Skip to content

SC-084: Deepfake Detection Evasion -- Operation MIRROR IMAGE

Scenario Overview

Field Detail
ID SC-084
Category AI Security / Adversarial ML / Identity Fraud
Severity High
ATT&CK Tactics Reconnaissance, Resource Development, Initial Access, Defense Evasion, Impact
ATT&CK Techniques T1566 (Phishing), T1078 (Valid Accounts), T1036 (Masquerading), T1565 (Data Manipulation), T1595 (Active Scanning)
AI-Specific Techniques Adversarial Perturbation, Detector Poisoning, GAN Identity Synthesis, Liveness Detection Bypass, Model Inversion
Target Environment VeridianBank -- a fictional digital-first bank using AI-powered KYC (Know Your Customer) with deepfake detection, liveness verification, biometric authentication, and automated fraud scoring
Difficulty ★★★★☆
Duration 2-3 hours
Estimated Impact 23 synthetic identities created and verified; $2.8M in fraudulent accounts opened; deepfake detector bypass rate increased from 0.3% to 94%; liveness detection evaded in 18 of 23 attempts; regulatory SAR filings required; customer trust impact

Narrative

VeridianBank is a fictional digital-first bank that onboards 100% of its customers remotely through a mobile application. The bank's identity verification pipeline is considered industry-leading, processing approximately 4,500 KYC verification requests per day through a multi-layer AI system:

  1. Document Verification -- OCR + template matching for government-issued IDs (drivers licenses, passports), deployed at verify.veridianbank.example.com (198.51.100.30)
  2. Facial Matching -- ResNet-50 based embedding comparison between selfie and ID photo (cosine similarity threshold: 0.89)
  3. Deepfake Detection -- Binary classifier trained on FaceForensics++ and internal datasets, analyzing frequency-domain artifacts (DCT coefficients), spatial inconsistencies, and temporal coherence
  4. Liveness Detection -- Challenge-response system (blink twice, turn head left/right, read random numbers) plus passive liveness analysis (skin texture, moire pattern detection)
  5. Fraud Scoring -- Behavioral risk model combining device fingerprint, IP reputation, velocity checks, and credit bureau thin-file indicators

The system runs on a Kubernetes cluster at 10.40.0.0/16, with ML models served via a TensorFlow Serving instance at 10.40.5.20. The KYC API processes verification requests from the mobile app at api.veridianbank.example.com (198.51.100.30). Model updates are deployed bi-weekly through an internal MLOps pipeline.

In April 2026, PHANTOM FACE v2 -- an AI-enabled identity fraud syndicate that evolved from the original PHANTOM FACE group (known for business email compromise using basic deepfakes) -- launches a sophisticated campaign to systematically defeat VeridianBank's AI-powered identity verification system and create synthetic identities for bust-out fraud.

Attack Flow

graph TD
    A[Phase 1: Detector Reconnaissance<br/>Black-box probing of deepfake detection model] --> B[Phase 2: Adversarial Perturbation Development<br/>Train U-Net to generate detector-evading modifications]
    B --> C[Phase 3: Synthetic Identity Generation<br/>GAN-generated faces + fabricated documents]
    C --> D[Phase 4: Liveness Detection Bypass<br/>Real-time face swap with adversarial overlay]
    D --> E[Phase 5: Account Creation at Scale<br/>23 synthetic identities verified and onboarded]
    E --> F[Phase 6: Bust-Out Fraud Execution<br/>Credit line exploitation and fund extraction]
    F --> G[Phase 7: Detection<br/>Fraud pattern + ML model anomaly detection]
    G --> H[Phase 8: Response<br/>Account freeze + model hardening + regulatory filing]

Phase Details

Phase 1: Detector Reconnaissance

ATT&CK Technique: T1595 (Active Scanning)

PHANTOM FACE v2 conducts systematic black-box probing of VeridianBank's deepfake detection model. Using a network of prepaid phones and residential proxy IPs (198.51.100.0/24 range), the group submits approximately 500 KYC verification attempts over 3 weeks with carefully varied deepfake quality levels to map the detector's decision boundary.

# Simulated detector probing campaign (educational only)
# PHANTOM FACE v2 submits test verifications via VeridianBank mobile API

# Step 1: Generate test images at varying quality levels
# Using StyleGAN3 to create 500 synthetic faces
# Quality levels: raw GAN output, post-processed, adversarially modified

# Step 2: Submit via API with unique device fingerprints per attempt
$ curl -X POST https://api.veridianbank.example.com/v2/kyc/verify \
    -H "Authorization: Bearer REDACTED" \
    -H "Content-Type: multipart/form-data" \
    -H "X-Device-ID: device-synth-00147" \
    -F "selfie=@test_face_147.jpg" \
    -F "id_front=@test_id_front_147.jpg" \
    -F "id_back=@test_id_back_147.jpg" \
    -F "session_id=kyc-sess-00147"

# Response (simulated):
# {
#   "verification_id": "ver-00147",
#   "status": "REJECTED",
#   "rejection_reason": "deepfake_detected",
#   "deepfake_score": 0.97,
#   "liveness_result": "NOT_ATTEMPTED",
#   "face_match_score": 0.91
# }

# Step 3: Analyze response patterns across 500 submissions
# Results matrix (simulated — educational only):
# Raw GAN output (StyleGAN3):     95% detected (avg score: 0.94)
# Post-processed (blur + noise):  72% detected (avg score: 0.81)
# Frequency-filtered:             34% detected (avg score: 0.62)
# Adversarial perturbation v1:     8% detected (avg score: 0.31)
# Adversarial perturbation v2:     6% detected (avg score: 0.12)

# Key findings from probing:
# 1. Detector relies heavily on DCT coefficient analysis (frequency domain)
# 2. Confidence threshold is 0.85 (below = flagged as deepfake)
# 3. Detector struggles with artifacts below 4x4 pixel block boundaries
# 4. No rate limiting on KYC attempts per device fingerprint class
# 5. Rejection responses include detailed score breakdown (information leak)

Phase 2: Adversarial Perturbation Development

AI-Specific Technique: Adversarial Perturbation Generation

Using insights from the reconnaissance phase, PHANTOM FACE v2 trains a custom adversarial perturbation generator -- a U-Net architecture that learns to add imperceptible pixel modifications to GAN-generated faces, specifically targeting the frequency-domain features that VeridianBank's detector relies on.

# Simulated adversarial perturbation generator (educational only)
# Architecture: U-Net with skip connections
# This is a simplified educational representation

import torch
import torch.nn as nn

class AdversarialPerturbationGenerator(nn.Module):
    """
    Educational example: generates imperceptible perturbations
    that evade frequency-domain deepfake detectors.
    NOT functional attack code -- simplified for learning.
    """
    def __init__(self):
        super().__init__()
        # Encoder: extract features from GAN-generated face
        self.encoder = nn.Sequential(
            nn.Conv2d(3, 64, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(64, 128, 3, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(128, 256, 3, stride=2, padding=1),
            nn.ReLU(),
        )
        # Decoder: generate perturbation mask
        self.decoder = nn.Sequential(
            nn.ConvTranspose2d(256, 128, 4, stride=2, padding=1),
            nn.ReLU(),
            nn.ConvTranspose2d(128, 64, 4, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(64, 3, 3, padding=1),
            nn.Tanh(),  # Output in [-1, 1]
        )
        self.epsilon = 2.0 / 255.0  # Max perturbation: 2 LSB per channel

    def forward(self, face_image):
        features = self.encoder(face_image)
        perturbation = self.decoder(features) * self.epsilon
        # Perturbed image is visually identical to original
        return face_image + perturbation

# Training loop (educational pseudocode)
# for epoch in range(100):
#     fake_faces = stylegan3.generate(batch_size=32)
#     perturbed = generator(fake_faces)
#     detector_score = veridian_detector_proxy(perturbed)
#     # Loss: minimize detector confidence + maximize image quality
#     loss = detector_score.mean() + lambda * lpips(fake_faces, perturbed)
#     loss.backward()
#     optimizer.step()

# Results after training (simulated):
# Detector confidence on raw GAN faces: 0.94 average
# Detector confidence after perturbation: 0.12 average
# LPIPS perceptual distance: 0.003 (imperceptible to humans)
# SSIM between original and perturbed: 0.998
# Perturbation magnitude: < 2 LSB per channel (invisible)

Phase 3: Synthetic Identity Generation

ATT&CK Technique: T1036 (Masquerading)

PHANTOM FACE v2 generates 23 complete synthetic identities, each consisting of a GAN-generated face (with adversarial perturbation applied), fabricated government ID documents, and a backstory including a synthetic thin credit file. The group uses a fictional US state ("New Cascadia") driver's license template.

# Simulated synthetic identity generation pipeline (educational only)
# Each identity includes: face, ID document, credit profile, device fingerprint

# Step 1: Generate GAN faces with adversarial perturbation
# Tool: StyleGAN3 + custom adversarial perturbation generator
# Output: 23 unique faces, each with perturbation applied
# Quality check: human review confirms visual realism
# Deepfake score check: all 23 score below 0.20 (threshold: 0.85)

# Step 2: Generate synthetic government IDs
# Template: New Cascadia driver's license (fictional state)
# For each identity:
#   - GAN face composited onto ID template
#   - Name: randomly generated (e.g., "Jordan Mitchell", "Alex Reeves")
#   - DOB: 1988-2000 range (plausible for new digital bank customer)
#   - Address: valid format, fictional street/city in New Cascadia
#   - License number: valid format (NC-XXXXXXXXX)
#   - Barcode: encoded matching front-side data
#   - Document aging: slight wear, realistic JPEG compression artifacts
#   - Hologram simulation: subtle iridescent overlay

# Step 3: Prepare device environments
# 23 virtual Android devices with unique fingerprints:
#   - Unique IMEI, MAC address, screen resolution
#   - Different Android versions (12, 13, 14)
#   - Varied installed app profiles
#   - Geographic distribution: 23 different cities
#   - Residential proxy IPs: 198.51.100.40 through 198.51.100.62

# Sample identity (synthetic -- educational only):
# Name: Jordan Mitchell
# DOB: 1994-07-15
# Address: 2847 Cedar Lane, Portfield, NC 98401
# License: NC-847291036
# Device: Pixel 7, Android 14, IP: 198.51.100.44
# GAN seed: 7291 | Perturbation model: v2.3
# Deepfake score: 0.08 | Face match score: 0.93

Phase 4: Liveness Detection Bypass

AI-Specific Technique: Liveness Detection Evasion

PHANTOM FACE v2 develops three techniques to bypass VeridianBank's liveness detection challenges. The primary technique uses a real human operator whose face provides genuine 3D movement data, while a real-time face swap overlay replaces their appearance with the target synthetic identity.

# Simulated liveness bypass techniques (educational only)
# All techniques are described for defensive understanding

# Technique A: Real-Time Face Swap with Adversarial Overlay
# Primary method -- used in 18 of 23 successful attempts

class LivenessBypass:
    """
    Educational representation of liveness evasion.
    NOT functional code -- simplified for learning purposes.
    """
    def __init__(self, target_identity):
        self.target_face = target_identity.gan_face
        self.perturbation = target_identity.adversarial_mask
        self.face_swap_model = None  # Placeholder
        self.anti_moire_filter = None  # Placeholder

    def process_frame(self, webcam_frame):
        """
        Real operator provides genuine 3D movement (passes liveness).
        Face swap replaces appearance with synthetic identity.
        Adversarial overlay suppresses deepfake artifacts.
        Anti-moire filter removes screen-capture patterns.
        """
        # Step 1: Detect real operator face landmarks
        landmarks = detect_landmarks(webcam_frame)

        # Step 2: Swap face with target synthetic identity
        swapped = self.face_swap_model.swap(
            webcam_frame, landmarks, self.target_face
        )

        # Step 3: Apply adversarial perturbation
        perturbed = swapped + self.perturbation

        # Step 4: Anti-moire filtering (removes virtual camera artifacts)
        output = self.anti_moire_filter.apply(perturbed)

        return output

# Technique A results (simulated):
# Blink detection: PASS (real operator blinks naturally)
# Head turn challenge: PASS (real operator turns head)
# Number reading: PASS (lip sync from real operator speech)
# Passive liveness (texture): PASS in 78% of attempts
# Overall success rate: 78% (18/23 attempts)

# Technique B: 3D Face Model Injection (backup method)
# Create 3D mesh from GAN face using face reconstruction model
# Inject via virtual webcam driver
# Success rate: 52% (lower due to texture artifacts)

# Technique C: Hybrid Attack (used for 5 failed-then-retry attempts)
# Real face for all liveness challenges
# Face swap activates ONLY during selfie-to-ID comparison step
# Requires precise timing of swap activation
# Success rate: 60% (timing-dependent)

# Simulated liveness bypass execution (educational only)
# Example: Identity "Jordan Mitchell" verification attempt

# Challenge 1: "Please blink twice"
# Real operator blinks naturally -> face swap outputs synthetic face blinking
# Result: PASS (genuine 3D eye movement detected)

# Challenge 2: "Turn your head to the left"
# Real operator turns head -> face swap maintains identity during rotation
# Result: PASS (genuine head pose change detected)

# Challenge 3: "Read the numbers: 4-7-2"
# Real operator speaks "four seven two"
# Lip movements from real operator mapped to synthetic face
# Result: PASS (lip sync matches expected phonemes)

# Passive liveness checks:
# Skin texture analysis: 0.72 (threshold: 0.50) -> PASS
# Moire pattern detection: 0.03 (threshold: 0.30) -> PASS
# Depth estimation: 0.81 (threshold: 0.40) -> PASS

# Overall liveness result: PASS
# Combined with deepfake score: 0.08 (well below 0.85 threshold)
# Face match (selfie vs ID): 0.93 (above 0.89 threshold)
# KYC result: APPROVED

Phase 5: Account Creation at Scale

ATT&CK Technique: T1078 (Valid Accounts)

Over a 10-day period, PHANTOM FACE v2 creates 23 verified accounts at VeridianBank, each with a unique synthetic identity. The accounts are created at a rate of 2-3 per day to avoid velocity-based detection, with geographic distribution matching the synthetic identities' claimed locations.

Simulated account creation timeline (educational only)
[2026-04-01] Account created: Jordan Mitchell (Portfield, NC)
  KYC result: APPROVED | Deepfake: 0.08 | Liveness: PASS | Face match: 0.93
  Initial deposit: $1,200 (ACH from fictional employer)

[2026-04-01] Account created: Alex Reeves (Lakewood, NC)
  KYC result: APPROVED | Deepfake: 0.11 | Liveness: PASS | Face match: 0.91
  Initial deposit: $800 (debit card transfer)

[2026-04-03] Account created: Casey Lin (Bridgeton, NC)
  KYC result: APPROVED | Deepfake: 0.15 | Liveness: PASS | Face match: 0.90
  Initial deposit: $1,500 (ACH)

[2026-04-03] Account created: Morgan Blake (Clearfield, NC)
  KYC result: APPROVED | Deepfake: 0.09 | Liveness: PASS | Face match: 0.94
  Initial deposit: $2,000 (wire transfer)

... 19 additional accounts created through 2026-04-10 ...

Total accounts: 23 (all KYC approved)
Failed attempts: 5 (liveness failure -- retried with Technique C)
Average deepfake score: 0.12 (well below 0.85 detection threshold)
Average face match: 0.92 (above 0.89 threshold)
Total initial deposits: $28,400

Phase 6: Bust-Out Fraud Execution

ATT&CK Technique: T1565 (Data Manipulation)

After a 14-day aging period during which the accounts maintain normal transaction patterns, PHANTOM FACE v2 executes the bust-out phase -- rapidly maxing credit lines across all 23 accounts and extracting funds through cryptocurrency purchases and peer-to-peer transfers.

Simulated bust-out fraud timeline (educational only)
[2026-04-01 through 2026-04-14] AGING PHASE
  Each account: 3-5 small transactions per day
  Categories: groceries, coffee, gas, streaming services
  Average daily spend: $35-$75 per account
  Direct deposit set up: fictional employer "TechServ Solutions"
  Credit limit requests: submitted on day 7
  Credit limits granted: $50,000-$175,000 per account

[2026-04-15 through 2026-04-20] BUST-OUT PHASE
  Day 15: 8 accounts max credit lines
    Jordan Mitchell: $150,000 -> cryptocurrency purchase at 203.0.113.70
    Alex Reeves: $125,000 -> P2P transfer to 0x0000...mule01
    Casey Lin: $175,000 -> cryptocurrency purchase at 203.0.113.70
    ... 5 additional accounts ...
    Daily total: $980,000

  Day 16: 8 accounts max credit lines
    Daily total: $870,000

  Day 17-20: Remaining 7 accounts + second-round on first accounts
    Additional total: $950,000

TOTAL FRAUD EXPOSURE:
  Credit line exploitation: $2,400,000
  Checking account overdraft: $280,000
  ACH fraud (reversed deposits): $120,000
  Total: $2,800,000

FUND EXTRACTION:
  Cryptocurrency purchases: $1,900,000 (68%)
  P2P transfers to mule accounts: $600,000 (21%)
  Cash advances: $300,000 (11%)
  All funds extracted within 5 days
  Accounts abandoned -- synthetic identities untraceable

Phase 7: Detection

The fraud is detected through converging signals from multiple monitoring systems:

Channel 1 (T+16 days): Fraud Scoring Anomaly -- VeridianBank's transaction monitoring system flags 8 accounts with identical behavioral trajectories: small transactions for 14 days followed by sudden credit line maximization and cryptocurrency purchases. The behavioral similarity score triggers a pattern alert.

Channel 2 (T+18 days): Credit Bureau Discrepancy -- A routine credit bureau reconciliation discovers that 12 of the 23 accounts have Social Security Numbers that were issued within the past 18 months but claim birth dates in the 1990s -- a hallmark of synthetic identities.

Channel 3 (T+20 days): ML Model Drift Alert -- The MLOps team's model monitoring detects a subtle shift in the deepfake detector's score distribution: the average score for new-account verifications has decreased from 0.71 to 0.58 over the past month, indicating either model degradation or adversarial evasion.

Simulated detection timeline (educational only)
[2026-04-17 08:30:00 UTC] FRAUD MONITORING -- PATTERN ALERT
  Source: fraud-engine.veridianbank.example.com
  Alert: BEHAVIORAL_COHORT_DETECTED
  Details:
    8 accounts flagged with identical trajectory:
    - 14-day aging period with small transactions
    - Credit limit increase on day 7
    - Sudden credit maximization on day 15
    - Cryptocurrency purchases exceeding $100,000
  Cohort similarity score: 0.94 (threshold: 0.80)
  Accounts: JMitchell-0401, AReeves-0401, CLin-0403, ...
  Action: Fraud team investigation initiated

[2026-04-19 14:15:00 UTC] CREDIT BUREAU -- SYNTHETIC IDENTITY FLAG
  Source: credit-reconciliation.veridianbank.example.com
  Alert: SSN_AGE_MISMATCH
  Details:
    12 accounts with SSN issue date < 18 months
    All claim DOB in 1988-2000 range
    Expected SSN issue date for these DOBs: 2004-2016
    Synthetic identity probability: HIGH
  Action: Compliance team escalation

[2026-04-21 10:45:00 UTC] MLOPS MONITORING -- MODEL DRIFT ALERT
  Source: mlops.veridianbank.example.com
  Alert: DEEPFAKE_DETECTOR_SCORE_DRIFT
  Details:
    Average deepfake score (30-day window):
    - March 1-15: 0.71 (baseline)
    - March 16-31: 0.68
    - April 1-15: 0.62
    - April 16-21: 0.58
    Distribution shift: KL divergence = 0.34 (threshold: 0.20)
    Possible causes: model degradation, data drift, adversarial attack
  Action: ML security team investigation

Detection Queries:

// KQL -- Detect systematic KYC probing patterns
KYCVerificationLogs
| where TimeGenerated > ago(30d)
| where VerificationResult == "REJECTED"
| where RejectionReason == "deepfake_detected"
| summarize AttemptCount = count(),
            UniqueIPs = dcount(ClientIP),
            UniqueDeviceIDs = dcount(DeviceFingerprint),
            AvgDeepfakeScore = avg(DeepfakeScore)
  by bin(TimeGenerated, 1h)
| where AttemptCount > 20
    or (UniqueIPs > 5 and UniqueDeviceIDs > 10)
| project TimeGenerated, AttemptCount, UniqueIPs,
          UniqueDeviceIDs, AvgDeepfakeScore

// KQL -- Detect liveness evasion indicators
KYCVerificationLogs
| where TimeGenerated > ago(7d)
| where LivenessResult == "PASS"
| extend FaceConsistency = toreal(FaceEmbeddingSimilarity)
| extend TextureScore = toreal(SkinTextureAnalysis)
| where FaceConsistency < 0.7 or TextureScore < 0.5
| join kind=inner (
    KYCVerificationLogs
    | where DeepfakeScore < 0.3
    | project SessionID, DeepfakeScore
) on SessionID
| project TimeGenerated, SessionID, FaceConsistency,
          TextureScore, DeepfakeScore, DeviceFingerprint, ClientIP

// KQL -- Detect synthetic identity bust-out pattern
BankAccountActivity
| where TimeGenerated > ago(30d)
| where AccountAge < 30
| summarize MaxBalance = max(Balance),
            MinBalance = min(Balance),
            TxCount = count(),
            CryptoTxCount = countif(MerchantCategory == "crypto_exchange"),
            P2PTxCount = countif(MerchantCategory == "p2p_transfer"),
            AvgDailySpend = avg(TransactionAmount)
  by AccountID, AccountAge
| where CryptoTxCount > 0 and MaxBalance > 10000
| extend BustOutRisk = iff(
    AccountAge < 21 and CryptoTxCount > 2 and MaxBalance > 50000,
    "HIGH", "MEDIUM")
| where BustOutRisk == "HIGH"
| project AccountID, AccountAge, MaxBalance, CryptoTxCount,
          P2PTxCount, BustOutRisk

// KQL -- Detect deepfake detector model drift
MLModelMetrics
| where TimeGenerated > ago(30d)
| where ModelName == "deepfake_detector_v3"
| summarize AvgScore = avg(PredictionScore),
            StdDev = stdev(PredictionScore),
            P5Score = percentile(PredictionScore, 5),
            P95Score = percentile(PredictionScore, 95)
  by bin(TimeGenerated, 1d)
| extend ScoreTrend = AvgScore - prev(AvgScore)
| where ScoreTrend < -0.02  // Decreasing average scores
| project TimeGenerated, AvgScore, StdDev, P5Score, P95Score, ScoreTrend

# SPL -- Detect systematic KYC probing patterns
index=kyc sourcetype=verification_log result="REJECTED" reason="deepfake_detected"
| bin _time span=1h
| stats count as attempt_count,
        dc(client_ip) as unique_ips,
        dc(device_fingerprint) as unique_devices,
        avg(deepfake_score) as avg_score
  by _time
| where attempt_count > 20
    OR (unique_ips > 5 AND unique_devices > 10)
| table _time, attempt_count, unique_ips, unique_devices, avg_score

# SPL -- Detect liveness evasion indicators
index=kyc sourcetype=verification_log liveness_result="PASS"
| eval face_consistency=tonumber(face_embedding_similarity)
| eval texture_score=tonumber(skin_texture_analysis)
| where face_consistency < 0.7 OR texture_score < 0.5
| join type=inner session_id [
    search index=kyc sourcetype=verification_log deepfake_score < 0.3
    | fields session_id, deepfake_score
]
| table _time, session_id, face_consistency, texture_score,
        deepfake_score, device_fingerprint

# SPL -- Detect synthetic identity bust-out pattern
index=banking sourcetype=account_activity
| where account_age < 30
| stats max(balance) as max_balance,
        min(balance) as min_balance,
        count as tx_count,
        sum(eval(if(merchant_category="crypto_exchange", 1, 0))) as crypto_count,
        sum(eval(if(merchant_category="p2p_transfer", 1, 0))) as p2p_count
  by account_id, account_age
| where crypto_count > 0 AND max_balance > 10000
| eval bust_out_risk=if(
    account_age < 21 AND crypto_count > 2 AND max_balance > 50000,
    "HIGH", "MEDIUM")
| where bust_out_risk="HIGH"
| table account_id, account_age, max_balance, crypto_count, p2p_count, bust_out_risk

# SPL -- Detect deepfake detector model drift
index=mlops sourcetype=model_metrics model_name="deepfake_detector_v3"
| bin _time span=1d
| stats avg(prediction_score) as avg_score,
        stdev(prediction_score) as std_dev,
        perc5(prediction_score) as p5_score,
        perc95(prediction_score) as p95_score
  by _time
| streamstats current=f last(avg_score) as prev_score
| eval score_trend=avg_score - prev_score
| where score_trend < -0.02
| table _time, avg_score, std_dev, p5_score, p95_score, score_trend

Phase 8: Response

Immediate Actions (0-48 hours):

Simulated incident response (educational only)
[2026-04-17 10:00:00 UTC] ALERT: Fraud Investigation Team activated
[2026-04-17 10:30:00 UTC] ACTION: 8 flagged accounts FROZEN
  Credit lines: suspended
  Debit access: blocked
  Remaining funds: $47,200 (most already extracted)

[2026-04-19 16:00:00 UTC] ACTION: Expanded investigation
  Credit bureau cross-reference identifies 15 additional suspect accounts
  Total accounts frozen: 23
  Total fraud exposure confirmed: $2,800,000
  Funds recovered: $312,000 (cryptocurrency exchange holds)
  Funds unrecoverable: $2,488,000

[2026-04-20 09:00:00 UTC] ACTION: KYC pipeline emergency hardening
  Deepfake detector: confidence threshold raised from 0.85 to 0.92
  Liveness detection: injection attack check added (virtual camera detection)
  KYC API: score breakdown removed from rejection responses (info leak closed)
  Velocity checks: max 1 KYC attempt per device fingerprint per 24 hours

[2026-04-21 12:00:00 UTC] ACTION: ML model investigation
  Adversarial attack confirmed (not model drift)
  Perturbation analysis: frequency-domain modifications detected in
    approved applications from April 1-10
  Action: emergency model retraining with adversarial examples

[2026-04-22 08:00:00 UTC] ACTION: Regulatory notification
  BSA/AML: 23 SARs filed (Suspicious Activity Reports)
  State banking regulator: notified of synthetic identity fraud event
  Law enforcement: FBI IC3 report filed
  Customer notification: not required (synthetic identities, no real customers)

Recovery (2-6 weeks):

  • Deploy ensemble deepfake detection: frequency-domain + spatial + temporal analysis (3 models voting)
  • Add injection attack detection: verify camera feed originates from physical sensor, not virtual device
  • Implement adversarial training: continuously red-team detector with latest GAN outputs
  • Deploy passive liveness via involuntary micro-expressions (harder to fake than challenge-response)
  • Build behavioral biometric layer: typing patterns, touch dynamics, session interaction analysis
  • Cross-reference new accounts against synthetic identity indicators (thin credit file + recent SSN issuance)
  • Implement continuous authentication post-onboarding (not just one-time KYC)
  • Remove detailed ML scores from API responses to prevent adversarial reconnaissance
  • Establish industry consortium for sharing synthetic identity indicators
  • Model monitoring: deploy adversarial drift detection alongside standard distribution monitoring

Decision Points (Tabletop Exercise)

Decision Point 1 -- Detector Confidence Drop

Your ML team notices that average deepfake detection scores have been gradually decreasing over the past month. Is this model drift due to changing input data, natural degradation, or an active adversarial campaign? How do you investigate without disrupting legitimate customer onboarding?

Decision Point 2 -- Account Freeze Threshold

You have identified 8 accounts with suspicious bust-out patterns, but freezing accounts incorrectly costs $2,500 per incident in customer support and legal review. With 23 suspect accounts, what confidence threshold do you require before freezing? How do you balance fraud prevention against customer experience?

Decision Point 3 -- Liveness Vendor Accountability

Your deepfake detection vendor claims 99.7% accuracy, but the attacker achieved a 94% bypass rate using adversarial perturbations. What SLA protections should your contract include? Should you switch vendors, add a second detector in ensemble, or build in-house capabilities?

Decision Point 4 -- Regulatory Timing

BSA/AML requires SAR filing within 30 days of detecting suspicious activity. When does the clock start -- when the fraud monitoring system first flagged the behavioral cohort (April 17), when the credit bureau discrepancy was found (April 19), or when each individual fraudulent account was opened (April 1-10)?

Lessons Learned

Key Takeaways

  1. Single-model deepfake detection is insufficient against adversarial attacks -- A determined attacker with black-box access can probe, map, and evade any single detector. Ensemble detection (multiple models analyzing different feature spaces) dramatically increases evasion difficulty.

  2. API responses should not leak model internals -- Returning detailed deepfake confidence scores in rejection responses enabled the attacker to precisely calibrate their adversarial perturbations. Security-critical ML systems should return only pass/fail decisions, not scores.

  3. Liveness detection must include injection attack prevention -- Challenge-response liveness works against static images but fails against real-time face-swap attacks. Virtual camera detection, depth sensing, and injection attack analysis must complement challenge-response systems.

  4. Synthetic identity fraud requires cross-system detection -- No single system (KYC, transaction monitoring, credit bureau) detected the full attack. Correlation across identity verification signals, credit history, and behavioral patterns is required.

  5. Adversarial robustness must be continuously tested -- ML models degrade against adversarial attacks over time as attackers develop new techniques. Regular red-teaming of deepfake detectors with state-of-the-art GAN outputs is essential.

  6. Velocity controls must consider sophisticated attackers -- The attacker circumvented basic velocity checks by using unique device fingerprints and distributing attempts over 3 weeks. Multi-dimensional velocity analysis (behavioral similarity, biometric clustering, temporal patterns) is needed.

MITRE ATT&CK Mapping

Technique ID Technique Name Phase
T1595 Active Scanning Reconnaissance (detector probing)
T1587.001 Develop Capabilities -- Malware Resource Development (perturbation generator)
T1036 Masquerading Defense Evasion (synthetic identity)
T1078 Valid Accounts Initial Access (fraudulent accounts)
T1566 Phishing Initial Access (employee credential theft)
T1565 Data Manipulation Impact (fraudulent transactions)
Custom: AML-T0043 Adversarial Perturbation Defense Evasion
Custom: AML-T0047 Model Evasion via Liveness Bypass Defense Evasion
Custom: AML-T0048 Synthetic Identity Generation Resource Development