Chapter 9: AI/ML in SOC - Quiz¶

Training Data:
Email 1: Subject: "Your account is locked" → Label: PHISHING
Email 2: Subject: "Meeting at 3pm" → Label: LEGITIMATE
... 10,000 labeled emails

Model learns patterns:
- Urgent language + suspicious link = PHISHING
- Internal sender + calendar invite = LEGITIMATE

Prediction on new email:
Email: "Verify your account now!" → Predicted: PHISHING (confidence: 92%)

Training Data: 1 million network connections (no labels)

Model finds clusters:
- Cluster 1: Web browsing (80% of traffic)
- Cluster 2: Email (15% of traffic)
- Cluster 3: SSH (4% of traffic)
- Cluster 4: Unusual (1% of traffic) ← Potential threats

Anomaly detected: Connection doesn't fit any cluster → Alert

Input: Email features (sender, subject, links, urgency words)
Output: PHISHING or LEGITIMATE
Classes: 2 (binary classification)

Input: File hash, behavioral features
Output: MALWARE or BENIGN
Classes: 2 (binary)

Input: Alert metadata (source IP, user, asset, threat intel)
Output: LOW, MEDIUM, HIGH, CRITICAL
Classes: 4 (multi-class classification)

Input: Network traffic features
Output: NORMAL, DOS, PROBE, R2L, U2R
Classes: 5 (multi-class)

Training Set: 10,000 labeled emails
- 6,000 LEGITIMATE
- 4,000 PHISHING

Model learns decision boundary:
IF (urgent_words > 3 AND external_sender AND suspicious_link):
    Predict: PHISHING
ELSE:
    Predict: LEGITIMATE

Input: User activity features (login times, applications used, data accessed)
Process: Clustering algorithm finds natural groups
Output:
  - Cluster 1: Sales team (CRM usage, daytime logins, high email volume)
  - Cluster 2: Engineers (IDE/Git, late-night logins, SSH usage)
  - Cluster 3: Finance (spreadsheets, 9-5 logins, financial system access)
  - Cluster 4: Executives (mobile access, travel, light usage)
  - Anomaly: User doesn't fit any cluster → Investigate

Input: Network flows (bytes, packets, duration, ports)
Output:
  - Cluster 1: HTTP/HTTPS browsing
  - Cluster 2: Email traffic
  - Cluster 3: Database queries
  - Cluster 4: Unknown (potential C2 traffic) → Alert

Input: Alerts (source, destination, type, time)
Output: Groups of related alerts (likely same incident)
Benefit: Reduces alert fatigue by grouping 50 alerts into 1 incident

Input: Alert features (threat intel match, asset criticality, user risk, historical FP rate)
Output: Risk score 0-100 (e.g., 87.3)
Model: Regression

Input: Vulnerability CVSS score, patch status, exposure, threat intel
Output: Predicted days until exploitation (e.g., 14.7 days)
Model: Regression

Input: Incident severity, complexity, team availability
Output: Predicted response time in minutes (e.g., 45.2 minutes)
Model: Regression

Classification: "Is this PHISHING or LEGITIMATE?" (discrete classes)
Regression: "What's the phishing probability?" (continuous: 0.873 = 87.3%)

Baseline: User typically accesses 50 files/day from HR database
Anomaly: User suddenly downloads 10,000 files
Alert: "Abnormal data access - potential exfiltration"

Baseline: User logs in from New York, 9am-5pm, Windows laptop
Anomaly: Login from Russia at 3am, Linux system
Alert: "Impossible travel + unusual OS"

Baseline: Workstation WKS-042 typically connects to 5 internal servers
Anomaly: WKS-042 connects to 50 servers in 10 minutes
Alert: "Abnormal network scanning behavior"

Baseline: User account "jdoe" never uses administrative tools
Anomaly: "jdoe" executes PowerShell with admin rights
Alert: "Unusual privilege usage"

Step 1: Baseline Learning (Training Phase)
- Collect 30-90 days of normal behavior
- Build user/entity profiles using unsupervised learning
- Example: User A baseline = [login_times, apps_used, data_accessed, ...]

Step 2: Anomaly Detection (Inference Phase)
- Compare current behavior to baseline
- Calculate anomaly score (0-100)
- Alert if score exceeds threshold

Step 3: Continuous Learning
- Update baselines as behavior evolves
- Handle concept drift (job changes, new applications)

User: jane.doe@company.com
Anomaly Score: 89/100
Reasons:
  - Login from unusual location (China vs typical New York)
  - Login at unusual time (3am vs typical 9am-5pm)
  - Unusual application (SSH vs typical Outlook/Chrome)
  - High data access (500 files vs typical 20)
Recommendation: Investigate potential compromise

Email Subject: "URGENT: Verify your account now!"
Email Body: "Click here to verify: http://evil.com/verify"
Sender: "security@paypa1.com"

- urgent_word_count: 2 (URGENT, now)
- external_sender: True
- sender_domain_typosquat: True (paypa1 vs paypal)
- link_count: 1
- link_domain_mismatch: True (evil.com != paypa1.com)
- capitalization_ratio: 0.15 (15% caps)
- has_attachment: False
- sender_in_contacts: False

File: malware.exe
Size: 2,456,032 bytes
Strings: ["cmd.exe", "powershell", "encrypt", ...]

- file_size: 2456032
- entropy: 7.8 (high entropy suggests encryption/packing)
- suspicious_imports: 5 (count of CreateRemoteThread, VirtualAllocEx, ...)
- string_matches: ["cmd.exe", "powershell"] (count: 2)
- packed: True (detected via entropy analysis)
- age_days: 3 (first seen 3 days ago)
- prevalence: 0.0001% (very rare file)

TCP connection: 10.0.1.50:49234 → 203.0.113.45:443
Duration: 3600 seconds
Bytes sent: 1024
Bytes received: 52,428,800

- duration_minutes: 60
- bytes_ratio: 51200 (received/sent - indicates data download)
- destination_is_external: True
- destination_threat_intel_match: True
- connection_time: 02:00 (unusual hour)
- port_443_with_non_http: True (suspicious)

Email 1: Subject "Verify account" From: attacker@evil.com → PHISHING
Email 2: Subject "Meeting notes" From: colleague@company.com → LEGITIMATE
... 100 total

Model memorizes: "If sender = attacker@evil.com → PHISHING"

Training Accuracy: 100% (perfect!)

Test Data:
Email: Subject "Verify account" From: attacker2@evil2.com → Model predicts LEGITIMATE

Why? Model memorized specific sender "attacker@evil.com" instead of learning pattern "Verify account + external sender = PHISHING"

Test Accuracy: 60% (poor!)

Underfitting: Model too simple (straight line through scattered data)
Good Fit: Model captures pattern (smooth curve fitting trend)
Overfitting: Model too complex (zigzag line through every training point, including noise)

100 samples → 10,000 samples
Harder for model to memorize, must learn patterns

Penalize complex models
Techniques: L1/L2 regularization, dropout (neural networks)

Reduce features from 100 → 20 most important
Removes noise and reduces complexity

Split data into 5 folds, train on 4, test on 1
Repeat 5 times, average performance
Ensures model generalizes across different data splits

Stop training when validation performance stops improving
Prevents model from over-learning training data

Problem: Model achieves 98% accuracy on historical alerts but only 70% on new alerts

Diagnosis: Overfitting (memorized specific IPs/users in training data)

Solution:
- Collected 10x more training data
- Reduced features from 50 → 15 most predictive
- Applied regularization

Result: Training 92%, Test 89% (better generalization)

Data: 10,000 alerts with features [source_ip, destination_ip, time, user, ...]
Task: Predict severity (Low/Medium/High/Critical)

Underfit Model: Uses only 1 feature (time of day)
Rule: "If time between 2am-6am → High, else → Low"

Result:
Training Accuracy: 55%
Test Accuracy: 54%

Why? Model ignores critical features (threat intel, asset criticality, user risk)
Patterns are too complex for simple time-based rule

Data: Complex curved pattern
Underfit Model: Straight horizontal line (ignores pattern)
Good Model: Curve that follows trend
Overfit Model: Zigzag through every point

Change: Linear model → Decision tree or neural network
Allows model to learn non-linear patterns

Before: [time]
After: [time, source_ip, threat_intel_match, asset_criticality, user_risk, ...]
Gives model more information to learn from

Increase training epochs/iterations
Give model more time to learn patterns

If regularization too strong → model too constrained
Relax constraints to allow learning

Problem: Model only checks file size
Rule: "If size > 10MB → Malware"

Training Accuracy: 52% (barely better than random)
Test Accuracy: 51%

Diagnosis: Underfitting (too simple)

Solution:
- Added features: entropy, imports, strings, behavior
- Changed algorithm: Simple rule → Random Forest

Result: Training 94%, Test 91% (learned real patterns)

Underfitting: Both training and test accuracy LOW
Good Fit: Both training and test accuracy HIGH (close values)
Overfitting: Training accuracy HIGH, test accuracy LOW (large gap)

Training (2023):
- Model trained on 2023 malware samples
- Learns: Malware uses DLL injection, specific C2 patterns

Production (2026):
- New malware families use fileless techniques, cloud C2
- Model doesn't recognize new patterns
- Accuracy: 95% (2023) → 70% (2026)
- Reason: Attack techniques evolved

Training (2024):
- Phishing emails use poor grammar, obvious spoofing

Production (2026):
- Attackers use AI-generated perfect grammar
- Compromise legitimate accounts (no spoofing)
- Model trained on old patterns misses new sophisticated phishing
- Accuracy: 92% (2024) → 68% (2026)

Training (Q1 2025):
- Company uses on-prem infrastructure
- UEBA baseline: 80% traffic to internal servers

Production (Q4 2025):
- Company migrates to cloud
- 70% traffic now to AWS/Azure
- UEBA alerts on legitimate cloud usage as "abnormal"
- False Positive Rate: 10% → 40%

Baseline (January):
- User "jdoe" is engineer, accesses code repos

Reality (June):
- User promoted to manager, now accesses HR systems
- UEBA flags as anomalous (legitimate job change)

Monitor:
1. Prediction accuracy over time (weekly/monthly)
2. Feature distributions (are inputs different than training data?)
3. False positive/negative rates
4. User feedback (analysts marking predictions as wrong)

Alert when:
- Accuracy drops >10% from baseline
- FP rate increases >20%
- Feature distributions shift significantly

Schedule: Retrain model monthly/quarterly
Data: Use recent labeled data (last 90 days)
Benefit: Model adapts to new patterns

Update model incrementally with new data
No full retraining needed
Adapts in real-time

Combine multiple models trained on different time periods
More robust to drift

Analysts label predictions
Feed back to training pipeline
Improves model continuously

Malware Detection Model:
- Week 1: Accuracy 94%
- Week 10: Accuracy 91%
- Week 20: Accuracy 85% ← Alert: Significant drift detected
- Action: Retrain on recent malware samples
- Week 21: Accuracy 93% (post-retraining)

Original Malware:
- Hash: abc123
- Entropy: 7.9 (high)
- Imports: VirtualAlloc, CreateRemoteThread
- ML Prediction: MALWARE (confidence: 98%)

Adversarial Malware:
- Add benign comments/strings to reduce entropy
- Entropy: 6.2 (now within benign range)
- Functionality: UNCHANGED (still malicious)
- ML Prediction: BENIGN (confidence: 65%)

Result: Attacker evades detection by manipulating features

Original Phishing:
Subject: "URGENT: Verify account now!"
ML Model: Flags "URGENT" + "Verify" as phishing indicators
Prediction: PHISHING (95%)

Adversarial Phishing:
Subject: "Important notice regarding your account verification"
- Replaces urgent words with softer language
- Same malicious intent
- ML Prediction: LEGITIMATE (70%)

Original C2 Traffic:
- Large payload (10MB data exfil)
- ML detects unusual volume
- Prediction: MALICIOUS

Adversarial C2:
- Split payload into 1000 small requests (10KB each)
- Mimics normal web traffic pattern
- ML Prediction: BENIGN (traffic volume per connection is normal)

- Attacker has access to model architecture and weights
- Calculates gradients to find minimal perturbation
- Crafts input to maximize misclassification

Example: Attacker knows malware detector uses entropy feature
→ Adds padding to reduce entropy below threshold

- Attacker queries model with test inputs
- Learns decision boundary through trial and error
- Crafts evasion based on observed behavior

Example: Test 100 malware variants to find which features trigger detection
→ Modify those specific features

Include adversarial examples in training data
Model learns to recognize evasion attempts

Training Data:
- Original malware samples
- + Adversarially modified samples (still labeled malware)

Result: Model robust to evasion

Use multiple models with different architectures
Harder for attacker to evade all simultaneously

Vote: If 2+ models detect malware → Flag as malicious

Don't rely solely on ML
Combine with:
- Signature-based detection (hash matching)
- Behavioral analysis (runtime monitoring)
- Sandboxing (execute in isolated environment)

Use features that are hard to manipulate without breaking functionality

Example: For malware detection
- Bad feature: File size (easily manipulated with padding)
- Good feature: Control flow graph (changing breaks functionality)

Detect manipulation attempts
Example: File with unusually high padding ratio → Suspicious

Attack: APT group crafts malware to evade EDR ML model
Method:
- Analyzed EDR vendor's published research
- Identified entropy threshold (> 7.5 = malware)
- Added benign strings to reduce entropy to 7.2

Defense:
- EDR vendor retrains model with adversarial samples
- Adds new feature: "unusual padding ratio"
- Detects evasion attempt

Use Case: Phishing detection
Reason: Phishing techniques evolve rapidly
Schedule: Retrain weekly with last 30 days of labeled emails
Benefit: Catches latest phishing trends

Use Case: Malware detection
Reason: New malware families emerge regularly
Schedule: Retrain quarterly with recent malware samples
Benefit: Adapts to new techniques while maintaining stability

Use Case: UEBA
Trigger: Major infrastructure change (cloud migration)
Action: Retrain immediately to establish new baseline
Benefit: Prevents false positive flood

Step 1: Collect Recent Data
- Last 90 days of labeled incidents
- Analyst feedback (false positives/negatives)
- New threat intelligence

Step 2: Prepare Training Set
- Combine historical data (for stability) with recent data (for adaptation)
- Ratio: 70% recent, 30% historical
- Balance classes (equal malicious/benign samples)

Step 3: Retrain Model
- Use same architecture or improved version
- Validate on hold-out test set
- Compare performance to previous model

Step 4: A/B Testing
- Deploy new model to 10% of traffic
- Monitor metrics (accuracy, FP rate, analyst feedback)
- If improved → Full deployment
- If worse → Rollback to previous model

Step 5: Monitor Performance
- Track accuracy over time
- Schedule next retraining

Initial Model (January 2025):
- Trained on 2024 data
- Accuracy: 92%

March 2025:
- Accuracy degrades to 85% (drift detected)
- New attack campaigns using different TTPs

Retraining (April 2025):
- Collected Q1 2025 labeled alerts (5,000 samples)
- Combined with 2024 data (15,000 samples)
- Retrained model

Post-Retraining:
- Accuracy: 93% (improved)
- Adapted to Q1 2025 threats

            Predicted MALWARE    Predicted BENIGN
Actually
MALWARE         TP = 90              FN = 10         (100 total malware)

Actually
BENIGN          FP = 20              TN = 880        (900 total benign)

Precision = TP / (TP + FP) = 90 / (90 + 20) = 90/110 = 81.8%
- "Of 110 malware predictions, 90 were correct"
- "18.2% of malware alerts are false positives"

Recall = TP / (TP + FN) = 90 / (90 + 10) = 90/100 = 90%
- "Of 100 actual malware samples, we detected 90"
- "We missed 10% of malware (false negatives)"

Model: Very conservative (only flags obvious malware)
Precision: 95% (very few FPs)
Recall: 60% (misses 40% of malware)

Impact:
+ Analysts trust alerts (low FP rate)
- Misses sophisticated threats (high FN rate)

Use Case: Auto-blocking (only block high-confidence threats)

Model: Very aggressive (flags anything suspicious)
Precision: 50% (high FP rate)
Recall: 98% (catches almost all malware)

Impact:
+ Catches nearly all threats (low FN rate)
- Alert fatigue (analysts waste time on FPs)

Use Case: Initial screening (send to analyst for review)

F1 Score = 2 × (Precision × Recall) / (Precision + Recall)

Example:
Precision: 85%
Recall: 80%
F1 = 2 × (0.85 × 0.80) / (0.85 + 0.80) = 0.824 = 82.4%

Use Case: Most SOC use cases (balance FP and FN)

Threshold adjustment example (malware detector):

Threshold: 0.9 (very strict)
→ High Precision (95%), Low Recall (65%)

Threshold: 0.5 (balanced)
→ Medium Precision (85%), Medium Recall (82%)

Threshold: 0.3 (aggressive)
→ Low Precision (70%), High Recall (95%)

Choose based on use case!

Alert: Unusual data access (model predicted: MALICIOUS)
Reality: Legitimate audit (actual: BENIGN)
Result: FALSE POSITIVE (alarm when shouldn't be)

Predicted: MALICIOUS
Actual: MALICIOUS
Example: UEBA flags impossible travel, investigation confirms account compromise
Result: ✅ Correct detection

Predicted: BENIGN
Actual: BENIGN
Example: Normal file access, no alert
Result: ✅ Correct (no alarm needed)

Predicted: MALICIOUS
Actual: BENIGN
Example: Legitimate audit flagged as data exfiltration
Result: ❌ Incorrect alarm (wasted analyst time)

Predicted: BENIGN
Actual: MALICIOUS
Example: Insider slowly exfiltrates data, stays under threshold, not flagged
Result: ❌ Missed threat (dangerous!)

Problem: Static threshold (50 files) doesn't account for legitimate changes
Solution: Adaptive baseline
- Detect user started audit project (new behavior cluster)
- Adjust expected range: 50-500 files for audit period
- Alert only if exceeds new range (e.g., 1000 files)

Alert: 500 file access
Enrichment:
- Check calendar: "Annual audit week" scheduled
- Check ticket system: Audit ticket assigned to user
- Check manager approval: Access approved
Result: Auto-close as expected behavior

Analyst marks alert as FP
→ Feeds back to model training
→ Model learns: "High file access during audit week = normal"
→ Future audits don't trigger alerts

Current: Alert if > 10x baseline (50 → 500)
Tuned: Alert if > 20x baseline (50 → 1000)
Trade-off: Fewer FPs but might miss some real threats (higher FN)

Day 1: User "new_hire" starts
UEBA: No historical data, can't establish baseline

Problem: If new_hire is malicious or compromised from day 1, UEBA won't detect anomalies

Risk Window: 30-90 days until baseline is established

New database server deployed
UEBA: No baseline for normal network traffic patterns

Problem: Can't detect if server is immediately compromised (no anomaly reference)

User "jdoe" promoted from Sales → IT Admin
Old Baseline: CRM access, 9-5 logins
New Role: Server access, on-call hours

Problem: New behavior looks anomalous compared to old baseline
Result: False positive flood OR need to rebuild baseline (cold start again)

Instead of individual baseline, use role-based baseline

New hire "jdoe" role: Software Engineer
Baseline: Aggregate behavior of all engineers
- Expected apps: IDE, Git, Slack
- Expected access: Code repos, dev servers
- Expected hours: Flexible (some work nights)

Anomaly: If jdoe accesses HR database → Alert (engineers don't typically access HR)

During baseline learning period (30 days):
- Apply stricter rule-based detections
- Flag high-risk actions (e.g., privilege escalation)
- Don't rely solely on behavioral anomaly detection

Use baselines from similar users/entities

New Sales user → Use existing Sales team baseline
New web server → Use existing web server baseline

Advantage: Immediate anomaly detection
Caveat: Assumes role similarity

Compressed baseline period:
- Traditional: 90 days
- Accelerated: 7-14 days with more aggressive data collection
- Trade-off: Less accurate baseline but faster coverage

Combine:
- Peer group baseline (immediate coverage)
- + Individual baseline (building over 30-90 days)
- + Rule-based detections (catch obvious threats)

Transition: Start with peer baseline, gradually shift to personalized baseline

New employee "alice" starts as Finance Analyst

Week 1-4: Cold Start Period
- Use Finance team baseline
- UEBA compares alice to average Finance behavior
- Alert if alice accesses engineering code (unusual for Finance)

Week 5-8: Hybrid Period
- Combine team baseline (70%) with emerging individual baseline (30%)
- Refine understanding of alice's specific patterns

Week 9+: Individual Baseline
- Sufficient data for personalized baseline
- Anomaly detection tailored to alice's specific behavior

Alert: Malware detected (confidence: 95%)

Black Box: "File is malware" (no explanation)
→ Analyst: "Why? I need to investigate before blocking"

Explainable: "Malware because:"
  - High entropy (7.9) - indicates packing/encryption
  - Suspicious imports: VirtualAlloc, CreateRemoteThread
  - Rare file (prevalence: 0.001%)
  - Threat intel: Hash matches Emotet variant
→ Analyst: "Makes sense, approved for blocking"

Analysts trust models they understand
Black box predictions → skepticism, manual override
Explainable predictions → confidence, faster response

Problem: Model flags benign software as malware

Black Box: Hard to diagnose why

Explainable: "Flagged because high entropy (7.8)"
→ Diagnosis: Legitimate software also has high entropy (compression)
→ Fix: Add additional features (digital signature, prevalence)

Regulations (GDPR, CCPA) require "right to explanation"
- If AI blocks user action, must explain why
- Auditors need to understand decision logic

Alert Scoring Model:
Which features most influenced score?

Feature Importance:
1. Threat Intel Match: 40%
2. Asset Criticality: 25%
3. User Risk Score: 20%
4. Time of Day: 10%
5. Other: 5%

Explanation: "Alert scored high primarily due to threat intel match"

Base risk score: 50/100

Feature Contributions:
+ Threat intel match: +35 points
+ Critical asset: +20 points
- Low user risk: -5 points
- Business hours: -3 points

Final score: 97/100

Explanation: "High score driven by threat intel and critical asset"

IF threat_intel_match = True:
    IF asset_critical = True:
        IF user_privileged = True:
            Risk = CRITICAL
        ELSE:
            Risk = HIGH

Path: threat_intel=True → asset=True → user=False → Risk=HIGH
Explanation: Clear decision path

Alert: Blocked (confidence: 92%)

Explanation: "Would have been allowed if:"
- Threat intel confidence < 80% (currently 95%)
- OR asset criticality = Low (currently Critical)

Helps analyst understand decision boundary

Email: "Urgent: Verify your account"
Prediction: PHISHING (confidence: 94%)

Black Box: No explanation
→ Analyst wastes 5 minutes manually analyzing

Explainable:
Reasons for PHISHING classification:
1. Urgent language: "Urgent", "Verify" (score: +30)
2. External sender + internal lookalike domain (score: +40)
3. Suspicious link (domain age: 2 days) (score: +20)
4. No previous email history with sender (score: +4)

→ Analyst: "Clear phishing, deleting and blocking" (1 minute decision)