Skip to content

Verification Debt Status

Generated: 2026-04-24 (s56-audit)

This page consolidates every "verification debt" item the project has accumulated — claims that ship in tools, content, or scaffolding but have not been independently verified against canonical sources. Each item below has one of four statuses:

  • CLOSED — verified by Nexus maintainer or a cross-checked test, or rendered impossible to drift via a regression test.
  • DOCUMENTED — the limitation is honestly disclosed in-product; closing it requires external work that's a deliberate non-goal for the educational scope.
  • PENDING-EXTERNAL — needs network access, vendor materials, or a paid resource that the audit session did not have. A specific recipe for closing it is documented below.
  • HUMAN-ONLY — fundamentally cannot be done by Claude. Requires a real practitioner, a real network, or both.

This file is the single source of truth for verification status. It supersedes the scattered "verification debt" lists in NEXT_SESSION.md, EVOLUTION_LOG.md, and individual tool docs.

Summary table

# Item Status Last action Closing requires
1 FSRS-5 weight role-comment vs code CLOSED (s56-audit) Comment table corrected to match code; golden test added (7/7 PASS) —
2 FSRS-5 numerical weight values vs canonical OSR repos PENDING-EXTERNAL Verification note in fsrs.js; golden test pins current behavior Network access to OSR repos; cross-check vs fsrs.js / py-fsrs / fsrs-rs defaults
3 VC proof-type vs Data Integrity URDNA2015 spec (4 demos) CLOSED (s56-audit) Per-field conformance audit at vc-demo-spec-conformance.md; deviations honestly disclosed in each demo —
4 Validating an actual conformant VC against the demo verifier DOCUMENTED Recipe in vc-demo-spec-conformance.md Verification Debt section A sample VC from the W3C VC test suite + manual paste-and-observe
5 Security+ SY0-701 cost / domain weights / DoD 8570 status VERIFIED 2026-04-24 (s56-audit) Cost CORRECTED $392→$425; weights CONFIRMED; prereq wording REPHRASED; DoD framework noted as 8570→8140 transition; DoD page direct verification blocked by SAML/CAC DoD page direct re-verification needs CAC login (out of scope)
6 CISSP AMF / CAT format / domain weights / endorsement window VERIFIED 2026-04-24 (s56-audit) AMF CONFIRMED $135; CAT CONFIRMED 100-150 / 3hr; non-English linear retired April 2024 (CORRECTED); D1 weight CORRECTED 15→16, D8 CORRECTED 11→10 (April 2024 outline); endorsement, Associate, prereqs, passing score all CONFIRMED —
7 CKS K8s version / cost / CKA prerequisite VERIFIED 2026-04-24 (s56-audit) 3 deltas: K8s version v1.27+→v1.34, cost $395→$445, CKA prereq recommended→MANDATORY; allowed sites EXPANDED to 10 domains; weights CONFIRMED unchanged —
8 CEH version (v12 vs v13) / cost / DoD 8570 / eligibility VERIFIED 2026-04-24 (s56-audit) Version CORRECTED v12→v13; cost CONFIRMED $1,199 Pearson VUE / ~$950 ECC online; eligibility EXPANDED ($100 application + 2yr exp); DoD CSSP Analyst CONFIRMED indirectly (DoD page CAC-walled); passing score band 60-85% per-form DoD page direct re-verification needs CAC login (out of scope)
9 Y.js POCs end-to-end CRDT replication test HUMAN-ONLY (procedure documented s56-audit) Procedure in y-js-poc-deployment.md § "HUMAN-ONLY end-to-end manual test" Two browser instances + two networks + 60-90 minutes
10 WCAG 2.2 AA manual third-party AT test (NVDA / VoiceOver / JAWS) HUMAN-ONLY (narrowed s56-audit) Audit + remediation s51-s54; automated axe-core 4.11.3 + pa11y 9.1.1 scan completed s56-audit (wcag-automated-scan.md) — 130 raw findings on 12 representative pages, 11 unique rules; R1-R5 regressions ALL CLOSED s56-audit (4 form-label fixes + 2 contrast fixes across did-web-vc-demo / incident-cost-calculator / skill-portfolio; +3 additional unlabeled fields uncovered + fixed during the rescan; final pa11y scan confirms 0 F68/H91/G18 errors on those 3 pages). Truly human-only scope now narrowed to screen-reader UX (announcement quality, live-region timing) + cognitive accessibility + reading-order (1.3.2) + the ~70 NEW findings on un-remediated tools that the next remediation pass should triage A practitioner with screen-reader expertise; 1-2 hours per tool
11 Digital Twin Bloom mapping vs peer-reviewed psychometric DOCUMENTED Honest disclosure in tool: "defensible heuristic, not peer-reviewed" A psychometrician's review; out of scope for the educational tool
12 Temporal Knowledge Graph snapshots 2026-01/02/03 DOCUMENTED Honest disclosure in tool: 3 of 4 snapshots are synthetic projections None — disclosure is the closure
13 Content Versioning "any commit counts as updated" DOCUMENTED Honest disclosure in tool None — disclosure is the closure
14 Cross-System Linking Matrix counts ID-mention not semantic coverage DOCUMENTED Honest disclosure in tool + report None — disclosure is the closure
15 Detection-vs-Scenario Coverage = technique-ID overlap, not real query execution DOCUMENTED Honest disclosure in detection-coverage.md; 140 uncatalogued = real signal Building a KQL/SPL parser + log-stream simulator; substantial separate task
16 Regulatory Radar 5 unverified RSS feeds DOCUMENTED TODO comments in regulatory_feed_consumer.py with discovery commands One-time URL discovery for FedRAMP / ENISA / CIS Benchmarks / EU AI Act / CCPA
17 Self-Modifying Brain META-EVALUATE without LLM = template-only DOCUMENTED Disclosure in nexus_brain.py SELF_MODIFY_LLM_FENCE constant; circuit breaker active LLM API key + Brain L1 active + reviewer covenant
18 Brain L1 dry-run guard does not exist (s55 honest discovery) CLOSED (s55) Documented in brain-l1-activation-playbook.md; no code change needed —
19 Brain L1 actual production activation HUMAN-ONLY (gated on GitHub Actions billing) Trigger attempted s56 (run #24879235467 → billing-block); 3 resolution paths documented Fix GH Actions billing OR repo public OR wait until 2026-05-01
20 Anthropic provider per-call cost cap / budget enforcement DOCUMENTED Disclosed in llm_router.py module docstring + ai-architecture.md Maintainer sets Anthropic console rate limits; not a router-side feature

Status totals

  • CLOSED: 3 (items 1, 3, 18) — items materially resolved by code/docs in earlier audit cycles. (Self-tally previously stated "4"; re-derived from live row parse 2026-04-25 in s56-audit.)
  • VERIFIED: 4 (items 5, 6, 7, 8 — closed in this 2026-04-24 cert-path audit cycle via live WebFetch + WebSearch cross-checks)
  • DOCUMENTED: 9 (items 4, 11, 12, 13, 14, 15, 16, 17, 20) — limitations honestly disclosed; no further action needed without external work. (Self-tally previously stated "8"; re-derived from live row parse 2026-04-25 in s56-audit.)
  • PENDING-EXTERNAL: 1 — needs network access for canonical cross-check
  • HUMAN-ONLY: 3 — fundamentally requires a real practitioner

Total: 3 + 4 + 9 + 1 + 3 = 20 ✓ (matches the 20-row table above; verified by awk -F'|' '/^\| [0-9]+ \|/ {...}' on 2026-04-25)

Note: items 5-8 (cert paths) were upgraded from DOCUMENTED → VERIFIED on 2026-04-24 via live WebFetch / WebSearch against the canonical vendor pages. Three of the four paths required real corrections (Sec+ price, CKS version+price+prereq, CEH version + CISSP weights + non-English format retirement). DoD baseline page direct verification was blocked for both Sec+ and CEH because public.cyber.mil/wid/cwmp/... now requires DoD CAC/SAML auth — DoD claims were verified indirectly via vendor pages and 8140 transition guidance.

How to use this file

  • Before publishing or recommending any tool/path/script to a real practitioner, read the corresponding row above. Make sure the deviation it discloses is acceptable for your use case.
  • When fixing or verifying an item, update its row here AND in the corresponding source doc/file. Then update the EVOLUTION_LOG.md with the closure entry.
  • When adding new tools or content with non-trivial external dependencies, add a new row here at creation time. Don't accumulate undisclosed assumptions.

Cross-references