"Name the six phases of the agent security assessment methodology (B12), and the prior module that supplies each phase's content." "(1) SCOPING → B0 (legal control plane → SOW + scope file). (2) RECONNAISSANCE → B1 (threat-model surface enumeration). (3) DISCOVERY → B9 (OWASP checklist) + B10 (Microsoft chains). (4) EXPLOITATION/VALIDATION → B0 (minimum-proof discipline, reproduce N/M). (5) REPORTING → B9 (scored output packaged). (6) RETESTING → B0.2 (residual-risk measurement). The methodology adapts PTES / NIST SP 800-115 to the agentic surface." c2b::b12::recall "What traditional pentest methodology standards does B12's six-phase assessment adapt, and what changes vs. what stays the same?" "PTES (Penetration Testing Execution Standard) and NIST SP 800-115 ('Technical Guide to Information Security Testing and Assessment'). The phase NAMES are unchanged (Scoping, Recon, Discovery, Exploitation/Validation, Reporting, Retesting). What changes is the CONTENT of each phase, because the target is an agent not a network — surfaces are tools/memory/prompts/edges, not IPs/ports; provider authorizations (B0) gate scoping; residuals are measured not binary." c2b::b12::recall "Why is 'run B9's checklist and hand over the output' only half an engagement?" "The checklist supplies two of six phases (Discovery + part of Validation). The other four are: Scoping (where B0's legal plane gets written into the SOW), Reconnaissance (B1 surface map), Reporting (where the scored output becomes a deliverable a board can read), and Retesting (where B0.2's residual-risk discipline proves the controls moved the needle). Skip any phase and the engagement is either illegal, unsellable, or unverifiable." c2b::b12::recall "State the four-point surface enumeration required at scoping." "(1) SURFACES — use B1's threat-model template (inputs, tools, memory, provider, identity, sandbox, inter-agent edges); the client under-reports, so the template ensures nothing stays silent. (2) MODEL VERSIONS — pinned checkpoints (a finding against v3 is meaningless against v4). (3) PROVIDER AUTHORIZATIONS — per technique, B0's three conditions (ToS permits / waiver on file / self-hosted); fail all three and the surface is OUT OF SCOPE until the gap closes. (4) EXCLUSIONS — prod PII, shared infra, forbidden techniques; written into the SOW so drift has a boundary." c2b::b12::recall "What are the three cost drivers for pricing an agent security assessment?" "(1) NUMBER OF AGENTS — each agent is a separate surface; multi-agent systems add a chain-complexity premium (B10 inter-agent trust attacks only exist there). (2) SURFACE COMPLEXITY — count B1's elements (tools, persistent memory, retrieval, code-exec, integrations, inter-agent edges); RAG+code-exec+MCP ≫ stateless chatbot. (3) DEPTH OF TESTING — checklist depth (B9's ten rows per agent, baseline) vs. chain depth (B10 compound-intent chains, premium for high-stakes agents)." c2b::b12::recall "Distinguish checklist depth from chain depth as sellable engagement depths." "CHECKLIST DEPTH runs B9's ten OWASP rows deterministically/measured against each agent — baseline engagement, reports 'no missing controls.' CHAIN DEPTH constructs B10's compound-intent chains (the seven failure modes + zero-click HITL bypass) against the highest-value surfaces — premium engagement for high-stakes agents (financial, healthcare, autonomous), reports 'and here is the compound chain that slips between them.' A client who buys only checklist depth gets a report that misses the gaps between controls." c2b::b12::recall "What is the honest pricing rule for assessment depth?" "Never sell a depth you cannot deliver with rigor. A chain-depth engagement priced at checklist-depth hours will ship a B10 chain that has not been validated end-to-end. Underpriced assessments produce under-rigored reports, and under-rigored reports are how an agent ships with a real vulnerability the assessment said was fixed." c2b::b12::recall "Name the seven clauses of an AI red-team SOW, and identify which three are AI-specific." "(1) Systems in scope (pinned versions). (2) PROVIDER AUTHORIZATION / ToS compliance [AI-SPECIFIC]. (3) Techniques permitted/prohibited (weight-read = minimum-proof only). (4) DUAL-USE & DISCLOSURE — 180d model-level / 90d harness-level, recipe suppressed by default [AI-SPECIFIC]. (5) DMCA § 1201 WAIVER if bypassing a model access control [AI-SPECIFIC]. (6) Data handling (Public/Provider-Only/Restricted/Destroy-on-Report). (7) Residual-risk measurement protocol (never binary 'fixed'). Clauses 2, 4, 5 are the AI-specific additions a traditional pentest SOW does not carry." c2b::b12::recall "Name the five sections of the engagement report and the purpose of each." "(1) EXECUTIVE SUMMARY — residual posture, findings count, ship recommendation (never 'secure'; calibrated to risk appetite). (2) FINDINGS — one entry per finding with the field set (taxonomy ref, attack procedure, evidence, residual risk, remediation). (3) CONTROL MATRIX — controls × surfaces grid: Present/Absent/Partial/Mismeasured (the gap analysis a regulator reads). (4) REMEDIATION ROADMAP — findings prioritized, routed to B2-B8, owner + effort. (5) APPENDIX — methodology, scope, evidence index (the reproducibility/audit layer)." c2b::b12::recall "What is the required field set for every finding in the report, and which field is the taxonomy reference?" "Finding ID; Title; Severity (Critical/High/Medium/Low); TAXONOMY REFERENCE (the OWASP ASI row from B9 and/or Microsoft failure mode from B10 — required, no orphan findings); Attack procedure; Evidence (B0 minimum-evidence: model version pinned, success rate over N attempts, sampling params, timestamp UTC, scope reference); Result (PASS/FAIL/MEASURED/N/A); Residual risk (measured rate after mitigation); Remediation module (which of B2-B8 builds the fix)." c2b::b12::recall "How does B12's engagement report relate to B9's checklist executor output?" "B9's checklist executor output IS the findings section's backbone. B12 does not re-test; it packages. The B9 rows (8 PASS/FAIL + 2 MEASURED) become findings F-01 through F-10. The B10 chains become additional findings (F-11+) that the checklist alone misses. The B9 risk-to-module mapping (ASI → B2/B3/B4/B5/B7/B8) becomes the control matrix read as a coverage view. Every finding maps to a taxonomy reference — a finding with no OWASP row and no Microsoft mode is a finding that has not been classified, and classification is what makes it actionable for the governance layer (B11)." c2b::b12::recall "What four classes does the report generator reject, and why is the strictness the honesty?" "The report generator rejects: (a) a finding with no taxonomy reference (unclassified = not actionable for governance); (b) a MEASURED finding without a success rate (anecdote, not a finding); (c) a Critical finding without a remediation route (not actionable); (d) [implicit] anything that would collapse MEASURED into PASS. The strictness is the honesty — a generator that emits whatever it is given produces the '10/10 PASS' lie B9 refuses. The generator enforces B9's discipline at the OUTPUT layer." c2b::b12::analysis "Why is the retest verdict never 'fixed'? State the four possible verdicts." "Because AI findings are not binary (B0.2). The verdict is based on measured residual risk under identical conditions (same harness, same sampling params, same pinned model version): (1) RESOLVED — residual → 0% (a deterministic control was added). (2) IMPROVED — e.g. 60% → 4% (mitigation reduced, did not eliminate). (3) UNCHANGED — remediation ineffective. (4) REGRESSED — a PASS row regressed to FAIL, or a B10 chain rerouted through a new gap. 'Is it fixed?' is the wrong question; 'did the residual move, and by how much, under identical conditions?' is the right one." c2b::b12::recall "Why does the retest re-run the FULL B9 checklist and re-attempt the B10 chains, rather than just re-checking the original finding?" "Because a remediation that closes the original finding may open a new one. A deterministic control added for one row can break another (a PASS regresses to FAIL). B10's compound-intent chains are especially prone: patching one step of a chain may reroute it through a different gap, so the compound still completes. A retest that checks only the original finding misses regressions and rerouted chains. The retest must also PIN the model version — if the client bumped the version, the before/after is invalid and both versions must be reported separately." c2b::b12::analysis "What is the role of the model-version pin in the retest, and what happens if the client bumped the version between test and retest?" "The pin isolates the remediation effect. If the client bumped the model version between the original test and the retest, the before/after comparison is INVALID — the residuals moved because of the version bump, not the remediation. The retest must EITHER run against the original pinned version (to isolate the remediation effect) OR report both versions separately (to show the combined effect of remediation + version). A retest against an unpinned or changed version is not a retest of the remediation." c2b::b12::analysis "Name the four layers of service packaging and what each makes the practice." "(1) METHODOLOGY — the six-phase playbook with phase→module mapping (makes it REPEATABLE: a new assessor runs an engagement the same way). (2) TOOLING — B9 executor, B10 chain harness, report generator, scope-file validator (makes it SCALABLE: a senior assessor runs concurrent engagements). (3) TEMPLATES — SOW (7 clauses), scope file, report (5 sections), retest delta, CVD timeline (makes it CONSISTENT: every engagement produces the same artifact shape). (4) EVIDENCE — B0 store (4 retention classes), B11 audit trail, B11 AI BOM (makes it DEFENSIBLE: every finding traces to scoped, authorized, minimum-proof evidence)." c2b::b12::recall "Why is comparability across releases the long-term value a CISO buys, and what breaks it?" "A CISO who buys the service across six releases wants to see the residual trend move (e.g. injection success rate over six releases). That is only possible if every engagement ran the same methodology, the same tooling, against the same report template. DRIFT in any layer — methodology, tooling, templates, or evidence — breaks the comparability. Comparability is the long-term value because it lets the client prove to their board/regulator that controls are improving over time, and it flags a regression before it reaches production." c2b::b12::analysis "From the course spec, who is the buyer of a B12 assessment, and what does that buyer read first?" "The CISO or AI security lead. They read FRAMEWORKS, not exploits. Before anything else they ask: which control framework does this map to? (B9 OWASP / B10 Microsoft / B11 NIST AI RMF); what is the residual risk in numbers? (B9's measured rows); can my auditor read this? (B11's audit trail). A beautiful exploit chain that does not map to a control row will not move budget. A measured residual that maps to a NIST AI RMF Measure function will. The methodology is calibrated to that buyer." c2b::b12::recall "A client demands the report say '10/10 PASS' after remediation. Refuse using the module's principles." "Refuse. B9's Result-type column is fixed by the KIND of control, not the outcome — two rows (ASI01, ASI06) are MEASURED because their controls are semantic and have a bypass/miss rate; they can never be PASS. The honest report says '8 PASS + 2 MEASURED at X% and Y%, characterized as the encoded-and-laundered class and the unverified-claim-propagation class.' Collapsing MEASURED into PASS is asking the engineer to lie about the probabilistic layers. The report generator enforces this at the output layer. The client's governance layer (B11) actually needs the measured residual — it maps to NIST AI RMF Measure, which is what moves budget." c2b::b12::analysis "You are scoping an assessment. The client's agent uses a frontier model via API, a retrieval store with customer PII, and a code-exec tool. The client says 'jailbreak and extract everything.' Walk the scoping steps." "(1) SURFACES via B1: API model (provider surface), retrieval store (deployer, PII), code-exec tool (deployer), inputs/memory/inter-agent edges. (2) PIN the model version. (3) PROVIDER AUTHORIZATION: for jailbreak + weight-read against the API model, check B0's three conditions — does the provider ToS permit it? is a waiver/preview enrollment on file? is it self-hosted? If all three fail, those techniques are OUT OF SCOPE until the client enrolls in the provider's red-team program or gets a waiver. (4) EXCLUSIONS: retrieval-store PII is Restricted/destroy-on-report (B0); weight-read is minimum-proof only (path+hash+bytecount, never the file). Write the seven-clause SOW; the client's counsel signs before testing." c2b::b12::analysis "Design the engagement report generator's validation function. What does it check per finding, and when does it refuse to ship?" "Per finding, validate(): (a) taxonomy_ref is non-empty (OWASP ASI row and/or Microsoft mode) — else REJECT 'missing taxonomy reference'; (b) model_version is pinned — else REJECT 'missing pinned model version'; (c) if result == MEASURED, success_rate is non-empty — else REJECT 'MEASURED requires a success rate'; (d) if severity in (Critical, High), remediation_module is non-empty — else REJECT 'no remediation route'. validate_all() runs every finding and returns the error list; the report does not ship until the list is empty. The strictness enforces B9's honesty at the output layer and prevents the '10/10 PASS' lie." c2b::b12::analysis "How does B12 synthesize B0–B11 into a single deliverable? Trace each prior module to its contribution in the engagement." "B0 → the SOW's seven clauses, the scope file, the residual-risk retest protocol, the minimum-evidence field set. B1 → the reconnaissance phase's surface-enumeration template. B2–B8 → the controls the report assesses and the remediation routes. B9 → the discovery-phase checklist backbone; its scored output (8/2) is the findings table; the risk→module map is the control matrix. B10 → the chain track; the additional findings the checklist misses; the chain-depth engagement. B11 → the framework (NIST AI RMF) the report maps to and the audit trail/AI BOM it evidences. B0.2 → the retest's residual-risk measurement. None is new; all are synthesized into one report." c2b::b12::analysis