Harness Security Assessments as a Service

Module B12 · Course 2B — Securing & Attacking Harnesses and LLMs

60 minutes · The capstone methodology module — turning B0–B11 techniques into a repeatable, scoped, deliverable engagement

The buyer is the CISO / AI security lead. They measure you on the artifact you hand them, not the techniques you ran. B0 built the control plane; B12 ships it as a profession.

Pillar 4 — Frameworks & Governance

From techniques to a service

Every prior module gave you a technique, a control, or a framework. None of them gave you an engagement.
Prior moduleContribution to the service
B0 — legal control planeThe SOW's seven clauses; the scope file; the residual-risk discipline
B1 — threat modelThe reconnaissance phase's surface-enumeration template
B2–B8 — defensesThe controls the report assesses; the remediation routes
B9 — OWASP checklistThe discovery-phase backbone; the findings table (8/2 split)
B10 — Microsoft taxonomyThe chain track; the findings the checklist alone misses
B11 — governanceThe framework the report maps to; the audit trail it evidences

B12 introduces no new technique. It introduces the operational layer that makes B0–B11 a profession.

The six-phase methodology

1. SCOPING ───────▶ 2. RECON ───────▶ 3. DISCOVERY
   B0 scope file       B1 surface map     B9 checklist + B10 chains
         │                    │                    │
         ▼                    ▼                    ▼
6. RETEST ◀─────── 5. REPORT ◀────── 4. EXPLOIT/VALIDATE
   residual risk      B9 output packaged   B0 minimum-proof
   before/after       (5 sections)         reproduce · measure N/M
         │
         └─────. recurring engagement ─────▶ back to 1
Only two phases are the testing B9/B10 supply. Scoping sets the legal plane; reporting makes it a deliverable; retesting measures the residual. An engagement that runs only the testing phases is half an engagement.

B12.1 — Methodology, Scoping & Pricing

The backbone, the four-point enumeration, the seven-clause SOW

Scoping: the four-point enumeration

EnumerateWhy it matters
1. Surfaces (B1 template)Inputs, tools, memory, provider, identity, sandbox, inter-agent edges. The client under-reports; B1 is how nothing stays silent.
2. Model versions (pinned)A finding against v3 is meaningless against v4. Pin checkpoints; record version per finding.
3. Provider authorizationsB0: per technique, ToS / waiver / self-hosted must hold. Fail all three → out of scope until the gap closes.
4. ExclusionsProd PII, shared infra, forbidden techniques. Written into the SOW so drift has a boundary.
Most common scoping mistake: a client who says "jailbreak everything" without checking their provider's terms. The deployer cannot authorize what the provider forbids.

Pricing: three cost drivers

1. Number of agents
Each agent is a separate surface. Multi-agent systems add a chain-complexity premium (B10 inter-agent trust attacks only exist there).
2. Surface complexity
Count B1's elements: tools, persistent memory, retrieval, code-exec, integrations, inter-agent edges. RAG+code-exec+MCP ≫ stateless chatbot.
3. Depth of testing — two sellable depths:
DepthWhat it deliversWhat it misses
Checklist depthB9's ten rows run per agentThe chains between controls
Chain depth (premium)B10 compound-intent chains on high-value surfacesNothing — but priced for high-stakes agents

Never sell a depth you cannot deliver with rigor. Chain depth at checklist-depth hours ships an unvalidated chain.

The SOW: seven clauses (B0 + B12)

#ClauseAI-specific?
1Systems in scope (pinned versions)
2Provider authorization / ToS complianceYES
3Techniques permitted / prohibited (weight-read = minimum-proof only)
4Dual-use & disclosure (180d model / 90d harness; recipe suppressed)YES
5DMCA § 1201 waiver (if bypassing a model access control)YES
6Data handling (Public / Provider-Only / Restricted / Destroy-on-Report)
7Residual-risk measurement protocol (never binary "fixed")
An SOW without clauses 2, 4, 5 fails at the moment a serious finding appears. The three red clauses are the AI-specific additions a traditional pentest SOW does not carry.

B12.2 — The Engagement Report

The scored artifact B9 produces, packaged for a CISO

The report: five sections

SectionContent
1. Executive summaryResidual posture · findings count · ship recommendation (never "secure")
2. FindingsTaxonomy ref · attack procedure · evidence · residual risk · remediation
3. Control matrixControls × surfaces: Present / Absent / Partial / Mismeasured
4. Remediation roadmapPrioritized, routed to B2–B8, owner + effort
5. AppendixMethodology · scope · evidence index (auditable)
B9's checklist output IS the findings section's backbone. B12 does not re-test; it packages. B9 rows → findings F-01..F-10; B10 chains → F-11+; B9 risk→module map → control matrix.

The finding field set (enforced)

Finding ID            F-04
Title                 Indirect injection → file exfil via send_email
Severity              Critical
Taxonomy ref          OWASP ASI01; Microsoft Mode 2 (goal-hijack drift)
Attack procedure      [the multi-step chain, each step's approval gate]
Model version         claude-opus-4-1-20260605   ← pinned
Success rate          60% over 100 attempts
Sampling params       {temperature: 1.0, top_p: 0.95}
Scope reference       SOW §2.3 / scope-file provider_authorization
Result                MEASURED
Residual risk         4% after L4 taint gate enabled mid-engagement
Remediation module    B2 (Prompt Injection Defense)
Timestamp UTC         2026-07-09T14:22:03Z
The report generator rejects: a finding with no taxonomy ref, a MEASURED finding with no success rate, a Critical finding with no remediation route. The strictness is the honesty.

B12.3 — Retesting & Packaging

Residual risk, not binary "fixed"; a repeatable practice

Retesting: residual, not "fixed"

The client asks "is it fixed?" — the answer is never yes. The answer is "injection success moved from 60% to 4% over 100 attempts, same harness, same sampling, same pinned version. Residual characterized at 4%."
VerdictMeaning
ResolvedResidual → 0% (deterministic control added)
Improved60% → 4% (mitigation reduced, did not eliminate)
Unchanged60% → 60% (remediation ineffective)
RegressedA PASS row → FAIL, or a B10 chain rerouted through a new gap

The retest re-runs the full B9 checklist and re-attempts the B10 chains — patching one step can reroute a chain through a different gap.

Packaging the service: four layers

LayerWhat it makes the practice
MethodologyThe six-phase playbook with phase→module mapping. Repeatable.
ToolingB9 executor · B10 chain harness · report generator · scope validator. Scalable.
TemplatesSOW (7 clauses) · scope file · report (5 sections) · retest delta. Consistent.
EvidenceB0 store (4 classes) · B11 audit trail · B11 AI BOM. Defensible.
The long-term value a CISO buys: comparable artifacts across releases — the injection-success-rate trend line over six releases. Drift in any layer breaks the comparability.

The capstone synthesis

B0 → SOW. B1 → reconnaissance. B2–B8 → the controls assessed. B9 + B10 → discovery. B11 → the framework + audit trail. B0.2 → the retest. None of it is new; all of it is synthesized.

At the end of B0–B12 you can do four things: build a hardened agent harness (Capstone B1), attack it with the full methodology, defend it with the full control set, and — the contribution of this module — deliver the assessment of it as a scoped, priced, reported, retestable service. That last capability is what turns the techniques into a practice a CISO will buy.

Lab & what's next

Lab (07): build the assessment report generator — ingest B9's checklist-executor JSON, emit the five-section engagement report with strict validation (no unclassified findings, no MEASURED without a rate, no Critical without remediation). Plus write a sample SOW with all seven clauses.

Next — Capstone B1: Build a Hardened Agent Harness. The synthesis module you've earned. B12's methodology is the lens; Capstone B1 is the system you assess. The report generator you build here is the artifact you'd run against it.