Skip to content

docs: close the seventh audit's verified findings - #835

Merged
khushalsonawat merged 1 commit into
devfrom
docs/audit-remediation
Aug 25, 2026
Merged

docs: close the seventh audit's verified findings#835
khushalsonawat merged 1 commit into
devfrom
docs/audit-remediation

Conversation

@khushalsonawat

Copy link
Copy Markdown
Contributor

What

Closes the five actionable findings from the seventh docs audit, every one verified against product source before a page changed.

  • Mermaid diagram source now survives into llms-full.txt as fenced blocks (51 diagrams across 47 pages were stripped with the JSX)
  • sdk/simulate rewritten around the SDK's single dual-mode run_test(): cloud args and local LiveKit args dispatch differently and never combine; the examples had been mixing them, which silently dropped parameters
  • Guardrail status codes aligned with the gateway across 8 pages: block returns 403 with a guardrail_error/content_blocked body; warn and log return 200 plus the x-agentcc-guardrail-triggered header. The 446/246 convention never shipped
  • Warnings on the caching and guardrails pages: an exact cache hit skips request-side guardrail screening; response-side checks still run
  • Experiment is no longer presented as a product; those signals flow into Evaluate

Why

An independent 742-page audit surfaced five claims that break working code or misrepresent the product. Verification confirmed three fully, narrowed one (cache bypass is input-side only), and disproved parts of another (446/246 exist only in dead code and an unwired client SDK).

What cases were covered

  • run_test verified against the released agent-simulate 0.1.3 wheel and its dispatch logic, not just repo source; the reference page checked out fully and was left untouched
  • Status codes traced to the gateway's error construction and header-setting call sites; Protect's SDK path confirmed to return plain 200s
  • Cache short-circuit traced through the plugin pipeline, including the post-stage skip list
  • Mermaid conversion tested against the corpus: 51 of 51 components convert
  • Four product bugs found during verification were filed separately, not papered over in docs

How

Four source-verification agents (simulate SDK, gateway pipeline, status codes, product framing) plus an extractor change, each landing the smallest edit that resolves its finding.

Recording

Not applicable: text and export changes only.

🤖 Generated with Claude Code

- export Mermaid diagram source into llms-full.txt as fenced blocks
  (51 diagrams across 47 pages were being stripped with the JSX)
- rewrite sdk/simulate around the SDK's single dual-mode run_test:
  cloud args and local LiveKit args dispatch differently and do not
  combine; examples no longer mix them
- align guardrail status codes with the gateway: 403 with a
  guardrail_error/content_blocked body on block, 200 plus the
  x-agentcc-guardrail-triggered header on warn and log; the 446/246
  convention never shipped and is gone from all five Protect pages
- document that an exact cache hit skips request-side guardrail
  screening while response-side checks still run
- stop presenting Experiment as a product; signals flow into Evaluate

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
@khushalsonawat khushalsonawat self-assigned this Aug 25, 2026
@khushalsonawat
khushalsonawat merged commit cc7028d into dev Aug 25, 2026
1 check passed
@khushalsonawat
khushalsonawat deleted the docs/audit-remediation branch August 25, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants