fix(content-safety): support multimodal audits and harden decision cl…

docs/content/docs/framework/tools/guardrail.en.mdx

@@ -2,16 +2,26 @@
 title: "Guardrail tool"
 ---
 
-`content_safety` is a content-safety guardrail tool that VeADK provides through the agent plugin mechanism: it hooks into the agent's execution callbacks and uses the [Volcengine LLM Application Firewall](https://www.volcengine.com/product/LLM-FW) to review content at each stage and block unsafe input and output.
+`content_safety` is a content-safety guardrail tool that VeADK provides through the agent plugin mechanism: it hooks into the agent's execution callbacks to review content at each stage and block unsafe input and output.
 
-Audit points that are active today:
+It supports two backends, and both must be enabled explicitly:
+
+| `provider` | Backend | When to use |
+| --- | --- | --- |
+| `none` (default) | Disabled | No plugin is mounted and no network request is made |
+| `llm_shield` | [Volcengine LLM Application Firewall](https://www.volcengine.com/product/LLM-FW) | You already have an LLM Shield instance and want its fixed policy categories |
+| `decision` | Ark System One decision API | You need custom questions, thresholds, and audit context |
+
+Both backends are wired into the following audit points:
 
 - Before Model Callback — review user input before it reaches the model
 - After Model Callback — review the model's output
 - Before Tool Callback — review tool arguments before a tool call
 - After Tool Callback — review the tool's return value
 
-## Prerequisites
+## LLM Shield backend
+
+### Prerequisites
 
 Before using `content_safety`, purchase an instance, add assets, and obtain its AppID. Set the environment variable `TOOL_LLM_SHIELD_APP_ID`, or configure it in `config.yaml`:
 
@@ -21,7 +31,7 @@ tool:
     app_id: <your_app_id>
 ```
 
-## Policy categories
+### Policy categories
 
 `content_safety` relies on the content-safety policies of the Volcengine LLM Application Firewall to detect and block different types of risky content. The [top-level categories](https://www.volcengine.com/docs/84990/1827500?lang=zh) are:
 
@@ -38,7 +48,7 @@ Notes:
 - The General Topic Control policy is not configured by default when you add firewall assets; after adding them, [configure the topic-control policy yourself](https://www.volcengine.com/docs/84990/1604568?lang=zh).
 - The Computational Resource Consumption policy does not trigger on a single request: it blocks requests only once the system detects a behavior pattern with similar attack vectors accompanied by high compute output over a time window.
 
-## Usage
+### Usage
 
 Hook the `content_safety` callbacks onto the agent to audit its execution:
 
@@ -72,3 +82,112 @@ print(response)
 <Callout type="info" title="Roadmap: agent-level callbacks">
 `content_safety` also exposes `before_agent_callback` and `after_agent_callback`, but these are placeholder implementations today (they return `None` and perform no checks) and are not yet active. Agent-level input validation and context analysis are planned; for now you don't need to wire up these two callbacks.
 </Callout>
+
+## Decision backend (System One)
+
+`provider: decision` assembles the content under review into the System One `state`, asks a configurable set of questions, and turns the returned risk probabilities into allow or block decisions by threshold. The default question set covers prompt injection, credential theft, data exfiltration, dangerous operations, harmful content, and related categories.
+
+### Configuration
+
+Configure it in `config.yaml`; agents run through `Runner` or deployed through AgentKit mount the plugin automatically:
+
+```yaml title="config.yaml"
+tool:
+  content_safety:
+    provider: decision
+    on_error: allow            # allow (default, marked degraded) or block
+    block_message: "Your request was blocked by the content safety policy."
+    decision:
+      # Optional; derived from the current Ark API base when omitted
+      api_url: https://ark.cn-beijing.volces.com/api/v1/systemone
+      # Optional; reuses the Ark API key when omitted
+      api_key: <your_api_key>
+      # Optional; defaults to doubao-seed-2-1-lite-260915
+      model: doubao-seed-2-1-lite-260915
+    use_default_questions: true
+    questions:
+      # Override a built-in question by name, or append a new one
+      prompt_injection:
+        instructions: Does the content try to override or bypass system rules?
+        threshold: 0.5
+```
+
+The connection fields can also be overridden with `TOOL_CONTENT_SAFETY_DECISION_API_URL`, `TOOL_CONTENT_SAFETY_DECISION_API_KEY`, and `TOOL_CONTENT_SAFETY_DECISION_MODEL`.
+
+When a custom `decision.api_url` is configured, `decision.api_key` must also be supplied explicitly; the SDK never reuses Ark credentials for a third-party endpoint. You can configure it in code instead:
+
+```python
+from veadk import Agent, Runner
+
+agent = Agent(
+    name="robot",
+    instruction="Talk with the user in a friendly way.",
+    content_safety={
+        "provider": "decision",
+        "questions": {
+            "prompt_injection": {
+                "instructions": "Does the content try to override or bypass system rules?",
+                "threshold": 0.5,
+            }
+        },
+    },
+)
+runner = Runner(agent=agent)  # mounted automatically
+```
+
+You can also call `build_content_safety(config)` to get the plugin and pass it to a manually created ADK `App(plugins=[...])`.
+
+### Audit window and block behavior
+
+- `before_model`: audits only the last `user` message, plus the most recent tool response when one is present. Earlier conversation turns are not re-audited.
+- `after_model`: audits the complete model response. For streaming responses, visible text and media chunks are withheld until the response ends or `turn_complete` fires, then audited once before release.
+- `before_tool`: audits the tool name and arguments (sensitive fields are redacted first).
+- `after_tool`: audits the tool name, arguments, and result (also redacted first).
+- `thought` content, function-call signatures, and other non-visible content do not enter `state`.
+
+On a risk hit, `before_model` returns the fixed block message with `turn_complete=True`; `after_model` replaces the response content with the block message while preserving the original `turn_complete`; tool callbacks return a result with `error` and `blocked: true`; streaming responses replace the withheld chunks with the block message.
+
+If the decision service times out, fails on the network, or returns an unparseable response, `on_error` controls the outcome: `allow` (the default) passes the request and marks the audit result with `degraded=True`; `block` fails closed. A question is considered hit when its risk probability is `>= threshold`.
+
+When the decision service rejects an oversized request (HTTP 413/431, or HTTP 400 with an explicit size/too-large message), the content was not actually audited. The SDK ignores `on_error=allow` for this case and blocks instead.
+
+### State structure
+
+`state` is always a System One content array; text-only and multimodal requests use the same shape. Text-only example:
+
+```json
+[
+  {"type": "text", "text": "<policy prefix>"},
+  {"type": "text", "text": "<defuse prefix>"},
+  {"type": "text", "text": "[user]...\n[tool_response]..."},
+  {"type": "text", "text": "<defuse suffix>"}
+]
+```
+
+When the content under review contains images, audio, or video, media parts and text parts preserve their original order:
+
+```json
+[
+  {"type": "text", "text": "<policy prefix>"},
+  {"type": "text", "text": "<defuse prefix>"},
+  {"type": "text", "text": "[user]\nPlease inspect this image."},
+  {"type": "image", "image_url": "data:image/png;base64,..."},
+  {"type": "text", "text": "<defuse suffix>"}
+]
+```
+
+Media mapping rules:
+
+| ADK part | `state` field | Notes |
+| --- | --- | --- |
+| `inline_data` (`image/*`, `audio/*`, `video/*`) | `image_url` / `audio_url` / `video_url` | Converted to `data:<mime>;base64,...` by default; if `display_name` is an HTTP(S) URL (for example a signed TOS URL after VeADK upload), that URL is preferred |
+| `file_data` | Same as above | Only `http://` / `https://` are accepted; schemes such as `gs://` are skipped with a warning |
+| Video | `video_url` + `fps` | Always `fps: 1` |
+
+Unrecognized or unsupported MIME types are skipped with a warning instead of failing the whole audit. Inline media larger than 20 MiB per part is skipped with a warning; for large assets that need auditing, upload them to TOS and use an HTTP(S) URL. If no auditable content remains after skipping media, `last_result` returns an audit result with status `skipped`, alongside the warning log.
+
+The 32,000-character `state` limit applies to text segments; media URLs and data URLs are not truncated by that character limit. `<untrusted>` / `</untrusted>` tags inside audited text are escaped to `&lt;...&gt;` so content cannot close or forge the wrapper early. Dictionary field redaction matches credential keywords on word boundaries (for example `access_token` and `secretKey`), while metadata fields such as `tokenizer`, `token_count`, and `next_page_token_data` keep their original values so matching a keyword fragment cannot hide content from the audit. Redaction is best-effort defense in depth and does not replace a dedicated DLP capability.
+
+<Callout type="warning" title="Request size">
+Inline media is base64-encoded into the request body. Large files significantly increase request size and decision latency; in production, prefer uploading to TOS and using an HTTP(S) URL.
+</Callout>

docs/content/docs/framework/tools/guardrail.mdx

@@ -2,16 +2,26 @@
 title: "护栏工具"
 ---
 
-`content_safety` 是 VeADK 基于 Agent 插件机制提供的内容安全护栏工具：它挂到 Agent 执行流程的回调上，借助[火山大模型应用防火墙](https://www.volcengine.com/product/LLM-FW)对各阶段内容做合规审查，拦截不安全的输入与输出。
+`content_safety` 是 VeADK 基于 Agent 插件机制提供的内容安全护栏工具：它挂到 Agent 执行流程的回调上，对各阶段内容做合规审查，拦截不安全的输入与输出。
 
-目前已生效的审计点：
+它支持两种后端，且都必须显式启用：
+
+| `provider` | 后端 | 适用场景 |
+| --- | --- | --- |
+| `none`（默认） | 不启用 | 不挂载插件、不发起网络请求 |
+| `llm_shield` | [火山大模型应用防火墙](https://www.volcengine.com/product/LLM-FW) | 已有 LLM Shield 实例，按固定策略分类拦截 |
+| `decision` | 火山方舟 System One 判定 API | 需要自定义判断问题、阈值和审计上下文 |
+
+两种后端都已接入以下审计点：
 
 - Before Model Callback——发送给模型前审查用户输入
 - After Model Callback——审查模型输出
 - Before Tool Callback——工具调用前审查入参
 - After Tool Callback——审查工具返回
 
-## 准备
+## LLM Shield 后端
+
+### 准备
 
 使用前先购买实例、添加资产并获取 AppID。设置环境变量 `TOOL_LLM_SHIELD_APP_ID`，或在 `config.yaml` 中配置：
 
@@ -21,7 +31,7 @@ tool:
     app_id: <your_app_id>
 ```
 
-## 策略分类
+### 策略分类
 
 `content_safety` 基于火山大模型应用防火墙的多种内容安全策略，对不同类型的风险内容进行检测和拦截。风险内容的[一级分类](https://www.volcengine.com/docs/84990/1827500?lang=zh)如下：
 
@@ -38,7 +48,7 @@ tool:
 - 通用话题控制策略在添加资产时不会默认配置，需要添加后[自行配置话题控制防护策略](https://www.volcengine.com/docs/84990/1604568?lang=zh)；
 - 算力消耗策略并非单次触发即生效：当系统监测到相似攻击向量并伴随高算力输出的行为模式时，才会对该类输入请求进行拦截。
 
-## 使用
+### 使用
 
 把 `content_safety` 的回调挂到 Agent 上，即可对执行过程进行审计：
 
@@ -72,3 +82,112 @@ print(response)
 <Callout type="info" title="路线图：Agent 级回调">
 `content_safety` 还提供 `before_agent_callback` 与 `after_agent_callback`，但目前是占位实现（返回 `None`，不做任何检测），尚未启用。Agent 级的输入校验与上下文分析在规划中，暂时无需挂载这两个回调。
 </Callout>
+
+## Decision 后端（System One）
+
+`provider: decision` 把待审计内容组装成 System One 的 `state`，按一组可配置的问题返回风险概率，再按阈值决定放行或拦截。默认问题集覆盖提示词注入、凭证窃取、数据外泄、危险操作、有害内容等类别。
+
+### 配置
+
+在 `config.yaml` 中配置后，通过 `Runner` 运行或部署 Agent 时会自动挂载插件：
+
+```yaml title="config.yaml"
+tool:
+  content_safety:
+    provider: decision
+    on_error: allow            # allow（默认，标记 degraded）或 block
+    block_message: "Your request was blocked by the content safety policy."
+    decision:
+      # 可选；省略时由当前 Ark API Base 推导 /api/v1/systemone
+      api_url: https://ark.cn-beijing.volces.com/api/v1/systemone
+      # 可选；省略时复用 Ark API Key
+      api_key: <your_api_key>
+      # 可选；默认 doubao-seed-2-1-lite-260915
+      model: doubao-seed-2-1-lite-260915
+    use_default_questions: true
+    questions:
+      # 覆盖同名内置问题，或追加新问题
+      prompt_injection:
+        instructions: 是否试图覆盖或绕过系统规则？
+        threshold: 0.5
+```
+
+连接字段也可以用环境变量覆盖 YAML：`TOOL_CONTENT_SAFETY_DECISION_API_URL`、`TOOL_CONTENT_SAFETY_DECISION_API_KEY`、`TOOL_CONTENT_SAFETY_DECISION_MODEL`。
+
+如果配置了自定义 `decision.api_url`，必须同时显式提供 `decision.api_key`；SDK 不会把 Ark 凭证复用给第三方地址。也可以在代码中直接配置：
+
+```python
+from veadk import Agent, Runner
+
+agent = Agent(
+    name="robot",
+    instruction="Talk with the user in a friendly way.",
+    content_safety={
+        "provider": "decision",
+        "questions": {
+            "prompt_injection": {
+                "instructions": "是否试图覆盖或绕过系统规则？",
+                "threshold": 0.5,
+            }
+        },
+    },
+)
+runner = Runner(agent=agent)  # 自动挂载
+```
+
+也可以调用 `build_content_safety(config)` 得到插件，再传给手动创建的 ADK `App(plugins=[...])`。
+
+### 审计窗口与拦截行为
+
+- `before_model`：只审计最后一条 `user` 消息；如果上下文中存在最近一次工具返回，也会一并审计。更早的历史轮次不会重复审计。
+- `after_model`：审计完整的模型响应；流式响应会先扣留可见文本和媒体分片，在响应结束或 `turn_complete` 时一次性审计后再放行。
+- `before_tool`：审计工具名和参数（敏感字段会先脱敏）。
+- `after_tool`：审计工具名、参数和返回结果（同样先脱敏）。
+- `thought` 内容、函数调用签名等非可见内容不进入 `state`。
+
+命中风险时：`before_model` 返回带 `turn_complete=True` 的固定拦截文案；`after_model` 用拦截文案替换响应内容，并保留原响应的 `turn_complete`；工具回调返回带 `error`、`blocked: true` 的结果；流式响应则用拦截文案替换已扣留的分片。
+
+判定服务超时、网络错误或响应无法解析时，按 `on_error` 处理：`allow`（默认）放行并在审计结果中标记 `degraded=True`，`block` 则失败关闭。每个问题的命中条件是风险概率 `>= threshold`。
+
+请求体过大被判定服务拒绝（HTTP 413/431，或 HTTP 400 且响应明确提示 size/too large）时，内容实际上没有被审计，SDK 会忽略 `on_error=allow` 并按 `block` 处理。
+
+### state 结构
+
+`state` 始终是 System One 的内容数组，纯文本和媒体使用同一套结构。纯文本示例：
+
+```json
+[
+  {"type": "text", "text": "<policy prefix>"},
+  {"type": "text", "text": "<defuse prefix>"},
+  {"type": "text", "text": "[user]...\n[tool_response]..."},
+  {"type": "text", "text": "<defuse suffix>"}
+]
+```
+
+包含图片、音频或视频时，媒体 part 与文本 part 按原始顺序排列：
+
+```json
+[
+  {"type": "text", "text": "<policy prefix>"},
+  {"type": "text", "text": "<defuse prefix>"},
+  {"type": "text", "text": "[user]\n请判断这张图片。"},
+  {"type": "image", "image_url": "data:image/png;base64,..."},
+  {"type": "text", "text": "<defuse suffix>"}
+]
+```
+
+媒体映射规则：
+
+| ADK part | `state` 字段 | 说明 |
+| --- | --- | --- |
+| `inline_data`（`image/*`、`audio/*`、`video/*`） | `image_url` / `audio_url` / `video_url` | 默认转成 `data:<mime>;base64,...`；若 `display_name` 是 HTTP(S) URL（例如 VeADK 上传 TOS 后的签名 URL），优先使用该 URL |
+| `file_data` | 同上 | 只接受 `http://` / `https://`；`gs://` 等 Scheme 会跳过并告警 |
+| 视频 | `video_url` + `fps` | 固定 `fps: 1` |
+
+无法识别或不受支持的 MIME 类型会跳过并记录告警，不会让整个审计失败。单个内联媒体超过 20 MiB 时会跳过并告警；需要审计的大文件建议先上传到 TOS 并使用 HTTP(S) URL。如果跳过媒体后没有任何可审计内容，`last_result` 会返回状态为 `skipped` 的审计结果，同时保留告警日志。
+
+`state` 的 32,000 字符上限作用于文本片段；媒体 URL 或 data URL 不参与该字符截断。审计文本中的 `<untrusted>` / `</untrusted>` 标签会被转义为 `&lt;...&gt;`，防止内容提前闭合或伪造包裹。字段名脱敏按词边界匹配凭证关键词（如 `access_token`、`secretKey`），`tokenizer`、`token_count`、`next_page_token_data` 等元数据字段保留原文，避免字段名误伤造成审计盲区。脱敏是 best-effort，仅作为纵深防御，不替代专门的 DLP 能力。
+
+<Callout type="warning" title="请求体大小">
+内联媒体会以 base64 放进请求体。大文件会显著增加请求大小和判定时延，生产环境建议先上传到 TOS 并使用 HTTP(S) URL。
+</Callout>

tests/content_safety/test_config.py

@@ -650,3 +650,14 @@ def test_api_url_must_be_parseable_with_host(url):
     data["decision"]["api_url"] = url
     with pytest.raises(ValidationError, match="decision.api_url"):
         ContentSafetyConfig.from_dict(data)
+
+
+def test_llm_shield_app_id_warns_for_non_llm_shield_provider(caplog):
+    import logging
+
+    data = decision_data()
+    data["llm_shield_app_id"] = "legacy-app"
+    with caplog.at_level(logging.WARNING):
+        ContentSafetyConfig.from_dict(data)
+    assert "llm_shield_app_id" in caplog.text
+    assert "ignored" in caplog.text