Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@ All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.16.0]

### Added
- **AG034 — Cloud/infrastructure destruction exposed to the agent.** Flags an agent tool that
can tear down cloud infrastructure with no approval: AWS `terminate_instances`,
`delete_bucket`, `delete_db_instance`, `delete_cluster`, `delete_stack`, `delete_volume`,
and Kubernetes `delete_namespaced_*` / `delete_collection_*` / `delete_namespace`. Mapped to
MITRE ATT&CK T1485 + T1531.
- **AG035 — Money movement exposed to the agent without approval.** Flags `Refund.create`,
`Payout.create`, and `Transfer.create` (Stripe-style) inside an agent tool with no approval
gate — the classic prompt-injection payout attack.

Both reuse AG033's call-based, tool-scoped, approval-suppressed model (the shared
`_iter_tool_sinks` spine), so they stay zero-false-positive. Verified: corpus precision/recall
1.000, and 0 findings across the 41-repo real benchmark.

## [0.15.0]

### Added
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,9 @@ passthrough, guardrail self-modification, secrets in model context, insecure des
framework flags (`allow_dangerous_*`, `trust_remote_code`), code/shell interpreter tools
(`PythonREPLTool`, `ShellTool`), a disabled code-execution sandbox (`use_docker=False`),
irreversible datastore/filesystem wipes exposed to an agent tool with no approval
(`drop_all`, `flushall`, `shutil.rmtree`, `DROP DATABASE`),
(`drop_all`, `flushall`, `shutil.rmtree`, `DROP DATABASE`), cloud/infrastructure
destruction (`terminate_instances`, `delete_bucket`, `delete_cluster`, k8s teardown),
money movement without approval (`Refund`/`Payout`/`Transfer.create`),
known-vulnerable framework dependencies (version-validated CVEs), and more. Run
`autonomyproof rules list` for the full catalogue and `autonomyproof rules explain AG001`
for details. Every finding carries **OWASP Agentic, NIST AI RMF, ISO 42001, MITRE
Expand Down Expand Up @@ -138,7 +140,7 @@ re-run `autonomyproof baseline .` and commit the updated file in the same PR.
Use the action directly:

```yaml
- uses: autonomyproof/autonomyproof-cli@v0.15.0
- uses: autonomyproof/autonomyproof-cli@v0.16.0
with:
target: .
fail-on: high
Expand All @@ -165,7 +167,7 @@ Gate locally before a commit ever leaves your machine:
# .pre-commit-config.yaml
repos:
- repo: https://github.com/autonomyproof/autonomyproof-cli
rev: v0.15.0
rev: v0.16.0
hooks:
- id: autonomyproof
```
Expand Down
6 changes: 4 additions & 2 deletions benchmark/CORPUS_RESULTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Labeled-corpus results (ground-truth precision & recall)

**Cases:** 127 · **Rules covered:** 27 · **Overall precision:** 1.000 · **Overall recall:** 1.000
**Cases:** 136 · **Rules covered:** 29 · **Overall precision:** 1.000 · **Overall recall:** 1.000

| Rule | pos | neg | TP | FP | FN | Precision | Recall | F1 |
|---|--:|--:|--:|--:|--:|--:|--:|--:|
Expand Down Expand Up @@ -31,8 +31,10 @@
| AG031 | 2 | 2 | 2 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG032 | 1 | 1 | 1 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG033 | 4 | 3 | 4 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG034 | 3 | 2 | 3 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG035 | 2 | 2 | 2 | 0 | 0 | 1.00 | 1.00 | 1.00 |

**Totals:** TP 67 · FP 0 · FN 0 · TN 60
**Totals:** TP 72 · FP 0 · FN 0 · TN 64

Precision = of the cases where a rule fired, how many were true positives. Recall = of the cases where a rule should fire, how many did. Reproduce with `python benchmark/corpus_eval.py`.

13 changes: 13 additions & 0 deletions benchmark/corpus.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,19 @@ cases:
- {id: ag033-neg-approval, rule: AG033, label: negative, code: "@tool\ndef reset_db():\n if not confirm:\n return\n Base.metadata.drop_all(engine)\n"}
- {id: ag033-neg-pandas, rule: AG033, label: negative, code: "@tool\ndef trim(df):\n return df.drop(columns=['x'])\n"}

# --- AG034 cloud/infra destruction via agent tool ---
- {id: ag034-pos-terminate, rule: AG034, label: positive, code: "@tool\ndef scale_down():\n ec2.terminate_instances(InstanceIds=ids)\n"}
- {id: ag034-pos-bucket, rule: AG034, label: positive, code: "@tool\ndef purge():\n s3.delete_bucket(Bucket=b)\n"}
- {id: ag034-pos-k8s, rule: AG034, label: positive, code: "@tool\ndef teardown():\n api.delete_namespaced_deployment(n, ns)\n"}
- {id: ag034-neg-nontool, rule: AG034, label: negative, code: "def scale_down():\n ec2.terminate_instances(InstanceIds=ids)\n"}
- {id: ag034-neg-benign, rule: AG034, label: negative, code: "@tool\ndef cleanup():\n queue.delete_message(handle)\n"}

# --- AG035 money movement via agent tool ---
- {id: ag035-pos-refund, rule: AG035, label: positive, code: "@tool\ndef handle(o):\n stripe.Refund.create(charge=o.charge)\n"}
- {id: ag035-pos-payout, rule: AG035, label: positive, code: "from stripe import Payout\n@tool\ndef pay(v):\n Payout.create(amount=v.owed)\n"}
- {id: ag035-neg-nontool, rule: AG035, label: negative, code: "def handle(o):\n stripe.Refund.create(charge=o.charge)\n"}
- {id: ag035-neg-customer, rule: AG035, label: negative, code: "@tool\ndef signup(e):\n stripe.Customer.create(email=e)\n"}

# --- AG021 broadened deserialization sinks ---
- {id: ag021-pos-joblib, rule: AG021, label: positive, code: "import joblib\njoblib.load(f)\n"}
- {id: ag021-pos-pandas, rule: AG021, label: positive, code: "import pandas\npandas.read_pickle(f)\n"}
Expand Down
Loading
Loading