Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@ All notable changes to this project are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.21.0]

### Added
- **Cross-function taint (phase 2: parameter propagation).** AG040 now also follows taint in
the caller→callee direction: a sink on a bare parameter (`def run_it(code): exec(code)`) is
flagged when a same-file caller passes model output for it
(`run_it(llm.invoke(p))`) — including keyword and non-first-position args. Combined with
phase 1 (return propagation), AG040 now tracks model output across the function boundary in
both directions, bounded depth, still single-file.

## [0.20.0]

### Added
Expand Down
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,11 @@ indirection, and whether a URL comes from a hardcoded constant / trusted config
`os.environ`) versus a tool parameter. SSRF classification uses real `ipaddress` range checks,
not string matching.

It also follows **cross-function taint within a file**: a value returned by a local helper is
tracked into its caller — e.g. model output laundered through a helper into `eval`/`exec` is
caught (AG040). It does **not** yet do whole-program / cross-file call-graph reachability, so a
value laundered across modules can still be missed. This is deliberately conservative and
It also follows **cross-function taint within a file**, in both directions: a value returned by
a local helper is tracked into its caller, and model output passed *into* a helper's parameter
is tracked to a sink inside it — so model output laundered through a helper into `eval`/`exec`
is caught either way (AG040). It does **not** yet do whole-program / cross-file call-graph
reachability, so a value laundered across modules can still be missed. This is deliberately conservative and
improving; treat findings as "this authority is reachable in the code," not a proof of
exploitability.

Expand Down Expand Up @@ -145,7 +146,7 @@ re-run `autonomyproof baseline .` and commit the updated file in the same PR.
Use the action directly:

```yaml
- uses: autonomyproof/autonomyproof-cli@v0.20.0
- uses: autonomyproof/autonomyproof-cli@v0.21.0
with:
target: .
fail-on: high
Expand All @@ -172,7 +173,7 @@ Gate locally before a commit ever leaves your machine:
# .pre-commit-config.yaml
repos:
- repo: https://github.com/autonomyproof/autonomyproof-cli
rev: v0.20.0
rev: v0.21.0
hooks:
- id: autonomyproof
```
Expand Down
6 changes: 3 additions & 3 deletions benchmark/CORPUS_RESULTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Labeled-corpus results (ground-truth precision & recall)

**Cases:** 156 · **Rules covered:** 34 · **Overall precision:** 1.000 · **Overall recall:** 1.000
**Cases:** 160 · **Rules covered:** 34 · **Overall precision:** 1.000 · **Overall recall:** 1.000

| Rule | pos | neg | TP | FP | FN | Precision | Recall | F1 |
|---|--:|--:|--:|--:|--:|--:|--:|--:|
Expand Down Expand Up @@ -37,9 +37,9 @@
| AG037 | 1 | 2 | 1 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG038 | 2 | 2 | 2 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG039 | 1 | 2 | 1 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG040 | 3 | 3 | 3 | 0 | 0 | 1.00 | 1.00 | 1.00 |
| AG040 | 5 | 5 | 5 | 0 | 0 | 1.00 | 1.00 | 1.00 |

**Totals:** TP 81 · FP 0 · FN 0 · TN 75
**Totals:** TP 83 · FP 0 · FN 0 · TN 77

Precision = of the cases where a rule fired, how many were true positives. Recall = of the cases where a rule should fire, how many did. Reproduce with `python benchmark/corpus_eval.py`.

6 changes: 6 additions & 0 deletions benchmark/corpus.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,12 @@ cases:
- {id: ag040-neg-nonmodel, rule: AG040, label: negative, code: "def run(u):\n data = requests.get(u).text\n exec(data)\n"}
- {id: ag040-neg-constant, rule: AG040, label: negative, code: "exec('print(1)')\n"}

# --- AG040 cross-function taint (return + parameter propagation) ---
- {id: ag040-pos-xfn-return, rule: AG040, label: positive, code: "def get_response(p):\n return llm.invoke(p)\ndef run(p):\n exec(get_response(p))\n"}
- {id: ag040-pos-xfn-param, rule: AG040, label: positive, code: "def dangerous(code):\n exec(code)\ndef tool(p):\n dangerous(llm.invoke(p))\n"}
- {id: ag040-neg-xfn-nonmodel, rule: AG040, label: negative, code: "def get_response(p):\n return requests.get(p).text\ndef run(p):\n exec(get_response(p))\n"}
- {id: ag040-neg-xfn-nocaller, rule: AG040, label: negative, code: "def dangerous(code):\n exec(code)\n"}

# --- AG021 broadened deserialization sinks ---
- {id: ag021-pos-joblib, rule: AG021, label: positive, code: "import joblib\njoblib.load(f)\n"}
- {id: ag021-pos-pandas, rule: AG021, label: positive, code: "import pandas\npandas.read_pickle(f)\n"}
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "autonomyproof"
version = "0.20.0"
version = "0.21.0"
description = "Open-source local scanner that finds unsafe capabilities and missing guardrails in Python AI-agent code."
readme = "README.md"
requires-python = ">=3.11"
Expand Down
2 changes: 1 addition & 1 deletion src/autonomyproof/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@

__all__ = ["__version__"]

__version__ = "0.20.0"
__version__ = "0.21.0"
35 changes: 35 additions & 0 deletions src/autonomyproof/rules/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,15 @@ def _is_model_call(node: ast.expr) -> bool:
return False


def _param_index(func: ast.FunctionDef | ast.AsyncFunctionDef, name: str) -> int | None:
"""Positional index of parameter ``name`` in ``func``, or None (keyword-only excluded)."""
positional = func.args.posonlyargs + func.args.args
for index, arg in enumerate(positional):
if arg.arg == name:
return index
return None


class InsecureModelOutputRule(Rule):
"""AG040 — Model output executed as code or a shell command."""

Expand Down Expand Up @@ -233,4 +242,30 @@ def _from_model(
assigned = ctx.analysis.resolve_local_value(base.id, origin)
if assigned is not None:
return self._from_model(ctx, assigned, origin, fmap, depth + 1)
# Phase 2: parameter propagation — an unassigned name that is a parameter of the
# enclosing function is tainted if any caller passes model output for it.
if self._param_fed_by_model(ctx, base.id, origin, fmap, depth):
return True
return False

def _param_fed_by_model(
self,
ctx: RuleContext,
name: str,
origin: ast.AST,
fmap: dict[str, ast.FunctionDef | ast.AsyncFunctionDef],
depth: int,
) -> bool:
scope = ctx.analysis.enclosing_scope(origin)
if not isinstance(scope, ast.FunctionDef | ast.AsyncFunctionDef):
return False
index = _param_index(scope, name)
if index is None:
return False
for call in ctx.analysis.calls:
if not (isinstance(call.func, ast.Name) and call.func.id == scope.name):
continue
arg = call.args[index] if index < len(call.args) else keyword(call, name)
if arg is not None and self._from_model(ctx, arg, call, fmap, depth + 1):
return True
return False
2 changes: 1 addition & 1 deletion tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def _home(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
def test_version(runner: CliRunner) -> None:
result = runner.invoke(cli.main, ["--version"])
assert result.exit_code == 0
assert "0.20.0" in result.output
assert "0.21.0" in result.output


def test_init_creates_and_is_idempotent(runner: CliRunner) -> None:
Expand Down
44 changes: 44 additions & 0 deletions tests/test_rules_execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -212,3 +212,47 @@ def test_ag040_xfn_deep_chain_not_tracked_clean() -> None:
"def run(p):\n exec(h1(p))\n"
)
assert run_rule(InsecureModelOutputRule(), code) == []


# --- AG040 cross-function phase 2 (parameter propagation) --------------------
def test_ag040_param_fed_model_positional() -> None:
code = "def dangerous(code):\n exec(code)\ndef tool(p):\n dangerous(llm.invoke(p))\n"
assert run_rule(InsecureModelOutputRule(), code)


def test_ag040_param_fed_model_keyword() -> None:
code = (
"def dangerous(code):\n exec(code)\ndef tool(p):\n dangerous(code=llm.predict(p))\n"
)
assert run_rule(InsecureModelOutputRule(), code)


def test_ag040_param_fed_model_second_position() -> None:
code = (
"def dangerous(a, code):\n exec(code)\ndef tool(p):\n dangerous(1, llm.invoke(p))\n"
)
assert run_rule(InsecureModelOutputRule(), code)


def test_ag040_param_caller_constant_clean() -> None:
code = "def dangerous(code):\n exec(code)\ndef tool():\n dangerous('print(1)')\n"
assert run_rule(InsecureModelOutputRule(), code) == []


def test_ag040_param_no_callers_clean() -> None:
assert run_rule(InsecureModelOutputRule(), "def dangerous(code):\n exec(code)\n") == []


def test_ag040_param_caller_passes_param_clean() -> None:
# tool forwards its own param; tool itself has no callers feeding model output.
code = "def dangerous(code):\n exec(code)\ndef tool(p):\n dangerous(p)\n"
assert run_rule(InsecureModelOutputRule(), code) == []


def test_ag040_module_level_exec_of_free_name_clean() -> None:
assert run_rule(InsecureModelOutputRule(), "exec(undefined_global)\n") == []


def test_ag040_exec_of_non_param_free_name_clean() -> None:
code = "def f():\n exec(g)\n"
assert run_rule(InsecureModelOutputRule(), code) == []
Loading