diff --git a/CHANGELOG.md b/CHANGELOG.md index 47240a8..55a3639 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,24 @@ # Changelog +## [Unreleased] + +### Changed +- **The reverse lot index carries the same gate as trace-forward**: `GET /list-artifact/reverse/{geoid}` + answered "which lots is this field in" for any hub login, with no grant and no audit entry; + measured live, that was five of the six lots the gated `/traceforward` returns (review + ledger AG-015). It now requires a grant for the seed or an authority credential with a + scope, exactly as `/traceforward` does, and an authority answer is written to the MEAL + ledger before it is returned. Callers holding only a login receive 403. Pancake's audit + route, which calls this index, already forwards the caller's authority token. +- `docs/MCP_SURFACE.md` gating note for `get_list_artifact_reverse`; the MCP tool takes + `grant_token`, `authority_token` and `scope`. README endpoint table corrected: it had + described a gate the code did not have. + +### Added +- `app/tests/testkit/grants.py`: one shared minter for field-access grants, replacing the + copy in `test_traceforward.py`. Six direct tests of the reverse index, including the + control that a bare login is refused and the proof that the same check can fire. + ## [v0.9-review] ### Added diff --git a/README.md b/README.md index 7af74d5..8c8a3bc 100644 --- a/README.md +++ b/README.md @@ -65,8 +65,10 @@ Follow these steps to run the Node locally alongside the `ar2-hub` and `Pancake` | `GET` | `/geoid/{geoid}/eudr-export` | Exports an EUDR-compliant GeoJSON artifact. | Yes (Grant) | | `POST` | `/list-artifact` | Registers a content-derived list (a lot/case/pallet/retail item) over GeoIDs, nested lists (`L:`), or regions (`R:`). Append-only. | Yes (Hub JWT) | | `POST` | `/region-artifact` | Registers a content-derived region from a WKT boundary (no acreage cap) or from member GeoIDs/lists. | Yes (Hub JWT) | -| `GET` | `/list-artifact/{list_id}` | Returns the members of a list artifact. | Yes (Hub JWT) | -| `GET` | `/list-artifact/reverse/{geoid}` | Trace-forward: every list/region containing a GeoID, climbed recursively to terminal products. | Yes (Hub JWT + `trace-forward` capability + seed authorization; accredited authority credential for the identity tier) | +| `GET` | `/list-artifact/{list_id}` | Returns the members of a list artifact. | Yes (Hub JWT, then a grant for the list or an authority credential) | +| `GET` | `/list-artifact/reverse/{geoid}` | The first hop of trace-forward: the lists that name a GeoID directly. | Yes (Hub JWT, then a grant for the seed or an authority credential with `?scope=`; the same gate as `/traceforward`) | +| `POST` | `/traceforward` | Trace-forward: every list/region containing a GeoID, climbed recursively to terminal products. | Yes (Hub JWT with the `trace-forward` capability or a grant, then a grant for the seed or an authority credential; identities only on the authority path, audited) | +| `GET` | `/list-artifact/{list_id}/traceback` | Trace-back: a lot walked down to its input lots and fields, one hop per lot. | Yes (Hub JWT, then a grant for the list or an authority credential) | See [TRACEABILITY.md](./TRACEABILITY.md) for how these endpoints compose into trace-back and trace-forward. diff --git a/app/mcp_server.py b/app/mcp_server.py index 66dc3c3..0d493ec 100644 --- a/app/mcp_server.py +++ b/app/mcp_server.py @@ -613,16 +613,21 @@ async def get_list_artifact( "GET /list-artifact/reverse/{geoid}. Which lots a given field " "appears in. The reverse index behind a recall: given a " "contaminated field, the lots that carried its produce. Requires a " - "hub token. The seed is expanded through its equivalence set " - "first, so a v1 Geo Id, a superseded alias and the canonical Geo Id " - "all return the same lots." + "hub token, and gated beyond that exactly like run_traceforward: " + "grant_token for this field, or authority_token with a scope. The " + "seed is expanded through its equivalence set first, so a v1 Geo " + "Id, a superseded alias and the canonical Geo Id all return the " + "same lots." ) ) async def get_list_artifact_reverse( - geoid: str, token: str | None = None, + geoid: str, grant_token: str | None = None, + authority_token: str | None = None, scope: str | None = None, + token: str | None = None, ) -> dict[str, Any]: return await _handle( - trace.get_list_artifact_reverse, geoid=geoid, + trace.get_list_artifact_reverse, geoid=geoid, scope=scope, + x_grant_token=grant_token, x_authority_token=authority_token, user=_required(token, "the reverse lot index")) # -- GET /list-artifact/{list_id}/traceback --------------------------- diff --git a/app/routers/traceforward.py b/app/routers/traceforward.py index 34eaa29..34e8d68 100644 --- a/app/routers/traceforward.py +++ b/app/routers/traceforward.py @@ -230,22 +230,49 @@ class ListReverseResponse(BaseModel): @router.get("/list-artifact/reverse/{geoid}", response_model=ListReverseResponse) def get_list_artifact_reverse( geoid: str, + scope: str | None = None, + x_grant_token: str | None = Header(None), + x_authority_token: str | None = Header(None), user: dict = Depends(require_hub_user), db: Session = Depends(get_db) ): """ Retrieves all list IDs that contain the given geoid. + Gated exactly as POST /traceforward is: a verified grant for this seed, or + an authority credential. This route answers the first hop of the same + question -- which lots is this field in -- and until 2026-10 it answered it + for any hub login, with no grant and no audit entry. Measured live, that + was five of the six lots the gated route returns (review ledger AG-015). + A second door to a gated answer has to carry the same gate. + Expands the seed through its equivalence set first, so a v1 GeoID, a superseded content-hash alias and the canonical GeoID all return the same lists. Without this the reverse lookup silently disagrees with /traceforward about what counts as the same field. """ + from app.auth import authorize_artifact + from app.meal_logger import log_traceforward + if x_authority_token and not scope: + raise HTTPException(status_code=400, detail="scope is required with an authority credential") + auth_result = authorize_artifact( + grant_token=x_grant_token, + authority_token=x_authority_token, + geoid=geoid, + scope=scope, + raise_404_on_fail=False, + ) + seeds = _equivalence_set(db, geoid) - list_ids = db.execute( + list_ids = sorted(set(db.execute( select(ListMemberEdge.list_id).where(ListMemberEdge.geoid.in_(seeds)) - ).scalars().all() - return ListReverseResponse(list_ids=sorted(set(list_ids))) + ).scalars().all())) + if auth_result.get("used_authority"): + # Same ledger event as the gated route: an authority asked which lots + # a field is in, and got an answer. Audit before disclosure. + log_traceforward(user.get("sub"), auth_result.get("authority_jti"), + geoid, scope, len(list_ids), list_ids) + return ListReverseResponse(list_ids=list_ids) # -------------------------------------------------------------------------- diff --git a/app/tests/test_geoid_v2_live.py b/app/tests/test_geoid_v2_live.py index 5553d03..e46d45c 100644 --- a/app/tests/test_geoid_v2_live.py +++ b/app/tests/test_geoid_v2_live.py @@ -46,6 +46,7 @@ ListMemberEdge, ) from app.s2_services import S2Service +from app.tests.testkit.grants import grant_header_for # noqa: E402 from app.utils import Utils app.dependency_overrides[require_hub_user] = lambda: { @@ -457,7 +458,7 @@ def test_reverse_lookup_finds_a_list_through_a_v1_identifier(): v1_id = "legacy-" + uuid.uuid4().hex[:12] _alias_v1(v1_id, v2_id) - r = client.get(f"/list-artifact/reverse/{v1_id}") + r = client.get(f"/list-artifact/reverse/{v1_id}", headers=grant_header_for(v1_id)) assert r.status_code == 200, r.text assert list_id in r.json()["list_ids"], ( "the v1 identifier must reach lists registered against its v2 identity" @@ -501,6 +502,6 @@ def test_a_list_registered_against_a_v1_geoid_is_still_traceable(): finally: db.close() - r = client.get(f"/list-artifact/reverse/{v2_id}") + r = client.get(f"/list-artifact/reverse/{v2_id}", headers=grant_header_for(v2_id)) assert r.status_code == 200, r.text assert list_id in r.json()["list_ids"] diff --git a/app/tests/test_point_polygon_peers.py b/app/tests/test_point_polygon_peers.py index 505c625..48ac22d 100644 --- a/app/tests/test_point_polygon_peers.py +++ b/app/tests/test_point_polygon_peers.py @@ -35,10 +35,18 @@ from app import geoid_v2 # noqa: E402 from app.auth import require_hub_user # noqa: E402 from app.main import app # noqa: E402 +from app.tests.testkit.grants import grant_header_for # noqa: E402 app.dependency_overrides[require_hub_user] = lambda: {"sub": "peers@demo.com"} client = TestClient(app) + +@pytest.fixture(autouse=True) +def set_test_status_list_dir(monkeypatch): + # The reverse lot index takes a grant, and a grant's status list is + # resolved locally under test rather than fetched from Pancake. + monkeypatch.setenv("TEST_STATUS_LIST_DIR", TESTKIT_DIR) + # Two plots in the Honduran coffee belt, far enough apart that neither resolves # onto the other and neither collides with anything registered by another test. POINT_LAT, POINT_LNG = 14.812345, -88.312345 @@ -245,7 +253,7 @@ def test_both_kinds_join_one_list_and_prove_membership(plots): assert sorted(fetched.json()["members"]) == members for kind, geo_id in plots.items(): - reverse = client.get(f"/list-artifact/reverse/{geo_id}") + reverse = client.get(f"/list-artifact/reverse/{geo_id}", headers=grant_header_for(geo_id)) assert reverse.status_code == 200, kind assert list_id in reverse.json()["list_ids"], f"{kind} is not findable in its own list" diff --git a/app/tests/test_traceforward.py b/app/tests/test_traceforward.py index 190a6be..fba3d79 100644 --- a/app/tests/test_traceforward.py +++ b/app/tests/test_traceforward.py @@ -1,4 +1,3 @@ -import json import os import uuid @@ -221,35 +220,7 @@ def _hub(capabilities): app.dependency_overrides[require_hub_user] = lambda: { "sub": "authority@demo.agstack.org", "capabilities": capabilities} -def valid_grant_for(geoid: str) -> str: - import base64 - import hashlib - import secrets - import time - - import jwt - - def _b64url(data: bytes) -> str: - return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii") - - private_pem = (Path(__file__).parent / "testkit" / "dev_keys" / "dev_issuer_private.pem").read_bytes() - - salt = _b64url(secrets.token_bytes(16)) - encoded_disclosure = _b64url(json.dumps([salt, "fields.0", geoid]).encode("utf-8")) - digest = _b64url(hashlib.sha256(encoded_disclosure.encode("ascii")).digest()) - - claims = { - "iss": "did:web:pancake.test", - "sub": "owner@demo.com", - "iat": int(time.time()), - "exp": int(time.time()) + 3600, - "vct": "agstack.org/credentials/field-access-grant/v1", - "status": {"status_list": {"uri": "http://localhost:8100/grants/status-list", "idx": 1}}, - "_sd": [digest], - "_sd_alg": "sha-256" - } - token = jwt.encode(claims, private_pem, algorithm="EdDSA", headers={"typ": "vc+sd-jwt", "kid": "pancake-test-1"}) - return f"{token}~{encoded_disclosure}~" +from app.tests.testkit.grants import valid_grant_for # noqa: E402 # 1. Gate A: no capability -> 403 def test_gate_a_missing_capability_403(): @@ -373,3 +344,92 @@ def test_authority_round_trip(): fwd = client.post("/traceforward", json={"seed_geoid": geoids[0], "scope": "demo-recall"}, headers=auth) assert fwd.status_code == 200 assert retail_list_id in [m["list_id"] for m in fwd.json()["matches"]] + + +# --------------------------------------------------------------------------- +# The second door. GET /list-artifact/reverse/{geoid} answers the first hop of +# the same question /traceforward answers, and until 2026-10 it answered it +# for any hub login with no grant and no audit entry (review ledger AG-015: +# five of the six lots the gated route returned, live). These tests hold the +# gate on the second door to the same shape as the first. +# --------------------------------------------------------------------------- + +def _lot_of(geoid: str) -> str: + r = client.post("/list-artifact", json={"members": [geoid]}) + assert r.status_code == 200, r.text + return r.json()["list_id"] + + +def test_reverse_refuses_a_bare_login(): + """The control: a login with every capability and no credential gets nothing.""" + _hub(["trace-forward"]) + seed = create_test_geoid(["111"]) + lot = _lot_of(seed) + r = client.get(f"/list-artifact/reverse/{seed}") + assert r.status_code == 403, r.text + assert lot not in r.text + + +def test_reverse_refuses_a_grant_for_a_different_field(): + _hub(["trace-forward"]) + seed = create_test_geoid(["111"]) + other = create_test_geoid(["222"]) + _lot_of(seed) + r = client.get(f"/list-artifact/reverse/{seed}", + headers={"X-Grant-Token": valid_grant_for(other)}) + assert r.status_code == 403 + + +def test_reverse_answers_a_grant_for_the_seed_and_the_check_can_fire(): + """The same lot the control must not reveal is revealed here, so the + control is shown to be testing a real answer rather than an empty index.""" + _hub([]) # no capability at all: the grant is sufficient, as on /traceforward + seed = create_test_geoid(["111"]) + lot = _lot_of(seed) + r = client.get(f"/list-artifact/reverse/{seed}", + headers={"X-Grant-Token": valid_grant_for(seed)}) + assert r.status_code == 200, r.text + assert r.json()["list_ids"] == [lot] + + +def test_reverse_with_an_authority_credential_needs_a_scope_and_is_audited(): + _hub(["trace-forward"]) + seed = create_test_geoid(["111"]) + lot = _lot_of(seed) + auth = {"X-Authority-Token": _read("valid_authority")} + + no_scope = client.get(f"/list-artifact/reverse/{seed}", headers=auth) + assert no_scope.status_code == 400 + + packets = [] + with patch("app.meal_logger._append_to_meal_chain", side_effect=packets.append): + r = client.get(f"/list-artifact/reverse/{seed}", params={"scope": "demo-recall"}, headers=auth) + assert r.status_code == 200, r.text + assert r.json()["list_ids"] == [lot] + assert packets and packets[-1]["seed_geoid"] == seed, "an authority answer must leave a MEAL entry" + + +def test_reverse_with_a_grant_is_not_audited(): + """Tier 1 on /traceforward is not logged; the second door matches it.""" + _hub([]) + seed = create_test_geoid(["111"]) + _lot_of(seed) + packets = [] + with patch("app.meal_logger._append_to_meal_chain", side_effect=packets.append): + r = client.get(f"/list-artifact/reverse/{seed}", + headers={"X-Grant-Token": valid_grant_for(seed)}) + assert r.status_code == 200 + assert packets == [] + + +def test_reverse_and_traceforward_agree_on_the_first_hop(): + """Same seed, same grant: the second door's answer is contained in the first's.""" + _hub([]) + seed = create_test_geoid(["111"]) + lot = _lot_of(seed) + parent = client.post("/list-artifact", json={"members": [f"L:{lot}"]}).json()["list_id"] + g = {"X-Grant-Token": valid_grant_for(seed)} + rev = set(client.get(f"/list-artifact/reverse/{seed}", headers=g).json()["list_ids"]) + fwd = set(client.post("/traceforward", json={"seed_geoid": seed}, headers=g).json()["list_ids"]) + assert rev == {lot} + assert rev < fwd and parent in fwd diff --git a/app/tests/testkit/grants.py b/app/tests/testkit/grants.py new file mode 100644 index 0000000..8aac22a --- /dev/null +++ b/app/tests/testkit/grants.py @@ -0,0 +1,52 @@ +"""Mint a field-access grant for one GeoID with the dev issuer key. + +Shared by every suite that has to pass the grant gate on a trace route. Lives +here rather than in one test module so that a second copy cannot drift from +the first; the credential shape is the one Pancake's testkit mints +(vct agstack.org/credentials/field-access-grant/v1, one selective disclosure +naming the field, a status-list reference the tests resolve locally). +""" +from __future__ import annotations + +import base64 +import hashlib +import json +import secrets +import time +from pathlib import Path + +import jwt + +DEV_KEYS = Path(__file__).parent / "dev_keys" +GRANT_VCT = "agstack.org/credentials/field-access-grant/v1" +STATUS_URI = "http://localhost:8100/grants/status-list" + + +def _b64url(data: bytes) -> str: + return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii") + + +def valid_grant_for(geoid: str, *, ttl_seconds: int = 3600) -> str: + """An SD-JWT grant whose single disclosure names ``geoid``.""" + private_pem = (DEV_KEYS / "dev_issuer_private.pem").read_bytes() + salt = _b64url(secrets.token_bytes(16)) + disclosure = _b64url(json.dumps([salt, "fields.0", geoid]).encode("utf-8")) + digest = _b64url(hashlib.sha256(disclosure.encode("ascii")).digest()) + now = int(time.time()) + claims = { + "iss": "did:web:pancake.test", + "sub": "owner@demo.com", + "iat": now, + "exp": now + ttl_seconds, + "vct": GRANT_VCT, + "status": {"status_list": {"uri": STATUS_URI, "idx": 1}}, + "_sd": [digest], + "_sd_alg": "sha-256", + } + token = jwt.encode(claims, private_pem, algorithm="EdDSA", + headers={"typ": "vc+sd-jwt", "kid": "pancake-test-1"}) + return f"{token}~{disclosure}~" + + +def grant_header_for(geoid: str) -> dict[str, str]: + return {"X-Grant-Token": valid_grant_for(geoid)} diff --git a/docs/MCP_SURFACE.md b/docs/MCP_SURFACE.md index a271cf9..df40fb7 100644 --- a/docs/MCP_SURFACE.md +++ b/docs/MCP_SURFACE.md @@ -25,7 +25,7 @@ Endpoint: `POST /mcp`, streamable HTTP, bearer token from the hub. Discovery at | `GET /geoid/{geo_id}/eudr-export` | `eudr_export` | optional | refuses without | The only read route that refuses without a grant rather than answering coarsely, because a masked cell is not a filing. | | `POST /get-percentage-overlap-two-fields` | `get_percentage_overlap_two_fields` | none | -- | Answers whether two Geo Ids name the same ground while disclosing only the one number. | | `POST /list-artifact` | `register_list_artifact` | hub token | -- | FSMA 204 lot. `location_geo_id` is the traceability lot code source (21 CFR 1.1330(a)(14)); omitting it records a gap rather than failing. | -| `GET /list-artifact/reverse/{geoid}` | `get_list_artifact_reverse` | hub token | -- | | +| `GET /list-artifact/reverse/{geoid}` | `get_list_artifact_reverse` | hub token | -- | Gated twice, like run_traceforward: a hub token to call at all, then a grant for this field or an authority credential with a scope. The first hop of trace-forward, carrying the same gate. | | `GET /list-artifact/{list_id}` | `get_list_artifact` | hub token | -- | Gated twice: a hub token to call at all, then a grant or authority credential to see the members. | | `GET /list-artifact/{list_id}/traceback` | `trace_back` | hub token | -- | One hop is one lot, which is the unit 21 CFR 1.1320(a) records. | | `POST /populate-country-in-geo-ids` | `populate_country_in_geo_ids` | none | -- | Maintenance. Walks the whole table; not for answering a question. | diff --git a/scripts/mcp_surface.py b/scripts/mcp_surface.py index a9cef2a..1f67982 100644 --- a/scripts/mcp_surface.py +++ b/scripts/mcp_surface.py @@ -84,6 +84,10 @@ "get_list_artifact": "Gated twice: a hub token to call at all, then a grant or authority " "credential to see the members.", + "get_list_artifact_reverse": + "Gated twice, like run_traceforward: a hub token to call at all, then a " + "grant for this field or an authority credential with a scope. The first " + "hop of trace-forward, carrying the same gate.", "trace_back": "One hop is one lot, which is the unit 21 CFR 1.1320(a) records.", "run_traceforward":