fix(sandbox): acknowledge unchanged policy revisions

[NaveCohenMonday]

Summary

Decouple sandbox policy-revision acknowledgement from OPA policy loading. A
newer sandbox revision whose effective policy hash is already loaded can now be
acknowledged without reloading identical policy content, allowing desired and
current revisions to converge.

Related Issue

Fixes #2518

Changes

• Track the acknowledged sandbox policy version separately from the loaded
  policy hash.
• Acknowledge only newer, sandbox-scoped revisions with the same non-empty
  effective policy hash.
• Wait for required policy-runtime or middleware reconciliation to succeed
  before acknowledging the revision.
• Exclude global policy, local overrides, equal or older versions, empty
  hashes, and different policy content from the shortcut.
• Emit acknowledgement success telemetry only after the gateway accepts the
  loaded-status report.
• Document the same-hash revision acknowledgement invariant.

Testing

• CARGO_INCREMENTAL=0 mise run pre-commit passes
• Unit tests added/updated
• E2E tests added/updated (maintainer requested test:e2e after review)

Validation:

CARGO_INCREMENTAL=0 mise exec -- cargo test -p openshell-sandbox --lib

test result: ok. 99 passed; 0 failed

CARGO_INCREMENTAL=0 mise exec -- cargo test -p openshell-sandbox \
  same_hash_ --lib

test result: ok. 3 passed; 0 failed

The scripted poll-loop tests cover v1/H → v2/H exactly-once acknowledgement
without an OPA reload, failed middleware reconciliation with no premature
status followed by retry and one acknowledgement, and the local/global,
changed/empty-hash, equal, and older-version scopes.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

All contributors have signed the DCO ✍️ ✅
_{Posted by the DCO Assistant Lite bot.}

[NaveCohenMonday]

I have read the DCO document and I hereby sign the DCO.

[drew]

gator-agent

PR Review Status

Validation: This concentrated sandbox-policy fix directly addresses confirmed issue #2518 and has no duplicate candidate.
Head SHA: 182ef51768dfa66becb6847e0349170621cbddab

Review findings:

• Warning — missing stateful regression coverage: the added predicate tests do not exercise the poll loop, so they cannot catch duplicate or premature acknowledgements. Please add a scripted-poll test proving v1/H → v2/H emits exactly one LOADED(2) without reloading OPA, a failed required middleware reconciliation emits nothing and retries, and a later success emits exactly one acknowledgement. Include the relevant negative scopes (local override, global source, changed/empty hash, and equal/older version).
• See the inline warning about emitting success telemetry before gateway acknowledgement succeeds.

Docs: The architecture invariant is documented. No Fern docs/navigation update is required because this fixes existing --wait semantics without changing a command or user workflow.

Tests: No local tests were run. The author reports pre-commit and two focused predicate tests passing; required CI is still waiting for the copy-PR mirror.

Next state: gator:in-review pending an author update. A test:e2e pipeline run will be required after review feedback is resolved because this changes sandbox policy-enforcement reconciliation.

[drew]

gator-agent

Warning: This success OCSF event fires immediately after queueing the status update, before report_policy_status succeeds. A backlog or terminal RPC failure would therefore produce a false “Acknowledged” operational event. Carry the hash/event kind in PolicyStatusUpdate and emit this event from run_policy_status_reporter only after the RPC returns Ok(()); alternatively, record queueing here with ordinary debug tracing.

[NaveCohenMonday]

Addressed in c20b6b8. PolicyStatusUpdate now carries an explicit success-event kind and policy hash. The reporter emits the acknowledgement OCSF event only after report_policy_status returns Ok(()); terminal delivery failures discard the update without success telemetry.

[NaveCohenMonday]

Addressed in c20b6b8c.

The poll loop now accepts a scripted gateway and middleware connector in tests,
while production continues to use the cached gRPC client and normal registry
connector. The new stateful coverage proves:

• v1/H → v2/H emits exactly one LOADED(2) and leaves the OPA generation
  unchanged;
• a failed required middleware reconciliation emits no LOADED(2);
• the identical next poll retries reconciliation, and a later success emits
  exactly one LOADED(2);
• local override, global source, empty hash, equal version, and older version
  do not use the shortcut;
• a changed hash follows the normal OPA reload and status path.

I also carried the acknowledgement event kind and policy hash in
PolicyStatusUpdate. The OCSF success event now emits from
run_policy_status_reporter only after report_policy_status returns
successfully.

Validation:

CARGO_INCREMENTAL=0 mise exec -- cargo test -p openshell-sandbox --lib
test result: ok. 99 passed; 0 failed

CARGO_INCREMENTAL=0 mise run pre-commit
Finished successfully

I additionally mutation-checked the two critical guards: allowing equal
versions caused the exactly-once test to fail, and allowing acknowledgement
before runtime reconciliation caused the failed-middleware test to fail.

[drew]

gator-agent

Re-check After Author Update

Thanks @NaveCohenMonday. I reviewed current head c20b6b8c365dd5821594673d5fe06bf4cbe2ae93 after your update explaining the new scripted poll-loop coverage and post-RPC acknowledgement telemetry.

Validation: This concentrated sandbox-policy fix directly addresses confirmed issue #2518.
Head SHA: c20b6b8c365dd5821594673d5fe06bf4cbe2ae93

What I checked: the full current-head diff, the same-hash candidate and reconciliation gates, status-reporter delivery path, stateful positive/negative poll tests, and the architecture invariant.

Disposition: the two prior gator findings are resolved. The poll-loop tests now cover same-hash exactly-once acknowledgement, middleware failure/retry, and the requested negative scopes; success OCSF telemetry now follows a successful gateway status RPC. The independent review found no blocking correctness or security issue. It left one non-blocking determinism warning and one optional telemetry-test suggestion inline.

Docs: architecture/sandbox.md accurately documents the invariant. No Fern docs/navigation update is required because this fixes existing semantics without adding a direct UX workflow.

Tests: Code-only review; no local tests were run. A test:e2e pipeline run is required because this changes sandbox policy-enforcement reconciliation.

Next state: gator:watch-pipeline.

[github-actions]

Label test:e2e applied, but pull-request/2557 does not exist yet. A maintainer needs to comment /ok to test c20b6b8c365dd5821594673d5fe06bf4cbe2ae93 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

[drew]

/ok to test c20b6b8