Skip to content

feat(evidence): retain opt-in test execution receipts and dependency observations - #358

Draft
dinakarjs wants to merge 7 commits into
agentrust-io:mainfrom
dinakarjs:feat/execution-evidence-receipts
Draft

dinakarjs wants to merge 7 commits into
agentrust-io:mainfrom
dinakarjs:feat/execution-evidence-receipts

Conversation

@dinakarjs

@dinakarjs dinakarjs commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Compatibility-floor validation currently checks declarations, not execution evidence. This adds an opt-in Python command recorder that retains the source revision/cleanliness, manifest digest, declared and installed package versions, timestamps, command outcome, evidence class, Actions run URL and log digest.

A separate checker rejects missing, stale, unsuccessful or inconsistent evidence. It does not promote community/verified tiers or imply conformance. Receipts are unsigned observations, not adversarial attestations; command coverage and provenance still require review.

The existing LangChain interop command is wrapped and its receipt/log uploaded for 30 days even on failure. Receipt consistency is now a required check for this opt-in job. The first run exposed a stale LangChain declaration, so tested_against was updated from TRACE 0.9.0 to the observed 0.11.0; its tier is unchanged. No vendor code is newly discovered or executed, and no compatibility floors or tier declarations change.

Validation: nine recorder tests cover actual subprocess success/failure/timeout, log substitution, missing evidence, changed manifests and version/source mismatches. Recorder and compatibility tests: 11 passed. Default-rule Ruff passes on both new Python files. The normal validation workflow now runs recorder regression tests.

See docs/test-evidence.md for same-interpreter use, retention, sensitive-log handling and evidence limitations. AI-assisted implementation; DCO sign-offs included.

Validation evidence

Full repository validation test suite: 69 passed; manifest validator: 48 integrations, zero failures. At head 044ca19, validation, lint, workflow lint and trace-adapters CI passed. LangChain receipt and log retained at https://github.com/agentrust-io/integrations/actions/runs/39668849253/artifacts/12378777164 . The consistency report detected installed versions differing from the existing declaration.

Unrelated existing failures: Nobulex links return HTTP 451 (handled by #357); WCM example imports lack OpenSSL (handled by #356).

Latest head d213b84 includes the corrected declaration and required consistency gate. Validation, lint, workflow lint, CodeQL and trace-adapters CI passed. The retained receipt reports consistent with no errors: https://github.com/agentrust-io/integrations/actions/runs/39669274550/artifacts/12378772781 . Existing unrelated #356/#357 blockers remain.

Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>
Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>
Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>
Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>
Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Contributor Check: MEDIUM

Check Result
Profile MEDIUM
Credential LOW
Overall MEDIUM

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:MEDIUM Contributor check flagged MEDIUM risk label Oct 21, 2026
…ce receipt

Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>
…ce receipt

Signed-off-by: Srinivasa Consultants <dinakarjs@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:MEDIUM Contributor check flagged MEDIUM risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant