diff --git a/papers/.gitattributes b/papers/.gitattributes new file mode 100644 index 0000000..8237688 --- /dev/null +++ b/papers/.gitattributes @@ -0,0 +1,3 @@ +# Published source packages: keep their bytes exactly as released, so each +# package manifest and SHA256SUMS still verifies after checkout. +* -text diff --git a/papers/README.md b/papers/README.md new file mode 100644 index 0000000..53ec585 --- /dev/null +++ b/papers/README.md @@ -0,0 +1,16 @@ +# Papers + +LaTeX or Markdown source for the technical reports about this project. Each folder holds the +published `source.zip` of the version listed, unpacked and unchanged, so the build and the +reproduction steps in its own README work from here. The report page carries the built PDF, +citation files and every earlier version. + +Paper text is licensed CC BY 4.0. Code inside a paper folder states its own license in that +folder; the rest of this repository keeps its existing license. + +| Report | Version and date | Page | DOI | +|---|---|---|---| +| [From Agent Observability to Governance Evidence: A Privacy-Constrained Telemetry Contract](agentrust-telemetry/) | 2, 2026-10-01 | [page](https://agentrust-io.com/research/agentrust-telemetry/v2/) | [10.5281/zenodo.23091282](https://doi.org/10.5281/zenodo.23091282) | +| [Acceptance Before Export: A Bounded Evidence Contract for Governance Telemetry](telemetry-evidence-acceptance/) | 3, 2026-10-01 | [page](https://agentrust-io.com/research/telemetry-evidence-acceptance/v3/) | [10.5281/zenodo.23091285](https://doi.org/10.5281/zenodo.23091285) | + +These sources previously lived in a private papers repository, which has been retired. diff --git a/papers/agentrust-telemetry/README.md b/papers/agentrust-telemetry/README.md new file mode 100644 index 0000000..0f16723 --- /dev/null +++ b/papers/agentrust-telemetry/README.md @@ -0,0 +1,20 @@ +# From Agent Observability to Governance Evidence: A Privacy-Constrained Telemetry Contract + +Technical report, version 2, October 1, 2026. Not peer reviewed. Patent Pending. +Version 2 corrects the statement that approval digests prevent an approval from being attached to +another action, which the reference finalizer does not check, and restores the bold type that +the version 1 build dropped. Results are unchanged. + +Build: `tectonic paper.tex` (prepared with Tectonic 0.16.9; set SOURCE_DATE_EPOCH=1790812800 +to reproduce the published PDF byte for byte). + +Paper text is licensed CC BY 4.0: https://creativecommons.org/licenses/by/4.0/ +Project code retains its existing project license; see https://github.com/agentrust-io/agentrust-telemetry. + +This package contains the manuscript source and the rerun records. The evaluated code is +agentrust-telemetry 0.1.0-alpha.2, commit 29c13cfa4047a0f5784a90df63b22a3568453158. +`experiments/results/rerun-2026-09-27.txt` records the September 27 rerun and +`experiments/results/rerun-2026-10-01/` the October 1 audit rerun at that commit, a later +revision and main, with the probe scripts it used. It is not an independently replicated +reproduction package. +Current implementation guidance: https://agentrust-io.com/telemetry/ diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-09-27.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-09-27.txt new file mode 100644 index 0000000..7132a5a --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-09-27.txt @@ -0,0 +1,80 @@ +AgenTrust Telemetry software rerun, 2026-09-27 +Repository: https://github.com/agentrust-io/agentrust-telemetry +Evaluated commit: 29c13cfa4047a0f5784a90df63b22a3568453158 + (release 0.1.0-alpha.2, tag v0.1.0-alpha.2; main as of 2026-09-03; next main commit is dated 2026-09-05) +Source obtained with: git archive (no working-tree state) +Python: 3.12.10 on Windows 11 +Install: pip install -e ".[test,otel]" resolved on 2026-09-27, then the TRACE + dependency pinned to the release current on 2026-09-03 and PyYAML added so + the two YAML repository gates run instead of skipping: + agentrust-trace==0.9.0 jsonschema==4.26.0 referencing==0.37.0 rfc8785==0.1.4 + opentelemetry-api==1.45.0 opentelemetry-sdk==1.45.0 PyYAML==6.0.3 cbor2==5.9.0 + +## python -m unittest discover -s tests -v +Ran 111 tests +OK (111 passed, 0 skipped, 0 failed) +Per module: adapters 10, agt_adapter 4, agt_approval 6, agt_audit 4, + conformance 6, data_flow 4, evidence 11, otel_projectors 6, propagation 5, + reference_scenario 1, repository_gates 16, sdk 10, trace_adapter 9, + trace_adapter_refusals 10, usage 9 + +Same command with the dependency resolution of 2026-09-27 (agentrust-trace 0.11.0, no PyYAML): +Ran 111 tests: 108 passed, 2 skipped (PyYAML not installed), 1 failed + FAIL test_trace_adapter_refusals.test_a_failure_inside_official_signing_is_reported_as_one + cause: agentrust-trace 0.10.0 (published 2026-09-06) rejects a non-Ed25519 + stand-in key ("key must be an Ed25519PrivateKey, got FailsWhenUsed"); + the repository replaced the stand-in in commit 5d963d3 (2026-09-06). + +## python conformance/runner/validate.py +PASS valid/action.json +PASS valid/approval.json +PASS valid/data-flow.json +PASS valid/evidence.json +PASS valid/policy-decision.json +PASS valid/usage.json +PASS invalid/action-without-digest.json +PASS invalid/aggregate-cost-without-rollup.json +PASS invalid/model-call-with-rollup.json +PASS invalid/numeric-timestamp.json +PASS invalid/raw-content.json +PASS invalid/usage-without-measurement.json +PASS invalid/zero-trace-id.json +13/13 expected verdicts (6 valid accepted, 7 invalid rejected); exit 0 + +## Repository contract checks +python tools/check_otel_compatibility.py + PASS OTel GenAI matrix covers 16 span attributes, 8 metrics, and 6 event families + (profile agentrust-otel-genai-compatibility-v1, captured 2026-08-18, upstream + open-telemetry/semantic-conventions-genai @ a685613a, status development) +python tools/check_typescript_schemas.py + PASS TypeScript schemas match normative bytes +python tools/check_schemas.py + PASS 8 bundled schemas match normative bytes +python tools/check_versions.py + PASS contract=0.1.0-alpha.2 python=0.1.0a2 npm_name=@agentrust-io/telemetry npm=0.1.0-alpha.2 + +## Boundary probes (scripted, against the evaluated source; probe script not part of the repository) +P1 durable callback returns False: EvidencePersistenceError raised; log projection calls=0; accepted entries=0 +P2 log exporter raises after acceptance: accepted=True; entries=1; + projection_errors=['log projection failed: RuntimeError: exporter down'] +P3 empty accumulator sealed: entries=0 completeness=complete +P4 callback commits externally then raises once, caller retries: external writes=2, local entries=1 + +## Post-evaluation regression tests run against the evaluated source +tests/test_trace_adapter_refusals.py copied from main c3515681 (fix merged 2026-09-25, PR #65) +Ran 14 tests: 3 failures, 1 error + FAIL test_an_event_edited_after_sealing_is_refused + FAIL test_a_dropped_entry_is_refused + FAIL test_a_substituted_chain_digest_is_refused + ERROR test_a_consistent_chain_over_a_malformed_event_is_refused +The evaluated finalizer signs snapshots whose entries no longer match the chain digest. + +## Comparison: current main +Commit c3515681261fd0ab6e6aa38e20eb2f071aee7a20 (release 0.1.0-alpha.5), same Python, +dependencies resolved 2026-09-27 (agentrust-trace 0.11.0, PyYAML installed) +python -m unittest discover -s tests: Ran 128 tests, OK +conformance runner: 13/13 expected verdicts (same 13 fixtures) +check_otel_compatibility: 16 span attributes, 8 metrics, 6 event families + +NOT RERUN: TypeScript test suite (npm run check); collector or backend interoperability; +performance. No hardware is involved in this evaluation. diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/RERUN.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/RERUN.txt new file mode 100644 index 0000000..b961cb5 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/RERUN.txt @@ -0,0 +1,65 @@ +AgenTrust Telemetry software rerun, 2026-10-01 (audit of technical report v1) +Repository: https://github.com/agentrust-io/agentrust-telemetry +Evaluated commit: 29c13cfa4047a0f5784a90df63b22a3568453158 (tag v0.1.0-alpha.2) +Later revision: c3515681261fd0ab6e6aa38e20eb2f071aee7a20 (release 0.1.0-alpha.5, 2026-09-26) +Current main: c86753cad6bbe9f00c5080489a12e0f94df34899 (2026-09-30; no src/ or + packages/typescript/src change since c351568) +Source obtained with: git archive ; tar commit id checked (git get-tar-commit-id) +Python 3.12.10, Node v24.18.1, npm 11.16.0, Windows 11. Fresh virtual environments. + +## Pinned commit, Python +Install: pip install -e ".[test,otel]" (resolved 2026-10-01: agentrust-trace 0.11.0, + opentelemetry 1.45.0), then agentrust-trace==0.9.0 PyYAML==6.0.3 cbor2==5.9.0 + (dependencies-pinned.txt) +python -m unittest discover -s tests -v: Ran 111 tests, OK, 0 skipped (python-unittest-pinned.txt) +Same command before the TRACE pin and PyYAML (agentrust-trace 0.11.0): + Ran 111 tests, FAILED (failures=1, skipped=2) + FAIL test_trace_adapter_refusals.test_a_failure_inside_official_signing_is_reported_as_one +python conformance/runner/validate.py: 13 PASS (6 valid accepted, 7 invalid rejected), exit 0 +python tools/check_otel_compatibility.py: PASS 16 span attributes, 8 metrics, 6 event families +python tools/check_typescript_schemas.py: PASS TypeScript schemas match normative bytes +python tools/check_schemas.py: PASS 8 bundled schemas match normative bytes +python tools/check_versions.py: PASS contract=0.1.0-alpha.2 python=0.1.0a2 npm=0.1.0-alpha.2 + +## Pinned commit, TypeScript (not rerun in the v1 report) +packages/typescript: npm ci (lockfile), npm run check (tsc build, tsx --test, npm pack --dry-run) + tests 41, pass 41, fail 0, skipped 0; pack dry run produced + agentrust-io-telemetry-0.1.0-alpha.2.tgz (typescript-check-pinned.txt) + +## Boundary probes P1 to P4 (telemetry_probes.py, reconstructed from Section 6) +pin, later revision and current main give identical results (probes-output.txt): +P1 refused ack: EvidencePersistenceError raised; log projection calls=0; entries=0 +P2 failing exporter: entries=1; projection_errors=['log projection failed: RuntimeError: exporter down'] +P3 empty seal: entries=0 completeness=complete +P4 lost ack then retry: external writes=2; local entries=1 + +## Section 7 findings, rechecked (validation_probes.py and regression tests) +At the pin: 5000-deep nesting raises RecursionError; a non-string key raises + AttributeError; NaN and Infinity attributes pass validate() and are refused only + by the accumulator (EvidenceError); a durable callback that calls snapshot() + on the same thread deadlocks (no return in 5 s). +At c351568 and current main: all four cases return EventValidationError or succeed. +tests/test_trace_adapter_refusals.py from c351568 run against the pinned source: + Ran 14 tests, FAILED (failures=3, errors=1) (regression-tests-at-pin-python.txt) +The TypeScript regression test added by the same fix, appended to the pinned + test/trace-finalizer.test.ts: pass 5, fail 1, "Missing expected exception" + (regression-test-at-pin-typescript.txt). The TypeScript finalizer at the pin + also signs an edited, truncated or digest-substituted snapshot. + +## Current main (c86753c), dependencies resolved 2026-10-01 plus PyYAML +python -m unittest discover -s tests: Ran 129 tests, OK (python-unittest-current-main.txt) + (128 at c351568; one repository-gate test added in #70) +conformance runner: 13/13; OTel matrix: 16 / 8 / 6; check_versions: 0.1.0-alpha.5 +TypeScript npm run check: tests 42, pass 42 (typescript-check-current-main.txt) + +## Not run +Collector or backend interoperability; performance; durable store; crash and restart. + +## Approval to executed-action binding (approval_binding_probe.py, added by this audit) +Challenge decision, approval.requested and approval.approved bound to action digest b*64, +then action.executed carrying the same approval_id with digest b*64 or c*64, sealed +complete and finalized with agentrust-trace 0.11.0 (approval-binding-output.txt): + pin: same digest -> affirming; different digest -> affirming + main: same digest -> affirming; different digest -> affirming +The finalizer checks request-to-approval binding (including action_digest) but does not +compare the approved digest with the executed action's digest. diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/approval-binding-output.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/approval-binding-output.txt new file mode 100644 index 0000000..f344bf3 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/approval-binding-output.txt @@ -0,0 +1,6 @@ +== pin +executed action with same digest as approved: appraisal=affirming +executed action with different digest: appraisal=affirming +== main +executed action with same digest as approved: appraisal=affirming +executed action with different digest: appraisal=affirming diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/approval_binding_probe.py b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/approval_binding_probe.py new file mode 100644 index 0000000..4b49de2 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/approval_binding_probe.py @@ -0,0 +1,26 @@ +"""Does TRACE finalization notice an executed action whose digest differs from the approved one? + +Usage: python approval_binding_probe.py SOURCE_DIR (an agentrust-telemetry tree) +""" +import json, sys +from pathlib import Path +ROOT = Path(sys.argv[1]); sys.path.insert(0, str(ROOT / 'src')) +from agentrust_telemetry import EvidenceAccumulator, SchemaValidator, TraceConfiguration, finalize_trace +from agentrust_trace import generate_key +fx = lambda n: json.loads((ROOT / 'conformance/fixtures/valid' / n).read_text()) +v = SchemaValidator(ROOT / 'spec/schema') +config = TraceConfiguration(subject='spiffe://example.test/agent/workflow', model_provider='example', model_id='m', model_version='1', + build_digest='sha256:' + 'b' * 64, build_slsa_level=1, origin_kind='self', origin_producer='example-runtime', + appraisal_verifier='https://example.test/verifier', classification_taxonomy='example.enterprise.v1', + classification_order=('public', 'internal', 'confidential', 'restricted')) +policy = fx('policy-decision.json'); policy['decision'] = 'challenge' +request = fx('approval.json'); request.update(event_type='approval.requested', event_id='018f0f7d-7a13-7cc2-8000-000000000010', + policy_event_id=policy['event_id'], chain_id='operators', chain_version='3', expires_at_unix_nano='1787080100000000000', time_unix_nano='1787079500000000000') +approval = fx('approval.json'); approval.update(event_type='approval.approved', policy_event_id=policy['event_id'], chain_id='operators', + chain_version='3', expires_at_unix_nano=request['expires_at_unix_nano']) +for label, digest in (('same digest as approved', 'b' * 64), ('different digest', 'c' * 64)): + action = fx('action.json'); action['action_digest'] = {'algorithm': 'sha256', 'value': digest} + acc = EvidenceAccumulator(policy['run_id'], v) + for e in (policy, request, approval, fx('data-flow.json'), action): acc.append(e) + rec = finalize_trace(acc.seal(completeness='complete'), config, signing_key=generate_key()) + print(f'executed action with {label}: appraisal={rec["appraisal"]["status"]}') diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/dependencies-current-main.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/dependencies-current-main.txt new file mode 100644 index 0000000..e48f4f6 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/dependencies-current-main.txt @@ -0,0 +1,32 @@ +agentrust-trace==0.11.0 +annotated-types==0.8.0 +arrow==1.4.0 +attrs==26.1.0 +cffi==2.1.1 +cryptography==50.0.2 +fqdn==1.6.0 +idna==3.20 +isoduration==20.11.0 +jsonpointer==3.1.1 +jsonschema==4.26.0 +jsonschema-specifications==2025.9.1 +opentelemetry-api==1.45.0 +opentelemetry-sdk==1.45.0 +opentelemetry-semantic-conventions==0.66b0 +pycparser==3.0 +pydantic==2.13.5 +pydantic_core==2.46.5 +python-dateutil==2.9.0.post0 +PyYAML==6.0.3 +referencing==0.37.0 +rfc3339-validator==0.1.4 +rfc3986-validator==0.1.1 +rfc3987==1.3.8 +rfc8785==0.1.4 +rpds-py==2026.6.3 +six==1.17.0 +typing-inspection==0.4.4 +typing_extensions==4.16.0 +tzdata==2026.4 +uri-template==1.3.0 +webcolors==25.10.0 diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/dependencies-pinned.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/dependencies-pinned.txt new file mode 100644 index 0000000..50c9df4 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/dependencies-pinned.txt @@ -0,0 +1,33 @@ +agentrust-trace==0.9.0 +annotated-types==0.8.0 +arrow==1.4.0 +attrs==26.1.0 +cbor2==5.9.0 +cffi==2.1.1 +cryptography==50.0.2 +fqdn==1.6.0 +idna==3.20 +isoduration==20.11.0 +jsonpointer==3.1.1 +jsonschema==4.26.0 +jsonschema-specifications==2025.9.1 +opentelemetry-api==1.45.0 +opentelemetry-sdk==1.45.0 +opentelemetry-semantic-conventions==0.66b0 +pycparser==3.0 +pydantic==2.13.5 +pydantic_core==2.46.5 +python-dateutil==2.9.0.post0 +PyYAML==6.0.3 +referencing==0.37.0 +rfc3339-validator==0.1.4 +rfc3986-validator==0.1.1 +rfc3987==1.3.8 +rfc8785==0.1.4 +rpds-py==2026.6.3 +six==1.17.0 +typing-inspection==0.4.4 +typing_extensions==4.16.0 +tzdata==2026.4 +uri-template==1.3.0 +webcolors==25.10.0 diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/probes-output.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/probes-output.txt new file mode 100644 index 0000000..5250685 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/probes-output.txt @@ -0,0 +1,33 @@ +== pin +P1 refused ack: EvidencePersistenceError raised=True; log projection calls=0; accepted entries=0 +P2 failing exporter: entries=1; projection_errors=['log projection failed: RuntimeError: exporter down'] +P3 empty seal: entries=0 completeness=complete +P4 lost ack then retry: external writes=2; local entries=1 +deep nesting (5000): RecursionError +non-string key: AttributeError +NaN attribute, validate: accepted +Infinity attribute, validate: accepted +NaN attribute, accumulator append: EvidenceError +callback re-entering snapshot(): deadlocked (no return in 5 s) +== later +P1 refused ack: EvidencePersistenceError raised=True; log projection calls=0; accepted entries=0 +P2 failing exporter: entries=1; projection_errors=['log projection failed: RuntimeError: exporter down'] +P3 empty seal: entries=0 completeness=complete +P4 lost ack then retry: external writes=2; local entries=1 +deep nesting (5000): EventValidationError +non-string key: EventValidationError +NaN attribute, validate: EventValidationError +Infinity attribute, validate: EventValidationError +NaN attribute, accumulator append: EventValidationError +callback re-entering snapshot(): accepted +== main +P1 refused ack: EvidencePersistenceError raised=True; log projection calls=0; accepted entries=0 +P2 failing exporter: entries=1; projection_errors=['log projection failed: RuntimeError: exporter down'] +P3 empty seal: entries=0 completeness=complete +P4 lost ack then retry: external writes=2; local entries=1 +deep nesting (5000): EventValidationError +non-string key: EventValidationError +NaN attribute, validate: EventValidationError +Infinity attribute, validate: EventValidationError +NaN attribute, accumulator append: EventValidationError +callback re-entering snapshot(): accepted diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/python-unittest-current-main.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/python-unittest-current-main.txt new file mode 100644 index 0000000..4c874e1 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/python-unittest-current-main.txt @@ -0,0 +1,152 @@ +test_cedar_preserves_final_decision_and_reports_skipped_errors (test_adapters.AdapterTests.test_cedar_preserves_final_decision_and_reports_skipped_errors) ... ok +test_cedar_rejects_error_messages_as_codes (test_adapters.AdapterTests.test_cedar_rejects_error_messages_as_codes) ... ok +test_cedar_rejects_string_as_identifier_collection (test_adapters.AdapterTests.test_cedar_rejects_string_as_identifier_collection) ... ok +test_factory_builds_and_validates_any_supported_family (test_adapters.AdapterTests.test_factory_builds_and_validates_any_supported_family) ... ok +test_factory_matches_cross_language_golden_event (test_adapters.AdapterTests.test_factory_matches_cross_language_golden_event) ... ok +test_factory_prevents_envelope_override (test_adapters.AdapterTests.test_factory_prevents_envelope_override) ... ok +test_factory_serializes_nanoseconds_without_precision_loss (test_adapters.AdapterTests.test_factory_serializes_nanoseconds_without_precision_loss) ... ok +test_opa_boolean_log_maps_without_copying_input_or_result (test_adapters.AdapterTests.test_opa_boolean_log_maps_without_copying_input_or_result) ... ok +test_opa_complex_result_requires_explicit_mapper (test_adapters.AdapterTests.test_opa_complex_result_requires_explicit_mapper) ... ok +test_opa_rejects_malformed_source_collections (test_adapters.AdapterTests.test_opa_rejects_malformed_source_collections) ... ok +test_latency_that_overflows_nanoseconds_is_a_value_error (test_agt_adapter.AgtAdapterTests.test_latency_that_overflows_nanoseconds_is_a_value_error) ... ok +test_policy_event_maps_challenge_and_excludes_free_form_content (test_agt_adapter.AgtAdapterTests.test_policy_event_maps_challenge_and_excludes_free_form_content) ... ok +test_policy_mapper_rejects_ambiguous_decision_and_reason_message (test_agt_adapter.AgtAdapterTests.test_policy_mapper_rejects_ambiguous_decision_and_reason_message) ... ok +test_sink_prevalidates_whole_batch_before_emission (test_agt_adapter.AgtAdapterTests.test_sink_prevalidates_whole_batch_before_emission) ... ok +test_sink_uses_source_result_sentinels_and_reports_projection_failure (test_agt_adapter.AgtAdapterTests.test_sink_uses_source_result_sentinels_and_reports_projection_failure) ... ok +test_structured_kind_or_decision_is_a_value_error (test_agt_adapter.AgtAdapterTests.test_structured_kind_or_decision_is_a_value_error) ... ok +test_bindings_reject_fields_that_are_absent_on_both_records (test_agt_approval.AgtApprovalAdapterTests.test_bindings_reject_fields_that_are_absent_on_both_records) ... ok +test_protocol_chain_preserves_policy_request_and_resolution_links (test_agt_approval.AgtApprovalAdapterTests.test_protocol_chain_preserves_policy_request_and_resolution_links) ... ok +test_request_rejects_wrong_policy_action_and_chain (test_agt_approval.AgtApprovalAdapterTests.test_request_rejects_wrong_policy_action_and_chain) ... ok +test_resolution_rejects_pre_request_time_and_malformed_chain_digest (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_pre_request_time_and_malformed_chain_digest) ... ok +test_resolution_rejects_wrong_request_and_wrong_action (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_wrong_request_and_wrong_action) ... ok +test_structured_outcome_or_verdict_is_a_value_error (test_agt_approval.AgtApprovalAdapterTests.test_structured_outcome_or_verdict_is_a_value_error) ... ok +test_terminal_outcomes_map_without_treating_votes_as_resolutions (test_agt_approval.AgtApprovalAdapterTests.test_terminal_outcomes_map_without_treating_votes_as_resolutions) ... ok +test_action_audit_requires_full_action_digest_and_computes_duration (test_agt_audit.AgtAuditAdapterTests.test_action_audit_requires_full_action_digest_and_computes_duration) ... ok +test_action_rejects_missing_or_reversed_timing (test_agt_audit.AgtAuditAdapterTests.test_action_rejects_missing_or_reversed_timing) ... ok +test_blocked_action_is_denied_even_if_source_outcome_is_success (test_agt_audit.AgtAuditAdapterTests.test_blocked_action_is_denied_even_if_source_outcome_is_success) ... ok +test_policy_audit_maps_without_free_form_data_or_resource (test_agt_audit.AgtAuditAdapterTests.test_policy_audit_maps_without_free_form_data_or_resource) ... ok +test_structured_decision_or_outcome_is_a_value_error (test_agt_audit.AgtAuditAdapterTests.test_structured_decision_or_outcome_is_a_value_error) ... ok +test_action_payload_field_is_rejected_by_schema_and_privacy_gate (test_conformance.ConformanceTests.test_action_payload_field_is_rejected_by_schema_and_privacy_gate) ... ok +test_all_schemas_are_valid_draft_2020_12 (test_conformance.ConformanceTests.test_all_schemas_are_valid_draft_2020_12) ... ok +test_committed_suite_has_expected_verdicts (test_conformance.ConformanceTests.test_committed_suite_has_expected_verdicts) ... ok +test_nested_prohibited_key_is_rejected_independently (test_conformance.ConformanceTests.test_nested_prohibited_key_is_rejected_independently) ... ok +test_unknown_event_type_is_rejected (test_conformance.ConformanceTests.test_unknown_event_type_is_rejected) ... ok +test_uppercase_trace_id_is_rejected (test_conformance.ConformanceTests.test_uppercase_trace_id_is_rejected) ... ok +test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason (test_data_flow.DataFlowTests.test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason) ... ok +test_classifier_cannot_return_arbitrary_metadata (test_data_flow.DataFlowTests.test_classifier_cannot_return_arbitrary_metadata) ... ok +test_classifier_sees_content_but_event_contains_only_closed_metadata (test_data_flow.DataFlowTests.test_classifier_sees_content_but_event_contains_only_closed_metadata) ... ok +test_metadata_identifiers_reject_prose_paths_and_query_values (test_data_flow.DataFlowTests.test_metadata_identifiers_reject_prose_paths_and_query_values) ... ok +test_structured_agt_classification_is_a_value_error (test_data_flow.DataFlowTests.test_structured_agt_classification_is_a_value_error) ... ok +test_callback_and_returned_entry_cannot_mutate_retained_evidence (test_evidence.EvidenceTests.test_callback_and_returned_entry_cannot_mutate_retained_evidence) ... ok +test_chain_is_ordered_and_has_stable_golden_digest (test_evidence.EvidenceTests.test_chain_is_ordered_and_has_stable_golden_digest) ... ok +test_client_does_not_project_when_evidence_fails (test_evidence.EvidenceTests.test_client_does_not_project_when_evidence_fails) ... ok +test_client_isolates_operational_event_from_evidence_sink (test_evidence.EvidenceTests.test_client_isolates_operational_event_from_evidence_sink) ... ok +test_client_reports_evidence_persistence (test_evidence.EvidenceTests.test_client_reports_evidence_persistence) ... ok +test_context_mismatch_never_enters_evidence (test_evidence.EvidenceTests.test_context_mismatch_never_enters_evidence) ... ok +test_durable_callback_acknowledges_before_local_acceptance (test_evidence.EvidenceTests.test_durable_callback_acknowledges_before_local_acceptance) ... ok +test_failed_durable_ack_is_fail_closed_and_retryable (test_evidence.EvidenceTests.test_failed_durable_ack_is_fail_closed_and_retryable) ... ok +test_non_finite_number_cannot_enter_digest_chain (test_evidence.EvidenceTests.test_non_finite_number_cannot_enter_digest_chain) ... ok +test_reentrant_durable_append_call_is_refused_not_deadlocked (test_evidence.EvidenceTests.test_reentrant_durable_append_call_is_refused_not_deadlocked) ... ok +test_reentrant_seal_call_is_refused_not_deadlocked (test_evidence.EvidenceTests.test_reentrant_seal_call_is_refused_not_deadlocked) ... ok +test_reentrant_snapshot_call_is_safe_and_reflects_committed_state (test_evidence.EvidenceTests.test_reentrant_snapshot_call_is_safe_and_reflects_committed_state) ... ok +test_rejects_cross_run_duplicate_overflow_and_append_after_seal (test_evidence.EvidenceTests.test_rejects_cross_run_duplicate_overflow_and_append_after_seal) ... ok +test_snapshot_is_defensive_and_unsealed_never_claims_complete (test_evidence.EvidenceTests.test_snapshot_is_defensive_and_unsealed_never_claims_complete) ... ok +test_action_metrics_exclude_action_and_agent_identifiers (test_otel_projectors.OTelProjectorTests.test_action_metrics_exclude_action_and_agent_identifiers) ... ok +test_classification_allowlist_is_bounded (test_otel_projectors.OTelProjectorTests.test_classification_allowlist_is_bounded) ... ok +test_classification_metric_is_allowlisted_or_collapsed (test_otel_projectors.OTelProjectorTests.test_classification_metric_is_allowlisted_or_collapsed) ... ok +test_metric_failure_is_reported_independently (test_otel_projectors.OTelProjectorTests.test_metric_failure_is_reported_independently) ... ok +test_real_metrics_record_usage_without_high_cardinality_ids (test_otel_projectors.OTelProjectorTests.test_real_metrics_record_usage_without_high_cardinality_ids) ... ok +test_real_otel_log_preserves_event_and_active_trace_context (test_otel_projectors.OTelProjectorTests.test_real_otel_log_preserves_event_and_active_trace_context) ... ok +test_async_handoff_creates_link_without_forcing_parentage (test_propagation.PropagationTests.test_async_handoff_creates_link_without_forcing_parentage) ... ok +test_header_lookup_is_case_insensitive (test_propagation.PropagationTests.test_header_lookup_is_case_insensitive) ... ok +test_injection_rejects_header_injection (test_propagation.PropagationTests.test_injection_rejects_header_injection) ... ok +test_link_requires_valid_w3c_context (test_propagation.PropagationTests.test_link_requires_valid_w3c_context) ... ok +test_sync_handoff_preserves_trace_and_agent_parent (test_propagation.PropagationTests.test_sync_handoff_preserves_trace_and_agent_parent) ... ok +test_complete_governed_workflow (test_reference_scenario.ReferenceScenarioTests.test_complete_governed_workflow) ... ok +test_contract_version_rejects_unsupported_spellings (test_repository_gates.RepositoryGateTests.test_contract_version_rejects_unsupported_spellings) ... ok +test_contract_versions_map_to_ecosystem_spellings (test_repository_gates.RepositoryGateTests.test_contract_versions_map_to_ecosystem_spellings) ... ok +test_declared_versions_are_consistent (test_repository_gates.RepositoryGateTests.test_declared_versions_are_consistent) ... ok +test_every_lock_keeps_every_input (test_repository_gates.RepositoryGateTests.test_every_lock_keeps_every_input) ... ok +test_fuzz_seeds_match_the_conformance_fixtures (test_repository_gates.RepositoryGateTests.test_fuzz_seeds_match_the_conformance_fixtures) ... ok +test_issue_templates_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_issue_templates_are_valid_yaml_when_parser_available) ... ok +test_npm_dist_tag_matches_the_declared_contract (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_matches_the_declared_contract) ... ok +test_npm_dist_tag_never_moves_latest_for_a_prerelease (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_never_moves_latest_for_a_prerelease) ... ok +test_npm_dist_tag_rejects_an_unsupported_version (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_rejects_an_unsupported_version) ... ok +test_otel_matrix_gate_detects_attribute_metric_and_family_drift (test_repository_gates.RepositoryGateTests.test_otel_matrix_gate_detects_attribute_metric_and_family_drift) ... ok +test_otel_matrix_matches_shipped_projection (test_repository_gates.RepositoryGateTests.test_otel_matrix_matches_shipped_projection) ... ok +test_release_tag_must_match_contract (test_repository_gates.RepositoryGateTests.test_release_tag_must_match_contract) ... ok +test_schema_gate_detects_changed_bytes (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_changed_bytes) ... ok +test_schema_gate_detects_missing_and_extra_files (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_missing_and_extra_files) ... ok +test_source_manifest_includes_otel_matrix (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_otel_matrix) ... ok +test_source_manifest_includes_release_policy (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_release_policy) ... ok +test_typescript_schemas_match_normative_bytes (test_repository_gates.RepositoryGateTests.test_typescript_schemas_match_normative_bytes) ... ok +test_workflows_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_workflows_are_valid_yaml_when_parser_available) ... ok +test_bundled_schemas_match_normative_schemas (test_sdk.SdkTests.test_bundled_schemas_match_normative_schemas) ... ok +test_context_mismatch_rejects_before_projection (test_sdk.SdkTests.test_context_mismatch_rejects_before_projection) ... ok +test_deeply_nested_input_is_a_validation_error (test_sdk.SdkTests.test_deeply_nested_input_is_a_validation_error) ... ok +test_emits_correlated_span_event_and_log (test_sdk.SdkTests.test_emits_correlated_span_event_and_log) ... ok +test_explicit_attribute_allowlist_is_narrow (test_sdk.SdkTests.test_explicit_attribute_allowlist_is_narrow) ... ok +test_extension_attributes_are_deny_by_default (test_sdk.SdkTests.test_extension_attributes_are_deny_by_default) ... ok +test_log_body_is_a_deep_copy (test_sdk.SdkTests.test_log_body_is_a_deep_copy) ... ok +test_no_span_still_emits_run_correlated_log (test_sdk.SdkTests.test_no_span_still_emits_run_correlated_log) ... ok +test_non_finite_numbers_are_a_validation_error (test_sdk.SdkTests.test_non_finite_numbers_are_a_validation_error) ... ok +test_non_string_keys_are_a_validation_error (test_sdk.SdkTests.test_non_string_keys_are_a_validation_error) ... ok +test_privacy_violation_rejects_before_projection (test_sdk.SdkTests.test_privacy_violation_rejects_before_projection) ... ok +test_projection_failures_are_reported_independently (test_sdk.SdkTests.test_projection_failures_are_reported_independently) ... ok +test_real_otel_span_receives_event (test_sdk.SdkTests.test_real_otel_span_receives_event) ... ok +test_self_referencing_input_is_a_validation_error (test_sdk.SdkTests.test_self_referencing_input_is_a_validation_error) ... ok +test_action_events_create_truthful_tool_transcript (test_trace_adapter.TraceAdapterTests.test_action_events_create_truthful_tool_transcript) ... ok +test_finalizes_signs_and_validates_official_trace_record (test_trace_adapter.TraceAdapterTests.test_finalizes_signs_and_validates_official_trace_record) ... ok +test_linked_terminal_approval_resolves_policy_challenge (test_trace_adapter.TraceAdapterTests.test_linked_terminal_approval_resolves_policy_challenge) ... ok +test_refuses_conflicting_policy_bundles (test_trace_adapter.TraceAdapterTests.test_refuses_conflicting_policy_bundles) ... ok +test_refuses_missing_signer (test_trace_adapter.TraceAdapterTests.test_refuses_missing_signer) ... ok +test_refuses_policy_digest_algorithm_trace_cannot_represent (test_trace_adapter.TraceAdapterTests.test_refuses_policy_digest_algorithm_trace_cannot_represent) ... ok +test_refuses_unranked_classification (test_trace_adapter.TraceAdapterTests.test_refuses_unranked_classification) ... ok +test_refuses_unsealed_or_incomplete_evidence (test_trace_adapter.TraceAdapterTests.test_refuses_unsealed_or_incomplete_evidence) ... ok +test_unbound_or_expired_approval_cannot_resolve_policy_challenge (test_trace_adapter.TraceAdapterTests.test_unbound_or_expired_approval_cannot_resolve_policy_challenge) ... ok +test_a_consistent_chain_over_a_malformed_event_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_consistent_chain_over_a_malformed_event_is_refused) +Digests recomputed over an event the schema never accepted. ... ok +test_a_dropped_entry_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_dropped_entry_is_refused) ... ok +test_a_failure_inside_official_signing_is_reported_as_one (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_failure_inside_official_signing_is_reported_as_one) +A key that satisfies the type check and then fails when used. ... ok +test_a_policy_decision_without_a_bundle_digest_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_policy_decision_without_a_bundle_digest_is_refused) ... ok +test_a_repeated_classification_cannot_rank_anything (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_repeated_classification_cannot_rank_anything) ... ok +test_a_run_with_no_data_flow_has_nothing_to_classify (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_data_flow_has_nothing_to_classify) ... ok +test_a_run_with_no_policy_decision_has_no_policy_to_bind (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_policy_decision_has_no_policy_to_bind) ... ok +test_a_substituted_chain_digest_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_substituted_chain_digest_is_refused) ... ok +test_a_taxonomy_the_configuration_does_not_name_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_taxonomy_the_configuration_does_not_name_is_refused) ... ok +test_an_empty_run_carries_no_chained_evidence (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_empty_run_carries_no_chained_evidence) ... ok +test_an_event_edited_after_sealing_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_event_edited_after_sealing_is_refused) ... ok +test_every_required_configuration_field_is_named_when_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_every_required_configuration_field_is_named_when_absent) ... ok +test_the_optional_dependency_is_named_when_it_is_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_the_optional_dependency_is_named_when_it_is_absent) ... ok +test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode) ... ok +test_accumulator_rejects_non_finite_external_cost (test_usage.UsageTests.test_accumulator_rejects_non_finite_external_cost) ... ok +test_bool_is_not_accepted_as_token_integer (test_usage.UsageTests.test_bool_is_not_accepted_as_token_integer) ... ok +test_cost_requires_finite_amount_and_explicit_provenance (test_usage.UsageTests.test_cost_requires_finite_amount_and_explicit_provenance) ... ok +test_reused_event_id_with_changed_data_is_rejected (test_usage.UsageTests.test_reused_event_id_with_changed_data_is_rejected) ... ok +test_rollup_deduplicates_and_reports_partial_coverage (test_usage.UsageTests.test_rollup_deduplicates_and_reports_partial_coverage) ... ok +test_rollup_rejects_non_leaf_and_mixed_currency (test_usage.UsageTests.test_rollup_rejects_non_leaf_and_mixed_currency) ... ok +test_rollups_match_shared_cross_language_vector (test_usage.UsageTests.test_rollups_match_shared_cross_language_vector) ... ok +test_schema_rejects_aggregate_cost_without_run_rollup (test_usage.UsageTests.test_schema_rejects_aggregate_cost_without_run_rollup) ... ok +test_unknown_price_is_absent_not_zero (test_usage.UsageTests.test_unknown_price_is_absent_not_zero) ... ok + +---------------------------------------------------------------------- +Ran 129 tests in 7.992s + +OK +PASS conformance\fixtures\valid\action.json +PASS conformance\fixtures\valid\approval.json +PASS conformance\fixtures\valid\data-flow.json +PASS conformance\fixtures\valid\evidence.json +PASS conformance\fixtures\valid\policy-decision.json +PASS conformance\fixtures\valid\usage.json +PASS conformance\fixtures\invalid\action-without-digest.json +PASS conformance\fixtures\invalid\aggregate-cost-without-rollup.json +PASS conformance\fixtures\invalid\model-call-with-rollup.json +PASS conformance\fixtures\invalid\numeric-timestamp.json +PASS conformance\fixtures\invalid\raw-content.json +PASS conformance\fixtures\invalid\usage-without-measurement.json +PASS conformance\fixtures\invalid\zero-trace-id.json +PASS contract=0.1.0-alpha.5 python=0.1.0a5 npm_name=@agentrust-io/telemetry npm=0.1.0-alpha.5 +PASS OTel GenAI matrix covers 16 span attributes, 8 metrics, and 6 event families +PASS TypeScript schemas match normative bytes diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/python-unittest-pinned.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/python-unittest-pinned.txt new file mode 100644 index 0000000..e2daa06 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/python-unittest-pinned.txt @@ -0,0 +1,133 @@ +test_cedar_preserves_final_decision_and_reports_skipped_errors (test_adapters.AdapterTests.test_cedar_preserves_final_decision_and_reports_skipped_errors) ... ok +test_cedar_rejects_error_messages_as_codes (test_adapters.AdapterTests.test_cedar_rejects_error_messages_as_codes) ... ok +test_cedar_rejects_string_as_identifier_collection (test_adapters.AdapterTests.test_cedar_rejects_string_as_identifier_collection) ... ok +test_factory_builds_and_validates_any_supported_family (test_adapters.AdapterTests.test_factory_builds_and_validates_any_supported_family) ... ok +test_factory_matches_cross_language_golden_event (test_adapters.AdapterTests.test_factory_matches_cross_language_golden_event) ... ok +test_factory_prevents_envelope_override (test_adapters.AdapterTests.test_factory_prevents_envelope_override) ... ok +test_factory_serializes_nanoseconds_without_precision_loss (test_adapters.AdapterTests.test_factory_serializes_nanoseconds_without_precision_loss) ... ok +test_opa_boolean_log_maps_without_copying_input_or_result (test_adapters.AdapterTests.test_opa_boolean_log_maps_without_copying_input_or_result) ... ok +test_opa_complex_result_requires_explicit_mapper (test_adapters.AdapterTests.test_opa_complex_result_requires_explicit_mapper) ... ok +test_opa_rejects_malformed_source_collections (test_adapters.AdapterTests.test_opa_rejects_malformed_source_collections) ... ok +test_policy_event_maps_challenge_and_excludes_free_form_content (test_agt_adapter.AgtAdapterTests.test_policy_event_maps_challenge_and_excludes_free_form_content) ... ok +test_policy_mapper_rejects_ambiguous_decision_and_reason_message (test_agt_adapter.AgtAdapterTests.test_policy_mapper_rejects_ambiguous_decision_and_reason_message) ... ok +test_sink_prevalidates_whole_batch_before_emission (test_agt_adapter.AgtAdapterTests.test_sink_prevalidates_whole_batch_before_emission) ... ok +test_sink_uses_source_result_sentinels_and_reports_projection_failure (test_agt_adapter.AgtAdapterTests.test_sink_uses_source_result_sentinels_and_reports_projection_failure) ... ok +test_bindings_reject_fields_that_are_absent_on_both_records (test_agt_approval.AgtApprovalAdapterTests.test_bindings_reject_fields_that_are_absent_on_both_records) ... ok +test_protocol_chain_preserves_policy_request_and_resolution_links (test_agt_approval.AgtApprovalAdapterTests.test_protocol_chain_preserves_policy_request_and_resolution_links) ... ok +test_request_rejects_wrong_policy_action_and_chain (test_agt_approval.AgtApprovalAdapterTests.test_request_rejects_wrong_policy_action_and_chain) ... ok +test_resolution_rejects_pre_request_time_and_malformed_chain_digest (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_pre_request_time_and_malformed_chain_digest) ... ok +test_resolution_rejects_wrong_request_and_wrong_action (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_wrong_request_and_wrong_action) ... ok +test_terminal_outcomes_map_without_treating_votes_as_resolutions (test_agt_approval.AgtApprovalAdapterTests.test_terminal_outcomes_map_without_treating_votes_as_resolutions) ... ok +test_action_audit_requires_full_action_digest_and_computes_duration (test_agt_audit.AgtAuditAdapterTests.test_action_audit_requires_full_action_digest_and_computes_duration) ... ok +test_action_rejects_missing_or_reversed_timing (test_agt_audit.AgtAuditAdapterTests.test_action_rejects_missing_or_reversed_timing) ... ok +test_blocked_action_is_denied_even_if_source_outcome_is_success (test_agt_audit.AgtAuditAdapterTests.test_blocked_action_is_denied_even_if_source_outcome_is_success) ... ok +test_policy_audit_maps_without_free_form_data_or_resource (test_agt_audit.AgtAuditAdapterTests.test_policy_audit_maps_without_free_form_data_or_resource) ... ok +test_action_payload_field_is_rejected_by_schema_and_privacy_gate (test_conformance.ConformanceTests.test_action_payload_field_is_rejected_by_schema_and_privacy_gate) ... ok +test_all_schemas_are_valid_draft_2020_12 (test_conformance.ConformanceTests.test_all_schemas_are_valid_draft_2020_12) ... ok +test_committed_suite_has_expected_verdicts (test_conformance.ConformanceTests.test_committed_suite_has_expected_verdicts) ... ok +test_nested_prohibited_key_is_rejected_independently (test_conformance.ConformanceTests.test_nested_prohibited_key_is_rejected_independently) ... ok +test_unknown_event_type_is_rejected (test_conformance.ConformanceTests.test_unknown_event_type_is_rejected) ... ok +test_uppercase_trace_id_is_rejected (test_conformance.ConformanceTests.test_uppercase_trace_id_is_rejected) ... ok +test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason (test_data_flow.DataFlowTests.test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason) ... ok +test_classifier_cannot_return_arbitrary_metadata (test_data_flow.DataFlowTests.test_classifier_cannot_return_arbitrary_metadata) ... ok +test_classifier_sees_content_but_event_contains_only_closed_metadata (test_data_flow.DataFlowTests.test_classifier_sees_content_but_event_contains_only_closed_metadata) ... ok +test_metadata_identifiers_reject_prose_paths_and_query_values (test_data_flow.DataFlowTests.test_metadata_identifiers_reject_prose_paths_and_query_values) ... ok +test_callback_and_returned_entry_cannot_mutate_retained_evidence (test_evidence.EvidenceTests.test_callback_and_returned_entry_cannot_mutate_retained_evidence) ... ok +test_chain_is_ordered_and_has_stable_golden_digest (test_evidence.EvidenceTests.test_chain_is_ordered_and_has_stable_golden_digest) ... ok +test_client_does_not_project_when_evidence_fails (test_evidence.EvidenceTests.test_client_does_not_project_when_evidence_fails) ... ok +test_client_isolates_operational_event_from_evidence_sink (test_evidence.EvidenceTests.test_client_isolates_operational_event_from_evidence_sink) ... ok +test_client_reports_evidence_persistence (test_evidence.EvidenceTests.test_client_reports_evidence_persistence) ... ok +test_context_mismatch_never_enters_evidence (test_evidence.EvidenceTests.test_context_mismatch_never_enters_evidence) ... ok +test_durable_callback_acknowledges_before_local_acceptance (test_evidence.EvidenceTests.test_durable_callback_acknowledges_before_local_acceptance) ... ok +test_failed_durable_ack_is_fail_closed_and_retryable (test_evidence.EvidenceTests.test_failed_durable_ack_is_fail_closed_and_retryable) ... ok +test_non_finite_number_cannot_enter_digest_chain (test_evidence.EvidenceTests.test_non_finite_number_cannot_enter_digest_chain) ... ok +test_rejects_cross_run_duplicate_overflow_and_append_after_seal (test_evidence.EvidenceTests.test_rejects_cross_run_duplicate_overflow_and_append_after_seal) ... ok +test_snapshot_is_defensive_and_unsealed_never_claims_complete (test_evidence.EvidenceTests.test_snapshot_is_defensive_and_unsealed_never_claims_complete) ... ok +test_action_metrics_exclude_action_and_agent_identifiers (test_otel_projectors.OTelProjectorTests.test_action_metrics_exclude_action_and_agent_identifiers) ... ok +test_classification_allowlist_is_bounded (test_otel_projectors.OTelProjectorTests.test_classification_allowlist_is_bounded) ... ok +test_classification_metric_is_allowlisted_or_collapsed (test_otel_projectors.OTelProjectorTests.test_classification_metric_is_allowlisted_or_collapsed) ... ok +test_metric_failure_is_reported_independently (test_otel_projectors.OTelProjectorTests.test_metric_failure_is_reported_independently) ... ok +test_real_metrics_record_usage_without_high_cardinality_ids (test_otel_projectors.OTelProjectorTests.test_real_metrics_record_usage_without_high_cardinality_ids) ... ok +test_real_otel_log_preserves_event_and_active_trace_context (test_otel_projectors.OTelProjectorTests.test_real_otel_log_preserves_event_and_active_trace_context) ... ok +test_async_handoff_creates_link_without_forcing_parentage (test_propagation.PropagationTests.test_async_handoff_creates_link_without_forcing_parentage) ... ok +test_header_lookup_is_case_insensitive (test_propagation.PropagationTests.test_header_lookup_is_case_insensitive) ... ok +test_injection_rejects_header_injection (test_propagation.PropagationTests.test_injection_rejects_header_injection) ... ok +test_link_requires_valid_w3c_context (test_propagation.PropagationTests.test_link_requires_valid_w3c_context) ... ok +test_sync_handoff_preserves_trace_and_agent_parent (test_propagation.PropagationTests.test_sync_handoff_preserves_trace_and_agent_parent) ... ok +test_complete_governed_workflow (test_reference_scenario.ReferenceScenarioTests.test_complete_governed_workflow) ... ok +test_contract_version_rejects_unsupported_spellings (test_repository_gates.RepositoryGateTests.test_contract_version_rejects_unsupported_spellings) ... ok +test_contract_versions_map_to_ecosystem_spellings (test_repository_gates.RepositoryGateTests.test_contract_versions_map_to_ecosystem_spellings) ... ok +test_declared_versions_are_consistent (test_repository_gates.RepositoryGateTests.test_declared_versions_are_consistent) ... ok +test_issue_templates_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_issue_templates_are_valid_yaml_when_parser_available) ... ok +test_npm_dist_tag_matches_the_declared_contract (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_matches_the_declared_contract) ... ok +test_npm_dist_tag_never_moves_latest_for_a_prerelease (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_never_moves_latest_for_a_prerelease) ... ok +test_npm_dist_tag_rejects_an_unsupported_version (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_rejects_an_unsupported_version) ... ok +test_otel_matrix_gate_detects_attribute_metric_and_family_drift (test_repository_gates.RepositoryGateTests.test_otel_matrix_gate_detects_attribute_metric_and_family_drift) ... ok +test_otel_matrix_matches_shipped_projection (test_repository_gates.RepositoryGateTests.test_otel_matrix_matches_shipped_projection) ... ok +test_release_tag_must_match_contract (test_repository_gates.RepositoryGateTests.test_release_tag_must_match_contract) ... ok +test_schema_gate_detects_changed_bytes (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_changed_bytes) ... ok +test_schema_gate_detects_missing_and_extra_files (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_missing_and_extra_files) ... ok +test_source_manifest_includes_otel_matrix (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_otel_matrix) ... ok +test_source_manifest_includes_release_policy (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_release_policy) ... ok +test_typescript_schemas_match_normative_bytes (test_repository_gates.RepositoryGateTests.test_typescript_schemas_match_normative_bytes) ... ok +test_workflows_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_workflows_are_valid_yaml_when_parser_available) ... ok +test_bundled_schemas_match_normative_schemas (test_sdk.SdkTests.test_bundled_schemas_match_normative_schemas) ... ok +test_context_mismatch_rejects_before_projection (test_sdk.SdkTests.test_context_mismatch_rejects_before_projection) ... ok +test_emits_correlated_span_event_and_log (test_sdk.SdkTests.test_emits_correlated_span_event_and_log) ... ok +test_explicit_attribute_allowlist_is_narrow (test_sdk.SdkTests.test_explicit_attribute_allowlist_is_narrow) ... ok +test_extension_attributes_are_deny_by_default (test_sdk.SdkTests.test_extension_attributes_are_deny_by_default) ... ok +test_log_body_is_a_deep_copy (test_sdk.SdkTests.test_log_body_is_a_deep_copy) ... ok +test_no_span_still_emits_run_correlated_log (test_sdk.SdkTests.test_no_span_still_emits_run_correlated_log) ... ok +test_privacy_violation_rejects_before_projection (test_sdk.SdkTests.test_privacy_violation_rejects_before_projection) ... ok +test_projection_failures_are_reported_independently (test_sdk.SdkTests.test_projection_failures_are_reported_independently) ... ok +test_real_otel_span_receives_event (test_sdk.SdkTests.test_real_otel_span_receives_event) ... ok +test_action_events_create_truthful_tool_transcript (test_trace_adapter.TraceAdapterTests.test_action_events_create_truthful_tool_transcript) ... ok +test_finalizes_signs_and_validates_official_trace_record (test_trace_adapter.TraceAdapterTests.test_finalizes_signs_and_validates_official_trace_record) ... ok +test_linked_terminal_approval_resolves_policy_challenge (test_trace_adapter.TraceAdapterTests.test_linked_terminal_approval_resolves_policy_challenge) ... ok +test_refuses_conflicting_policy_bundles (test_trace_adapter.TraceAdapterTests.test_refuses_conflicting_policy_bundles) ... ok +test_refuses_missing_signer (test_trace_adapter.TraceAdapterTests.test_refuses_missing_signer) ... ok +test_refuses_policy_digest_algorithm_trace_cannot_represent (test_trace_adapter.TraceAdapterTests.test_refuses_policy_digest_algorithm_trace_cannot_represent) ... ok +test_refuses_unranked_classification (test_trace_adapter.TraceAdapterTests.test_refuses_unranked_classification) ... ok +test_refuses_unsealed_or_incomplete_evidence (test_trace_adapter.TraceAdapterTests.test_refuses_unsealed_or_incomplete_evidence) ... ok +test_unbound_or_expired_approval_cannot_resolve_policy_challenge (test_trace_adapter.TraceAdapterTests.test_unbound_or_expired_approval_cannot_resolve_policy_challenge) ... ok +test_a_failure_inside_official_signing_is_reported_as_one (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_failure_inside_official_signing_is_reported_as_one) +A key that satisfies the shape check and then fails when used. ... ok +test_a_policy_decision_without_a_bundle_digest_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_policy_decision_without_a_bundle_digest_is_refused) ... ok +test_a_repeated_classification_cannot_rank_anything (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_repeated_classification_cannot_rank_anything) ... ok +test_a_run_with_no_data_flow_has_nothing_to_classify (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_data_flow_has_nothing_to_classify) ... ok +test_a_run_with_no_policy_decision_has_no_policy_to_bind (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_policy_decision_has_no_policy_to_bind) ... ok +test_a_taxonomy_the_configuration_does_not_name_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_taxonomy_the_configuration_does_not_name_is_refused) ... ok +test_an_empty_run_carries_no_chained_evidence (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_empty_run_carries_no_chained_evidence) ... ok +test_every_required_configuration_field_is_named_when_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_every_required_configuration_field_is_named_when_absent) ... ok +test_the_optional_dependency_is_named_when_it_is_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_the_optional_dependency_is_named_when_it_is_absent) ... ok +test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode) ... ok +test_accumulator_rejects_non_finite_external_cost (test_usage.UsageTests.test_accumulator_rejects_non_finite_external_cost) ... ok +test_bool_is_not_accepted_as_token_integer (test_usage.UsageTests.test_bool_is_not_accepted_as_token_integer) ... ok +test_cost_requires_finite_amount_and_explicit_provenance (test_usage.UsageTests.test_cost_requires_finite_amount_and_explicit_provenance) ... ok +test_reused_event_id_with_changed_data_is_rejected (test_usage.UsageTests.test_reused_event_id_with_changed_data_is_rejected) ... ok +test_rollup_deduplicates_and_reports_partial_coverage (test_usage.UsageTests.test_rollup_deduplicates_and_reports_partial_coverage) ... ok +test_rollup_rejects_non_leaf_and_mixed_currency (test_usage.UsageTests.test_rollup_rejects_non_leaf_and_mixed_currency) ... ok +test_rollups_match_shared_cross_language_vector (test_usage.UsageTests.test_rollups_match_shared_cross_language_vector) ... ok +test_schema_rejects_aggregate_cost_without_run_rollup (test_usage.UsageTests.test_schema_rejects_aggregate_cost_without_run_rollup) ... ok +test_unknown_price_is_absent_not_zero (test_usage.UsageTests.test_unknown_price_is_absent_not_zero) ... ok + +---------------------------------------------------------------------- +Ran 111 tests in 7.340s + +OK +PASS conformance\fixtures\valid\action.json +PASS conformance\fixtures\valid\approval.json +PASS conformance\fixtures\valid\data-flow.json +PASS conformance\fixtures\valid\evidence.json +PASS conformance\fixtures\valid\policy-decision.json +PASS conformance\fixtures\valid\usage.json +PASS conformance\fixtures\invalid\action-without-digest.json +PASS conformance\fixtures\invalid\aggregate-cost-without-rollup.json +PASS conformance\fixtures\invalid\model-call-with-rollup.json +PASS conformance\fixtures\invalid\numeric-timestamp.json +PASS conformance\fixtures\invalid\raw-content.json +PASS conformance\fixtures\invalid\usage-without-measurement.json +PASS conformance\fixtures\invalid\zero-trace-id.json +PASS contract=0.1.0-alpha.2 python=0.1.0a2 npm_name=@agentrust-io/telemetry npm=0.1.0-alpha.2 +PASS OTel GenAI matrix covers 16 span attributes, 8 metrics, and 6 event families +PASS TypeScript schemas match normative bytes diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/regression-test-at-pin-typescript.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/regression-test-at-pin-typescript.txt new file mode 100644 index 0000000..34cc93d --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/regression-test-at-pin-typescript.txt @@ -0,0 +1,34 @@ +✔ finalizer derives claims then signs, validates, and self-verifies (14.7155ms) +✔ tool transcript uses the shared RFC 8785 sequence/event digest (4.1443ms) +✔ linked approval resolves a challenge while an unrelated approval does not (6.3127ms) +✔ finalization refuses incomplete evidence, conflicts, unranked data, and missing trust inputs (3.9279ms) +✔ codec validation or self-verification failure is fail-closed (1.1009ms) +✖ finalizer refuses a snapshot whose entries no longer hash to its chain digest (2.1761ms) +ℹ tests 6 +ℹ suites 0 +ℹ pass 5 +ℹ fail 1 +ℹ cancelled 0 +ℹ skipped 0 +ℹ todo 0 +ℹ duration_ms 824.5197 + +✖ failing tests: + +test at test\trace-finalizer.test.ts:56:1 +✖ finalizer refuses a snapshot whose entries no longer hash to its chain digest (2.1761ms) + AssertionError [ERR_ASSERTION]: Missing expected exception. + at TestContext. (\ts-pin\packages\typescript\test\trace-finalizer.test.ts:60:10) + at Test.runInAsyncScope (node:async_hooks:227:14) + at Test.run (node:internal/test_runner/test:1325:25) + at Test.processPendingSubtests (node:internal/test_runner/test:911:18) + at Test.postRun (node:internal/test_runner/test:1465:19) + at Test.run (node:internal/test_runner/test:1390:12) + at async Test.processPendingSubtests (node:internal/test_runner/test:911:7) { + generatedMessage: false, + code: 'ERR_ASSERTION', + actual: undefined, + expected: /evidence chain does not verify/, + operator: 'throws', + diff: 'simple' + } diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/regression-tests-at-pin-python.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/regression-tests-at-pin-python.txt new file mode 100644 index 0000000..8b7c24a --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/regression-tests-at-pin-python.txt @@ -0,0 +1,65 @@ +test_a_consistent_chain_over_a_malformed_event_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_consistent_chain_over_a_malformed_event_is_refused) +Digests recomputed over an event the schema never accepted. ... ERROR +test_a_dropped_entry_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_dropped_entry_is_refused) ... FAIL +test_a_failure_inside_official_signing_is_reported_as_one (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_failure_inside_official_signing_is_reported_as_one) +A key that satisfies the type check and then fails when used. ... ok +test_a_policy_decision_without_a_bundle_digest_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_policy_decision_without_a_bundle_digest_is_refused) ... ok +test_a_repeated_classification_cannot_rank_anything (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_repeated_classification_cannot_rank_anything) ... ok +test_a_run_with_no_data_flow_has_nothing_to_classify (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_data_flow_has_nothing_to_classify) ... ok +test_a_run_with_no_policy_decision_has_no_policy_to_bind (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_policy_decision_has_no_policy_to_bind) ... ok +test_a_substituted_chain_digest_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_substituted_chain_digest_is_refused) ... FAIL +test_a_taxonomy_the_configuration_does_not_name_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_taxonomy_the_configuration_does_not_name_is_refused) ... ok +test_an_empty_run_carries_no_chained_evidence (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_empty_run_carries_no_chained_evidence) ... ok +test_an_event_edited_after_sealing_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_event_edited_after_sealing_is_refused) ... FAIL +test_every_required_configuration_field_is_named_when_absent (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_every_required_configuration_field_is_named_when_absent) ... ok +test_the_optional_dependency_is_named_when_it_is_absent (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_the_optional_dependency_is_named_when_it_is_absent) ... ok +test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode) ... ok + +====================================================================== +ERROR: test_a_consistent_chain_over_a_malformed_event_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_consistent_chain_over_a_malformed_event_is_refused) +Digests recomputed over an event the schema never accepted. +---------------------------------------------------------------------- +Traceback (most recent call last): + File "\src-pin-reg\tests\test_trace_adapter_refusals.py", line 277, in test_a_consistent_chain_over_a_malformed_event_is_refused + finalize_trace(snapshot, self.config, signing_key=self.key) + File "\src-pin-reg\src\agentrust_telemetry\trace_adapter.py", line 47, in finalize_trace + bundle_hash, enforcement_mode = _policy_binding(events) + ^^^^^^^^^^^^^^^^^^^^^^^ + File "\src-pin-reg\src\agentrust_telemetry\trace_adapter.py", line 136, in _policy_binding + if any("bundle_digest" not in event["policy"] for event in decisions): + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "\src-pin-reg\src\agentrust_telemetry\trace_adapter.py", line 136, in + if any("bundle_digest" not in event["policy"] for event in decisions): + ~~~~~^^^^^^^^^^ +KeyError: 'policy' + +====================================================================== +FAIL: test_a_dropped_entry_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_dropped_entry_is_refused) +---------------------------------------------------------------------- +Traceback (most recent call last): + File "\src-pin-reg\tests\test_trace_adapter_refusals.py", line 237, in test_a_dropped_entry_is_refused + self.assertIn("evidence chain does not verify", str(caught.exception)) +AssertionError: 'evidence chain does not verify' not found in 'no policy decision evidence is present' + +====================================================================== +FAIL: test_a_substituted_chain_digest_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_substituted_chain_digest_is_refused) +---------------------------------------------------------------------- +Traceback (most recent call last): + File "\src-pin-reg\tests\test_trace_adapter_refusals.py", line 242, in test_a_substituted_chain_digest_is_refused + with self.assertRaises(TraceFinalizationError) as caught: + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +AssertionError: TraceFinalizationError not raised + +====================================================================== +FAIL: test_an_event_edited_after_sealing_is_refused (tests.test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_event_edited_after_sealing_is_refused) +---------------------------------------------------------------------- +Traceback (most recent call last): + File "\src-pin-reg\tests\test_trace_adapter_refusals.py", line 228, in test_an_event_edited_after_sealing_is_refused + with self.assertRaises(TraceFinalizationError) as caught: + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +AssertionError: TraceFinalizationError not raised + +---------------------------------------------------------------------- +Ran 14 tests in 0.571s + +FAILED (failures=3, errors=1) diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/telemetry_probes.py b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/telemetry_probes.py new file mode 100644 index 0000000..98c4de7 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/telemetry_probes.py @@ -0,0 +1,47 @@ +"""Paper 1 (agentrust-telemetry v1) boundary probes P1 to P4, reconstructed from Section 6. + +Usage: python telemetry_probes.py SOURCE_DIR (an agentrust-telemetry tree) +Prints one line per probe; records observations, does not assert. +""" +import json, sys +from pathlib import Path +SRC = Path(sys.argv[1]) +sys.path.insert(0, str(SRC / 'src')) +from agentrust_telemetry import EvidenceAccumulator, SchemaValidator, TelemetryClient, EvidencePersistenceError + +v = SchemaValidator(SRC / 'spec/schema') +event = json.loads((SRC / 'conformance/fixtures/valid/policy-decision.json').read_text()) + +class Log: + def __init__(self, fail=False): self.calls = 0; self.fail = fail + def emit(self, rec): + self.calls += 1 + if self.fail: raise RuntimeError('exporter down') + +# P1: durable callback refuses (returns False) +a = EvidenceAccumulator(event['run_id'], v, durable_append=lambda e: False) +log = Log(); raised = False +try: TelemetryClient(v, span_resolver=lambda: None, evidence_sink=a, log_emitter=log).emit(event) +except EvidencePersistenceError: raised = True +print(f'P1 refused ack: EvidencePersistenceError raised={raised}; log projection calls={log.calls}; accepted entries={len(a.snapshot().entries)}') + +# P2: log exporter raises after acceptance +a = EvidenceAccumulator(event['run_id'], v) +r = TelemetryClient(v, span_resolver=lambda: None, evidence_sink=a, log_emitter=Log(True)).emit(event) +print(f'P2 failing exporter: entries={len(a.snapshot().entries)}; projection_errors={list(r.projection_errors)}') + +# P3: empty accumulator sealed as complete +s = EvidenceAccumulator('empty-run', v).seal(completeness='complete') +print(f'P3 empty seal: entries={len(s.entries)} completeness={s.completeness}') + +# P4: callback commits externally, then raises once; caller retries +external = []; state = {'n': 0} +def flaky(entry): + external.append(entry.event_id); state['n'] += 1 + if state['n'] == 1: raise RuntimeError('ack lost after external commit') + return True +a = EvidenceAccumulator(event['run_id'], v, durable_append=flaky) +for _ in range(2): + try: a.append(event); break + except EvidencePersistenceError: pass +print(f'P4 lost ack then retry: external writes={len(external)}; local entries={len(a.snapshot().entries)}') diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/typescript-check-current-main.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/typescript-check-current-main.txt new file mode 100644 index 0000000..26b4916 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/typescript-check-current-main.txt @@ -0,0 +1,151 @@ + +> @agentrust-io/telemetry@0.1.0-alpha.5 check +> npm run build && npm test && npm pack --dry-run + + +> @agentrust-io/telemetry@0.1.0-alpha.5 build +> tsc -p tsconfig.json + + +> @agentrust-io/telemetry@0.1.0-alpha.5 test +> tsx --test test/*.test.ts + +✔ OPA boolean decision logs preserve safe facts and match Python identifiers (12.8381ms) +✔ OPA rejects ambiguous results, malformed collections, and normalized invalid dates (2.1347ms) +✔ Cedar sorts stable codes, retains final decision, and rejects prose errors (1.1367ms) +✔ AGT policy mapping excludes free-form content and sink prevalidates a whole batch (5.4747ms) +✔ AGT sink reports projection failure and identifiers cannot carry prose (0.4324ms) +✔ approval chain preserves Python-compatible policy, request, and resolution links (7.4834ms) +✔ request mapping rejects every policy binding mismatch (2.4333ms) +✔ resolution mapping rejects request mismatch, chronology reversal, and malformed evidence (1.06ms) +✔ only terminal outcomes map to terminal approval events (2.2161ms) +✔ audit policy mapping matches Python ID and excludes free-form and partial hashes (10.8772ms) +✔ audit action requires caller full digest and computes safe duration (7.474ms) +✔ audit action rejects absent, reversed, and unsafe timing (2.4852ms) +✔ classifier sees content while the event contains only closed metadata (3.3033ms) +✔ classification boundary rejects arbitrary results and prose identifiers (0.5564ms) +✔ AGT access maps only tier, decision, agent, and time (2.1632ms) +✔ evidence chain matches the shared Python and TypeScript golden vector (145.7215ms) +✔ durable acknowledgement is fail-closed, retryable, and copy-isolated (252.169ms) +✔ cross-run, duplicate, overflow, invalid number, and sealed mutations are rejected (2.6848ms) +✔ reentrant durable callbacks cannot corrupt sequence state (3.8419ms) +✔ client does not project an event when evidence persistence fails (143.2795ms) +✔ usage metrics record totals without high-cardinality identity dimensions (6.5471ms) +✔ classification dimensions are bounded by an explicit allowlist (1.819ms) +✔ policy, approval, and action metrics expose only bounded semantic dimensions (2.2672ms) +✔ W3C and durable AgentTrust context round-trip (5.5184ms) +✔ headers are case-insensitive and reject line breaks (0.5124ms) +✔ valid and invalid shared conformance fixtures have identical verdict classes (12.3895ms) +✔ event construction matches the cross-language golden bytes semantically (6.8337ms) +✔ uint64-scale nanoseconds survive construction exactly (1.231ms) +✔ projection matches Python allowlist and does not flatten nested content (0.4337ms) +✔ client validates context before all caller-owned projections (0.6983ms) +✔ invalid all-zero OTel context is treated as absent (0.4114ms) +✔ log body and evidence input are defensive copies (0.5304ms) +✔ finalizer derives claims then signs, validates, and self-verifies (14.42ms) +✔ tool transcript uses the shared RFC 8785 sequence/event digest (19.8728ms) +✔ linked approval resolves a challenge while an unrelated approval does not (100.5955ms) +✔ finalization refuses incomplete evidence, conflicts, unranked data, and missing trust inputs (13.5107ms) +✔ codec validation or self-verification failure is fail-closed (0.9121ms) +✔ finalizer refuses a snapshot whose entries no longer hash to its chain digest (35.9274ms) +✔ unknown cost stays absent and observations validate provenance (11.7523ms) +✔ rollups deduplicate leaves, preserve partial coverage, and add costs decimally (6.2445ms) +✔ rollups reject changed duplicate IDs, non-leaves, mixed currency, and unsafe totals (2.2886ms) +✔ agent and workflow rollups match the shared Python and TypeScript vector (2.037ms) +ℹ tests 42 +ℹ suites 0 +ℹ pass 42 +ℹ fail 0 +ℹ cancelled 0 +ℹ skipped 0 +ℹ todo 0 +ℹ duration_ms 13496.0361 +npm notice +npm notice package: @agentrust-io/telemetry@0.1.0-alpha.5 +npm notice Tarball Contents +npm notice 1.1kB LICENSE +npm notice 1.0kB README.md +npm notice 2.2kB dist/adapters.d.ts +npm notice 2.2kB dist/adapters.d.ts.map +npm notice 9.3kB dist/adapters.js +npm notice 10.3kB dist/adapters.js.map +npm notice 1.0kB dist/agt-approval.d.ts +npm notice 1.1kB dist/agt-approval.d.ts.map +npm notice 6.8kB dist/agt-approval.js +npm notice 6.9kB dist/agt-approval.js.map +npm notice 760B dist/agt-audit.d.ts +npm notice 816B dist/agt-audit.d.ts.map +npm notice 5.6kB dist/agt-audit.js +npm notice 6.3kB dist/agt-audit.js.map +npm notice 1.0kB dist/client.d.ts +npm notice 930B dist/client.d.ts.map +npm notice 2.6kB dist/client.js +npm notice 2.6kB dist/client.js.map +npm notice 1.8kB dist/data-flow.d.ts +npm notice 1.8kB dist/data-flow.d.ts.map +npm notice 4.8kB dist/data-flow.js +npm notice 4.9kB dist/data-flow.js.map +npm notice 219B dist/evidence-digest.d.ts +npm notice 272B dist/evidence-digest.d.ts.map +npm notice 918B dist/evidence-digest.js +npm notice 897B dist/evidence-digest.js.map +npm notice 1.4kB dist/evidence.d.ts +npm notice 1.2kB dist/evidence.d.ts.map +npm notice 4.1kB dist/evidence.js +npm notice 3.7kB dist/evidence.js.map +npm notice 921B dist/factory.d.ts +npm notice 963B dist/factory.d.ts.map +npm notice 2.4kB dist/factory.js +npm notice 2.5kB dist/factory.js.map +npm notice 484B dist/index.d.ts +npm notice 412B dist/index.d.ts.map +npm notice 449B dist/index.js +npm notice 387B dist/index.js.map +npm notice 336B dist/metrics.d.ts +npm notice 357B dist/metrics.d.ts.map +npm notice 4.1kB dist/metrics.js +npm notice 3.9kB dist/metrics.js.map +npm notice 444B dist/projection.d.ts +npm notice 511B dist/projection.d.ts.map +npm notice 1.4kB dist/projection.js +npm notice 1.3kB dist/projection.js.map +npm notice 1.1kB dist/propagation.d.ts +npm notice 1.0kB dist/propagation.d.ts.map +npm notice 2.6kB dist/propagation.js +npm notice 3.0kB dist/propagation.js.map +npm notice 1.3kB dist/trace-finalizer.d.ts +npm notice 1.3kB dist/trace-finalizer.d.ts.map +npm notice 10.7kB dist/trace-finalizer.js +npm notice 11.1kB dist/trace-finalizer.js.map +npm notice 1.1kB dist/types.d.ts +npm notice 1.2kB dist/types.d.ts.map +npm notice 44B dist/types.js +npm notice 102B dist/types.js.map +npm notice 1.7kB dist/usage.d.ts +npm notice 1.5kB dist/usage.d.ts.map +npm notice 5.8kB dist/usage.js +npm notice 6.1kB dist/usage.js.map +npm notice 479B dist/validation.d.ts +npm notice 471B dist/validation.d.ts.map +npm notice 4.2kB dist/validation.js +npm notice 4.2kB dist/validation.js.map +npm notice 1.3kB package.json +npm notice 1.7kB schemas/action.schema.json +npm notice 1.6kB schemas/approval.schema.json +npm notice 1.4kB schemas/common.schema.json +npm notice 1.9kB schemas/data-flow.schema.json +npm notice 1.2kB schemas/envelope.schema.json +npm notice 900B schemas/evidence.schema.json +npm notice 1.6kB schemas/policy-decision.schema.json +npm notice 3.4kB schemas/usage.schema.json +npm notice Tarball Details +npm notice name: @agentrust-io/telemetry +npm notice version: 0.1.0-alpha.5 +npm notice filename: agentrust-io-telemetry-0.1.0-alpha.5.tgz +npm notice package size: 37.9 kB +npm notice unpacked size: 183.7 kB +npm notice shasum: 5235c6a7ba8da9be08e5d44870ef0809ea3e7dc7 +npm notice integrity: sha512-CRB1SwcOTiuDB[...]Eg5imHfi2tR9Q== +npm notice total files: 75 +npm notice +agentrust-io-telemetry-0.1.0-alpha.5.tgz diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/typescript-check-pinned.txt b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/typescript-check-pinned.txt new file mode 100644 index 0000000..7a66994 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/typescript-check-pinned.txt @@ -0,0 +1,146 @@ + +> @agentrust-io/telemetry@0.1.0-alpha.2 check +> npm run build && npm test && npm pack --dry-run + + +> @agentrust-io/telemetry@0.1.0-alpha.2 build +> tsc -p tsconfig.json + + +> @agentrust-io/telemetry@0.1.0-alpha.2 test +> tsx --test test/*.test.ts + +✔ OPA boolean decision logs preserve safe facts and match Python identifiers (9.4232ms) +✔ OPA rejects ambiguous results, malformed collections, and normalized invalid dates (2.1862ms) +✔ Cedar sorts stable codes, retains final decision, and rejects prose errors (0.6554ms) +✔ AGT policy mapping excludes free-form content and sink prevalidates a whole batch (0.8977ms) +✔ AGT sink reports projection failure and identifiers cannot carry prose (0.539ms) +✔ approval chain preserves Python-compatible policy, request, and resolution links (6.9146ms) +✔ request mapping rejects every policy binding mismatch (0.6131ms) +✔ resolution mapping rejects request mismatch, chronology reversal, and malformed evidence (0.733ms) +✔ only terminal outcomes map to terminal approval events (1.1321ms) +✔ audit policy mapping matches Python ID and excludes free-form and partial hashes (18.8166ms) +✔ audit action requires caller full digest and computes safe duration (3.9528ms) +✔ audit action rejects absent, reversed, and unsafe timing (0.5037ms) +✔ classifier sees content while the event contains only closed metadata (3.2515ms) +✔ classification boundary rejects arbitrary results and prose identifiers (1.6484ms) +✔ AGT access maps only tier, decision, agent, and time (1.8071ms) +✔ evidence chain matches the shared Python and TypeScript golden vector (10.4962ms) +✔ durable acknowledgement is fail-closed, retryable, and copy-isolated (10.9122ms) +✔ cross-run, duplicate, overflow, invalid number, and sealed mutations are rejected (4.1169ms) +✔ reentrant durable callbacks cannot corrupt sequence state (0.9573ms) +✔ client does not project an event when evidence persistence fails (13.2601ms) +✔ usage metrics record totals without high-cardinality identity dimensions (4.5101ms) +✔ classification dimensions are bounded by an explicit allowlist (1.1192ms) +✔ policy, approval, and action metrics expose only bounded semantic dimensions (11.0404ms) +✔ W3C and durable AgentTrust context round-trip (3.0052ms) +✔ headers are case-insensitive and reject line breaks (0.3918ms) +✔ valid and invalid shared conformance fixtures have identical verdict classes (8.3736ms) +✔ event construction matches the cross-language golden bytes semantically (1.826ms) +✔ uint64-scale nanoseconds survive construction exactly (0.6991ms) +✔ projection matches Python allowlist and does not flatten nested content (0.2767ms) +✔ client validates context before all caller-owned projections (0.5157ms) +✔ invalid all-zero OTel context is treated as absent (0.2903ms) +✔ log body and evidence input are defensive copies (0.4044ms) +✔ finalizer derives claims then signs, validates, and self-verifies (6.4703ms) +✔ tool transcript uses the shared RFC 8785 sequence/event digest (3.5356ms) +✔ linked approval resolves a challenge while an unrelated approval does not (5.8819ms) +✔ finalization refuses incomplete evidence, conflicts, unranked data, and missing trust inputs (3.3329ms) +✔ codec validation or self-verification failure is fail-closed (1.1157ms) +✔ unknown cost stays absent and observations validate provenance (6.8512ms) +✔ rollups deduplicate leaves, preserve partial coverage, and add costs decimally (2.5112ms) +✔ rollups reject changed duplicate IDs, non-leaves, mixed currency, and unsafe totals (6.9783ms) +✔ agent and workflow rollups match the shared Python and TypeScript vector (4.4286ms) +ℹ tests 41 +ℹ suites 0 +ℹ pass 41 +ℹ fail 0 +ℹ cancelled 0 +ℹ skipped 0 +ℹ todo 0 +ℹ duration_ms 2916.3694 +npm notice +npm notice package: @agentrust-io/telemetry@0.1.0-alpha.2 +npm notice Tarball Contents +npm notice 1.1kB LICENSE +npm notice 1.0kB README.md +npm notice 2.2kB dist/adapters.d.ts +npm notice 2.2kB dist/adapters.d.ts.map +npm notice 9.3kB dist/adapters.js +npm notice 10.3kB dist/adapters.js.map +npm notice 1.0kB dist/agt-approval.d.ts +npm notice 1.1kB dist/agt-approval.d.ts.map +npm notice 6.8kB dist/agt-approval.js +npm notice 6.9kB dist/agt-approval.js.map +npm notice 760B dist/agt-audit.d.ts +npm notice 816B dist/agt-audit.d.ts.map +npm notice 5.6kB dist/agt-audit.js +npm notice 6.3kB dist/agt-audit.js.map +npm notice 1.0kB dist/client.d.ts +npm notice 930B dist/client.d.ts.map +npm notice 2.6kB dist/client.js +npm notice 2.6kB dist/client.js.map +npm notice 1.8kB dist/data-flow.d.ts +npm notice 1.8kB dist/data-flow.d.ts.map +npm notice 4.8kB dist/data-flow.js +npm notice 4.9kB dist/data-flow.js.map +npm notice 1.4kB dist/evidence.d.ts +npm notice 1.2kB dist/evidence.d.ts.map +npm notice 4.7kB dist/evidence.js +npm notice 4.4kB dist/evidence.js.map +npm notice 921B dist/factory.d.ts +npm notice 963B dist/factory.d.ts.map +npm notice 2.4kB dist/factory.js +npm notice 2.5kB dist/factory.js.map +npm notice 484B dist/index.d.ts +npm notice 412B dist/index.d.ts.map +npm notice 449B dist/index.js +npm notice 387B dist/index.js.map +npm notice 336B dist/metrics.d.ts +npm notice 357B dist/metrics.d.ts.map +npm notice 4.1kB dist/metrics.js +npm notice 3.9kB dist/metrics.js.map +npm notice 444B dist/projection.d.ts +npm notice 511B dist/projection.d.ts.map +npm notice 1.4kB dist/projection.js +npm notice 1.3kB dist/projection.js.map +npm notice 1.1kB dist/propagation.d.ts +npm notice 1.0kB dist/propagation.d.ts.map +npm notice 2.6kB dist/propagation.js +npm notice 3.0kB dist/propagation.js.map +npm notice 1.3kB dist/trace-finalizer.d.ts +npm notice 1.3kB dist/trace-finalizer.d.ts.map +npm notice 8.8kB dist/trace-finalizer.js +npm notice 9.7kB dist/trace-finalizer.js.map +npm notice 1.1kB dist/types.d.ts +npm notice 1.2kB dist/types.d.ts.map +npm notice 44B dist/types.js +npm notice 102B dist/types.js.map +npm notice 1.7kB dist/usage.d.ts +npm notice 1.5kB dist/usage.d.ts.map +npm notice 5.8kB dist/usage.js +npm notice 6.1kB dist/usage.js.map +npm notice 479B dist/validation.d.ts +npm notice 471B dist/validation.d.ts.map +npm notice 4.2kB dist/validation.js +npm notice 4.2kB dist/validation.js.map +npm notice 1.3kB package.json +npm notice 1.7kB schemas/action.schema.json +npm notice 1.6kB schemas/approval.schema.json +npm notice 1.4kB schemas/common.schema.json +npm notice 1.9kB schemas/data-flow.schema.json +npm notice 1.2kB schemas/envelope.schema.json +npm notice 900B schemas/evidence.schema.json +npm notice 1.6kB schemas/policy-decision.schema.json +npm notice 3.4kB schemas/usage.schema.json +npm notice Tarball Details +npm notice name: @agentrust-io/telemetry +npm notice version: 0.1.0-alpha.2 +npm notice filename: agentrust-io-telemetry-0.1.0-alpha.2.tgz +npm notice package size: 36.9 kB +npm notice unpacked size: 179.5 kB +npm notice shasum: 0db73c21520242dba3b80dcca6b3025d49ec8d5b +npm notice integrity: sha512-78qIuL0HkzGL5[...]lFMI5aXiqghuQ== +npm notice total files: 71 +npm notice +agentrust-io-telemetry-0.1.0-alpha.2.tgz diff --git a/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/validation_probes.py b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/validation_probes.py new file mode 100644 index 0000000..ae6f2a7 --- /dev/null +++ b/papers/agentrust-telemetry/experiments/results/rerun-2026-10-01/validation_probes.py @@ -0,0 +1,43 @@ +"""Paper 1 Section 7 claims: validation edge cases and reentrant-callback deadlock. + +Usage: python validation_probes.py SOURCE_DIR (an agentrust-telemetry tree) +Records what each case raised; does not assert. +""" +import json, sys, threading +from copy import deepcopy +from pathlib import Path +SRC = Path(sys.argv[1]) +sys.path.insert(0, str(SRC / 'src')) +from agentrust_telemetry import EvidenceAccumulator, SchemaValidator + +v = SchemaValidator(SRC / 'spec/schema') +base = json.loads((SRC / 'conformance/fixtures/valid/policy-decision.json').read_text()) + +def outcome(fn): + try: + fn(); return 'accepted' + except Exception as exc: + return type(exc).__name__ + +deep = deepcopy(base); node = {}; deep['attributes'] = node +for _ in range(5000): + node['x'] = {}; node = node['x'] +nonstr = deepcopy(base); nonstr.setdefault('attributes', {})[1] = 'v' +va = SchemaValidator(SRC / 'spec/schema', allowed_attribute_keys=frozenset({'k'})) +nan = deepcopy(base); nan['attributes'] = {'k': float('nan')} +inf = deepcopy(base); inf['attributes'] = {'k': float('inf')} +print('deep nesting (5000):', outcome(lambda: v.validate(deep))) +print('non-string key:', outcome(lambda: v.validate(nonstr))) +print('NaN attribute, validate:', outcome(lambda: va.validate(nan))) +print('Infinity attribute, validate:', outcome(lambda: va.validate(inf))) +print('NaN attribute, accumulator append:', outcome(lambda: EvidenceAccumulator(base['run_id'], va).append(nan))) + +holder = {} +def reenter(entry): + holder['acc'].snapshot() + return True +acc = EvidenceAccumulator(base['run_id'], v, durable_append=reenter); holder['acc'] = acc +result = {} +t = threading.Thread(target=lambda: result.setdefault('r', outcome(lambda: acc.append(base))), daemon=True) +t.start(); t.join(5) +print('callback re-entering snapshot():', 'deadlocked (no return in 5 s)' if t.is_alive() else result['r']) diff --git a/papers/agentrust-telemetry/paper.tex b/papers/agentrust-telemetry/paper.tex new file mode 100644 index 0000000..df66998 --- /dev/null +++ b/papers/agentrust-telemetry/paper.tex @@ -0,0 +1,134 @@ +\documentclass[10pt,letterpaper]{article} +\usepackage[margin=1in]{geometry} +\usepackage{amsmath,booktabs,natbib,microtype,parskip,enumitem} +\usepackage{fontspec} +\setmainfont{texgyretermes}[Extension=.otf,UprightFont=*-regular,BoldFont=*-bold,ItalicFont=*-italic,BoldItalicFont=*-bolditalic] +\usepackage[colorlinks=true,linkcolor=blue,citecolor=blue,urlcolor=blue]{hyperref} +\setlength{\parindent}{0pt} +\setlength{\emergencystretch}{3em} + +\title{\textbf{From Agent Observability to Governance Evidence: A Privacy-Constrained Telemetry Contract}} +\author{ + Imran Siddique \\ + OPAQUE Systems \\[4pt] + \normalsize \url{https://github.com/agentrust-io/agentrust-telemetry} +} +\date{October 1, 2026 \\ \small Technical report, version 2} + +\begin{document} +\maketitle + +\begin{abstract} +Agent observability conventions describe model, tool, and agent operations, but governance facts are commonly fragmented across policy engines, approval stores, cost modules, and audit systems. Copying those facts into ordinary traces creates two hazards: sensitive payload capture and the false inference that sampled operational telemetry is complete audit evidence. This technical report describes AgenTrust Telemetry, a backend-neutral contract for six governance event families: policy decisions, approval lifecycles, usage, classified data flows, action execution, and evidence lifecycle. The contract correlates with W3C Trace Context and OpenTelemetry without installing a provider, exporter, or competing tracing model. A metadata-only profile rejects prompts, outputs, source code, tool arguments and results, credentials, and authorization tokens by key. Durable run and action identifiers survive process and asynchronous handoffs; propagated metadata remains untrusted and does not confer identity or authority. An optional accumulator accepts events before lossy export and can be finalized into a separately verifiable TRACE record. Its completeness status is a producer assertion that the accumulator records but does not measure. The evaluated release, 0.1.0-alpha.2, ships Python and TypeScript reference SDKs over shared schemas and a portable conformance set of six valid and seven invalid fixtures. At that commit 111 Python unit tests pass and all 13 fixtures produce their expected verdicts, on September 3, 2026 and again when rerun for this edition. A defect found after the evaluation let an edited evidence snapshot be signed; it is reported beside the results it qualifies. The contribution is not another agent tracing convention; it is a narrow semantic boundary between operational observation and governance evidence whose completeness must be stated rather than inferred. +\end{abstract} + +\paragraph{Scope of this edition.} +This is an author-prepared technical report, not peer reviewed and not a normative specification. Version 2 (October 1, 2026) corrects two errors in version 1 (September 27, 2026): the statement that approval digests prevent an approval from being attached to another action, which the reference finalizer does not check, and a typesetting fault that dropped every bold face from the published PDF; results are unchanged. Version 1 revised a manuscript dated September 3, 2026. The contract it describes is alpha and may change incompatibly; current behavior, normative schemas, and limitations are maintained in the project repository. The Python unit suite, the portable conformance runner, and the repository's cross-runtime contract checks were rerun on September 27, 2026 at the evaluated commit (\texttt{29c13cf}, release 0.1.0-alpha.2). The TypeScript test suite was not rerun. Findings published after the evaluation are reported in Section~\ref{sec:after}. + +\noindent\textbf{Patent Pending.} + +\section{Introduction} + +An agent run can cross model providers, tool servers, queues, approval systems, policy engines, databases, and human decision points. OpenTelemetry (OTel) supplies a broadly adopted model for traces, metrics, logs, events, baggage, and context propagation. Its generative-AI conventions, now maintained in a dedicated repository with development status, describe model, tool, and agent operations \citep{otelgenai}. OpenInference similarly defines AI-observability span conventions \citep{openinference}. Those systems answer operational questions: what ran, how long it took, where it failed, and how calls relate. + +Governance asks different questions. Which policy and enforcement mode produced a decision? Was approval requested before execution, and was the executed action the one approved? Which cost source produced an amount? Did classified data move between named trust zones? Was evidence accumulation complete before finalization? These facts may appear in proprietary logs, but inconsistent names and identifiers prevent portable analysis. + +A second problem is semantic, not syntactic. Telemetry pipelines are routinely sampled, batched, retried, transformed, and dropped. An audit record that claims complete execution evidence cannot be reconstructed from a backend query unless completeness was established before the lossy boundary. AgenTrust Telemetry therefore treats OTel projection and durable evidence as two consumers of a validated event, not as interchangeable representations. + +\section{Requirements and non-goals} + +The contract has five requirements: + +\begin{enumerate}[leftmargin=*] + \item remain independent of agent framework, policy engine, collector, and storage backend; + \item reuse OTel context and fields where their semantics already fit; + \item correlate governance events across processes and asynchronous handoffs; + \item reject raw or secret-bearing content in the metadata-only profile; + \item make evidence completeness an explicit producer claim rather than an inference from observed events. +\end{enumerate} + +The project is not a collector, dashboard, tracing backend, policy evaluator, approval workflow, pricing catalog, or agent auto-instrumentation library. It does not duplicate spans already produced by OTel GenAI or OpenInference instrumentation. An event reports a fact observed by another component; it does not authorize an action merely by existing. + +\section{Event contract} + +Six event families provide a common envelope and family-specific fields. + +\begin{table}[h] +\centering +\begin{tabular}{p{0.22\linewidth}p{0.70\linewidth}} +\toprule +Family & Governance fact \\ +\midrule +Policy decision & allow, deny, challenge, or error; enforcement mode; policy identity; latency \\ +Approval lifecycle & request, decision, expiry, cancellation, and execution outcome bound to an action digest \\ +Usage & input/output units, cost, currency, scope, and provenance of measurement or estimate \\ +Data flow & classified source-to-destination metadata without payload capture \\ +Action execution & resolved tool, MCP, A2A, file, HTTP, or database attempt and outcome \\ +Evidence lifecycle & checkpoint, accumulation state, completeness declaration, and optional TRACE finalization \\ +\bottomrule +\end{tabular} +\caption{Normalized governance event families.} +\end{table} + +Every event carries a durable \texttt{run\_id}, an \texttt{agent\_id}, an event time encoded as a decimal string of nanoseconds so that no runtime loses precision, and when applicable a stable \texttt{action\_id} and canonical action digest. \texttt{trace\_id} and \texttt{span\_id} correlate the event with operational context but do not replace durable identifiers. This matters when one logical run spans multiple traces, waits in a queue, or must be verified after the tracing backend's retention period. + +Usage events distinguish direct measurements from estimates and distinguish an individual operation from a rollup. A cost value without currency and provenance is not portable. Approval events carry the action digest of the approved intent, so a consumer can detect an approval attached to an action with different arguments. The reference finalizer does not make that comparison: at the evaluated release, and on the main branch on October 1, 2026, a run whose executed action carries a digest other than the approved one is still appraised as affirming. Data-flow events record classifications, logical sources, destinations, and applicable policy labels without recording the content that crossed the boundary. + +All six families are defined by the schemas. At the evaluated release, however, neither SDK provides a helper that emits \texttt{approval.cancelled}, the approval execution outcomes, or the \texttt{evidence.*} lifecycle events; a producer must construct those events directly. + +\section{Privacy and propagation} + +The metadata-only profile prohibits raw prompts, model outputs, source code, tool arguments, tool results, credentials, and authorization tokens. Schema validation is supplemented by recursive key inspection because permissive extension objects could otherwise tunnel prohibited fields, and extension attributes are rejected unless each key is explicitly allowlisted. This is a minimization boundary, not a proof that arbitrary metadata cannot be identifying: a sensitive value placed under an allowed key is not detected. + +For synchronous cross-process calls, the SDK injects W3C trace context plus durable AgenTrust identifiers. A receiver extracts the OTel context and uses it as a remote parent. For queue handoffs, the receiver starts a new trace and links it to the extracted context, preserving causality without misrepresenting queued work as a synchronous child span. Propagated identifiers are untrusted input: they establish neither workload identity nor authorization. + +W3C Trace Context standardizes trace propagation \citep{tracecontext}; it intentionally does not provide identity or security. AgenTrust Telemetry follows that boundary and requires authoritative identity and policy systems to emit their own verified facts. + +\section{Projection and evidence finalization} + +The normalized JSON event is the source contract. OTel is a projection. Standard OTel attributes take precedence, and AgenTrust extensions are added only for governance semantics not represented by the pinned convention. Trace and span identifiers remain native context rather than duplicated string attributes. Event projection follows OTel's guidance that events represent point-in-time occurrences or state transitions within a longer operation \citep{otelevents}. + +The SDK can deliver a validated event to two independent sinks. An OTel emitter supports operational search and metrics. An evidence accumulator receives the event before sampling or export and maintains an ordered SHA-256 chain over RFC 8785 canonical bytes of accepted events. It records a caller-supplied completeness label; it does not independently measure whether every source event was captured. Finalization may produce a TRACE record containing the chain commitment, policy and approval summaries, usage totals with provenance, data-flow classifications, and the completeness status \citep{tracepaper}. + +With the reference accumulator configured as the evidence sink, each operational projection attempted by an invocation follows successful evidence acceptance. In durable callback mode, acceptance additionally follows the callback's acknowledgment, and a refused or failed acknowledgment raises before any projection is attempted. A projection that fails after acceptance is reported separately and does not remove the accepted event. This is an ordering property, not a completeness property. + +A complete-capture claim needs more than that ordering: an independently defined set of required events, instrumentation at every declared boundary, and a recovery policy. The accumulator verifies none of these. An empty accumulator can be sealed as complete, so the label alone proves neither that required events were emitted nor that they were retained; the TRACE finalizer at the evaluated release refuses open, incomplete, or empty evidence and marks its output \texttt{software-only}. A durable callback must supply its own storage, idempotency, and recovery: if an external store commits but its acknowledgment is lost, a retry repeats the external write. A backend query cannot establish these properties after the fact. TRACE supplies a signed evidence container and verification semantics; AgenTrust Telemetry supplies the event normalization and pre-export accumulation boundary. + +\section{Implementation and conformance} + +The project publishes JSON Schema 2020-12 contracts, Python and TypeScript SDKs, valid and invalid fixtures, and an independent Python conformance runner. Both SDKs consume the same schema corpus, and a repository check compares the TypeScript schema copies byte for byte with the normative files. TypeScript preserves nanosecond timestamps as decimal strings to avoid JavaScript numeric truncation. Neither SDK installs a global OTel provider, processor, exporter, meter, or propagator; callers retain operational ownership. + +The portable fixture corpus includes six valid examples, one for each family, and seven invalid examples. Negative cases reject raw content, an all-zero trace identifier, numeric timestamp ambiguity, usage without a measurement basis, a model call incorrectly marked as a rollup, aggregate cost without rollup semantics, and action execution without an action digest. Repository tests cover policy adapters, approval state, AGT audit and data projection, OTel projection, propagation, evidence accumulation, TRACE finalization, cross-language schema synchronization, and repository gates. The checked OTel projection matrix, pinned to a development-status commit of the GenAI conventions captured on August 18, 2026, covers 16 span attributes, eight metrics, and six event families. + +\textbf{Results.} The evaluated commit is \texttt{29c13cf} (release 0.1.0-alpha.2, the repository's main branch on September 3, 2026). On that date 111 Python unit tests passed and all 13 portable fixtures produced their expected results. A rerun at the same commit on September 27, 2026 reproduced both counts: 111 tests passed with no skips, the runner reported six valid and seven invalid fixtures as expected, the projection matrix check reported 16 span attributes, eight metrics, and six event families, and the TypeScript schema copies matched the normative bytes. The rerun pinned the optional TRACE dependency to agentrust-trace 0.9.0, the release current on September 3. With the version resolved on September 27 (0.11.0), one test fails: agentrust-trace 0.10.0 began rejecting a non-Ed25519 stand-in signing key that the test used, which the repository corrected in its test on September 6. Four scripted probes of the evaluated code confirmed the ordering and completeness statements in the previous section: a refused acknowledgment raised before any log projection; a failing log exporter left the accepted event in place and was reported as a projection error; an empty accumulator sealed as complete; and a lost acknowledgment followed by a retry produced two external writes and one local entry. For comparison, the current main branch (release 0.1.0-alpha.5) passes 128 Python tests with the same 13 fixtures and the same matrix counts. + +\section{Findings after the evaluation} +\label{sec:after} + +Two changes merged after September 3, 2026 narrow the results above. + +\textbf{Finalization did not recheck the evidence chain.} At the evaluated release, TRACE finalization took the runtime measurement from the snapshot's chain digest and derived the appraisal from the snapshot's entries without checking that the entries still hashed to that digest. Because the snapshot holds mutable objects, a caller could seal a run holding a deny decision, edit the entry to allow, and obtain a signed record affirming the edited decision under a measurement that commits to the original. The fix, merged on September 25, 2026 and released in 0.1.0-alpha.5, recomputes the chain before signing in both SDKs and refuses on mismatch. Run against the evaluated code, the four regression tests added with that fix produce three failures (an edited event, a dropped entry, and a substituted chain digest are all signed) and one error. The finalization results in the previous section therefore do not extend to snapshots that can be modified between sealing and signing. + +\textbf{Validation and locking edge cases.} The same change found that validation at the evaluated release raised an uncaught recursion error on deeply nested input, raised on non-string keys, and accepted NaN and infinite numbers until the evidence chain rejected them; all three now fail validation. A change merged on September 6, 2026 stopped the Python accumulator from deadlocking when a durable callback re-entered it on the same thread. + +\section{Related work and novelty boundary} + +\textbf{Observability conventions.} OTel defines the telemetry data model, context propagation, and event guidance \citep{otelevents}. Its GenAI conventions define agent, tool, and inference spans and treat message content as opt-in attributes \citep{otelgenai}. OpenInference defines span attributes such as \texttt{input.value} and \texttt{output.value} that carry payloads unless hiding options are configured \citep{openinference}. W3C Trace Context provides interoperable propagation \citep{tracecontext}, and CloudEvents standardizes event envelopes across systems \citep{cloudevents}. AgenTrust Telemetry composes with these surfaces and does not replace them. + +\textbf{Governance telemetry.} Pathak and Jain's Governance-Aware Agent Telemetry (GAAT) is the closest prior work \citep{gaat}. It defines a Governance Telemetry Schema that extends OTel with governance attributes such as classification, jurisdiction, and sensitivity; evaluates OPA-compatible rules with sub-200 ms enforcement; routes graduated interventions through an enforcement bus; and adds a Trusted Telemetry Plane with ECDSA P-256 signed events, a Merkle-tree audit log, replay detection, and statistical omission detection trained on expected event sequences. Its evaluation reports violation prevention rates on synthetic and trace-derived workloads. GAAT therefore already covers governance attributes on OTel, policy evaluation over telemetry, and cryptographic provenance of telemetry. Kulkarni's Governance-Aware Observability Pipeline (GAOP) enforces compliance rules, including PII redaction, inside the telemetry pipeline and adds cryptographic lineage \citep{gaop}. Policy engines and approval systems also expose their own decision logs. + +\textbf{Tamper-evident agent records.} Hash-chained and Merkle-structured audit logs are long established \citep{schneierkelsey,crosbywallach}. The Agent Audit Trail Internet-Draft, first posted in March 2026, specifies JSON agent audit records linked by SHA-256 hash chaining over RFC 8785 canonical bytes, with optional signatures and, in later revisions, a record phase that distinguishes pre-execution from post-execution records \citep{aatdraft}. Two reports posted in September 2026, after the repository was public, are concurrent with this work: Agent Flight Recorder adds Merkle batching and on-chain anchoring to canonical agent-action events \citep{afr}, and NovaFabric seals agent runs into signed, replayable capsules built on OTel and measures declared-stream completeness \citep{novafabric}. + +\textbf{What this report does not claim.} It does not claim governance event families, governance attributes on OTel, policy enforcement from telemetry, payload redaction, hash-chained or signed audit records, or a measured completeness guarantee. Governance event families alone are not novel. + +\textbf{Narrow delta.} The remaining contribution is the conjunction of (1) stable run and action joins across synchronous and queued handoffs, independent of trace identifiers; (2) metadata-only exclusions enforced by the producer SDK before any export, instead of redaction in a downstream pipeline; (3) explicit measurement and cost provenance; and (4) a stated separation between lossy operational export and governance evidence, in which events are accepted before export and completeness is an explicit producer assertion carried into a separately verifiable TRACE record, never inferred from a backend. The claim would be narrowed further by a prior cross-runtime convention that supplies the same producer-side exclusions, durable joins, and separation of export from evidence. + +\section{Limitations and conclusion} + +The contract is alpha and may change. Schemas constrain representation, not the truth of producer observations. Metadata can still be sensitive. Coverage depends on instrumentation at every declared governance boundary. An accumulator cannot observe bypassed code, and a signature cannot repair an incomplete capture. The accumulator's memory mode is not durable, and in callback mode the adopter owns storage, idempotency, and recovery. Interoperability was exercised locally with OpenTelemetry Python, not across collectors, backends, or independent implementations, and no performance measurement is reported. + +AgenTrust Telemetry's central result is a boundary: operational traces may be rich but lossy, while a complete-capture claim requires independently justified instrumentation and coverage in addition to accumulation before loss. A small event contract, shared schemas, privacy failures, durable joins, and optional TRACE finalization make that boundary portable without creating another tracing backend. + +\bibliographystyle{plainnat} +\bibliography{refs} +\end{document} diff --git a/papers/agentrust-telemetry/refs.bib b/papers/agentrust-telemetry/refs.bib new file mode 100644 index 0000000..39debbc --- /dev/null +++ b/papers/agentrust-telemetry/refs.bib @@ -0,0 +1,13 @@ +@misc{otelgenai, title={{OpenTelemetry} {GenAI} Semantic Conventions}, author={{OpenTelemetry Authors}}, year={2026}, howpublished={Development status; the evaluated compatibility matrix pins commit a685613}, url={https://github.com/open-telemetry/semantic-conventions-genai}} +@misc{otelevents, title={Semantic Conventions for Events}, author={{OpenTelemetry Authors}}, year={2026}, howpublished={Development status}, url={https://opentelemetry.io/docs/specs/semconv/general/events/}} +@misc{openinference, title={{OpenInference} Specification}, author={{Arize AI}}, year={2026}, url={https://github.com/Arize-ai/openinference/tree/main/spec}} +@techreport{tracecontext, title={{Trace Context} Level 2}, author={Kanzhelev, Sergey and Dyla, Daniel and Shkuro, Yuri and Sundaram, J. Kalyana and Krol, Bastian}, institution={{W3C}}, type={Candidate Recommendation Draft}, year={2024}, url={https://www.w3.org/TR/2024/CRD-trace-context-2-20240328/}} +@misc{cloudevents, title={{CloudEvents} Specification, version 1.0.2}, author={{CloudEvents Authors}}, year={2022}, howpublished={Cloud Native Computing Foundation}, url={https://github.com/cloudevents/spec}} +@techreport{tracepaper, title={{TRACE}: Trust, Runtime Attestation, and Compliance Evidence ({A} Portable Attestation Format for {AI} Agent Runtime Governance)}, author={Poddar, Rishabh and Fulkerson, Aaron and Siddique, Imran}, institution={{OPAQUE} Systems}, type={Technical report, version 1}, year={2026}, url={https://agentrust-io.com/research/trace/v1/}} +@misc{gaat, title={{Governance-Aware Agent Telemetry} for Closed-Loop Enforcement in Multi-Agent {AI} Systems}, author={Pathak, Anshul and Jain, Nishant}, year={2026}, howpublished={arXiv:2604.05119}, url={https://arxiv.org/abs/2604.05119}} +@article{gaop, title={{Governance-Aware Observability Pipeline} ({GAOP}): Embedding Compliance Enforcement and Cryptographic Lineage into Telemetry Data Flows}, author={Kulkarni, Priyanka}, journal={International Journal of Computer Applications}, volume={187}, number={50}, year={2025}, doi={10.5120/ijca2025925867}} +@article{schneierkelsey, title={Secure Audit Logs to Support Computer Forensics}, author={Schneier, Bruce and Kelsey, John}, journal={{ACM} Transactions on Information and System Security}, volume={2}, number={2}, year={1999}, doi={10.1145/317087.317089}} +@inproceedings{crosbywallach, title={Efficient Data Structures for Tamper-Evident Logging}, author={Crosby, Scott A. and Wallach, Dan S.}, booktitle={Proceedings of the 18th {USENIX} Security Symposium}, year={2009}, url={https://www.usenix.org/conference/usenixsecurity09/technical-sessions/presentation/efficient-data-structures-tamper-evident}} +@misc{aatdraft, title={{Agent Audit Trail}: A Standard Logging Format for Autonomous {AI} Systems}, author={Sharif, R.}, year={2026}, howpublished={Internet-Draft draft-sharif-agent-audit-trail-05, {IETF}, first posted in March}, url={https://datatracker.ietf.org/doc/draft-sharif-agent-audit-trail/}} +@misc{afr, title={{Agent Flight Recorder}: Tamper-Evident Audit Trails with On-Chain Anchoring for Long-Horizon Tool-Using Agents}, author={Bindschaedler, Laurent and Botha, Quentin and Siebenbrunner, Christoph}, year={2026}, howpublished={arXiv:2609.01931}, url={https://arxiv.org/abs/2609.01931}} +@misc{novafabric, title={{NovaFabric}: Tamper-Evident, Replayable Evidence for Autonomous {AI} Agent Runs}, author={Ardebili, Mohsen Seyedkazemi}, year={2026}, howpublished={arXiv:2609.12582}, url={https://arxiv.org/abs/2609.12582}} diff --git a/papers/telemetry-evidence-acceptance/README.md b/papers/telemetry-evidence-acceptance/README.md new file mode 100644 index 0000000..d49b15e --- /dev/null +++ b/papers/telemetry-evidence-acceptance/README.md @@ -0,0 +1,22 @@ +# Acceptance Before Export: A Bounded Evidence Contract for Governance Telemetry + +Technical report, version 3, October 1, 2026. Not peer reviewed. Patent Pending. +Version 3 adds the pinned revision's TRACE finalizer defect, which the paper's truncation +argument depends on, and the Internet-Draft that already chains agent audit records over RFC 8785 +bytes. Results are unchanged. + +Build: `cd telemetry-evidence-acceptance && tectonic paper.tex` (prepared with Tectonic 0.16.9; set +SOURCE_DATE_EPOCH=1790812800 to reproduce the published PDF byte for byte). The fonts and their OFL +licenses are in `_paper-assets/fonts/`, the same relative path the manuscript uses. + +Paper text is licensed CC BY 4.0: https://creativecommons.org/licenses/by/4.0/ +Project code retains its existing project license; see https://github.com/agentrust-io/agentrust-telemetry. + +This package contains the manuscript source, the experiment scripts and the records of the +September 27 and October 1, 2026 reruns. The evaluated code is agentrust-telemetry commit +29c13cfa4047a0f5784a90df63b22a3568453158 (0.1.0-alpha.2): 111 Python unit tests passed and all +13 portable fixtures produced their expected verdicts on both dates. The October 1 README points +to TypeScript and finalizer records kept with the AgenTrust Telemetry report's source package +(https://agentrust-io.com/research/agentrust-telemetry/). No durable store, crash-restart, +multi-process or deployed-exporter experiment was run. +Current implementation guidance: https://agentrust-io.com/telemetry/ diff --git a/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Montserrat-OFL.txt b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Montserrat-OFL.txt new file mode 100644 index 0000000..fe576a2 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Montserrat-OFL.txt @@ -0,0 +1,93 @@ +Copyright 2024 The Montserrat.Git Project Authors (https://github.com/JulietaUla/Montserrat.git) + +This Font Software is licensed under the SIL Open Font License, Version 1.1. +This license is copied below, and is also available with a FAQ at: +https://openfontlicense.org + + +----------------------------------------------------------- +SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 +----------------------------------------------------------- + +PREAMBLE +The goals of the Open Font License (OFL) are to stimulate worldwide +development of collaborative font projects, to support the font creation +efforts of academic and linguistic communities, and to provide a free and +open framework in which fonts may be shared and improved in partnership +with others. + +The OFL allows the licensed fonts to be used, studied, modified and +redistributed freely as long as they are not sold by themselves. The +fonts, including any derivative works, can be bundled, embedded, +redistributed and/or sold with any software provided that any reserved +names are not used by derivative works. The fonts and derivatives, +however, cannot be released under any other type of license. The +requirement for fonts to remain under this license does not apply +to any document created using the fonts or their derivatives. + +DEFINITIONS +"Font Software" refers to the set of files released by the Copyright +Holder(s) under this license and clearly marked as such. This may +include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the +copyright statement(s). + +"Original Version" refers to the collection of Font Software components as +distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, +or substituting -- in part or in whole -- any of the components of the +Original Version, by changing formats or by porting the Font Software to a +new environment. + +"Author" refers to any designer, engineer, programmer, technical +writer or other person who contributed to the Font Software. + +PERMISSION & CONDITIONS +Permission is hereby granted, free of charge, to any person obtaining +a copy of the Font Software, to use, study, copy, merge, embed, modify, +redistribute, and sell modified and unmodified copies of the Font +Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, +in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, +redistributed and/or sold with any software, provided that each copy +contains the above copyright notice and this license. These can be +included either as stand-alone text files, human-readable headers or +in the appropriate machine-readable metadata fields within text or +binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font +Name(s) unless explicit written permission is granted by the corresponding +Copyright Holder. This restriction only applies to the primary font name as +presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font +Software shall not be used to promote, endorse or advertise any +Modified Version, except to acknowledge the contribution(s) of the +Copyright Holder(s) and the Author(s) or with their explicit written +permission. + +5) The Font Software, modified or unmodified, in part or in whole, +must be distributed entirely under this license, and must not be +distributed under any other license. The requirement for fonts to +remain under this license does not apply to any document created +using the Font Software. + +TERMINATION +This license becomes null and void if any of the above conditions are +not met. + +DISCLAIMER +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT +OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE +COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL +DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM +OTHER DEALINGS IN THE FONT SOFTWARE. \ No newline at end of file diff --git a/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Montserrat-SemiBold.ttf b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Montserrat-SemiBold.ttf new file mode 100644 index 0000000..2c0a104 Binary files /dev/null and b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Montserrat-SemiBold.ttf differ diff --git a/papers/telemetry-evidence-acceptance/_paper-assets/fonts/README.md b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/README.md new file mode 100644 index 0000000..bcf1969 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/README.md @@ -0,0 +1 @@ +Open-license fonts from google/fonts, downloaded September 4, 2026. Static review instances generated with fontTools from Montserrat[wght].ttf and Roboto[wdth,wght].ttf. Corresponding OFL licenses are retained. diff --git a/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-Bold.ttf b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-Bold.ttf new file mode 100644 index 0000000..5f938b8 Binary files /dev/null and b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-Bold.ttf differ diff --git a/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-OFL.txt b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-OFL.txt new file mode 100644 index 0000000..9c48e05 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-OFL.txt @@ -0,0 +1,93 @@ +Copyright 2011 The Roboto Project Authors (https://github.com/googlefonts/roboto-classic) + +This Font Software is licensed under the SIL Open Font License, Version 1.1. +This license is copied below, and is also available with a FAQ at: +https://openfontlicense.org + + +----------------------------------------------------------- +SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 +----------------------------------------------------------- + +PREAMBLE +The goals of the Open Font License (OFL) are to stimulate worldwide +development of collaborative font projects, to support the font creation +efforts of academic and linguistic communities, and to provide a free and +open framework in which fonts may be shared and improved in partnership +with others. + +The OFL allows the licensed fonts to be used, studied, modified and +redistributed freely as long as they are not sold by themselves. The +fonts, including any derivative works, can be bundled, embedded, +redistributed and/or sold with any software provided that any reserved +names are not used by derivative works. The fonts and derivatives, +however, cannot be released under any other type of license. The +requirement for fonts to remain under this license does not apply +to any document created using the fonts or their derivatives. + +DEFINITIONS +"Font Software" refers to the set of files released by the Copyright +Holder(s) under this license and clearly marked as such. This may +include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the +copyright statement(s). + +"Original Version" refers to the collection of Font Software components as +distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, +or substituting -- in part or in whole -- any of the components of the +Original Version, by changing formats or by porting the Font Software to a +new environment. + +"Author" refers to any designer, engineer, programmer, technical +writer or other person who contributed to the Font Software. + +PERMISSION & CONDITIONS +Permission is hereby granted, free of charge, to any person obtaining +a copy of the Font Software, to use, study, copy, merge, embed, modify, +redistribute, and sell modified and unmodified copies of the Font +Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, +in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, +redistributed and/or sold with any software, provided that each copy +contains the above copyright notice and this license. These can be +included either as stand-alone text files, human-readable headers or +in the appropriate machine-readable metadata fields within text or +binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font +Name(s) unless explicit written permission is granted by the corresponding +Copyright Holder. This restriction only applies to the primary font name as +presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font +Software shall not be used to promote, endorse or advertise any +Modified Version, except to acknowledge the contribution(s) of the +Copyright Holder(s) and the Author(s) or with their explicit written +permission. + +5) The Font Software, modified or unmodified, in part or in whole, +must be distributed entirely under this license, and must not be +distributed under any other license. The requirement for fonts to +remain under this license does not apply to any document created +using the Font Software. + +TERMINATION +This license becomes null and void if any of the above conditions are +not met. + +DISCLAIMER +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT +OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE +COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL +DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM +OTHER DEALINGS IN THE FONT SOFTWARE. diff --git a/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-Regular.ttf b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-Regular.ttf new file mode 100644 index 0000000..302a48e Binary files /dev/null and b/papers/telemetry-evidence-acceptance/_paper-assets/fonts/Roboto-Regular.ttf differ diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/Makefile b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/Makefile new file mode 100644 index 0000000..849a3e8 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/Makefile @@ -0,0 +1,5 @@ +all: + tectonic paper.tex + +clean: + rm -f paper.aux paper.bbl paper.blg paper.log paper.out paper.pdf diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/README.md b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/README.md new file mode 100644 index 0000000..eb8bf9d --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/README.md @@ -0,0 +1,22 @@ +# Experiments + +Run from this directory, with the dependencies listed in +`results/rerun-2026-09-27/pinned/results.json`: + +``` +python reproduce.py --prepare-from /path/to/clones --source-root /path/to/empty-dir --output out +python boundary_probes.py --source-root /path/to/empty-dir --output out +``` + +`--prepare-from` takes a directory holding clones of agentrust-telemetry, +weight-custody-manifest and agent-manifest. It runs `git archive` of each pinned commit +(no checkout, no branch switch), checks the commit id embedded in the tar and extracts it. +Alternatively point `--source-root` at checkouts that are already at the pinned commits with +clean tracked files. The runner refuses any other revision. Outputs contain no absolute paths. + +The runner covers all three reports that share it (memory sweep, human approval, Acceptance Before Export); this paper relies on telemetry-tests.txt and telemetry-conformance.txt and on the +matching entries in `boundary-probes.json`. `boundary_probes.py` records what each probe +observed instead of asserting, so it can also be run against later revisions. + +Neither script reproduces the historical protected-hardware receipt. `reproduce.py` and +`boundary_probes.py` are identical copies of the files in `../../_paper-assets/`. diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/boundary_probes.py b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/boundary_probes.py new file mode 100644 index 0000000..e92a251 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/boundary_probes.py @@ -0,0 +1,108 @@ +"""Adversarial boundary probes, separate from repository test suites. + +Each probe records what it observed (true or false) instead of asserting, so the +same script can be run against the pinned revisions and against later revisions. +A probe that raises is recorded with its exception type. Output contains no paths. +""" +from pathlib import Path +import sys, json, hashlib, zipfile, io +from copy import deepcopy +from datetime import datetime, timezone + +import argparse +p=argparse.ArgumentParser() +p.add_argument('--source-root',type=Path,default=Path.home()/'source') +p.add_argument('--output',type=Path,required=True) +args=p.parse_args() +SRC=args.source_root +OUT=args.output +OUT.mkdir(parents=True,exist_ok=True) +for path in ['agentrust-telemetry/src', 'weight-custody-manifest/python/src', 'agent-manifest/python/src']: + sys.path.insert(0, str(SRC/path)) +from agentrust_telemetry import EvidenceAccumulator, SchemaValidator, TelemetryClient, EvidencePersistenceError +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from wcm import BytearrayMemoryRange, run_memory_sweep, verify_memory_sweep, memory_sweep_public_key +from agent_manifest._delegation import HitlApprovalSigner, verify_hitl_approval +from agent_manifest._signing import generate_ed25519, signing_pre_image + +results = [] +def record(name, value, meaning): + results.append({'probe': name, 'observed': bool(value), 'meaning': meaning}) + +def verifies(fn, *a): + try: + fn(*a) + return True + except Exception: + return False + +v = SchemaValidator(SRC/'agentrust-telemetry/spec/schema') +event = json.loads((SRC/'agentrust-telemetry/conformance/fixtures/valid/policy-decision.json').read_text()) +class Log: + def __init__(self, fail=False): self.calls=0; self.fail=fail + def emit(self, rec): + self.calls += 1 + if self.fail: raise RuntimeError('controlled export failure') + +a = EvidenceAccumulator(event['run_id'], v) +log = Log(True) +r = TelemetryClient(v, span_resolver=lambda: None, evidence_sink=a, log_emitter=log).emit(event) +record('export_failure_preserves_accepted_event', len(a.snapshot().entries)==1 and len(r.projection_errors)==1, 'Evidence acceptance survives a later operational projection error; memory mode is not durable storage.') + +a = EvidenceAccumulator(event['run_id'], v, durable_append=lambda e: False) +log = Log() +try: TelemetryClient(v, span_resolver=lambda: None, evidence_sink=a, log_emitter=log).emit(event) +except EvidencePersistenceError: pass +record('failed_ack_suppresses_projection', log.calls==0 and not a.snapshot().entries, 'No operational projection or local chain advance after callback refusal.') + +empty = EvidenceAccumulator('empty-run', v).seal(completeness='complete') +record('empty_complete_declaration_is_accepted', empty.completeness=='complete' and not empty.entries, 'Counterexample: seal does not independently prove capture completeness.') + +committed=[] +def ambiguous(entry): committed.append(entry.event_id); raise RuntimeError('ack lost after external commit') +a = EvidenceAccumulator(event['run_id'], v, durable_append=ambiguous) +for _ in range(2): + try: a.append(event) + except EvidencePersistenceError: pass +record('external_commit_retry_requires_sink_idempotency', len(committed)==2 and not a.snapshot().entries, 'Counterexample: no cross-store transaction or built-in exactly-once persistence.') + +class Alias: + page_size=64; page_count=4 + def __init__(self): self.p=[bytearray(64) for _ in range(4)] + def write_page(self,i,x): self.p[0 if i==2 else i][:]=x + def read_page(self,i): return bytes(self.p[0 if i==2 else i]) +key=Ed25519PrivateKey.generate() +sweep=run_memory_sweep(Alias(),challenge_nonce='review-nonce',signing_key=key,sweep_secret=b's'*32) +record('signed_bad_memory_is_authentic_not_acceptable', sweep.aliasing_detected and verify_memory_sweep(sweep,memory_sweep_public_key(key))[0], 'Signature verification authenticates the negative finding. The broker must reject it separately.') +good=run_memory_sweep(BytearrayMemoryRange(64,page_size=64),challenge_nonce='old-challenge',signing_key=key,sweep_secret=b's'*32) +record('signature_verifier_has_no_freshness_input', verify_memory_sweep(good,memory_sweep_public_key(key))[0], 'Broker challenge comparison, not signature verification alone, establishes request freshness.') + +kp=generate_ed25519() +scope={'approval_duration_seconds':3600,'risk_tier':'high','action':'review'} +stamp=datetime.now(timezone.utc).isoformat() +signer=HitlApprovalSigner(kp) +approval={'approver_id':'mailto:reviewer@example.test','approved_at':stamp,'approved_scope':scope,'approval_method':'software-key'} +import inspect +method_param='approval_method' in inspect.signature(signer.sign_approval).parameters +extra={'approval_method':approval['approval_method']} if method_param else {} +approval['approval_signature']=signer.sign_approval(manifest_id='review-manifest',approved_at=stamp,approved_scope=scope,approver_id=approval['approver_id'],**extra) +record('approval_signer_accepts_method_parameter', method_param, 'Whether this revision lets the human signer cover approval_method; the probe passes it when available.') +original_ok=verifies(verify_hitl_approval,approval,'review-manifest',kp.public_bytes) +changed=deepcopy(approval); changed['approval_method']='hardware-key' +changed_ok=verifies(verify_hitl_approval,changed,'review-manifest',kp.public_bytes) +record('approval_method_label_is_outside_approval_signature', original_ok and changed_ok, 'True means the unaltered approval verifies and the method-label alteration still verifies (label outside the signature). False with the unaltered approval verifying means the label is bound. Not a full runtime test.') +record('approval_unaltered_verifies', original_ok, 'Control for the previous probe.') +def duration_case(dur): + sc={'risk_tier':'high','action':'review'} + if dur is not None: sc['approval_duration_seconds']=dur + ap={'approver_id':'mailto:reviewer@example.test','approved_at':stamp,'approved_scope':sc,'approval_method':'software-key'} + ap['approval_signature']=signer.sign_approval(manifest_id='review-manifest',approved_at=stamp,approved_scope=sc,approver_id=ap['approver_id'],**({'approval_method':'software-key'} if method_param else {})) + return verifies(verify_hitl_approval,ap,'review-manifest',kp.public_bytes) +record('approval_helper_accepts_zero_or_missing_duration', duration_case(0) and duration_case(None), 'True means the single-approval helper accepts approval_duration_seconds of 0 or absent. The integrated verifier applies its own expiry logic.') +base={'manifest_id':'review-manifest','hitl_record':{'required':True,'risk_tier':'high','approvals':[]}} +attached=deepcopy(base); attached['hitl_record']['approvals']=[approval] +weakened=deepcopy(base); weakened['hitl_record']['required']=False +record('requirement_bound_approval_appendable', signing_pre_image(base)==signing_pre_image(attached) and signing_pre_image(base)!=signing_pre_image(weakened), 'Issuer pre-image preserves requirements while normalizing only approvals.') + +(OUT/'boundary-probes.json').write_text(json.dumps({'captured_at':datetime.now(timezone.utc).isoformat(timespec='seconds'),'results':results,'limits':'Single-process software probes; no hardware or legal conclusion.'},indent=2)+chr(10),encoding='utf-8',newline=chr(10)) +print(json.dumps(results,indent=2)) diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/reproduce.py b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/reproduce.py new file mode 100644 index 0000000..62ad9c3 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/reproduce.py @@ -0,0 +1,139 @@ +"""Bounded-suite runner for three technical reports (the challenge-bound memory sweep, +requirement-preserving human approval and Acceptance Before Export); refuses unexpected +source revisions. + +Sources live under --source-root/. Each one must be either + + * a git checkout whose HEAD equals the pin and whose tracked files are unmodified, or + * an extracted `git archive` of the pin. Create one with --prepare-from CLONES_ROOT, + which runs `git archive ` in CLONES_ROOT/ (read only, no checkout or + branch switch), checks the commit id embedded in the tar and extracts it. + +Everything written to --output is free of absolute local paths: commands name the +interpreter as `python`, working directories are relative to the source root, and +logs have the source root, interpreter prefix and home directory replaced by +placeholders. +""" +from pathlib import Path +from datetime import datetime, timezone +import argparse, hashlib, importlib.metadata, io, json, os, platform, re, subprocess, sys, tarfile + +PINS = { + 'agentrust-telemetry': '29c13cfa4047a0f5784a90df63b22a3568453158', + 'weight-custody-manifest': '2acedfa883204f7d1fd457d3ee603198d85be82c', + 'agent-manifest': '934809709a2815695d65cfacb45dc0a164286046', +} +TASKS = [ + ('telemetry-tests', 'agentrust-telemetry', ['-m', 'unittest', 'discover', '-s', 'tests', '-v']), + ('telemetry-conformance', 'agentrust-telemetry', ['conformance/runner/validate.py']), + ('memory-tests', 'weight-custody-manifest/python', + ['-m', 'pytest', '-p', 'no:cacheprovider', '-o', 'pythonpath=src', + 'tests/test_memory_sweep.py', 'tests/test_kbs.py', '-v']), + ('approval-tests', 'agent-manifest/python', + ['-m', 'pytest', '-p', 'no:cacheprovider', '-o', 'pythonpath=src', + 'tests/test_signing.py', 'tests/test_delegation_hitl.py', '-v']), +] +DEPENDENCIES = ['pytest', 'cryptography', 'pydantic', 'jsonschema', 'referencing', 'rfc8785', + 'cbor2', 'agentrust-trace', 'opentelemetry-sdk', 'jcs', 'PyYAML'] +MARKER = '.source-commit' + + +def git(repo, *args, **kw): + return subprocess.check_output(['git', '-c', 'safe.directory=' + str(repo), '-C', str(repo), *args], **kw) + + +def prepare(clones, root): + for name, pin in PINS.items(): + dest = root / name + if dest.exists(): + raise SystemExit(f'{name}: {dest.name} already exists under the source root; use an empty root') + data = git((clones / name).resolve(), 'archive', '--format=tar', pin) + commit = subprocess.run(['git', 'get-tar-commit-id'], input=data, capture_output=True, check=True).stdout.decode().strip() + if commit != pin: + raise SystemExit(f'{name}: archive carries {commit}, expected {pin}') + dest.mkdir(parents=True) + with tarfile.open(fileobj=io.BytesIO(data)) as tar: + tar.extractall(dest, filter='data') + (dest / MARKER).write_text(pin + '\n', encoding='ascii') + + +def verify(root): + method = {} + for name, pin in PINS.items(): + repo = (root / name).resolve() + if (repo / MARKER).exists(): + found = (repo / MARKER).read_text(encoding='ascii').strip() + if found != pin: + raise SystemExit(f'{name}: archive marker {found}, expected {pin}') + method[name] = 'git archive of the pinned commit (tar commit id checked)' + continue + head = git(repo, 'rev-parse', 'HEAD', text=True).strip() + if head != pin: + raise SystemExit(f'{name}: expected {pin}, found {head}') + if git(repo, 'diff', '--name-only', 'HEAD', text=True).strip(): + raise SystemExit(f'{name}: tracked files differ from the pinned revision') + method[name] = 'git checkout at the pinned commit, tracked files clean' + return method + + +def scrub(text, root): + subs = [(str(root.resolve()), ''), (sys.prefix, ''), + (sys.base_prefix, ''), (str(Path.home()), '')] + for old, new in subs: + for variant in {old, old.replace('\\', '/'), old.replace('\\', '\\\\')}: + text = text.replace(variant, new) + return re.sub(r'[A-Za-z]:\\\\?Users\\\\?[^\\\s"\']+', '', text) + + +def summary(name, text): + if name == 'telemetry-tests': + m = re.search(r'Ran (\d+) tests?.*?\n\n(OK.*|FAILED.*)', text, re.S) + return f'{m.group(1)} tests, {m.group(2).strip()}' if m else 'unparsed' + if name == 'telemetry-conformance': + verdicts = [l.split()[0] for l in text.splitlines() if l.startswith(('PASS ', 'FAIL '))] + return f"{verdicts.count('PASS')} of {len(verdicts)} fixtures returned the expected verdict" + m = re.findall(r'=+ (.*?) in [0-9.]+s', text) + return m[-1] if m else 'unparsed' + + +def main(): + p = argparse.ArgumentParser() + p.add_argument('--source-root', type=Path, required=True) + p.add_argument('--output', type=Path, required=True) + p.add_argument('--prepare-from', type=Path, help='directory of local clones to `git archive` from') + a = p.parse_args() + a.output.mkdir(parents=True, exist_ok=True) + if a.prepare_from: + a.source_root.mkdir(parents=True, exist_ok=True) + prepare(a.prepare_from, a.source_root) + method = verify(a.source_root) + results = [] + env = dict(os.environ, PYTHONDONTWRITEBYTECODE='1', PYTHONUTF8='1') + for name, path, args in TASKS: + run = subprocess.run([sys.executable, *args], cwd=a.source_root / path, env=env, + capture_output=True, text=True, encoding='utf-8', errors='replace') + log = scrub(run.stdout + '\nSTDERR\n' + run.stderr, a.source_root) + (a.output / (name + '.txt')).write_text(log, encoding='utf-8', newline='\n') + results.append({'name': name, 'cwd': path, 'command': ['python', *args], 'exit_code': run.returncode, + 'summary': summary(name, log), 'log': name + '.txt', + 'log_sha256': hashlib.sha256(log.encode('utf-8')).hexdigest()}) + deps = {} + for d in DEPENDENCIES: + try: + deps[d] = importlib.metadata.version(d) + except importlib.metadata.PackageNotFoundError: + deps[d] = None + record = {'captured_at': datetime.now(timezone.utc).isoformat(timespec='seconds'), + 'python': platform.python_version(), 'platform': platform.platform(terse=True), + 'dependencies': deps, + 'sources': {n: {'commit': PINS[n], 'verified_by': method[n]} for n in PINS}, + 'results': results, + 'limits': ['Local software tests only; no protected-hardware execution.', + 'Tests and implementation share a repository and are not independent validation.']} + (a.output / 'results.json').write_text(json.dumps(record, indent=2) + '\n', encoding='utf-8', newline='\n') + print(json.dumps(results, indent=2)) + raise SystemExit(any(r['exit_code'] for r in results)) + + +if __name__ == '__main__': + main() diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/README.md b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/README.md new file mode 100644 index 0000000..d33a0b5 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/README.md @@ -0,0 +1,16 @@ +# September 27, 2026 rerun + +- `pinned/`: `reproduce.py` and `boundary_probes.py` against `git archive` extracts of the + three pinned commits (agentrust-telemetry 29c13cf, weight-custody-manifest 2acedfa, + agent-manifest 9348097). The runner checked each tar's embedded commit id before extracting. + `results.json` records the interpreter version, package versions, commands and log hashes. +- `later-revision/`: `boundary_probes.py` only, against `git archive` extracts of the + commits listed in `COMMITS.txt` (origin/main on September 27, 2026). The repository + test suites were not run at these commits. + +Environment: Windows, Python 3.12.10, pytest 8.4.2, cryptography 50.0.0, jsonschema 4.26.0, +cbor2 5.9.0. The September 4 run used cbor2 6.1.4; local application control blocks the +6.x compiled extension on this machine. The selected test modules do not test COSE or CBOR encoding. + +Logs have local paths replaced with ``, `` and ``. +Software only: no protected hardware, durable store or external identity provider. diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/later-revision/COMMITS.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/later-revision/COMMITS.txt new file mode 100644 index 0000000..0c2d2a0 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/later-revision/COMMITS.txt @@ -0,0 +1,3 @@ +agent-manifest 3c124914e3dcd64369a16d27a5cae6943b918916 +weight-custody-manifest 94d5519fd604f7be5a784539a37c1fc021e3d607 +agentrust-telemetry c3515681261fd0ab6e6aa38e20eb2f071aee7a20 diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/later-revision/boundary-probes.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/later-revision/boundary-probes.json new file mode 100644 index 0000000..904e5dc --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/later-revision/boundary-probes.json @@ -0,0 +1,61 @@ +{ + "captured_at": "2026-09-28T04:55:59+00:00", + "results": [ + { + "probe": "export_failure_preserves_accepted_event", + "observed": true, + "meaning": "Evidence acceptance survives a later operational projection error; memory mode is not durable storage." + }, + { + "probe": "failed_ack_suppresses_projection", + "observed": true, + "meaning": "No operational projection or local chain advance after callback refusal." + }, + { + "probe": "empty_complete_declaration_is_accepted", + "observed": true, + "meaning": "Counterexample: seal does not independently prove capture completeness." + }, + { + "probe": "external_commit_retry_requires_sink_idempotency", + "observed": true, + "meaning": "Counterexample: no cross-store transaction or built-in exactly-once persistence." + }, + { + "probe": "signed_bad_memory_is_authentic_not_acceptable", + "observed": true, + "meaning": "Signature verification authenticates the negative finding. The broker must reject it separately." + }, + { + "probe": "signature_verifier_has_no_freshness_input", + "observed": true, + "meaning": "Broker challenge comparison, not signature verification alone, establishes request freshness." + }, + { + "probe": "approval_signer_accepts_method_parameter", + "observed": true, + "meaning": "Whether this revision lets the human signer cover approval_method; the probe passes it when available." + }, + { + "probe": "approval_method_label_is_outside_approval_signature", + "observed": false, + "meaning": "True means the unaltered approval verifies and the method-label alteration still verifies (label outside the signature). False with the unaltered approval verifying means the label is bound. Not a full runtime test." + }, + { + "probe": "approval_unaltered_verifies", + "observed": true, + "meaning": "Control for the previous probe." + }, + { + "probe": "approval_helper_accepts_zero_or_missing_duration", + "observed": false, + "meaning": "True means the single-approval helper accepts approval_duration_seconds of 0 or absent. The integrated verifier applies its own expiry logic." + }, + { + "probe": "requirement_bound_approval_appendable", + "observed": true, + "meaning": "Issuer pre-image preserves requirements while normalizing only approvals." + } + ], + "limits": "Single-process software probes; no hardware or legal conclusion." +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/approval-tests.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/approval-tests.txt new file mode 100644 index 0000000..c2324f7 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/approval-tests.txt @@ -0,0 +1,91 @@ +============================= test session starts ============================= +platform win32 -- Python 3.12.10, pytest-8.4.2, pluggy-1.6.0 -- \Scripts\python.exe +rootdir: \agent-manifest\python +configfile: pyproject.toml +collecting ... collected 74 items + +tests/test_signing.py::test_pre_image_excludes_attestation PASSED [ 1%] +tests/test_signing.py::test_pre_image_excludes_signature PASSED [ 2%] +tests/test_signing.py::test_pre_image_includes_all_signed_fields PASSED [ 4%] +tests/test_signing.py::test_pre_image_is_rfc8785 PASSED [ 5%] +tests/test_signing.py::test_ed25519_sign_verify_roundtrip PASSED [ 6%] +tests/test_signing.py::test_ed25519_wrong_message_fails PASSED [ 8%] +tests/test_signing.py::test_ed25519_wrong_key_fails PASSED [ 9%] +tests/test_signing.py::test_ed25519_truncated_signature_fails PASSED [ 10%] +tests/test_signing.py::test_ed25519_wrong_algorithm_label_not_accepted PASSED [ 12%] +tests/test_signing.py::test_ed25519_signed_fields_list_correct PASSED [ 13%] +tests/test_signing.py::test_ed25519_sign_block_validates_against_model PASSED [ 14%] +tests/test_signing.py::test_ed25519_small_order_key_rejected PASSED [ 16%] +tests/test_signing.py::test_ed25519_all_small_order_points_rejected PASSED [ 17%] +tests/test_signing.py::test_ed25519_truncated_sig_raises_invalid_signature PASSED [ 18%] +tests/test_signing.py::test_b64url_decode_rejects_plus PASSED [ 20%] +tests/test_signing.py::test_b64url_decode_rejects_slash PASSED [ 21%] +tests/test_signing.py::test_b64url_decode_accepts_valid PASSED [ 22%] +tests/test_signing.py::test_ed25519_public_key_roundtrip_b64url PASSED [ 24%] +tests/test_signing.py::test_ml_dsa65_sign_verify_roundtrip PASSED [ 25%] +tests/test_signing.py::test_ml_dsa65_wrong_message_fails PASSED [ 27%] +tests/test_signing.py::test_hybrid_sign_verify_roundtrip PASSED [ 28%] +tests/test_signing.py::test_hybrid_tampered_classical_fails PASSED [ 29%] +tests/test_signing.py::test_hybrid_tampered_pq_fails PASSED [ 31%] +tests/test_signing.py::test_hybrid_both_components_cover_same_pre_image PASSED [ 32%] +tests/test_signing.py::test_signed_fields_match_spec_coverage_table PASSED [ 33%] +tests/test_signing.py::test_pre_image_normalizes_hitl_approvals_to_empty PASSED [ 35%] +tests/test_signing.py::test_pre_image_keeps_hitl_required_tamper_evident PASSED [ 36%] +tests/test_signing.py::test_approvals_attach_post_issuance_without_resigning PASSED [ 37%] +tests/test_signing.py::test_newly_signed_fields_are_tamper_evident PASSED [ 39%] +tests/test_signing.py::test_ed25519_keypair_repr_redacts_private_key PASSED [ 40%] +tests/test_signing.py::test_ed25519_keypair_str_equals_repr PASSED [ 41%] +tests/test_signing.py::test_ed25519_keypair_private_b64url_roundtrip PASSED [ 43%] +tests/test_signing.py::test_ed25519_from_private_bytes_valid PASSED [ 44%] +tests/test_signing.py::test_ed25519_from_private_bytes_malformed_raises PASSED [ 45%] +tests/test_signing.py::test_ed25519_from_private_bytes_too_short_raises PASSED [ 47%] +tests/test_signing.py::test_ed25519_verifier_wrong_length_key_raises PASSED [ 48%] +tests/test_signing.py::test_ed25519_verifier_zero_length_key_raises PASSED [ 50%] +tests/test_signing.py::test_ed25519_verifier_rsa_sized_bytes_raises PASSED [ 51%] +tests/test_signing.py::test_ed25519_signer_signed_at_is_iso8601 PASSED [ 52%] +tests/test_signing.py::test_ml_dsa65_signer_signed_at_is_iso8601 PASSED [ 54%] +tests/test_signing.py::test_hybrid_signer_signed_at_is_iso8601 PASSED [ 55%] +tests/test_delegation_hitl.py::test_hop_pre_image_includes_manifest_id PASSED [ 56%] +tests/test_delegation_hitl.py::test_hop_pre_image_includes_scope PASSED [ 58%] +tests/test_delegation_hitl.py::test_hop_pre_image_deterministic PASSED [ 59%] +tests/test_delegation_hitl.py::test_hop_pre_image_different_hops PASSED [ 60%] +tests/test_delegation_hitl.py::test_delegation_sign_verify_single_hop PASSED [ 62%] +tests/test_delegation_hitl.py::test_delegation_wrong_key_fails PASSED [ 63%] +tests/test_delegation_hitl.py::test_delegation_wrong_manifest_id_fails PASSED [ 64%] +tests/test_delegation_hitl.py::test_scope_laundering_detected PASSED [ 66%] +tests/test_delegation_hitl.py::test_depth_exceeded_raises PASSED [ 67%] +tests/test_delegation_hitl.py::test_empty_chain_passes PASSED [ 68%] +tests/test_delegation_hitl.py::test_missing_public_key_raises PASSED [ 70%] +tests/test_delegation_hitl.py::test_root_principal_must_match_manifest_issuer PASSED [ 71%] +tests/test_delegation_hitl.py::test_root_principal_matching_issuer_passes PASSED [ 72%] +tests/test_delegation_hitl.py::test_root_principal_matches_via_principal_manifest_id PASSED [ 74%] +tests/test_delegation_hitl.py::test_child_dropping_parent_constraint_is_rejected PASSED [ 75%] +tests/test_delegation_hitl.py::test_child_adding_constraint_is_allowed PASSED [ 77%] +tests/test_delegation_hitl.py::test_child_raising_ttl_is_rejected PASSED [ 78%] +tests/test_delegation_hitl.py::test_child_unbounded_ttl_under_bounded_parent_is_rejected PASSED [ 79%] +tests/test_delegation_hitl.py::test_child_raising_max_delegation_depth_is_rejected PASSED [ 81%] +tests/test_delegation_hitl.py::test_child_delegated_late_with_equal_ttl_outlives_parent_is_rejected PASSED [ 82%] +tests/test_delegation_hitl.py::test_child_delegated_late_with_shorter_ttl_still_within_parent_window_passes PASSED [ 83%] +tests/test_delegation_hitl.py::test_mixed_z_and_naive_delegated_at_still_refuses_on_absolute_expiry PASSED [ 85%] +tests/test_delegation_hitl.py::test_approval_pre_image_includes_manifest_id PASSED [ 86%] +tests/test_delegation_hitl.py::test_approval_pre_image_includes_scope PASSED [ 87%] +tests/test_delegation_hitl.py::test_approval_sign_verify PASSED [ 89%] +tests/test_delegation_hitl.py::test_approval_wrong_key_fails PASSED [ 90%] +tests/test_delegation_hitl.py::test_approval_wrong_manifest_id_fails PASSED [ 91%] +tests/test_delegation_hitl.py::test_approval_scope_change_fails PASSED [ 93%] +tests/test_delegation_hitl.py::test_approval_expired_raises PASSED [ 94%] +tests/test_delegation_hitl.py::test_approval_no_duration_does_not_expire PASSED [ 95%] +tests/test_delegation_hitl.py::test_hitl_approval_rejects_spiffe_approver_id PASSED [ 97%] +tests/test_delegation_hitl.py::test_hitl_approval_accepts_mailto_approver_id PASSED [ 98%] +tests/test_delegation_hitl.py::test_hitl_approval_accepts_did_approver_id PASSED [100%] + +============================== warnings summary =============================== +..\..\..\venv\Lib\site-packages\_pytest\config\__init__.py:1474 + \Lib\site-packages\_pytest\config\__init__.py:1474: PytestConfigWarning: Unknown config option: asyncio_mode + + self._warn_or_fail_if_strict(f"Unknown config option: {key}\n") + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +======================== 74 passed, 1 warning in 0.44s ======================== + +STDERR diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/boundary-probes.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/boundary-probes.json new file mode 100644 index 0000000..c821286 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/boundary-probes.json @@ -0,0 +1,61 @@ +{ + "captured_at": "2026-09-28T04:55:58+00:00", + "results": [ + { + "probe": "export_failure_preserves_accepted_event", + "observed": true, + "meaning": "Evidence acceptance survives a later operational projection error; memory mode is not durable storage." + }, + { + "probe": "failed_ack_suppresses_projection", + "observed": true, + "meaning": "No operational projection or local chain advance after callback refusal." + }, + { + "probe": "empty_complete_declaration_is_accepted", + "observed": true, + "meaning": "Counterexample: seal does not independently prove capture completeness." + }, + { + "probe": "external_commit_retry_requires_sink_idempotency", + "observed": true, + "meaning": "Counterexample: no cross-store transaction or built-in exactly-once persistence." + }, + { + "probe": "signed_bad_memory_is_authentic_not_acceptable", + "observed": true, + "meaning": "Signature verification authenticates the negative finding. The broker must reject it separately." + }, + { + "probe": "signature_verifier_has_no_freshness_input", + "observed": true, + "meaning": "Broker challenge comparison, not signature verification alone, establishes request freshness." + }, + { + "probe": "approval_signer_accepts_method_parameter", + "observed": false, + "meaning": "Whether this revision lets the human signer cover approval_method; the probe passes it when available." + }, + { + "probe": "approval_method_label_is_outside_approval_signature", + "observed": true, + "meaning": "True means the unaltered approval verifies and the method-label alteration still verifies (label outside the signature). False with the unaltered approval verifying means the label is bound. Not a full runtime test." + }, + { + "probe": "approval_unaltered_verifies", + "observed": true, + "meaning": "Control for the previous probe." + }, + { + "probe": "approval_helper_accepts_zero_or_missing_duration", + "observed": true, + "meaning": "True means the single-approval helper accepts approval_duration_seconds of 0 or absent. The integrated verifier applies its own expiry logic." + }, + { + "probe": "requirement_bound_approval_appendable", + "observed": true, + "meaning": "Issuer pre-image preserves requirements while normalizing only approvals." + } + ], + "limits": "Single-process software probes; no hardware or legal conclusion." +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/memory-tests.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/memory-tests.txt new file mode 100644 index 0000000..2f18b75 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/memory-tests.txt @@ -0,0 +1,45 @@ +============================= test session starts ============================= +platform win32 -- Python 3.12.10, pytest-8.4.2, pluggy-1.6.0 -- \Scripts\python.exe +rootdir: \weight-custody-manifest\python +configfile: pyproject.toml +collecting ... collected 35 items + +tests/test_memory_sweep.py::test_real_full_range_write_read_transcript_verifies PASSED [ 2%] +tests/test_memory_sweep.py::test_controlled_aliased_mapping_is_detected PASSED [ 5%] +tests/test_memory_sweep.py::test_omitted_range_and_host_tampering_are_rejected PASSED [ 8%] +tests/test_memory_sweep.py::test_host_authored_result_signed_by_wrong_key_is_rejected PASSED [ 11%] +tests/test_memory_sweep.py::test_short_secret_and_invalid_range_fail_closed PASSED [ 14%] +tests/test_kbs.py::test_happy_path_releases_key PASSED [ 17%] +tests/test_kbs.py::test_release_denies_when_manifest_has_no_out_of_band_authorization PASSED [ 20%] +tests/test_kbs.py::test_attacker_manifest_for_same_weights_is_rejected_by_exact_pin PASSED [ 22%] +tests/test_kbs.py::test_replayed_nonce_denied PASSED [ 25%] +tests/test_kbs.py::test_wrong_platform_denied PASSED [ 28%] +tests/test_kbs.py::test_unknown_serving_image_denied PASSED [ 31%] +tests/test_kbs.py::test_revoked_serving_image_denied PASSED [ 34%] +tests/test_kbs.py::test_prefer_current_refuses_retiring PASSED [ 37%] +tests/test_kbs.py::test_retiring_past_retire_after_denied PASSED [ 40%] +tests/test_kbs.py::test_gpu_absent_denied PASSED [ 42%] +tests/test_kbs.py::test_gpu_binding_broken_denied PASSED [ 45%] +tests/test_kbs.py::test_gpu_measurement_mismatch_denied PASSED [ 48%] +tests/test_kbs.py::test_attestation_key_revoked_denied PASSED [ 51%] +tests/test_kbs.py::test_stale_revocation_cache_denied PASSED [ 54%] +tests/test_kbs.py::test_no_key_for_weights_hash_denied PASSED [ 57%] +tests/test_kbs.py::test_memory_fingerprint_required_but_absent_denied PASSED [ 60%] +tests/test_kbs.py::test_memory_fingerprint_aliasing_denied PASSED [ 62%] +tests/test_kbs.py::test_memory_fingerprint_clean_releases PASSED [ 65%] +tests/test_kbs.py::test_memory_fingerprint_host_authored_and_replayed_results_are_denied PASSED [ 68%] +tests/test_kbs.py::test_channel_binding_off_by_default_returns_raw_key PASSED [ 71%] +tests/test_kbs.py::test_channel_binding_required_seals_key_to_enclave PASSED [ 74%] +tests/test_kbs.py::test_channel_binding_required_but_absent_denied PASSED [ 77%] +tests/test_kbs.py::test_channel_binding_malformed_transport_key_denied PASSED [ 80%] +tests/test_kbs.py::test_naive_retire_after_is_read_as_utc_and_does_not_raise PASSED [ 82%] +tests/test_kbs.py::test_naive_retire_after_still_expires PASSED [ 85%] +tests/test_kbs.py::test_aware_retire_after_keeps_its_offset PASSED [ 88%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[soon] PASSED [ 91%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[] PASSED [ 94%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[2026-13-45T99:99:99] PASSED [ 97%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[next tuesday] PASSED [100%] + +============================= 35 passed in 0.19s ============================== + +STDERR diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/results.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/results.json new file mode 100644 index 0000000..41f5b3d --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/results.json @@ -0,0 +1,107 @@ +{ + "captured_at": "2026-09-28T04:55:58+00:00", + "python": "3.12.10", + "platform": "Windows-11", + "dependencies": { + "pytest": "8.4.2", + "cryptography": "50.0.0", + "pydantic": "2.13.4", + "jsonschema": "4.26.0", + "referencing": "0.37.0", + "rfc8785": "0.1.4", + "cbor2": "5.9.0", + "agentrust-trace": "0.9.0", + "opentelemetry-sdk": "1.44.0", + "jcs": "0.2.1", + "PyYAML": "6.0.3" + }, + "sources": { + "agentrust-telemetry": { + "commit": "29c13cfa4047a0f5784a90df63b22a3568453158", + "verified_by": "git archive of the pinned commit (tar commit id checked)" + }, + "weight-custody-manifest": { + "commit": "2acedfa883204f7d1fd457d3ee603198d85be82c", + "verified_by": "git archive of the pinned commit (tar commit id checked)" + }, + "agent-manifest": { + "commit": "934809709a2815695d65cfacb45dc0a164286046", + "verified_by": "git archive of the pinned commit (tar commit id checked)" + } + }, + "results": [ + { + "name": "telemetry-tests", + "cwd": "agentrust-telemetry", + "command": [ + "python", + "-m", + "unittest", + "discover", + "-s", + "tests", + "-v" + ], + "exit_code": 0, + "summary": "111 tests, OK", + "log": "telemetry-tests.txt", + "log_sha256": "e68fd132e85776a347a4f6401b9ce9c60d48947b2cf81e59bce03d8c8a43fc50" + }, + { + "name": "telemetry-conformance", + "cwd": "agentrust-telemetry", + "command": [ + "python", + "conformance/runner/validate.py" + ], + "exit_code": 0, + "summary": "13 of 13 fixtures returned the expected verdict", + "log": "telemetry-conformance.txt", + "log_sha256": "f1af2ff732a78ef035226e1190421fcb67881644ff2936b5768131d08ac8e3ab" + }, + { + "name": "memory-tests", + "cwd": "weight-custody-manifest/python", + "command": [ + "python", + "-m", + "pytest", + "-p", + "no:cacheprovider", + "-o", + "pythonpath=src", + "tests/test_memory_sweep.py", + "tests/test_kbs.py", + "-v" + ], + "exit_code": 0, + "summary": "35 passed", + "log": "memory-tests.txt", + "log_sha256": "c8ec45488f99012cee49b452376d00cd01f0bf28bea3793bd8e7b7a2606f6366" + }, + { + "name": "approval-tests", + "cwd": "agent-manifest/python", + "command": [ + "python", + "-m", + "pytest", + "-p", + "no:cacheprovider", + "-o", + "pythonpath=src", + "tests/test_signing.py", + "tests/test_delegation_hitl.py", + "-v" + ], + "exit_code": 0, + "summary": "74 passed, 1 warning", + "log": "approval-tests.txt", + "log_sha256": "56a0b9a6a7668cb05cfd91077e5599e61b36b72d2abc9eb6cbccce5506352d5c" + } + ], + "limits": [ + "Local software tests only; no protected-hardware execution.", + "Tests and implementation share a repository and are not independent validation." + ] +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/telemetry-conformance.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/telemetry-conformance.txt new file mode 100644 index 0000000..6fd65da --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/telemetry-conformance.txt @@ -0,0 +1,15 @@ +PASS conformance\fixtures\valid\action.json +PASS conformance\fixtures\valid\approval.json +PASS conformance\fixtures\valid\data-flow.json +PASS conformance\fixtures\valid\evidence.json +PASS conformance\fixtures\valid\policy-decision.json +PASS conformance\fixtures\valid\usage.json +PASS conformance\fixtures\invalid\action-without-digest.json +PASS conformance\fixtures\invalid\aggregate-cost-without-rollup.json +PASS conformance\fixtures\invalid\model-call-with-rollup.json +PASS conformance\fixtures\invalid\numeric-timestamp.json +PASS conformance\fixtures\invalid\raw-content.json +PASS conformance\fixtures\invalid\usage-without-measurement.json +PASS conformance\fixtures\invalid\zero-trace-id.json + +STDERR diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/telemetry-tests.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/telemetry-tests.txt new file mode 100644 index 0000000..3ee54c5 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-09-27/pinned/telemetry-tests.txt @@ -0,0 +1,135 @@ +PASS conformance\fixtures\valid\action.json +PASS conformance\fixtures\valid\approval.json +PASS conformance\fixtures\valid\data-flow.json +PASS conformance\fixtures\valid\evidence.json +PASS conformance\fixtures\valid\policy-decision.json +PASS conformance\fixtures\valid\usage.json +PASS conformance\fixtures\invalid\action-without-digest.json +PASS conformance\fixtures\invalid\aggregate-cost-without-rollup.json +PASS conformance\fixtures\invalid\model-call-with-rollup.json +PASS conformance\fixtures\invalid\numeric-timestamp.json +PASS conformance\fixtures\invalid\raw-content.json +PASS conformance\fixtures\invalid\usage-without-measurement.json +PASS conformance\fixtures\invalid\zero-trace-id.json +PASS contract=0.1.0-alpha.2 python=0.1.0a2 npm_name=@agentrust-io/telemetry npm=0.1.0-alpha.2 +PASS OTel GenAI matrix covers 16 span attributes, 8 metrics, and 6 event families +PASS TypeScript schemas match normative bytes + +STDERR +test_cedar_preserves_final_decision_and_reports_skipped_errors (test_adapters.AdapterTests.test_cedar_preserves_final_decision_and_reports_skipped_errors) ... ok +test_cedar_rejects_error_messages_as_codes (test_adapters.AdapterTests.test_cedar_rejects_error_messages_as_codes) ... ok +test_cedar_rejects_string_as_identifier_collection (test_adapters.AdapterTests.test_cedar_rejects_string_as_identifier_collection) ... ok +test_factory_builds_and_validates_any_supported_family (test_adapters.AdapterTests.test_factory_builds_and_validates_any_supported_family) ... ok +test_factory_matches_cross_language_golden_event (test_adapters.AdapterTests.test_factory_matches_cross_language_golden_event) ... ok +test_factory_prevents_envelope_override (test_adapters.AdapterTests.test_factory_prevents_envelope_override) ... ok +test_factory_serializes_nanoseconds_without_precision_loss (test_adapters.AdapterTests.test_factory_serializes_nanoseconds_without_precision_loss) ... ok +test_opa_boolean_log_maps_without_copying_input_or_result (test_adapters.AdapterTests.test_opa_boolean_log_maps_without_copying_input_or_result) ... ok +test_opa_complex_result_requires_explicit_mapper (test_adapters.AdapterTests.test_opa_complex_result_requires_explicit_mapper) ... ok +test_opa_rejects_malformed_source_collections (test_adapters.AdapterTests.test_opa_rejects_malformed_source_collections) ... ok +test_policy_event_maps_challenge_and_excludes_free_form_content (test_agt_adapter.AgtAdapterTests.test_policy_event_maps_challenge_and_excludes_free_form_content) ... ok +test_policy_mapper_rejects_ambiguous_decision_and_reason_message (test_agt_adapter.AgtAdapterTests.test_policy_mapper_rejects_ambiguous_decision_and_reason_message) ... ok +test_sink_prevalidates_whole_batch_before_emission (test_agt_adapter.AgtAdapterTests.test_sink_prevalidates_whole_batch_before_emission) ... ok +test_sink_uses_source_result_sentinels_and_reports_projection_failure (test_agt_adapter.AgtAdapterTests.test_sink_uses_source_result_sentinels_and_reports_projection_failure) ... ok +test_bindings_reject_fields_that_are_absent_on_both_records (test_agt_approval.AgtApprovalAdapterTests.test_bindings_reject_fields_that_are_absent_on_both_records) ... ok +test_protocol_chain_preserves_policy_request_and_resolution_links (test_agt_approval.AgtApprovalAdapterTests.test_protocol_chain_preserves_policy_request_and_resolution_links) ... ok +test_request_rejects_wrong_policy_action_and_chain (test_agt_approval.AgtApprovalAdapterTests.test_request_rejects_wrong_policy_action_and_chain) ... ok +test_resolution_rejects_pre_request_time_and_malformed_chain_digest (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_pre_request_time_and_malformed_chain_digest) ... ok +test_resolution_rejects_wrong_request_and_wrong_action (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_wrong_request_and_wrong_action) ... ok +test_terminal_outcomes_map_without_treating_votes_as_resolutions (test_agt_approval.AgtApprovalAdapterTests.test_terminal_outcomes_map_without_treating_votes_as_resolutions) ... ok +test_action_audit_requires_full_action_digest_and_computes_duration (test_agt_audit.AgtAuditAdapterTests.test_action_audit_requires_full_action_digest_and_computes_duration) ... ok +test_action_rejects_missing_or_reversed_timing (test_agt_audit.AgtAuditAdapterTests.test_action_rejects_missing_or_reversed_timing) ... ok +test_blocked_action_is_denied_even_if_source_outcome_is_success (test_agt_audit.AgtAuditAdapterTests.test_blocked_action_is_denied_even_if_source_outcome_is_success) ... ok +test_policy_audit_maps_without_free_form_data_or_resource (test_agt_audit.AgtAuditAdapterTests.test_policy_audit_maps_without_free_form_data_or_resource) ... ok +test_action_payload_field_is_rejected_by_schema_and_privacy_gate (test_conformance.ConformanceTests.test_action_payload_field_is_rejected_by_schema_and_privacy_gate) ... ok +test_all_schemas_are_valid_draft_2020_12 (test_conformance.ConformanceTests.test_all_schemas_are_valid_draft_2020_12) ... ok +test_committed_suite_has_expected_verdicts (test_conformance.ConformanceTests.test_committed_suite_has_expected_verdicts) ... ok +test_nested_prohibited_key_is_rejected_independently (test_conformance.ConformanceTests.test_nested_prohibited_key_is_rejected_independently) ... ok +test_unknown_event_type_is_rejected (test_conformance.ConformanceTests.test_unknown_event_type_is_rejected) ... ok +test_uppercase_trace_id_is_rejected (test_conformance.ConformanceTests.test_uppercase_trace_id_is_rejected) ... ok +test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason (test_data_flow.DataFlowTests.test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason) ... ok +test_classifier_cannot_return_arbitrary_metadata (test_data_flow.DataFlowTests.test_classifier_cannot_return_arbitrary_metadata) ... ok +test_classifier_sees_content_but_event_contains_only_closed_metadata (test_data_flow.DataFlowTests.test_classifier_sees_content_but_event_contains_only_closed_metadata) ... ok +test_metadata_identifiers_reject_prose_paths_and_query_values (test_data_flow.DataFlowTests.test_metadata_identifiers_reject_prose_paths_and_query_values) ... ok +test_callback_and_returned_entry_cannot_mutate_retained_evidence (test_evidence.EvidenceTests.test_callback_and_returned_entry_cannot_mutate_retained_evidence) ... ok +test_chain_is_ordered_and_has_stable_golden_digest (test_evidence.EvidenceTests.test_chain_is_ordered_and_has_stable_golden_digest) ... ok +test_client_does_not_project_when_evidence_fails (test_evidence.EvidenceTests.test_client_does_not_project_when_evidence_fails) ... ok +test_client_isolates_operational_event_from_evidence_sink (test_evidence.EvidenceTests.test_client_isolates_operational_event_from_evidence_sink) ... ok +test_client_reports_evidence_persistence (test_evidence.EvidenceTests.test_client_reports_evidence_persistence) ... ok +test_context_mismatch_never_enters_evidence (test_evidence.EvidenceTests.test_context_mismatch_never_enters_evidence) ... ok +test_durable_callback_acknowledges_before_local_acceptance (test_evidence.EvidenceTests.test_durable_callback_acknowledges_before_local_acceptance) ... ok +test_failed_durable_ack_is_fail_closed_and_retryable (test_evidence.EvidenceTests.test_failed_durable_ack_is_fail_closed_and_retryable) ... ok +test_non_finite_number_cannot_enter_digest_chain (test_evidence.EvidenceTests.test_non_finite_number_cannot_enter_digest_chain) ... ok +test_rejects_cross_run_duplicate_overflow_and_append_after_seal (test_evidence.EvidenceTests.test_rejects_cross_run_duplicate_overflow_and_append_after_seal) ... ok +test_snapshot_is_defensive_and_unsealed_never_claims_complete (test_evidence.EvidenceTests.test_snapshot_is_defensive_and_unsealed_never_claims_complete) ... ok +test_action_metrics_exclude_action_and_agent_identifiers (test_otel_projectors.OTelProjectorTests.test_action_metrics_exclude_action_and_agent_identifiers) ... ok +test_classification_allowlist_is_bounded (test_otel_projectors.OTelProjectorTests.test_classification_allowlist_is_bounded) ... ok +test_classification_metric_is_allowlisted_or_collapsed (test_otel_projectors.OTelProjectorTests.test_classification_metric_is_allowlisted_or_collapsed) ... ok +test_metric_failure_is_reported_independently (test_otel_projectors.OTelProjectorTests.test_metric_failure_is_reported_independently) ... ok +test_real_metrics_record_usage_without_high_cardinality_ids (test_otel_projectors.OTelProjectorTests.test_real_metrics_record_usage_without_high_cardinality_ids) ... ok +test_real_otel_log_preserves_event_and_active_trace_context (test_otel_projectors.OTelProjectorTests.test_real_otel_log_preserves_event_and_active_trace_context) ... ok +test_async_handoff_creates_link_without_forcing_parentage (test_propagation.PropagationTests.test_async_handoff_creates_link_without_forcing_parentage) ... ok +test_header_lookup_is_case_insensitive (test_propagation.PropagationTests.test_header_lookup_is_case_insensitive) ... ok +test_injection_rejects_header_injection (test_propagation.PropagationTests.test_injection_rejects_header_injection) ... ok +test_link_requires_valid_w3c_context (test_propagation.PropagationTests.test_link_requires_valid_w3c_context) ... ok +test_sync_handoff_preserves_trace_and_agent_parent (test_propagation.PropagationTests.test_sync_handoff_preserves_trace_and_agent_parent) ... ok +test_complete_governed_workflow (test_reference_scenario.ReferenceScenarioTests.test_complete_governed_workflow) ... ok +test_contract_version_rejects_unsupported_spellings (test_repository_gates.RepositoryGateTests.test_contract_version_rejects_unsupported_spellings) ... ok +test_contract_versions_map_to_ecosystem_spellings (test_repository_gates.RepositoryGateTests.test_contract_versions_map_to_ecosystem_spellings) ... ok +test_declared_versions_are_consistent (test_repository_gates.RepositoryGateTests.test_declared_versions_are_consistent) ... ok +test_issue_templates_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_issue_templates_are_valid_yaml_when_parser_available) ... ok +test_npm_dist_tag_matches_the_declared_contract (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_matches_the_declared_contract) ... ok +test_npm_dist_tag_never_moves_latest_for_a_prerelease (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_never_moves_latest_for_a_prerelease) ... ok +test_npm_dist_tag_rejects_an_unsupported_version (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_rejects_an_unsupported_version) ... ok +test_otel_matrix_gate_detects_attribute_metric_and_family_drift (test_repository_gates.RepositoryGateTests.test_otel_matrix_gate_detects_attribute_metric_and_family_drift) ... ok +test_otel_matrix_matches_shipped_projection (test_repository_gates.RepositoryGateTests.test_otel_matrix_matches_shipped_projection) ... ok +test_release_tag_must_match_contract (test_repository_gates.RepositoryGateTests.test_release_tag_must_match_contract) ... ok +test_schema_gate_detects_changed_bytes (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_changed_bytes) ... ok +test_schema_gate_detects_missing_and_extra_files (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_missing_and_extra_files) ... ok +test_source_manifest_includes_otel_matrix (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_otel_matrix) ... ok +test_source_manifest_includes_release_policy (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_release_policy) ... ok +test_typescript_schemas_match_normative_bytes (test_repository_gates.RepositoryGateTests.test_typescript_schemas_match_normative_bytes) ... ok +test_workflows_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_workflows_are_valid_yaml_when_parser_available) ... ok +test_bundled_schemas_match_normative_schemas (test_sdk.SdkTests.test_bundled_schemas_match_normative_schemas) ... ok +test_context_mismatch_rejects_before_projection (test_sdk.SdkTests.test_context_mismatch_rejects_before_projection) ... ok +test_emits_correlated_span_event_and_log (test_sdk.SdkTests.test_emits_correlated_span_event_and_log) ... ok +test_explicit_attribute_allowlist_is_narrow (test_sdk.SdkTests.test_explicit_attribute_allowlist_is_narrow) ... ok +test_extension_attributes_are_deny_by_default (test_sdk.SdkTests.test_extension_attributes_are_deny_by_default) ... ok +test_log_body_is_a_deep_copy (test_sdk.SdkTests.test_log_body_is_a_deep_copy) ... ok +test_no_span_still_emits_run_correlated_log (test_sdk.SdkTests.test_no_span_still_emits_run_correlated_log) ... ok +test_privacy_violation_rejects_before_projection (test_sdk.SdkTests.test_privacy_violation_rejects_before_projection) ... ok +test_projection_failures_are_reported_independently (test_sdk.SdkTests.test_projection_failures_are_reported_independently) ... ok +test_real_otel_span_receives_event (test_sdk.SdkTests.test_real_otel_span_receives_event) ... ok +test_action_events_create_truthful_tool_transcript (test_trace_adapter.TraceAdapterTests.test_action_events_create_truthful_tool_transcript) ... ok +test_finalizes_signs_and_validates_official_trace_record (test_trace_adapter.TraceAdapterTests.test_finalizes_signs_and_validates_official_trace_record) ... ok +test_linked_terminal_approval_resolves_policy_challenge (test_trace_adapter.TraceAdapterTests.test_linked_terminal_approval_resolves_policy_challenge) ... ok +test_refuses_conflicting_policy_bundles (test_trace_adapter.TraceAdapterTests.test_refuses_conflicting_policy_bundles) ... ok +test_refuses_missing_signer (test_trace_adapter.TraceAdapterTests.test_refuses_missing_signer) ... ok +test_refuses_policy_digest_algorithm_trace_cannot_represent (test_trace_adapter.TraceAdapterTests.test_refuses_policy_digest_algorithm_trace_cannot_represent) ... ok +test_refuses_unranked_classification (test_trace_adapter.TraceAdapterTests.test_refuses_unranked_classification) ... ok +test_refuses_unsealed_or_incomplete_evidence (test_trace_adapter.TraceAdapterTests.test_refuses_unsealed_or_incomplete_evidence) ... ok +test_unbound_or_expired_approval_cannot_resolve_policy_challenge (test_trace_adapter.TraceAdapterTests.test_unbound_or_expired_approval_cannot_resolve_policy_challenge) ... ok +test_a_failure_inside_official_signing_is_reported_as_one (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_failure_inside_official_signing_is_reported_as_one) +A key that satisfies the shape check and then fails when used. ... ok +test_a_policy_decision_without_a_bundle_digest_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_policy_decision_without_a_bundle_digest_is_refused) ... ok +test_a_repeated_classification_cannot_rank_anything (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_repeated_classification_cannot_rank_anything) ... ok +test_a_run_with_no_data_flow_has_nothing_to_classify (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_data_flow_has_nothing_to_classify) ... ok +test_a_run_with_no_policy_decision_has_no_policy_to_bind (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_policy_decision_has_no_policy_to_bind) ... ok +test_a_taxonomy_the_configuration_does_not_name_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_taxonomy_the_configuration_does_not_name_is_refused) ... ok +test_an_empty_run_carries_no_chained_evidence (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_empty_run_carries_no_chained_evidence) ... ok +test_every_required_configuration_field_is_named_when_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_every_required_configuration_field_is_named_when_absent) ... ok +test_the_optional_dependency_is_named_when_it_is_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_the_optional_dependency_is_named_when_it_is_absent) ... ok +test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode) ... ok +test_accumulator_rejects_non_finite_external_cost (test_usage.UsageTests.test_accumulator_rejects_non_finite_external_cost) ... ok +test_bool_is_not_accepted_as_token_integer (test_usage.UsageTests.test_bool_is_not_accepted_as_token_integer) ... ok +test_cost_requires_finite_amount_and_explicit_provenance (test_usage.UsageTests.test_cost_requires_finite_amount_and_explicit_provenance) ... ok +test_reused_event_id_with_changed_data_is_rejected (test_usage.UsageTests.test_reused_event_id_with_changed_data_is_rejected) ... ok +test_rollup_deduplicates_and_reports_partial_coverage (test_usage.UsageTests.test_rollup_deduplicates_and_reports_partial_coverage) ... ok +test_rollup_rejects_non_leaf_and_mixed_currency (test_usage.UsageTests.test_rollup_rejects_non_leaf_and_mixed_currency) ... ok +test_rollups_match_shared_cross_language_vector (test_usage.UsageTests.test_rollups_match_shared_cross_language_vector) ... ok +test_schema_rejects_aggregate_cost_without_run_rollup (test_usage.UsageTests.test_schema_rejects_aggregate_cost_without_run_rollup) ... ok +test_unknown_price_is_absent_not_zero (test_usage.UsageTests.test_unknown_price_is_absent_not_zero) ... ok + +---------------------------------------------------------------------- +Ran 111 tests in 2.793s + +OK diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/README.md b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/README.md new file mode 100644 index 0000000..914e44f --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/README.md @@ -0,0 +1,24 @@ +# October 1, 2026 rerun + +Fresh virtual environment, Python 3.12.10 on Windows 11, with the package versions recorded in +`pinned/results.json` (the same set as the September 27 rerun: cryptography 50.0.0, +jsonschema 4.26.0, cbor2 5.9.0, agentrust-trace 0.9.0, opentelemetry-sdk 1.44.0). + +- `pinned/`: `reproduce.py --prepare-from` against fresh clones, so every source is a + `git archive` extract of its pin with the tar commit id checked, then `boundary_probes.py`. + agentrust-telemetry 111 tests OK; 13 of 13 fixtures; memory tests 35 passed; approval tests + 74 passed. The four telemetry logs match the September 27 logs except for timings. All + eleven probes match September 27. +- `later-revision/`: `boundary_probes.py` against the commits in + `../rerun-2026-09-27/later-revision/COMMITS.txt`. All eleven probes match September 27. +- `current-main/`: `boundary_probes.py` against origin/main on October 1 (agentrust-telemetry + c86753cad6bbe9f00c5080489a12e0f94df34899, weight-custody-manifest + 5ef8c122479f5df79ad8a7c04b70f4a43bbdb313, agent-manifest + fed9aeb091e4c77e0b40e12e959bee30b52e3c75). The four telemetry probes this paper relies on + give the same result as at the pin. + +Also checked on October 1 and recorded in +`../../../agentrust-telemetry/experiments/results/rerun-2026-10-01/`: the TypeScript suite at +the pin (41 of 41 pass), and the TRACE finalizer defect fixed in agentrust-telemetry PR 65 +(the pinned Python and TypeScript finalizers sign an edited, truncated or digest-substituted +snapshot). diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/current-main/boundary-probes.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/current-main/boundary-probes.json new file mode 100644 index 0000000..8579f52 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/current-main/boundary-probes.json @@ -0,0 +1,61 @@ +{ + "captured_at": "2026-10-01T18:46:45+00:00", + "results": [ + { + "probe": "export_failure_preserves_accepted_event", + "observed": true, + "meaning": "Evidence acceptance survives a later operational projection error; memory mode is not durable storage." + }, + { + "probe": "failed_ack_suppresses_projection", + "observed": true, + "meaning": "No operational projection or local chain advance after callback refusal." + }, + { + "probe": "empty_complete_declaration_is_accepted", + "observed": true, + "meaning": "Counterexample: seal does not independently prove capture completeness." + }, + { + "probe": "external_commit_retry_requires_sink_idempotency", + "observed": true, + "meaning": "Counterexample: no cross-store transaction or built-in exactly-once persistence." + }, + { + "probe": "signed_bad_memory_is_authentic_not_acceptable", + "observed": true, + "meaning": "Signature verification authenticates the negative finding. The broker must reject it separately." + }, + { + "probe": "signature_verifier_has_no_freshness_input", + "observed": true, + "meaning": "Broker challenge comparison, not signature verification alone, establishes request freshness." + }, + { + "probe": "approval_signer_accepts_method_parameter", + "observed": true, + "meaning": "Whether this revision lets the human signer cover approval_method; the probe passes it when available." + }, + { + "probe": "approval_method_label_is_outside_approval_signature", + "observed": false, + "meaning": "True means the unaltered approval verifies and the method-label alteration still verifies (label outside the signature). False with the unaltered approval verifying means the label is bound. Not a full runtime test." + }, + { + "probe": "approval_unaltered_verifies", + "observed": true, + "meaning": "Control for the previous probe." + }, + { + "probe": "approval_helper_accepts_zero_or_missing_duration", + "observed": false, + "meaning": "True means the single-approval helper accepts approval_duration_seconds of 0 or absent. The integrated verifier applies its own expiry logic." + }, + { + "probe": "requirement_bound_approval_appendable", + "observed": true, + "meaning": "Issuer pre-image preserves requirements while normalizing only approvals." + } + ], + "limits": "Single-process software probes; no hardware or legal conclusion." +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/later-revision/boundary-probes.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/later-revision/boundary-probes.json new file mode 100644 index 0000000..93016b9 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/later-revision/boundary-probes.json @@ -0,0 +1,61 @@ +{ + "captured_at": "2026-10-01T18:46:42+00:00", + "results": [ + { + "probe": "export_failure_preserves_accepted_event", + "observed": true, + "meaning": "Evidence acceptance survives a later operational projection error; memory mode is not durable storage." + }, + { + "probe": "failed_ack_suppresses_projection", + "observed": true, + "meaning": "No operational projection or local chain advance after callback refusal." + }, + { + "probe": "empty_complete_declaration_is_accepted", + "observed": true, + "meaning": "Counterexample: seal does not independently prove capture completeness." + }, + { + "probe": "external_commit_retry_requires_sink_idempotency", + "observed": true, + "meaning": "Counterexample: no cross-store transaction or built-in exactly-once persistence." + }, + { + "probe": "signed_bad_memory_is_authentic_not_acceptable", + "observed": true, + "meaning": "Signature verification authenticates the negative finding. The broker must reject it separately." + }, + { + "probe": "signature_verifier_has_no_freshness_input", + "observed": true, + "meaning": "Broker challenge comparison, not signature verification alone, establishes request freshness." + }, + { + "probe": "approval_signer_accepts_method_parameter", + "observed": true, + "meaning": "Whether this revision lets the human signer cover approval_method; the probe passes it when available." + }, + { + "probe": "approval_method_label_is_outside_approval_signature", + "observed": false, + "meaning": "True means the unaltered approval verifies and the method-label alteration still verifies (label outside the signature). False with the unaltered approval verifying means the label is bound. Not a full runtime test." + }, + { + "probe": "approval_unaltered_verifies", + "observed": true, + "meaning": "Control for the previous probe." + }, + { + "probe": "approval_helper_accepts_zero_or_missing_duration", + "observed": false, + "meaning": "True means the single-approval helper accepts approval_duration_seconds of 0 or absent. The integrated verifier applies its own expiry logic." + }, + { + "probe": "requirement_bound_approval_appendable", + "observed": true, + "meaning": "Issuer pre-image preserves requirements while normalizing only approvals." + } + ], + "limits": "Single-process software probes; no hardware or legal conclusion." +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/approval-tests.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/approval-tests.txt new file mode 100644 index 0000000..e061b16 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/approval-tests.txt @@ -0,0 +1,91 @@ +============================= test session starts ============================= +platform win32 -- Python 3.12.10, pytest-8.4.2, pluggy-1.6.0 -- \Scripts\python.exe +rootdir: \agent-manifest\python +configfile: pyproject.toml +collecting ... collected 74 items + +tests/test_signing.py::test_pre_image_excludes_attestation PASSED [ 1%] +tests/test_signing.py::test_pre_image_excludes_signature PASSED [ 2%] +tests/test_signing.py::test_pre_image_includes_all_signed_fields PASSED [ 4%] +tests/test_signing.py::test_pre_image_is_rfc8785 PASSED [ 5%] +tests/test_signing.py::test_ed25519_sign_verify_roundtrip PASSED [ 6%] +tests/test_signing.py::test_ed25519_wrong_message_fails PASSED [ 8%] +tests/test_signing.py::test_ed25519_wrong_key_fails PASSED [ 9%] +tests/test_signing.py::test_ed25519_truncated_signature_fails PASSED [ 10%] +tests/test_signing.py::test_ed25519_wrong_algorithm_label_not_accepted PASSED [ 12%] +tests/test_signing.py::test_ed25519_signed_fields_list_correct PASSED [ 13%] +tests/test_signing.py::test_ed25519_sign_block_validates_against_model PASSED [ 14%] +tests/test_signing.py::test_ed25519_small_order_key_rejected PASSED [ 16%] +tests/test_signing.py::test_ed25519_all_small_order_points_rejected PASSED [ 17%] +tests/test_signing.py::test_ed25519_truncated_sig_raises_invalid_signature PASSED [ 18%] +tests/test_signing.py::test_b64url_decode_rejects_plus PASSED [ 20%] +tests/test_signing.py::test_b64url_decode_rejects_slash PASSED [ 21%] +tests/test_signing.py::test_b64url_decode_accepts_valid PASSED [ 22%] +tests/test_signing.py::test_ed25519_public_key_roundtrip_b64url PASSED [ 24%] +tests/test_signing.py::test_ml_dsa65_sign_verify_roundtrip PASSED [ 25%] +tests/test_signing.py::test_ml_dsa65_wrong_message_fails PASSED [ 27%] +tests/test_signing.py::test_hybrid_sign_verify_roundtrip PASSED [ 28%] +tests/test_signing.py::test_hybrid_tampered_classical_fails PASSED [ 29%] +tests/test_signing.py::test_hybrid_tampered_pq_fails PASSED [ 31%] +tests/test_signing.py::test_hybrid_both_components_cover_same_pre_image PASSED [ 32%] +tests/test_signing.py::test_signed_fields_match_spec_coverage_table PASSED [ 33%] +tests/test_signing.py::test_pre_image_normalizes_hitl_approvals_to_empty PASSED [ 35%] +tests/test_signing.py::test_pre_image_keeps_hitl_required_tamper_evident PASSED [ 36%] +tests/test_signing.py::test_approvals_attach_post_issuance_without_resigning PASSED [ 37%] +tests/test_signing.py::test_newly_signed_fields_are_tamper_evident PASSED [ 39%] +tests/test_signing.py::test_ed25519_keypair_repr_redacts_private_key PASSED [ 40%] +tests/test_signing.py::test_ed25519_keypair_str_equals_repr PASSED [ 41%] +tests/test_signing.py::test_ed25519_keypair_private_b64url_roundtrip PASSED [ 43%] +tests/test_signing.py::test_ed25519_from_private_bytes_valid PASSED [ 44%] +tests/test_signing.py::test_ed25519_from_private_bytes_malformed_raises PASSED [ 45%] +tests/test_signing.py::test_ed25519_from_private_bytes_too_short_raises PASSED [ 47%] +tests/test_signing.py::test_ed25519_verifier_wrong_length_key_raises PASSED [ 48%] +tests/test_signing.py::test_ed25519_verifier_zero_length_key_raises PASSED [ 50%] +tests/test_signing.py::test_ed25519_verifier_rsa_sized_bytes_raises PASSED [ 51%] +tests/test_signing.py::test_ed25519_signer_signed_at_is_iso8601 PASSED [ 52%] +tests/test_signing.py::test_ml_dsa65_signer_signed_at_is_iso8601 PASSED [ 54%] +tests/test_signing.py::test_hybrid_signer_signed_at_is_iso8601 PASSED [ 55%] +tests/test_delegation_hitl.py::test_hop_pre_image_includes_manifest_id PASSED [ 56%] +tests/test_delegation_hitl.py::test_hop_pre_image_includes_scope PASSED [ 58%] +tests/test_delegation_hitl.py::test_hop_pre_image_deterministic PASSED [ 59%] +tests/test_delegation_hitl.py::test_hop_pre_image_different_hops PASSED [ 60%] +tests/test_delegation_hitl.py::test_delegation_sign_verify_single_hop PASSED [ 62%] +tests/test_delegation_hitl.py::test_delegation_wrong_key_fails PASSED [ 63%] +tests/test_delegation_hitl.py::test_delegation_wrong_manifest_id_fails PASSED [ 64%] +tests/test_delegation_hitl.py::test_scope_laundering_detected PASSED [ 66%] +tests/test_delegation_hitl.py::test_depth_exceeded_raises PASSED [ 67%] +tests/test_delegation_hitl.py::test_empty_chain_passes PASSED [ 68%] +tests/test_delegation_hitl.py::test_missing_public_key_raises PASSED [ 70%] +tests/test_delegation_hitl.py::test_root_principal_must_match_manifest_issuer PASSED [ 71%] +tests/test_delegation_hitl.py::test_root_principal_matching_issuer_passes PASSED [ 72%] +tests/test_delegation_hitl.py::test_root_principal_matches_via_principal_manifest_id PASSED [ 74%] +tests/test_delegation_hitl.py::test_child_dropping_parent_constraint_is_rejected PASSED [ 75%] +tests/test_delegation_hitl.py::test_child_adding_constraint_is_allowed PASSED [ 77%] +tests/test_delegation_hitl.py::test_child_raising_ttl_is_rejected PASSED [ 78%] +tests/test_delegation_hitl.py::test_child_unbounded_ttl_under_bounded_parent_is_rejected PASSED [ 79%] +tests/test_delegation_hitl.py::test_child_raising_max_delegation_depth_is_rejected PASSED [ 81%] +tests/test_delegation_hitl.py::test_child_delegated_late_with_equal_ttl_outlives_parent_is_rejected PASSED [ 82%] +tests/test_delegation_hitl.py::test_child_delegated_late_with_shorter_ttl_still_within_parent_window_passes PASSED [ 83%] +tests/test_delegation_hitl.py::test_mixed_z_and_naive_delegated_at_still_refuses_on_absolute_expiry PASSED [ 85%] +tests/test_delegation_hitl.py::test_approval_pre_image_includes_manifest_id PASSED [ 86%] +tests/test_delegation_hitl.py::test_approval_pre_image_includes_scope PASSED [ 87%] +tests/test_delegation_hitl.py::test_approval_sign_verify PASSED [ 89%] +tests/test_delegation_hitl.py::test_approval_wrong_key_fails PASSED [ 90%] +tests/test_delegation_hitl.py::test_approval_wrong_manifest_id_fails PASSED [ 91%] +tests/test_delegation_hitl.py::test_approval_scope_change_fails PASSED [ 93%] +tests/test_delegation_hitl.py::test_approval_expired_raises PASSED [ 94%] +tests/test_delegation_hitl.py::test_approval_no_duration_does_not_expire PASSED [ 95%] +tests/test_delegation_hitl.py::test_hitl_approval_rejects_spiffe_approver_id PASSED [ 97%] +tests/test_delegation_hitl.py::test_hitl_approval_accepts_mailto_approver_id PASSED [ 98%] +tests/test_delegation_hitl.py::test_hitl_approval_accepts_did_approver_id PASSED [100%] + +============================== warnings summary =============================== +..\..\..\venv-pin\Lib\site-packages\_pytest\config\__init__.py:1474 + \Lib\site-packages\_pytest\config\__init__.py:1474: PytestConfigWarning: Unknown config option: asyncio_mode + + self._warn_or_fail_if_strict(f"Unknown config option: {key}\n") + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +======================== 74 passed, 1 warning in 1.41s ======================== + +STDERR diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/boundary-probes.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/boundary-probes.json new file mode 100644 index 0000000..3ddcfd8 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/boundary-probes.json @@ -0,0 +1,61 @@ +{ + "captured_at": "2026-10-01T18:46:08+00:00", + "results": [ + { + "probe": "export_failure_preserves_accepted_event", + "observed": true, + "meaning": "Evidence acceptance survives a later operational projection error; memory mode is not durable storage." + }, + { + "probe": "failed_ack_suppresses_projection", + "observed": true, + "meaning": "No operational projection or local chain advance after callback refusal." + }, + { + "probe": "empty_complete_declaration_is_accepted", + "observed": true, + "meaning": "Counterexample: seal does not independently prove capture completeness." + }, + { + "probe": "external_commit_retry_requires_sink_idempotency", + "observed": true, + "meaning": "Counterexample: no cross-store transaction or built-in exactly-once persistence." + }, + { + "probe": "signed_bad_memory_is_authentic_not_acceptable", + "observed": true, + "meaning": "Signature verification authenticates the negative finding. The broker must reject it separately." + }, + { + "probe": "signature_verifier_has_no_freshness_input", + "observed": true, + "meaning": "Broker challenge comparison, not signature verification alone, establishes request freshness." + }, + { + "probe": "approval_signer_accepts_method_parameter", + "observed": false, + "meaning": "Whether this revision lets the human signer cover approval_method; the probe passes it when available." + }, + { + "probe": "approval_method_label_is_outside_approval_signature", + "observed": true, + "meaning": "True means the unaltered approval verifies and the method-label alteration still verifies (label outside the signature). False with the unaltered approval verifying means the label is bound. Not a full runtime test." + }, + { + "probe": "approval_unaltered_verifies", + "observed": true, + "meaning": "Control for the previous probe." + }, + { + "probe": "approval_helper_accepts_zero_or_missing_duration", + "observed": true, + "meaning": "True means the single-approval helper accepts approval_duration_seconds of 0 or absent. The integrated verifier applies its own expiry logic." + }, + { + "probe": "requirement_bound_approval_appendable", + "observed": true, + "meaning": "Issuer pre-image preserves requirements while normalizing only approvals." + } + ], + "limits": "Single-process software probes; no hardware or legal conclusion." +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/memory-tests.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/memory-tests.txt new file mode 100644 index 0000000..306aa06 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/memory-tests.txt @@ -0,0 +1,45 @@ +============================= test session starts ============================= +platform win32 -- Python 3.12.10, pytest-8.4.2, pluggy-1.6.0 -- \Scripts\python.exe +rootdir: \weight-custody-manifest\python +configfile: pyproject.toml +collecting ... collected 35 items + +tests/test_memory_sweep.py::test_real_full_range_write_read_transcript_verifies PASSED [ 2%] +tests/test_memory_sweep.py::test_controlled_aliased_mapping_is_detected PASSED [ 5%] +tests/test_memory_sweep.py::test_omitted_range_and_host_tampering_are_rejected PASSED [ 8%] +tests/test_memory_sweep.py::test_host_authored_result_signed_by_wrong_key_is_rejected PASSED [ 11%] +tests/test_memory_sweep.py::test_short_secret_and_invalid_range_fail_closed PASSED [ 14%] +tests/test_kbs.py::test_happy_path_releases_key PASSED [ 17%] +tests/test_kbs.py::test_release_denies_when_manifest_has_no_out_of_band_authorization PASSED [ 20%] +tests/test_kbs.py::test_attacker_manifest_for_same_weights_is_rejected_by_exact_pin PASSED [ 22%] +tests/test_kbs.py::test_replayed_nonce_denied PASSED [ 25%] +tests/test_kbs.py::test_wrong_platform_denied PASSED [ 28%] +tests/test_kbs.py::test_unknown_serving_image_denied PASSED [ 31%] +tests/test_kbs.py::test_revoked_serving_image_denied PASSED [ 34%] +tests/test_kbs.py::test_prefer_current_refuses_retiring PASSED [ 37%] +tests/test_kbs.py::test_retiring_past_retire_after_denied PASSED [ 40%] +tests/test_kbs.py::test_gpu_absent_denied PASSED [ 42%] +tests/test_kbs.py::test_gpu_binding_broken_denied PASSED [ 45%] +tests/test_kbs.py::test_gpu_measurement_mismatch_denied PASSED [ 48%] +tests/test_kbs.py::test_attestation_key_revoked_denied PASSED [ 51%] +tests/test_kbs.py::test_stale_revocation_cache_denied PASSED [ 54%] +tests/test_kbs.py::test_no_key_for_weights_hash_denied PASSED [ 57%] +tests/test_kbs.py::test_memory_fingerprint_required_but_absent_denied PASSED [ 60%] +tests/test_kbs.py::test_memory_fingerprint_aliasing_denied PASSED [ 62%] +tests/test_kbs.py::test_memory_fingerprint_clean_releases PASSED [ 65%] +tests/test_kbs.py::test_memory_fingerprint_host_authored_and_replayed_results_are_denied PASSED [ 68%] +tests/test_kbs.py::test_channel_binding_off_by_default_returns_raw_key PASSED [ 71%] +tests/test_kbs.py::test_channel_binding_required_seals_key_to_enclave PASSED [ 74%] +tests/test_kbs.py::test_channel_binding_required_but_absent_denied PASSED [ 77%] +tests/test_kbs.py::test_channel_binding_malformed_transport_key_denied PASSED [ 80%] +tests/test_kbs.py::test_naive_retire_after_is_read_as_utc_and_does_not_raise PASSED [ 82%] +tests/test_kbs.py::test_naive_retire_after_still_expires PASSED [ 85%] +tests/test_kbs.py::test_aware_retire_after_keeps_its_offset PASSED [ 88%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[soon] PASSED [ 91%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[] PASSED [ 94%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[2026-13-45T99:99:99] PASSED [ 97%] +tests/test_kbs.py::test_unparseable_retire_after_fails_closed[next tuesday] PASSED [100%] + +============================= 35 passed in 0.53s ============================== + +STDERR diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/results.json b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/results.json new file mode 100644 index 0000000..6951dac --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/results.json @@ -0,0 +1,107 @@ +{ + "captured_at": "2026-10-01T18:45:50+00:00", + "python": "3.12.10", + "platform": "Windows-11", + "dependencies": { + "pytest": "8.4.2", + "cryptography": "50.0.0", + "pydantic": "2.13.4", + "jsonschema": "4.26.0", + "referencing": "0.37.0", + "rfc8785": "0.1.4", + "cbor2": "5.9.0", + "agentrust-trace": "0.9.0", + "opentelemetry-sdk": "1.44.0", + "jcs": "0.2.1", + "PyYAML": "6.0.3" + }, + "sources": { + "agentrust-telemetry": { + "commit": "29c13cfa4047a0f5784a90df63b22a3568453158", + "verified_by": "git archive of the pinned commit (tar commit id checked)" + }, + "weight-custody-manifest": { + "commit": "2acedfa883204f7d1fd457d3ee603198d85be82c", + "verified_by": "git archive of the pinned commit (tar commit id checked)" + }, + "agent-manifest": { + "commit": "934809709a2815695d65cfacb45dc0a164286046", + "verified_by": "git archive of the pinned commit (tar commit id checked)" + } + }, + "results": [ + { + "name": "telemetry-tests", + "cwd": "agentrust-telemetry", + "command": [ + "python", + "-m", + "unittest", + "discover", + "-s", + "tests", + "-v" + ], + "exit_code": 0, + "summary": "111 tests, OK", + "log": "telemetry-tests.txt", + "log_sha256": "2374d65a78eb3fffda834d8ec4fc358e18ec851a557327e207760ef0dc5b4584" + }, + { + "name": "telemetry-conformance", + "cwd": "agentrust-telemetry", + "command": [ + "python", + "conformance/runner/validate.py" + ], + "exit_code": 0, + "summary": "13 of 13 fixtures returned the expected verdict", + "log": "telemetry-conformance.txt", + "log_sha256": "f1af2ff732a78ef035226e1190421fcb67881644ff2936b5768131d08ac8e3ab" + }, + { + "name": "memory-tests", + "cwd": "weight-custody-manifest/python", + "command": [ + "python", + "-m", + "pytest", + "-p", + "no:cacheprovider", + "-o", + "pythonpath=src", + "tests/test_memory_sweep.py", + "tests/test_kbs.py", + "-v" + ], + "exit_code": 0, + "summary": "35 passed", + "log": "memory-tests.txt", + "log_sha256": "da1988be46556341113eebcf5ff9b70efa42ce2501871bb3a7a256b22cbddd78" + }, + { + "name": "approval-tests", + "cwd": "agent-manifest/python", + "command": [ + "python", + "-m", + "pytest", + "-p", + "no:cacheprovider", + "-o", + "pythonpath=src", + "tests/test_signing.py", + "tests/test_delegation_hitl.py", + "-v" + ], + "exit_code": 0, + "summary": "74 passed, 1 warning", + "log": "approval-tests.txt", + "log_sha256": "64a4116df7a121eecacf3af9555fe11807501694b8d12f8329615318a87c2b42" + } + ], + "limits": [ + "Local software tests only; no protected-hardware execution.", + "Tests and implementation share a repository and are not independent validation." + ] +} diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/telemetry-conformance.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/telemetry-conformance.txt new file mode 100644 index 0000000..6fd65da --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/telemetry-conformance.txt @@ -0,0 +1,15 @@ +PASS conformance\fixtures\valid\action.json +PASS conformance\fixtures\valid\approval.json +PASS conformance\fixtures\valid\data-flow.json +PASS conformance\fixtures\valid\evidence.json +PASS conformance\fixtures\valid\policy-decision.json +PASS conformance\fixtures\valid\usage.json +PASS conformance\fixtures\invalid\action-without-digest.json +PASS conformance\fixtures\invalid\aggregate-cost-without-rollup.json +PASS conformance\fixtures\invalid\model-call-with-rollup.json +PASS conformance\fixtures\invalid\numeric-timestamp.json +PASS conformance\fixtures\invalid\raw-content.json +PASS conformance\fixtures\invalid\usage-without-measurement.json +PASS conformance\fixtures\invalid\zero-trace-id.json + +STDERR diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/telemetry-tests.txt b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/telemetry-tests.txt new file mode 100644 index 0000000..0f31cec --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/experiments/results/rerun-2026-10-01/pinned/telemetry-tests.txt @@ -0,0 +1,135 @@ +PASS conformance\fixtures\valid\action.json +PASS conformance\fixtures\valid\approval.json +PASS conformance\fixtures\valid\data-flow.json +PASS conformance\fixtures\valid\evidence.json +PASS conformance\fixtures\valid\policy-decision.json +PASS conformance\fixtures\valid\usage.json +PASS conformance\fixtures\invalid\action-without-digest.json +PASS conformance\fixtures\invalid\aggregate-cost-without-rollup.json +PASS conformance\fixtures\invalid\model-call-with-rollup.json +PASS conformance\fixtures\invalid\numeric-timestamp.json +PASS conformance\fixtures\invalid\raw-content.json +PASS conformance\fixtures\invalid\usage-without-measurement.json +PASS conformance\fixtures\invalid\zero-trace-id.json +PASS contract=0.1.0-alpha.2 python=0.1.0a2 npm_name=@agentrust-io/telemetry npm=0.1.0-alpha.2 +PASS OTel GenAI matrix covers 16 span attributes, 8 metrics, and 6 event families +PASS TypeScript schemas match normative bytes + +STDERR +test_cedar_preserves_final_decision_and_reports_skipped_errors (test_adapters.AdapterTests.test_cedar_preserves_final_decision_and_reports_skipped_errors) ... ok +test_cedar_rejects_error_messages_as_codes (test_adapters.AdapterTests.test_cedar_rejects_error_messages_as_codes) ... ok +test_cedar_rejects_string_as_identifier_collection (test_adapters.AdapterTests.test_cedar_rejects_string_as_identifier_collection) ... ok +test_factory_builds_and_validates_any_supported_family (test_adapters.AdapterTests.test_factory_builds_and_validates_any_supported_family) ... ok +test_factory_matches_cross_language_golden_event (test_adapters.AdapterTests.test_factory_matches_cross_language_golden_event) ... ok +test_factory_prevents_envelope_override (test_adapters.AdapterTests.test_factory_prevents_envelope_override) ... ok +test_factory_serializes_nanoseconds_without_precision_loss (test_adapters.AdapterTests.test_factory_serializes_nanoseconds_without_precision_loss) ... ok +test_opa_boolean_log_maps_without_copying_input_or_result (test_adapters.AdapterTests.test_opa_boolean_log_maps_without_copying_input_or_result) ... ok +test_opa_complex_result_requires_explicit_mapper (test_adapters.AdapterTests.test_opa_complex_result_requires_explicit_mapper) ... ok +test_opa_rejects_malformed_source_collections (test_adapters.AdapterTests.test_opa_rejects_malformed_source_collections) ... ok +test_policy_event_maps_challenge_and_excludes_free_form_content (test_agt_adapter.AgtAdapterTests.test_policy_event_maps_challenge_and_excludes_free_form_content) ... ok +test_policy_mapper_rejects_ambiguous_decision_and_reason_message (test_agt_adapter.AgtAdapterTests.test_policy_mapper_rejects_ambiguous_decision_and_reason_message) ... ok +test_sink_prevalidates_whole_batch_before_emission (test_agt_adapter.AgtAdapterTests.test_sink_prevalidates_whole_batch_before_emission) ... ok +test_sink_uses_source_result_sentinels_and_reports_projection_failure (test_agt_adapter.AgtAdapterTests.test_sink_uses_source_result_sentinels_and_reports_projection_failure) ... ok +test_bindings_reject_fields_that_are_absent_on_both_records (test_agt_approval.AgtApprovalAdapterTests.test_bindings_reject_fields_that_are_absent_on_both_records) ... ok +test_protocol_chain_preserves_policy_request_and_resolution_links (test_agt_approval.AgtApprovalAdapterTests.test_protocol_chain_preserves_policy_request_and_resolution_links) ... ok +test_request_rejects_wrong_policy_action_and_chain (test_agt_approval.AgtApprovalAdapterTests.test_request_rejects_wrong_policy_action_and_chain) ... ok +test_resolution_rejects_pre_request_time_and_malformed_chain_digest (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_pre_request_time_and_malformed_chain_digest) ... ok +test_resolution_rejects_wrong_request_and_wrong_action (test_agt_approval.AgtApprovalAdapterTests.test_resolution_rejects_wrong_request_and_wrong_action) ... ok +test_terminal_outcomes_map_without_treating_votes_as_resolutions (test_agt_approval.AgtApprovalAdapterTests.test_terminal_outcomes_map_without_treating_votes_as_resolutions) ... ok +test_action_audit_requires_full_action_digest_and_computes_duration (test_agt_audit.AgtAuditAdapterTests.test_action_audit_requires_full_action_digest_and_computes_duration) ... ok +test_action_rejects_missing_or_reversed_timing (test_agt_audit.AgtAuditAdapterTests.test_action_rejects_missing_or_reversed_timing) ... ok +test_blocked_action_is_denied_even_if_source_outcome_is_success (test_agt_audit.AgtAuditAdapterTests.test_blocked_action_is_denied_even_if_source_outcome_is_success) ... ok +test_policy_audit_maps_without_free_form_data_or_resource (test_agt_audit.AgtAuditAdapterTests.test_policy_audit_maps_without_free_form_data_or_resource) ... ok +test_action_payload_field_is_rejected_by_schema_and_privacy_gate (test_conformance.ConformanceTests.test_action_payload_field_is_rejected_by_schema_and_privacy_gate) ... ok +test_all_schemas_are_valid_draft_2020_12 (test_conformance.ConformanceTests.test_all_schemas_are_valid_draft_2020_12) ... ok +test_committed_suite_has_expected_verdicts (test_conformance.ConformanceTests.test_committed_suite_has_expected_verdicts) ... ok +test_nested_prohibited_key_is_rejected_independently (test_conformance.ConformanceTests.test_nested_prohibited_key_is_rejected_independently) ... ok +test_unknown_event_type_is_rejected (test_conformance.ConformanceTests.test_unknown_event_type_is_rejected) ... ok +test_uppercase_trace_id_is_rejected (test_conformance.ConformanceTests.test_uppercase_trace_id_is_rejected) ... ok +test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason (test_data_flow.DataFlowTests.test_agt_decision_maps_tier_and_omits_categories_owner_geography_reason) ... ok +test_classifier_cannot_return_arbitrary_metadata (test_data_flow.DataFlowTests.test_classifier_cannot_return_arbitrary_metadata) ... ok +test_classifier_sees_content_but_event_contains_only_closed_metadata (test_data_flow.DataFlowTests.test_classifier_sees_content_but_event_contains_only_closed_metadata) ... ok +test_metadata_identifiers_reject_prose_paths_and_query_values (test_data_flow.DataFlowTests.test_metadata_identifiers_reject_prose_paths_and_query_values) ... ok +test_callback_and_returned_entry_cannot_mutate_retained_evidence (test_evidence.EvidenceTests.test_callback_and_returned_entry_cannot_mutate_retained_evidence) ... ok +test_chain_is_ordered_and_has_stable_golden_digest (test_evidence.EvidenceTests.test_chain_is_ordered_and_has_stable_golden_digest) ... ok +test_client_does_not_project_when_evidence_fails (test_evidence.EvidenceTests.test_client_does_not_project_when_evidence_fails) ... ok +test_client_isolates_operational_event_from_evidence_sink (test_evidence.EvidenceTests.test_client_isolates_operational_event_from_evidence_sink) ... ok +test_client_reports_evidence_persistence (test_evidence.EvidenceTests.test_client_reports_evidence_persistence) ... ok +test_context_mismatch_never_enters_evidence (test_evidence.EvidenceTests.test_context_mismatch_never_enters_evidence) ... ok +test_durable_callback_acknowledges_before_local_acceptance (test_evidence.EvidenceTests.test_durable_callback_acknowledges_before_local_acceptance) ... ok +test_failed_durable_ack_is_fail_closed_and_retryable (test_evidence.EvidenceTests.test_failed_durable_ack_is_fail_closed_and_retryable) ... ok +test_non_finite_number_cannot_enter_digest_chain (test_evidence.EvidenceTests.test_non_finite_number_cannot_enter_digest_chain) ... ok +test_rejects_cross_run_duplicate_overflow_and_append_after_seal (test_evidence.EvidenceTests.test_rejects_cross_run_duplicate_overflow_and_append_after_seal) ... ok +test_snapshot_is_defensive_and_unsealed_never_claims_complete (test_evidence.EvidenceTests.test_snapshot_is_defensive_and_unsealed_never_claims_complete) ... ok +test_action_metrics_exclude_action_and_agent_identifiers (test_otel_projectors.OTelProjectorTests.test_action_metrics_exclude_action_and_agent_identifiers) ... ok +test_classification_allowlist_is_bounded (test_otel_projectors.OTelProjectorTests.test_classification_allowlist_is_bounded) ... ok +test_classification_metric_is_allowlisted_or_collapsed (test_otel_projectors.OTelProjectorTests.test_classification_metric_is_allowlisted_or_collapsed) ... ok +test_metric_failure_is_reported_independently (test_otel_projectors.OTelProjectorTests.test_metric_failure_is_reported_independently) ... ok +test_real_metrics_record_usage_without_high_cardinality_ids (test_otel_projectors.OTelProjectorTests.test_real_metrics_record_usage_without_high_cardinality_ids) ... ok +test_real_otel_log_preserves_event_and_active_trace_context (test_otel_projectors.OTelProjectorTests.test_real_otel_log_preserves_event_and_active_trace_context) ... ok +test_async_handoff_creates_link_without_forcing_parentage (test_propagation.PropagationTests.test_async_handoff_creates_link_without_forcing_parentage) ... ok +test_header_lookup_is_case_insensitive (test_propagation.PropagationTests.test_header_lookup_is_case_insensitive) ... ok +test_injection_rejects_header_injection (test_propagation.PropagationTests.test_injection_rejects_header_injection) ... ok +test_link_requires_valid_w3c_context (test_propagation.PropagationTests.test_link_requires_valid_w3c_context) ... ok +test_sync_handoff_preserves_trace_and_agent_parent (test_propagation.PropagationTests.test_sync_handoff_preserves_trace_and_agent_parent) ... ok +test_complete_governed_workflow (test_reference_scenario.ReferenceScenarioTests.test_complete_governed_workflow) ... ok +test_contract_version_rejects_unsupported_spellings (test_repository_gates.RepositoryGateTests.test_contract_version_rejects_unsupported_spellings) ... ok +test_contract_versions_map_to_ecosystem_spellings (test_repository_gates.RepositoryGateTests.test_contract_versions_map_to_ecosystem_spellings) ... ok +test_declared_versions_are_consistent (test_repository_gates.RepositoryGateTests.test_declared_versions_are_consistent) ... ok +test_issue_templates_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_issue_templates_are_valid_yaml_when_parser_available) ... ok +test_npm_dist_tag_matches_the_declared_contract (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_matches_the_declared_contract) ... ok +test_npm_dist_tag_never_moves_latest_for_a_prerelease (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_never_moves_latest_for_a_prerelease) ... ok +test_npm_dist_tag_rejects_an_unsupported_version (test_repository_gates.RepositoryGateTests.test_npm_dist_tag_rejects_an_unsupported_version) ... ok +test_otel_matrix_gate_detects_attribute_metric_and_family_drift (test_repository_gates.RepositoryGateTests.test_otel_matrix_gate_detects_attribute_metric_and_family_drift) ... ok +test_otel_matrix_matches_shipped_projection (test_repository_gates.RepositoryGateTests.test_otel_matrix_matches_shipped_projection) ... ok +test_release_tag_must_match_contract (test_repository_gates.RepositoryGateTests.test_release_tag_must_match_contract) ... ok +test_schema_gate_detects_changed_bytes (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_changed_bytes) ... ok +test_schema_gate_detects_missing_and_extra_files (test_repository_gates.RepositoryGateTests.test_schema_gate_detects_missing_and_extra_files) ... ok +test_source_manifest_includes_otel_matrix (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_otel_matrix) ... ok +test_source_manifest_includes_release_policy (test_repository_gates.RepositoryGateTests.test_source_manifest_includes_release_policy) ... ok +test_typescript_schemas_match_normative_bytes (test_repository_gates.RepositoryGateTests.test_typescript_schemas_match_normative_bytes) ... ok +test_workflows_are_valid_yaml_when_parser_available (test_repository_gates.RepositoryGateTests.test_workflows_are_valid_yaml_when_parser_available) ... ok +test_bundled_schemas_match_normative_schemas (test_sdk.SdkTests.test_bundled_schemas_match_normative_schemas) ... ok +test_context_mismatch_rejects_before_projection (test_sdk.SdkTests.test_context_mismatch_rejects_before_projection) ... ok +test_emits_correlated_span_event_and_log (test_sdk.SdkTests.test_emits_correlated_span_event_and_log) ... ok +test_explicit_attribute_allowlist_is_narrow (test_sdk.SdkTests.test_explicit_attribute_allowlist_is_narrow) ... ok +test_extension_attributes_are_deny_by_default (test_sdk.SdkTests.test_extension_attributes_are_deny_by_default) ... ok +test_log_body_is_a_deep_copy (test_sdk.SdkTests.test_log_body_is_a_deep_copy) ... ok +test_no_span_still_emits_run_correlated_log (test_sdk.SdkTests.test_no_span_still_emits_run_correlated_log) ... ok +test_privacy_violation_rejects_before_projection (test_sdk.SdkTests.test_privacy_violation_rejects_before_projection) ... ok +test_projection_failures_are_reported_independently (test_sdk.SdkTests.test_projection_failures_are_reported_independently) ... ok +test_real_otel_span_receives_event (test_sdk.SdkTests.test_real_otel_span_receives_event) ... ok +test_action_events_create_truthful_tool_transcript (test_trace_adapter.TraceAdapterTests.test_action_events_create_truthful_tool_transcript) ... ok +test_finalizes_signs_and_validates_official_trace_record (test_trace_adapter.TraceAdapterTests.test_finalizes_signs_and_validates_official_trace_record) ... ok +test_linked_terminal_approval_resolves_policy_challenge (test_trace_adapter.TraceAdapterTests.test_linked_terminal_approval_resolves_policy_challenge) ... ok +test_refuses_conflicting_policy_bundles (test_trace_adapter.TraceAdapterTests.test_refuses_conflicting_policy_bundles) ... ok +test_refuses_missing_signer (test_trace_adapter.TraceAdapterTests.test_refuses_missing_signer) ... ok +test_refuses_policy_digest_algorithm_trace_cannot_represent (test_trace_adapter.TraceAdapterTests.test_refuses_policy_digest_algorithm_trace_cannot_represent) ... ok +test_refuses_unranked_classification (test_trace_adapter.TraceAdapterTests.test_refuses_unranked_classification) ... ok +test_refuses_unsealed_or_incomplete_evidence (test_trace_adapter.TraceAdapterTests.test_refuses_unsealed_or_incomplete_evidence) ... ok +test_unbound_or_expired_approval_cannot_resolve_policy_challenge (test_trace_adapter.TraceAdapterTests.test_unbound_or_expired_approval_cannot_resolve_policy_challenge) ... ok +test_a_failure_inside_official_signing_is_reported_as_one (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_failure_inside_official_signing_is_reported_as_one) +A key that satisfies the shape check and then fails when used. ... ok +test_a_policy_decision_without_a_bundle_digest_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_policy_decision_without_a_bundle_digest_is_refused) ... ok +test_a_repeated_classification_cannot_rank_anything (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_repeated_classification_cannot_rank_anything) ... ok +test_a_run_with_no_data_flow_has_nothing_to_classify (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_data_flow_has_nothing_to_classify) ... ok +test_a_run_with_no_policy_decision_has_no_policy_to_bind (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_run_with_no_policy_decision_has_no_policy_to_bind) ... ok +test_a_taxonomy_the_configuration_does_not_name_is_refused (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_a_taxonomy_the_configuration_does_not_name_is_refused) ... ok +test_an_empty_run_carries_no_chained_evidence (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_an_empty_run_carries_no_chained_evidence) ... ok +test_every_required_configuration_field_is_named_when_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_every_required_configuration_field_is_named_when_absent) ... ok +test_the_optional_dependency_is_named_when_it_is_absent (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_the_optional_dependency_is_named_when_it_is_absent) ... ok +test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode (test_trace_adapter_refusals.TraceAdapterRefusalTests.test_two_decisions_disagreeing_on_enforcement_cannot_produce_one_mode) ... ok +test_accumulator_rejects_non_finite_external_cost (test_usage.UsageTests.test_accumulator_rejects_non_finite_external_cost) ... ok +test_bool_is_not_accepted_as_token_integer (test_usage.UsageTests.test_bool_is_not_accepted_as_token_integer) ... ok +test_cost_requires_finite_amount_and_explicit_provenance (test_usage.UsageTests.test_cost_requires_finite_amount_and_explicit_provenance) ... ok +test_reused_event_id_with_changed_data_is_rejected (test_usage.UsageTests.test_reused_event_id_with_changed_data_is_rejected) ... ok +test_rollup_deduplicates_and_reports_partial_coverage (test_usage.UsageTests.test_rollup_deduplicates_and_reports_partial_coverage) ... ok +test_rollup_rejects_non_leaf_and_mixed_currency (test_usage.UsageTests.test_rollup_rejects_non_leaf_and_mixed_currency) ... ok +test_rollups_match_shared_cross_language_vector (test_usage.UsageTests.test_rollups_match_shared_cross_language_vector) ... ok +test_schema_rejects_aggregate_cost_without_run_rollup (test_usage.UsageTests.test_schema_rejects_aggregate_cost_without_run_rollup) ... ok +test_unknown_price_is_absent_not_zero (test_usage.UsageTests.test_unknown_price_is_absent_not_zero) ... ok + +---------------------------------------------------------------------- +Ran 111 tests in 7.937s + +OK diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/paper.tex b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/paper.tex new file mode 100644 index 0000000..5f220c7 --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/paper.tex @@ -0,0 +1,110 @@ +\documentclass[10pt,letterpaper]{article} +\usepackage[margin=0.9in,headheight=14pt]{geometry} +\usepackage{fontspec,xcolor,parskip,titlesec,fancyhdr,xurl,microtype} +\usepackage[colorlinks=true,linkcolor=black,citecolor=black,urlcolor=black]{hyperref} +\definecolor{OceanBlue}{HTML}{00B0FF} +\definecolor{CoralGreen}{HTML}{10BA87} +\setmainfont{Roboto-Regular.ttf}[Path=../_paper-assets/fonts/,BoldFont=Roboto-Bold.ttf] +\newfontfamily\headingfont{Montserrat-SemiBold.ttf}[Path=../_paper-assets/fonts/] +\titleformat{\section}{\headingfont\fontsize{13}{16}\selectfont}{\thesection}{0.6em}{}[\color{CoralGreen}\titlerule] +\titlespacing*{\section}{0pt}{17pt}{8pt} +\setlength{\parindent}{0pt} +\setlength{\emergencystretch}{3em} +\setlength{\parskip}{6pt} +\pagestyle{fancy}\fancyhf{} +\fancyhead[L]{\small OPAQUE Systems / Research} +\fancyhead[R]{\small Technical report, version 3 / October 1, 2026} +\fancyfoot[L]{\small Not peer reviewed} +\fancyfoot[R]{\thepage} +\renewcommand{\headrulewidth}{0.4pt} +\begin{document} + + +{\raggedright\headingfont\fontsize{21}{26}\selectfont\color{OceanBlue} Acceptance Before Export: A Bounded Evidence Contract for Governance Telemetry\par} + +\vspace{8pt}Imran Siddique\par OPAQUE Systems\par\vspace{4pt}October 1, 2026\par Technical report, version 3\par + +\vspace{5pt}{\color{CoralGreen}\hrule}\vspace{8pt} + +\textbf{Scope of this edition.} This is an author-prepared technical report, not peer reviewed. Version 3 (October 1, 2026) corrects version 2 (September 30, 2026), which left out a defect in the pinned revision's TRACE finalizer that the paper's truncation argument depends on, and the Internet-Draft that already chains agent audit records over RFC 8785 bytes; results are unchanged. Version 2 added one prior-art citation that version 1 (September 27, 2026) omitted. Version 1 revised a manuscript dated September 4, 2026. Every result reported here is software-only: the Python test suite, the portable conformance fixtures and separate review probes, run on one Windows workstation against a pinned source revision, rerun on September 27, 2026, and compared with a later revision. No durable store, crash-restart, multi-process or deployed-exporter experiment was run.\par + +\noindent\textbf{Patent Pending.} + +\section*{Abstract} + +An operational telemetry backend can lose events through sampling, retention and exporter failures. Its observations therefore cannot, by themselves, establish complete governance evidence. We describe a source-side acceptance boundary in AgenTrust Telemetry: validate a metadata-constrained event, check context correlation, accept an isolated representation into a run-scoped cryptographic prefix, and only then attempt operational projections. In a durable configuration, an external callback must acknowledge persistence before the local prefix advances. Export failures remain distinct from evidence failures. We formalize the conditional ordering property and test its limits, including acknowledgment loss and an empty accumulator declared complete. At a pinned source revision, 111 Python tests and 13 portable conformance fixtures produced expected results on September 4 and again on September 27, 2026. Four additional probes demonstrate both the useful separation and the absence of automatic completeness or distributed exactly-once persistence. The contribution is a precise contract for acceptance before export, with explicit trust and recovery obligations, rather than a claim that a hash chain proves every event occurred. + + +\section{Motivation and related systems} + +Agent runs cross policy engines, approval services, tools and asynchronous execution boundaries. The AgenTrust contract represents policy decisions, approval lifecycle, usage, classified data flow, action execution and evidence lifecycle as six event families. A durable run identifier and, when applicable, action identifier and digest allow governance joins to survive changes in operational trace context. Here, durable means a stable logical identifier; it does not imply a durable storage medium. + + +OpenTelemetry supplies an operational data model and sampling machinery \cite{ref1}. Secure cryptographically linked audit logs long predate agent telemetry \cite{ref2}, and tamper-evident history trees give logarithmic-size proofs that a later log version extends an earlier one \cite{ref8}. Recent governance telemetry work combines policy events, provenance and enforcement \cite{ref3}, while append-only telemetry storage also appears in patent literature \cite{ref4}. These are substantial overlaps. This paper does not introduce hash chains, event sourcing, governance telemetry or a complete audit protocol. Its subject is the SDK boundary between local evidence acceptance and fallible operational projection. + + +\section{Event and trust contract} + +The normalized JSON event is validated before use. The metadata profile excludes raw prompts, outputs, source code, tool arguments/results, credentials and authorization tokens through schemas and recursive key checks. Such restrictions reduce accidental payload capture; they are not a semantic proof that arbitrary strings contain no secrets. Nanosecond timestamps are represented as decimal strings to avoid numeric precision loss across languages. + + +Run and action identifiers are distinct from trace/span identifiers. If an active span exists, supplied correlation fields must agree before evidence is accepted. Propagated metadata is untrusted and confers no workload identity or authorization. The caller owns instrumentation, the schema configuration, its evidence sink, operational emitters and finalization. The SDK does not install an OpenTelemetry provider or authorize the underlying action. + + +The trusted boundary includes an honest event producer and, for the specified evidence behavior, the reference accumulator. A custom sink can behave differently. A callback configured as durable must honestly acknowledge committed storage and supply idempotency and recovery. The accumulator cannot observe operations that bypass instrumentation or recover source facts fabricated by a producer. + + +\section{Acceptance and projection protocol} + +An accumulator owns one run, an accepted entry list, an event-ID set, a sealed flag, a completeness label, a capacity and a lock. It begins empty, unsealed and unknown. Validation and copying precede a locked acceptance operation. A run mismatch, duplicate event ID, closed accumulator or exceeded cap is rejected. + + +For sequence i, the next commitment is SHA-256 of the previous raw 32-byte digest, i encoded as unsigned 64-bit big-endian bytes and RFC 8785 canonical event bytes. Genesis uses 32 zero bytes. The stored entry includes its sequence, event ID, previous digest, digest and event representation. Canonicalization is an established interoperability primitive \cite{ref5}, and the Agent Audit Trail Internet-Draft, first posted in March 2026, already links agent audit records by SHA-256 over RFC 8785 canonical bytes \cite{ref9}. + + +When a persistence callback exists, the accumulator passes it an isolated candidate entry and requires the exact Boolean value true. Any exception or other result prevents local advancement. After acknowledgment, the local list and event-ID set advance, and isolated snapshots/return values prevent external mutation through shared object references. Without a callback, acceptance is memory-only. + + +The client next attempts span, log and metric projections. Each projection exception is recorded separately; it does not delete an accepted entry and does not prevent attempts of the remaining projections. The current Python implementation calls these projections sequentially. The result field named evidence\_persisted means the selected sink returned successfully; with an in-memory accumulator it must not be read as proof of disk persistence. + + +\section{Properties and counterexamples} + +The ordering property is conditional: when the reference evidence sink is configured, any operational projection attempted by an emit invocation follows successful return from its evidence append. In durable callback mode, successful append additionally follows callback acknowledgment. This follows from program order and exception propagation at the boundary. It is not an atomic transaction across the SDK, external storage and every exporter. + + +If an external store commits and then loses its acknowledgment, the local prefix remains unchanged and a retry may call the store again with the same event. A deliberately non-idempotent callback in our probe records two external writes. The production callback must deduplicate retries and detect conflicting bytes; the SDK does not implement a distributed exactly-once protocol or crash recovery. + + +Sealing closes the prefix and stores a producer-supplied completeness label: complete, incomplete or unknown. Unsealed snapshots are unknown. A separate probe seals an empty accumulator as complete successfully. Thus the label is not an independently verified property. Establishing complete capture would also require a defined required-event set, correct instrumentation, terminal-state coverage and recovery evidence. A hash chain alone provides none of those premises. + + +The commitment authenticates neither the event source nor the final head by itself. A trusted checkpoint or separately signed final evidence container is needed to detect substitution or truncation relative to a known head. Optional TRACE finalization is an integration boundary, not a repair for missing source events. At the pinned revision it did not check the chain either. The finalizer took its measurement from the snapshot's chain digest and its appraisal from the snapshot's entries without recomputing one from the other, and a sealed snapshot holds mutable objects, so a snapshot with an edited entry, a dropped entry or a substituted chain digest was signed. Pull request 65 of the source repository, released in 0.1.0-alpha.5 on September 26, 2026, recomputes the chain before signing in both SDKs; its regression tests fail against the pinned code. + + +\section{Reproducible evaluation} + +The source revision is agentrust-telemetry 29c13cfa4047a0f5784a90df63b22a3568453158 \cite{ref7}. September 4, 2026 execution produced 111 passing Python unit tests and expected verdicts for all 13 portable fixtures, six valid and seven invalid. A rerun on September 27, 2026 against a git archive of that commit, whose embedded commit identifier was checked before extraction, produced the same counts. The fixtures cover the six event families and failures such as prohibited content, invalid trace identifiers, timestamp ambiguity, missing action digests and unsupported usage aggregation. The repository suite also exercises adapters, propagation, projection, evidence and finalization. + + +Four additional review probes establish: an exporter exception preserves an accepted event; callback refusal prevents projection and local advancement; an empty complete declaration is accepted; and commit-before-ack loss requires external idempotency. The last two are counterexamples to stronger claims. The probes use the actual modules at the pinned revision and record results in boundary-probes.json. All four probes gave the same result at the later revision c351568 (September 26, 2026); between the two revisions the Python evidence module changed once, to remove a deadlock on reentrant append or seal (pull request 46), while validation and TRACE finalization changed in pull request 65. Commands, source commit identifiers, dependency versions and test logs accompany the paper. + + +The September 27 rerun used Windows with Python 3.12.10, cryptography 50.0.0 and jsonschema 4.26.0; the September 4 run used Python 3.12.13 with the same two packages. The implementation and its repository tests share assumptions; the review probes are separate scenarios but are not an independent organizational reproduction. TypeScript schemas and golden fixtures are part of the project, but its TypeScript suite was not freshly executed in this review. No crash-restart store, multi-process correctness proof or performance benchmark is reported. + + +\section{Relation to prior logging systems} + +The useful distinction is preservation of an accepted evidence prefix across operational export failures, together with refusal-before-projection when the evidence path fails. Prior write-ahead and fault-tolerant logging systems may supply closely related ordering. One example tags and checksums batches between log agents and collectors so that delivery can be confirmed \cite{ref6}; its claims were not compared with this contract. Terminology and deployment in an agent system are not, by themselves, evidence of a new ordering property. + + +\section{Limits and next experiments} + +The result concerns accepted event representations. It does not prove source truth, complete instrumentation, physical execution, secrecy of arbitrary metadata or authoritative policy enforcement. Next experiments should inject durable-store acknowledgment loss and restart, compare recovered prefixes, test same-ID/different-content conflicts and explicitly define completeness criteria before attempting to verify them. A deployable contract should make memory acceptance, durable acknowledgment, export success and completeness visible as distinct outcomes. + + +\bibliographystyle{unsrt} + +\bibliography{refs} + +\end{document} \ No newline at end of file diff --git a/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/refs.bib b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/refs.bib new file mode 100644 index 0000000..5980fea --- /dev/null +++ b/papers/telemetry-evidence-acceptance/telemetry-evidence-acceptance/refs.bib @@ -0,0 +1,44 @@ +@misc{ref1, + title = {{OpenTelemetry. Sampling. Project documentation, accessed September 27, 2026}}, + howpublished = {\url{https://opentelemetry.io/docs/concepts/sampling/}} +} + +@misc{ref2, + title = {{B. Schneier and J. M. Kelsey (Counterpane Internet Security). Event auditing system. US Patent 5,978,475, granted November 2, 1999}}, + howpublished = {\url{https://patents.google.com/patent/US5978475A/en}} +} + +@misc{ref3, + title = {{A. Pathak and N. Jain. Governance-Aware Agent Telemetry for Closed-Loop Enforcement in Multi-Agent AI Systems. arXiv:2604.05119v1, April 6, 2026}}, + howpublished = {\url{https://arxiv.org/abs/2604.05119v1}} +} + +@misc{ref4, + title = {{X. Ma and Z. Yin (Cisco Technology). Configuration and operational network observability using sliding window for append-only telemetry log. US Patent Application Publication 2025/0016076 A1, January 9, 2025}}, + howpublished = {\url{https://patents.google.com/patent/US20250016076A1/en}} +} + +@misc{ref5, + title = {{A. Rundgren, B. Jordan and S. Erdtman. JSON Canonicalization Scheme (JCS). RFC 8785, June 2020}}, + howpublished = {\url{https://www.rfc-editor.org/rfc/rfc8785}} +} + +@misc{ref6, + title = {{J. M. Hsieh and H. N. Robinson (Cloudera). Collecting and aggregating log data with fault tolerance. US Patent 9,081,888 B2, granted July 14, 2015}}, + howpublished = {\url{https://patents.google.com/patent/US9081888B2/en}} +} + +@misc{ref7, + title = {{AgenTrust Telemetry source, pinned revision 29c13cfa4047a0f5784a90df63b22a3568453158}}, + howpublished = {\url{https://github.com/agentrust-io/agentrust-telemetry/tree/29c13cfa4047a0f5784a90df63b22a3568453158}} +} + +@misc{ref8, + title = {{S. A. Crosby and D. S. Wallach. Efficient Data Structures for Tamper-Evident Logging. 18th USENIX Security Symposium, 2009}}, + howpublished = {\url{https://www.usenix.org/legacy/event/sec09/tech/full_papers/crosby.pdf}} +} + +@misc{ref9, + title = {{R. Sharif. Agent Audit Trail: A Standard Logging Format for Autonomous AI Systems. IETF Internet-Draft draft-sharif-agent-audit-trail, revision -00 posted March 29, 2026; revision -06 current on October 1, 2026}}, + howpublished = {\url{https://datatracker.ietf.org/doc/draft-sharif-agent-audit-trail/}} +}