Encrypt attended state and the authority journal at rest by default

CAPABILITIES.md

@@ -317,7 +317,7 @@ improvising or retrying when something is off.
 | See a compiled program as a map (every step, exact edge, loop and stop rule, with each step's checks) and play a recorded run over it; one React package that Desktop, app.openadapt.ai and openadapt.ai share | On main ([packages/program-graph](packages/program-graph/README.md)) | Real compiled OpenEMR referral program and a recorded scoreboard run, both on synthetic data | `@openadapt/program-graph`; `openadapt-flow visualize --format json` |
 | Named sections for every compiled program ("Who: title, first name, last name", "Contact", "Create New Patient"), found without a model from the save, the recorded form structure, the fields and any narration; a person can rename, merge or split them on the review screen | On main | Unit tests; the certified OpenEMR referral bundle and a real recording of it on a local test copy with made-up patients | `openadapt-flow sections`; [SECTIONS.md](packages/flow/docs/SECTIONS.md) |
 | Encrypt PHI at rest: every bundle file that can hold patient data under one sealed index (unknown files refused), web recordings from the first frame, and the run directories of sealed bundles; compile, induce, scaffold-verifier, qualify, admit, run, resume, undo, visualize, explain, emit-skill, for-each, sections, check mark, compose, process, variants, route, teach and push read them in memory through one artifact reader and refuse without the key; teach keeps its skill library and taught bundle sealed, and push keeps its sanitized review copy and approved archive encrypted until the upload | On main | Synthetic (no-plaintext audit-hook test over record, compile and a VERIFIED run; per-frame cost measured) | `openadapt-flow seal`, `record`; [phi_at_rest.md](packages/flow/docs/phi_at_rest.md) |
-| Encryption at rest under a configured or locally generated key (0600): durable checkpoints by default; with `OPENADAPT_FLOW_AT_REST=strict`, every recording, bundle and run directory too. Each plaintext write is logged on stderr and in a ledger that Desktop shows as a notice | On main | Synthetic (key, opt-out, strict-mode and run-directory tests) | `openadapt_flow.at_rest`; [phi_at_rest.md](packages/flow/docs/phi_at_rest.md#default-opt-out-and-strict-mode) |
+| Encryption at rest under a configured or locally generated key (0600): durable checkpoints and attended state by default; with `OPENADAPT_FLOW_AT_REST=strict`, every recording, bundle and run directory too. Each plaintext write is logged on stderr and in a ledger that Desktop shows as a notice | On main | Synthetic (key, opt-out, strict-mode, run-directory and attended-state audit-hook tests) | `openadapt_flow.at_rest`; [phi_at_rest.md](packages/flow/docs/phi_at_rest.md#default-opt-out-and-strict-mode) |
 | Signed receipts (Seals), countersigned and timestamped, offline by default | Released | Unit tests, real TSA tokens | `openadapt-flow seal`, `countersign`, `local-seal` |
 | Offline Seals signed with a key the customer holds: a key file, the OS keychain, or a PKCS#11 token or HSM; pinned keys with rotation and revocation; no fallback to a generated key | On main ([#1392](https://github.com/OpenAdaptAI/OpenAdapt/pull/1392)) | Unit tests with synthetic data (keychain and PKCS#11 through stand-ins) | `openadapt-flow local-seal --key`, `local-seal key create`, `pin`, `revoke`; `openadapt seal verify FOLDER` checks every sealed run in a folder; [OFFLINE_SEALS.md](packages/flow/docs/OFFLINE_SEALS.md#customer-held-keys) |
 | Anchor an offline Seal in the public Sigstore Rekor transparency log (opt-in; sends one digest and the public key), and verify the anchor offline from the stored inclusion proof | On main ([#1394](https://github.com/OpenAdaptAI/OpenAdapt/pull/1394)) | Unit tests against a simulated Rekor log; real public Rekor entries and Sigstore TSA tokens for the landing demo's referral runs, verified offline in tests; Ed25519ph matches the RFC 8032 test vector | `openadapt-flow local-seal anchor`, `local-seal verify --require-anchor`; [OFFLINE_SEALS.md](packages/flow/docs/OFFLINE_SEALS.md#optional-anchor-a-seal-in-a-public-transparency-log) |

llms-full.txt

@@ -720,7 +720,7 @@ improvising or retrying when something is off.
 | See a compiled program as a map (every step, exact edge, loop and stop rule, with each step's checks) and play a recorded run over it; one React package that Desktop, app.openadapt.ai and openadapt.ai share | On main ([packages/program-graph](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/program-graph/README.md)) | Real compiled OpenEMR referral program and a recorded scoreboard run, both on synthetic data | `@openadapt/program-graph`; `openadapt-flow visualize --format json` |
 | Named sections for every compiled program ("Who: title, first name, last name", "Contact", "Create New Patient"), found without a model from the save, the recorded form structure, the fields and any narration; a person can rename, merge or split them on the review screen | On main | Unit tests; the certified OpenEMR referral bundle and a real recording of it on a local test copy with made-up patients | `openadapt-flow sections`; [SECTIONS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SECTIONS.md) |
 | Encrypt PHI at rest: every bundle file that can hold patient data under one sealed index (unknown files refused), web recordings from the first frame, and the run directories of sealed bundles; compile, induce, scaffold-verifier, qualify, admit, run, resume, undo, visualize, explain, emit-skill, for-each, sections, check mark, compose, process, variants, route, teach and push read them in memory through one artifact reader and refuse without the key; teach keeps its skill library and taught bundle sealed, and push keeps its sanitized review copy and approved archive encrypted until the upload | On main | Synthetic (no-plaintext audit-hook test over record, compile and a VERIFIED run; per-frame cost measured) | `openadapt-flow seal`, `record`; [phi_at_rest.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/phi_at_rest.md) |
-| Encryption at rest under a configured or locally generated key (0600): durable checkpoints by default; with `OPENADAPT_FLOW_AT_REST=strict`, every recording, bundle and run directory too. Each plaintext write is logged on stderr and in a ledger that Desktop shows as a notice | On main | Synthetic (key, opt-out, strict-mode and run-directory tests) | `openadapt_flow.at_rest`; [phi_at_rest.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/phi_at_rest.md#default-opt-out-and-strict-mode) |
+| Encryption at rest under a configured or locally generated key (0600): durable checkpoints and attended state by default; with `OPENADAPT_FLOW_AT_REST=strict`, every recording, bundle and run directory too. Each plaintext write is logged on stderr and in a ledger that Desktop shows as a notice | On main | Synthetic (key, opt-out, strict-mode, run-directory and attended-state audit-hook tests) | `openadapt_flow.at_rest`; [phi_at_rest.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/phi_at_rest.md#default-opt-out-and-strict-mode) |
 | Signed receipts (Seals), countersigned and timestamped, offline by default | Released | Unit tests, real TSA tokens | `openadapt-flow seal`, `countersign`, `local-seal` |
 | Offline Seals signed with a key the customer holds: a key file, the OS keychain, or a PKCS#11 token or HSM; pinned keys with rotation and revocation; no fallback to a generated key | On main ([#1392](https://github.com/OpenAdaptAI/OpenAdapt/pull/1392)) | Unit tests with synthetic data (keychain and PKCS#11 through stand-ins) | `openadapt-flow local-seal --key`, `local-seal key create`, `pin`, `revoke`; `openadapt seal verify FOLDER` checks every sealed run in a folder; [OFFLINE_SEALS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/OFFLINE_SEALS.md#customer-held-keys) |
 | Anchor an offline Seal in the public Sigstore Rekor transparency log (opt-in; sends one digest and the public key), and verify the anchor offline from the stored inclusion proof | On main ([#1394](https://github.com/OpenAdaptAI/OpenAdapt/pull/1394)) | Unit tests against a simulated Rekor log; real public Rekor entries and Sigstore TSA tokens for the landing demo's referral runs, verified offline in tests; Ed25519ph matches the RFC 8032 test vector | `openadapt-flow local-seal anchor`, `local-seal verify --require-anchor`; [OFFLINE_SEALS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/OFFLINE_SEALS.md#optional-anchor-a-seal-in-a-public-transparency-log) |

packages/flow/docs/phi_at_rest.md

@@ -22,7 +22,7 @@ with your other secrets.
 
 | | Default | `OPENADAPT_FLOW_AT_REST=strict` |
 | --- | --- | --- |
-| Durable checkpoints | Encrypted | Encrypted |
+| Durable checkpoints and attended state | Encrypted | Encrypted |
 | Web recordings | Encrypted with a configured key; otherwise logged | Encrypted |
 | Bundles from `compile`, `induce`, `admit` | Encrypted from encrypted input; otherwise logged | Encrypted |
 | Run directories | Encrypted for a sealed bundle; otherwise logged | Encrypted |
@@ -190,6 +190,36 @@ step with them.
 The approved archive beside it (`<name>.approved.zip`) is one AES-256-GCM
 container under the same key.
 
+### Durable checkpoints and attended state
+
+In the default mode, under the configured or local key, the durable store
+writes its checkpoints, pauses and approvals as `<name>.enc` AES-256-GCM
+containers. The attended store does the same for the pause capability and its
+history, the decision log (operator and message), program and reconciliation
+receipts, and program-transition control frames. Each file's associated data
+names its run-relative path, so a file copied over another path does not
+decrypt, and once a run's attended state is encrypted a plaintext file in its
+place is refused. The external attended journal stores each decision-log
+snapshot encrypted and bound to its run and sequence; its hash chain and HMAC
+cover the plaintext. A run whose attended files are plaintext from an earlier
+version still resumes, and its first encrypted write converts them.
+The key is stretched with scrypt once per run, under a random salt kept in the
+run's `.attended_state.encrypted` marker, and each file and journal row gets
+its own nonce under keys derived from it, so the cost doesn't grow with the
+number of files or journal rows. Each file names the salt it was sealed
+under, and a file whose salt is not the one in the run's marker is refused, so
+a file copied in from another run under the same key does not read. If two
+processes convert the same run at once, the one that loses the race to write
+the marker reseals its files under the winner's salt. The replayer classifies
+a run's outcome with its own checkpoint key, so a run whose key was passed in
+code rather than through the environment still reads its sealed evidence.
+Once a run's attended state is encrypted,
+`OPENADAPT_FLOW_PLAINTEXT_AT_REST` does not turn it back to plaintext: a write
+with no key is refused. Containers written by the earlier per-file format
+still read.
+`tests/test_attended_at_rest.py` runs attended decisions under the same disk
+audit hook and checks that no attended file or journal row holds plaintext.
+
 ### Run directory of a sealed bundle
 
 Every file is encrypted except these structural ones: `.durable_run.claim`

packages/flow/openadapt_flow/crypto.py

@@ -170,6 +170,8 @@ def is_encrypted(data: bytes) -> bool:
     """
     if data[: len(SET_FILE_MAGIC)] == SET_FILE_MAGIC:
         return True
+    if data[: len(SALTED_FILE_MAGIC)] == SALTED_FILE_MAGIC:
+        return True
     try:
         obj = json.loads(data)
     except (ValueError, TypeError):
@@ -305,6 +307,25 @@ def open(self, container: bytes, *, aad: bytes) -> bytes:
             ) from exc
 
 
+#: Prefix of a container sealed under a key derived once per run (see
+#: :mod:`openadapt_flow.runtime.durable.attended_state`): this prefix, the
+#: run's 16-byte salt, a random 96-bit nonce, then the AES-256-GCM ciphertext.
+SALTED_FILE_MAGIC: Final[bytes] = b"OAFLOW-SALTED-1\n"
+SALT_LEN: Final[int] = _SALT_LEN
+NONCE_LEN: Final[int] = _NONCE_LEN
+
+
+def derive_data_key(passphrase: str, salt: bytes) -> bytes:
+    """The 256-bit scrypt key for ``passphrase`` and ``salt``, at the work
+    factors new containers record. A caller that derives once and caches the
+    key must keep it in memory only."""
+    if not passphrase:
+        raise MissingKeyError("no encryption passphrase configured")
+    if len(salt) != _SALT_LEN:
+        raise CryptoError("a data-key salt must be 16 bytes")
+    return _derive(passphrase, salt, n=_SCRYPT_N, r=_SCRYPT_R, p=_SCRYPT_P)
+
+
 def kdf_parameters() -> dict[str, int]:
     """The scrypt work factors new containers record."""
     return {"kdf_n": _SCRYPT_N, "kdf_r": _SCRYPT_R, "kdf_p": _SCRYPT_P}