lance-graph-dir-sim: directory desired-state simulation over SoA + Quack (no writes)

.claude/board/entries/2026-10-03-dir-sim-soa-quack.md

@@ -0,0 +1,52 @@
+# Directory desired-state simulation over SoA + Quack (2026-10-03)
+
+**Status:** MEASURED. The `crates/lance-graph-dir-sim` crate is excluded from
+the workspace because it path-depends on the OGAR sibling, the same shape as
+`lance-graph-report-ogar`. It consumes the semantic vocabulary in OGAR
+`ogar-dir-sim` (OGAR PR #314).
+
+A directory version is a shared `Arc<Snapshot>` of SoA lanes plus a
+delta-sized overlay. Invariants are Quack programs:
+
+- **Edge integrity** is an anti-join: `negate(Semijoin)`, lowered to two
+  `MaskOp::Gather` over the node-kind planes.
+- **SMTP / UPN uniqueness** is `GroupReduce Count` keyed on a normalized-key
+  dictionary id, folded over base + overlay. The overlay rows are admitted by a
+  `Semijoin` against the active-user plane.
+
+Population rules combine two folded `GROUP BY` sinks at the consumer. They
+never pass one program's mask to another program's `Semijoin`.
+
+| quantity (one membership mutation, `tests/alloc.rs`) | 1k users | 100k users |
+|---|---|---|
+| bytes allocated by `simulate` | 853 | 853 |
+| bytes allocated by same-root `diff` | 1,208 | 1,208 |
+
+The suite has 23 tests. Ten guards were disabled one at a time and each turned
+its test red.
+
+Open for the operator:
+
+- **`VersionedGraph` (`u32` node ids, additions-only diff).** It cannot
+  persist 128-bit directory identity. The choice is to widen it upstream or
+  use a dedicated directory dataset.
+- **Row-population masks.** There is still no shared row-population mask type
+  in the contract; this crate uses mask-risc bitmaps directly.
+
+## Addendum (same day): no produced mask into a Semijoin, now a compile error
+
+- Kept rows from a program are sealed in `Kept`, which only exits through
+  `rows()`. Nothing can build a `ForeignPlane` from it. A `compile_fail`
+  doctest pins this, with a passing twin. Disable-run: giving `Kept` a slice
+  `Deref` turns the doctest red.
+- Audit by reading: every `ForeignPlane` in the crate is resident (the user
+  plane, the group plane, the active-user plane).
+- ndarray is mandatory. It comes in through `lance-graph-mask-risc`, a
+  non-optional path dependency (`cargo tree -i ndarray`).
+- All builds and tests ran with `CARGO_PROFILE_DEV_DEBUG=0` and
+  `CARGO_INCREMENTAL=0`.
+- `ogar-loco` was not adopted. The only loco→mask-risc dialect is a test-local
+  `FoldDialect` in `r2il-mask-abi-probe`. Its `GROUP_SUM` sink keeps only the
+  last fold, so it cannot express ImplyGroup's two keyed counts.
+- OPEN: promote a fold dialect to a library, with a multi-sink `GROUP_SUM`.
+  Then rules can be loco program data.

.claude/board/entries/README.md

@@ -25,10 +25,11 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
 opposite directions; the stranding this convention prevents shows up in
 exactly one of them, never both.
 
-177 entries, 2026-08-06 .. 2026-09-30.
+178 entries, 2026-08-06 .. 2026-10-03.
 
 | date | entry id | finding | file |
 |---|---|---|---|
+| 2026-10-03 | `dir-sim-soa-quack` | Directory simulation on SoA + Quack: one-edge mutation 853 B at 1k and 100k users | [2026-10-03-dir-sim-soa-quack.md](2026-10-03-dir-sim-soa-quack.md) |
 | 2026-09-30 | `deepnsm-v2-coverage-bands` |  | [2026-09-30-deepnsm-v2-coverage-bands.md](2026-09-30-deepnsm-v2-coverage-bands.md) |
 | 2026-09-29 | `deepnsm-v2-counted-pick-tag-deltas` |  | [2026-09-29-deepnsm-v2-counted-pick-tag-deltas.md](2026-09-29-deepnsm-v2-counted-pick-tag-deltas.md) |
 | 2026-09-26 | `deepnsm-v2-lexical-evidence-survives-routing` |  | [2026-09-26-deepnsm-v2-lexical-evidence-survives-routing.md](2026-09-26-deepnsm-v2-lexical-evidence-survives-routing.md) |

Cargo.toml

@@ -110,6 +110,12 @@ exclude = [
     # reason as lance-graph-ogar: path-deps the OGAR sibling. Verify via
     # `cargo test --manifest-path crates/lance-graph-report-ogar/Cargo.toml`.
     "crates/lance-graph-report-ogar",
+    # Directory desired-state simulation over SoA lanes + Quack (versions as
+    # shared snapshot + delta overlay; invariants as Semijoin / GroupReduce
+    # programs). EXCLUDED for the same reason: path-deps the OGAR sibling
+    # (ogar-dir-core / ogar-ad / ogar-dir-sim). Verify via
+    # `cargo test --manifest-path crates/lance-graph-dir-sim/Cargo.toml`.
+    "crates/lance-graph-dir-sim",
     # Cognitive Compilation loop (claude/cognitive-compilation-lance-graph-h8sgym).
     # The NEW piece is the Elixir-shaped template — thinking styles, JITson, and
     # i4-32D thinking-style vectors already exist; the template was the gap. Four

crates/lance-graph-dir-sim/Cargo.toml

@@ -0,0 +1,29 @@
+# lance-graph-dir-sim — simulate a directory's desired state over the SoA
+# substrate. EXCLUDED from the lance-graph workspace (own [workspace] root)
+# because it path-deps the OGAR sibling — the same shape as
+# lance-graph-report-ogar / lance-graph-ogar.
+#
+# Direction: this crate depends on BOTH sides; neither depends on it. OGAR
+# owns the semantic vocabulary (ogar-dir-sim: Change, provenance, Violation,
+# ExecutionPlan) and the observation records (ogar-dir-core / ogar-ad);
+# lance-graph owns execution: every population query lowers through
+# lance-graph-quack onto lance-graph-mask-risc's one evaluator.
+#
+# Build/verify:  cargo test --manifest-path crates/lance-graph-dir-sim/Cargo.toml
+
+[package]
+name = "lance-graph-dir-sim"
+version = "0.1.0"
+edition = "2021"
+publish = false
+license = "Apache-2.0"
+description = "Versioned directory graph over SoA lanes: observed snapshots shared structurally across simulated versions (parent + delta), pure population rules, invariants as Quack semijoin / GroupReduce programs, semantic diff and a side-effect-free ExecutionPlan. No AD / Graph / Exchange / LDAP / PowerShell I/O."
+
+[workspace]
+
+[dependencies]
+lance-graph-quack = { path = "../lance-graph-quack" }
+lance-graph-mask-risc = { path = "../lance-graph-mask-risc" }
+ogar-dir-core = { path = "../../../OGAR/crates/ogar-dir-core" }
+ogar-dir-sim = { path = "../../../OGAR/crates/ogar-dir-sim" }
+ogar-ad = { path = "../../../OGAR/crates/ogar-ad" }

crates/lance-graph-dir-sim/src/exec.rs

@@ -0,0 +1,100 @@
+//! The one place a Quack [`Query`] is lowered and run.
+//!
+//! Every population operation in this crate is a `Filter` + `Agg` over
+//! borrowed lanes and resident planes, lowered by `lance-graph-quack` and
+//! executed by `lance-graph-mask-risc`. Nothing here iterates rows, builds a
+//! hash table, or produces joined tuples. The caller owns every buffer:
+//! scratch is one tile per slot (`Scratch::for_program`), a `Keep` lands in a
+//! `words_for(n_rows)` bitmap, a `GroupReduce` in a `K`-slot sink.
+
+use lance_graph_mask_risc::{
+    execute_into, materialize_rows, words_for, Foreign, Out, Planes, Program, Scratch, Terminal,
+    Value,
+};
+use lance_graph_quack::{lower, Agg, Col, Filter, GroupAddr, GroupAgg, Query};
+
+/// Lower a directory query. Directory queries are fixed shapes built in this
+/// crate, so a lowering failure is a programming error, not input.
+pub(crate) fn program(filter: Filter, agg: Agg) -> Program {
+    lower(&Query { filter, agg }).expect("directory queries lower")
+}
+
+fn run(p: &Program, planes: &Planes<'_>, foreign: &Foreign<'_>, out: Out<'_>) -> Value {
+    let mut scratch = Scratch::for_program(p, planes.n_rows).expect("scratch carves");
+    execute_into(p, planes, foreign, &mut scratch, out).expect("directory program runs")
+}
+
+/// The rows one program kept. Its only exit is [`Kept::rows`], the evidence
+/// boundary: it has no `&[u64]` view, so it cannot become another program's
+/// `ForeignPlane`. A `Semijoin` gathers only from resident planes (node kinds,
+/// the active-user plane), never from a population a program produced.
+///
+/// ```compile_fail
+/// use lance_graph_dir_sim::Kept;
+/// use lance_graph_mask_risc::ForeignPlane;
+/// fn feed(k: &Kept) -> ForeignPlane<'_> {
+///     ForeignPlane { words: k, rows: 0 }
+/// }
+/// ```
+///
+/// The same imports and shape compile against a resident plane:
+///
+/// ```
+/// use lance_graph_dir_sim::Kept;
+/// use lance_graph_mask_risc::ForeignPlane;
+/// fn feed<'a>(_k: &Kept, resident: &'a [u64]) -> ForeignPlane<'a> {
+///     ForeignPlane { words: resident, rows: 0 }
+/// }
+/// ```
+#[derive(Debug, PartialEq, Eq)]
+pub struct Kept {
+    bits: Vec<u64>,
+    n_rows: usize,
+}
+
+impl Kept {
+    /// Kept row indices, ascending. Bounded by the number of survivors.
+    pub fn rows(&self) -> Vec<usize> {
+        materialize_rows(&self.bits, self.n_rows)
+    }
+}
+
+/// Surviving rows (`Agg::Rows` → `Keep`), sealed in a [`Kept`].
+pub(crate) fn keep(p: &Program, planes: &Planes<'_>, foreign: &Foreign<'_>) -> Kept {
+    debug_assert!(matches!(p.terminal, Terminal::Keep { .. }));
+    let mut bits = vec![0u64; words_for(planes.n_rows)];
+    if planes.n_rows > 0 {
+        run(p, planes, foreign, Out::Mask(&mut bits));
+    }
+    Kept {
+        bits,
+        n_rows: planes.n_rows,
+    }
+}
+
+/// `GROUP BY key COUNT(*)` over the rows `filter` keeps, ADDED into `sink`
+/// (whose length is the group universe; keys past it — e.g. `NONE` — drop).
+pub(crate) fn group_count_into(
+    filter: Filter,
+    key: Col,
+    planes: &Planes<'_>,
+    foreign: &Foreign<'_>,
+    sink: &mut [i64],
+) {
+    if planes.n_rows == 0 || sink.is_empty() {
+        return;
+    }
+    let p = program(
+        filter,
+        Agg::GroupReduce {
+            key: GroupAddr::Local(key),
+            agg: GroupAgg::Count,
+        },
+    );
+    let mut part = vec![0i64; sink.len()];
+    let v = run(&p, planes, foreign, Out::I64(&mut part));
+    debug_assert_eq!(v, Value::GroupReduced);
+    for (s, x) in sink.iter_mut().zip(part) {
+        *s += x;
+    }
+}

crates/lance-graph-dir-sim/src/lib.rs

@@ -0,0 +1,79 @@
+//! # lance-graph-dir-sim — explore a directory future over the SoA substrate
+//!
+//! ```text
+//! observed G0 ──rule──► G1 ──rule──► G2 ──validate──► "desired" ──diff(G0,G2)──► ExecutionPlan ──X
+//! ```
+//!
+//! OGAR owns the meaning (`ogar-dir-sim`: `Change`, provenance, `Violation`,
+//! `ExecutionPlan`); this crate owns execution:
+//!
+//! * [`snapshot`] — one observation as SoA lanes and bit planes, `Guid128`
+//!   sorted so the ordinal is the index; strings in store dictionaries.
+//! * [`view`] — a version = shared `Arc<Snapshot>` + delta-sized overlay.
+//! * [`rule`] — pure population rules over a borrowed [`View`].
+//! * [`validate`] — invariants as Quack programs (`Semijoin` anti-joins,
+//!   `GroupReduce` counts).
+//! * [`store`] — append-only versions, tags, diff, plan, audit.
+//! * [`observe`] — `ogar-ad` records → observation.
+//!
+//! No network, process or file I/O. Nothing writes to AD, Entra, Exchange,
+//! LDAP or PowerShell. `#![forbid(unsafe_code)]`.
+
+#![forbid(unsafe_code)]
+
+mod exec;
+pub mod observe;
+pub mod rule;
+pub mod snapshot;
+pub mod store;
+pub mod validate;
+pub mod view;
+
+pub use exec::Kept;
+pub use rule::{member_counts, GrantGroup, ImplyGroup, Rule, SetPrimarySmtp};
+pub use snapshot::{
+    pack_ou, BuildError, Dict, Dicts, NodeKind, Observation, ObservedNode, Snapshot, NONE,
+};
+pub use store::{Rejection, SimError, VersionStore};
+pub use view::{ApplyError, View};
+
+use lance_graph_mask_risc::{Foreign, LaneRef, Planes};
+use lance_graph_quack::{Cmp, Col, Filter, Mask};
+use ogar_dir_core::OuHhtl;
+
+/// A subtree prefix deeper than the packed lane (4 levels) can address.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub struct SubtreeTooDeep(pub usize);
+
+/// Nodes located in the OU subtree `prefix` (ancestor-or-self), as a node
+/// bitmap — one ternary match on the packed OU lane (`Cmp::MatchU64`), no DN
+/// strings. Prefixes up to depth 4 are exact; deeper ones are refused.
+pub fn subtree(v: &View<'_>, prefix: &OuHhtl) -> Result<exec::Kept, SubtreeTooDeep> {
+    let d = prefix.depth();
+    if d > 4 {
+        return Err(SubtreeTooDeep(d));
+    }
+    let care = if d == 0 { 0 } else { u64::MAX << (64 - 16 * d) };
+    let s = v.snap;
+    let lanes = [LaneRef::U64(&s.ou_hi)];
+    let masks: [&[u64]; 1] = [&s.ou_present];
+    let planes = Planes {
+        n_rows: s.len(),
+        masks: &masks,
+        lanes: &lanes,
+    };
+    let p = exec::program(
+        Filter::and([
+            Filter::plane(Mask(0)),
+            Filter::cmp(
+                Col(0),
+                Cmp::MatchU64 {
+                    pattern: pack_ou(prefix),
+                    care,
+                },
+            ),
+        ]),
+        lance_graph_quack::Agg::Rows,
+    );
+    Ok(exec::keep(&p, &planes, &Foreign::NONE))
+}

crates/lance-graph-dir-sim/src/observe.rs

@@ -0,0 +1,59 @@
+//! `ogar-ad` records (OGAR PR #313) → an [`Observation`].
+//!
+//! The ingestion boundary: values are read out of the record's value pool
+//! once and handed to [`Snapshot::build`](crate::Snapshot::build) for
+//! interning. "Active" is derived from `userAccountControl` bit `0x2`
+//! (ACCOUNTDISABLE); the primary SMTP is the `SMTP:` proxy; the OU-HHTL is
+//! taken as-is (never a DN string). Memberships are relations that `ogar-ad`
+//! records do not carry; the caller adds observed ones.
+
+use crate::snapshot::{NodeKind, Observation, ObservedNode};
+use ogar_ad::{AdKind, SCHEMA_V1};
+use ogar_dir_core::{DirRecord, ValuePool};
+
+const UAC_ACCOUNTDISABLE: u32 = 0x2;
+
+fn slot(name: &str) -> usize {
+    SCHEMA_V1
+        .iter()
+        .find(|d| d.name == name)
+        .map(|d| d.slot as usize)
+        .expect("ogar-ad schema v1")
+}
+
+/// Users and groups among `records` (other kinds are skipped).
+pub fn from_ad(records: &[DirRecord], pool: &ValuePool) -> Observation {
+    let text = |r: &DirRecord, name: &str| {
+        r.str_ref(slot(name))
+            .and_then(|s| pool.get(s))
+            .and_then(|b| std::str::from_utf8(b).ok())
+            .map(str::to_string)
+    };
+    let mut obs = Observation::default();
+    for r in records {
+        let kind = match r.object_kind() {
+            k if k == AdKind::User as u16 => NodeKind::User,
+            k if k == AdKind::Group as u16 => NodeKind::Group,
+            _ => continue,
+        };
+        let primary_smtp = r
+            .str_ref(slot("proxyAddresses"))
+            .and_then(|s| pool.get_multi(s))
+            .and_then(|vs| {
+                vs.into_iter()
+                    .filter_map(|v| std::str::from_utf8(v).ok())
+                    .find_map(|v| v.strip_prefix("SMTP:").map(str::to_string))
+            });
+        obs.nodes.push((
+            r.node_guid(),
+            ObservedNode {
+                kind,
+                active: r.num(0).is_none_or(|uac| uac & UAC_ACCOUNTDISABLE == 0),
+                upn: text(r, "userPrincipalName"),
+                primary_smtp,
+                ou: r.ou_hhtl(),
+            },
+        ));
+    }
+    obs
+}