diff --git a/AGENTS.md b/AGENTS.md index d337d3294..f766de70c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,9 +6,9 @@ MetaObjects is a **cross-language metadata standard** for declaring typed entity The metamodel is the **durable spine**; generated code is the **disposable artifact**. Substrate is local-first: typed metadata lives in your repo, generated code is idiomatic per-language output that runs without any MetaObjects dependency at runtime. If `@metaobjectsdev/*` disappears tomorrow, you keep working code. -## Five pillars +## Six pillars -The first four ship per-language today across the five ports (TS / C# / Java / Python / Kotlin), with cross-port conformance corpora verifying byte-identical behavior. The fifth ships its vocabulary and its `verify` checks in every port, and its test scaffolding in TypeScript only: +The first four ship per-language today across the five ports (TS / C# / Java / Python / Kotlin), with cross-port conformance corpora verifying byte-identical behavior. The fifth ships its vocabulary and its `verify` checks in every port, and its test scaffolding in TypeScript only. The sixth is content the other five compose, shipped as named opt-in artifacts: 1. **Codegen** — emit idiomatic per-language code (Drizzle/Zod + Fastify for TS, EF Core + ASP.NET for C#, Spring REST + DTO + Repository for Java via `codegen-spring`, Pydantic + FastAPI for Python, KotlinPoet + Exposed + Spring for Kotlin via `codegen-kotlin`). Hand-edit-preserving regen via three-way merge. 2. **Runtime metadata** — load metadata at runtime, drive behavior dynamically (CRUD, validation, relationships, dynamic admin UIs, LLM tool registration). On Kysely (TS), a DB-API 2 driver via ObjectManager (Python), modernized JDBC + Spring-tx via OMDB (Java), Exposed (Kotlin), EF Core (C#). @@ -16,9 +16,11 @@ The first four ship per-language today across the five ports (TS / C# / Java / P 4. **Prompt construction** — a prompt is code, not a string scattered across services. Declare a prompt's payload as a typed projection (payload bloat becomes a diff), keep its text external and provider-resolved, and render it deterministically: snapshot-testable, cache-stable (no whitespace change silently breaking exact-prefix prompt-cache hits), and drift-checked at build time so a renamed field can't degrade a prompt. Conformance-gated, so the guarantee holds in every language port. **Render + payload-VO codegen + `verify` + parser-on-receipt for a *responding* `template.prompt` — one carrying `@responseRef` (FR-006) — + the output-format prompt fragment & tolerant `extract` parser (FR-010) ship in all five ports today** (since 0.24.0 the whole inbound tier keys off `@responseRef`; a `template.output` is outbound-only and emits no parser — ADR-0052) — the library-side building blocks of the pillar are complete. The one remaining library-side piece is MCP exposure of declared prompts/tools (see `spec/roadmap.md`); the application-level consolidation (eval harness, end-to-end declared-prompt orchestration) and consumer adoption are exercised in adopter projects, not in this library repo. Designed in `docs/superpowers/specs/2026-05-22-fr-004-cross-language-prompt-construction-design.md`. 5. **Requirements and testing** — declare what the software is supposed to *do* in the same model as the entities, so a capability claim is checkable instead of prose. The other four pillars keep the code honest about the *model*; this one asks whether the thing you said the software does is actually built — an absence no test can fail on, because a test exercises code that exists. `requirement.functional` (fails when *nothing* implements it) and `requirement.architectural` (fails when something *violates* it) are registered vocabulary in all five ports, with the loader enforcing the closed `@status` enum. `@implementedBy` is **resolved, not trusted** — it names a real member of the real model, so a claim whose implementation was renamed or deleted fails the build instead of going quietly stale. `meta verify` reports the ledger on every run (unresolved links, entities no claim covers, gaps recorded versus gaps nobody has ruled on) plus an authoring lint whose findings can never fail a build; `meta docs` renders it for humans and for agents. **The port split, stated exactly: the vocabulary and the `verify` checks are cross-port; `requirementTests()` — which scaffolds a test stub per claim — is TypeScript-only.** A project that declares no `requirement.*` nodes sees no change at all. Note the standing carve-out: `agent-context/skills/metaobjects-fit-assessment/SKILL.md` deliberately does NOT treat `requirement.*` as an assessment axis — see the ruling in that file before "finishing the job" there. +6. **Libraries** — reusable declared design, shipped as metadata and opted into by name (`"libraries": ["iam"]` in `.metaobjects/config.json`). Not a sixth verb: a library is the **reuse unit that composes the other five** — entities, requirements, the generators its design implies, its runtime packages — as one named, opt-in, drift-gated artifact. What makes it a pillar rather than a folder of YAML is the fifth: without requirements a library is a schema snippet; with them it is design an adopter's build is held to, which is the same test the requirements pillar passes (*does it change what an agent can be checked against?*). **A library is LAYERED and its core layer is INERT** — the core declares no `source.rdb`, and a sourceless object generates nothing and migrates to nothing (#248), so `["iam"]` adds zero tables and zero generated code while making the design present and resolvable; `["iam", "iam/db"]` is the separate opt-in that proposes the schema. **Copy is the expected mode** (`meta eject ` — ADR-0034's ruling applied to metadata), and an ejected copy still named in `libraries` is refused at load (`ERR_LIBRARY_PACKAGE_COLLISION`) rather than merging asymmetrically. Object coverage activates on ADOPTER-authored requirements only, so a library cannot volunteer a project for a gate it did not ask for. `iam` (preview) and `ai` (stable) ship today; rows appear in `meta gen --list` beside the generators. See [docs/features/libraries.md](docs/features/libraries.md) and `docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md`. + ## Status -_Last refreshed 2026-09-12._ +_Last refreshed 2026-09-13._ **1.0 gating — the quiet period is RETIRED (2026-09-06).** `docs/1.0-readiness.md` §G3 no longer asks for "one coordinated release with no metamodel-breaking change." It measured a @@ -68,7 +70,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (322 fixtures; 22 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (325 fixtures; 22 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. diff --git a/CHANGELOG.md b/CHANGELOG.md index efa4fea4c..a815e7686 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,155 @@ here.** ## [Unreleased] +### Added + +- **Libraries: reusable declared design you opt into** (FR-043, the sixth pillar). + `"libraries": ["iam"]` in `.metaobjects/config.json` brings a shipped, requirement-backed + model into your project. Two ship: **`iam`** (`preview`) — users, nestable typed groups, + roles as permission bundles, grants global or scoped to a group, nine entities and eleven + requirements — and **`ai`** (`stable`), the LLM-call trace envelope that already existed. + + **A library is LAYERED, and the core layer is INERT.** The core declares no `source.rdb`, + and a sourceless object generates nothing and migrates to nothing (#248), so + `["iam"]` adds **zero tables and zero generated code** — the design is present and + resolvable, and nothing else happens until you add `["iam", "iam/db"]`. A layer token + implies its core; a token whose layer is unknown is dropped whole rather than reduced to + it, because answering a mistyped `iam/database` with an inert core and no tables is the + worst of the available outcomes. + + **Copy is the expected mode.** `meta eject ` copies every layer into your first + DECLARED source root with a provenance header, and `meta eject --list` reports how far + your copy has drifted from the shipped tree — nodes changed, only-upstream, only-yours — + matched by name with the package neutralized, because renaming the package is something + you are invited to do. Ejecting and leaving the library in `libraries` is refused at load + (`ERR_LIBRARY_PACKAGE_COLLISION`): both trees merge, and the merge is asymmetric — + additions take effect, deletions do not. Its mirror `ERR_LIBRARY_PACKAGE_NOT_OWNED` + refuses a NEW node declared into a library's package; an `overlay: true` amendment stays + open. See [libraries.md](docs/features/libraries.md). + + `meta gen --list` carries `kind: "library"` rows beside the generators — one door, one + namespace — with `useWhen`, `layers`, `provides`, and under `--probe` what your selection + actually added here. + +- **`overlay: true` licenses an attribute override.** `ERR_MERGE_CONFLICT` now fires only + on an UNMARKED conflicting redeclaration. The flag is the author saying "I know about the + other declaration and I mean to change it", and without this an adopter could not disagree + with a library's shipped requirement without ejecting the whole ledger. All four loaders + (Kotlin inherits the JVM's); one new conformance fixture takes over the unmarked-conflict + error branch, so the coverage moved rather than being deleted. + +### Changed + +- **`libraries` moved to `.metaobjects/config.json`**, out of `metaobjects.config.ts`, + outright and with no dual-read — a sweep of the estate found zero uses of the key. Which + designs a project adopts is a fact about the PROJECT, not about how one port generates + code from it. A config still carrying the old key gets a pointed error rather than + silence. + +- **`library/ai` is SPLIT into `model` + `db` layers, and its requirements are new.** + Opting into `"ai"` alone no longer proposes `CREATE TABLE llm_call` — that moved to + `"ai/db"`. Breaking-ish for an `ai` adopter tracking the library: add `"ai/db"` to keep + the table. The concrete-`LlmCall` wart was previously carried as accepted on the grounds + that splitting would change what existing adopters get; the estate sweep found there are + none, so it was closed rather than documented. + +- **`trace-helper` keys on a declared ANCHOR, not a hard-coded entity name** — and the + name it hard-coded was never actually matching the shipped base. It compared + `"LlmCallBase"` against the SHORT name, so any adopter entity of that name in any package + emitted a helper writing columns that entity does not declare. It now resolves the anchor + its library's manifest declares and compares by node identity, in all three ports that + ship it. Two self-extinguishing warnings cover the halves of the choice: a library opted + into whose implied generator is not wired, and a generator wired whose library is not. + +- **Object coverage activates on ADOPTER-authored requirements only.** A library shipping + its own ledger would otherwise switch the unclaimed-entity gate on across a project that + has never written a requirement. Library entries are still counted and still checked; + they simply cannot volunteer you. `meta verify` prints `coverage: not measured (no + project-authored requirements)` rather than a ratio, and the JSON omits the pair rather + than zeroing it — `0/0 claimed` and "not measured" mean opposite things. + + **A project that already has a ledger sees its coverage denominator grow** to include an + opted-in library's entities. They are all claimed by the library's own ledger, so no new + warnings appear, but the printed numbers move. + +- **A shipped library's files carry a stable `library:.yaml` source id in every + build.** It was the file's basename in a checkout and `library:…` when embedded, so one + node's error envelope read differently depending on how the library was resolved — and + collided with an adopter file of that name. + +### Fixed + +- **Both shipped libraries failed `meta verify`'s requirement gate**, in metadata an + adopter cannot fix: every L4 in `ai` claimed FIELDS (`ERR_REQUIREMENT_L4_NOT_OBJECT`), + and both libraries wrote their concerns as SIBLINGS of the L2 segment their own comments + said they were children of, leaving that L2 claiming nothing in its whole subtree. The + load test proved they LOAD clean, which is a different claim, and nothing checked the + other one. Both ledgers are fixed as the model intends — concerns nested under their L2, + each L4 naming the OBJECT with its fields in an L5 child — and a new standalone gate + holds every shipped library, and every future one, to zero loader errors, zero loader + warnings, zero gate findings, zero lint findings, no unruled gaps, and every entity + claimed by its own ledger. + +### Changed + +- **Codegen is OPT-IN: no port ships a default generator suite** (ADR-0034 Amendment 2). + A new project got code it never asked for — TypeScript's `meta init` copied and wired + five generators, C# ran nine for a caller who named none, Python eight; Java never had + a default set and has been right all along. Deciding which code an application needs + belongs to whoever is building it, increasingly an LLM in the repo, which is well able + to make that call given a truthful catalog and is badly served by a default that + pre-empts it. + + **This is a PATCH and no existing project changes by one byte.** An adopter already has + their owned copies on disk and their selection committed in their own config; `meta gen` + keeps running exactly that list, `verify --codegen` keeps checking exactly that output, + and re-running `init` never clobbers a file that exists. What changes is what a *new* + project starts with. `docs/compatibility-policy.md` is narrowed in the same change: the + scaffold-and-own promise is the LAYOUT and the INTERFACES, not which generators a fresh + scaffold wires. + + What this means per port: + - **TypeScript** — `meta init` scaffolds `codegen/generators/` **empty**, a config with + `generators: []`, and no dependencies. `dbImport` and the throwing `src/db.ts` stub are + gone with it: both existed only because the scaffold wired `routesFile()`, whose output + emits `import { db } from …`. `dbImport` is now a declared `configKey` on the `routes` + catalog entry, reported by `meta eject routes` to the adopter who chose routes. + - **C# / Python** — `--generators` is REQUIRED; a run that names none is a usage error + and writes nothing. `verify --codegen` re-runs the SELECTION, so with none named it + reports that there is nothing to check rather than regenerating a suite the project + never ran. Python's `verify` gains `--generators`, matching C#. + +- **`meta gen --list` is now the generator CATALOG, and `--probe` answers it against your + own model.** `--list --format json` emits one document per generator: its `layer`, + `framework`, what it emits, what it `requires`, the consolidated install set, the config + keys it reads, and whether this project already owns a copy. `--probe` constructs every + generator and dry-runs it against the loaded model, reporting how many files each would + emit — so `output-parser: 3, callable: 0, requirement-tests: 7` replaces a category + label, and cannot go stale, because it runs the generators rather than describing them. + +- **`meta eject` takes many names and reports one consolidated install set.** A real + selection is several generators, and three separate invocations produced three separate + install lines for the same package. `--format json` carries, per file, the import line + and the entry to wire, plus one install set with third-party ranges read from the runtime + package's own `peerDependencies`. An unknown name refuses the whole call before writing + anything. + +### Added + +- **`layer` joins the cross-port generator manifest** — `model` / `persistence` / `api` / + `client` / `docs` / `capability` — gated by all five ports' registry-conformance tests + exactly as `tier` is. Five TypeScript generators that were in no manifest at all (`form`, + `hooks`, `grid`, `grid-hook`, `requirement-tests`) join it too, so the catalog describes + 34 generators where the manifest described 29. + +- **`meta gen` audits the selection.** Two self-extinguishing warnings, neither a build + failure: a wired generator whose `requires` are not wired (its output will import a + module nothing emits), and two `api`-layer generators declaring different frameworks + (`routes` + `routes-hono` emit to different paths, so nothing conflicts and two complete + HTTP surfaces appear silently). There is deliberately no equivalent rule on the `client` + layer: `@metaobjectsdev/tanstack` peers on `react`, so a form generator plus the TanStack + hook/grid generators is the intended composition. + ### Fixed - **Generated files no longer point at a plugin point that does not exist ([#367]).** diff --git a/agent-context/skills/metaobjects-audit/SKILL.md b/agent-context/skills/metaobjects-audit/SKILL.md index 82ed5693b..5f1238aff 100644 --- a/agent-context/skills/metaobjects-audit/SKILL.md +++ b/agent-context/skills/metaobjects-audit/SKILL.md @@ -351,7 +351,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on THREE of five ports an existing project emits none**. C# and Python have a real default suite and get it by upgrading; TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, so `meta init` scaffolding `namesFile()` covers only a project initialized at 1.0 and upgrading the package never edits a config written earlier. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. diff --git a/agent-context/skills/metaobjects-audit/references/capability-checklist.md b/agent-context/skills/metaobjects-audit/references/capability-checklist.md index 3a05b86ca..8b3265072 100644 --- a/agent-context/skills/metaobjects-audit/references/capability-checklist.md +++ b/agent-context/skills/metaobjects-audit/references/capability-checklist.md @@ -71,10 +71,10 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove migration script, a body-to-column map (drift signature 11). Every port emits a per-object names artifact from the declaration, so a literal is a second source of truth even when it agrees with the naming strategy today. A typed ORM handle in its place is correct. **Check - the artifact is emitted at all before scoring the literals: on TypeScript and the JVM the - generator list in the config IS the complete list, so an existing project emits none and the - un-wired generator is the finding FIRST** (C# and Python have a real default suite and get it - by upgrading). **This entry is the physical-name INSTANCE of signature 11.** The rule that + the artifact is emitted at all before scoring the literals: on every port the selection IS + the complete list — the config's on TypeScript and the JVM, `--generators` on C# and Python — + so an existing project emits none and the un-wired generator is the finding FIRST** (no port + ships a default suite; ADR-0034 Amendment 2). **This entry is the physical-name INSTANCE of signature 11.** The rule that generates the rest — and the reason an enum member compared as a bare string is NOT one — is the Cross-cutting entry below. - **`@kind` = `view` / `materializedView`** — hunt hand-written SQL views where an authored diff --git a/agent-context/skills/metaobjects-audit/references/csharp.md b/agent-context/skills/metaobjects-audit/references/csharp.md index e41b9f841..a2bee8c5d 100644 --- a/agent-context/skills/metaobjects-audit/references/csharp.md +++ b/agent-context/skills/metaobjects-audit/references/csharp.md @@ -51,7 +51,7 @@ rejected (exit 2). | `FromSqlInterpolated(` outside `.g.cs` | stored-proc call — candidate for the `callable` generator | | `// keep in sync with` / `// mirrors the` | second-source-of-truth comment — always a finding | | `HasPrecision(` hand-coded | `field.decimal` with `@precision`/`@scale` drives this from the `entity` generator | -| a table/column string in raw-SQL EF calls, or `nameof(Entity.Prop)` standing in for a column | second spelling of a declared physical name — reference `Names.g.cs` (`AuthorNames.SourcePrimaryTable` / `Column`, default suite — `Names.Name` is the OBJECT's name, not the table); an EF property inside LINQ is the typed handle — correct | +| a table/column string in raw-SQL EF calls, or `nameof(Entity.Prop)` standing in for a column | second spelling of a declared physical name — reference `Names.g.cs` (`AuthorNames.SourcePrimaryTable` / `Column`, emitted when `names` is named in `--generators` — `Names.Name` is the OBJECT's name, not the table); an EF property inside LINQ is the typed handle — correct | --- @@ -66,7 +66,7 @@ selection uses stable names via `dotnet meta gen --generators `, over a **So do not score a C# project down for "not owning its generators", and do not recommend writing one.** The customization path here is the **declarative template**: `dotnet meta gen --template-spec --template-root `, whose entries append to -the default suite. A finding of the form "the built-ins do not emit the shape this +your `--generators` selection. A finding of the form "the built-ins do not emit the shape this project needs" resolves to a template-spec, not to generator code. Worked example with the full JSON: `docs/ports/csharp.md`. diff --git a/agent-context/skills/metaobjects-audit/references/python.md b/agent-context/skills/metaobjects-audit/references/python.md index 7d9bc1d30..7f9450aeb 100644 --- a/agent-context/skills/metaobjects-audit/references/python.md +++ b/agent-context/skills/metaobjects-audit/references/python.md @@ -52,7 +52,7 @@ regardless of server language — see the migration reference. | `def get_all_` / `def create_` / `def update_` / `def delete_` in non-generated files | hand-rolled CRUD — compare to the generated router | | `# keep in sync with` / `# mirrors the` | second-source-of-truth comment — always a finding | | `try: ... except KeyError` / `?? ''` around format strings in prompt code | silent-degradation hack around a prompt payload — flag it | -| a table/column string in the repository implementation (SQLAlchemy Core, asyncpg/psycopg SQL) | second spelling of a declared physical name — reference `_names.py` (`AUTHOR_SOURCE_PRIMARY_TABLE` / `AUTHOR__COLUMN`, default suite); no typed handle exists on this port, so that is never the reason to waive it | +| a table/column string in the repository implementation (SQLAlchemy Core, asyncpg/psycopg SQL) | second spelling of a declared physical name — reference `_names.py` (`AUTHOR_SOURCE_PRIMARY_TABLE` / `AUTHOR__COLUMN`, emitted when `names` is named in `--generators`); no typed handle exists on this port, so that is never the reason to waive it | --- @@ -66,8 +66,8 @@ seam to register a generator of your own. (`--provider module:symbol` registers **So do not score a Python project down for "not owning its generators", and do not recommend writing one.** The customization path here is the **declarative template**: -`metaobjects gen --template-spec --templates `, whose entries append to the -default suite. A finding of the form "the built-ins do not emit the shape this project +`metaobjects gen --template-spec --templates `, whose entries append to your +`--generators` selection. A finding of the form "the built-ins do not emit the shape this project needs" resolves to a template-spec, not to generator code. Worked example with the full JSON: `docs/ports/python.md`. diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 86bfc2dfd..c40bce3c3 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -46,22 +46,28 @@ is, this once:** **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: -1. **Search the vocabulary** — `meta types `, or `meta types --all +1. **Check whether the design already ships.** `meta gen --list --format json` + carries `kind: "library"` rows — declared design MetaObjects ships, each with a + `useWhen`. If one matches the capability you are about to declare, opt in and adapt + rather than author it from scratch: you inherit its entities, its requirements, and + the rulings recorded with them. The bare name is the core layer (sourceless — no + tables, no generated code); `/db` is the separate opt-in that adds the schema. +2. **Search the vocabulary** — `meta types `, or `meta types --all ` to search by behavior. There are field subtypes, relationships, projections, origins, identities, sources, and attributes you may not know exist. Find the construct that models it. Add `--detail` for one construct's valid `@attrs`, or `--format json` for the same answer as one machine-readable document — that form carries every match (`--limit` never truncates it) with each attr's `allowedValues`, so you read the accepted values rather than guessing them. -2. **Declare it and generate** — then *consume* the generated query/type/route; +3. **Declare it and generate** — then *consume* the generated query/type/route; never reimplement it alongside. -3. **If the model is right but the generated OUTPUT is wrong, change your generator.** +4. **If the model is right but the generated OUTPUT is wrong, change your generator.** Naming, file layout, imports, framework, signatures are generator concerns, not reasons to hand-write. The generators are in *your* repo and are yours to edit — a standing rule not to change the MetaObjects repo does not reach them; they are a different repository. Editing one is ordinary work, not an escalation. (See `metaobjects-codegen` → "Your generators are yours".) -4. **Only if no construct can express it** — and you have actually looked — +5. **Only if no construct can express it** — and you have actually looked — hand-write it, wired to generated types. Business algorithms, external integrations, and bespoke interactions are legitimately hand-written; CRUD, validation, finders, relationships, and derived/aggregate data are not. diff --git a/agent-context/skills/metaobjects-codegen/SKILL.md b/agent-context/skills/metaobjects-codegen/SKILL.md index 247af390d..5f3350cf2 100644 --- a/agent-context/skills/metaobjects-codegen/SKILL.md +++ b/agent-context/skills/metaobjects-codegen/SKILL.md @@ -124,19 +124,72 @@ implementing the port's generator interface elsewhere. Your language reference h mechanism; see also "The commands and config keys that implement the steps above differ per port" below. -## Selecting generators by stable name +## Selecting generators — NOTHING is generated until you choose it + +**`meta init` wires no generators, and no port ships a default suite.** A fresh +scaffold has `generators: []` and an empty `codegen/generators/`; `--generators` is +required on the C# and Python CLIs; Java has never had a default set. Choosing what an +application needs is a judgment over its purpose and stack, and the tool's job is to +make that choice cheap and truthful, not to make it for you. + +Each generator has a **stable name** (kebab-case) that is the same in every port and +surfaces in diagnostics — reference generators by that name, never by inlining what +they emit. + +### The procedure + +1. **Read the app's purpose and stack.** What is it for; what does it already use. +2. **`meta gen --list --format json --probe`** — the catalog. Every generator with its + `layer`, `framework`, what it emits, what it requires, what it costs to install, + and — with `--probe`, which constructs each generator and dry-runs it against YOUR + model — how many files each would actually emit. +3. **Check the libraries before you choose generators.** The same catalog carries + `kind: "library"` rows — declared design MetaObjects ships, each with a `useWhen`. + If one matches a capability you are about to model, **opt in and adapt rather than + author**: you inherit its entities, its requirements, and the rulings recorded with + them. Layers are how much of it you take. The bare name is the CORE layer — the + model and its ledger, sourceless, so it adds **no tables and no generated code**; + `/db` is the separate opt-in that proposes the schema. Opt in with + `"libraries": ["iam", "iam/db"]` in `.metaobjects/config.json`. +4. **Choose by `layer`.** Satisfy every `requires`. Take what `wouldEmit > 0` says your + model is already asking for. + - Pick **ONE** framework on the `api` layer: `routes` and `routes-hono` are + alternatives, and wiring both silently produces two complete HTTP surfaces. + - Do **NOT** apply that rule to `client`. `@metaobjectsdev/tanstack` peers on + `react`, so a form generator plus the TanStack hook/grid generators is the + intended composition, not a conflict. +5. **`meta eject --format json`** — copies each into `codegen/generators/` + (yours to edit), and reports the import line, the entry to add to `generators`, one + consolidated install command, and any config keys those generators read. +6. **`meta gen`** — read its warnings, then typecheck. + +A library row also carries `provides` (what is in the box) and, under `--probe`, a +`project` block: which layers you selected, how many tables and requirements they +added here, which of your entities `extends` into it, and any generator the library +implies that you have not wired. + +### The six layers, and what picks them + +| layer | what it is | chosen by | +|---|---|---| +| `model` | entity/DTO/value-object modules and the constants beside them | app shape | +| `persistence` | how rows are read and written | app shape | +| `api` | the HTTP surface — pick one framework | app shape | +| `client` | the browser tier | app shape | +| `docs` | on by default; the canonical door is `meta docs` | — | +| `capability` | prompts, parsers, payloads, traces, test stubs | **`--probe`** | + +At intent level: a **headless data service** is `model` + `persistence`; an **HTTP API** +adds `api` with one framework; an **admin UI** adds `client`. Members come from the live +catalog, never from a list in prose — a list here would go stale the day a generator is +added, and `--probe` cannot, because it runs the generators rather than describing them. -Codegen is a set of named generators you opt into. Each generator has a **stable -name** (kebab-case) that surfaces in diagnostics — reference generators by that -name, never by inlining what they emit. Typical generators cover: the entity -type/model, the DB table/schema, query/finder helpers, REST routes, client -form/grid/hook artifacts, filter + sort allowlists, payload value-objects, and -parsers for a responding `template.prompt` (one carrying `@responseRef`). You -enable the subset your project needs; an abstract entity never emits -instance/write artifacts regardless. +`capability` looking like one large bucket is the point: you are not meant to choose +inside it by reading labels. You declared a `template.prompt`, or a +`requirement.functional`, or a stored-proc source — `--probe` reports the file count +that follows, for your model. -Per-entity opt-outs exist (e.g. skipping client-side artifacts for a given -entity) and are set as attributes on the entity in metadata, not in code. +An abstract entity never emits instance/write artifacts regardless of what is wired. ## A dependency's metadata is load-only by default — codegen excludes it @@ -239,8 +292,9 @@ the data access too. **`@unmanaged: true`** (view or table); migrate/verify then never touch it. `@sql` and `@unmanaged` are mutually exclusive. -`meta gen --list` prints every generator by stable name; the `generators` array in -`metaobjects.config.ts` is where you opt each one in or out. +`meta gen --list` prints every generator by stable name (add `--probe` for a file count +against your own model); the `generators` array in `metaobjects.config.ts` is where you +opt each one in. It starts empty. ### Adopting onto existing code — make codegen match the code, not the code match codegen @@ -435,7 +489,8 @@ generator sets are **closed built-in registries** — `--generators` *selects* f ships, and there is no seam to register a `Generator` of your own. (Python's `--provider module:symbol` registers **metamodel vocabulary**, not a generator; do not reach for it here.) Use `--template-spec ` — plus `--templates ` on Python or -`--template-root ` on C# — and your entries are appended to the default suite. Worked +`--template-root ` on C# — and your entries are appended to your `--generators` +selection. Worked examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`. **The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both @@ -490,21 +545,21 @@ everywhere — **each physical name is spelled once, and generated code referenc | Kotlin | `Names.kt` | `ProgramNames.CREATED_AT_COLUMN` | | Python | `_names.py` | `PROGRAM_CREATED_AT_COLUMN` | -**Check that it is actually wired before you reference it — on three of five ports an -EXISTING project emits none.** "In the default suite" and "what a fresh scaffold writes" -are different facts, and only C# and Python have the first: +**Check that it is actually wired before you reference it — on ALL FIVE ports a project +emits none until it asks for it.** ADR-0034 Amendment 2 made codegen opt-in everywhere: no +port ships a default suite, so there is no port on which upgrading the package starts +emitting this artifact. "Wired" is the only fact there is. -| Port | Where the suite is decided | An existing project upgrading gets it? | +| Port | Where the selection is declared | An existing project upgrading gets it? | |---|---|---| -| C# | `GenCommand.DefaultGeneratorNames` — a real default | **Yes**, with no edit | -| Python | `cli.py` `_default_generators()` — a real default | **Yes**, with no edit | -| TypeScript | `metaobjects.config.ts` `generators: [...]` — **the complete list; there is no default suite** | **No** — add `namesFile()` | +| C# | `--generators ` on `dotnet meta gen` — required, no default | **No** — name `names` | +| Python | `--generators ` on `metaobjects gen` — required, no default | **No** — name `names` | +| TypeScript | `metaobjects.config.ts` `generators: [...]` — the complete list | **No** — add `namesFile()` | | Java / Kotlin | the pom's `` — the complete list | **No** — add `SpringNamesGenerator` / `KotlinNamesGenerator` | -TypeScript's `meta init` scaffolds `namesFile()`, so a project *initialized* at 1.0 has it; -a project initialized earlier has the config `meta init` wrote then, and upgrading the -package never edits a config. So on TypeScript the artifact is opt-in exactly as it is on -the JVM — the scaffold is not a default. To wire it into an existing TS project: +`meta init` scaffolds `generators: []` and an empty `codegen/generators/`, so even a +project *initialized* at 1.0 has to choose this one — the scaffold is deliberately not a +default by another name. To wire it into a TS project: ```ts import { namesFile } from "./codegen/generators/names.js"; // after `meta eject names` diff --git a/agent-context/skills/metaobjects-codegen/references/csharp.md b/agent-context/skills/metaobjects-codegen/references/csharp.md index 43879969f..5b6883478 100644 --- a/agent-context/skills/metaobjects-codegen/references/csharp.md +++ b/agent-context/skills/metaobjects-codegen/references/csharp.md @@ -28,6 +28,10 @@ dotnet meta gen metaobjects --out Generated --generators entity,db-context,route dotnet meta verify metaobjects --codegen --out Generated # codegen-drift gate (regenerate + diff vs committed) ``` +`dotnet meta verify --codegen` re-runs the SELECTION and diffs, so it takes the same +`--generators` the `gen` that produced the output used; with none named it reports that +there is nothing to check. + `dotnet meta verify` defaults to `--templates` (the FR-004 prompt/template drift gate, see the prompts reference); `--codegen` is the codegen-output drift gate. **Schema migration + live-DB drift are NOT `dotnet meta`** — they run through the @@ -35,8 +39,10 @@ Node `meta` tool (see the migration reference). ## `MetaObjects.Codegen` generators -Wire generators by their stable name (`dotnet meta gen --generators `), -or run the default set. Output lands under `--namespace` in `--output-dir`. +Wire generators by their stable name — **`--generators ` is REQUIRED**. There is +no default set: a run that names none is a usage error and writes nothing (ADR-0034 +Amendment 2). `dotnet meta gen --list` is the catalog. Output lands under `--namespace` +in `--output-dir`. | Stable name | Output | |---|---| diff --git a/agent-context/skills/metaobjects-codegen/references/python.md b/agent-context/skills/metaobjects-codegen/references/python.md index 8d69832f5..b7290c799 100644 --- a/agent-context/skills/metaobjects-codegen/references/python.md +++ b/agent-context/skills/metaobjects-codegen/references/python.md @@ -50,7 +50,11 @@ meta docs --out ./docs # Node: run from the PROJECT ROOT (no ## Generators -Wire generators by their stable name (`--generators `), or run the default set. +Wire generators by their stable name — **`--generators ` is REQUIRED**. There is +no default set: a run that names none is a usage error and writes nothing (ADR-0034 +Amendment 2). `metaobjects gen --list` is the catalog. `verify --codegen` re-runs the +SELECTION and diffs, so it takes the same `--generators`; with none named it reports that +there is nothing to check. Output lands under `--out` (with the `@generated` guard header). Metadata is the same canonical JSON every port reads (fused-key form, `source.rdb` + `@table`, `@column` for a renamed physical column). diff --git a/agent-context/skills/metaobjects-codegen/references/typescript.md b/agent-context/skills/metaobjects-codegen/references/typescript.md index db252d560..cb9c21fed 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript.md @@ -32,9 +32,13 @@ npm install --save-dev @metaobjectsdev/codegen-ts-react @metaobjectsdev/codegen- Codegen is wired in a type-checked TS config at the project root. `defineConfig` comes from `@metaobjectsdev/cli`; the generators come from their packages. +`meta init` scaffolds this file with **`generators: []`** — nothing is generated until +you choose it. Each import below appears once you `meta eject` that generator, which +prints the exact line to add. + ```ts import { defineConfig } from "@metaobjectsdev/cli"; -// Owned generators scaffolded by `meta init` (ADR-0034 scaffold-and-own). +// Owned generators — copied in by `meta eject` (ADR-0034 scaffold-and-own). import { entityFile } from "./codegen/generators/entity"; import { queriesFile } from "./codegen/generators/queries"; import { routesFile } from "./codegen/generators/routes"; @@ -83,13 +87,19 @@ PROJECT ROOT that CONTAINS the metadata — never the metadata directory itself. ## The generators -Server-side, framework-neutral. The first four are **scaffolded into your repo** by -`meta init` and imported from `./codegen/generators/*` (ADR-0034) — 1.0 REMOVED their -`@metaobjectsdev/codegen-ts/generators` export, so an owned copy is the only path. The -engine primitives come from the package main entry, `@metaobjectsdev/codegen-ts`. The -`/generators` subpath itself is NOT deprecated: it is the supported home of the generators -with no ownable copy — `promptRender`, `outputParser`, `outputPrompt`, `extractor`, -`renderHelper`, `traceHelperFile`, `routesFileHono`, `namesFile`, `callableFile`. +Server-side, framework-neutral. **None is wired by default** — `meta init` writes +`generators: []` and an empty `codegen/generators/`. `meta eject ...` copies the +ownable ones into your repo, imported from `./codegen/generators/*` (ADR-0034); 1.0 +REMOVED their `@metaobjectsdev/codegen-ts/generators` export, so an owned copy is the only +path for those. The engine primitives come from the package main entry, +`@metaobjectsdev/codegen-ts`. The `/generators` subpath itself is NOT deprecated: it is the +supported home of the generators with no ownable copy — `promptRender`, `outputParser`, +`outputPrompt`, `extractor`, `renderHelper`, `traceHelperFile`, `namesFile`, +`callableFile`, `requirementTests`. + +The table below is a per-emission reference, NOT the selection surface. Select with +`meta gen --list --format json --probe`, which is generated from the live registry and +reports a file count for your own model; a table in a document cannot do either. | Generator | Emits per entity | |---|---| diff --git a/agent-context/skills/metaobjects-runtime-ui/SKILL.md b/agent-context/skills/metaobjects-runtime-ui/SKILL.md index 722a14971..adb6a8c32 100644 --- a/agent-context/skills/metaobjects-runtime-ui/SKILL.md +++ b/agent-context/skills/metaobjects-runtime-ui/SKILL.md @@ -64,14 +64,14 @@ handle: raw SQL, a string-keyed query builder, and a hand-written repository on whose generated model carries no persistence binding at all (Java, Python). Your server reference names the handle and the artifact for this stack. -**First check the artifact exists — on TypeScript and the JVM it is opt-in, and an -existing project almost certainly has none.** C# and Python emit it from a real default -suite, so upgrading is enough. TypeScript's `generators: [...]` and the JVM's -`` are each the COMPLETE list: `meta init` scaffolds `namesFile()` for a -project initialized at 1.0, and upgrading the package never edits a config that was -written earlier. Look for `.names.ts` / `Names` in the generated output -before you write `ProgramNames.fields.x.column` against it; if it is not there, wiring the -generator is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL +**First check the artifact exists — it is opt-in on EVERY port, and an existing project +almost certainly has none.** Every port's generator list is now the COMPLETE list: C# and +Python dropped their default suites (ADR-0034 Amendment 2), TypeScript's `generators: []` +starts empty, and the JVM's `` never had a default. Upgrading the package +never edits a config that was written earlier, so `names` arrives only when someone wires +it. Look for `.names.ts` / `Names` in the generated output before you +write `ProgramNames.fields.x.column` against it; if it is not there, wiring the generator +is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL with literal names because "there is no constant" is the loop this closes. ## The REST contract diff --git a/agent-context/skills/metaobjects-runtime-ui/references/csharp.md b/agent-context/skills/metaobjects-runtime-ui/references/csharp.md index de3002982..f9060799a 100644 --- a/agent-context/skills/metaobjects-runtime-ui/references/csharp.md +++ b/agent-context/skills/metaobjects-runtime-ui/references/csharp.md @@ -94,8 +94,9 @@ the Java, Kotlin, and Python backends byte-for-byte. Inside LINQ keep the property (`db.Authors.Where(a => a.Name == …)`): it is type-checked against the model, and a string constant there trades a compile error for a runtime one. Where LINQ does not reach — raw SQL, a migration script, a log line — take the physical -name from the generated `Names.g.cs`. `names` is in the default suite, and the -generated entity and `AppDbContext` already read it, so it cannot disagree with the +name from the generated `Names.g.cs`. `names` is opt-in — name it in +`--generators` — and once selected the generated entity and `AppDbContext` already read +it, so it cannot disagree with the mapping: `AuthorNames.SourcePrimaryTable` is the table (the member is named for the source's `@kind`, so a view reads `SourceReplicaView` / `SourcePrimaryView` and a stored procedure `SourcePrimaryProc`), `AuthorNames.Column` the column, diff --git a/agent-context/skills/metaobjects-runtime-ui/references/python.md b/agent-context/skills/metaobjects-runtime-ui/references/python.md index 5102730ac..73f8484d9 100644 --- a/agent-context/skills/metaobjects-runtime-ui/references/python.md +++ b/agent-context/skills/metaobjects-runtime-ui/references/python.md @@ -100,7 +100,7 @@ resolve the column themselves, so that path never needs a physical name. The mom repository `Protocol` is backed by your own SQLAlchemy Core / asyncpg / psycopg code, it does — and nothing Python generates carries one: the Pydantic models, create/patch shapes, router and allowlist all key by field. Take it from the generated -`_names.py` (`names` is in the default suite): +`_names.py` (`names` is opt-in — name it in `--generators`): ```python from generated.author_names import ( diff --git a/agent-context/templates/always-on.md.mustache b/agent-context/templates/always-on.md.mustache index b403ef251..0eefde6cd 100644 --- a/agent-context/templates/always-on.md.mustache +++ b/agent-context/templates/always-on.md.mustache @@ -51,7 +51,8 @@ itself for those three: it is the source those pages are generated from. - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 144d1eae2..48f32e008 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -25,7 +25,7 @@ regenerate with `ls -d fixtures//*/ | wc -l`. | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 322 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 325 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 16 | 16 / 16 | 15 / 16 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 15 / 16 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 16 / 16 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 31 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | @@ -119,7 +119,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. ## Fixture-to-doc mapping -### `fixtures/conformance/` — metamodel loader + canonical serializer (322) +### `fixtures/conformance/` — metamodel loader + canonical serializer (325) | Fixture prefix | Feature doc | |---|---| @@ -131,7 +131,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. | `auto-set-on-*` | [features/entities.md](features/entities.md) (auto-set timestamps) | | `attr-filter-*`, `loader-filterable-*`, `warning-filterable-*`, `layout-data-grid-*`, `error-data-grid-*` | [features/entities.md](features/entities.md) (filter / sort / grid) | | `overlay-*` | [features/entities.md](features/entities.md) (overlay / merge) | -| `merge-three-way-no-conflict`, `error-merge-conflict-attr`, `warning-duplicate-declaration` | [features/loaders.md](features/loaders.md) (multi-file merge attribution, FR5c) | +| `merge-three-way-no-conflict`, `error-merge-conflict-attr`, `merge-conflict-unmarked-attr-redeclaration`, `warning-duplicate-declaration` | [features/loaders.md](features/loaders.md) (multi-file merge attribution, FR5c) | | `field-string-*`, `field-decimal-*`, `field-object-storage-*`, `error-field-object-storage-*` | [features/field-types.md](features/field-types.md) | | `currency-*` | [features/field-types.md](features/field-types.md) (currency) | | `enum-*`, `error-enum-*`, `warning-enum-*` | [features/field-types.md](features/field-types.md) (enum) | @@ -271,7 +271,7 @@ Phase 1a is TypeScript + Python only; those three ports arrive in Phase 2. ## Orphaned fixtures (tested but not yet documented) -The fixtures in the nine corpora mapped above (metamodel 322 + yaml 16 + verify 31 +The fixtures in the nine corpora mapped above (metamodel 325 + yaml 16 + verify 31 + render 15 + persistence 33 + api-contract 41 + source-resolution 25 + scope 10 + dependency 23) each map to a feature doc. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/README.md b/docs/README.md index e645de4e2..4082ccde7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -39,6 +39,7 @@ docs/ │ ├── image-upload.md # view.image form control (TS-web) │ ├── metadata-sources.md # where metadata comes from: sources, scope, discovery │ ├── metadata-dependencies.md # building on a metadata model published elsewhere (FR-023) +│ ├── libraries.md # reusable declared design you opt into, a layer at a time (FR-043) │ └── own-your-codegen.md # scaffold-and-own generator ownership (ADR-0034) └── ports/ # one file per language/framework port ├── typescript.md @@ -62,6 +63,7 @@ this tree is documentation, not the source of truth. | Author metadata in YAML instead of JSON | [`features/yaml-authoring.md`](features/yaml-authoring.md) | | Point the toolchain at metadata that lives somewhere other than `metaobjects/`, or scope what a project generates and migrates | [`features/metadata-sources.md`](features/metadata-sources.md) | | Build on a metadata model another repository publishes (`dependencies`, `meta deps sync`, overlay/extend across the boundary) | [`features/metadata-dependencies.md`](features/metadata-dependencies.md) | +| Adopt a design MetaObjects already ships — users/groups/roles, an LLM trace envelope — instead of authoring it (`libraries`, `meta eject `) | [`features/libraries.md`](features/libraries.md) | | Record what the system is supposed to do, and stop agents reviving retired features | [`features/requirements.md`](features/requirements.md) | | Wire prompt construction (FR-004) | [`features/templates-and-payloads.md`](features/templates-and-payloads.md) | | Share a metadata shape across multiple instances (abstracts, `extends:`) | [`features/abstracts-and-inheritance.md`](features/abstracts-and-inheritance.md) | diff --git a/docs/compatibility-policy.md b/docs/compatibility-policy.md index d39ea2d41..906eab16e 100644 --- a/docs/compatibility-policy.md +++ b/docs/compatibility-policy.md @@ -50,8 +50,12 @@ work*. A release can move one without the other, and most releases move neither. - **The CLI command surface** — `init` / `gen` / `verify` and their *documented* flags, per port (`meta`, `dotnet meta`, `mvn metaobjects:*`, `metaobjects`). -- **The scaffold-and-own contract** — what `meta init` scaffolds and the `Generator` - interface owned templates implement. +- **The scaffold-and-own contract** — the *layout and the interfaces*: + `codegen/generators/`, the local-import config shape, `.metaobjects/`, and the + `Generator` interface owned templates implement. **Not** *which* generators a fresh + scaffold wires: codegen is opt-in and the scaffolded selection is empty by design + (ADR-0034 Amendment 2), so changing what `meta init` starts you with changes nothing + for a project that already exists. > **What this costs you, stated plainly.** Post-1.0 the caret rule stops being a gate — > `^1.0.0` accepts `1.1.0` — so a metamodel change can reach you on a routine update @@ -71,6 +75,29 @@ work*. A release can move one without the other, and most releases move neither. (e.g. the reserved-but-unregistered declared-API vocabulary `api.*`/`operation.*`/ `binding.*`, and reserved index subtypes `index.fulltext`/`vector`/`spatial`). +## Shipped libraries (`libraries: [...]`) + +A [library](features/libraries.md) is declared design MetaObjects ships as metadata — +nodes an adopter opts into and then generates from. What its SHAPE promises depends on +the `stability` its manifest declares, and `meta gen --list` prints it: + +- **`stable`** — additive only within a MINOR. A field, an index, a requirement may be + added; a node or field is not removed or renamed, and a physical name does not change, + without a MAJOR. +- **`preview`** — exempt from that promise. The shape may change in a MINOR, including + removals and renames. A library ships `preview` while its shape is still being learned + from use, and is promoted on evidence: one external estate running it with the drift + gate enforced (the same bar G3d set for the 1.0 cut). + +Two things bound what that costs you. **Copy is the expected mode** — `meta eject ` +hands you the metadata to own, and a later change to the library then reaches only the +adopters who chose to track it. And the layering means the core layer generates nothing +until you opt into `db`, so a shape change in a library you took for its design alone +cannot move your schema. + +The library's own REQUIREMENTS carry the same reading rule as its model: `live` means +"the model as shipped realises this", never "your application does". + ## MINOR vs. PATCH (what a version bump means) The trigger is **new public surface, not code size**: diff --git a/docs/features/cli.md b/docs/features/cli.md index c84685762..2aaae18f8 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -134,7 +134,7 @@ filesystem provider rooted at `--prompts`), and `verify --db` which is ADR-0015"). Bare `verify` stays `--codegen` for back-compat. The **C# `dotnet meta`** port likewise ships the codegen-side subverbs: `verify --templates` (its historical template/prompt drift gate, the C# back-compat default), `verify ---codegen` (regenerate the default generator suite to a temp dir and diff against +--codegen` (regenerate the configured generators to a temp dir and diff against the committed `--out` tree, never touching it), and a **clean `--db` rejection (exit 2)** — bare `dotnet meta verify` keeps `--templates` and prints the subverb note. The **Java/Kotlin `mvn metaobjects:verify`** port expresses the same vocabulary as a @@ -275,7 +275,8 @@ libraries: [ai] # optional; MetaObjects-shipped library package targets: api: outDir: src/generated/api - generators: [entity, routes] # optional; stable names from `metaobjects gen --list`; omit = default suite + generators: [entity, routes] # REQUIRED; stable names from `metaobjects gen --list`. + # There is no default suite — see ADR-0034 Amendment 2. admin: outDir: src/generated/admin entities: [Author, Book] # optional allowlist; omit = every entity @@ -311,15 +312,13 @@ targets: ## `libraries` — opting into a MetaObjects-shipped library package -MetaObjects ships a small set of standard metadata packages under `library/`. A project -opts into one by name, and its nodes become available to `extends`: +MetaObjects ships a small set of declared designs under `library/` — see +[libraries.md](libraries.md) for what they are and how to adapt one. A project opts into +one by name, and its nodes become available to `extends`: -```ts -// metaobjects.config.ts (Node `meta`) -export default defineConfig({ - libraries: ["ai"], // makes metaobjects::ai::LlmCallBase resolvable - generators: [entityFile()], -}); +```jsonc +// .metaobjects/config.json — the port-neutral file, beside `dependencies` +{ "schema_version": 1, "sources": [], "libraries": ["ai"] } ``` ```yaml @@ -332,6 +331,17 @@ libraries: [ai] { "object.entity": { "name": "AgentCall", "extends": "metaobjects::ai::LlmCallBase", ... } } ``` +- **A LAYER at a time.** A token is `` or `/`; the bare name is + the CORE layer, which declares no `source.rdb` and therefore adds **no tables and no + generated code**. `"ai/db"` adds the persistence layer and IMPLIES `"ai"` (a db layer + is nothing but `overlay: true` redeclarations, so without its base it would be + `ERR_OVERLAY_NO_TARGET`). A token whose LAYER is unknown is dropped WHOLE rather than + reduced to its core — implying the core from a mistyped `ai/database` would hand you + an inert core and no tables, with no diagnostic. +- **It lives in `.metaobjects/config.json`**, not in `metaobjects.config.ts`. Which + designs a project adopts is a fact about the PROJECT, not about how one port generates + code from it; that file is the port-neutral one every port already reads. A config + still carrying the old key gets a pointed error rather than silence. - **Opt-in, never automatic.** A library package registers real top-level nodes. A project that never references one should not find them in its model, its generated output or its docs — so nothing is loaded until the key names it. @@ -370,6 +380,25 @@ the `dotnet meta` CLI has no project-config file to carry a key. All five ports `metaobjects::ai::LlmCallBase`; three of them (Node `meta`, Python `metaobjects`, Maven) expose it declaratively. +### `meta eject ` — taking the metadata to own + +`meta eject` takes generator names AND library names. A library ejects every layer into +your first DECLARED source root (resolved through `sources`, never a hard-coded +directory) as `meta...yaml`, each stamped with a provenance header, and +never overwrites without `--force`. It does not edit your config. + +**One step remains, and the loader enforces it**: remove the library from `libraries`. +Left there, the shipped tree and your copy both load and merge asymmetrically — +additions take effect, deletions do not — so the load is refused with +`ERR_LIBRARY_PACKAGE_COLLISION`. Its mirror, `ERR_LIBRARY_PACKAGE_NOT_OWNED`, fires on a +NEW node declared into a library's package while the library is opted in; an +`overlay: true` amendment of one of its own nodes is the documented door and is +untouched. + +`meta eject --list` reports, per ejected library, `identical` or `differs` against the +shipped tree with counts changed / only-upstream / only-yours — the same staleness +report it gives for owned generators, one level up. + ## `meta gen` / `meta verify` run an advisory anti-pattern pass (Node `meta`) Both `meta verify` and a real `meta gen` write run (not `--dry-run`) end with a diff --git a/docs/features/codegen-concepts.md b/docs/features/codegen-concepts.md index f51bb2254..07aab0e02 100644 --- a/docs/features/codegen-concepts.md +++ b/docs/features/codegen-concepts.md @@ -256,7 +256,8 @@ JSON-schema'd beside the TS port at `codegen-ts/src/template-codegen/template-spec.schema.json`). Pass it on `gen`: `metaobjects gen --out --template-spec spec.json --templates ` (Python), or `dotnet meta gen --out --template-spec spec.json --template-root ` (C#). -The named generators are appended to the default suite, resolving template refs +The named generators are appended to the SELECTION (there is no default suite — +ADR-0034 Amendment 2), resolving template refs under the templates root (`--templates` in Python, `--template-root` in C#; default `templates`). `scope` must be one of `perEntity`/`perPackage`/`perModel` and `format` (when present) one of the diff --git a/docs/features/entities.md b/docs/features/entities.md index 518e5cbe9..af59da255 100644 --- a/docs/features/entities.md +++ b/docs/features/entities.md @@ -347,9 +347,33 @@ The following conformance fixtures gate this feature's behavior across ports: **Overlay / merge** - [`fixtures/conformance/overlay-same-object-different-files/`](../../fixtures/conformance/overlay-same-object-different-files/) — same `package` + `name` merge across files -- [`fixtures/conformance/overlay-attr-last-writer-wins/`](../../fixtures/conformance/overlay-attr-last-writer-wins/) — attr conflict resolution +- [`fixtures/conformance/overlay-attr-last-writer-wins/`](../../fixtures/conformance/overlay-attr-last-writer-wins/) — a MARKED overlay overriding an attr: last-writer-wins, and **no error** +- [`fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/`](../../fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/) — the same collision UNMARKED: `ERR_MERGE_CONFLICT` - [`fixtures/conformance/overlay-merge-flag-explicit/`](../../fixtures/conformance/overlay-merge-flag-explicit/) — `overlay: true` is explicit-merge-intent +### `overlay: true` licenses an attribute override + +Two files can set the same attribute to different values two ways, and they mean +different things: + +- **Unmarked** — two declarations of the same `(type, package::name)` that collided + *without knowing about each other*. The value merges last-writer-wins so the loader + sees one canonical tree, and the load emits **`ERR_MERGE_CONFLICT`** so somebody can + fix it. +- **Marked `overlay: true`** — the author saying *"I know about the other declaration + and I mean to change it."* The value wins and the load is **silent**. + +The loader already treats the flag specially — `overlay: true` is find-or-throw, a plain +redeclaration is create-or-find — and this makes it mean one coherent thing rather than +two. It is how an adopter retunes an inherited attribute (a library's `@status`, a +dependency's `@maxLength`) without the toolchain calling a deliberate act a defect. + +**Mark the whole ancestor chain, not just the leaf.** Addressing a nested node means +re-declaring its ancestors, and every one of them must carry `overlay: true` too. Left +plain, each ancestor is a same-shape redeclaration and emits +`WARN_DUPLICATE_DECLARATION` — three warnings to change one leaf in a depth-4 tree. +Marked, the load is silent. + Cross-port runner coverage: TS / Java / Kotlin / C# / Python all execute these via their respective conformance runners. See [`docs/CONFORMANCE.md`](../CONFORMANCE.md) for the per-port pass/skip ledger. diff --git a/docs/features/libraries.md b/docs/features/libraries.md new file mode 100644 index 000000000..59eca636f --- /dev/null +++ b/docs/features/libraries.md @@ -0,0 +1,170 @@ +# Libraries: declared design you opt into + +Most applications contain a handful of models that are not their idea. Users, groups, +roles, permissions. A trace row for every call to a language model. You did not invent +them, you will not differentiate on them, and you will spend a day re-deriving them +anyway — badly enough that six months later someone asks why a grant is a string +compared to a literal in a branch. + +A **library** is that design, declared as metadata and shipped with MetaObjects: + +```jsonc +// .metaobjects/config.json +{ "schema_version": 1, "sources": [], "libraries": ["iam"] } +``` + +An agent working in your repo can now see that the capability exists, resolve against +it, and build on it. Nothing else happens — which is the part worth reading twice. + +## A library is LAYERED, and the core layer is INERT + +A library ships its **core model**, its **DB persistence** and (where it has one) its +**UI rendering** as separate layers. You take as much of it as you want: + +```jsonc +"libraries": ["iam"] // core model only — inert +"libraries": ["iam", "iam/db"] // + persistence: the tables +"libraries": ["iam", "iam/db", "iam/ui"] // + UI +``` + +**The core layer declares no `source.rdb`, and a sourceless object is inert by a +contract that already shipped.** `meta migrate` skips an object with no writable source +and codegen emits no route, queries, hooks, grid or form for one (both citing #248: +persistability derives from source presence, never from the object subtype). So +`libraries: ["iam"]` adds **zero tables and zero generated code**. The design is +present and resolvable; the schema is a second, separate decision. + +A sourceless object still gets a type-only interface, so `extends` and references work +against it from day one. + +`"iam/db"` **implies** `"iam"`, and the implication is not a convenience: a db layer is +nothing but `overlay: true` redeclarations, and an overlay whose target was never +declared is `ERR_OVERLAY_NO_TARGET`. A token whose LAYER is unknown is dropped WHOLE +rather than reduced to its core — implying the core from a mistyped `iam/database` would +hand you an inert core and no tables, with no diagnostic. + +An unknown library name is refused by the config reader with the tokens this build +actually ships (`ERR_UNKNOWN_LIBRARY`). Skipping it silently would resurface later as +`ERR_UNRESOLVED_SUPER` pointing at your own metadata — the wrong place to go looking. + +## Finding one: they are rows in the generator catalog + +There is one door, not two: + +``` +meta gen --list --format json --probe +``` + +Library rows carry `kind: "library"`, a `useWhen` sentence, the `layers` you can select, +the `packages` the library owns, and a computed `provides` — how many entities, how many +abstracts, how many requirements are in the box. With a project present there is also a +`project` block: which layers you selected, how many tables and requirements they added +*here*, which of your entities `extends` into the library, and any generator the library +implies that you have not wired. + +The rule for an agent is the one the `metaobjects-codegen` skill states: **if a +`useWhen` matches the capability you are about to model, opt in and adapt rather than +author.** + +## Copy is the expected mode + +**A library is first a reference — something to copy and make your own.** That is the +same ruling ADR-0034 made about generators: the reference templates are copied into your +repo because you own your code. Metadata is no different. + +| you want to | door | +|---|---| +| **the design, as a starting point you own** — rename the package, delete what you do not need, change a PK strategy, keep the requirements and edit them | **`meta eject `** — the expected path | +| a new shape sharing a library base, tracking upstream | `extends` | +| add to a shipped node while tracking upstream (fields, indexes, views) | `overlay: true` on the same `(type, metaobjects::::Name)` | +| change a shipped requirement's verdict while tracking upstream | `overlay: true` on the requirement node | + +``` +meta eject iam +``` + +copies every layer into your project's **first declared source root** (resolved through +your `sources` — never a hard-coded directory name), as `meta.iam.model.yaml`, +`meta.iam.db.yaml`, `meta.iam.requirements.yaml`, each stamped with a provenance header. +It does not edit your config, and it never overwrites a file without `--force`. + +**One step remains, and the loader enforces it: remove the library from `libraries`.** +Left there, the shipped tree and your copy both load and merge — and the merge is +asymmetric. Additions in your copy take effect; **deletions do not**, because the library +still declares what you removed. That is `ERR_LIBRARY_PACKAGE_COLLISION`, refused at +load rather than left to be discovered. + +The mirror error is `ERR_LIBRARY_PACKAGE_NOT_OWNED`: a **new** node declared into a +package a library owns while that library is opted in. A later release of the library +may ship a node of that name and merge into yours. Declare it in a package you own and +`extends` the library's node, or say `overlay: true` and mean it. + +### Staleness, after you own it + +``` +meta eject --list +``` + +reports, per ejected library, whether your copy is `identical` to the shipped tree or +`differs`, with three counts: nodes **changed**, nodes **only upstream** (the library +gained one, or you deleted it) and nodes **only yours**. The comparison runs through the +canonical serializer in own mode, so re-indentation and key order never show up — only a +declaration that actually changed. Nodes are matched by NAME with the package +neutralized, because renaming the package is something you are invited to do and is not +drift. + +`meta verify` also advises on any node under `metaobjects::` in your own metadata that +carries no ejection provenance — a hand-copy whose origin nothing can reconstruct, or a +package name that will collide the day a library ships into it. + +## Requirements come with the design + +A library ships `requirement.*` nodes describing what its model promises, and on opt-in +they enter your ledger with no new machinery. One reading rule, and it is load-bearing: +**`live` in a library means "the model as shipped realises this"**, never "your +application does". Behaviour the model cannot carry ships as `partial` + +`disposition: accepted` with a note naming what you must do. + +**Object coverage activates on requirements YOU authored.** A library cannot volunteer +you for the unclaimed-entity gate: opt into `iam` with no ledger of your own and +`meta verify` prints its entries, checks them, and says `coverage: not measured (no +project-authored requirements)`. Write your first requirement and coverage turns on — +over the library's entities too, which by then its own ledger claims, so they add no +warnings. + +To disagree with a shipped requirement, overlay it: same `(type, package::path)`, +`overlay: true`, your `status` / `disposition` / `notes`. Addressing a nested node means +re-declaring its ancestors, and **every one of them must also carry `overlay: true`** — +left plain, each emits `WARN_DUPLICATE_DECLARATION`, so a depth-4 tree costs three +warnings to change one leaf. + +## Generators a library implies + +A library may declare generators its design implies, each with an **anchor** — the +library node the generator keys on. `ai` declares `trace-helper`, anchored on +`metaobjects::ai::LlmCallBase`. + +Nothing is wired for you. Two self-extinguishing warnings do the rest: *library opted +in, implied generator not wired* and *generator wired, its library not opted in*. The +second matters more than it looks: a generator whose anchor is absent matches nothing +and emits zero files, which reads exactly like "my model has no trace entities yet". + +## What ships today + +| library | what it is | stability | +|---|---|---| +| `iam` | Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group. Nine entities; grants are two junctions rather than one with a nullable scope, because a NULL in a unique key is distinct from every other NULL in SQL. | `preview` | +| `ai` | The LLM-call trace envelope: what was asked, what came back, what it cost, how long it took. | `stable` | + +`preview` means the SHAPE may change within a MINOR; `stable` means additive-only. See +[compatibility-policy.md](../compatibility-policy.md). Copy-and-own is the real answer +to both: a later change to a library reaches only the adopters who chose to track it. + +## Scope, stated plainly + +A library's nodes are generated, migrated and ledgered **as if you had written them** — +the opposite of a [metadata dependency](metadata-dependencies.md), which is someone +else's model you must not drift from and is excluded from your codegen, schema and +ledger by default. You opt into a library to *have the thing*. The layering is what +keeps that from meaning "nine tables you did not ask for": the core layer produces +nothing until you add `db`. diff --git a/docs/features/migrations/0.x-to-1.0.md b/docs/features/migrations/0.x-to-1.0.md index 719e1b26b..5683b125e 100644 --- a/docs/features/migrations/0.x-to-1.0.md +++ b/docs/features/migrations/0.x-to-1.0.md @@ -247,7 +247,7 @@ moved fails to compile. **Grep for `Names.name` before you regenerate.** One key a table, a view and a stored procedure, told apart only by a sibling `kind`. **If you have never seen a `Names` file, that is expected and it is worth fixing -now.** On C# and Python the artifact comes from a real default suite, so upgrading is +now.** On C# and Python the artifact used to come from a default suite, so upgrading was enough. On TypeScript and the JVM the generator list in your config IS the complete list — `metaobjects.config.ts`'s `generators: [...]` and the pom's `` — and upgrading a package never edits a config. `meta init` scaffolds `namesFile()`, but only into a diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 3c008c570..fd7c594eb 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -10,7 +10,7 @@ port** — this is intentional (ADR-0035 §3, ratified), not a parity gap: 1. **You own the invocation** — codegen runs through your own build, on your terms, in every port. -2. **You own the templates** — in TypeScript, `meta init` scaffolds the reference +2. **You own the templates** — in TypeScript, `meta eject ...` copies the reference generators *into your repo* so you can edit them (ADR-0034 scaffold-and-own). The JVM/Python/C# ports own codegen through **build configuration** rather than copied template files; template customization there is via the declarative @@ -194,7 +194,7 @@ you get is the first thing to establish, because it changes what you can plan. | Port | Invocation | Programmatic — write a `Generator` | Declarative — template + scope | |---|---|---|---| -| **TypeScript** | `meta init` → `meta gen` (Bun/Node CLI) | **Yes — scaffold-and-own.** `meta init` copies `entityFile`/`queriesFile`/`routesFile`/`namesFile`/`barrel` into `codegen/generators/*.ts`; `metaobjects.config.ts` imports those local copies. Edit them freely, or `meta eject ` any other one — except `shared-model` (FR-023's publisher generator), which is registered and discoverable but deliberately not ejectable, since it emits a hash-pinned cross-port contract artifact. | **Yes** — `templateGenerator({ template, scope, outputPattern })` in the config's `generators: [...]`. No CLI flag: the config already takes generator values. | +| **TypeScript** | `meta init` → `meta gen --list --probe` → `meta eject ` → `meta gen` (Bun/Node CLI) | **Yes — scaffold-and-own.** `meta init` scaffolds the LAYOUT and an empty selection (ADR-0034 Amendment 2); `meta eject ...` copies each generator you choose into `codegen/generators/*.ts` and prints the import to add to `metaobjects.config.ts`. Edit them freely. Not every registered generator is ejectable — `shared-model` (FR-023's publisher generator) is discoverable but deliberately not, since it emits a hash-pinned cross-port contract artifact. | **Yes** — `templateGenerator({ template, scope, outputPattern })` in the config's `generators: [...]`. No CLI flag: the config already takes generator values. | | **Java / Kotlin** | `mvn metaobjects:generate` / `mvn metaobjects:verify` (`metaobjects-maven-plugin`) | **Yes.** Every generator — built-in or your own — is named in `` and loaded from the project classpath: one seam, not two. There is no default suite, so `` is the complete list. Kotlin runs through the same goal. | **Yes** — `TemplateScopeGenerator` wired as an ordinary ``. No CLI flag: `` is already the seam. | | **C#** | `dotnet meta gen` / `dotnet meta verify` (.NET tool) | **No.** `GeneratorRegistry` is a closed built-in registry; `--generators` *selects* from what ships. There is no registration seam. | **Yes, and it is your only path** — `dotnet meta gen --template-spec --template-root `. | | **Python** | `metaobjects gen` / `metaobjects verify` (console-script) | **No.** `GENERATOR_REGISTRY` is a closed built-in registry, same as C#. (`--provider module:symbol` registers **metamodel vocabulary**, not a generator — do not reach for it here.) | **Yes, and it is your only path** — `metaobjects gen --template-spec --templates `. | @@ -290,5 +290,5 @@ the same "idiomatic per port" principle as generator ownership (ADR-0035 §3). Importing the built-in generators from `@metaobjectsdev/codegen-ts/generators` (`entityFile`, `queriesFile`, `routesFile`, `barrel`) is **deprecated** (ADR-0034) and -**removed at the 1.0/8.0 release**. Use the owned copies `meta init` scaffolds into +**removed at the 1.0/8.0 release**. Use the owned copies `meta eject` writes into `codegen/generators/*` and import those from your `metaobjects.config.ts`. diff --git a/docs/features/source-kinds.md b/docs/features/source-kinds.md index dbd4ba2d7..326693ea2 100644 --- a/docs/features/source-kinds.md +++ b/docs/features/source-kinds.md @@ -330,7 +330,7 @@ object AuthorViewTable : Table("v_author") { `MetaObjects.Codegen` emits `OwnsOne` / `DbSet` wiring as appropriate; for `@kind: "view"` the generated `AppDbContext` calls `entity.ToView(AuthorViewNames.SourcePrimaryView)` — -`names` is in C#'s default generator suite, so the view name is referenced from the +`names` is the opt-in C# constants generator, so when selected the view name is referenced from the generated constants artifact, not respelled. The `CREATE VIEW` body is emitted by the **Node** `meta migrate` (schema is Node-owned — ADR-0015; the C# migrate surface was removed), not by `dotnet meta`. diff --git a/docs/llms/llms-full.txt b/docs/llms/llms-full.txt index 4693b8620..ee439ab69 100644 --- a/docs/llms/llms-full.txt +++ b/docs/llms/llms-full.txt @@ -250,12 +250,16 @@ metadata.root: The project config (`metaobjects.config.ts`) wires up the generators and target settings: +Codegen is **opt-in**: `meta init` writes `generators: []`, and no port ships a default +suite. `meta gen --list --probe` is the catalog — it runs every generator against your +model and reports how many files each would emit — and `meta eject ...` copies the +ones you choose into your repo, printing the import line and the entry to add. + ```ts import { defineConfig } from "@metaobjectsdev/cli"; -// Owned generators scaffolded by `meta init` (ADR-0034 scaffold-and-own) — these files are -// copied into YOUR repo and are yours to edit. The default emit targets Fastify on Node; -// retarget it by editing these, not by switching tools. `meta eject ` takes ownership -// of any other generator. +// Owned generators — copied in by `meta eject` (ADR-0034 scaffold-and-own). These files +// are in YOUR repo and are yours to edit. The default emit targets Fastify on Node; +// retarget it by editing these, not by switching tools. import { entityFile } from "./codegen/generators/entity.js"; import { queriesFile } from "./codegen/generators/queries.js"; import { routesFile } from "./codegen/generators/routes.js"; @@ -274,10 +278,16 @@ Run the CLI: ```bash # Scaffold metaobjects/, .metaobjects/, metaobjects.config.ts, AND the agent context -# (.metaobjects/AGENTS.md + CLAUDE.md + .claude/skills/metaobjects-*) +# (.metaobjects/AGENTS.md + CLAUDE.md + .claude/skills/metaobjects-*). Wires no generators. $ meta init -# Generate code (entities + queries + routes + barrel per config) +# The catalog: every generator by layer, with a file count for YOUR model +$ meta gen --list --probe + +# Take the ones you want — copies them into codegen/generators/ and says what to wire +$ meta eject entity queries routes barrel + +# Generate code from exactly what you wired $ meta gen # Optional positional filter diff --git a/docs/llms/llms.txt b/docs/llms/llms.txt index 1d8bb9343..f9920d6e4 100644 --- a/docs/llms/llms.txt +++ b/docs/llms/llms.txt @@ -62,6 +62,7 @@ MetaObjects deliberately does **not** ship one universal binary. Schema operatio - Node CLI binary name: `meta` - Project config file: `metaobjects.config.ts` (configures `outDir`, `targets`, `dialect`, `dbImport`, `apiPrefix`, `columnNamingStrategy`, and the `generators` array) +- Codegen is **opt-in**: no port ships a default generator suite, `meta init` scaffolds `generators: []`, and `meta gen --list` is the catalog (`--probe` reports how many files each generator would emit for your model) - Project marker directory: `.metaobjects/` - Install (TypeScript reference): `npm install -D @metaobjectsdev/cli` (or `bun add @metaobjectsdev/cli`) - Install (Java): `com.metaobjectsmetaobjects-metadata8.0.3` plus per-concern artifacts (`metaobjects-codegen-spring`, `metaobjects-omdb`, `metaobjects-maven-plugin`, etc.) diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index b50bd9cf2..f9ae25b0e 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -108,8 +108,11 @@ The codegen emits: ### `Names` — the physical names, as constants -`names` ships in the **default generator suite** — a new project gets -`Names.g.cs` without configuring anything. It carries the physical +`names` is opt-in — select it with `--generators names` on `dotnet meta gen` +(`dotnet meta gen --list` names the whole catalog). This port ships no `eject` +verb — ejecting belongs to the Node `meta` CLI, and it is about OWNING a +generator's source, never about selecting one for a run. When selected, a project gets +`Names.g.cs`. It carries the physical database names for one object as `const string`s: ```csharp @@ -394,7 +397,8 @@ dotnet meta verify --codegen ./metadata --out ./Generated --template-root ./temp Each spec entry derives the neutral template data dict for its scope (`MetaObjects.Codegen.TemplateCodegen.TemplateData`) and names each file via the `outputPattern` placeholders (`{name}`, `{Name}`, `{package}`). The named generators -are **appended** to the default suite and gated byte-identical against the shared +are **appended** to the `--generators` selection (there is no default suite) and gated +byte-identical against the shared `fixtures/template-codegen-conformance/` corpus. A `target` field is rejected (C# has no output-target concept); a bad template ref or wrong `--template-root` surfaces as a clean error, not a stack trace. For output to be regenerable, the **template** must emit diff --git a/docs/ports/python.md b/docs/ports/python.md index 90ae354ba..fb8979e68 100644 --- a/docs/ports/python.md +++ b/docs/ports/python.md @@ -103,8 +103,8 @@ byte-identical. ### Taking one tier and not the rest -`--generators ` runs exactly the named generators instead of the default -suite (`--list` names all of them). This is the answer for a project that wants +`--generators ` selects exactly the named generators from the available +catalog (`--list` names all of them). This is the answer for a project that wants one tier without adopting the others — most often a **schema-only adopter** whose tables come from `meta migrate` and whose application code is not generated at all, but which still has physical table and column names hard-coded across its data @@ -117,8 +117,8 @@ metaobjects gen ./metadata --out ./generated --generators names It emits one `_names.py` per object and nothing else. Each carries `_SOURCE_PRIMARY_TABLE`, a `__COLUMN` per field, a `_COLUMNS_BY_FIELD` map, and `_SOURCE_PRIMARY_SCHEMA` when the -source declares a `@schema`. On the 16-entity persistence-conformance model the -default suite emits 68 files and this emits 19 — so adopting the names tier does +source declares a `@schema`. On the 16-entity persistence-conformance model the full +server-side selection emits 68 files and this emits 19 — so adopting the names tier does not drag a REST surface into a repo that does not want one. **Pass the same `--column-naming` the schema was created with.** It defaults to @@ -199,8 +199,11 @@ app.dependency_overrides[get_repository] = lambda: SqlAlchemyAuthorRepository(se ### `_names.py` — the physical names, as constants -The `names` generator ships in the **default generator suite** — a new -project gets `_names.py` without configuring anything. The module +The `names` generator is opt-in — select it with `--generators names` on +`metaobjects gen` (`--list` names the whole catalog). The Python console-script +has no `eject` verb: ejecting is the Node `meta` CLI's, and it is about OWNING a +generator's source, never about selecting one for a run. When selected, a project +gets `_names.py`. The module mirrors the metadata that declared it: every node it describes — the object, each `source.rdb` child, each `identity.*` and `index.*` child — carries its own `_TYPE`, `_SUB_TYPE` and `_NAME`, and a source's physical name sits under @@ -384,7 +387,8 @@ spec is ignored there rather than refused. Each spec entry derives the neutral template data dict for its scope and names each file via the `outputPattern` placeholders (`{name}`, `{Name}`, `{package}`). -The named generators are **appended** to the default suite and gated byte-identical +The named generators are **appended** to the `--generators` selection (there is no +default suite) and gated byte-identical against the shared `fixtures/template-codegen-conformance/` corpus. Output is format-agnostic (text/markdown/csv/json/xml/html), so the template-spec pass emits no `__init__.py` into its tree. A `target` field is rejected (the Python port has diff --git a/docs/superpowers/plans/2026-09-13-opt-in-codegen-and-generator-catalog.md b/docs/superpowers/plans/2026-09-13-opt-in-codegen-and-generator-catalog.md new file mode 100644 index 000000000..cb6f5548c --- /dev/null +++ b/docs/superpowers/plans/2026-09-13-opt-in-codegen-and-generator-catalog.md @@ -0,0 +1,1027 @@ +# Opt-in codegen and the generator catalog — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development +> (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use +> checkbox (`- [ ]`) syntax for tracking. + +**Goal:** No code generator runs until the adopter chooses it, and the choice is made from a +truthful, machine-readable catalog rather than from a default suite the CLI picked. + +**Architecture:** The catalog is the generator registry we already have, completed — `layer` +joins the cross-port manifest, the TypeScript entry type gains the facts a selection needs +(`framework`, `requires`, `runtimePeers`, `configKeys`, `kind`), the react and tanstack packages +export their own registry slices and the CLI unions them, and `meta gen --list --format json` +plus `--probe` is the agent-facing door. `meta init` shrinks to layout + an empty documented +selection; `meta eject` becomes the copy door and takes multiple names. Every declared +compatibility fact is **resolved, not trusted** — gates run the generators and read the +imports back. + +**Tech Stack:** TypeScript (Bun workspace), C# (.NET tool), Python (console script), Java/Kotlin +(Maven; already correct — no default suite). + +**Spec:** `docs/superpowers/specs/2026-09-12-opt-in-codegen-and-generator-catalog-design.md` + +## Global Constraints + +- **This ships as a PATCH.** `docs/compatibility-policy.md:53` is narrowed in the same change: + the scaffold-and-own promise is the **layout and the interfaces**, not which generators a + fresh scaffold wires. +- **`metamodelVersion` does not move.** `registry.json` is the *generator* manifest, a different + contract from `expected-registry.json`. No metamodel vocabulary is added; ADR-0023 is not engaged. +- **`layer` has exactly SIX values** — `model`, `persistence`, `api`, `client`, `docs`, + `capability`. Do not reintroduce `trace` / `requirements` / `publish` / `primitive`; they are + `capability`, discriminated by `--probe` (spec §8b). +- **34 generator names total** — the 29 already in `fixtures/generator-registry-conformance/registry.json` + plus `form`, `hooks`, `grid`, `grid-hook`, `requirement-tests`, all `ports: ["typescript"]`. +- **Named constants for metamodel strings.** Generator stable names are not metamodel strings, but + `layer` values are a closed set — declare them `as const` with a derived union type. +- **No `any`.** Use `unknown` and narrow. +- **Never a bare `bun test` at the repo root.** `cd server/typescript && bun test `. +- **`scripts/ci-local.sh` prints "LOCAL CI FAILED" and still returns 0.** Grep the `SUMMARY` + block; never branch on `$?`. +- **Agent-context prose changes need the corpus regenerated in the SAME commit** or `ts-unit` + goes red. Read back the **python** fixture — it is the only port-only stack. +- **The repo is PUBLIC.** No private project names, no absolute home paths, in code, docs, + fixtures, commit messages or branch names. + +--- + +## File structure + +| file | responsibility | +|---|---| +| `fixtures/generator-registry-conformance/registry.json` | cross-port truth: 34 names, `tier`, **`layer`**, `ports` | +| `server/typescript/packages/codegen-ts/src/generator-registry.ts` | the codegen-ts **slice** + the shared `GeneratorRegistryEntry` type and `Layer` union | +| `server/typescript/packages/codegen-ts-react/src/generator-registry.ts` | the react slice (`form`) | +| `server/typescript/packages/codegen-ts-tanstack/src/generator-registry.ts` | the tanstack slice (`hooks`, `grid`, `grid-hook`) | +| `server/typescript/packages/cli/src/lib/catalog.ts` | **new** — composes the three slices; the one place the TS port's full catalog exists | +| `server/typescript/packages/cli/src/commands/gen.ts` | `--list --format`, `--probe`, `project.*`, first-run pointer | +| `server/typescript/packages/cli/src/commands/eject.ts` | multi-name, `--format json`, consolidated install set | +| `server/typescript/packages/cli/src/commands/init.ts` | net shrink: no scaffolded generators, no db stub, no scaffold deps | +| `server/typescript/packages/codegen-ts/src/runner.ts` | the `requires` gate + the `api`-layer framework advisory | +| `server/csharp/MetaObjects.Cli/GenCommand.cs` | default suite removed; `--generators` required | +| `server/python/src/metaobjects/cli.py` | `_default_generators` removed; `--generators` required | + +--- + +## Task 1: `layer` joins the cross-port manifest + +**Files:** +- Modify: `fixtures/generator-registry-conformance/registry.json` +- Modify: `fixtures/generator-registry-conformance/README.md` +- Test: `server/typescript/packages/codegen-ts/test/golden/generator-registry-conformance.test.ts` + +**Interfaces:** +- Produces: every manifest entry carries `"layer": ""`. The five new entries + (`form`, `hooks`, `grid`, `grid-hook`, `requirement-tests`) exist with `ports: ["typescript"]`. + +The assignment, verbatim from spec §8b — do not improvise: + +| layer | members | +|---|---| +| `model` | entity, names, barrel, dto, value-object | +| `persistence` | queries, db-context, repository, exposed-table, relations, stored-proc | +| `api` | routes, routes-hono, filter-allowlist, validator, spring-config | +| `client` | form, hooks, grid, grid-hook | +| `docs` | docs, mermaid-er, api-docs | +| `capability` | prompt-render, output-parser, output-prompt, extractor, render-helper, payload, trace-helper, requirement-tests, shared-model, template, callable | + +- [ ] **Step 1: Write the failing test** — extend the TS conformance test with a layer block. + +```ts +const LAYERS = ["model", "persistence", "api", "client", "docs", "capability"] as const; + +it("every manifest entry declares one of the six layers", () => { + const bad = Object.entries(manifest.generators) + .filter(([, e]) => !LAYERS.includes(e.layer as (typeof LAYERS)[number])) + .map(([n, e]) => `${n}=${String(e.layer)}`); + expect(bad, `entries with a missing/unknown layer: ${bad.join(", ")}`).toEqual([]); +}); +``` + +Also widen `interface ManifestEntry` with `layer: string;`. + +- [ ] **Step 2: Run it and watch it fail** + +`cd server/typescript && bun test packages/codegen-ts/test/golden/generator-registry-conformance.test.ts` +Expected: FAIL — every entry reports `=undefined`. + +- [ ] **Step 3: Add `layer` to all 29 manifest entries and add the five new ones** + +Each new entry follows the existing shape, e.g.: + +```json +"form": { + "concept": "Per-entity React form component over the generated Zod schema.", + "tier": "native", + "layer": "client", + "ports": ["typescript"] +}, +"requirement-tests": { + "concept": "Per-requirement test stub, one per requirement.functional claim.", + "tier": "native", + "layer": "capability", + "ports": ["typescript"] +} +``` + +Update the `$comment` to name `layer` as a gated facet, and update +`fixtures/generator-registry-conformance/README.md` to document the six values and the +"a layer with one member does no grouping work" rationale. + +- [ ] **Step 4: Run it and watch it pass** (the set-equality assertion will still fail — the TS + registry does not yet expose the five new names. That is Task 2/3's job; leave it red and + note it, or land Tasks 1-3 as one commit. **Land them as one commit** — a red conformance + gate must never sit on `main`.) + +- [ ] **Step 5: Do not commit yet.** Continue into Task 2. + +--- + +## Task 2: the registry entry type gains the facts a selection needs + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/generator-registry.ts` +- Modify: `server/typescript/packages/codegen-ts/src/index.ts` (export the new types) +- Test: `server/typescript/packages/codegen-ts/test/generator-registry.test.ts` + +**Interfaces:** +- Produces: + +```ts +export const GENERATOR_LAYERS = ["model", "persistence", "api", "client", "docs", "capability"] as const; +export type Layer = (typeof GENERATOR_LAYERS)[number]; + +export type GeneratorFramework = "fastify" | "hono" | "react" | "tanstack"; + +export interface GeneratorRegistryEntry { + name: string; + /** Forward compatibility (spec §9 / FR-043): today every entry is "generator". */ + kind: "generator"; + layer: Layer; + tier: GeneratorTier; + description: string; + factory: () => Generator; + options?: string; + /** Absent = framework-neutral. */ + framework?: GeneratorFramework; + /** Stable names whose output this generator's output imports. */ + requires?: readonly string[]; + /** The @metaobjectsdev runtime the EMITTED code imports. */ + runtimePackage?: string; + /** Third-party packages the EMITTED code imports. Ranges come from the runtime + * package's own peerDependencies, never from here. */ + runtimePeers?: readonly string[]; + /** Config keys this generator reads: dbImport, apiPrefix, extStyle, … */ + configKeys?: readonly string[]; + /** True iff a reference template exists for `meta eject`. */ + ejectable: boolean; + note?: string; +} +``` + +- [ ] **Step 1: Write the failing test** + +```ts +import { generatorRegistry, GENERATOR_LAYERS } from "../src/generator-registry.js"; + +it("every entry declares kind, layer and ejectable", () => { + for (const [name, e] of Object.entries(generatorRegistry)) { + expect(e.kind, name).toBe("generator"); + expect(GENERATOR_LAYERS, name).toContain(e.layer); + expect(typeof e.ejectable, name).toBe("boolean"); + } +}); + +it("requires only names entries that exist in some slice", () => { + // codegen-ts's slice is self-contained: no entry here requires a react/tanstack name. + for (const [name, e] of Object.entries(generatorRegistry)) { + for (const dep of e.requires ?? []) { + expect(Object.keys(generatorRegistry), `${name} requires ${dep}`).toContain(dep); + } + } +}); +``` + +- [ ] **Step 2: Run it and watch it fail** + +`cd server/typescript && bun test packages/codegen-ts/test/generator-registry.test.ts` +Expected: FAIL — `kind` undefined. + +- [ ] **Step 3: Extend the type and populate every codegen-ts entry** + +Add `requirement-tests` to the registry (importing `requirementTests` from +`./generators/requirement-tests.js`; its factory is `() => requirementTests()`). + +`ejectable` is `true` exactly for the names in `REFERENCE_GENERATOR_NAMES` — assert that +rather than hand-maintaining it: + +```ts +import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; +// … in each entry: ejectable: REFERENCE_GENERATOR_NAMES.includes(name as ReferenceGeneratorName) +``` + +Leave `requires` / `runtimePeers` **empty for now** — Task 9's gates derive the truth and this +plan populates them there. Declaring a guess first and gating it second is the failure mode this +design exists to prevent. + +- [ ] **Step 4: Run it and watch it pass** + +- [ ] **Step 5: Do not commit yet.** Continue into Task 3. + +--- + +## Task 3: registry slices in react + tanstack, composed in the CLI + +**Files:** +- Create: `server/typescript/packages/codegen-ts-react/src/generator-registry.ts` +- Create: `server/typescript/packages/codegen-ts-tanstack/src/generator-registry.ts` +- Modify: both packages' `src/index.ts` +- Create: `server/typescript/packages/cli/src/lib/catalog.ts` +- Create: `server/typescript/packages/cli/test/catalog-conformance.test.ts` +- Modify: `server/typescript/packages/codegen-ts/test/golden/generator-registry-conformance.test.ts` + (narrow to one direction — see below) + +**Interfaces:** +- Consumes: `GeneratorRegistryEntry`, `Layer` from `@metaobjectsdev/codegen-ts` (Task 2). +- Produces: + - `export const reactGeneratorRegistry: Record` (`form`) + - `export const tanstackGeneratorRegistry: Record` + (`hooks`, `grid`, `grid-hook`) + - `export function composeCatalog(): Record` in + `cli/src/lib/catalog.ts`, plus `export function listCatalog(): GeneratorRegistryEntry[]` + sorted by layer (in `GENERATOR_LAYERS` order) then name. + +Why the conformance test moves: `codegen-ts` cannot import `codegen-ts-react` (dependency +direction), so its own registry is a **slice** and can only be checked one way — every name it +exposes is in the manifest with `typescript` in `ports`. Set equality is a property of the +**composed** catalog, so it is asserted in the CLI, which is the only package that can see all +three. + +- [ ] **Step 1: Write the failing CLI conformance test** + +```ts +// server/typescript/packages/cli/test/catalog-conformance.test.ts +import { describe, it, expect } from "bun:test"; +import { readFileSync, existsSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { composeCatalog } from "../src/lib/catalog.js"; + +function findRepoRoot(start: string): string { + let dir = start; + while (true) { + if (existsSync(join(dir, "fixtures")) && existsSync(join(dir, "server"))) return dir; + const parent = dirname(dir); + if (parent === dir) throw new Error("Could not locate repo root"); + dir = parent; + } +} + +const manifest = JSON.parse(readFileSync( + join(findRepoRoot(import.meta.dir), "fixtures/generator-registry-conformance/registry.json"), + "utf-8", +)) as { generators: Record }; + +describe("composed TS catalog == the manifest's typescript slice", () => { + const expected = new Set(Object.entries(manifest.generators) + .filter(([, e]) => e.ports.includes("typescript")).map(([n]) => n)); + const actual = new Set(Object.keys(composeCatalog())); + + it("no rogue, no missing", () => { + expect({ + extra: [...actual].filter((n) => !expected.has(n)).sort(), + missing: [...expected].filter((n) => !actual.has(n)).sort(), + }).toEqual({ extra: [], missing: [] }); + }); + + it("layer and tier agree with the manifest", () => { + const catalog = composeCatalog(); + for (const name of expected) { + expect(catalog[name]!.layer, name).toBe(manifest.generators[name]!.layer); + expect(catalog[name]!.tier, name).toBe(manifest.generators[name]!.tier); + } + }); + + it("no name is registered by two slices", () => { + // composeCatalog throws on a duplicate; this pins the behaviour. + expect(() => composeCatalog()).not.toThrow(); + }); +}); +``` + +- [ ] **Step 2: Run it and watch it fail** + +`cd server/typescript && bun test packages/cli/test/catalog-conformance.test.ts` +Expected: FAIL — `../src/lib/catalog.js` does not exist. + +- [ ] **Step 3: Write the two slices and the composer** + +The react slice: + +```ts +// server/typescript/packages/codegen-ts-react/src/generator-registry.ts +import type { GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; +import { formFile } from "./form-file.js"; +import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; + +export const reactGeneratorRegistry: Record = { + form: { + name: "form", + kind: "generator", + layer: "client", + tier: "native", + framework: "react", + description: "Per-entity React form component over the generated Zod schema.", + factory: () => formFile(), + options: "filter?, target?", + ejectable: REFERENCE_GENERATOR_NAMES.includes("form"), + }, +}; +``` + +The tanstack slice mirrors it with `hooks` → `tanstackQuery()`, `grid` → `tanstackGrid()`, +`grid-hook` → `tanstackGridHook()`, all `framework: "tanstack"`, `layer: "client"`. + +The composer: + +```ts +// server/typescript/packages/cli/src/lib/catalog.ts +import { generatorRegistry, GENERATOR_LAYERS, type GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; +import { reactGeneratorRegistry } from "@metaobjectsdev/codegen-ts-react"; +import { tanstackGeneratorRegistry } from "@metaobjectsdev/codegen-ts-tanstack"; + +const SLICES: ReadonlyArray]> = [ + ["@metaobjectsdev/codegen-ts", generatorRegistry], + ["@metaobjectsdev/codegen-ts-react", reactGeneratorRegistry], + ["@metaobjectsdev/codegen-ts-tanstack", tanstackGeneratorRegistry], +]; + +/** Which package a catalog entry came from — the install boundary, and the only + * fact composition adds that a slice cannot know about itself. */ +export function packageOf(name: string): string | undefined { + return SLICES.find(([, slice]) => name in slice)?.[0]; +} + +export function composeCatalog(): Record { + const out: Record = {}; + for (const [pkg, slice] of SLICES) { + for (const [name, entry] of Object.entries(slice)) { + const prior = packageOf(name); + if (name in out) { + throw new Error( + `generator "${name}" is registered by both ${prior} and ${pkg} — a stable name ` + + "identifies ONE generator across the whole catalog (ADR-0021 D3).", + ); + } + out[name] = entry; + } + } + return out; +} + +export function listCatalog(): GeneratorRegistryEntry[] { + return Object.values(composeCatalog()).sort((a, b) => + GENERATOR_LAYERS.indexOf(a.layer) - GENERATOR_LAYERS.indexOf(b.layer) || + a.name.localeCompare(b.name)); +} +``` + +Note the composer must not call `packageOf` for the duplicate message before inserting — read +the prior package from the already-built `out` instead by tracking a parallel `Record`; fix the sketch above accordingly when implementing (the message must name both +packages). + +Add `@metaobjectsdev/codegen-ts-react` and `-tanstack` to the CLI's `dependencies` if not +already there (they are — `eject.ts` imports both). + +- [ ] **Step 4: Narrow the codegen-ts conformance test to one direction** + +Replace the set-equality assertion with: + +```ts +it("every name codegen-ts registers is a typescript name in the manifest", () => { + const rogue = [...actualNames].filter((n) => !expectedNames.has(n)).sort(); + expect(rogue, `codegen-ts registers names the manifest does not give to typescript: ${rogue.join(", ")}`).toEqual([]); +}); +``` + +…and add a comment saying the other direction is asserted on the COMPOSED catalog in +`packages/cli/test/catalog-conformance.test.ts`, because codegen-ts is a slice. + +- [ ] **Step 5: Run both** + +``` +cd server/typescript && bun test packages/codegen-ts/test/generator-registry.test.ts \ + packages/codegen-ts/test/golden/generator-registry-conformance.test.ts \ + packages/cli/test/catalog-conformance.test.ts +``` +Expected: PASS. + +- [ ] **Step 6: Commit Tasks 1-3 together** + +```bash +git add fixtures/generator-registry-conformance server/typescript/packages/codegen-ts \ + server/typescript/packages/codegen-ts-react server/typescript/packages/codegen-ts-tanstack \ + server/typescript/packages/cli +git commit -m "feat(catalog): layer joins the generator manifest, and the catalog is composed across packages" +``` + +--- + +## Task 4: `layer` reaches the other four ports + +**Files:** +- Modify: `server/csharp/MetaObjects.Codegen/GeneratorRegistry.cs` (or wherever the entry record lives) + `server/csharp/MetaObjects.Codegen.Tests/GeneratorRegistryConformanceTests.cs` +- Modify: `server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java` + `.../GeneratorRegistryConformanceTest.java` +- Modify: `server/java/codegen-kotlin/.../GeneratorRegistryConformanceTest.kt` (+ the Kotlin registry) +- Modify: `server/python/src/metaobjects/codegen/generator_registry.py` + `server/python/tests/conformance/test_generator_registry_conformance.py` + +**Interfaces:** +- Consumes: the manifest's `layer` field (Task 1). +- Produces: each port's registry entry carries a `layer`, and each port's conformance test + asserts agreement with the manifest, exactly as it already asserts `tier`. + +- [ ] **Step 1: Read each port's registry + conformance test before editing.** They are not + identically shaped; the assertion to copy is the existing `tier` agreement block. + +- [ ] **Step 2: Add the failing layer-agreement assertion in each port**, mirroring that block. + +- [ ] **Step 3: Run each port's test and watch it fail.** + +``` +cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter GeneratorRegistryConformance +cd server/java && mvn -q -pl codegen-spring test -Dtest=GeneratorRegistryConformanceTest +cd server/java && mvn -q -pl codegen-kotlin test -Dtest=GeneratorRegistryConformanceTest +cd server/python && uv run --extra integration pytest tests/conformance/test_generator_registry_conformance.py +``` + +**Trap:** `dotnet test` prints `Passed!` for a project that did not compile. Count the result +lines, do not read the last one. + +- [ ] **Step 4: Add `layer` to each port's registry entries** using the Task 1 table. + +- [ ] **Step 5: Re-run all four; all green.** + +- [ ] **Step 6: Commit** + +```bash +git commit -m "feat(catalog): layer is gated in all five ports" +``` + +--- + +## Task 5: `meta gen --list --format json|toon` and `--probe` + +**Files:** +- Modify: `server/typescript/packages/cli/src/lib/args.ts` (`GEN_OPTIONS` gains `probe`) +- Modify: `server/typescript/packages/cli/src/commands/gen.ts` +- Create: `server/typescript/packages/cli/src/lib/catalog-listing.ts` +- Create: `server/typescript/packages/cli/test/gen-list-catalog.test.ts` + +**Interfaces:** +- Consumes: `listCatalog()`, `packageOf()` (Task 3). +- Produces: `buildCatalogListing(opts): CatalogRow[]` where + +```ts +export interface CatalogRow { + name: string; + kind: "generator"; + layer: Layer; + framework?: string; + tier: "native" | "neutral"; + package: string; + description: string; + useWhen?: string; + emits?: string; + requires?: readonly string[]; + configKeys?: readonly string[]; + install?: { dev: string[]; runtime: string[] }; + source: { kind: "reference-template" | "package-only"; ejectable: boolean; owned: boolean | null }; + project?: { wired: boolean; frameworkDetected: boolean | null; wouldEmit: number | null }; +} +``` + +`--list` with no project loads nothing (like `meta types`). `--probe` requires a project: it +resolves the collection + config, loads the model, constructs **every** catalog generator and +dry-runs it in memory, and reports the file count per generator. + +- [ ] **Step 1: Write the failing tests** + +```ts +it("--list --format json emits one document, every row layered", async () => { + const out = await captureStdout(() => genCommand(["--list"], tmpdir, "json")); + const rows = JSON.parse(out) as CatalogRow[]; + expect(rows.length).toBe(23); // the manifest's typescript slice + for (const r of rows) { + expect(GENERATOR_LAYERS).toContain(r.layer); + expect(r.kind).toBe("generator"); + expect(r.package).toMatch(/^@metaobjectsdev\//); + } +}); + +it("--list needs no project at all", async () => { + // an empty tmpdir — no metaobjects/, no config + expect(await genCommand(["--list"], emptyTmp, "json")).toBe(0); +}); + +it("--probe reports wouldEmit from the real model", async () => { + const rows = JSON.parse(await captureStdout(() => genCommand(["--list", "--probe"], projectTmp, "json"))); + const entity = rows.find((r) => r.name === "entity")!; + expect(entity.project!.wouldEmit).toBeGreaterThan(0); + const parser = rows.find((r) => r.name === "output-parser")!; + expect(parser.project!.wouldEmit).toBe(0); // the fixture declares no template.prompt +}); + +it("--probe without a project is a usage error, not an empty listing", async () => { + expect(await genCommand(["--list", "--probe"], emptyTmp, "json")).toBe(2); +}); +``` + +- [ ] **Step 2: Run and watch fail.** + +`cd server/typescript && bun test packages/cli/test/gen-list-catalog.test.ts` + +- [ ] **Step 3: Implement.** + +`useWhen` and `emits` come from the reference-template headers, which already carry them +(`use-when`, `emits`) — parse them the way `eject.ts` parses the import line, in +`catalog-listing.ts`, and leave them `undefined` for a non-ejectable entry. `owned` is +`true`/`false` when a project is present (does `codegen/generators/.ts` exist?) and `null` +with no project. `frameworkDetected` reads `package.json` dependencies for the framework's own +package (`fastify`, `hono`, `react`, `@tanstack/react-query`); `null` with no manifest. + +`wouldEmit` construction must not throw: wrap each `factory()` + dry-run in a try/catch and +report `null` with a `probeError` string rather than failing the whole listing — a generator +that needs required options (`template`, `shared-model`) cannot be probed and must say so. + +Text format keeps the existing grouped rendering, regrouped by **layer** instead of tier, with +tier shown per row. + +- [ ] **Step 4: Run; green.** + +- [ ] **Step 5: Commit** + +```bash +git commit -m "feat(cli): meta gen --list is the catalog, with --probe answering 'what would this emit for MY model'" +``` + +--- + +## Task 6: the `requires` gate and the `api`-framework advisory + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/runner.ts` +- Create: `server/typescript/packages/codegen-ts/test/requires-gate.test.ts` + +**Interfaces:** +- Consumes: `GeneratorRegistryEntry.requires`, `.framework`, `.layer`. +- Produces: two self-extinguishing warnings on `RunGenResult.warnings`. Neither fails a build. + +`runGen` receives `generators: Generator[]` (factory instances), not names. Resolve each +instance's stable name via `generator.name` — the `Generator` interface already carries a +kebab-case `name`. Map it back to the catalog by that name; a generator whose name is not a +catalog key is an owned/third-party generator and is skipped silently (it has no declaration to +check). + +- [ ] **Step 1: Write the failing tests** + +```ts +it("warns when a wired generator's requires are not wired", async () => { + const res = await runGen({ config: { ...base, generators: [tanstackGridHook()] }, ... }); + expect(res.warnings.join("\n")).toContain("grid-hook"); + expect(res.warnings.join("\n")).toContain("grid"); +}); + +it("says nothing when the requirement is wired", async () => { + const res = await runGen({ config: { ...base, generators: [tanstackGrid(), tanstackGridHook()] }, ... }); + expect(res.warnings.filter((w) => w.includes("requires"))).toEqual([]); +}); + +it("warns when two api-layer generators declare different frameworks", async () => { + const res = await runGen({ config: { ...base, generators: [routesFile(), routesFileHono()] }, ... }); + expect(res.warnings.join("\n")).toMatch(/fastify.*hono|hono.*fastify/); +}); + +it("does NOT warn on form + hooks + grid — client is a composition, not a conflict", async () => { + const res = await runGen({ config: { ...base, generators: [formFile(), tanstackQuery(), tanstackGrid()] }, ... }); + expect(res.warnings.filter((w) => w.includes("framework"))).toEqual([]); +}); +``` + +The last two need the react/tanstack entries, which `codegen-ts` cannot import. Put those two +tests in `packages/cli/test/` and drive them through the composed catalog, passing the catalog +into `runGen` as an option: + +```ts +/** The catalog the requires/framework gates check against. Injected because the + * composition lives in the CLI (codegen-ts cannot import its own dependents). */ +catalog?: Record; +``` + +Defaulting to `generatorRegistry` keeps `runGen` correct for a programmatic embedder that never +composes. + +- [ ] **Step 2: Run and watch fail.** + +- [ ] **Step 3: Implement both gates** in `runner.ts`, after generator resolution and before + emission. Message shapes: + +``` +meta gen: "grid-hook" is wired but "grid" is not. grid-hook's output imports the + module grid emits, so `tsc` will report an unresolved import. Wire grid, or keep + your own hand-written equivalent at that path. +meta gen: two api-layer generators declare different frameworks — "routes" (fastify) + and "routes-hono" (hono). Both emit a complete HTTP surface over the same entities, + to different paths, so nothing fails; this is only a warning because migrating + between them, or serving Node and edge from one model, is legitimate. +``` + +- [ ] **Step 4: Run; green.** + +- [ ] **Step 5: Commit** + +```bash +git commit -m "feat(codegen): warn on an unsatisfied requires edge and on two api frameworks" +``` + +--- + +## Task 7: `meta eject` takes many names, reports JSON, consolidates the install set + +**Files:** +- Modify: `server/typescript/packages/cli/src/lib/args.ts` (`parseEjectArgs`: many positionals; `EJECT_OPTIONS` unchanged) +- Modify: `server/typescript/packages/cli/src/commands/eject.ts` +- Modify: `server/typescript/packages/cli/src/index.ts` (`eject` joins `FORMAT_AWARE_COMMANDS`; help text) +- Create: `server/typescript/packages/cli/test/eject-multi.test.ts` + +**Interfaces:** +- Produces: `ejectCommand(args, cwd, fmt)`. JSON payload exactly as spec §D3: + +```ts +interface EjectPayload { + ejected: Array<{ + name: string; path: string; status: "created" | "preserved" | "replaced"; + wire: { import: string; entry: string }; + requires: readonly string[]; + }>; + install: { dev: string[]; runtime: string[]; command: string }; + config: { keys: string[] }; +} +``` + +`entry` is `${exportName}()`. `install.dev` is the set of packages the ejected FILES import +(what `requiredPackages()` already derives) plus each entry's own package; `install.runtime` is +the union of each entry's `runtimePackage` and `runtimePeers`, **with ranges read from the +runtime package's own `peerDependencies`** (a new `lib/peer-ranges.ts` helper — `runtimeTsPeerRanges()` +in `init.ts` already does exactly this for one package; generalize and move it, then delete the +init copy in Task 8). + +- [ ] **Step 1: Write the failing tests** + +```ts +it("ejects several names in one call", async () => { + expect(await ejectCommand(["entity", "queries", "routes"], tmp, "text")).toBe(0); + for (const n of ["entity", "queries", "routes"]) { + expect(existsSync(join(tmp, "codegen/generators", `${n}.ts`))).toBe(true); + } +}); + +it("--format json emits ONE document and nothing else on stdout", async () => { + const out = await captureStdout(() => ejectCommand(["form"], tmp, "json")); + const payload = JSON.parse(out) as EjectPayload; + expect(payload.ejected[0]!.name).toBe("form"); + expect(payload.ejected[0]!.wire.entry).toBe("formFile()"); + expect(payload.install.dev).toContain("@metaobjectsdev/codegen-ts-react@^" + cliVersion()); +}); + +it("the install set is consolidated, not per-name", async () => { + const payload = JSON.parse(await captureStdout(() => ejectCommand(["hooks", "grid"], tmp, "json"))); + const tanstack = payload.install.dev.filter((d) => d.startsWith("@metaobjectsdev/codegen-ts-tanstack")); + expect(tanstack.length).toBe(1); +}); + +it("an unknown name refuses the WHOLE call, ejecting nothing", async () => { + expect(await ejectCommand(["entity", "nonesuch"], tmp, "text")).toBe(2); + expect(existsSync(join(tmp, "codegen/generators/entity.ts"))).toBe(false); +}); +``` + +That last one matters: a partial eject leaves a repo half-changed with a non-zero exit, which is +the worst outcome. Validate every name before writing any file. + +- [ ] **Step 2: Run and watch fail.** + +- [ ] **Step 3: Implement.** Keep every existing text-mode message (the preserved/differs/ + replaced branches are load-bearing and were written against real incidents) — they now + print per name. Print the consolidated install block once, at the end. + +- [ ] **Step 4: Run; green.** + +- [ ] **Step 5: Commit** + +```bash +git commit -m "feat(cli): meta eject takes many names and reports one consolidated install set" +``` + +--- + +## Task 8: `meta init` scaffolds the layout and an empty selection + +**Files:** +- Modify: `server/typescript/packages/cli/src/commands/init.ts` (net shrink) +- Modify: `server/typescript/packages/cli/src/commands/gen.ts` (the first-run pointer) +- Modify: `server/typescript/packages/cli/test/init*.test.ts` (roughly a dozen assertions about the five) +- Create: `server/typescript/packages/cli/test/init-empty-selection.test.ts` + +**What goes:** +- `SCAFFOLDED_GENERATOR_NAMES` and `writeOwnedGenerators`'s copy loop (the function stays, now + only calling `writeCodegenTsconfig` — or fold that call into `init` and delete the wrapper). +- `DB_STUB_BODY`, `DB_STUB_NOTE`, `DB_STUB_REL_PATH`, `SCAFFOLD_DB_IMPORT` and the whole db-stub + block in `init` — including the three-branch warning. `dbImport` leaves the scaffold, so + nothing points at `src/db.ts`. +- `addScaffoldDevDependencies`, `addScaffoldRuntimeDependencies`, `scaffoldRuntimeDependencies`, + `SCAFFOLD_OUTPUT_PEERS`, `runtimeTsPeerRanges` (the last one MOVES to `lib/peer-ranges.ts` in + Task 7 — delete the copy here, import if still needed). +- `extStyle`, `dbImport`, `apiPrefix` from the scaffolded config body. + +**What stays:** `metaobjects/`, `.metaobjects/`, `metaobjects.config.ts`, `tsconfig.codegen.json`, +the gitignores, agent context, ESM `"type": "module"` handling, `codegen/generators/` as an +empty directory (the tsconfig's `include` also names `metaobjects.config.ts`, so an empty +`codegen/` never makes the include match nothing). + +The new config body is spec §D3's, verbatim, with `dialect` interpolated. + +- [ ] **Step 1: Write the failing tests** + +```ts +it("scaffolds no generators and wires none", async () => { + await init({ cwd: tmp }); + const body = readFileSync(join(tmp, "metaobjects.config.ts"), "utf8"); + expect(body).toContain("generators: []"); + expect(body).not.toContain("./codegen/generators/"); + expect(readdirSync(join(tmp, "codegen/generators"))).toEqual([]); +}); + +it("scaffolds no db stub and no dbImport", async () => { + await init({ cwd: tmp }); + expect(existsSync(join(tmp, "src/db.ts"))).toBe(false); + expect(readFileSync(join(tmp, "metaobjects.config.ts"), "utf8")).not.toContain("dbImport"); +}); + +it("adds no runtime or codegen dependencies to package.json", async () => { + writeFileSync(join(tmp, "package.json"), JSON.stringify({ name: "x", version: "0.0.0" })); + await init({ cwd: tmp }); + const pkg = JSON.parse(readFileSync(join(tmp, "package.json"), "utf8")); + for (const d of ["drizzle-orm", "zod", "fastify", "@metaobjectsdev/codegen-ts"]) { + expect(Object.keys({ ...pkg.dependencies, ...pkg.devDependencies })).not.toContain(d); + } +}); + +it("still sets type: module", async () => { /* … the ESM rule is unchanged … */ }); + +it("the config points at the catalog", async () => { + await init({ cwd: tmp }); + expect(readFileSync(join(tmp, "metaobjects.config.ts"), "utf8")) + .toContain("meta gen --list --format json --probe"); +}); +``` + +And in `gen.ts`: + +```ts +it("an empty selection prints a pointer, not a warning, and exits 0", async () => { + const code = await genCommand([], scaffoldedTmp, "text"); + expect(code).toBe(0); + expect(stderr).toContain("Nothing is generated until you choose it"); + expect(stderr).toContain("meta gen --list"); + expect(stderr).not.toContain("WARN"); +}); +``` + +- [ ] **Step 2: Run and watch fail.** + +- [ ] **Step 3: Implement the shrink.** Then sweep the existing init tests: every assertion + naming one of the five, `src/db.ts`, `dbImport`, or a scaffolded dependency has to be + deleted or inverted. `git grep -n "SCAFFOLDED_GENERATOR_NAMES\|src/db.ts\|DB_STUB" server/typescript` + finds them. + +- [ ] **Step 4: Run the whole CLI suite.** `cd server/typescript && bun test packages/cli` + +- [ ] **Step 5: Commit** + +```bash +git commit -m "feat(cli): meta init scaffolds the layout and an empty documented selection" +``` + +--- + +## Task 9: the resolved-not-trusted gates, and the declarations they force + +**Files:** +- Create: `server/typescript/packages/cli/test/catalog-declarations-resolved.test.ts` +- Modify: all three registry slices (populate `requires`, `runtimePeers`, `runtimePackage`, `configKeys`) + +**Interfaces:** +- Consumes: `composeCatalog()`, a fixture model rich enough to make every generator emit. +- Produces: no runtime interface — this task's product is the *declarations*, made true. + +The fixture model must carry: two entities with a relationship, a `template.prompt` with a +`@responseRef`, a `template.output`, a `layout.dataGrid`, a `source.rdb` with `@kind: storedProc`, +and one `requirement.functional`. `fixtures/conformance/` has pieces of each; assemble a +dedicated one at `server/typescript/packages/cli/test/fixtures/catalog-probe/` rather than +reaching into the conformance corpus (a corpus fixture changing shape must never break this). + +- [ ] **Step 1: Write the failing gates** + +```ts +it("runtimePeers ⊇ the third-party packages the emitted files import", async () => { + for (const [name, entry] of Object.entries(composeCatalog())) { + const files = await dryRunOne(entry, model); + const imported = thirdPartyImportsOf(files); // excludes @metaobjectsdev/* and relative + const declared = new Set(entry.runtimePeers ?? []); + const undeclared = [...imported].filter((p) => !declared.has(p)); + expect(undeclared, `${name} emits imports it does not declare: ${undeclared.join(", ")}`).toEqual([]); + } +}); + +it("requires ⊇ the generators whose emitted paths this generator's output imports", async () => { + const emitters = await pathOwners(composeCatalog(), model); // path -> generator name + for (const [name, entry] of Object.entries(composeCatalog())) { + const files = await dryRunOne(entry, model); + const needed = new Set(relativeImportTargets(files).map((p) => emitters.get(p)).filter(Boolean)); + needed.delete(name); + const declared = new Set(entry.requires ?? []); + const undeclared = [...needed].filter((n) => !declared.has(n!)); + expect(undeclared, `${name} depends on ${undeclared.join(", ")} but declares none of it`).toEqual([]); + } +}); + +it("every ejectable entry has a reference template, and every template has an entry", () => { + const catalog = composeCatalog(); + const templates = new Set([...coreTpl.REFERENCE_GENERATOR_NAMES, + ...reactTpl.REFERENCE_GENERATOR_NAMES, ...tanstackTpl.REFERENCE_GENERATOR_NAMES]); + expect([...templates].filter((n) => !(n in catalog))).toEqual([]); + expect(Object.values(catalog).filter((e) => e.ejectable && !templates.has(e.name)).map((e) => e.name)).toEqual([]); +}); + +it("every reference-template header's documented facets agree with its catalog entry", () => { + // use-when / emits parsed out of the header == entry.useWhen / entry.emits +}); +``` + +- [ ] **Step 2: Run and read off the truth.** The first two tests FAIL and their messages are + the answer: they name, per generator, exactly what to declare. + +- [ ] **Step 3: Populate `requires` / `runtimePeers` / `runtimePackage` from the failures.** + Do NOT guess ahead of the gate. `configKeys` is not derivable — read each generator for + the `ctx.config.*` keys it touches and declare them by hand; add a fifth test asserting + each declared key is a real `MetaobjectsConfig` property (a compile-time + `keyof MetaobjectsConfig` type on the field does this for free — prefer that). + +- [ ] **Step 4: Run; green.** + +- [ ] **Step 5: Commit** + +```bash +git commit -m "test(catalog): resolve every compatibility declaration against what the generators actually emit" +``` + +--- + +## Task 10: C# and Python lose their default suites + +**Files:** +- Modify: `server/csharp/MetaObjects.Cli/GenCommand.cs`, `VerifyCommand.cs:250` +- Modify: `server/csharp/MetaObjects.Codegen.Tests/{CodegenDriftTests,NamesGeneratorTests,NoMagicPhysicalNamesTests,IntegrationFixtureDriftTests}.cs` +- Modify: `server/python/src/metaobjects/cli.py` (`_default_generators`, its call site at ~:510) +- Modify: the Python CLI tests that rely on a default run + +**Interfaces:** +- Produces: `dotnet meta gen` and `metaobjects gen` with no `--generators` exit **2** with a + usage error naming the catalog door, and generate nothing. + +Note `VerifyCommand.cs:250` falls back to `DefaultGeneratorNames` when the config names none — +`verify --codegen` re-runs the config's list, so with no default suite it must report "nothing +selected, nothing to check" rather than silently checking nine artifacts nobody generates. + +The test suites use the default set as a convenience list. Replace each with an explicit local +array named for what that test is about (`CodegenDriftTests` already documents that it was once +a hand-copied duplicate — the honest fix now is an explicit list, since there is no default to +track). + +- [ ] **Step 1: Write the failing tests** + +```csharp +[Fact] +public void GenWithNoGeneratorsIsAUsageError() +{ + var outcome = GenCommand.Run(metadataDir, outDir, "Acme", emitAbstractShapes: false, + generatorNames: null, templateRoot: null); + Assert.False(outcome.Ok); + Assert.Contains("--generators", outcome.Error); + Assert.Empty(Directory.GetFiles(outDir, "*", SearchOption.AllDirectories)); +} +``` + +```python +def test_gen_without_generators_is_a_usage_error(tmp_path, capsys): + rc = main(["gen", "--metadata", str(md), "--out", str(out)]) + assert rc == 2 + assert "--generators" in capsys.readouterr().err + assert not list(out.rglob("*")) +``` + +- [ ] **Step 2: Run and watch fail.** + +- [ ] **Step 3: Remove both default suites** and make `--generators` required. The error message + in both ports: + +``` +gen: no generators selected. Nothing is generated until you choose it — + pass --generators . See the catalog: +``` + +- [ ] **Step 4: Run both ports' suites.** Count the `dotnet test` result lines. + +- [ ] **Step 5: Commit** + +```bash +git commit -m "feat(csharp,python): no default generator suite — --generators is required" +``` + +--- + +## Task 11: ADR-0034 Amendment 2, the compatibility-policy narrowing, and the docs + +**Files:** +- Modify: `spec/decisions/ADR-0034-*.md` (Amendment 2) +- Modify: `docs/compatibility-policy.md:53` +- Modify: `docs/features/own-your-codegen.md`, `docs/features/cli.md`, `docs/features/codegen-concepts.md` +- Modify: `server/typescript/packages/cli/README.md` (quickstart), root `README.md` +- Modify: the `llms.txt` pair — ONE line, never the enumeration +- Modify: `agent-context/skills/metaobjects-codegen/SKILL.md` (+ references) and + `metaobjects-runtime-ui`, and the always-on template +- Regenerate: the agent-context conformance corpus **in the same commit** +- Modify: `CHANGELOG.md` + +ADR-0034 Amendment 2, verbatim intent from spec §6: Decision 2's "`meta init` scaffolds a +sensible default generator set" and the consequence "first-run still works (init scaffolds +defaults)" are replaced by — init scaffolds the layout and an empty documented selection; +`eject` is the copy door; the catalog is the composed registry behind `--list`. Everything else +in ADR-0034 stands. FR-040 was Amendment 1; do not renumber it. + +`docs/compatibility-policy.md:53` becomes: + +> - **The scaffold-and-own contract** — the *layout and the interfaces* +> (`codegen/generators/`, the local-import config shape, `.metaobjects/`, the `Generator` +> interface owned templates implement). NOT *which* generators a fresh scaffold wires: codegen +> is opt-in, and the scaffolded selection is empty by design. + +The skill gains spec §D4's **procedure**, not a list, plus intent-level recipes naming layers +only. It is gated by the existing capability-grounding test. + +- [ ] **Step 1: Find the grounding test and read what it checks.** + `git grep -rln "capability-grounding\|skill prose\|grounded" server/typescript/packages/*/test` + +- [ ] **Step 2: Extend it** so every stable name AND every layer token the skill mentions must + exist in the composed catalog. + +- [ ] **Step 3: Write the prose.** Then regenerate the agent-context corpus and read back the + **python** fixture. + +- [ ] **Step 4: Prove the cross-cutting gates** + +```bash +cd +bun test scripts/site && bun scripts/build-site-payload.ts --check +cd server/typescript && bun test packages/cli packages/codegen-ts +``` + +- [ ] **Step 5: Commit** + +```bash +git commit -m "docs(catalog): codegen is opt-in — ADR-0034 Amendment 2, the narrowed compat clause, and the selection procedure" +``` + +--- + +## Task 12: full local CI + +- [ ] **Step 1:** `nm-triage` in the repo; note the pre-gate line. +- [ ] **Step 2:** `scripts/ci-local.sh --strict-toolchains`, then **grep the SUMMARY block** — + the script exits 0 on a red gate. +- [ ] **Step 3:** Fix whatever it names; re-run the affected lane only. + +--- + +## Self-review notes + +- **Spec coverage.** §2 items 1-4 → Tasks 8, 10; §3 D1 → Task 1 (no stored bundle exists + anywhere in the plan); D2(a) → Task 3, D2(b) → Tasks 2 + 9; D3 → Tasks 5 + 7 + 8; D4 → Tasks 6 + + 11; D5 → Task 9 + Tasks 1/4. §4 cross-port → Tasks 1, 4, 10. §6 → Task 11. §7 → every task. + §8a → Task 6. §8b → Task 1. §9 (`kind`) → Task 2. +- **Not covered, deliberately:** §10's validation probe (put an adopting agent on a fresh + `meta init`) is a measurement, not an implementation step; it runs after this ships. +- **Type consistency.** `Layer`, `GENERATOR_LAYERS`, `GeneratorRegistryEntry`, `composeCatalog`, + `listCatalog`, `packageOf`, `CatalogRow`, `EjectPayload` are used with those exact names in + every task that names them. diff --git a/docs/superpowers/plans/2026-09-13-overlay-mechanism-conformance.md b/docs/superpowers/plans/2026-09-13-overlay-mechanism-conformance.md new file mode 100644 index 000000000..d9aea5822 --- /dev/null +++ b/docs/superpowers/plans/2026-09-13-overlay-mechanism-conformance.md @@ -0,0 +1,492 @@ +# Overlay Mechanism Conformance Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +> ## STATUS: COMPLETE (2026-09-13) — executed inline, `7cdd886f3` + `f6624b6e1` +> +> All three tasks done. Both fixtures landed and are **green in all five ports** +> (TS, C#, Java, Kotlin, Python), no expected-failure ledger entries, corpus +> 322 → 324, count gate and site payload verified. +> +> **Task 1 — `overlay-adds-source`: clean.** Zero errors, zero warnings. The +> layered-library design is verified cross-port. +> +> **Task 2 — `overlay-nested-requirement`: Q1 answered YES structurally**, plus +> two findings the spec had assumed away — every ancestor in an overlay chain +> must also be marked `overlay: true` or each emits +> `WARN_DUPLICATE_DECLARATION`, and an attribute override emits +> `ERR_MERGE_CONFLICT` even under an explicit `overlay: true`. Both are recorded +> in **FR-043 Amendment 2**, along with the maintainer's ruling that the flag +> licenses the override, and the coverage trap in implementing it. +> +> The plan's "stop and re-scope" branch did NOT fire: §5.5 survives. + +**Goal:** Prove, in all five ports, the two overlay behaviours that FR-043's layered library design and its `§5.5` adaptation door both rest on — and which are currently gated nowhere. + +**Architecture:** Two additions to the shared metamodel conformance corpus at `fixtures/conformance/`. Discovery is automatic — no runner code changes in any port — so each fixture reaches TypeScript, Java, Kotlin (via `metadata-ktx`), Python and C# by existing. A port that cannot yet pass one is recorded in its `conformance-expected-failures.json` ledger rather than left silently red. The corpus count is a **derived value with three committed copies**, so each fixture-adding commit must move them together or the `gates` lane goes red in a way no port lane can see. + +**Tech Stack:** Bun (TS test runner), the `@metaobjectsdev/conformance` engine, JSON fixtures, `scripts/ci-local.sh` for the port fan-out. + +**Spec:** `docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md` — **Amendment 1** ("Two things this makes worth gating") and **§12 Q1**. + +## Why this plan comes before the two feature plans + +Both remaining plans build on behaviour neither spec verified: + +- **Q1 gates "Phase 1 shippable"** by the spec's own words. If `overlay: true` does not merge on a nested `requirement.*` node, an adopter cannot disagree with a library requirement without ejecting the whole ledger, and FR-043 §5.5 collapses to eject-only — which changes what Phase 1 is. +- **Amendment 1's layering rests on an overlay adding a `source.rdb`.** That is documented in `CLAUDE.md` and has no fixture and no test in any port. It has been verified in TypeScript only, by a throwaway probe, at commit `72b103320`. + +Planning the feature work before these land would be planning on an unverified premise. + +## Global Constraints + +- **This repository is PUBLIC.** No other-project or client names, no absolute home paths (`/home//…`), in code, fixtures, docs **or commit messages**. Use repo-relative paths or ``. +- **Never run a bare `bun test` at the repo root** — it walks `java/`, `python/`, `csharp/` and `fixtures/`. Scope every run. +- **Never `git add -A`.** Stage the exact paths listed in each task; other sessions may hold uncommitted work in this checkout. +- **`main` is forward-only** — commit to `main`, no side branches unless asked; never rebase, reset or force. +- **`CLAUDE.md` is a symlink to `AGENTS.md`.** Edit `AGENTS.md`. A Python rewrite or `sed -i` on the symlink path breaks the link. +- **No metamodel vocabulary is added by this plan.** `expected-registry.json` and `metamodelVersion` do not move, and `node scripts/check-metamodel-version.mjs` must stay green. +- Adding a fixture is a **four-file change**: the fixture directory, `AGENTS.md`'s count, **both** sites in `docs/CONFORMANCE.md`, and `examples/showcase/site-payload.json`. + +## File Structure + +| Path | Responsibility | +|---|---| +| `fixtures/conformance/overlay-adds-source/` | **New.** Base declares a sourceless entity; a second file overlays a `source.rdb` + `index.lookup` onto it. Proves the layering mechanism. | +| `fixtures/conformance/overlay-nested-requirement/` | **New.** Base declares a 3-deep `requirement.functional` tree; a second file overlays the *innermost* node, changing `@status` and adding `@disposition`. Proves FR-043 §5.5. | +| `AGENTS.md:71` | Corpus count (`322` → `324`). `CLAUDE.md` is a symlink to this file. | +| `docs/CONFORMANCE.md:28,122,274` | Corpus count, in the corpora table row, the section heading, and the total line. | +| `examples/showcase/site-payload.json` | Regenerated by `bun run site:payload`; the count is baked into what the site publishes. | +| `server/{java,python,csharp}/…/conformance-expected-failures.json` | Ledger — only touched if a port genuinely cannot pass a new fixture. | + +**Do not touch** any port's conformance runner. Discovery is automatic (`fixtures/conformance/README.md`, "Adding a fixture": *"No runner code changes — discovery is automatic."*). + +--- + +### Task 1: Fixture — an overlay can add a `source.rdb` + +This is the mechanism Amendment 1's whole layered-library design rests on: `model.yaml` ships a sourceless entity, `db.yaml` overlays persistence onto it. + +**Files:** +- Create: `fixtures/conformance/overlay-adds-source/input/meta.a-model.json` +- Create: `fixtures/conformance/overlay-adds-source/input/meta.b-db.json` +- Create: `fixtures/conformance/overlay-adds-source/expected.json` +- Modify: `AGENTS.md:71`, `docs/CONFORMANCE.md:28,122,274`, `examples/showcase/site-payload.json` + +**Interfaces:** +- Consumes: nothing — first task. +- Produces: a corpus at **323** fixtures, which Task 2 takes to 324. Task 2's count edits assume this one landed. + +**Naming note:** input files are read in sorted order, so the `a-`/`b-` prefixes make "base first, overlay second" explicit. ADR-0055 made the overlay pass deferred and order-independent, so this ordering is documentary, not load-bearing — `overlay-mixed-file-base-in-later-file` already covers the inverse. + +- [x] **Step 1: Write the base — a sourceless entity** + +Create `fixtures/conformance/overlay-adds-source/input/meta.a-model.json`: + +```json +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "object.entity": { + "name": "User", + "children": [ + { "field.uuid": { "name": "id", "@required": true } }, + { "field.string": { "name": "username", "@required": true, "@maxLength": 64 } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "uuid" } } + ] + } + } + ] + } +} +``` + +- [x] **Step 2: Write the overlay — persistence only** + +Create `fixtures/conformance/overlay-adds-source/input/meta.b-db.json`: + +```json +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "object.entity": { + "name": "User", + "overlay": true, + "children": [ + { "source.rdb": { "@table": "iam_user", "@role": "primary" } }, + { "index.lookup": { "name": "ixUsername", "@fields": ["username"] } } + ] + } + } + ] + } +} +``` + +- [x] **Step 3: Write the expected golden** + +Overlay children **append** to the base's in declaration order, and `overlay: true` is not serialized (confirmed against `fixtures/conformance/overlay-same-object-different-files/expected.json`). Attributes are `@`-prefixed and sorted alphabetically after `name`. + +Create `fixtures/conformance/overlay-adds-source/expected.json`: + +```json +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "object.entity": { + "name": "User", + "children": [ + { "field.uuid": { "name": "id", "@required": true } }, + { "field.string": { "name": "username", "@maxLength": 64, "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "uuid" } }, + { "source.rdb": { "@role": "primary", "@table": "iam_user" } }, + { "index.lookup": { "name": "ixUsername", "@fields": ["username"] } } + ] + } + } + ] + } +} +``` + +- [x] **Step 4: Run it — and treat a golden mismatch as information, not failure** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts -t "overlay-adds-source" +``` + +Two outcomes, and they mean opposite things: + +- **A diff in which attributes are present/absent or ordered differently from Step 3** — the golden is wrong, not the loader. Canonical serialization omits defaulted attributes (`@role: primary` may not survive), and a defaulted `@kind` may be *added*. Replace `expected.json` with the serializer's actual output **only after** reading it and confirming the entity has exactly one `source.rdb` and one `index.lookup`. +- **A merge failure — `ERR_OVERLAY_NO_TARGET`, a second `User` object, or zero sources on the merged entity** — that is the real finding. **Stop and report it.** Amendment 1's layering does not work, and FR-043 needs re-opening before any further work. + +- [x] **Step 5: Verify the entity is genuinely persisted, not just structurally merged** + +A merged `source.rdb` that does not satisfy `isWritableSource` would pass the golden and still leave the entity inert — the exact failure the layering must not have. Confirm against the shipped predicate: + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts -t "overlay-adds-source" --reporter=verbose +``` + +Expected: both the `lint:` and `conformance:` tests for this fixture PASS. + +- [x] **Step 6: Move all four count sites in this same commit** + +The count is derived by `scripts/site/counts.test.ts` and asserted against three committed copies plus the site payload. Bump `322` → `323`: + +```bash +cd +# AGENTS.md line 71 — CLAUDE.md is a SYMLINK to this; edit AGENTS.md, never the symlink path +# docs/CONFORMANCE.md — the corpora table row (:28), the section heading (:122), the total (:274) +bun run site:payload +``` + +- [x] **Step 7: Prove the count gate green — a port lane cannot tell you** + +```bash +cd && bun test scripts/site && bun scripts/build-site-payload.ts --check +``` + +Expected: PASS. This gate lives in the `gates` lane; `--only ` never runs it. On 2026-09-12 a fixtures commit moved 314 → 322, updated only `docs/CONFORMANCE.md`, and left `site payload is true` red across the entire port fan-out. + +- [x] **Step 8: Commit** + +```bash +git add fixtures/conformance/overlay-adds-source AGENTS.md docs/CONFORMANCE.md examples/showcase/site-payload.json +git commit -m "test(conformance): an overlay can add a source.rdb + +The layered-library design in FR-043 Amendment 1 rests on a core model +declaring no source and a db layer overlaying one in. That behaviour is +documented in the layered-overlay section of the project context and was +gated nowhere — no fixture, no test, in any port. + +Corpus 322 -> 323." +``` + +--- + +### Task 2: Fixture — an overlay merges a nested `requirement.*` + +This settles FR-043 **§12 Q1**, which the spec names as the question gating "Phase 1 shippable". + +**Files:** +- Create: `fixtures/conformance/overlay-nested-requirement/input/meta.a-requirements.json` +- Create: `fixtures/conformance/overlay-nested-requirement/input/meta.b-adopter.json` +- Create: `fixtures/conformance/overlay-nested-requirement/expected.json` +- Modify: `AGENTS.md:71`, `docs/CONFORMANCE.md:28,122,274`, `examples/showcase/site-payload.json` + +**Interfaces:** +- Consumes: Task 1's corpus count of **323**. +- Produces: corpus at **324**. No code interface. + +**Why this is not already covered.** `overlay-nested-under-plain-parent-base-later` overlays a `field.string` one level inside an object. Requirements nest arbitrarily deep (the corpus's `requirement-levels-and-nesting` runs level 1 → 5), and §5.5 asks an adopter to overlay a node **two or more levels down** inside a library's tree. Depth, and a non-`field` child type, are what this adds. + +**What the fixture also documents.** To address a nested node, the overlay file re-declares the ancestor chain — plain parents, `overlay: true` on the node actually being changed. That is the shape `overlay-nested-under-plain-parent-base-later` uses, and it is what an adopter following §5.5 will have to write. It is not obvious, and the fixture is where it becomes legible. + +- [x] **Step 1: Write the base — a library-shaped requirement tree** + +Create `fixtures/conformance/overlay-nested-requirement/input/meta.a-requirements.json`: + +```json +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "requirement.functional": { + "name": "accessControl", + "@level": 2, + "@status": "live", + "@statement": "Who may do what is answered from stored grants, never from a name compared to a literal in code", + "@counterexample": "A superuser recognised by username", + "children": [ + { + "requirement.functional": { + "name": "grants", + "@level": 3, + "@status": "live", + "@statement": "A role is granted to a user either system-wide or within one group", + "@counterexample": "A grant that cannot say where it applies", + "children": [ + { + "requirement.functional": { + "name": "scopedGrantRequiresMembership", + "@level": 4, + "@status": "live", + "@statement": "A role granted within a group is held only by a member of that group", + "@counterexample": "A scoped grant naming a user who never joined the group" + } + } + ] + } + } + ] + } + } + ] + } +} +``` + +- [x] **Step 2: Write the adopter's disagreement** + +This is §5.5 exactly: same `(type, package::path)`, `overlay: true` on the innermost node, downgrading the verdict and recording why. Ancestors are re-declared plain. + +Create `fixtures/conformance/overlay-nested-requirement/input/meta.b-adopter.json`: + +```json +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "requirement.functional": { + "name": "accessControl", + "children": [ + { + "requirement.functional": { + "name": "grants", + "children": [ + { + "requirement.functional": { + "name": "scopedGrantRequiresMembership", + "overlay": true, + "@status": "partial", + "@disposition": "accepted", + "@notes": "The schema cannot express it; enforced in the service layer." + } + } + ] + } + } + ] + } + } + ] + } +} +``` + +- [x] **Step 3: Write the expected golden** + +Last-writer-wins on attribute conflicts, so `@status` becomes `partial`; `@disposition` and `@notes` are added; everything else survives. + +Create `fixtures/conformance/overlay-nested-requirement/expected.json`: + +```json +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "requirement.functional": { + "name": "accessControl", + "@counterexample": "A superuser recognised by username", + "@level": 2, + "@statement": "Who may do what is answered from stored grants, never from a name compared to a literal in code", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "grants", + "@counterexample": "A grant that cannot say where it applies", + "@level": 3, + "@statement": "A role is granted to a user either system-wide or within one group", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "scopedGrantRequiresMembership", + "@counterexample": "A scoped grant naming a user who never joined the group", + "@disposition": "accepted", + "@level": 4, + "@notes": "The schema cannot express it; enforced in the service layer.", + "@statement": "A role granted within a group is held only by a member of that group", + "@status": "partial" + } + } + ] + } + } + ] + } + } + ] + } +} +``` + +- [x] **Step 4: Run it** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts -t "overlay-nested-requirement" +``` + +Read the outcome the same way as Task 1 Step 4 — an attribute-ordering diff is a wrong golden; a merge failure is the real finding. + +**If `@status` stays `live`, or a second `scopedGrantRequiresMembership` appears, or the load errors — STOP.** That is Q1 answered NO. Report it and do not start the FR-043 plan: §5.5 collapses to eject-only, and Phase 1 has to be re-scoped before it can be planned. Record the finding in the spec's §12 Q1 in the same session. + +- [x] **Step 5: Confirm the ancestors were not duplicated** + +The failure mode that passes a naive golden is the overlay creating a *parallel* tree rather than merging. Assert the root has exactly one child: + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts -t "overlay-nested-requirement" --reporter=verbose +``` + +Expected: both `lint:` and `conformance:` PASS, and the golden above — which declares exactly one `accessControl` — is what matched. + +- [x] **Step 6: Move all four count sites in this same commit** + +Bump `323` → `324` in `AGENTS.md:71` and all three `docs/CONFORMANCE.md` sites, then: + +```bash +cd && bun run site:payload +``` + +- [x] **Step 7: Prove the count gate green** + +```bash +cd && bun test scripts/site && bun scripts/build-site-payload.ts --check +``` + +Expected: PASS. + +- [x] **Step 8: Commit** + +```bash +git add fixtures/conformance/overlay-nested-requirement AGENTS.md docs/CONFORMANCE.md examples/showcase/site-payload.json +git commit -m "test(conformance): an overlay merges a nested requirement node + +Settles FR-043 section 12 Q1, which that spec names as the question gating +'Phase 1 shippable': an adopter disagreeing with a library requirement +overlays it in place rather than ejecting the whole ledger. + +Existing nested-overlay coverage stops at a field one level inside an +object. Requirements nest arbitrarily deep, and the adaptation door needs +a node two or more levels down. + +Corpus 323 -> 324." +``` + +--- + +### Task 3: Prove both fixtures across the remaining four ports + +TypeScript is the reference implementation; a fixture green there says nothing about Java, Kotlin, Python or C#. This is where a cross-port gap becomes visible, and where the honest answer may be a ledger entry rather than a fix. + +**Files:** +- Modify (only if a port genuinely fails): `server/java/metadata/conformance-expected-failures.json`, `server/python/tests/conformance/conformance-expected-failures.json`, `server/csharp/MetaObjects.Conformance.Tests/conformance-expected-failures.json` + +**Interfaces:** +- Consumes: both fixtures from Tasks 1 and 2, committed. +- Produces: the cross-port verdict that Amendment 1's gate #1 asked for. The FR-043 plan reads this to know whether the layering is proven or ledgered. + +- [x] **Step 1: Run the four port lanes** + +```bash +cd && scripts/ci-local.sh --only java --only python --only csharp --only ts-unit +``` + +Kotlin inherits the corpus through `metadata-ktx` and runs inside the `java` lane. + +- [x] **Step 2: Read the SUMMARY, never the exit code** + +`ci-local.sh` **prints "LOCAL CI FAILED" and still returns 0.** Grep the summary block; do not branch on `$?`. + +```bash +cd && scripts/ci-local.sh --only java --only python --only csharp --only ts-unit 2>&1 | tee /tmp/ci.log; grep -A20 "SUMMARY" /tmp/ci.log +``` + +Expected: every lane `ok`. + +- [x] **Step 3: If a port fails, diagnose before ledgering** + +A ledger entry is a recorded gap, not a way to make a lane green. Read the port's actual error first: + +- **`ERR_OVERLAY_NO_TARGET` on `overlay-adds-source`** in a port means that port's deferred-overlay pass does not reach a `source.*` child. That is a real cross-port defect in the mechanism Amendment 1 depends on — **report it**, do not ledger it silently. +- **An unknown-type error on `overlay-nested-requirement`** means that port has not registered the requirement vocabulary in its conformance provider set. Check whether the port passes the existing `requirement-levels-and-nesting` fixture; if it does, the vocabulary is there and the failure is about overlay, not registration. + +- [x] **Step 4: Ledger only a genuine, understood gap** + +If and only if Step 3 establishes the port cannot pass yet, add the fixture name to that port's `conformance-expected-failures.json` with the reason. Then re-run that lane and confirm it reports `ok`. + +- [x] **Step 5: Commit — only if Step 4 changed a ledger** + +```bash +git add server//.../conformance-expected-failures.json +git commit -m "test(conformance): ledger on + +" +``` + +If no ledger changed, there is nothing to commit — say so and move on. + +- [x] **Step 6: Record the verdict in the spec** + +Amendment 1 says the overlay-adds-source behaviour "earns one conformance fixture", and §12 Q1 is still written as open. Update both in one docs commit: replace §12 Q1's text with the answer and the fixture name, and add the fixture name to Amendment 1's gating list. If any port is ledgered, say which and why — a green corpus with a ledgered port is not the same claim as a green corpus. + +```bash +git add docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md +git commit -m "docs(fr-043): Q1 answered — , by conformance fixture + +" +``` + +--- + +## Self-Review + +**Spec coverage.** Amendment 1's two gating items: item 1 (overlay adds a source, five ports) is Tasks 1 + 3; item 2 (core layer declares no source) is **deliberately not here** — it is a per-library assertion that needs `library/iam/` to exist, so it belongs to the FR-043 plan, not to a corpus plan. §12 Q1 is Tasks 2 + 3, including the "stop and re-scope" branch the spec's own wording implies. + +**Placeholder scan.** Every fixture body is complete JSON. Every command is runnable as written. The one place the plan declines to pre-compute an exact value — the canonical serializer's treatment of defaulted attributes — is handled by a step that says how to obtain it and, crucially, how to tell that case apart from a real merge failure. + +**Type consistency.** `overlay-adds-source` and `overlay-nested-requirement` are used identically in fixture paths, `-t` filters, count arithmetic (322 → 323 → 324) and commit messages. The package `acme::iam` is used in both fixtures and collides with nothing in the corpus — the existing requirement fixtures use `acme::caps` and `acme::shop`. + +**One risk worth stating.** Task 2 has a real chance of answering Q1 **no**. That is not a plan failure; it is the plan doing its job, and it is why this plan runs before the feature plans rather than inside one. diff --git a/docs/superpowers/specs/2026-06-04-generated-api-reference-design.md b/docs/superpowers/specs/2026-06-04-generated-api-reference-design.md index f35d85764..9aff6ba23 100644 --- a/docs/superpowers/specs/2026-06-04-generated-api-reference-design.md +++ b/docs/superpowers/specs/2026-06-04-generated-api-reference-design.md @@ -62,7 +62,7 @@ For each **`template.output`**: → `string` (document) / `EmailDocument` (email). - **Output parser / prompt** where generated. -Coverage **tracks the default generator suite** — document exactly what a normal +Coverage **is configuration-driven** — document exactly what the currently-configured `meta gen` produces, nothing speculative. (The api-docs generator inspects which generators are configured/registered and documents their output.) diff --git a/docs/superpowers/specs/2026-09-12-opt-in-codegen-and-generator-catalog-design.md b/docs/superpowers/specs/2026-09-12-opt-in-codegen-and-generator-catalog-design.md index 75616cf04..44fcca302 100644 --- a/docs/superpowers/specs/2026-09-12-opt-in-codegen-and-generator-catalog-design.md +++ b/docs/superpowers/specs/2026-09-12-opt-in-codegen-and-generator-catalog-design.md @@ -1,6 +1,6 @@ # Opt-in codegen and the generator catalog -**Status:** design, pending maintainer review +**Status:** approved 2026-09-13 (§8a and §8b ruled; see §10 for what is deliberately NOT here) **Date:** 2026-09-12 **Ships as:** PATCH (see §6) @@ -227,9 +227,11 @@ list: 1. Read the app's purpose and stack. 2. `meta gen --list --format json --probe`. -3. Choose by `layer`; at most one `framework` per `api` and per `client` layer; - satisfy every `requires`; take what `wouldEmit > 0` says the model already - asks for. +3. Choose by `layer`; satisfy every `requires`; take what `wouldEmit > 0` says + the model already asks for. Pick ONE `api` framework — `routes` and + `routes-hono` are alternatives. Do NOT apply that rule to `client`: + `@metaobjectsdev/tanstack` peers on `react`, so `form` + `hooks` + `grid` + is the intended composition, not a conflict. 4. `meta eject --format json`; apply `wire`, run `install.command`, set `config.keys`. 5. `meta gen`; read its warnings; typecheck. @@ -261,9 +263,26 @@ registry, or the build fails. - A test asserts every ejectable name has a registry entry and every `ejectable: true` entry has a template; `requires ⊆ registry keys` is a compile-time check. -- `runtimePeers` is checked against what the generators actually emit. +- **Every compatibility declaration is resolved, not trusted** — the doctrine + `@implementedBy` already runs on, applied to the catalog. A declaration is a + promise someone has to remember to keep, and "someone remembers" scales badly + across five ports and a growing framework set: + - `runtimePeers` — run each generator over a fixture model; assert the emitted + files' third-party imports are a subset of what the entry declares. + - `requires` — resolve each generator's emitted RELATIVE imports back to + whichever generator emits those paths; assert that set is a subset of the + declared `requires`. A new framework generator that quietly depends on + `entity` cannot ship claiming it depends on nothing. + - `framework` — the manifest is byte-matched by all five ports, so a framework + generator cannot be added without an entry that declares itself. - Skill prose is checked against the composed registry. +Note what this separates. **Applicability** — "would this emit anything for MY +model?" — is answered by `--probe`, which cannot drift because it does not +describe the generators, it runs them. **Compatibility** — "what does this need +in order to work?" — is declared, and therefore has to be gated. Conflating the +two is how a catalog goes quietly wrong as frameworks are added. + ## 4. Cross-port **The vocabulary and the policy are cross-port; the mechanism is per-port.** @@ -368,44 +387,95 @@ README quickstart, the root README, the `llms.txt` pair (one line — "codegen i opt-in; `meta gen --list` is the catalog" — never the enumeration), a migration note, `docs/compatibility-policy.md:53`, and the CHANGELOG. -## 8. Open questions for the maintainer - -### 8a. Is `framework` exclusivity a gate, or only part of the procedure? - -D4 tells the builder to wire at most one `framework` per `api` and per `client` -layer, but nothing enforces it: wiring both `routes` and `routes-hono` would emit -two HTTP surfaces over the same entities. Three options, in ascending cost — -leave it advisory (the generated files have different names, so the failure is -visible and cheap to undo); add a warning to the `requires` gate when two entries -in the same layer declare different `framework` values; or make it a real -exclusivity slot, the way Quarkus fails a build when two extensions provide the -same capability. Advisory is consistent with every other gate here being -self-extinguishing, and with not hard-coding decisions into the CLI — but it is a -ruling, not an omission, and the plan needs it settled. - -### 8b. The closed `layer` set - -It becomes cross-port gated vocabulary, so it needs a ruling rather than a guess. -Verified to cover all 34 generator names exactly — the 29 in the manifest plus -the five being added — with no gaps and no invented entries. Proposed: - -| layer | members | -|---|---| -| `model` | entity, names, barrel, dto, value-object | -| `persistence` | queries, db-context, repository, exposed-table, relations, stored-proc, callable | -| `api` | routes, routes-hono, filter-allowlist, validator, spring-config | -| `client` | form, hooks, grid, grid-hook | -| `prompt` | prompt-render, output-parser, output-prompt, extractor, render-helper, payload | -| `trace` | trace-helper | -| `requirements` | requirement-tests | -| `publish` | shared-model | -| `docs` | docs, mermaid-er, api-docs | -| `primitive` | template | +## 8. Rulings (was: open questions) — settled 2026-09-13 + +### 8a. `framework` exclusivity is an ADVISORY, and only on the `api` layer + +Verified first: `routes` emits `.routes.ts` and `routes-hono` emits +`.routes.hono.ts`. **Different paths** — so wiring both does not trip the +runner's conflicting-output-path error, does not fail `tsc`, and silently +produces two complete HTTP surfaces over the same entities. + +It is nonetheless not an error. Migrating Fastify→Hono, or serving Node and edge +from one model, are legitimate. So: a self-extinguishing warning when two wired +`api`-layer entries declare different `framework` values, consistent with every +other gate here. No build failure. + +**And no general per-layer rule**, because the `client` layer disproves it: +`@metaobjectsdev/tanstack` declares `react` as a peer, so `form` (react) + +`hooks`/`grid` (tanstack) is the documented, normal composition. An earlier draft +of this spec stated "at most one framework per `api` and per `client` layer"; +that rule would have forbidden the single most common client selection, and is +corrected in D4. + +### 8b. Six layers, not ten + +The ten-layer draft had four single-member layers (`trace`, `requirements`, +`publish`, `primitive`). A layer with one member does no grouping work, and the +draft conflated two different kinds of choice: the app-shape decisions a builder +makes, and the model-driven ones the model has already made. Nobody picks +`prompt-render` by browsing a taxonomy — they pick it because they declared a +`template.prompt`, which `--probe` reports exactly, with a file count from the +real model. + +| layer | members | chosen by | +|---|---|---| +| `model` | entity, names, barrel, dto, value-object | app shape | +| `persistence` | queries, db-context, repository, exposed-table, relations, stored-proc | app shape | +| `api` | routes, routes-hono, filter-allowlist, validator, spring-config | app shape | +| `client` | form, hooks, grid, grid-hook | app shape | +| `docs` | docs, mermaid-er, api-docs | on by default | +| `capability` | prompt-render, output-parser, output-prompt, extractor, render-helper, payload, trace-helper, requirement-tests, shared-model, template, callable | `--probe` | + +All 34 names covered — the 29 in the manifest plus the five being added — with no +gaps and no invented entries. + +`capability` looking like a large undifferentiated bucket is the point: you are +not meant to choose inside it by reading labels. `output-parser: 3, callable: 0, +requirement-tests: 7` from your own model is strictly better information than a +category name, and it cannot go stale. Six gated values instead of ten, and one +fewer invented taxonomy — the same principle the opt-in ruling rests on. `layer` rather than a reuse of "tier", which is already taken twice (native/neutral in ADR-0020, and server/UI elsewhere). -## 9. Validation +## 9. Forward compatibility: this catalog is the first `kind` + +There is a larger idea this design must not foreclose, specified separately in +**FR-043**: shipping *packages* an agent pulls into a project and adapts — +**feature packages**, about a capability the application has, and +**non-functional packages**, about a property its construction has. + +That is not greenfield. `library/ai/llm-call.yaml` already ships one: the +LLM-call audit model, opted into by name through the loader's `libraries` +option, embedded per port under an `embedded-library drift` gate, with the +`trace-helper` generator and OMDB runtime beside it. Read FR-043 before +assuming the two kinds differ by whether they carry code — that shipped package +is a *feature* package which nonetheless carries a generator, so the real model +is a set of components (metadata, requirements, generator selection, runtime +helpers) of which any subset may be present. + +Two decisions here keep that reachable without building any of it now. + +**A catalog entry gains a `kind`, rather than packages becoming a parallel +system.** Today every entry is `kind: "generator"`. A package is a later value. +Nothing else in this spec needs to change for that: entries are already keyed on +ADR-0021 stable names, which is exactly what a package would reference, and +`requires` already expresses "this needs that" in a form that extends from +generator→generator to package→package. + +**Three mechanisms exist, and FR-023 is not the one.** `libraries: [...]` is +embed-and-use (what the shipped `ai` package does); FR-023 `dependencies` is +*sync-and-pin* — hash-locked, deliberately excluded from your own codegen and +ledger — which is right for sharing a model you do NOT own; `eject` is +copy-and-own. A package pulled in to be modified and implemented against wants +its requirements IN your ledger, which is the opposite of what a dependency +does. So the shape is embed by default, eject to own — `library` composed with +ADR-0034 — and FR-023 stays what it is for. + +Nothing in §3 is built speculatively for this. The `kind` field is one string. + +## 10. Validation FR-040 §1's own bar: put an adopting agent on a fresh `meta init` with the new catalog and skill, on a stack nobody wrote a recipe for, and check whether it diff --git a/docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md b/docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md new file mode 100644 index 000000000..9f5c94e32 --- /dev/null +++ b/docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md @@ -0,0 +1,892 @@ +# FR-043 — Libraries: reusable declared design + +**Status:** design approved 2026-09-13. Supersedes the "deferred" sketch of the +same date — the maintainer reversed that deferral once it became clear the +mechanism already ships. +**Vocabulary impact: none.** No type, subtype or attribute is added. +`expected-registry.json` and `metamodelVersion` do not move. This is shipped +*content* plus toolchain surface. + +**Maintainer rulings, 2026-09-13:** the library is named **`iam`** (§7.1); +**Permission is an entity with two grant junctions** (§7.2); **object coverage +activates on adopter-authored requirements only** (§5.4); **`iam` ships +`stability: preview`** with a promotion bar (§10) — and an adopter who wants a +different shape can always copy the metadata into their own project and rename +the packaging, which is the eject door (§3.4) and the reason a shape freeze is +survivable. + +## Amendment 1 (2026-09-13) — libraries are LAYERED; the core model is inert + +**Ruled by the maintainer, and it supersedes §3.3, §7.4, §10 and §12 Q2 as +originally written.** A library ships its **core model**, its **DB persistence** +and its **UI rendering** as separate layers, the second and third applied as +`overlay: true` files over the first. Requirements split the same way. An adopter +takes the core model alone, or adds the DB layer, or adds the UI layer. + +This is the layered overlay pattern the project already documents (`CLAUDE.md`, +"Optional layered overlay pattern") and the shape the maintainer used habitually +on the predecessor JVM implementation — DB and UI metadata in their own files. + +**Why it changes the design rather than decorating it.** The core layer declares +no `source.rdb`, and a sourceless object is inert by a contract that already +ships: + +- `server/typescript/packages/migrate-ts/src/expected-schema.ts:201` — + `if (!hasWritableSource) continue;` — no writable source, no table. +- `server/typescript/packages/codegen-ts/src/instance-artifacts.ts:15-25` — + sourceless means no route, no queries, no hooks, no grid, no form. It still + gets a type-only interface, so `extends` and reference still work. + +Both cite #248: *persistability derives from source presence, never from the +object subtype.* So `libraries: ["iam"]` adds **zero tables and zero generated +code**. What an adopter gains is the design being present and resolvable — which +is the whole point: an agent working in the repo knows the capability exists and +can draw on it, and nothing else happens until the adopter asks for it. + +**Verified for this amendment** (real loader, `strict: true`, at `72b103320`): +a core `object.entity` with no source loads clean with zero writable sources; a +second file declaring the same `(type, package::name)` with `overlay: true` and a +`source.rdb` + `index.lookup` child merges to exactly one writable source, no +errors. + +### What this supersedes + +| section | as written | amended | +|---|---|---| +| §3.3 | opting in proposes nine `CREATE TABLE`s | opting into the **core** proposes none; the **db layer** is the opt-in that proposes tables | +| §7.4 | one `model.yaml` carrying `source.rdb` | `model.yaml` (sourceless) + `db.yaml` (overlay: sources, lookup indexes) | +| §10 | risk: "Opt-in means tables. Nine for `iam`" | **withdrawn** — the core layer adds none | +| §10, §0 | "the `ai` concrete `LlmCall` wart persists" | **withdrawn** — `ai` splits the same way (see below) | +| §12 Q2 | `migrate.scope.exclude` — Phase 1 or Phase 2? | **dissolved.** Nothing is added to subtract. `migrate.scope` stays include-only and its committed rationale stands | +| §12 Q3 | composite `identity.reference` conformance case | **dropped.** `iam` uses no composite FK; this was a metamodel question wearing a library costume | +| §12 Q4 | should `libraries` move to `.metaobjects/config.json`? | **yes, outright.** A sweep of ~70 estate `metaobjects.config.ts` / `.metaobjects/config.json` files found **zero** uses of the key — there is no installed base to dual-read for | +| §11 | a `delete` overlay directive | **ruled: not Phase 1**, unchanged. It stays a candidate awaiting evidence that the track-upstream minority is real | + +### The `ai` split falls out of the same ruling + +`library/ai/llm-call.yaml` ships `LlmCallBase` (abstract, sourceless) beside a +concrete `LlmCall` carrying `source.rdb: { table: llm_call }`. §0 and §10 carried +that as an accepted wart, on the grounds that splitting it changes what existing +`ai` adopters get. The estate sweep removes the objection: there are no existing +`ai` adopters. `ai` splits into `ai/model.yaml` + `ai/db.yaml` like everything +else, and the wart is closed rather than documented. + +### Opt-in surface + +Layers are addressed path-like, which needs no config schema change — `libraries` +stays `string[]`: + +```jsonc +"libraries": ["iam"] // core model only — inert +"libraries": ["iam", "iam/db"] // + persistence +"libraries": ["iam", "iam/db", "iam/ui"] // + UI +``` + +`"iam/db"` **implies** `"iam"`: a db layer is an overlay, and an overlay whose +target was never declared is already `ERR_OVERLAY_NO_TARGET`, so implication is +the only coherent reading. `librarySources()` today is package-granular +(`REFS_BY_PACKAGE[pkg]` returns every ref under the package), so layer selection +is real work — see the plan. + +### Two things this makes worth gating + +Neither gates the library; both gate what the layering **rests on**. + +1. **"An overlay can add a `source.rdb`" is documented in `CLAUDE.md` and gated + nowhere** — no conformance fixture, no test, in any port. Verified working in + TypeScript for this amendment; Java, Python, C# and Kotlin are unverified. The + layered design makes this behaviour load-bearing in all five ports, so it + earns one conformance fixture. +2. **The core layer must declare no source.** That is the inertness promise, and + an added `source.rdb` in `model.yaml` would break it silently. One assertion + per library, in the doctrine of §4's "every manifest fact is resolved, not + trusted". + +## Amendment 2 (2026-09-13) — Q1 is answered, and `overlay: true` licenses an override + +**Both of Amendment 1's gating fixtures have landed and are green in all five +ports** (TS, C#, Java, Kotlin, Python — `scripts/ci-local.sh`, no expected-failure +ledger entries): + +| fixture | what it settles | +|---|---| +| `fixtures/conformance/overlay-adds-source` | Amendment 1's layering. A core model declares no source; a db layer overlays `source.rdb` + `index.lookup` onto it. **Zero errors, zero warnings.** | +| `fixtures/conformance/overlay-nested-requirement` | §12 **Q1**. An adopter overlays a library requirement nested three deep. **The tree merges correctly** — right shape, no duplicates, adopter's values applied. | + +**Q1 is answered YES structurally**, so §5.5 is not withdrawn and Phase 1 keeps +its shape. But running it surfaced two things the spec had assumed away. + +### Finding 1 — the whole ancestor chain must be marked, not just the leaf + +Addressing a nested node means re-declaring its ancestors. Every one of them must +**also** carry `overlay: true`. Left plain — which is the shape the pre-existing +`overlay-nested-under-plain-parent-base-later` fixture uses — each ancestor emits +`WARN_DUPLICATE_DECLARATION` ("duplicate declaration … with no semantic change"), +so a depth-4 library tree costs **three warnings to change one leaf**. Marked, the +load is silent. Measured, not inferred. This is a documentation obligation on +§5.5 and on `docs/features/libraries.md`. + +### Finding 2 — and the ruling that follows + +**§0's fact row is wrong as written.** It states: *"Overlay attr conflicts are +last-writer-wins, and adopter files load after library files ⇒ The adopter always +wins. This is the adaptation door."* The adopter's value does win — but the load +emits `ERR_MERGE_CONFLICT`, **even under an explicit `overlay: true`**: + +``` +ERR_MERGE_CONFLICT attr '@status' conflicts: + existing value "live" differs from new value "partial" on scopedGrantRequiresMembership +``` + +That is deliberate FR5c behaviour shared by every overlay, not something specific +to requirements, and `parser-core.ts:1035-1040` records the intent: *"the error +surfaces the conflict so a consumer can fix the metadata."* The loader treats an +override as a defect to fix. Adding a **new** attribute is clean — so `@disposition` +and `@notes` merge silently and only `@status: live → partial` conflicts, which is +precisely what §5.5 asks an adopter to do. + +**MAINTAINER RULING: `overlay: true` licenses the override.** `ERR_MERGE_CONFLICT` +fires only when the conflicting redeclaration is **not** marked `overlay: true`. + +> **SHIPPED 2026-09-13** in all four loaders (Kotlin inherits the JVM's), with the +> fixture pair below. One consequence beyond what this amendment weighed: the ruling is +> unconditional, so it also reaches a `dependency` node an adopter overlays. +> `fixtures/dependency-conformance`'s `an-overlay-attr-the-base-now-sets-differently-conflicts` +> expected `ERR_MERGE_CONFLICT` there and now loads clean; it was renamed +> `…-is-licensed` and an unmarked sibling added, so the accident case stays covered on +> that axis too. What guards a dependency is the hash lock plus `meta deps check` +> (upstream moved) and `refuseUnownedPackages` (a NEW node in their package), not the +> merge-conflict error — but the loss of that one signal is recorded here rather than +> discovered later. + +The reasoning: the conflict error exists to catch two files that collided without +knowing about each other. `overlay: true` is the author saying "I know about the +other declaration and I mean to change it." The loader already treats the flag +specially (find-or-throw versus create-or-find), and this makes it mean one +coherent thing instead of two. It is a loader behaviour change, **no vocabulary**, +so `expected-registry.json` and `metamodelVersion` do not move. + +It is also not only a requirements fix — it removes the same papercut from the db +and ui layers the moment an adopter *retunes* an inherited attribute rather than +only adding one. + +### What the ruling costs — and the coverage trap in it + +Four loaders (TS, Java, Python, C#; Kotlin inherits the JVM's), plus a fixture +change that must not be a blanket flip: + +- **`overlay-attr-last-writer-wins` currently marks its overlay `overlay: true` + and expects `ERR_MERGE_CONFLICT`.** Under the ruling it expects **zero** errors. + Flipping it alone would silently delete the only coverage of the accident case. +- So a **new fixture must take over the error branch** — an unmarked + redeclaration whose attribute conflicts. The loader merges a same-`(type, name)` + redeclaration either way, so that case stays reachable and stays an error. +- `overlay-nested-requirement`'s `expected-errors.json` drops to zero errors in + the same change. + +Net corpus effect: +1 fixture, two expectation flips, and the four-site count bump. + +## 0. Facts this rests on — verified in the tree, not assumed + +| Fact | Consequence | +|---|---| +| `libraries: [...]` **prepends** library sources; `Collection.imported()` is FR-023-only | Library nodes are **in scope by default** for codegen, migrate and the ledger. This is what distinguishes a library from a dependency. It must be stated, probed and guarded (§3.3). | +| `library/ai/llm-call.yaml` ships a concrete `LlmCall` with `source.rdb` | Opting into `ai` proposes `CREATE TABLE llm_call`. Already recorded as an un-split cross-port wart. | +| Every port embeds `library/**/*.yaml`; `knownLibraryPackages()` derives from the embedded refs | Adding `library//` already reaches five ports with no port edit. A manifest rides the same embed. | +| `refuseUnownedPackages` exists for dependencies; **nothing reserves `metaobjects::`** | An adopter can today declare a new node into `metaobjects::ai` silently. | +| `checkRequirements` early-returns only when the tree has **zero** requirements | A library shipping requirements would switch the unclaimed-entity gate on for every adopter entity. §5.4 is the rule that prevents it. | +| Architectural claims propagate down `extends`; functional ones do not. Bare `@implementedBy` binds package-locally (ADR-0042) | A library's architectural requirement on its abstract base claims every adopter subtype for free. | +| An M:N `@through` junction must declare **exactly two** `identity.reference` children | A three-FK scoped-grant table cannot be an M:N relationship; it is read by explicit finders. | +| Overlay attr conflicts are last-writer-wins, and adopter files load after library files | The adopter's VALUE always wins — but see **Amendment 2**: today the load also emits `ERR_MERGE_CONFLICT`, even under `overlay: true`. Ruled to be licensed by the flag. | +| `requirementTests()` is filter-driven | No day-one stub ambush. | + +**Independently verified for this FR:** the §7.4 model **loads clean under +`strict: true`** against the real registry — `identity.reference` +(`references`/`onDelete`), `identity.primary` (`generation`), `autoSet`, +`stringFormat: email`, `field.enum` (`values`/`default`), +`relationship.association` (`through`) and `index.lookup` are all registered +vocabulary today. + +## 1. The pillar + +A library is not a sixth verb; it is the **reuse unit that composes the other +five** — the first pillar's inputs, the fifth's, the first's generator +selection, and the second's runtime helpers, shipped as one named, opt-in, +drift-gated artifact. What makes it a pillar rather than a folder of YAML is the +fifth: without requirements a library is a schema snippet; with them it is design +an adopter's build is held to. The test applied is the one the requirements +pillar itself passes — *does it change what an agent can be checked against?* It +does: an adopter who opts into `iam` gets `meta verify` holding their build to +"no authorization decision is hard-wired to a name", which no snippet can do. + +Proposed paragraph, in the register of the existing five (to land in `CLAUDE.md` +when Phase 1 ships, not before): + +> 6. **Libraries** — a capability is declared once and pulled in, not re-derived +> per project. A library is a named bundle of the other pillars' inputs — model +> metadata, the requirements that make its design checkable, the generators it +> implies, and the runtime helpers those generators emit against — opted into by +> name (`libraries: [...]`), embedded in every port under a byte-identity drift +> gate, discovered through the same catalog as generators (`kind: "library"`), +> adapted with `extends` and `overlay: true`, and taken over with `meta eject`. +> Model and requirements are cross-port by construction; generator selection and +> runtime are per-port by nature. Two ship: `ai` (the LLM-call trace envelope) +> and `iam` (users, groups, roles, permissions). A library is first a REFERENCE: +> the expected use is to copy it, repackage it and make it yours, exactly as the +> codegen reference templates are copied under ADR-0034 — using one in place and +> adapting it by overlay is the deliberate choice of an adopter who wants to +> track upstream. A project that names no library sees nothing. + +## 2. The model + +A **library** is `library//` containing: + +| component | file | cross-port | required | +|---|---|---|---| +| manifest | `library.json` | yes (embedded like the YAML) | yes | +| model | `model.yaml` (or several) | yes | no | +| requirements | `requirements.yaml` | yes | no, but a model with none is the gap this FR closes; the manifest test warns | +| generator selection | manifest `generators[]` — stable names + optional `anchor` | names are cross-port; generators are per-port | no | +| runtime helpers | manifest `runtime.{…}`, informational | per-port | no | + +**Feature vs non-functional is what a library is *about*, never a structural +constraint.** `ai` is a feature library carrying a generator and runtime; `iam` +is a feature library carrying model + requirements only; an NFR library ("this +has an HTTP tier") carries generators + architectural requirements and possibly +no model. The manifest's `kind` is a catalog facet, not a switch. + +**Three things a library is not.** It never ships metamodel vocabulary — the +sealed registry stays sealed (ADR-0023); a library that "needs an attr" has +failed ADR-0037 step 0. It is not a dependency (§3.6). It is not a generator +bundle — the catalog's no-stored-bundles ruling holds: a library *implies* +generators by stable name and warns; it never wires. + +Every node lives in `metaobjects::`, and `metaobjects::` is reserved for +shipped libraries (§3.5). + +## 3. Mechanism + +### 3.1 Authoring discipline + +1. Ship a concrete entity **only where the design needs a table** — a foreign key + must point at one, so a relational library is necessarily concrete. Every + concrete entity is a table the adopter gets; `--probe` prints the count. +2. Physical names carry a library prefix (`iam_user`, `llm_call`) — collision + with the adopter's tables, and `user`/`group` are reserved words in Postgres. +3. `field.uuid` + `generation: uuid` on principals; composite assigned keys on + junctions. Never `increment` — a library cannot know the adopter's id strategy. +4. Ship no adopter-facing profile data; that arrives by `overlay: true`. +5. No credentials (§7.3). + +### 3.2 Opt-in — unchanged + +`libraries: ["iam"]`, per port as today; unknown name is a hard config error +listing what ships. One change: `knownLibraryPackages()` derives from the +embedded **manifests** rather than file refs, so a pure-NFR library with no YAML +is still a known name. + +### 3.3 Scope: in by default, and why + +> **AMENDED — read Amendment 1 first.** The paragraph below describes the +> pre-layering design, in which one undivided library put nine tables into your +> migration. Under the amendment the **core layer is sourceless and therefore +> inert**: in scope, resolvable, visible to an agent, and generating nothing. +> "In scope by default" survives and still distinguishes a library from an +> FR-023 dependency — but what is in scope by default now *produces nothing* +> until the adopter opts into the db layer. The `migrate.scope` asymmetry the +> last sentence carries is no longer reachable from here. + +A library's nodes are generated, migrated and ledgered **as if you had written +them** — the opposite of FR-023, because you opt in to *have the thing*. +Consequences stated plainly: `meta migrate` after `libraries: ["iam"]` proposes +nine `CREATE TABLE`s, and `--probe` says so before you commit. Dropping one +library table from a migration is awkward because `migrate.scope` is +include-only; that asymmetry predates this FR and is carried as an open question +rather than solved here. + +### 3.4 Adaptation: copy is the expected mode + +**A library is first a reference — something to copy and make your own.** That is +the same ruling ADR-0034 made on the generator side: the reference templates are +copied into the adopter's repo because the adopter owns their code. Metadata is +no different, and a design that told adopters to use library metadata in place +while adapting it through a merge would contradict the project's own doctrine. + +So the ladder leads with copy, and using a library in place is the deliberate +minority choice made by someone who wants to track upstream: + +| you want to | door | +|---|---| +| **the design, as a starting point you own** — rename the package, delete what you do not need, change a PK strategy, keep the requirements and edit them | **`meta eject `** — the expected path | +| a new shape sharing a library base, tracking upstream | `extends` | +| add to a shipped node while tracking upstream (fields, indexes, views, `@filterable`) | `overlay: true` on the same `(type, metaobjects::::Name)` | +| change a shipped requirement's verdict while tracking upstream | `overlay: true` on the requirement node (§5.5) | + +Two consequences follow, and both are good: + +- **The shape freeze mostly evaporates.** If most adopters copy, a later change + to `iam` reaches only the minority who opted to track it. That is the real + answer to the `stability` question, of which `preview` is only the belt. +- **Phase 1's weight shifts to the copy path.** The provenance header, a clean + package rename, and the staleness report are the parts that have to be + excellent; `libraries: [...]` polish matters less than it looked. + +`meta eject iam` +copies `library/iam/*.yaml` into the project's first resolved source root (via +`resolveCollection()`, never a hard-coded directory name), stamps a provenance +header, and prints the next step: remove `iam` from `libraries`. From there the +adopter owns it and may rename the package freely. It does not edit the config. +Two guards: + +- **Ejected-and-still-opted-in is refused at load** (`ERR_LIBRARY_PACKAGE_COLLISION`). + Without it the copy silently merges as an overlay: additions take, deletions do + not, and nothing says so. +- **`meta eject --list` reports per-node staleness** — load shipped and ejected + trees standalone, canonical-serialize each root node in own-mode, report + `identical | differs` with counts changed / upstream-only / local-only. + Metadata drift *is* summarisable, through the serializer that already exists, + and it is formatter-proof by construction. + +### 3.5 Ownership + +Reuse `refuseUnownedPackages` (TS + Python) against opted-in libraries: +declaring a new root node into `metaobjects::` while it is opted in is +refused, with the fix named (own a package and `extends`, or `overlay: true`). +`metaobjects::` in adopter sources with no ejection provenance is a `verify` +advisory. Loader-level cross-port versions are Phase 2. + +### 3.6 Why not FR-023 + +A dependency is *someone else's model you must not drift from* — excluded from +your codegen/schema/ledger, hash-locked. A library is *a design you adopt as +yours* — in scope, overlay-able, ejectable. The one device libraries borrow is +the ownership refusal. FR-023's reserved `npm`/`python` transports are the right +vehicle for **third-party** libraries in Phase 2. + +## 4. The registry + +**It is the codegen catalog**, `kind: "library"`. Every reason that design gave +for one table holds: one door, one namespace, one `--probe`, one skill procedure. + +The record is `library//library.json`, embedded beside the YAML: + +```json +{ + "name": "iam", + "kind": "feature", + "stability": "preview", + "since": "1.1.0", + "description": "Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group.", + "useWhen": "the application has people who log in and things some of them may not do", + "packages": ["metaobjects::iam"], + "model": ["iam/model"], + "requirements": ["iam/requirements"], + "generators": [], + "runtime": {} +} +``` + +`ai`'s adds `"generators": [{ "name": "trace-helper", "anchor": "metaobjects::ai::LlmCallBase" }]` +and its runtime packages. + +`meta gen --list --format json` emits a `kind: "library"` row carrying +`libraryKind`, `stability`, `ports`, `description`, `useWhen`, `packages`, +a computed `provides` (`entities`, `abstracts`, `requirements`, `generators`) and, +under `--probe`, a `project` block: `optedIn`, `extendedBy`, `tablesAdded`, +`requirementsAdded`, `impliedGeneratorsNotWired`. The cross-port subset is +`name`, `kind`, `ports`, `description`, `useWhen`, `stability`. + +**Agent consumption:** the `metaobjects-codegen` procedure gains one step before +"choose by layer" — *list libraries; if a `useWhen` matches the capability you +are about to model, opt in and adapt rather than author* — and +`metaobjects-authoring` gets the mirror rule where it teaches declaring a new +entity. Both are gated by the existing capability-grounding test. + +**Every manifest fact is resolved, not trusted**, by one test per port: +`packages` against the library loaded standalone; `model`/`requirements` refs +against the embedded set; `generators[].name` against the registry; +`generators[].anchor` against the library's own nodes; `name` = last package +segment; every `@implementedBy` resolving **within the library standalone** — a +library's ledger must be self-contained; and the prose grounded like skill prose. +A library name may not equal a generator stable name. + +## 5. Requirements in libraries + +### 5.1 Semantics + +`requirements.yaml` is ordinary `requirement.*` metadata in `metaobjects::`. +On opt-in it enters the adopter's ledger with **no new machinery**. One reading +rule: **`live` in a library means "the model as shipped realises this"**, never +"your application does". Behaviour the model cannot carry ships as `partial` + +`disposition: accepted` with a `notes` sentence naming what the adopter must do. +That is the honest boundary — a ledger binds to model nodes; runtime guarantees +are the runtime package's tests, and this FR does not invent a way to point a +requirement at code (`@verifiedBy` was retired for exactly that). + +### 5.2 Levels + +A library's functional tree roots at **L2** — L1 is the adopter's solution, and a +library is by definition a segment of someone's. A root-level L2 is legal. +Architectural requirements ship flat. + +### 5.3 `@implementedBy` across the boundary + +| direction | works | mechanism | +|---|---|---| +| library requirement → library node | yes | bare name binds package-locally; the standalone gate proves every claim resolves with no adopter present | +| library **architectural** → adopter entity | yes, free | propagation down `extends` | +| library **functional** → adopter entity | no, by design | the adopter says why their entity exists | +| adopter requirement → library node | yes | FQN | + +### 5.4 Day one — the ruled rule + +**Object coverage activates on adopter-authored requirements only.** Provenance +is already on the node. Library requirements are always counted, always +gate-checked for their own integrity, and always claim what they claim — but a +library cannot volunteer you for coverage. The moment you write your first +requirement, coverage includes the library's entities too, which by then are +claimed by the library's own ledger and add no warnings. + +Day-one output for a no-ledger project opting into `iam`: + +``` +meta verify — requirements: 21 entries (17 functional, 4 architectural) — + 19 live, 2 partial; coverage: not measured (no project-authored requirements). +meta verify — requirements: 0 recorded gap(s) with no @disposition. +``` + +A library must not ship unruled gaps; the standalone gate asserts it. + +### 5.5 Disagreeing with a library requirement + +Overlay it: same `(type, package::path)`, `overlay: true`, set `status: partial`, +`disposition: accepted`, `notes: …`. Last-writer-wins gives the adopter +precedence. **Evidence still needed** — see §11 Q1. + +### 5.6 The `ai` retrofit + +`library/ai/requirements.yaml` is the worked example, and it is a retrofit rather +than new design: `library/ai/llm-call.yaml` landed 2026-06-03 and +`requirement.functional` first appears 2026-08-11, so the library could not have +carried requirements when it was written. + +An L2 `llmTracing` with `envelope` / `accounting` / `typedIo` beneath it, and two +flat architectural claims (`traceRowsCarryTiming`, `traceRowsCarryOutcome`) on +`LlmCallBase`, which propagate to every adopter entity extending it. The entry +that earns its keep is `typedIo`, honestly `partial` + `accepted`: the library +declares the envelope and the **adopter** declares the typed VO columns, so the +requirement records the seam ADR-0024 drew — in the ledger, where an agent reads +it before adding a fourth trace column. + +## 6. NFR / codegen support + +The manifest lists stable names this library implies, each with an optional +`anchor` — the library node the generator keys on. Nothing is wired; two +self-extinguishing warnings do the rest: *library opted in, implied generator not +wired* and *generator wired, its library not opted in*. + +**Retiring the hard-coded entity name.** `runGen` gains +`ctx.libraries: LibraryManifest[]`, and `trace-helper` replaces +`const LLM_CALL_BASE = "LlmCallBase"` with the anchor of whichever opted-in +library lists it, resolved to a node and compared by **node identity** rather +than `.name` — which also fixes a latent bug, since today any adopter entity +named `LlmCallBase` in any package triggers the generator. Java and Python read +the same embedded manifest. Floor if a port's plumbing slips: a test asserting +the port's constant equals the manifest anchor. + +"Live only if wired" for an NFR library's architectural requirement is +**derivable** — requirement's package → library → `generators` → wired list — so +it needs no vocabulary (ADR-0037 step 0). Phase 2, because it is the first time +`verify` reads generator wiring for a requirement verdict. + +## 7. The `iam` library + +### 7.1 Name — ruled: `iam` + +Identity and access management names both halves. `user` is too narrow for nine +entities and makes `metaobjects::user::User` stutter; `rbac` names only the +authorization half. + +### 7.2 Permission — ruled: an entity, with two grant junctions + +The legacy shape (User / Group / Role + two junctions) forces every authorization +check to compare a role name to a literal, so adding a role is a code change and +roles multiply. The assignable unit is the permission; a role is a reusable +bundle; code asks `can(user, "invoice:approve")` and the mapping is data. +Permission earns entity status on ADR-0037's own reasoning: its own identity (a +stable key), its own lifecycle, and a junction with real foreign keys. + +Keep the legacy's genuinely good idea — **group-scoped role grants** — and add +the case it lacked, a system-wide grant. **Two junctions rather than one with a +nullable scope**: a nullable column in a unique key is the SQL trap (NULLs are +distinct, so global grants could duplicate), and fixing it needs a partial-index +`@where` whose expression carries a physical column name. Two composite-keyed +tables need no escape hatch and survive three dialects and five ports unchanged. + +`GroupType` stays an entity, not an enum: "which roles may be held in this kind +of group" is data an adopter extends, and an enum's `@values` cannot be extended +by overlay. + +### 7.3 Deliberately out + +**Authentication.** No password, no secret question, no session. An adopter's +legacy model of this shape stored a length-bounded plaintext password and a +knowledge-based secret pair on the user row — practices an agent extending "the +user model" re-derives on sight, which is why the library ships a **negative +architectural requirement** against them rather than pretending the risk is not +there. Credentials are a separate capability with an entity per factor. Also out: +profile/PII (adopter overlay), audit history (a future library), and a runtime +`can()` helper (Phase 2 — the first library runtime needing five implementations). + +### 7.4 The model + +> **AMENDED — read Amendment 1 first.** The single document below is the +> pre-layering form. It splits into `library/iam/model.yaml` (everything +> shown here EXCEPT the `source.rdb` and `index.lookup` children) and +> `library/iam/db.yaml` (an `overlay: true` redeclaration of each entity +> carrying only those two child kinds). The field/identity/relationship +> content is unchanged and still loads clean under `strict: true`; the +> split is where the children live, not what they are. + +`library/iam/model.yaml`. Verified to load clean under `strict: true`. + +```yaml +metadata: + package: metaobjects::iam + children: + - object.entity: + name: IamBase + abstract: true + description: Shared shape of every iam principal and definition — a stable uuid plus change timestamps. Junctions do not extend it; they are addressed by their participants. + children: + - field.uuid: { name: id, required: true } + - field.timestamp: { name: createdAt, autoSet: onCreate } + - field.timestamp: { name: updatedAt, autoSet: onUpdate } + + - object.entity: + name: User + extends: IamBase + children: + - source.rdb: { table: iam_user, role: primary } + - field.string: { name: username, required: true, maxLength: 64, filterable: true } + - field.string: { name: email, required: true, maxLength: 254, stringFormat: email, filterable: true } + - field.string: { name: displayName, maxLength: 120 } + - field.enum: { name: status, required: true, values: [invited, active, suspended, closed], default: active, filterable: true } + - field.timestamp: { name: emailVerifiedAt } + - field.timestamp: { name: lastSeenAt } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqUsername, fields: [username] } + - identity.secondary: { name: uqEmail, fields: [email] } + - relationship.association: { name: groups, objectRef: Group, cardinality: many, through: GroupMember } + - relationship.association: { name: roles, objectRef: Role, cardinality: many, through: UserRole } + + - object.entity: + name: GroupType + extends: IamBase + children: + - source.rdb: { table: iam_group_type, role: primary } + - field.string: { name: key, required: true, maxLength: 64 } + - field.string: { name: name, required: true, maxLength: 120 } + - field.string: { name: description, maxLength: 500 } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + + - object.entity: + name: Group + extends: IamBase + children: + - source.rdb: { table: iam_group, role: primary } + - field.uuid: { name: groupTypeId, required: true } + - field.uuid: { name: parentId } + - field.string: { name: key, required: true, maxLength: 64 } + - field.string: { name: name, required: true, maxLength: 120, filterable: true } + - field.string: { name: description, maxLength: 500 } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict } + - identity.reference: { name: fkParent, fields: [parentId], references: Group, onDelete: restrict } + - index.lookup: { name: ixParent, fields: [parentId] } + + - object.entity: + name: Role + extends: IamBase + children: + - source.rdb: { table: iam_role, role: primary } + - field.string: { name: key, required: true, maxLength: 64 } + - field.string: { name: name, required: true, maxLength: 120 } + - field.string: { name: description, maxLength: 500 } + - field.uuid: { name: groupTypeId, description: "When set, this role may be held only within groups of this type; absent means grantable anywhere." } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict } + - relationship.association: { name: permissions, objectRef: Permission, cardinality: many, through: RolePermission } + + - object.entity: + name: Permission + extends: IamBase + children: + - source.rdb: { table: iam_permission, role: primary } + - field.string: { name: key, required: true, maxLength: 128, description: "Stable : key the application checks against." } + - field.string: { name: description, maxLength: 500 } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + + # ---- grant surface: every grant is a row, addressed by its participants ---- + + - object.entity: + name: GroupMember + children: + - source.rdb: { table: iam_group_member, role: primary } + - field.uuid: { name: userId, required: true } + - field.uuid: { name: groupId, required: true } + - field.timestamp: { name: joinedAt, autoSet: onCreate } + - identity.primary: { name: pk, fields: [userId, groupId], generation: assigned } + - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade } + - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade } + - index.lookup: { name: ixGroup, fields: [groupId] } + + - object.entity: + name: RolePermission + children: + - source.rdb: { table: iam_role_permission, role: primary } + - field.uuid: { name: roleId, required: true } + - field.uuid: { name: permissionId, required: true } + - identity.primary: { name: pk, fields: [roleId, permissionId], generation: assigned } + - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: cascade } + - identity.reference: { name: fkPermission, fields: [permissionId], references: Permission, onDelete: restrict } + - index.lookup: { name: ixPermission, fields: [permissionId] } + + - object.entity: + name: UserRole + description: A system-wide grant of a role to a user. + children: + - source.rdb: { table: iam_user_role, role: primary } + - field.uuid: { name: userId, required: true } + - field.uuid: { name: roleId, required: true } + - field.timestamp: { name: grantedAt, autoSet: onCreate } + - identity.primary: { name: pk, fields: [userId, roleId], generation: assigned } + - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade } + - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict } + - index.lookup: { name: ixRole, fields: [roleId] } + + - object.entity: + name: GroupMemberRole + description: A grant of a role to a user within one group. Three foreign keys, so it is not an M:N @through junction; it is read by explicit finders. + children: + - source.rdb: { table: iam_group_member_role, role: primary } + - field.uuid: { name: userId, required: true } + - field.uuid: { name: groupId, required: true } + - field.uuid: { name: roleId, required: true } + - field.timestamp: { name: grantedAt, autoSet: onCreate } + - identity.primary: { name: pk, fields: [userId, groupId, roleId], generation: assigned } + - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade } + - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade } + - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict } + - index.lookup: { name: ixGroupRole, fields: [groupId, roleId] } +``` + +The referential rule is one sentence, and is also a requirement: **deleting a +principal cascades its grants; deleting a definition still in use is refused; +deleting a role cascades only its own permission mapping.** + +### 7.5 The requirements + +Twenty-one entries. An L2 `accessControl` — *who may do what is answered from +stored grants, never from a name compared to a literal in code* — with +`identity`, `grouping`, `grants` and `decision` beneath it, and four flat +architectural claims. Two invariants the schema cannot express ship as +`partial` + `accepted`: **acyclic group nesting**, and **a role bound to a group +type is granted only in groups of that type**. + +The load-bearing architectural entries: + +```yaml +- requirement.architectural: + name: grantsAreRows + status: live + statement: A grant exists only as a stored row; nothing is granted by naming, position or convention. + counterexample: A superuser recognised by username. + implementedBy: [UserRole, GroupMemberRole, RolePermission, GroupMember] +- requirement.architectural: + name: noCredentialsOnUser + status: live + statement: A user row carries no authentication secret — no password, no hash, no knowledge-based question or answer. + counterexample: A password or secret-answer column on the user table. + description: Authentication is a separate capability with an entity per factor; this library is identity and authorization only. + implementedBy: [User] +- requirement.architectural: + name: principalDeletionRevokesGrants + status: live + statement: Deleting a user or group removes its grants; deleting a role or permission still in use is refused. + counterexample: A grant row pointing at a user who no longer exists. + implementedBy: [GroupMember, UserRole, GroupMemberRole, RolePermission] +- requirement.architectural: + name: stableIdentifiers + status: live + statement: Every principal and definition is addressed by a uuid that never changes; every grant by its participants. + counterexample: A group referenced by its display name. + implementedBy: [IamBase] +``` + +`noCredentialsOnUser` is what earns the library its keep with an agent: the one +thing every reader of a user table proposes adding, stated as a prohibition in +force, claimable, and rendered on `agent/requirements.md`. It is `architectural` +rather than `retired` — `retired` is chartered for a capability built here and +removed, and the library never built it. + +## 8. Phase 1 / Phase 2 + +> **AMENDED — read Amendment 1 first.** Items 1 and 2 are restated below to +> carry the layering; items 3–7 stand as written. Three items are added (1a, 2a, +> 2b) and one is added to item 7. + +**PHASE 1 IS SHIPPED (2026-09-13): items 1, 1a, 1b, 2, 2a, 2b, 2c, 3, 4, 5, 6, 7.** + +Three findings from building the second half, recorded because each contradicts something +this spec assumed: + +- **Item 2's "TS standalone-verify gate" was declared shipped and did not exist**, and + writing it found that **BOTH shipped libraries failed the requirement gate** — eleven + errors and two warnings, in metadata an adopter cannot fix. Every L4 in `ai` claimed + FIELDS (`ERR_REQUIREMENT_L4_NOT_OBJECT`), and both libraries wrote their concerns as + SIBLINGS of the L2 segment their own comments described them as children of, leaving + that L2 claiming nothing in its subtree. Loading clean and VERIFYING clean are different + claims; only the first was gated. Both ledgers are fixed as the model intends and the + gate now holds every library — and every future one, since it iterates + `knownLibraryTokens()` — to zero loader errors, zero loader warnings, zero gate + findings, zero lint findings, no unruled gaps, and every entity claimed by its own + ledger. +- **§6's "latent bug" in `trace-helper` was the WHOLE behaviour, not a corner of it.** The + short-name compare meant the generator never actually keyed on the shipped base: every + trace fixture in three ports declared its own `LlmCallBase` and passed. Fixing the + anchor required pointing all of them at the real library — which is the bypass ADR-0024 + named, closed rather than documented. +- **Library files carried an ambiguous source id** — the file's basename on disk, + `library:.yaml` when embedded. The collision guard needs to tell a library's + contribution from an adopter's, and an ejected copy is named after the library's own + files, so the id is now stable in every build and in all four ports. + +§8's table stands as the record of what each item covered. + +| # | Phase 1 | scope | +|---|---|---| +| 1 | `library.json` per library, embedded, declaring its **layers**; `knownLibraryPackages()` from manifests and accepting layer tokens (`iam`, `iam/db`, `iam/ui`); manifest-resolution test | 4 embeds, 4 code sites, 4 tests | +| 1a | **Layer selection in `librarySources()`** — today `REFS_BY_PACKAGE[pkg]` returns every ref under a package, so a bare `iam` would pull the db and ui layers too. Selection resolves a layer token to its manifest-declared refs, and `iam/db` implies `iam` | TS, then Java + Python + C# embeds | +| 1b | **`libraries` moves to `.metaobjects/config.json`** beside FR-023 `dependencies`, and OUT of `metaobjects.config.ts` — outright, no dual-read (§12 Q4: zero estate uses) | TS + Python | +| 2 | `library/iam/{model,db,requirements}.yaml`; `library/ai` **split** into `{model,db}.yaml` + `requirements.yaml`; per-port standalone strict-load test; TS standalone-verify gate | content cross-port, gates per port | +| 2a | **Conformance fixture: an overlay adds a `source.rdb`.** Documented in `CLAUDE.md`, gated nowhere, and now load-bearing in five ports | fixtures + all 5 ports | +| 2b | **Assertion: every core layer declares no source** — the inertness promise, resolved not trusted | TS (per library) | +| 2c | **`overlay: true` licenses an attribute override** (Amendment 2 ruling) — `ERR_MERGE_CONFLICT` fires only on an UNMARKED conflicting redeclaration. Flip `overlay-attr-last-writer-wins` to zero errors and `overlay-nested-requirement` likewise, and add a new fixture taking over the unmarked-conflict error branch so the coverage is moved, not deleted | 4 loaders (Kotlin inherits JVM) + 3 fixtures | +| 3 | Coverage-activation rule (§5.4) | TS | +| 4 | Catalog `kind: "library"` + `--probe` project block; one-namespace test; skill amendments + agent-context corpus regen **in the same commit** | TS | +| 5 | Implied-generator warnings; `GenContext.libraries`; `trace-helper` anchor from manifest | TS + Java + Python | +| 6 | `meta eject `; `ERR_LIBRARY_PACKAGE_COLLISION`; ownership refusal; `eject --list` per-node staleness | TS (+ Python for the refusal) | +| 7 | `docs/features/libraries.md` (leading with the layer model); `cli.md`; compat-policy clause for shipped-library shape; the `CLAUDE.md` pillar paragraph; **the `ai` split as a CHANGELOG breaking-ish note**; CHANGELOG | docs | + +**Phase 2:** third-party libraries over FR-023's `npm`/`python` transports; +`requires` edges library→library; loader-level collision/ownership errors in all +five ports; "live only if wired" in `verify`; Python/C# catalog rows and probe; +`migrate.scope.exclude`; composite-FK evidence; a runtime `can()` per port; an +authentication library; splitting `ai`'s concrete `LlmCall` if adopters ask. + +## 9. Compatibility + +No metamodel change, so `expected-registry.json`, `registry.json` and +`metamodelVersion` are untouched and no conformance corpus gains a port-matrix +row. What touches every port is bounded: the manifest embed, `knownPackages` +derivation, the standalone-load test, and the `trace-helper` anchor. Kotlin +inherits from the JVM loader. Everything else is TypeScript. + +The compatibility policy gains a clause for shipped-library metadata: +`stability: preview` is exempt from the shape promise; `stable` means +additive-only within a MINOR. + +## 10. Risks + +- **Shipping `iam` freezes a shape — but only for adopters who track it.** + Because copy-and-own is the expected mode (§3.4), a later change to `iam` + reaches only those who chose `libraries: ["iam"]` over ejecting. That is the + primary mitigation; `stability: preview` at ship and a promotion bar to + `stable` (one external estate running it with the drift gate enforced, the G3d + precedent) are the belt. +- **Opt-in means tables.** Nine for `iam`; an adopter wanting only `User` will + feel over-served. `--probe` makes it visible before commitment, overlay and + eject make it adaptable, and the design deliberately refuses optional + sub-components — a bundle of bundles is the rot the catalog design documented. +- **The `ai` concrete `LlmCall` wart persists**, documented in the manifest + rather than split, because splitting changes what existing `ai` adopters get. +- **A project that already has a ledger** sees its coverage denominator grow to + include the library's entities — all claimed by the library, so no new + warnings, but the summary numbers move. CHANGELOG it. +- **Requirements bind only to model nodes.** Runtime guarantees have no ledger + address and this FR refuses to invent one; they arrive as `partial`/`accepted` + notes and requirement-test stubs. If that proves too weak the fix is an ADR, + not an attribute. +- **Whether declared design measurably helps an agent is still n=1.** Building + ahead of that measurement is the maintainer's call; the design keeps the cost + of being wrong low — `preview`, no vocabulary, everything opt-in — and `iam` + gives FR-041 a concrete "more declared metadata" arm to measure. + +## 11. Candidate: a delete directive for overlays + +Overlay today can add and can override an attribute; it cannot **remove**. The +eight reserved structural keys are `name`, `package`, `extends`, `abstract`, +`overlay`, `isArray`, `children`, `value` — there is no removal semantic +anywhere, so an adopter tracking a library upstream cannot drop a field, +identity or requirement they do not want. They must eject. + +**Shape, if built.** It is a merge directive, the same class as `overlay: true`, +so it is a reserved structural key on the child being removed — not an +`@`-attribute. `delete` and `remove` are both free: no registered attribute in +any port uses either name. + +**The constraint that decides the semantics.** Overlay merge is deliberately +order-independent — ADR-0055 made it a deferred pass, and #188 established the +same property for super-resolution: the result is a pure function of the source +SET, not of load order. A naive delete breaks that. It is preserved by making +delete **absorbing**: present anywhere in the source set, the node is absent from +the result, and nothing can add it back. That keeps the merge a set operation. +The cost of absorbing semantics is that a library which later legitimately +reintroduces a member cannot reach an adopter who deleted it — which is the +correct outcome and should be stated rather than discovered. + +**The ledger interaction is a feature.** Deleting `User.email` dangles +`uniqueLogin`'s `implementedBy: [User.username, User.email]`, which is an ERROR +on a live requirement. The adopter must also overlay the requirement to say what +they now claim. Removing a capability forces you to amend the design that +promised it — which is precisely what the requirements pillar is for. + +**Why it is a candidate and not Phase 1.** A new reserved structural key is a +change to the canonical interchange format, so it lands in all five ports' +parsers and serializers and in the canonical body-key order, and it moves +`metamodelVersion` (post-1.0, that must be called out in the CHANGELOG). Against +that: once copy-and-own is the expected mode (§3.4), you delete by editing your +copy, and the directive serves only the track-upstream minority. Build it when +that minority turns out to be real — their existence is the evidence that +justifies the vocabulary. + +## 12. Remaining open questions + +> **AMENDED — read Amendments 1 and 2 first. NOTHING IN THIS SECTION IS STILL +> OPEN.** Q2, Q3 and Q4 are ruled in Amendment 1. Q1 is ANSWERED in Amendment 2 +> — yes structurally, by `fixtures/conformance/overlay-nested-requirement`, +> green in all five ports — with the attribute-override conflict it exposed +> ruled there too. + +1. **Evidence, and the one that gates "shippable": does `overlay: true` merge on + a NESTED `requirement.*` node?** If not, an adopter cannot disagree with a + library requirement without ejecting the whole ledger and §5.5 collapses to + eject-only. One conformance fixture settles it. An attempt to settle it with a + throwaway loader harness failed for an unrelated reason — requirement + vocabulary needs the CLI's provider bootstrap, not the bare loader — so it + wants a real fixture rather than a spot check. +2. **`migrate.scope` has no `exclude`**, so dropping one library table from a + migration means enumerating everything else. Add it (config-only, TS, cheap) + in Phase 1, or leave it to author discipline? +3. **Composite `identity.reference`** — `GroupMemberRole(userId, groupId)` → + `GroupMember` would put "a scoped grant requires membership" in the schema. + Unverified across five ports' DDL/ORM paths; shipped as `partial`/`accepted` + instead. Worth a persistence-conformance case? +4. **Should `libraries` move to the neutral `.metaobjects/config.json`** beside + `dependencies`, so TS and Python read one key and the JVM/C# ports can follow? + Not blocking; it is the consistency debt the cross-port sources decision noted. diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index 2b04bfcb9..1b3ee0ff9 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -7,7 +7,7 @@ "metamodel": "1.0" }, "counts": { - "fixtures": 322, + "fixtures": 325, "corpora": 22, "baseTypes": 14 }, diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md index 82ed5693b..5f1238aff 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -351,7 +351,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on THREE of five ports an existing project emits none**. C# and Python have a real default suite and get it by upgrading; TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, so `meta init` scaffolding `namesFile()` covers only a project initialized at 1.0 and upgrading the package never edits a config written earlier. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index 3a05b86ca..8b3265072 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -71,10 +71,10 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove migration script, a body-to-column map (drift signature 11). Every port emits a per-object names artifact from the declaration, so a literal is a second source of truth even when it agrees with the naming strategy today. A typed ORM handle in its place is correct. **Check - the artifact is emitted at all before scoring the literals: on TypeScript and the JVM the - generator list in the config IS the complete list, so an existing project emits none and the - un-wired generator is the finding FIRST** (C# and Python have a real default suite and get it - by upgrading). **This entry is the physical-name INSTANCE of signature 11.** The rule that + the artifact is emitted at all before scoring the literals: on every port the selection IS + the complete list — the config's on TypeScript and the JVM, `--generators` on C# and Python — + so an existing project emits none and the un-wired generator is the finding FIRST** (no port + ships a default suite; ADR-0034 Amendment 2). **This entry is the physical-name INSTANCE of signature 11.** The rule that generates the rest — and the reason an enum member compared as a bare string is NOT one — is the Cross-cutting entry below. - **`@kind` = `view` / `materializedView`** — hunt hand-written SQL views where an authored diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 86bfc2dfd..c40bce3c3 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -46,22 +46,28 @@ is, this once:** **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: -1. **Search the vocabulary** — `meta types `, or `meta types --all +1. **Check whether the design already ships.** `meta gen --list --format json` + carries `kind: "library"` rows — declared design MetaObjects ships, each with a + `useWhen`. If one matches the capability you are about to declare, opt in and adapt + rather than author it from scratch: you inherit its entities, its requirements, and + the rulings recorded with them. The bare name is the core layer (sourceless — no + tables, no generated code); `/db` is the separate opt-in that adds the schema. +2. **Search the vocabulary** — `meta types `, or `meta types --all ` to search by behavior. There are field subtypes, relationships, projections, origins, identities, sources, and attributes you may not know exist. Find the construct that models it. Add `--detail` for one construct's valid `@attrs`, or `--format json` for the same answer as one machine-readable document — that form carries every match (`--limit` never truncates it) with each attr's `allowedValues`, so you read the accepted values rather than guessing them. -2. **Declare it and generate** — then *consume* the generated query/type/route; +3. **Declare it and generate** — then *consume* the generated query/type/route; never reimplement it alongside. -3. **If the model is right but the generated OUTPUT is wrong, change your generator.** +4. **If the model is right but the generated OUTPUT is wrong, change your generator.** Naming, file layout, imports, framework, signatures are generator concerns, not reasons to hand-write. The generators are in *your* repo and are yours to edit — a standing rule not to change the MetaObjects repo does not reach them; they are a different repository. Editing one is ordinary work, not an escalation. (See `metaobjects-codegen` → "Your generators are yours".) -4. **Only if no construct can express it** — and you have actually looked — +5. **Only if no construct can express it** — and you have actually looked — hand-write it, wired to generated types. Business algorithms, external integrations, and bespoke interactions are legitimately hand-written; CRUD, validation, finders, relationships, and derived/aggregate data are not. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md index 247af390d..5f3350cf2 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -124,19 +124,72 @@ implementing the port's generator interface elsewhere. Your language reference h mechanism; see also "The commands and config keys that implement the steps above differ per port" below. -## Selecting generators by stable name +## Selecting generators — NOTHING is generated until you choose it + +**`meta init` wires no generators, and no port ships a default suite.** A fresh +scaffold has `generators: []` and an empty `codegen/generators/`; `--generators` is +required on the C# and Python CLIs; Java has never had a default set. Choosing what an +application needs is a judgment over its purpose and stack, and the tool's job is to +make that choice cheap and truthful, not to make it for you. + +Each generator has a **stable name** (kebab-case) that is the same in every port and +surfaces in diagnostics — reference generators by that name, never by inlining what +they emit. + +### The procedure + +1. **Read the app's purpose and stack.** What is it for; what does it already use. +2. **`meta gen --list --format json --probe`** — the catalog. Every generator with its + `layer`, `framework`, what it emits, what it requires, what it costs to install, + and — with `--probe`, which constructs each generator and dry-runs it against YOUR + model — how many files each would actually emit. +3. **Check the libraries before you choose generators.** The same catalog carries + `kind: "library"` rows — declared design MetaObjects ships, each with a `useWhen`. + If one matches a capability you are about to model, **opt in and adapt rather than + author**: you inherit its entities, its requirements, and the rulings recorded with + them. Layers are how much of it you take. The bare name is the CORE layer — the + model and its ledger, sourceless, so it adds **no tables and no generated code**; + `/db` is the separate opt-in that proposes the schema. Opt in with + `"libraries": ["iam", "iam/db"]` in `.metaobjects/config.json`. +4. **Choose by `layer`.** Satisfy every `requires`. Take what `wouldEmit > 0` says your + model is already asking for. + - Pick **ONE** framework on the `api` layer: `routes` and `routes-hono` are + alternatives, and wiring both silently produces two complete HTTP surfaces. + - Do **NOT** apply that rule to `client`. `@metaobjectsdev/tanstack` peers on + `react`, so a form generator plus the TanStack hook/grid generators is the + intended composition, not a conflict. +5. **`meta eject --format json`** — copies each into `codegen/generators/` + (yours to edit), and reports the import line, the entry to add to `generators`, one + consolidated install command, and any config keys those generators read. +6. **`meta gen`** — read its warnings, then typecheck. + +A library row also carries `provides` (what is in the box) and, under `--probe`, a +`project` block: which layers you selected, how many tables and requirements they +added here, which of your entities `extends` into it, and any generator the library +implies that you have not wired. + +### The six layers, and what picks them + +| layer | what it is | chosen by | +|---|---|---| +| `model` | entity/DTO/value-object modules and the constants beside them | app shape | +| `persistence` | how rows are read and written | app shape | +| `api` | the HTTP surface — pick one framework | app shape | +| `client` | the browser tier | app shape | +| `docs` | on by default; the canonical door is `meta docs` | — | +| `capability` | prompts, parsers, payloads, traces, test stubs | **`--probe`** | + +At intent level: a **headless data service** is `model` + `persistence`; an **HTTP API** +adds `api` with one framework; an **admin UI** adds `client`. Members come from the live +catalog, never from a list in prose — a list here would go stale the day a generator is +added, and `--probe` cannot, because it runs the generators rather than describing them. -Codegen is a set of named generators you opt into. Each generator has a **stable -name** (kebab-case) that surfaces in diagnostics — reference generators by that -name, never by inlining what they emit. Typical generators cover: the entity -type/model, the DB table/schema, query/finder helpers, REST routes, client -form/grid/hook artifacts, filter + sort allowlists, payload value-objects, and -parsers for a responding `template.prompt` (one carrying `@responseRef`). You -enable the subset your project needs; an abstract entity never emits -instance/write artifacts regardless. +`capability` looking like one large bucket is the point: you are not meant to choose +inside it by reading labels. You declared a `template.prompt`, or a +`requirement.functional`, or a stored-proc source — `--probe` reports the file count +that follows, for your model. -Per-entity opt-outs exist (e.g. skipping client-side artifacts for a given -entity) and are set as attributes on the entity in metadata, not in code. +An abstract entity never emits instance/write artifacts regardless of what is wired. ## A dependency's metadata is load-only by default — codegen excludes it @@ -239,8 +292,9 @@ the data access too. **`@unmanaged: true`** (view or table); migrate/verify then never touch it. `@sql` and `@unmanaged` are mutually exclusive. -`meta gen --list` prints every generator by stable name; the `generators` array in -`metaobjects.config.ts` is where you opt each one in or out. +`meta gen --list` prints every generator by stable name (add `--probe` for a file count +against your own model); the `generators` array in `metaobjects.config.ts` is where you +opt each one in. It starts empty. ### Adopting onto existing code — make codegen match the code, not the code match codegen @@ -435,7 +489,8 @@ generator sets are **closed built-in registries** — `--generators` *selects* f ships, and there is no seam to register a `Generator` of your own. (Python's `--provider module:symbol` registers **metamodel vocabulary**, not a generator; do not reach for it here.) Use `--template-spec ` — plus `--templates ` on Python or -`--template-root ` on C# — and your entries are appended to the default suite. Worked +`--template-root ` on C# — and your entries are appended to your `--generators` +selection. Worked examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`. **The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both @@ -490,21 +545,21 @@ everywhere — **each physical name is spelled once, and generated code referenc | Kotlin | `Names.kt` | `ProgramNames.CREATED_AT_COLUMN` | | Python | `_names.py` | `PROGRAM_CREATED_AT_COLUMN` | -**Check that it is actually wired before you reference it — on three of five ports an -EXISTING project emits none.** "In the default suite" and "what a fresh scaffold writes" -are different facts, and only C# and Python have the first: +**Check that it is actually wired before you reference it — on ALL FIVE ports a project +emits none until it asks for it.** ADR-0034 Amendment 2 made codegen opt-in everywhere: no +port ships a default suite, so there is no port on which upgrading the package starts +emitting this artifact. "Wired" is the only fact there is. -| Port | Where the suite is decided | An existing project upgrading gets it? | +| Port | Where the selection is declared | An existing project upgrading gets it? | |---|---|---| -| C# | `GenCommand.DefaultGeneratorNames` — a real default | **Yes**, with no edit | -| Python | `cli.py` `_default_generators()` — a real default | **Yes**, with no edit | -| TypeScript | `metaobjects.config.ts` `generators: [...]` — **the complete list; there is no default suite** | **No** — add `namesFile()` | +| C# | `--generators ` on `dotnet meta gen` — required, no default | **No** — name `names` | +| Python | `--generators ` on `metaobjects gen` — required, no default | **No** — name `names` | +| TypeScript | `metaobjects.config.ts` `generators: [...]` — the complete list | **No** — add `namesFile()` | | Java / Kotlin | the pom's `` — the complete list | **No** — add `SpringNamesGenerator` / `KotlinNamesGenerator` | -TypeScript's `meta init` scaffolds `namesFile()`, so a project *initialized* at 1.0 has it; -a project initialized earlier has the config `meta init` wrote then, and upgrading the -package never edits a config. So on TypeScript the artifact is opt-in exactly as it is on -the JVM — the scaffold is not a default. To wire it into an existing TS project: +`meta init` scaffolds `generators: []` and an empty `codegen/generators/`, so even a +project *initialized* at 1.0 has to choose this one — the scaffold is deliberately not a +default by another name. To wire it into a TS project: ```ts import { namesFile } from "./codegen/generators/names.js"; // after `meta eject names` diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md index 722a14971..adb6a8c32 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md @@ -64,14 +64,14 @@ handle: raw SQL, a string-keyed query builder, and a hand-written repository on whose generated model carries no persistence binding at all (Java, Python). Your server reference names the handle and the artifact for this stack. -**First check the artifact exists — on TypeScript and the JVM it is opt-in, and an -existing project almost certainly has none.** C# and Python emit it from a real default -suite, so upgrading is enough. TypeScript's `generators: [...]` and the JVM's -`` are each the COMPLETE list: `meta init` scaffolds `namesFile()` for a -project initialized at 1.0, and upgrading the package never edits a config that was -written earlier. Look for `.names.ts` / `Names` in the generated output -before you write `ProgramNames.fields.x.column` against it; if it is not there, wiring the -generator is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL +**First check the artifact exists — it is opt-in on EVERY port, and an existing project +almost certainly has none.** Every port's generator list is now the COMPLETE list: C# and +Python dropped their default suites (ADR-0034 Amendment 2), TypeScript's `generators: []` +starts empty, and the JVM's `` never had a default. Upgrading the package +never edits a config that was written earlier, so `names` arrives only when someone wires +it. Look for `.names.ts` / `Names` in the generated output before you +write `ProgramNames.fields.x.column` against it; if it is not there, wiring the generator +is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL with literal names because "there is no constant" is the loop this closes. ## The REST contract diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/AGENTS.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/AGENTS.md index 3cb2c2ffc..fb77b6b14 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/AGENTS.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/AGENTS.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/CLAUDE.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/CLAUDE.md index 3cb2c2ffc..fb77b6b14 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/CLAUDE.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.metaobjects/CLAUDE.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md index 82ed5693b..5f1238aff 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -351,7 +351,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on THREE of five ports an existing project emits none**. C# and Python have a real default suite and get it by upgrading; TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, so `meta init` scaffolding `namesFile()` covers only a project initialized at 1.0 and upgrading the package never edits a config written earlier. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index 3a05b86ca..8b3265072 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -71,10 +71,10 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove migration script, a body-to-column map (drift signature 11). Every port emits a per-object names artifact from the declaration, so a literal is a second source of truth even when it agrees with the naming strategy today. A typed ORM handle in its place is correct. **Check - the artifact is emitted at all before scoring the literals: on TypeScript and the JVM the - generator list in the config IS the complete list, so an existing project emits none and the - un-wired generator is the finding FIRST** (C# and Python have a real default suite and get it - by upgrading). **This entry is the physical-name INSTANCE of signature 11.** The rule that + the artifact is emitted at all before scoring the literals: on every port the selection IS + the complete list — the config's on TypeScript and the JVM, `--generators` on C# and Python — + so an existing project emits none and the un-wired generator is the finding FIRST** (no port + ships a default suite; ADR-0034 Amendment 2). **This entry is the physical-name INSTANCE of signature 11.** The rule that generates the rest — and the reason an enum member compared as a bare string is NOT one — is the Cross-cutting entry below. - **`@kind` = `view` / `materializedView`** — hunt hand-written SQL views where an authored diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md index 86bfc2dfd..c40bce3c3 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -46,22 +46,28 @@ is, this once:** **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: -1. **Search the vocabulary** — `meta types `, or `meta types --all +1. **Check whether the design already ships.** `meta gen --list --format json` + carries `kind: "library"` rows — declared design MetaObjects ships, each with a + `useWhen`. If one matches the capability you are about to declare, opt in and adapt + rather than author it from scratch: you inherit its entities, its requirements, and + the rulings recorded with them. The bare name is the core layer (sourceless — no + tables, no generated code); `/db` is the separate opt-in that adds the schema. +2. **Search the vocabulary** — `meta types `, or `meta types --all ` to search by behavior. There are field subtypes, relationships, projections, origins, identities, sources, and attributes you may not know exist. Find the construct that models it. Add `--detail` for one construct's valid `@attrs`, or `--format json` for the same answer as one machine-readable document — that form carries every match (`--limit` never truncates it) with each attr's `allowedValues`, so you read the accepted values rather than guessing them. -2. **Declare it and generate** — then *consume* the generated query/type/route; +3. **Declare it and generate** — then *consume* the generated query/type/route; never reimplement it alongside. -3. **If the model is right but the generated OUTPUT is wrong, change your generator.** +4. **If the model is right but the generated OUTPUT is wrong, change your generator.** Naming, file layout, imports, framework, signatures are generator concerns, not reasons to hand-write. The generators are in *your* repo and are yours to edit — a standing rule not to change the MetaObjects repo does not reach them; they are a different repository. Editing one is ordinary work, not an escalation. (See `metaobjects-codegen` → "Your generators are yours".) -4. **Only if no construct can express it** — and you have actually looked — +5. **Only if no construct can express it** — and you have actually looked — hand-write it, wired to generated types. Business algorithms, external integrations, and bespoke interactions are legitimately hand-written; CRUD, validation, finders, relationships, and derived/aggregate data are not. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md index 247af390d..5f3350cf2 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -124,19 +124,72 @@ implementing the port's generator interface elsewhere. Your language reference h mechanism; see also "The commands and config keys that implement the steps above differ per port" below. -## Selecting generators by stable name +## Selecting generators — NOTHING is generated until you choose it + +**`meta init` wires no generators, and no port ships a default suite.** A fresh +scaffold has `generators: []` and an empty `codegen/generators/`; `--generators` is +required on the C# and Python CLIs; Java has never had a default set. Choosing what an +application needs is a judgment over its purpose and stack, and the tool's job is to +make that choice cheap and truthful, not to make it for you. + +Each generator has a **stable name** (kebab-case) that is the same in every port and +surfaces in diagnostics — reference generators by that name, never by inlining what +they emit. + +### The procedure + +1. **Read the app's purpose and stack.** What is it for; what does it already use. +2. **`meta gen --list --format json --probe`** — the catalog. Every generator with its + `layer`, `framework`, what it emits, what it requires, what it costs to install, + and — with `--probe`, which constructs each generator and dry-runs it against YOUR + model — how many files each would actually emit. +3. **Check the libraries before you choose generators.** The same catalog carries + `kind: "library"` rows — declared design MetaObjects ships, each with a `useWhen`. + If one matches a capability you are about to model, **opt in and adapt rather than + author**: you inherit its entities, its requirements, and the rulings recorded with + them. Layers are how much of it you take. The bare name is the CORE layer — the + model and its ledger, sourceless, so it adds **no tables and no generated code**; + `/db` is the separate opt-in that proposes the schema. Opt in with + `"libraries": ["iam", "iam/db"]` in `.metaobjects/config.json`. +4. **Choose by `layer`.** Satisfy every `requires`. Take what `wouldEmit > 0` says your + model is already asking for. + - Pick **ONE** framework on the `api` layer: `routes` and `routes-hono` are + alternatives, and wiring both silently produces two complete HTTP surfaces. + - Do **NOT** apply that rule to `client`. `@metaobjectsdev/tanstack` peers on + `react`, so a form generator plus the TanStack hook/grid generators is the + intended composition, not a conflict. +5. **`meta eject --format json`** — copies each into `codegen/generators/` + (yours to edit), and reports the import line, the entry to add to `generators`, one + consolidated install command, and any config keys those generators read. +6. **`meta gen`** — read its warnings, then typecheck. + +A library row also carries `provides` (what is in the box) and, under `--probe`, a +`project` block: which layers you selected, how many tables and requirements they +added here, which of your entities `extends` into it, and any generator the library +implies that you have not wired. + +### The six layers, and what picks them + +| layer | what it is | chosen by | +|---|---|---| +| `model` | entity/DTO/value-object modules and the constants beside them | app shape | +| `persistence` | how rows are read and written | app shape | +| `api` | the HTTP surface — pick one framework | app shape | +| `client` | the browser tier | app shape | +| `docs` | on by default; the canonical door is `meta docs` | — | +| `capability` | prompts, parsers, payloads, traces, test stubs | **`--probe`** | + +At intent level: a **headless data service** is `model` + `persistence`; an **HTTP API** +adds `api` with one framework; an **admin UI** adds `client`. Members come from the live +catalog, never from a list in prose — a list here would go stale the day a generator is +added, and `--probe` cannot, because it runs the generators rather than describing them. -Codegen is a set of named generators you opt into. Each generator has a **stable -name** (kebab-case) that surfaces in diagnostics — reference generators by that -name, never by inlining what they emit. Typical generators cover: the entity -type/model, the DB table/schema, query/finder helpers, REST routes, client -form/grid/hook artifacts, filter + sort allowlists, payload value-objects, and -parsers for a responding `template.prompt` (one carrying `@responseRef`). You -enable the subset your project needs; an abstract entity never emits -instance/write artifacts regardless. +`capability` looking like one large bucket is the point: you are not meant to choose +inside it by reading labels. You declared a `template.prompt`, or a +`requirement.functional`, or a stored-proc source — `--probe` reports the file count +that follows, for your model. -Per-entity opt-outs exist (e.g. skipping client-side artifacts for a given -entity) and are set as attributes on the entity in metadata, not in code. +An abstract entity never emits instance/write artifacts regardless of what is wired. ## A dependency's metadata is load-only by default — codegen excludes it @@ -239,8 +292,9 @@ the data access too. **`@unmanaged: true`** (view or table); migrate/verify then never touch it. `@sql` and `@unmanaged` are mutually exclusive. -`meta gen --list` prints every generator by stable name; the `generators` array in -`metaobjects.config.ts` is where you opt each one in or out. +`meta gen --list` prints every generator by stable name (add `--probe` for a file count +against your own model); the `generators` array in `metaobjects.config.ts` is where you +opt each one in. It starts empty. ### Adopting onto existing code — make codegen match the code, not the code match codegen @@ -435,7 +489,8 @@ generator sets are **closed built-in registries** — `--generators` *selects* f ships, and there is no seam to register a `Generator` of your own. (Python's `--provider module:symbol` registers **metamodel vocabulary**, not a generator; do not reach for it here.) Use `--template-spec ` — plus `--templates ` on Python or -`--template-root ` on C# — and your entries are appended to the default suite. Worked +`--template-root ` on C# — and your entries are appended to your `--generators` +selection. Worked examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`. **The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both @@ -490,21 +545,21 @@ everywhere — **each physical name is spelled once, and generated code referenc | Kotlin | `Names.kt` | `ProgramNames.CREATED_AT_COLUMN` | | Python | `_names.py` | `PROGRAM_CREATED_AT_COLUMN` | -**Check that it is actually wired before you reference it — on three of five ports an -EXISTING project emits none.** "In the default suite" and "what a fresh scaffold writes" -are different facts, and only C# and Python have the first: +**Check that it is actually wired before you reference it — on ALL FIVE ports a project +emits none until it asks for it.** ADR-0034 Amendment 2 made codegen opt-in everywhere: no +port ships a default suite, so there is no port on which upgrading the package starts +emitting this artifact. "Wired" is the only fact there is. -| Port | Where the suite is decided | An existing project upgrading gets it? | +| Port | Where the selection is declared | An existing project upgrading gets it? | |---|---|---| -| C# | `GenCommand.DefaultGeneratorNames` — a real default | **Yes**, with no edit | -| Python | `cli.py` `_default_generators()` — a real default | **Yes**, with no edit | -| TypeScript | `metaobjects.config.ts` `generators: [...]` — **the complete list; there is no default suite** | **No** — add `namesFile()` | +| C# | `--generators ` on `dotnet meta gen` — required, no default | **No** — name `names` | +| Python | `--generators ` on `metaobjects gen` — required, no default | **No** — name `names` | +| TypeScript | `metaobjects.config.ts` `generators: [...]` — the complete list | **No** — add `namesFile()` | | Java / Kotlin | the pom's `` — the complete list | **No** — add `SpringNamesGenerator` / `KotlinNamesGenerator` | -TypeScript's `meta init` scaffolds `namesFile()`, so a project *initialized* at 1.0 has it; -a project initialized earlier has the config `meta init` wrote then, and upgrading the -package never edits a config. So on TypeScript the artifact is opt-in exactly as it is on -the JVM — the scaffold is not a default. To wire it into an existing TS project: +`meta init` scaffolds `generators: []` and an empty `codegen/generators/`, so even a +project *initialized* at 1.0 has to choose this one — the scaffold is deliberately not a +default by another name. To wire it into a TS project: ```ts import { namesFile } from "./codegen/generators/names.js"; // after `meta eject names` diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md index 722a14971..adb6a8c32 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md @@ -64,14 +64,14 @@ handle: raw SQL, a string-keyed query builder, and a hand-written repository on whose generated model carries no persistence binding at all (Java, Python). Your server reference names the handle and the artifact for this stack. -**First check the artifact exists — on TypeScript and the JVM it is opt-in, and an -existing project almost certainly has none.** C# and Python emit it from a real default -suite, so upgrading is enough. TypeScript's `generators: [...]` and the JVM's -`` are each the COMPLETE list: `meta init` scaffolds `namesFile()` for a -project initialized at 1.0, and upgrading the package never edits a config that was -written earlier. Look for `.names.ts` / `Names` in the generated output -before you write `ProgramNames.fields.x.column` against it; if it is not there, wiring the -generator is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL +**First check the artifact exists — it is opt-in on EVERY port, and an existing project +almost certainly has none.** Every port's generator list is now the COMPLETE list: C# and +Python dropped their default suites (ADR-0034 Amendment 2), TypeScript's `generators: []` +starts empty, and the JVM's `` never had a default. Upgrading the package +never edits a config that was written earlier, so `names` arrives only when someone wires +it. Look for `.names.ts` / `Names` in the generated output before you +write `ProgramNames.fields.x.column` against it; if it is not there, wiring the generator +is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL with literal names because "there is no constant" is the loop this closes. ## The REST contract diff --git a/fixtures/agent-context-conformance/java-react/expected/.metaobjects/AGENTS.md b/fixtures/agent-context-conformance/java-react/expected/.metaobjects/AGENTS.md index d2d5f7e48..9de479b32 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.metaobjects/AGENTS.md +++ b/fixtures/agent-context-conformance/java-react/expected/.metaobjects/AGENTS.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/java-react/expected/.metaobjects/CLAUDE.md b/fixtures/agent-context-conformance/java-react/expected/.metaobjects/CLAUDE.md index d2d5f7e48..9de479b32 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.metaobjects/CLAUDE.md +++ b/fixtures/agent-context-conformance/java-react/expected/.metaobjects/CLAUDE.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md index 82ed5693b..5f1238aff 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -351,7 +351,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on THREE of five ports an existing project emits none**. C# and Python have a real default suite and get it by upgrading; TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, so `meta init` scaffolding `namesFile()` covers only a project initialized at 1.0 and upgrading the package never edits a config written earlier. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index 3a05b86ca..8b3265072 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -71,10 +71,10 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove migration script, a body-to-column map (drift signature 11). Every port emits a per-object names artifact from the declaration, so a literal is a second source of truth even when it agrees with the naming strategy today. A typed ORM handle in its place is correct. **Check - the artifact is emitted at all before scoring the literals: on TypeScript and the JVM the - generator list in the config IS the complete list, so an existing project emits none and the - un-wired generator is the finding FIRST** (C# and Python have a real default suite and get it - by upgrading). **This entry is the physical-name INSTANCE of signature 11.** The rule that + the artifact is emitted at all before scoring the literals: on every port the selection IS + the complete list — the config's on TypeScript and the JVM, `--generators` on C# and Python — + so an existing project emits none and the un-wired generator is the finding FIRST** (no port + ships a default suite; ADR-0034 Amendment 2). **This entry is the physical-name INSTANCE of signature 11.** The rule that generates the rest — and the reason an enum member compared as a bare string is NOT one — is the Cross-cutting entry below. - **`@kind` = `view` / `materializedView`** — hunt hand-written SQL views where an authored diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md index 7d9bc1d30..7f9450aeb 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md @@ -52,7 +52,7 @@ regardless of server language — see the migration reference. | `def get_all_` / `def create_` / `def update_` / `def delete_` in non-generated files | hand-rolled CRUD — compare to the generated router | | `# keep in sync with` / `# mirrors the` | second-source-of-truth comment — always a finding | | `try: ... except KeyError` / `?? ''` around format strings in prompt code | silent-degradation hack around a prompt payload — flag it | -| a table/column string in the repository implementation (SQLAlchemy Core, asyncpg/psycopg SQL) | second spelling of a declared physical name — reference `_names.py` (`AUTHOR_SOURCE_PRIMARY_TABLE` / `AUTHOR__COLUMN`, default suite); no typed handle exists on this port, so that is never the reason to waive it | +| a table/column string in the repository implementation (SQLAlchemy Core, asyncpg/psycopg SQL) | second spelling of a declared physical name — reference `_names.py` (`AUTHOR_SOURCE_PRIMARY_TABLE` / `AUTHOR__COLUMN`, emitted when `names` is named in `--generators`); no typed handle exists on this port, so that is never the reason to waive it | --- @@ -66,8 +66,8 @@ seam to register a generator of your own. (`--provider module:symbol` registers **So do not score a Python project down for "not owning its generators", and do not recommend writing one.** The customization path here is the **declarative template**: -`metaobjects gen --template-spec --templates `, whose entries append to the -default suite. A finding of the form "the built-ins do not emit the shape this project +`metaobjects gen --template-spec --templates `, whose entries append to your +`--generators` selection. A finding of the form "the built-ins do not emit the shape this project needs" resolves to a template-spec, not to generator code. Worked example with the full JSON: `docs/ports/python.md`. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md index 86bfc2dfd..c40bce3c3 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -46,22 +46,28 @@ is, this once:** **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: -1. **Search the vocabulary** — `meta types `, or `meta types --all +1. **Check whether the design already ships.** `meta gen --list --format json` + carries `kind: "library"` rows — declared design MetaObjects ships, each with a + `useWhen`. If one matches the capability you are about to declare, opt in and adapt + rather than author it from scratch: you inherit its entities, its requirements, and + the rulings recorded with them. The bare name is the core layer (sourceless — no + tables, no generated code); `/db` is the separate opt-in that adds the schema. +2. **Search the vocabulary** — `meta types `, or `meta types --all ` to search by behavior. There are field subtypes, relationships, projections, origins, identities, sources, and attributes you may not know exist. Find the construct that models it. Add `--detail` for one construct's valid `@attrs`, or `--format json` for the same answer as one machine-readable document — that form carries every match (`--limit` never truncates it) with each attr's `allowedValues`, so you read the accepted values rather than guessing them. -2. **Declare it and generate** — then *consume* the generated query/type/route; +3. **Declare it and generate** — then *consume* the generated query/type/route; never reimplement it alongside. -3. **If the model is right but the generated OUTPUT is wrong, change your generator.** +4. **If the model is right but the generated OUTPUT is wrong, change your generator.** Naming, file layout, imports, framework, signatures are generator concerns, not reasons to hand-write. The generators are in *your* repo and are yours to edit — a standing rule not to change the MetaObjects repo does not reach them; they are a different repository. Editing one is ordinary work, not an escalation. (See `metaobjects-codegen` → "Your generators are yours".) -4. **Only if no construct can express it** — and you have actually looked — +5. **Only if no construct can express it** — and you have actually looked — hand-write it, wired to generated types. Business algorithms, external integrations, and bespoke interactions are legitimately hand-written; CRUD, validation, finders, relationships, and derived/aggregate data are not. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md index 247af390d..5f3350cf2 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -124,19 +124,72 @@ implementing the port's generator interface elsewhere. Your language reference h mechanism; see also "The commands and config keys that implement the steps above differ per port" below. -## Selecting generators by stable name +## Selecting generators — NOTHING is generated until you choose it + +**`meta init` wires no generators, and no port ships a default suite.** A fresh +scaffold has `generators: []` and an empty `codegen/generators/`; `--generators` is +required on the C# and Python CLIs; Java has never had a default set. Choosing what an +application needs is a judgment over its purpose and stack, and the tool's job is to +make that choice cheap and truthful, not to make it for you. + +Each generator has a **stable name** (kebab-case) that is the same in every port and +surfaces in diagnostics — reference generators by that name, never by inlining what +they emit. + +### The procedure + +1. **Read the app's purpose and stack.** What is it for; what does it already use. +2. **`meta gen --list --format json --probe`** — the catalog. Every generator with its + `layer`, `framework`, what it emits, what it requires, what it costs to install, + and — with `--probe`, which constructs each generator and dry-runs it against YOUR + model — how many files each would actually emit. +3. **Check the libraries before you choose generators.** The same catalog carries + `kind: "library"` rows — declared design MetaObjects ships, each with a `useWhen`. + If one matches a capability you are about to model, **opt in and adapt rather than + author**: you inherit its entities, its requirements, and the rulings recorded with + them. Layers are how much of it you take. The bare name is the CORE layer — the + model and its ledger, sourceless, so it adds **no tables and no generated code**; + `/db` is the separate opt-in that proposes the schema. Opt in with + `"libraries": ["iam", "iam/db"]` in `.metaobjects/config.json`. +4. **Choose by `layer`.** Satisfy every `requires`. Take what `wouldEmit > 0` says your + model is already asking for. + - Pick **ONE** framework on the `api` layer: `routes` and `routes-hono` are + alternatives, and wiring both silently produces two complete HTTP surfaces. + - Do **NOT** apply that rule to `client`. `@metaobjectsdev/tanstack` peers on + `react`, so a form generator plus the TanStack hook/grid generators is the + intended composition, not a conflict. +5. **`meta eject --format json`** — copies each into `codegen/generators/` + (yours to edit), and reports the import line, the entry to add to `generators`, one + consolidated install command, and any config keys those generators read. +6. **`meta gen`** — read its warnings, then typecheck. + +A library row also carries `provides` (what is in the box) and, under `--probe`, a +`project` block: which layers you selected, how many tables and requirements they +added here, which of your entities `extends` into it, and any generator the library +implies that you have not wired. + +### The six layers, and what picks them + +| layer | what it is | chosen by | +|---|---|---| +| `model` | entity/DTO/value-object modules and the constants beside them | app shape | +| `persistence` | how rows are read and written | app shape | +| `api` | the HTTP surface — pick one framework | app shape | +| `client` | the browser tier | app shape | +| `docs` | on by default; the canonical door is `meta docs` | — | +| `capability` | prompts, parsers, payloads, traces, test stubs | **`--probe`** | + +At intent level: a **headless data service** is `model` + `persistence`; an **HTTP API** +adds `api` with one framework; an **admin UI** adds `client`. Members come from the live +catalog, never from a list in prose — a list here would go stale the day a generator is +added, and `--probe` cannot, because it runs the generators rather than describing them. -Codegen is a set of named generators you opt into. Each generator has a **stable -name** (kebab-case) that surfaces in diagnostics — reference generators by that -name, never by inlining what they emit. Typical generators cover: the entity -type/model, the DB table/schema, query/finder helpers, REST routes, client -form/grid/hook artifacts, filter + sort allowlists, payload value-objects, and -parsers for a responding `template.prompt` (one carrying `@responseRef`). You -enable the subset your project needs; an abstract entity never emits -instance/write artifacts regardless. +`capability` looking like one large bucket is the point: you are not meant to choose +inside it by reading labels. You declared a `template.prompt`, or a +`requirement.functional`, or a stored-proc source — `--probe` reports the file count +that follows, for your model. -Per-entity opt-outs exist (e.g. skipping client-side artifacts for a given -entity) and are set as attributes on the entity in metadata, not in code. +An abstract entity never emits instance/write artifacts regardless of what is wired. ## A dependency's metadata is load-only by default — codegen excludes it @@ -239,8 +292,9 @@ the data access too. **`@unmanaged: true`** (view or table); migrate/verify then never touch it. `@sql` and `@unmanaged` are mutually exclusive. -`meta gen --list` prints every generator by stable name; the `generators` array in -`metaobjects.config.ts` is where you opt each one in or out. +`meta gen --list` prints every generator by stable name (add `--probe` for a file count +against your own model); the `generators` array in `metaobjects.config.ts` is where you +opt each one in. It starts empty. ### Adopting onto existing code — make codegen match the code, not the code match codegen @@ -435,7 +489,8 @@ generator sets are **closed built-in registries** — `--generators` *selects* f ships, and there is no seam to register a `Generator` of your own. (Python's `--provider module:symbol` registers **metamodel vocabulary**, not a generator; do not reach for it here.) Use `--template-spec ` — plus `--templates ` on Python or -`--template-root ` on C# — and your entries are appended to the default suite. Worked +`--template-root ` on C# — and your entries are appended to your `--generators` +selection. Worked examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`. **The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both @@ -490,21 +545,21 @@ everywhere — **each physical name is spelled once, and generated code referenc | Kotlin | `Names.kt` | `ProgramNames.CREATED_AT_COLUMN` | | Python | `_names.py` | `PROGRAM_CREATED_AT_COLUMN` | -**Check that it is actually wired before you reference it — on three of five ports an -EXISTING project emits none.** "In the default suite" and "what a fresh scaffold writes" -are different facts, and only C# and Python have the first: +**Check that it is actually wired before you reference it — on ALL FIVE ports a project +emits none until it asks for it.** ADR-0034 Amendment 2 made codegen opt-in everywhere: no +port ships a default suite, so there is no port on which upgrading the package starts +emitting this artifact. "Wired" is the only fact there is. -| Port | Where the suite is decided | An existing project upgrading gets it? | +| Port | Where the selection is declared | An existing project upgrading gets it? | |---|---|---| -| C# | `GenCommand.DefaultGeneratorNames` — a real default | **Yes**, with no edit | -| Python | `cli.py` `_default_generators()` — a real default | **Yes**, with no edit | -| TypeScript | `metaobjects.config.ts` `generators: [...]` — **the complete list; there is no default suite** | **No** — add `namesFile()` | +| C# | `--generators ` on `dotnet meta gen` — required, no default | **No** — name `names` | +| Python | `--generators ` on `metaobjects gen` — required, no default | **No** — name `names` | +| TypeScript | `metaobjects.config.ts` `generators: [...]` — the complete list | **No** — add `namesFile()` | | Java / Kotlin | the pom's `` — the complete list | **No** — add `SpringNamesGenerator` / `KotlinNamesGenerator` | -TypeScript's `meta init` scaffolds `namesFile()`, so a project *initialized* at 1.0 has it; -a project initialized earlier has the config `meta init` wrote then, and upgrading the -package never edits a config. So on TypeScript the artifact is opt-in exactly as it is on -the JVM — the scaffold is not a default. To wire it into an existing TS project: +`meta init` scaffolds `generators: []` and an empty `codegen/generators/`, so even a +project *initialized* at 1.0 has to choose this one — the scaffold is deliberately not a +default by another name. To wire it into a TS project: ```ts import { namesFile } from "./codegen/generators/names.js"; // after `meta eject names` diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/python.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/python.md index 8d69832f5..b7290c799 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/python.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/python.md @@ -50,7 +50,11 @@ meta docs --out ./docs # Node: run from the PROJECT ROOT (no ## Generators -Wire generators by their stable name (`--generators `), or run the default set. +Wire generators by their stable name — **`--generators ` is REQUIRED**. There is +no default set: a run that names none is a usage error and writes nothing (ADR-0034 +Amendment 2). `metaobjects gen --list` is the catalog. `verify --codegen` re-runs the +SELECTION and diffs, so it takes the same `--generators`; with none named it reports that +there is nothing to check. Output lands under `--out` (with the `@generated` guard header). Metadata is the same canonical JSON every port reads (fused-key form, `source.rdb` + `@table`, `@column` for a renamed physical column). diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md index 722a14971..adb6a8c32 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md @@ -64,14 +64,14 @@ handle: raw SQL, a string-keyed query builder, and a hand-written repository on whose generated model carries no persistence binding at all (Java, Python). Your server reference names the handle and the artifact for this stack. -**First check the artifact exists — on TypeScript and the JVM it is opt-in, and an -existing project almost certainly has none.** C# and Python emit it from a real default -suite, so upgrading is enough. TypeScript's `generators: [...]` and the JVM's -`` are each the COMPLETE list: `meta init` scaffolds `namesFile()` for a -project initialized at 1.0, and upgrading the package never edits a config that was -written earlier. Look for `.names.ts` / `Names` in the generated output -before you write `ProgramNames.fields.x.column` against it; if it is not there, wiring the -generator is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL +**First check the artifact exists — it is opt-in on EVERY port, and an existing project +almost certainly has none.** Every port's generator list is now the COMPLETE list: C# and +Python dropped their default suites (ADR-0034 Amendment 2), TypeScript's `generators: []` +starts empty, and the JVM's `` never had a default. Upgrading the package +never edits a config that was written earlier, so `names` arrives only when someone wires +it. Look for `.names.ts` / `Names` in the generated output before you +write `ProgramNames.fields.x.column` against it; if it is not there, wiring the generator +is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL with literal names because "there is no constant" is the loop this closes. ## The REST contract diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/references/python.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/references/python.md index 5102730ac..73f8484d9 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/references/python.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-runtime-ui/references/python.md @@ -100,7 +100,7 @@ resolve the column themselves, so that path never needs a physical name. The mom repository `Protocol` is backed by your own SQLAlchemy Core / asyncpg / psycopg code, it does — and nothing Python generates carries one: the Pydantic models, create/patch shapes, router and allowlist all key by field. Take it from the generated -`_names.py` (`names` is in the default suite): +`_names.py` (`names` is opt-in — name it in `--generators`): ```python from generated.author_names import ( diff --git a/fixtures/agent-context-conformance/python/expected/.metaobjects/AGENTS.md b/fixtures/agent-context-conformance/python/expected/.metaobjects/AGENTS.md index 080ee6941..11b335843 100644 --- a/fixtures/agent-context-conformance/python/expected/.metaobjects/AGENTS.md +++ b/fixtures/agent-context-conformance/python/expected/.metaobjects/AGENTS.md @@ -30,7 +30,8 @@ itself for those three: it is the source those pages are generated from. - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/python/expected/.metaobjects/CLAUDE.md b/fixtures/agent-context-conformance/python/expected/.metaobjects/CLAUDE.md index 080ee6941..11b335843 100644 --- a/fixtures/agent-context-conformance/python/expected/.metaobjects/CLAUDE.md +++ b/fixtures/agent-context-conformance/python/expected/.metaobjects/CLAUDE.md @@ -30,7 +30,8 @@ itself for those three: it is the source those pages are generated from. - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md index 82ed5693b..5f1238aff 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -351,7 +351,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on THREE of five ports an existing project emits none**. C# and Python have a real default suite and get it by upgrading; TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, so `meta init` scaffolding `namesFile()` covers only a project initialized at 1.0 and upgrading the package never edits a config written earlier. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index 3a05b86ca..8b3265072 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -71,10 +71,10 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove migration script, a body-to-column map (drift signature 11). Every port emits a per-object names artifact from the declaration, so a literal is a second source of truth even when it agrees with the naming strategy today. A typed ORM handle in its place is correct. **Check - the artifact is emitted at all before scoring the literals: on TypeScript and the JVM the - generator list in the config IS the complete list, so an existing project emits none and the - un-wired generator is the finding FIRST** (C# and Python have a real default suite and get it - by upgrading). **This entry is the physical-name INSTANCE of signature 11.** The rule that + the artifact is emitted at all before scoring the literals: on every port the selection IS + the complete list — the config's on TypeScript and the JVM, `--generators` on C# and Python — + so an existing project emits none and the un-wired generator is the finding FIRST** (no port + ships a default suite; ADR-0034 Amendment 2). **This entry is the physical-name INSTANCE of signature 11.** The rule that generates the rest — and the reason an enum member compared as a bare string is NOT one — is the Cross-cutting entry below. - **`@kind` = `view` / `materializedView`** — hunt hand-written SQL views where an authored diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 86bfc2dfd..c40bce3c3 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -46,22 +46,28 @@ is, this once:** **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: -1. **Search the vocabulary** — `meta types `, or `meta types --all +1. **Check whether the design already ships.** `meta gen --list --format json` + carries `kind: "library"` rows — declared design MetaObjects ships, each with a + `useWhen`. If one matches the capability you are about to declare, opt in and adapt + rather than author it from scratch: you inherit its entities, its requirements, and + the rulings recorded with them. The bare name is the core layer (sourceless — no + tables, no generated code); `/db` is the separate opt-in that adds the schema. +2. **Search the vocabulary** — `meta types `, or `meta types --all ` to search by behavior. There are field subtypes, relationships, projections, origins, identities, sources, and attributes you may not know exist. Find the construct that models it. Add `--detail` for one construct's valid `@attrs`, or `--format json` for the same answer as one machine-readable document — that form carries every match (`--limit` never truncates it) with each attr's `allowedValues`, so you read the accepted values rather than guessing them. -2. **Declare it and generate** — then *consume* the generated query/type/route; +3. **Declare it and generate** — then *consume* the generated query/type/route; never reimplement it alongside. -3. **If the model is right but the generated OUTPUT is wrong, change your generator.** +4. **If the model is right but the generated OUTPUT is wrong, change your generator.** Naming, file layout, imports, framework, signatures are generator concerns, not reasons to hand-write. The generators are in *your* repo and are yours to edit — a standing rule not to change the MetaObjects repo does not reach them; they are a different repository. Editing one is ordinary work, not an escalation. (See `metaobjects-codegen` → "Your generators are yours".) -4. **Only if no construct can express it** — and you have actually looked — +5. **Only if no construct can express it** — and you have actually looked — hand-write it, wired to generated types. Business algorithms, external integrations, and bespoke interactions are legitimately hand-written; CRUD, validation, finders, relationships, and derived/aggregate data are not. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md index 247af390d..5f3350cf2 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -124,19 +124,72 @@ implementing the port's generator interface elsewhere. Your language reference h mechanism; see also "The commands and config keys that implement the steps above differ per port" below. -## Selecting generators by stable name +## Selecting generators — NOTHING is generated until you choose it + +**`meta init` wires no generators, and no port ships a default suite.** A fresh +scaffold has `generators: []` and an empty `codegen/generators/`; `--generators` is +required on the C# and Python CLIs; Java has never had a default set. Choosing what an +application needs is a judgment over its purpose and stack, and the tool's job is to +make that choice cheap and truthful, not to make it for you. + +Each generator has a **stable name** (kebab-case) that is the same in every port and +surfaces in diagnostics — reference generators by that name, never by inlining what +they emit. + +### The procedure + +1. **Read the app's purpose and stack.** What is it for; what does it already use. +2. **`meta gen --list --format json --probe`** — the catalog. Every generator with its + `layer`, `framework`, what it emits, what it requires, what it costs to install, + and — with `--probe`, which constructs each generator and dry-runs it against YOUR + model — how many files each would actually emit. +3. **Check the libraries before you choose generators.** The same catalog carries + `kind: "library"` rows — declared design MetaObjects ships, each with a `useWhen`. + If one matches a capability you are about to model, **opt in and adapt rather than + author**: you inherit its entities, its requirements, and the rulings recorded with + them. Layers are how much of it you take. The bare name is the CORE layer — the + model and its ledger, sourceless, so it adds **no tables and no generated code**; + `/db` is the separate opt-in that proposes the schema. Opt in with + `"libraries": ["iam", "iam/db"]` in `.metaobjects/config.json`. +4. **Choose by `layer`.** Satisfy every `requires`. Take what `wouldEmit > 0` says your + model is already asking for. + - Pick **ONE** framework on the `api` layer: `routes` and `routes-hono` are + alternatives, and wiring both silently produces two complete HTTP surfaces. + - Do **NOT** apply that rule to `client`. `@metaobjectsdev/tanstack` peers on + `react`, so a form generator plus the TanStack hook/grid generators is the + intended composition, not a conflict. +5. **`meta eject --format json`** — copies each into `codegen/generators/` + (yours to edit), and reports the import line, the entry to add to `generators`, one + consolidated install command, and any config keys those generators read. +6. **`meta gen`** — read its warnings, then typecheck. + +A library row also carries `provides` (what is in the box) and, under `--probe`, a +`project` block: which layers you selected, how many tables and requirements they +added here, which of your entities `extends` into it, and any generator the library +implies that you have not wired. + +### The six layers, and what picks them + +| layer | what it is | chosen by | +|---|---|---| +| `model` | entity/DTO/value-object modules and the constants beside them | app shape | +| `persistence` | how rows are read and written | app shape | +| `api` | the HTTP surface — pick one framework | app shape | +| `client` | the browser tier | app shape | +| `docs` | on by default; the canonical door is `meta docs` | — | +| `capability` | prompts, parsers, payloads, traces, test stubs | **`--probe`** | + +At intent level: a **headless data service** is `model` + `persistence`; an **HTTP API** +adds `api` with one framework; an **admin UI** adds `client`. Members come from the live +catalog, never from a list in prose — a list here would go stale the day a generator is +added, and `--probe` cannot, because it runs the generators rather than describing them. -Codegen is a set of named generators you opt into. Each generator has a **stable -name** (kebab-case) that surfaces in diagnostics — reference generators by that -name, never by inlining what they emit. Typical generators cover: the entity -type/model, the DB table/schema, query/finder helpers, REST routes, client -form/grid/hook artifacts, filter + sort allowlists, payload value-objects, and -parsers for a responding `template.prompt` (one carrying `@responseRef`). You -enable the subset your project needs; an abstract entity never emits -instance/write artifacts regardless. +`capability` looking like one large bucket is the point: you are not meant to choose +inside it by reading labels. You declared a `template.prompt`, or a +`requirement.functional`, or a stored-proc source — `--probe` reports the file count +that follows, for your model. -Per-entity opt-outs exist (e.g. skipping client-side artifacts for a given -entity) and are set as attributes on the entity in metadata, not in code. +An abstract entity never emits instance/write artifacts regardless of what is wired. ## A dependency's metadata is load-only by default — codegen excludes it @@ -239,8 +292,9 @@ the data access too. **`@unmanaged: true`** (view or table); migrate/verify then never touch it. `@sql` and `@unmanaged` are mutually exclusive. -`meta gen --list` prints every generator by stable name; the `generators` array in -`metaobjects.config.ts` is where you opt each one in or out. +`meta gen --list` prints every generator by stable name (add `--probe` for a file count +against your own model); the `generators` array in `metaobjects.config.ts` is where you +opt each one in. It starts empty. ### Adopting onto existing code — make codegen match the code, not the code match codegen @@ -435,7 +489,8 @@ generator sets are **closed built-in registries** — `--generators` *selects* f ships, and there is no seam to register a `Generator` of your own. (Python's `--provider module:symbol` registers **metamodel vocabulary**, not a generator; do not reach for it here.) Use `--template-spec ` — plus `--templates ` on Python or -`--template-root ` on C# — and your entries are appended to the default suite. Worked +`--template-root ` on C# — and your entries are appended to your `--generators` +selection. Worked examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`. **The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both @@ -490,21 +545,21 @@ everywhere — **each physical name is spelled once, and generated code referenc | Kotlin | `Names.kt` | `ProgramNames.CREATED_AT_COLUMN` | | Python | `_names.py` | `PROGRAM_CREATED_AT_COLUMN` | -**Check that it is actually wired before you reference it — on three of five ports an -EXISTING project emits none.** "In the default suite" and "what a fresh scaffold writes" -are different facts, and only C# and Python have the first: +**Check that it is actually wired before you reference it — on ALL FIVE ports a project +emits none until it asks for it.** ADR-0034 Amendment 2 made codegen opt-in everywhere: no +port ships a default suite, so there is no port on which upgrading the package starts +emitting this artifact. "Wired" is the only fact there is. -| Port | Where the suite is decided | An existing project upgrading gets it? | +| Port | Where the selection is declared | An existing project upgrading gets it? | |---|---|---| -| C# | `GenCommand.DefaultGeneratorNames` — a real default | **Yes**, with no edit | -| Python | `cli.py` `_default_generators()` — a real default | **Yes**, with no edit | -| TypeScript | `metaobjects.config.ts` `generators: [...]` — **the complete list; there is no default suite** | **No** — add `namesFile()` | +| C# | `--generators ` on `dotnet meta gen` — required, no default | **No** — name `names` | +| Python | `--generators ` on `metaobjects gen` — required, no default | **No** — name `names` | +| TypeScript | `metaobjects.config.ts` `generators: [...]` — the complete list | **No** — add `namesFile()` | | Java / Kotlin | the pom's `` — the complete list | **No** — add `SpringNamesGenerator` / `KotlinNamesGenerator` | -TypeScript's `meta init` scaffolds `namesFile()`, so a project *initialized* at 1.0 has it; -a project initialized earlier has the config `meta init` wrote then, and upgrading the -package never edits a config. So on TypeScript the artifact is opt-in exactly as it is on -the JVM — the scaffold is not a default. To wire it into an existing TS project: +`meta init` scaffolds `generators: []` and an empty `codegen/generators/`, so even a +project *initialized* at 1.0 has to choose this one — the scaffold is deliberately not a +default by another name. To wire it into a TS project: ```ts import { namesFile } from "./codegen/generators/names.js"; // after `meta eject names` diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md index db252d560..cb9c21fed 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md @@ -32,9 +32,13 @@ npm install --save-dev @metaobjectsdev/codegen-ts-react @metaobjectsdev/codegen- Codegen is wired in a type-checked TS config at the project root. `defineConfig` comes from `@metaobjectsdev/cli`; the generators come from their packages. +`meta init` scaffolds this file with **`generators: []`** — nothing is generated until +you choose it. Each import below appears once you `meta eject` that generator, which +prints the exact line to add. + ```ts import { defineConfig } from "@metaobjectsdev/cli"; -// Owned generators scaffolded by `meta init` (ADR-0034 scaffold-and-own). +// Owned generators — copied in by `meta eject` (ADR-0034 scaffold-and-own). import { entityFile } from "./codegen/generators/entity"; import { queriesFile } from "./codegen/generators/queries"; import { routesFile } from "./codegen/generators/routes"; @@ -83,13 +87,19 @@ PROJECT ROOT that CONTAINS the metadata — never the metadata directory itself. ## The generators -Server-side, framework-neutral. The first four are **scaffolded into your repo** by -`meta init` and imported from `./codegen/generators/*` (ADR-0034) — 1.0 REMOVED their -`@metaobjectsdev/codegen-ts/generators` export, so an owned copy is the only path. The -engine primitives come from the package main entry, `@metaobjectsdev/codegen-ts`. The -`/generators` subpath itself is NOT deprecated: it is the supported home of the generators -with no ownable copy — `promptRender`, `outputParser`, `outputPrompt`, `extractor`, -`renderHelper`, `traceHelperFile`, `routesFileHono`, `namesFile`, `callableFile`. +Server-side, framework-neutral. **None is wired by default** — `meta init` writes +`generators: []` and an empty `codegen/generators/`. `meta eject ...` copies the +ownable ones into your repo, imported from `./codegen/generators/*` (ADR-0034); 1.0 +REMOVED their `@metaobjectsdev/codegen-ts/generators` export, so an owned copy is the only +path for those. The engine primitives come from the package main entry, +`@metaobjectsdev/codegen-ts`. The `/generators` subpath itself is NOT deprecated: it is the +supported home of the generators with no ownable copy — `promptRender`, `outputParser`, +`outputPrompt`, `extractor`, `renderHelper`, `traceHelperFile`, `namesFile`, +`callableFile`, `requirementTests`. + +The table below is a per-emission reference, NOT the selection surface. Select with +`meta gen --list --format json --probe`, which is generated from the live registry and +reports a file count for your own model; a table in a document cannot do either. | Generator | Emits per entity | |---|---| diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md index 722a14971..adb6a8c32 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md @@ -64,14 +64,14 @@ handle: raw SQL, a string-keyed query builder, and a hand-written repository on whose generated model carries no persistence binding at all (Java, Python). Your server reference names the handle and the artifact for this stack. -**First check the artifact exists — on TypeScript and the JVM it is opt-in, and an -existing project almost certainly has none.** C# and Python emit it from a real default -suite, so upgrading is enough. TypeScript's `generators: [...]` and the JVM's -`` are each the COMPLETE list: `meta init` scaffolds `namesFile()` for a -project initialized at 1.0, and upgrading the package never edits a config that was -written earlier. Look for `.names.ts` / `Names` in the generated output -before you write `ProgramNames.fields.x.column` against it; if it is not there, wiring the -generator is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL +**First check the artifact exists — it is opt-in on EVERY port, and an existing project +almost certainly has none.** Every port's generator list is now the COMPLETE list: C# and +Python dropped their default suites (ADR-0034 Amendment 2), TypeScript's `generators: []` +starts empty, and the JVM's `` never had a default. Upgrading the package +never edits a config that was written earlier, so `names` arrives only when someone wires +it. Look for `.names.ts` / `Names` in the generated output before you +write `ProgramNames.fields.x.column` against it; if it is not there, wiring the generator +is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL with literal names because "there is no constant" is the loop this closes. ## The REST contract diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/AGENTS.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/AGENTS.md index eb65d8e43..b18a14ed2 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/AGENTS.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/AGENTS.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/CLAUDE.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/CLAUDE.md index eb65d8e43..b18a14ed2 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/CLAUDE.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.metaobjects/CLAUDE.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md index 82ed5693b..5f1238aff 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -351,7 +351,7 @@ Per finding: `file:line` → what → generated-equivalent exists? → recommend **Score only where a constant actually exists.** A fact declared in the project's CODEGEN CONFIG rather than in metadata — a bespoke route mount path, a hand-chosen JSON envelope key — is genuinely spelled twice when a client re-types it, but no generated constant holds it. That is a GENERATOR GAP, reported as one, never a literal-site finding: never invent a constant that the emitters do not emit. - **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on THREE of five ports an existing project emits none**. C# and Python have a real default suite and get it by upgrading; TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, so `meta init` scaffolding `namesFile()` covers only a project initialized at 1.0 and upgrading the package never edits a config written earlier. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. + **No verify subverb sees this** — `--codegen` diffs generated files, `--db` compares schema to metadata — so this audit is the only gate. Remedy: reference the constant — but **check the artifact is emitted at all before scoring the literals, because on ALL FIVE ports an existing project emits none until it asks**. ADR-0034 Amendment 2 made codegen opt-in everywhere: C# and Python require `--generators`, TypeScript's `generators: [...]` and the JVM's `` are each the COMPLETE list, and `meta init` scaffolds `generators: []` — so no port begins emitting this artifact merely because the package was upgraded. Where no artifact exists, the un-wired generator is the FIRST finding and the first remedy (`namesFile()` on TS after `meta eject names`; `SpringNamesGenerator` / `KotlinNamesGenerator` in the pom) — score the literals under it rather than as N independent findings, since one config line fixes the cause and generated code stops embedding the names too. **Report TWO numbers for this signature, always, and never just the first: (a) is the artifact EMITTED, and (b) how many of the literal sites actually IMPORT it — the adoption ratio, as `/ sites`.** Emission is the cheap half and it is the half that gets done: measured on an adopter whose audit proved the artifact emitted correctly, ran the generator to show it, and then adopted it at **0 of 53 sites** (37 table + 16 column). Wiring the generator changes the scorecard; it changes nothing a reader of the code experiences, because the second spelling is still the one every query uses. An audit that reports only (a) makes the estate look upgraded while every literal it found is still there — so a project that emits the artifact and imports it nowhere scores WORSE than one that has not started, not better, because it now carries a third spelling that claims to be the source of truth. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md index 3a05b86ca..8b3265072 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/capability-checklist.md @@ -71,10 +71,10 @@ classify it (using the classification scheme in `SKILL.md`) and route the cutove migration script, a body-to-column map (drift signature 11). Every port emits a per-object names artifact from the declaration, so a literal is a second source of truth even when it agrees with the naming strategy today. A typed ORM handle in its place is correct. **Check - the artifact is emitted at all before scoring the literals: on TypeScript and the JVM the - generator list in the config IS the complete list, so an existing project emits none and the - un-wired generator is the finding FIRST** (C# and Python have a real default suite and get it - by upgrading). **This entry is the physical-name INSTANCE of signature 11.** The rule that + the artifact is emitted at all before scoring the literals: on every port the selection IS + the complete list — the config's on TypeScript and the JVM, `--generators` on C# and Python — + so an existing project emits none and the un-wired generator is the finding FIRST** (no port + ships a default suite; ADR-0034 Amendment 2). **This entry is the physical-name INSTANCE of signature 11.** The rule that generates the rest — and the reason an enum member compared as a bare string is NOT one — is the Cross-cutting entry below. - **`@kind` = `view` / `materializedView`** — hunt hand-written SQL views where an authored diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md index 86bfc2dfd..c40bce3c3 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -46,22 +46,28 @@ is, this once:** **Before you hand-write anything data-shaped, STOP and find the model.** The moment you reach for a hand-written query, route, validator, form, relationship, or aggregate — that is almost always **metadata you have not declared yet.** In order: -1. **Search the vocabulary** — `meta types `, or `meta types --all +1. **Check whether the design already ships.** `meta gen --list --format json` + carries `kind: "library"` rows — declared design MetaObjects ships, each with a + `useWhen`. If one matches the capability you are about to declare, opt in and adapt + rather than author it from scratch: you inherit its entities, its requirements, and + the rulings recorded with them. The bare name is the core layer (sourceless — no + tables, no generated code); `/db` is the separate opt-in that adds the schema. +2. **Search the vocabulary** — `meta types `, or `meta types --all ` to search by behavior. There are field subtypes, relationships, projections, origins, identities, sources, and attributes you may not know exist. Find the construct that models it. Add `--detail` for one construct's valid `@attrs`, or `--format json` for the same answer as one machine-readable document — that form carries every match (`--limit` never truncates it) with each attr's `allowedValues`, so you read the accepted values rather than guessing them. -2. **Declare it and generate** — then *consume* the generated query/type/route; +3. **Declare it and generate** — then *consume* the generated query/type/route; never reimplement it alongside. -3. **If the model is right but the generated OUTPUT is wrong, change your generator.** +4. **If the model is right but the generated OUTPUT is wrong, change your generator.** Naming, file layout, imports, framework, signatures are generator concerns, not reasons to hand-write. The generators are in *your* repo and are yours to edit — a standing rule not to change the MetaObjects repo does not reach them; they are a different repository. Editing one is ordinary work, not an escalation. (See `metaobjects-codegen` → "Your generators are yours".) -4. **Only if no construct can express it** — and you have actually looked — +5. **Only if no construct can express it** — and you have actually looked — hand-write it, wired to generated types. Business algorithms, external integrations, and bespoke interactions are legitimately hand-written; CRUD, validation, finders, relationships, and derived/aggregate data are not. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md index 247af390d..5f3350cf2 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -124,19 +124,72 @@ implementing the port's generator interface elsewhere. Your language reference h mechanism; see also "The commands and config keys that implement the steps above differ per port" below. -## Selecting generators by stable name +## Selecting generators — NOTHING is generated until you choose it + +**`meta init` wires no generators, and no port ships a default suite.** A fresh +scaffold has `generators: []` and an empty `codegen/generators/`; `--generators` is +required on the C# and Python CLIs; Java has never had a default set. Choosing what an +application needs is a judgment over its purpose and stack, and the tool's job is to +make that choice cheap and truthful, not to make it for you. + +Each generator has a **stable name** (kebab-case) that is the same in every port and +surfaces in diagnostics — reference generators by that name, never by inlining what +they emit. + +### The procedure + +1. **Read the app's purpose and stack.** What is it for; what does it already use. +2. **`meta gen --list --format json --probe`** — the catalog. Every generator with its + `layer`, `framework`, what it emits, what it requires, what it costs to install, + and — with `--probe`, which constructs each generator and dry-runs it against YOUR + model — how many files each would actually emit. +3. **Check the libraries before you choose generators.** The same catalog carries + `kind: "library"` rows — declared design MetaObjects ships, each with a `useWhen`. + If one matches a capability you are about to model, **opt in and adapt rather than + author**: you inherit its entities, its requirements, and the rulings recorded with + them. Layers are how much of it you take. The bare name is the CORE layer — the + model and its ledger, sourceless, so it adds **no tables and no generated code**; + `/db` is the separate opt-in that proposes the schema. Opt in with + `"libraries": ["iam", "iam/db"]` in `.metaobjects/config.json`. +4. **Choose by `layer`.** Satisfy every `requires`. Take what `wouldEmit > 0` says your + model is already asking for. + - Pick **ONE** framework on the `api` layer: `routes` and `routes-hono` are + alternatives, and wiring both silently produces two complete HTTP surfaces. + - Do **NOT** apply that rule to `client`. `@metaobjectsdev/tanstack` peers on + `react`, so a form generator plus the TanStack hook/grid generators is the + intended composition, not a conflict. +5. **`meta eject --format json`** — copies each into `codegen/generators/` + (yours to edit), and reports the import line, the entry to add to `generators`, one + consolidated install command, and any config keys those generators read. +6. **`meta gen`** — read its warnings, then typecheck. + +A library row also carries `provides` (what is in the box) and, under `--probe`, a +`project` block: which layers you selected, how many tables and requirements they +added here, which of your entities `extends` into it, and any generator the library +implies that you have not wired. + +### The six layers, and what picks them + +| layer | what it is | chosen by | +|---|---|---| +| `model` | entity/DTO/value-object modules and the constants beside them | app shape | +| `persistence` | how rows are read and written | app shape | +| `api` | the HTTP surface — pick one framework | app shape | +| `client` | the browser tier | app shape | +| `docs` | on by default; the canonical door is `meta docs` | — | +| `capability` | prompts, parsers, payloads, traces, test stubs | **`--probe`** | + +At intent level: a **headless data service** is `model` + `persistence`; an **HTTP API** +adds `api` with one framework; an **admin UI** adds `client`. Members come from the live +catalog, never from a list in prose — a list here would go stale the day a generator is +added, and `--probe` cannot, because it runs the generators rather than describing them. -Codegen is a set of named generators you opt into. Each generator has a **stable -name** (kebab-case) that surfaces in diagnostics — reference generators by that -name, never by inlining what they emit. Typical generators cover: the entity -type/model, the DB table/schema, query/finder helpers, REST routes, client -form/grid/hook artifacts, filter + sort allowlists, payload value-objects, and -parsers for a responding `template.prompt` (one carrying `@responseRef`). You -enable the subset your project needs; an abstract entity never emits -instance/write artifacts regardless. +`capability` looking like one large bucket is the point: you are not meant to choose +inside it by reading labels. You declared a `template.prompt`, or a +`requirement.functional`, or a stored-proc source — `--probe` reports the file count +that follows, for your model. -Per-entity opt-outs exist (e.g. skipping client-side artifacts for a given -entity) and are set as attributes on the entity in metadata, not in code. +An abstract entity never emits instance/write artifacts regardless of what is wired. ## A dependency's metadata is load-only by default — codegen excludes it @@ -239,8 +292,9 @@ the data access too. **`@unmanaged: true`** (view or table); migrate/verify then never touch it. `@sql` and `@unmanaged` are mutually exclusive. -`meta gen --list` prints every generator by stable name; the `generators` array in -`metaobjects.config.ts` is where you opt each one in or out. +`meta gen --list` prints every generator by stable name (add `--probe` for a file count +against your own model); the `generators` array in `metaobjects.config.ts` is where you +opt each one in. It starts empty. ### Adopting onto existing code — make codegen match the code, not the code match codegen @@ -435,7 +489,8 @@ generator sets are **closed built-in registries** — `--generators` *selects* f ships, and there is no seam to register a `Generator` of your own. (Python's `--provider module:symbol` registers **metamodel vocabulary**, not a generator; do not reach for it here.) Use `--template-spec ` — plus `--templates ` on Python or -`--template-root ` on C# — and your entries are appended to the default suite. Worked +`--template-root ` on C# — and your entries are appended to your `--generators` +selection. Worked examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`. **The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both @@ -490,21 +545,21 @@ everywhere — **each physical name is spelled once, and generated code referenc | Kotlin | `Names.kt` | `ProgramNames.CREATED_AT_COLUMN` | | Python | `_names.py` | `PROGRAM_CREATED_AT_COLUMN` | -**Check that it is actually wired before you reference it — on three of five ports an -EXISTING project emits none.** "In the default suite" and "what a fresh scaffold writes" -are different facts, and only C# and Python have the first: +**Check that it is actually wired before you reference it — on ALL FIVE ports a project +emits none until it asks for it.** ADR-0034 Amendment 2 made codegen opt-in everywhere: no +port ships a default suite, so there is no port on which upgrading the package starts +emitting this artifact. "Wired" is the only fact there is. -| Port | Where the suite is decided | An existing project upgrading gets it? | +| Port | Where the selection is declared | An existing project upgrading gets it? | |---|---|---| -| C# | `GenCommand.DefaultGeneratorNames` — a real default | **Yes**, with no edit | -| Python | `cli.py` `_default_generators()` — a real default | **Yes**, with no edit | -| TypeScript | `metaobjects.config.ts` `generators: [...]` — **the complete list; there is no default suite** | **No** — add `namesFile()` | +| C# | `--generators ` on `dotnet meta gen` — required, no default | **No** — name `names` | +| Python | `--generators ` on `metaobjects gen` — required, no default | **No** — name `names` | +| TypeScript | `metaobjects.config.ts` `generators: [...]` — the complete list | **No** — add `namesFile()` | | Java / Kotlin | the pom's `` — the complete list | **No** — add `SpringNamesGenerator` / `KotlinNamesGenerator` | -TypeScript's `meta init` scaffolds `namesFile()`, so a project *initialized* at 1.0 has it; -a project initialized earlier has the config `meta init` wrote then, and upgrading the -package never edits a config. So on TypeScript the artifact is opt-in exactly as it is on -the JVM — the scaffold is not a default. To wire it into an existing TS project: +`meta init` scaffolds `generators: []` and an empty `codegen/generators/`, so even a +project *initialized* at 1.0 has to choose this one — the scaffold is deliberately not a +default by another name. To wire it into a TS project: ```ts import { namesFile } from "./codegen/generators/names.js"; // after `meta eject names` diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md index db252d560..cb9c21fed 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md @@ -32,9 +32,13 @@ npm install --save-dev @metaobjectsdev/codegen-ts-react @metaobjectsdev/codegen- Codegen is wired in a type-checked TS config at the project root. `defineConfig` comes from `@metaobjectsdev/cli`; the generators come from their packages. +`meta init` scaffolds this file with **`generators: []`** — nothing is generated until +you choose it. Each import below appears once you `meta eject` that generator, which +prints the exact line to add. + ```ts import { defineConfig } from "@metaobjectsdev/cli"; -// Owned generators scaffolded by `meta init` (ADR-0034 scaffold-and-own). +// Owned generators — copied in by `meta eject` (ADR-0034 scaffold-and-own). import { entityFile } from "./codegen/generators/entity"; import { queriesFile } from "./codegen/generators/queries"; import { routesFile } from "./codegen/generators/routes"; @@ -83,13 +87,19 @@ PROJECT ROOT that CONTAINS the metadata — never the metadata directory itself. ## The generators -Server-side, framework-neutral. The first four are **scaffolded into your repo** by -`meta init` and imported from `./codegen/generators/*` (ADR-0034) — 1.0 REMOVED their -`@metaobjectsdev/codegen-ts/generators` export, so an owned copy is the only path. The -engine primitives come from the package main entry, `@metaobjectsdev/codegen-ts`. The -`/generators` subpath itself is NOT deprecated: it is the supported home of the generators -with no ownable copy — `promptRender`, `outputParser`, `outputPrompt`, `extractor`, -`renderHelper`, `traceHelperFile`, `routesFileHono`, `namesFile`, `callableFile`. +Server-side, framework-neutral. **None is wired by default** — `meta init` writes +`generators: []` and an empty `codegen/generators/`. `meta eject ...` copies the +ownable ones into your repo, imported from `./codegen/generators/*` (ADR-0034); 1.0 +REMOVED their `@metaobjectsdev/codegen-ts/generators` export, so an owned copy is the only +path for those. The engine primitives come from the package main entry, +`@metaobjectsdev/codegen-ts`. The `/generators` subpath itself is NOT deprecated: it is the +supported home of the generators with no ownable copy — `promptRender`, `outputParser`, +`outputPrompt`, `extractor`, `renderHelper`, `traceHelperFile`, `namesFile`, +`callableFile`, `requirementTests`. + +The table below is a per-emission reference, NOT the selection surface. Select with +`meta gen --list --format json --probe`, which is generated from the live registry and +reports a file count for your own model; a table in a document cannot do either. | Generator | Emits per entity | |---|---| diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md index 722a14971..adb6a8c32 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-runtime-ui/SKILL.md @@ -64,14 +64,14 @@ handle: raw SQL, a string-keyed query builder, and a hand-written repository on whose generated model carries no persistence binding at all (Java, Python). Your server reference names the handle and the artifact for this stack. -**First check the artifact exists — on TypeScript and the JVM it is opt-in, and an -existing project almost certainly has none.** C# and Python emit it from a real default -suite, so upgrading is enough. TypeScript's `generators: [...]` and the JVM's -`` are each the COMPLETE list: `meta init` scaffolds `namesFile()` for a -project initialized at 1.0, and upgrading the package never edits a config that was -written earlier. Look for `.names.ts` / `Names` in the generated output -before you write `ProgramNames.fields.x.column` against it; if it is not there, wiring the -generator is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL +**First check the artifact exists — it is opt-in on EVERY port, and an existing project +almost certainly has none.** Every port's generator list is now the COMPLETE list: C# and +Python dropped their default suites (ADR-0034 Amendment 2), TypeScript's `generators: []` +starts empty, and the JVM's `` never had a default. Upgrading the package +never edits a config that was written earlier, so `names` arrives only when someone wires +it. Look for `.names.ts` / `Names` in the generated output before you +write `ProgramNames.fields.x.column` against it; if it is not there, wiring the generator +is the first step and the `metaobjects-codegen` skill says how. Writing raw SQL with literal names because "there is no constant" is the loop this closes. ## The REST contract diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/AGENTS.md b/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/AGENTS.md index 5d4e28f71..8c255ed26 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/AGENTS.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/AGENTS.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/CLAUDE.md b/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/CLAUDE.md index 5d4e28f71..8c255ed26 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/CLAUDE.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.metaobjects/CLAUDE.md @@ -32,7 +32,8 @@ Not there? Run `meta docs`. It reads the metadata and the config and needs nothi - Pattern-derivable from metadata = codegen, never hand-write — FKs, CRUD, validators, finders, and the database schema and migrations. The schema is a disposable, generated artifact: change the metadata and regenerate. Don't hand-write divergent SQL; where a stack owns its own migration files, GENERATE them — `meta migrate --migration-format flyway` emits the paired `V__`/`U__` files a Flyway runner expects, and leaves applying them to Flyway. - The **live database** is a derived artifact too — never hand-apply a schema change to a running DB (ad-hoc `psql`/console `ALTER`/`CREATE`/`DROP`), not even to preview a column or unblock a boot. Drive every schema change from the metadata, never ad-hoc: on the Node/TS-owned migrate stack that is `meta migrate` (metadata → DDL, ADR-0015); on a **Flyway**-owned JVM stack that is `meta migrate --migration-format flyway`, which emits the migration and deliberately refuses `--apply` because Flyway owns applying it. Hand-authoring a migration to match the generated schema is the LAST resort — for a runner MetaObjects has no writer for — never the default for Flyway. A hand-applied ad-hoc change drifts the live DB from the metadata + migration history and collides at the next migrate/boot ("column already exists") — a state no migration can reproduce. Run `meta verify --db ` after any DB-touching work to catch that drift early — the URL is required, and the bare form exits 2. - Never hand-edit generated **output** — change the metadata and regenerate. (That bounds the files codegen *emits*; it says nothing about the generator that emits them — next bullet.) What survives a regen DIFFERS BY TOOLCHAIN, so never rely on it: the Node/TS `meta gen` path three-way-merges, so an edit inside a generated file is preserved (and is refused rather than guessed at when it cannot tell yours from its own stale output); the JVM generators overwrite, and several Kotlin ones — including `.kt` — truncate unconditionally with no marker check. Put your own code in a subclass or a separate file, never in the generated one. -- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source scaffolded into your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. +- **Nothing is generated until you wire it.** No port ships a default generator suite: a fresh `meta init` writes `generators: []` and an empty `codegen/generators/`, and `--generators` is required on the C# and Python CLIs. `meta gen --list --probe` is the catalog — it runs every generator against your model and reports how many files each would emit — and `meta eject ...` copies the ones you choose into your repo. +- **The generators are yours — editing one is ordinary work, not an escalation.** Your generator list, and any generator source in your repo (`codegen/generators/` on the Node/TS path), are your code: no `@generated` header, no upstream ownership, nothing to ask permission for. **A standing rule not to change the MetaObjects repo is not a rule about your generators** — that is a different repository, and generalising the first into the second is how an agent ends up hand-writing the exact layer it was told to generate. When generated output does not fit, in order: change the **metadata** if the model is wrong; else change **your own generator**; only then hand-write. Hand-writing what the metadata already describes is the last resort, not the first. Per-port specifics — including which ports have an eject command and which mean implementing a generator interface — are in the `metaobjects-codegen` skill. - Use the generated constants for any string that names metadata — a type, subtype or attribute name, and a **physical table or column name**: those are declared once (`@table` / `@column`) and emitted as the per-object `Names` artifact, so never respell one as a literal outside its declaration. Prefer a typed ORM handle where one exists; the constant is for raw SQL, migrations and logs. - **Ownership has a converse — wire a generator only for output you will actually consume.** Generated code nothing imports is indistinguishable from generated code that does not COMPILE — one adopter carried 87 uncallable query helpers through two audits because a dead-file census read them as over-generation. Worse, an unused generated file still reads as an invitation: a routes file nobody mounted still says "register this as-is for stock CRUD", so the next reader adopts the thing you decided not to. If an entity needs no REST surface, grid or form, don't wire that generator at all, or narrow it with the generator's own `filter` — don't emit the file and leave it unimported. - The loaded metadata model is READ-ONLY — never inject nodes or mutate the tree at load time (no "enrich the model on load" hooks). Need an extra field/column? Author it in the metadata, or derive it during codegen (read the metadata, emit output). Mutating the loaded model makes it diverge from what's declared — a bad practice reserved for very rare cases. diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 18b0f4263..059d0858a 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -89,6 +89,9 @@ "ERR_ENUM_INT_VALUE_MAP_ARRAY": "A field.enum carries @intValueMap together with isArray=true. Int-backing is a persistence-layer codec and no port implements it element-wise over an array column, so the combination would silently persist member SYMBOLS into an integer array. An array-of-enum stays string-backed: drop @intValueMap, or make the field scalar.", "ERR_REQUIREMENT_RETIRED_HAS_IMPLEMENTORS": "A requirement.* with @status: retired declares @implementedBy. Refused rather than exempted (FR-039): a retired capability has no implementation by definition, so forbidding the attribute makes the dangling-reference class unreachable instead of silently tolerated.", "ERR_REQUIREMENT_SUPERSEDED_BY_NOT_RETIRED": "@supersededBy on a requirement whose @status is not `retired`. The attribute names what REPLACED a withdrawn capability; on a live one there is nothing to have replaced it.", + "ERR_UNKNOWN_LIBRARY": "FR-043: .metaobjects/config.json's `libraries` names a shipped library or layer this build does not have. Refused with the available tokens rather than skipped — skipped, it resurfaces as ERR_UNRESOLVED_SUPER against the adopter's own metadata.", + "ERR_LIBRARY_PACKAGE_COLLISION": "FR-043: a node is declared by BOTH an adopter's own metadata and a shipped library the project opts into — the `meta eject ` copy with the library still in `libraries`. The two merge silently and asymmetrically: additions take, deletions do not. TypeScript only (the SDK load path).", + "ERR_LIBRARY_PACKAGE_NOT_OWNED": "FR-043: a NEW top-level node is declared into a package a shipped library owns while that library is opted in — a later release of the library may ship a node of that name and merge into it. `overlay: true` on one of the library's OWN nodes is the documented amendment door and is untouched. TypeScript only (the SDK load path).", "ERR_DEPENDENCY_UNRESOLVED": "FR-023: a declared dependency's transport could not locate a directory holding metaobjects.pkg.json.", "ERR_DEPENDENCY_MANIFEST_INVALID": "FR-023: a dependency's metaobjects.pkg.json fails its schema, names a different dependency, points at a missing or hash-mismatched artifact, or its artifact does not load standalone / does not declare exactly the listed packages and nodes.", "ERR_DEPENDENCY_SNAPSHOT_STALE": "FR-023: the committed snapshot does not match .metaobjects/deps.lock.json (lock missing, entry missing or extra, artifact missing, or hash mismatch) — run `meta deps sync`.", diff --git a/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/README.md b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/README.md new file mode 100644 index 000000000..dad174220 --- /dev/null +++ b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/README.md @@ -0,0 +1,26 @@ +# merge-conflict-unmarked-attr-redeclaration + +Two files declare the same `(type, package::name)` and set the same attribute to +different values. Neither marks itself `overlay: true`. **`ERR_MERGE_CONFLICT`.** + +This is FR5c's original case: two files that collided **without knowing about each +other**. The value still merges last-writer-wins so the loader sees one canonical tree; +the error is what tells a consumer the collision happened. + +## Why this fixture exists + +It took over the error branch from `overlay-attr-last-writer-wins`, which used to prove +the same thing — with an overlay that DID mark itself. FR-043 Amendment 2 ruled that +`overlay: true` **licenses** the override: the flag is the author saying "I know about +the other declaration and I mean to change it", and the loader already treats it +specially (find-or-throw versus create-or-find), so honouring it here makes it mean one +thing rather than two. + +Flipping that fixture alone would have deleted the only coverage of the accident case. +This fixture holds it, so the coverage MOVED rather than disappearing. The two are a +pair and should be read together: + +| fixture | overlay marked? | verdict | +|---|---|---| +| `overlay-attr-last-writer-wins` | yes | no error — the override is licensed | +| `merge-conflict-unmarked-attr-redeclaration` | **no** | `ERR_MERGE_CONFLICT` | diff --git a/fixtures/conformance/overlay-attr-last-writer-wins/expected-errors.json b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/expected-errors.json similarity index 83% rename from fixtures/conformance/overlay-attr-last-writer-wins/expected-errors.json rename to fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/expected-errors.json index 0963d569f..d5d7490ce 100644 --- a/fixtures/conformance/overlay-attr-last-writer-wins/expected-errors.json +++ b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/expected-errors.json @@ -6,7 +6,7 @@ "format": "merged", "files": [ "meta.commerce.json", - "meta.commerce.overlay.json" + "meta.commerce.second.json" ] } } diff --git a/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/input/meta.commerce.json b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/input/meta.commerce.json new file mode 100644 index 000000000..0d43e0d87 --- /dev/null +++ b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/input/meta.commerce.json @@ -0,0 +1,26 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Product", + "@title": "Product v1", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": "id" + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/input/meta.commerce.second.json b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/input/meta.commerce.second.json new file mode 100644 index 000000000..43af0a943 --- /dev/null +++ b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/input/meta.commerce.second.json @@ -0,0 +1,13 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Product", + "@title": "Product v2" + } + } + ] + } +} diff --git a/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/providers.json b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/providers.json new file mode 100644 index 000000000..d901a6611 --- /dev/null +++ b/fixtures/conformance/merge-conflict-unmarked-attr-redeclaration/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-documentation"] diff --git a/fixtures/conformance/overlay-adds-source/expected.json b/fixtures/conformance/overlay-adds-source/expected.json new file mode 100644 index 000000000..ff6cc59a4 --- /dev/null +++ b/fixtures/conformance/overlay-adds-source/expected.json @@ -0,0 +1,51 @@ +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "object.entity": { + "name": "User", + "children": [ + { + "field.uuid": { + "name": "id", + "@required": true + } + }, + { + "field.string": { + "name": "username", + "@maxLength": 64, + "@required": true + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ], + "@generation": "uuid" + } + }, + { + "source.rdb": { + "@role": "primary", + "@table": "iam_user" + } + }, + { + "index.lookup": { + "name": "ixUsername", + "@fields": [ + "username" + ] + } + } + ] + } + } + ] + } +} + diff --git a/fixtures/conformance/overlay-adds-source/input/meta.a-model.json b/fixtures/conformance/overlay-adds-source/input/meta.a-model.json new file mode 100644 index 000000000..f13a8da27 --- /dev/null +++ b/fixtures/conformance/overlay-adds-source/input/meta.a-model.json @@ -0,0 +1,17 @@ +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "object.entity": { + "name": "User", + "children": [ + { "field.uuid": { "name": "id", "@required": true } }, + { "field.string": { "name": "username", "@required": true, "@maxLength": 64 } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "uuid" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/overlay-adds-source/input/meta.b-db.json b/fixtures/conformance/overlay-adds-source/input/meta.b-db.json new file mode 100644 index 000000000..fcad4aed5 --- /dev/null +++ b/fixtures/conformance/overlay-adds-source/input/meta.b-db.json @@ -0,0 +1,17 @@ +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "object.entity": { + "name": "User", + "overlay": true, + "children": [ + { "source.rdb": { "@table": "iam_user", "@role": "primary" } }, + { "index.lookup": { "name": "ixUsername", "@fields": ["username"] } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/overlay-attr-last-writer-wins/README.md b/fixtures/conformance/overlay-attr-last-writer-wins/README.md new file mode 100644 index 000000000..1cc66bf25 --- /dev/null +++ b/fixtures/conformance/overlay-attr-last-writer-wins/README.md @@ -0,0 +1,24 @@ +# overlay-attr-last-writer-wins + +A second file re-opens `acme::Product` with `overlay: true` and sets `@title` to a +different value. **Last writer wins, and there is NO error.** + +## Why no error + +`overlay: true` **licenses** the override (FR-043 Amendment 2). The `ERR_MERGE_CONFLICT` +this fixture used to expect exists to catch two files that collided *without knowing +about each other*; the flag is the author saying "I know about the other declaration and +I mean to change it". The loader already treats it specially — `overlay: true` is +find-or-throw, a plain redeclaration is create-or-find — so honouring it here makes it +mean one coherent thing rather than two. + +The accident case did NOT stop being an error, it MOVED: see +[`merge-conflict-unmarked-attr-redeclaration`](../merge-conflict-unmarked-attr-redeclaration/), +which is the identical collision with neither file marked. The two are a pair and should +be read together — flipping this fixture without adding that one would have deleted the +only coverage of the case FR5c was written for. + +| fixture | overlay marked? | verdict | +|---|---|---| +| this one | yes | no error — the override is licensed | +| `merge-conflict-unmarked-attr-redeclaration` | no | `ERR_MERGE_CONFLICT` | diff --git a/fixtures/conformance/overlay-nested-requirement/README.md b/fixtures/conformance/overlay-nested-requirement/README.md new file mode 100644 index 000000000..496b7bdab --- /dev/null +++ b/fixtures/conformance/overlay-nested-requirement/README.md @@ -0,0 +1,27 @@ +# overlay-nested-requirement + +An adopter overlays a library requirement nested three deep — the `requirement.*` node +is not a root child, so addressing it means re-declaring its ancestors — and changes +`@status` while adding `@disposition` and `@notes`. + +**The tree merges correctly and the load is clean: right shape, no duplicates, the +adopter's values applied, zero errors.** + +This is the case FR-043 §5.5 rests on. Without it, an adopter could not disagree with a +library's requirement without ejecting the whole ledger, and the adaptation door §5.5 +describes would collapse to eject-only. + +## Two things it pins that are easy to get wrong + +1. **Every ancestor in the chain carries `overlay: true`, not just the leaf.** Left + plain, each ancestor is a same-shape redeclaration and emits + `WARN_DUPLICATE_DECLARATION` — three warnings to change one leaf in a depth-4 tree. + Marked, the load is silent. (`overlay-nested-under-plain-parent-base-later` uses the + unmarked shape deliberately, to pin what that costs.) +2. **Changing `@status` is an OVERRIDE, and it does not error.** It used to: FR5c's + `ERR_MERGE_CONFLICT` fired even under an explicit `overlay: true`, which made the one + move §5.5 asks an adopter to make look like a defect. FR-043 Amendment 2 ruled the + flag licenses the override. Adding an attribute the base never set (`@disposition`, + `@notes` here) was always clean; now retuning one is too. See + [`overlay-attr-last-writer-wins`](../overlay-attr-last-writer-wins/) and its unmarked + pair. diff --git a/fixtures/conformance/overlay-nested-requirement/expected.json b/fixtures/conformance/overlay-nested-requirement/expected.json new file mode 100644 index 000000000..57ecb50e0 --- /dev/null +++ b/fixtures/conformance/overlay-nested-requirement/expected.json @@ -0,0 +1,41 @@ +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "requirement.functional": { + "name": "accessControl", + "@counterexample": "A superuser recognised by username", + "@level": 2, + "@statement": "Who may do what is answered from stored grants, never from a name compared to a literal in code", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "grants", + "@counterexample": "A grant that cannot say where it applies", + "@level": 3, + "@statement": "A role is granted to a user either system-wide or within one group", + "@status": "live", + "children": [ + { + "requirement.functional": { + "name": "scopedGrantRequiresMembership", + "@counterexample": "A scoped grant naming a user who never joined the group", + "@disposition": "accepted", + "@level": 4, + "@notes": "The schema cannot express it; enforced in the service layer.", + "@statement": "A role granted within a group is held only by a member of that group", + "@status": "partial" + } + } + ] + } + } + ] + } + } + ] + } +} + diff --git a/fixtures/conformance/overlay-nested-requirement/input/meta.a-requirements.json b/fixtures/conformance/overlay-nested-requirement/input/meta.a-requirements.json new file mode 100644 index 000000000..0ca9505ac --- /dev/null +++ b/fixtures/conformance/overlay-nested-requirement/input/meta.a-requirements.json @@ -0,0 +1,38 @@ +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "requirement.functional": { + "name": "accessControl", + "@level": 2, + "@status": "live", + "@statement": "Who may do what is answered from stored grants, never from a name compared to a literal in code", + "@counterexample": "A superuser recognised by username", + "children": [ + { + "requirement.functional": { + "name": "grants", + "@level": 3, + "@status": "live", + "@statement": "A role is granted to a user either system-wide or within one group", + "@counterexample": "A grant that cannot say where it applies", + "children": [ + { + "requirement.functional": { + "name": "scopedGrantRequiresMembership", + "@level": 4, + "@status": "live", + "@statement": "A role granted within a group is held only by a member of that group", + "@counterexample": "A scoped grant naming a user who never joined the group" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/overlay-nested-requirement/input/meta.b-adopter.json b/fixtures/conformance/overlay-nested-requirement/input/meta.b-adopter.json new file mode 100644 index 000000000..abff3f14e --- /dev/null +++ b/fixtures/conformance/overlay-nested-requirement/input/meta.b-adopter.json @@ -0,0 +1,32 @@ +{ + "metadata.root": { + "package": "acme::iam", + "children": [ + { + "requirement.functional": { + "name": "accessControl", + "overlay": true, + "children": [ + { + "requirement.functional": { + "name": "grants", + "overlay": true, + "children": [ + { + "requirement.functional": { + "name": "scopedGrantRequiresMembership", + "overlay": true, + "@status": "partial", + "@disposition": "accepted", + "@notes": "The schema cannot express it; enforced in the service layer." + } + } + ] + } + } + ] + } + } + ] + } +} \ No newline at end of file diff --git a/fixtures/dependency-conformance/README.md b/fixtures/dependency-conformance/README.md index a50c3df98..460ec84d0 100644 --- a/fixtures/dependency-conformance/README.md +++ b/fixtures/dependency-conformance/README.md @@ -133,10 +133,25 @@ where §2.5 predicts. | a node removed/never existed; local `extends` targets it | `an-extends-whose-target-was-removed-fails` | `ERR_UNRESOLVED_SUPER` | | a node removed/never existed; local `field.object @objectRef` targets it | `a-reference-whose-target-was-removed-fails` | `ERR_UNRESOLVED_OBJECT_REF` | | a member's subtype changed (v1 `Customer.email` is `field.string`); local dotted `extends: Customer.email` from a `field.int` | `a-dotted-extends-whose-member-changed-subtype-fails` | `ERR_EXTENDS_TARGET_MISMATCH` | -| an attr the consumer's overlay also sets is now set differently upstream (base `@maxLength: 120`, overlay `@maxLength: 80`) | `an-overlay-attr-the-base-now-sets-differently-conflicts` | `ERR_MERGE_CONFLICT` | +| an attr the consumer REDECLARES without marking it, set differently upstream (base `@maxLength: 120`, redeclaration `@maxLength: 80`) | `an-unmarked-attr-redeclaration-over-a-dependency-conflicts` | `ERR_MERGE_CONFLICT` | | the dependency's own artifact file fails to parse | `a-dependency-file-error-names-the-dependency` | `ERR_MALFORMED_JSON`, naming `dep:acme-common/acme-common.metaobjects.json` | -All seven pass with **no source change** (`sdk/test/dependency-conformance.test.ts`). +One further case is the same collision with the overlay MARKED, and it is deliberately +**not** an error: + +| Upstream change / consumer construct | Case | Outcome | +|---|---|---| +| the same attr collision, but the consumer's declaration carries `overlay: true` | `an-overlay-attr-the-base-sets-differently-is-licensed` | loads clean; the consumer's value wins | + +That pair is FR-043 Amendment 2 reaching the dependency axis. `overlay: true` licenses +the override — the flag is the author saying "I know about the other declaration and I +mean to change it", and an overlay pinned against a dependency node is exactly that +statement. What guards a dependency is the hash lock plus `meta deps check` (upstream +moved) and `refuseUnownedPackages` (you declared a NEW node into their package), not the +merge-conflict error; the unmarked row above is what keeps the collision-by-accident case +covered here. + +All eight pass with **no source change** (`sdk/test/dependency-conformance.test.ts`). `an-overlay-whose-target-was-removed-fails` is the one case that rests on a genuine upstream removal rather than "never existed" — it loads `acme-common-v2-email-removed.json` (v1's `Customer` minus `email`) as the dependency diff --git a/fixtures/dependency-conformance/cases.json b/fixtures/dependency-conformance/cases.json index 0c9894ab3..93a009db9 100644 --- a/fixtures/dependency-conformance/cases.json +++ b/fixtures/dependency-conformance/cases.json @@ -966,7 +966,7 @@ "expectLoadError": "ERR_EXTENDS_TARGET_MISMATCH" }, { - "name": "an-overlay-attr-the-base-now-sets-differently-conflicts", + "name": "an-overlay-attr-the-base-sets-differently-is-licensed", "tree": { "metaobjects/meta.ov-conflict.json": "{\"metadata.root\":{\"package\":\"acme::common\",\"children\":[{\"object.entity\":{\"name\":\"Customer\",\"overlay\":true,\"children\":[{\"field.string\":{\"name\":\"email\",\"overlay\":true,\"@maxLength\":80}}]}}]}}" }, @@ -1005,6 +1005,51 @@ } } }, + "expectFiles": [ + ".metaobjects/deps/acme-common/acme-common.metaobjects.json", + "metaobjects/meta.ov-conflict.json" + ] + }, + { + "name": "an-unmarked-attr-redeclaration-over-a-dependency-conflicts", + "tree": { + "metaobjects/meta.ov-conflict.json": "{\"metadata.root\":{\"package\":\"acme::common\",\"children\":[{\"object.entity\":{\"name\":\"Customer\",\"children\":[{\"field.string\":{\"name\":\"email\",\"@maxLength\":80}}]}}]}}" + }, + "treeFiles": { + ".metaobjects/deps/acme-common/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "path": "../acme-common/metaobjects" + } + ] + }, + "lock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "path": "../acme-common/metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + }, "expectFiles": [ ".metaobjects/deps/acme-common/acme-common.metaobjects.json", "metaobjects/meta.ov-conflict.json" diff --git a/fixtures/generator-registry-conformance/README.md b/fixtures/generator-registry-conformance/README.md index bd21bd9af..30da7245d 100644 --- a/fixtures/generator-registry-conformance/README.md +++ b/fixtures/generator-registry-conformance/README.md @@ -16,12 +16,45 @@ conformance test asserts: expose a name whose `ports` array omits it. 3. **Tier agreement** — a name marked `tier: "neutral"` is flagged neutral in the port (owned by `meta docs`, not the recommended native suite). +4. **Layer agreement** — a name's `layer` matches the port's registry entry. Because all five ports validate against this one file, a **shared concept is spelled identically everywhere** (e.g. the REST surface is `routes` in every port, never `controller`/`router`). That cross-port spelling stability is the whole point. +## `layer` — the six values, and why six + +`layer` is the grouping an adopter (increasingly an LLM in their repo) **selects +by**. Codegen is opt-in: nothing runs until it is chosen, so the catalog's job is +to make the choice cheap, and `layer` is the axis it is cheap along. + +| layer | members | chosen by | +|---|---|---| +| `model` | entity, names, barrel, dto, value-object | app shape | +| `persistence` | queries, db-context, repository, exposed-table, relations, stored-proc | app shape | +| `api` | routes, routes-hono, filter-allowlist, validator, spring-config | app shape | +| `client` | form, hooks, grid, grid-hook | app shape | +| `docs` | docs, mermaid-er, api-docs | on by default (`meta docs`) | +| `capability` | prompt-render, output-parser, output-prompt, extractor, render-helper, payload, trace-helper, requirement-tests, shared-model, template, callable | `meta gen --list --probe` | + +Six, not ten. An earlier draft split `capability` into `trace` / `requirements` / +`publish` / `primitive`, each with **one member** — a layer with one member does +no grouping work, and the split conflated two different kinds of choice. The +first four layers are app-shape decisions a builder makes. `capability` holds the +ones the **model has already made**: nobody picks `prompt-render` by browsing a +taxonomy, they pick it because they declared a `template.prompt`. `--probe` +constructs every generator and dry-runs it against the real model, so +`output-parser: 3, callable: 0, requirement-tests: 7` is strictly better +information than a category name — and unlike a category name it cannot go stale, +because it does not describe the generators, it runs them. + +`capability` looking like a large undifferentiated bucket is therefore the point, +not a defect to be tidied. Do not add a seventh value to make it look tidier. + +`layer` rather than a reuse of "tier", which is already taken twice +(native/neutral in ADR-0020, and server/UI elsewhere). + ## Shared (cross-port) names These concepts MUST use the same stable name wherever a port implements them: diff --git a/fixtures/generator-registry-conformance/registry.json b/fixtures/generator-registry-conformance/registry.json index c887cedf8..52cc0ac41 100644 --- a/fixtures/generator-registry-conformance/registry.json +++ b/fixtures/generator-registry-conformance/registry.json @@ -1,152 +1,211 @@ { - "$comment": "CANONICAL generator stable-name manifest (ADR-0021 D3). The single cross-port source of truth for generator stable names. Every port's generator registry is conformance-tested against this file: (1) every generator a port exposes MUST use a name listed here; (2) for each generator, the `ports` array lists exactly the ports expected to expose it (presence is checked both ways); (3) because all ports validate against this one file, a shared concept (e.g. `routes`) is spelled identically everywhere. Port ids: typescript, csharp, java, kotlin, python. `tier`: native (idiomatic per-port code) or neutral (Tier-2, owned by `meta docs`). Adding/renaming a generator = edit THIS file + the port registry together; the conformance gate fails otherwise.", + "$comment": "CANONICAL generator stable-name manifest (ADR-0021 D3). The single cross-port source of truth for generator stable names. Every port's generator registry is conformance-tested against this file: (1) every generator a port exposes MUST use a name listed here; (2) for each generator, the `ports` array lists exactly the ports expected to expose it (presence is checked both ways); (3) because all ports validate against this one file, a shared concept (e.g. `routes`) is spelled identically everywhere. Port ids: typescript, csharp, java, kotlin, python. `tier`: native (idiomatic per-port code) or neutral (Tier-2, owned by `meta docs`). `layer`: one of model, persistence, api, client, docs, capability — the grouping an adopter SELECTS BY, gated in every port exactly as `tier` is (see README.md). Adding/renaming a generator = edit THIS file + the port registry together; the conformance gate fails otherwise.", "ports": ["typescript", "csharp", "java", "kotlin", "python"], "generators": { "entity": { "concept": "Per-entity model/class — the entity module (table-backed or value object).", "tier": "native", + "layer": "model", "ports": ["typescript", "csharp", "java", "kotlin", "python"] }, "routes": { "concept": "Per-entity REST endpoint surface (controllers / routes / router).", "tier": "native", + "layer": "api", "ports": ["typescript", "csharp", "java", "kotlin", "python"] }, "output-parser": { "concept": "Per-template tolerant output parser (recover-on-receipt).", "tier": "native", + "layer": "capability", "ports": ["typescript", "csharp", "java", "kotlin", "python"] }, "output-prompt": { "concept": "Per-template output-format prompt fragment generator.", "tier": "native", + "layer": "capability", "ports": ["typescript", "csharp", "java", "kotlin", "python"] }, "render-helper": { "concept": "Per-template.output render helper (document/email typed wrappers).", "tier": "native", + "layer": "capability", "ports": ["typescript", "csharp", "java", "kotlin", "python"] }, "extractor": { "concept": "Per-template strict typed extract helper (strict payload extraction).", "tier": "native", + "layer": "capability", "ports": ["typescript", "csharp", "java", "kotlin", "python"] }, "template": { "concept": "Generic Mustache template primitive (walk + template -> files).", "tier": "native", + "layer": "capability", "ports": ["typescript", "csharp", "java", "python"] }, "filter-allowlist": { "concept": "Per-entity REST filter allowlist (queryable-field guard).", "tier": "native", + "layer": "api", "ports": ["csharp", "java", "kotlin", "python"] }, "payload": { "concept": "Per-template payload value object (the strict payload type).", "tier": "native", + "layer": "capability", "ports": ["csharp", "java", "kotlin", "python"] }, "queries": { "concept": "Per-entity typed query helpers (findById/create/...).", "tier": "native", + "layer": "persistence", "ports": ["typescript"] }, "callable": { "concept": "Per-entity callable wrapper for a callable source (storedProc / tableFunction); in TS it also covers the service surface wrapping the query helpers.", "tier": "native", + "layer": "capability", "ports": ["typescript", "csharp"] }, "routes-hono": { "concept": "Per-entity Hono CRUD routes.", "tier": "native", + "layer": "api", "ports": ["typescript"] }, "barrel": { "concept": "Single index module re-exporting every generated entity module.", "tier": "native", + "layer": "model", "ports": ["typescript"] }, "names": { "concept": "Per-entity physical database name constants (table/view name, schema, column names).", "tier": "native", + "layer": "model", "ports": ["typescript", "csharp", "kotlin", "python", "java"] }, "prompt-render": { "concept": "Per-template prompt-render helper over the render engine.", "tier": "native", + "layer": "capability", "ports": ["typescript"] }, "api-docs": { "concept": "Per-entity/template SDK API reference (generated-code API; human + agent forms).", "tier": "native", + "layer": "docs", "ports": ["typescript"] }, "trace-helper": { "concept": "Per-entity typed record/call trace helpers (extract + buildLlmCallRow + persist; LlmCallBase-derived entities only).", "tier": "native", + "layer": "capability", "note": "TS emits record + call; Java (codegen-spring) emits record only — the Java LlmClient seam is BYO/vendor-neutral and not ported (ADR-0024). Python emits record_ only (same BYO-caller rationale).", "ports": ["typescript", "java", "python"] }, "db-context": { "concept": "Single EF Core DbContext binding every generated entity.", "tier": "native", + "layer": "persistence", "ports": ["csharp"] }, "repository": { "concept": "Per-entity repository seam (Java: Spring Data interface; Kotlin: an open Exposed persistence base with CRUD + patch bodies).", "tier": "native", + "layer": "persistence", "ports": ["java", "kotlin"] }, "dto": { "concept": "Per-entity Spring DTO.", "tier": "native", + "layer": "model", "ports": ["java"] }, "value-object": { "concept": "Per-value-object Spring record with jakarta constraints (the DTO/Patch bind target for a field.object jsonb column).", "tier": "native", + "layer": "model", "ports": ["java"] }, "exposed-table": { "concept": "Per-entity Kotlin Exposed table object.", "tier": "native", + "layer": "persistence", "ports": ["kotlin"] }, "relations": { "concept": "Cross-entity relationship helpers.", "tier": "native", + "layer": "persistence", "ports": ["kotlin"] }, "spring-config": { "concept": "Spring wiring/configuration for the generated surface.", "tier": "native", + "layer": "api", "ports": ["kotlin"] }, "stored-proc": { "concept": "Stored-procedure binding helpers.", "tier": "native", + "layer": "persistence", "ports": ["kotlin"] }, "validator": { "concept": "Per-entity input validator.", "tier": "native", + "layer": "api", "ports": ["kotlin"] }, "shared-model": { "concept": "FR-023: a publisher's flattened shared-model artifact + manifest for a consumer's `meta deps sync`.", "tier": "native", + "layer": "capability", + "ports": ["typescript"] + }, + "form": { + "concept": "Per-entity React form component over the generated Zod schema.", + "tier": "native", + "layer": "client", + "ports": ["typescript"] + }, + "hooks": { + "concept": "Per-entity TanStack Query hooks (useEntity / useEntities / useCreate / useUpdate / useDelete).", + "tier": "native", + "layer": "client", + "ports": ["typescript"] + }, + "grid": { + "concept": "Per-entity TanStack Table column definitions, from a layout.dataGrid declaration.", + "tier": "native", + "layer": "client", + "ports": ["typescript"] + }, + "grid-hook": { + "concept": "Per-entity server-driven grid state hook (sort/filter/page) over the generated columns.", + "tier": "native", + "layer": "client", + "ports": ["typescript"] + }, + "requirement-tests": { + "concept": "Per-requirement test stub, one per requirement.functional claim in the ledger.", + "tier": "native", + "layer": "capability", "ports": ["typescript"] }, "docs": { "concept": "Neutral per-entity / per-template Markdown documentation pages.", "tier": "neutral", + "layer": "docs", "note": "Owned by `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended native suite.", "ports": ["typescript"] }, "mermaid-er": { "concept": "Neutral Mermaid ER diagram of the entity/relationship model.", "tier": "neutral", + "layer": "docs", "note": "Neutral artifact owned by the docs engine (ADR-0020); surfaced via `meta docs`.", "ports": ["typescript"] } diff --git a/library/ai/db.yaml b/library/ai/db.yaml new file mode 100644 index 000000000..2e39b8f92 --- /dev/null +++ b/library/ai/db.yaml @@ -0,0 +1,16 @@ +# library/ai/db.yaml — the DB PERSISTENCE layer for metaobjects::ai. +# +# Opted into as `"ai/db"`, which IMPLIES `"ai"`: `LlmCall` is declared in model.yaml and +# this file only re-opens it, so without the core layer the overlay has no target. +# +# `LlmCall` is the concrete, table-backed instance of the abstract envelope. An adopter +# who wants their OWN table (a different name, extra columns, a different id strategy) +# extends `LlmCallBase` in their own metadata and never opts into this layer at all. +metadata: + package: metaobjects::ai + children: + - object.entity: + name: LlmCall + overlay: true + children: + - source.rdb: { table: llm_call, role: primary } diff --git a/library/ai/library.json b/library/ai/library.json new file mode 100644 index 000000000..da94fff91 --- /dev/null +++ b/library/ai/library.json @@ -0,0 +1,20 @@ +{ + "$comment": "Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.", + "name": "ai", + "kind": "feature", + "stability": "stable", + "since": "0.20.0", + "description": "The LLM-call trace envelope: what was asked, what came back, what it cost, how long it took.", + "useWhen": "the application calls a language model and someone will ask what it cost or why a call failed", + "packages": ["metaobjects::ai"], + "layers": { + "": { "refs": ["ai/model", "ai/requirements"], "description": "the core model and its requirements — sourceless, so it adds no tables" }, + "db": { "refs": ["ai/db"], "description": "the concrete llm_call table" } + }, + "generators": [ + { "name": "trace-helper", "anchor": "metaobjects::ai::LlmCallBase" } + ], + "runtime": { + "typescript": ["@metaobjectsdev/runtime-ts"] + } +} diff --git a/library/ai/llm-call.yaml b/library/ai/model.yaml similarity index 56% rename from library/ai/llm-call.yaml rename to library/ai/model.yaml index 4001b17c0..619d7ad24 100644 --- a/library/ai/llm-call.yaml +++ b/library/ai/model.yaml @@ -1,6 +1,16 @@ -# library/ai/llm-call.yaml -# MetaObjects-shipped standard metadata. Adopters opt in via the loader's -# `libraries: ["ai"]` option, then `extends: "metaobjects::ai::LlmCallBase"`. +# library/ai/model.yaml — the CORE layer: the LLM-call trace envelope. +# +# Adopters opt in via `libraries: ["ai"]`, then `extends: "metaobjects::ai::LlmCallBase"`. +# +# This layer declares NO `source.rdb`, so opting into `"ai"` alone adds zero tables and +# zero generated code — the design is present and resolvable, and nothing else happens +# until the adopter adds `"ai/db"`. See library/iam/model.yaml for the full rationale. +# +# This file was split out of the former `library/ai/llm-call.yaml`, which shipped the +# abstract base and a concrete `LlmCall` carrying `source.rdb` together. That was +# recorded as an accepted wart on the grounds that splitting would change what existing +# `ai` adopters get; a sweep of the estate found there are none, so it was closed rather +# than documented (FR-043 Amendment 1). metadata: package: metaobjects::ai children: @@ -29,6 +39,6 @@ metadata: - object.entity: name: LlmCall extends: metaobjects::ai::LlmCallBase + description: The concrete trace row. Its `source.rdb` lives in db.yaml, so opting into the core layer alone declares the shape without proposing a table. children: - - source.rdb: { table: llm_call, role: primary } - identity.primary: { name: id, fields: ["spanId"] } diff --git a/library/ai/requirements.yaml b/library/ai/requirements.yaml new file mode 100644 index 000000000..ca9813247 --- /dev/null +++ b/library/ai/requirements.yaml @@ -0,0 +1,112 @@ +# library/ai/requirements.yaml — what the LLM-call trace envelope PROMISES. +# +# A RETROFIT, not new design: llm-call.yaml landed 2026-06-03 and `requirement.functional` +# first appears 2026-08-11, so the library could not have carried requirements when it was +# written. That is why this file is worth reading as a worked example — it shows what +# declaring the design of something that already exists actually turns up. +# +# The entry that earns its keep is `typedIo`, honestly `partial` + `accepted`: the library +# declares the ENVELOPE and the adopter declares the typed VO columns. Recording that seam +# in the ledger is where an agent meets it, before adding a fourth trace column. +# +# HIERARCHY IS NESTING, and the L4/L5 split is grain. An L4 names the OBJECT it is about; +# the fields that carry it hang off it as an L5 child. Writing the fields at L4 is +# ERR_REQUIREMENT_L4_NOT_OBJECT, and writing the concerns as SIBLINGS of the L2 leaves the +# L2 claiming nothing — both of which this file did until the standalone verify gate +# existed (`cli/test/shipped-library-verify.test.ts`). +metadata: + package: metaobjects::ai + children: + - requirement.functional: + name: llmTracing + level: 2 + status: live + statement: Every call to a language model leaves a row that says what was asked, what came back, what it cost and how long it took. + counterexample: A spend figure nobody can attribute to a call. + description: The segment this library covers. Its three children below are the concerns it decomposes into. + children: + - requirement.functional: + name: envelope + level: 4 + status: live + statement: A trace row identifies its call and its place in a trace — trace, span, parent span, session, call type, system. + counterexample: A log line that cannot be joined to the request that produced it. + implementedBy: [LlmCallBase] + children: + - requirement.functional: + name: traceAddressing + level: 5 + status: live + statement: The four addressing columns are declared on the base — trace, span, parent span and session. + counterexample: A row whose place in a trace is inferred from insertion order. + description: >- + The member grain exists here so the claim RESOLVES against the fields + themselves: renaming or dropping one of them dangles this reference and + fails the build, which naming the object alone would not. + implementedBy: [LlmCallBase.traceId, LlmCallBase.spanId, LlmCallBase.parentSpanId, LlmCallBase.sessionId] + + - requirement.functional: + name: accounting + level: 4 + status: live + statement: A trace row carries the tokens in, the tokens out, and the cost in integer minor units. + counterexample: A cost stored as a float. + description: >- + `field.currency` — integer minor units on the wire, always. Float arithmetic for + money is forbidden by the cross-port wire contract, and a spend total is exactly + the sum that exposes it. + implementedBy: [LlmCallBase] + children: + - requirement.functional: + name: tokenAndCostColumns + level: 5 + status: live + statement: Tokens in, tokens out and cost are three declared columns, the cost a field.currency. + counterexample: A cost column declared as a double. + implementedBy: [LlmCallBase.inputTokens, LlmCallBase.outputTokens, LlmCallBase.costMinor] + + - requirement.functional: + name: typedIo + level: 4 + status: partial + disposition: accepted + statement: The request and response bodies are stored as structured jsonb, not as opaque text. + counterexample: A prompt stored as a string nobody can query a field out of. + notes: >- + The library declares the two columns as generic jsonb with no `@objectRef`, + because it cannot know the adopter's request/response shape. Typing them is the + ADOPTER's move: declare an `object.value` and overlay the field with + `@objectRef` + `@storage: jsonb`. This is the seam ADR-0024 drew, recorded here + rather than in prose so it is in the ledger an agent reads before adding a + fourth trace column of its own. + implementedBy: [LlmCallBase] + children: + - requirement.functional: + name: jsonbBodies + level: 5 + status: live + statement: The request and response bodies are declared as jsonb columns on the base. + counterexample: A prompt stored in a text column. + description: >- + `live` where its parent is `partial`, and the split is the point: the + COLUMNS are shipped and this claim is fully realised; what is outstanding + is the TYPING of them, which is the parent's gap and the adopter's move. + implementedBy: [LlmCallBase.llmRequest, LlmCallBase.llmResponse] + + - requirement.architectural: + name: traceRowsCarryTiming + status: live + statement: Every trace row records when the call started and how long it took. + counterexample: A latency figure derived from log timestamps after the fact. + description: >- + Architectural, so it propagates down `extends` to every adopter entity deriving + from LlmCallBase — which is the point: an adopter's own trace table is claimed + by this requirement for free, and dropping the columns breaks the build. + implementedBy: [LlmCallBase] + + - requirement.architectural: + name: traceRowsCarryOutcome + status: live + statement: Every trace row records how the call ended — a status, a finish reason, and the error detail when there was one. + counterexample: A failed call indistinguishable from one that never happened. + implementedBy: [LlmCallBase] diff --git a/library/iam/db.yaml b/library/iam/db.yaml new file mode 100644 index 000000000..ac4021d1d --- /dev/null +++ b/library/iam/db.yaml @@ -0,0 +1,81 @@ +# library/iam/db.yaml — the DB PERSISTENCE layer for metaobjects::iam. +# +# Opted into as `"iam/db"`, which IMPLIES `"iam"`: this file is nothing but +# `overlay: true` redeclarations, and an overlay whose target was never declared is +# ERR_OVERLAY_NO_TARGET. +# +# It carries exactly two kinds of child — `source.rdb` and `index.lookup` — and nothing +# else. The field set, the identities and the relationships all live in model.yaml, +# because they are the DESIGN; what lives here is where the rows go and which lookups are +# worth an index. Add a field here and the core layer stops being the whole model, which +# is the thing the split exists to guarantee. +# +# Physical names are `iam_`-prefixed. Two reasons, both real: `user` and `group` are +# reserved words in Postgres, and an adopter very likely has tables of their own by those +# names. A library that collides on a table name is a library nobody can adopt. +metadata: + package: metaobjects::iam + children: + - object.entity: + name: User + overlay: true + children: + - source.rdb: { table: iam_user, role: primary } + + - object.entity: + name: GroupType + overlay: true + children: + - source.rdb: { table: iam_group_type, role: primary } + + - object.entity: + name: Group + overlay: true + children: + - source.rdb: { table: iam_group, role: primary } + # Nesting is walked parent-ward constantly; the FK alone gives no index. + - index.lookup: { name: ixParent, fields: [parentId] } + + - object.entity: + name: Role + overlay: true + children: + - source.rdb: { table: iam_role, role: primary } + + - object.entity: + name: Permission + overlay: true + children: + - source.rdb: { table: iam_permission, role: primary } + + - object.entity: + name: GroupMember + overlay: true + children: + - source.rdb: { table: iam_group_member, role: primary } + # The composite PK covers (userId, groupId), so "who is in this group?" — + # the other direction — has no index without this one. Same reasoning for + # every ixSecond below. + - index.lookup: { name: ixGroup, fields: [groupId] } + + - object.entity: + name: RolePermission + overlay: true + children: + - source.rdb: { table: iam_role_permission, role: primary } + - index.lookup: { name: ixPermission, fields: [permissionId] } + + - object.entity: + name: UserRole + overlay: true + children: + - source.rdb: { table: iam_user_role, role: primary } + - index.lookup: { name: ixRole, fields: [roleId] } + + - object.entity: + name: GroupMemberRole + overlay: true + children: + - source.rdb: { table: iam_group_member_role, role: primary } + # "who holds this role in this group?" — the scoped-grant read. + - index.lookup: { name: ixGroupRole, fields: [groupId, roleId] } diff --git a/library/iam/library.json b/library/iam/library.json new file mode 100644 index 000000000..39fcc18a5 --- /dev/null +++ b/library/iam/library.json @@ -0,0 +1,16 @@ +{ + "$comment": "Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.", + "name": "iam", + "kind": "feature", + "stability": "preview", + "since": "1.1.0", + "description": "Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group.", + "useWhen": "the application has people who log in and things some of them may not do", + "packages": ["metaobjects::iam"], + "layers": { + "": { "refs": ["iam/model", "iam/requirements"], "description": "the core model and its requirements — sourceless, so it adds no tables" }, + "db": { "refs": ["iam/db"], "description": "nine tables, iam_-prefixed, plus the lookup indexes the composite keys do not cover" } + }, + "generators": [], + "runtime": {} +} diff --git a/library/iam/model.yaml b/library/iam/model.yaml new file mode 100644 index 000000000..21e39908d --- /dev/null +++ b/library/iam/model.yaml @@ -0,0 +1,157 @@ +# library/iam/model.yaml — the CORE layer: identity and access management. +# +# Adopters opt in via `libraries: ["iam"]` in .metaobjects/config.json. +# +# This layer declares NO `source.rdb`, and that is the whole point of the split. A +# sourceless object is inert by a contract that already ships: migrate skips an object +# with no writable source, and codegen emits no route, queries, hooks, grid or form for +# one (both citing #248 — persistability derives from source presence, never from the +# object subtype). It still gets a type-only interface, so `extends` and reference work. +# +# So `libraries: ["iam"]` adds ZERO tables and ZERO generated code. What an adopter gains +# is the design being present and resolvable: an agent working in the repo knows the +# capability exists and can draw on it, and nothing else happens until the adopter adds +# `"iam/db"`. +# +# Authoring discipline (FR-043 §3.1), so the departures are visible: +# - `field.uuid` + `generation: uuid` on principals; composite ASSIGNED keys on +# junctions. Never `increment` — a library cannot know the adopter's id strategy. +# - Physical names carry the `iam_` prefix (in db.yaml): `user` and `group` are +# reserved words in Postgres, and an adopter has tables of their own. +# - No adopter-facing profile data. That arrives by `overlay: true`. +# - No credentials. See requirements.yaml → `noCredentialsOnUser`. +metadata: + package: metaobjects::iam + children: + - object.entity: + name: IamBase + abstract: true + description: Shared shape of every iam principal and definition — a stable uuid plus change timestamps. Junctions do not extend it; they are addressed by their participants. + children: + - field.uuid: { name: id, required: true } + - field.timestamp: { name: createdAt, autoSet: onCreate } + - field.timestamp: { name: updatedAt, autoSet: onUpdate } + + - object.entity: + name: User + extends: IamBase + description: A person or service account that can be granted access. Carries no authentication secret of any kind — see the noCredentialsOnUser requirement. + children: + - field.string: { name: username, required: true, maxLength: 64, filterable: true } + - field.string: { name: email, required: true, maxLength: 254, stringFormat: email, filterable: true } + - field.string: { name: displayName, maxLength: 120 } + # NOT `filterable: true`, deliberately. The loader warns when a filterable + # field is in no identity — filtering on it sequential-scans — and a library + # must not ship a warning to every adopter. `username` and `email` carry it + # because they have identity.secondary; `status` does not. An adopter who + # wants to filter on status overlays `filterable` AND an index together, + # which is exactly what the layer split is for. + - field.enum: { name: status, required: true, values: [invited, active, suspended, closed], default: active } + - field.timestamp: { name: emailVerifiedAt } + - field.timestamp: { name: lastSeenAt } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqUsername, fields: [username] } + - identity.secondary: { name: uqEmail, fields: [email] } + - relationship.association: { name: groups, objectRef: Group, cardinality: many, through: GroupMember } + - relationship.association: { name: roles, objectRef: Role, cardinality: many, through: UserRole } + + - object.entity: + name: GroupType + extends: IamBase + description: What KIND of group this is — a team, a tenant, a project. An entity rather than an enum, because "which roles may be held in this kind of group" is data an adopter extends, and an enum's values cannot be extended by overlay. + children: + - field.string: { name: key, required: true, maxLength: 64 } + - field.string: { name: name, required: true, maxLength: 120 } + - field.string: { name: description, maxLength: 500 } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + + - object.entity: + name: Group + extends: IamBase + description: A nestable collection of users, of a declared GroupType. Nesting is by parentId; acyclicity is an invariant the schema cannot express — see the acyclicGroupNesting requirement. + children: + - field.uuid: { name: groupTypeId, required: true } + - field.uuid: { name: parentId } + - field.string: { name: key, required: true, maxLength: 64 } + # Not filterable for the same reason as User.status above. + - field.string: { name: name, required: true, maxLength: 120 } + - field.string: { name: description, maxLength: 500 } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict } + - identity.reference: { name: fkParent, fields: [parentId], references: Group, onDelete: restrict } + + - object.entity: + name: Role + extends: IamBase + description: A reusable bundle of permissions. Code never compares a role NAME to a literal — it asks whether a user holds a permission, and the mapping is data. + children: + - field.string: { name: key, required: true, maxLength: 64 } + - field.string: { name: name, required: true, maxLength: 120 } + - field.string: { name: description, maxLength: 500 } + - field.uuid: { name: groupTypeId, description: "When set, this role may be held only within groups of this type; absent means grantable anywhere." } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict } + - relationship.association: { name: permissions, objectRef: Permission, cardinality: many, through: RolePermission } + + - object.entity: + name: Permission + extends: IamBase + description: "The assignable unit — a stable : key the application checks against. An entity, not an enum, on ADR-0037's own reasoning: it has its own identity, its own lifecycle, and a junction with real foreign keys." + children: + - field.string: { name: key, required: true, maxLength: 128, description: "Stable : key the application checks against." } + - field.string: { name: description, maxLength: 500 } + - identity.primary: { name: pk, fields: [id], generation: uuid } + - identity.secondary: { name: uqKey, fields: [key] } + + # ---- grant surface: every grant is a row, addressed by its participants ---- + # + # Junctions do NOT extend IamBase: they have no identity of their own, and adding a + # surrogate uuid to a row whose identity IS its participants invites a duplicate. + + - object.entity: + name: GroupMember + description: A user's membership of a group. + children: + - field.uuid: { name: userId, required: true } + - field.uuid: { name: groupId, required: true } + - field.timestamp: { name: joinedAt, autoSet: onCreate } + - identity.primary: { name: pk, fields: [userId, groupId], generation: assigned } + - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade } + - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade } + + - object.entity: + name: RolePermission + description: A permission granted by a role. + children: + - field.uuid: { name: roleId, required: true } + - field.uuid: { name: permissionId, required: true } + - identity.primary: { name: pk, fields: [roleId, permissionId], generation: assigned } + - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: cascade } + - identity.reference: { name: fkPermission, fields: [permissionId], references: Permission, onDelete: restrict } + + - object.entity: + name: UserRole + description: A system-wide grant of a role to a user. + children: + - field.uuid: { name: userId, required: true } + - field.uuid: { name: roleId, required: true } + - field.timestamp: { name: grantedAt, autoSet: onCreate } + - identity.primary: { name: pk, fields: [userId, roleId], generation: assigned } + - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade } + - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict } + + - object.entity: + name: GroupMemberRole + description: A grant of a role to a user WITHIN one group. Three foreign keys, so it is not an M:N @through junction (which must declare exactly two identity.reference children); it is read by explicit finders. + children: + - field.uuid: { name: userId, required: true } + - field.uuid: { name: groupId, required: true } + - field.uuid: { name: roleId, required: true } + - field.timestamp: { name: grantedAt, autoSet: onCreate } + - identity.primary: { name: pk, fields: [userId, groupId, roleId], generation: assigned } + - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade } + - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade } + - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict } diff --git a/library/iam/requirements.yaml b/library/iam/requirements.yaml new file mode 100644 index 000000000..0fb7cf092 --- /dev/null +++ b/library/iam/requirements.yaml @@ -0,0 +1,156 @@ +# library/iam/requirements.yaml — what this library's design PROMISES. +# +# This is what makes iam a library rather than a schema snippet. Without requirements an +# adopter gets nine tables; with them they get nine tables plus a build that is held to +# "no authorization decision is hard-wired to a name", which no snippet can do. +# +# Two reading rules, both load-bearing: +# +# `live` here means "the model AS SHIPPED realises this" — never "your application +# does". A ledger binds to model nodes; runtime guarantees are the runtime's tests, and +# this library does not invent a way to point a requirement at code (@verifiedBy was +# retired for exactly that). Behaviour the model cannot carry ships as `partial` + +# `disposition: accepted` with a notes sentence naming what the adopter must do. +# +# The functional tree roots at L2, not L1. L1 is the adopter's SOLUTION, and a library +# is by definition a segment of someone else's. Architectural claims ship flat. +# +# HIERARCHY IS NESTING, and the L4/L5 split is grain. The concerns are CHILDREN of the L2 +# rather than its siblings, and an L4 names the OBJECT it is about while the field that +# carries it hangs off it as an L5 child. Written flat, the L2 claims nothing in its whole +# subtree; written at L4, a field reference is ERR_REQUIREMENT_L4_NOT_OBJECT. Both shipped +# here until the standalone verify gate existed (`cli/test/shipped-library-verify.test.ts`). +metadata: + package: metaobjects::iam + children: + # ---- functional: the L2 segment and the concerns nested under it -------- + - requirement.functional: + name: accessControl + level: 2 + status: live + statement: Who may do what is answered from stored grants, never from a name compared to a literal in code. + counterexample: A branch that reads `if (user.role === "admin")`. + description: The segment this library covers. The concerns beneath it are what it decomposes into. + children: + - requirement.functional: + name: identity + level: 4 + status: live + statement: A person or service account is represented once, addressed by a uuid, and reachable by username or email. + counterexample: Two rows for the same person because the email changed. + implementedBy: [User] + + - requirement.functional: + name: grouping + level: 4 + status: live + statement: Users are collected into typed, nestable groups, and the kind of group is data rather than a hard-coded set. + counterexample: A `teamOrTenant` boolean. + implementedBy: [Group, GroupType, GroupMember] + + - requirement.functional: + name: acyclicGroupNesting + level: 4 + status: partial + disposition: accepted + statement: A group is never its own ancestor. + counterexample: Two groups each naming the other as parent. + notes: >- + The schema cannot express this — a self-referencing FK admits a cycle, and the + only relational forms that would catch it (a recursive CHECK, a closure table + maintained by trigger) are DB-specific and would not survive three dialects. + The adopter enforces it where the write happens. Recorded rather than omitted + so an agent reading the ledger before adding a parent-setting endpoint sees the + obligation. + implementedBy: [Group] + + - requirement.functional: + name: grants + level: 4 + status: live + statement: A role is granted to a user either system-wide or scoped to one group, and both are ordinary rows. + counterexample: A nullable `groupId` on one grant table, where NULL means "everywhere". + description: >- + Two junctions, not one with a nullable scope. A NULL in a unique key is DISTINCT + from every other NULL in SQL, so a nullable-scope design lets the same global + grant be inserted twice; the fix needs a partial index whose expression carries + a physical column name. Two composite-keyed tables need no escape hatch and + survive three dialects and five ports unchanged. + implementedBy: [UserRole, GroupMemberRole] + + - requirement.functional: + name: roleScopedToGroupType + level: 4 + status: partial + disposition: accepted + statement: A role bound to a group type is granted only within groups of that type. + counterexample: A "tenant admin" role granted inside a project group. + notes: >- + Expressing this relationally needs the grant row to carry the group's type and + a composite FK back to (group, type) — three foreign keys deep, unverified + across five ports' DDL and ORM paths. The adopter checks it at the point of + grant. The declared half is the L5 child below; the enforcement is not. + implementedBy: [Role, GroupMemberRole] + children: + - requirement.functional: + name: roleDeclaresItsGroupType + level: 5 + status: live + statement: A role declares the group type it is bound to, as a nullable reference. + counterexample: A role whose intended scope is recoverable only from its name. + description: >- + `live` where its parent is `partial`, and the split is grain as much as + verdict: the DECLARATION is shipped and resolves against the field itself, + so dropping the column fails the build — while the ENFORCEMENT, which no + schema here can carry, stays the parent's accepted gap. + implementedBy: [Role.groupTypeId] + + - requirement.functional: + name: decision + level: 4 + status: live + statement: An authorization decision is the question "does this user hold this permission key", answered from rows. + counterexample: A hard-coded list of usernames that bypass a check. + implementedBy: [Permission, RolePermission] + + # ---- architectural: prohibitions in force -------------------------------- + + - requirement.architectural: + name: grantsAreRows + status: live + statement: A grant exists only as a stored row; nothing is granted by naming, position or convention. + counterexample: A superuser recognised by username. + implementedBy: [UserRole, GroupMemberRole, RolePermission, GroupMember] + + - requirement.architectural: + name: noCredentialsOnUser + status: live + statement: A user row carries no authentication secret — no password, no hash, no knowledge-based question or answer. + counterexample: A password or secret-answer column on the user table. + description: >- + Authentication is a separate capability with an entity per factor; this library + is identity and authorization only. + notes: >- + This is the one thing every reader of a user table proposes adding, and a real + legacy model of this shape stored a length-bounded plaintext password and a + knowledge-based secret pair on the user row. Stating it as a prohibition IN + FORCE — claimable, and rendered on agent/requirements.md — is what stops an + agent extending "the user model" from re-deriving it on sight. It is + `architectural`, not `retired`: retired is chartered for a capability built + here and removed, and this library never built one. + implementedBy: [User] + + - requirement.architectural: + name: principalDeletionRevokesGrants + status: live + statement: Deleting a user or group removes its grants; deleting a role or permission still in use is refused. + counterexample: A grant row pointing at a user who no longer exists. + description: The referential rule in one sentence — cascade from a principal, restrict from a definition. + implementedBy: [GroupMember, UserRole, GroupMemberRole, RolePermission] + + - requirement.architectural: + name: stableIdentifiers + status: live + statement: Every principal and definition is addressed by a uuid that never changes; every grant by its participants. + counterexample: A group referenced by its display name. + implementedBy: [IamBase] diff --git a/scripts/generate-embedded-library.ts b/scripts/generate-embedded-library.ts index 015ea5601..926bf8872 100644 --- a/scripts/generate-embedded-library.ts +++ b/scripts/generate-embedded-library.ts @@ -91,6 +91,30 @@ if (files.length === 0) { process.exit(1); } +// The MANIFESTS ride the same embed, in their own map keyed by library NAME. +// +// Their own map rather than another entry in the ref map, because a ref is +// "path under library/ minus .yaml" and every reader appends `.yaml` back. Folding a +// `.json` into that would mean a special case at every call site in four ports; one more +// constant means none. +const manifests = readdirSync(libDir, { withFileTypes: true }) + .filter((e) => e.isDirectory()) + .map((e) => ({ name: e.name, path: join(libDir, e.name, "library.json") })) + .filter((m) => existsSync(m.path)) + .map((m) => ({ name: m.name, content: readFileSync(m.path, "utf-8") })) + .sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)); + +// A library directory with no manifest is a build error, not a silently-skipped one: the +// manifest is what declares the library's LAYERS, and without it `libraries: ["iam/db"]` +// resolves to nothing and the adopter gets an empty tree with no explanation. +const withoutManifest = readdirSync(libDir, { withFileTypes: true }) + .filter((e) => e.isDirectory() && !existsSync(join(libDir, e.name, "library.json"))) + .map((e) => e.name); +if (withoutManifest.length > 0) { + console.error(`error: library director${withoutManifest.length === 1 ? "y" : "ies"} with no library.json: ${withoutManifest.join(", ")}`); + process.exit(1); +} + // ref = path under library/ WITHOUT the .yaml suffix. // e.g. library/ai/llm-call.yaml -> "ai/llm-call" // Sorted by ref for deterministic, byte-stable output. @@ -118,6 +142,11 @@ const source = `// AUTO-GENERATED by scripts/generate-embedded-library.ts — DO export const EMBEDDED_LIBRARY: Record = { ${body} }; + +/** Library NAME -> the exact text of its \`library.json\` manifest. */ +export const EMBEDDED_LIBRARY_MANIFESTS: Record = { +${manifests.map((m) => ` ${JSON.stringify(m.name)}: ${JSON.stringify(m.content)},`).join("\n")} +}; `; writeFileSync(outFile, source, "utf-8"); @@ -164,10 +193,17 @@ public final class EmbeddedLibrary { /** Ref to exact file contents, insertion-ordered by ref. */ public static final Map CONTENT; + /** Library name to the exact text of its {@code library.json} manifest. */ + public static final Map MANIFESTS; + static { Map m = new LinkedHashMap<>(); ${javaEntries} CONTENT = Collections.unmodifiableMap(m); + + Map n = new LinkedHashMap<>(); +${manifests.map((x) => ` n.put(${JSON.stringify(x.name)}, ${JSON.stringify(x.content)});`).join("\n")} + MANIFESTS = Collections.unmodifiableMap(n); } } `; @@ -206,12 +242,20 @@ public static class EmbeddedLibrary { ${csharpEntries} }; + + /// Library name to the exact text of its library.json manifest. + public static readonly IReadOnlyDictionary Manifests = + new Dictionary + { +${manifests.map((x) => ` [${JSON.stringify(x.name)}] = ${JSON.stringify(x.content)},`).join("\n")} + }; } `; writeFileSync(csharpOutFile, csharpSource, "utf-8"); console.log( - `wrote ${entries.length} embedded library file(s) to ${relative(repoRoot, outFile)}, ` + - `${relative(repoRoot, javaOutFile)} and ${relative(repoRoot, csharpOutFile)}`, + `wrote ${entries.length} embedded library file(s) + ${manifests.length} manifest(s) to ` + + `${relative(repoRoot, outFile)}, ${relative(repoRoot, javaOutFile)} and ` + + `${relative(repoRoot, csharpOutFile)}`, ); diff --git a/server/csharp/MetaObjects.Cli.Tests/ColumnNamingFlagTests.cs b/server/csharp/MetaObjects.Cli.Tests/ColumnNamingFlagTests.cs index 9fed49976..7fb983496 100644 --- a/server/csharp/MetaObjects.Cli.Tests/ColumnNamingFlagTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/ColumnNamingFlagTests.cs @@ -56,7 +56,13 @@ public ColumnNamingFlagTests() private (string Entity, string Names) GenerateWith(params string[] extraArgs) { var outDir = Path.Combine(_tmp, "generated-" + Guid.NewGuid().ToString("N")); - var args = new List { "gen", MetaDir, "--out", outDir, "--namespace", "Acme.Generated" }; + // --generators is REQUIRED now: there is no default suite. The two this test + // reads (entity + names) are exactly what its assertions inspect. + var args = new List + { + "gen", MetaDir, "--out", outDir, "--namespace", "Acme.Generated", + "--generators", "entity,names", + }; args.AddRange(extraArgs); var (exit, stdout, stderr) = RunCli(_tmp, args.ToArray()); Assert.True(exit == 0, $"exit={exit}\n{stdout}{stderr}"); diff --git a/server/csharp/MetaObjects.Cli.Tests/GenCommandTests.cs b/server/csharp/MetaObjects.Cli.Tests/GenCommandTests.cs index 3dc57aa90..34e7e484f 100644 --- a/server/csharp/MetaObjects.Cli.Tests/GenCommandTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/GenCommandTests.cs @@ -32,7 +32,7 @@ public GenCommandTests() [Fact] public void Gen_writes_entity_and_dbcontext() { - var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + var outcome = GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); Assert.True(outcome.Ok, string.Join("; ", outcome.LoadErrors)); Assert.True(File.Exists(Path.Combine(OutDir, "Subscriber.g.cs"))); Assert.True(File.Exists(Path.Combine(OutDir, "AppDbContext.g.cs"))); @@ -52,7 +52,7 @@ public void Gen_writes_entity_and_dbcontext() [Fact] public void Gen_writes_the_hash_manifest_beside_the_metadata_not_in_cwd() { - var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + var outcome = GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); Assert.True(outcome.Ok, string.Join("; ", outcome.LoadErrors)); Assert.True( @@ -69,7 +69,7 @@ public void Gen_baseline_adopt_records_without_writing() { // The flag's whole value is threading: one that parses and is then dropped before // GenConfig would pass a parser test and change nothing. So this asks the RUN. - Assert.True(GenCommand.Run(MetaDir, OutDir, "Acme.Generated").Ok); + Assert.True(GenSuite.Run(MetaDir, OutDir, "Acme.Generated").Ok); var stale = Path.Combine(OutDir, "Subscriber.g.cs"); var onDisk = "// \n// from an older engine\n"; File.WriteAllText(stale, onDisk); @@ -77,7 +77,7 @@ public void Gen_baseline_adopt_records_without_writing() var outcome = GenCommand.Run( MetaObjects.Loader.MetaDataLoader.FromDirectory(MetaDir), OutDir, "Acme.Generated", - emitAbstractShapes: false, generatorNames: null, templateRoot: null, + emitAbstractShapes: false, generatorNames: GenSuite.Names, templateRoot: null, templateSpecPath: null, projectRoot: _tmp, columnNaming: MetaObjects.Codegen.ColumnNamingStrategy.Literal, baseline: "adopt"); diff --git a/server/csharp/MetaObjects.Cli.Tests/GenListAndSelectionTests.cs b/server/csharp/MetaObjects.Cli.Tests/GenListAndSelectionTests.cs index 9d3f6077d..83be5d214 100644 --- a/server/csharp/MetaObjects.Cli.Tests/GenListAndSelectionTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/GenListAndSelectionTests.cs @@ -53,16 +53,14 @@ public void ListLines_prints_all_generators_with_stable_names_and_descriptions() } [Fact] - public void Default_suite_is_python_parity_plus_names() + public void There_is_no_default_suite() { - // Parity with the Python default (entity / router / filter-allowlist / payload / - // output-parser / output-prompt / extractor), plus `names` — C# ships the - // per-object physical-database-names artifact default ON (program spec §A5; - // Python has not built it yet). render-helper is opt-in (needs --template-root); - // template / callable stay opt-in. - Assert.Equal( - ["entity", "names", "db-context", "routes", "filter-allowlist", "payload", "output-parser", "output-prompt", "extractor"], - GenCommand.DefaultGeneratorNames); + // This port used to run NINE generators for a caller who named none, and this + // test pinned that list. Codegen is opt-in now: what is pinned instead is that + // the concept is gone, and that the refusal says how to proceed. Java never had + // a default set and has been right all along. + Assert.Contains("--generators", GenCommand.NoGeneratorsSelected); + Assert.Contains("--list", GenCommand.NoGeneratorsSelected); } [Fact] @@ -91,14 +89,28 @@ public void Unknown_generator_name_surfaces_as_an_error_not_a_throw() } [Fact] - public void Null_generator_names_runs_the_default_suite() + public void Null_generator_names_is_a_usage_error_and_writes_nothing() { var outcome = GenCommand.Run( MetaDir, OutDir, "Acme.Generated", emitAbstractShapes: false, generatorNames: null, templateRoot: null); + Assert.False(outcome.Ok); + Assert.Contains(outcome.LoadErrors, e => e.Contains("--generators")); + // Nothing was emitted — a refusal that still wrote half a suite would be worse + // than the default it replaces. + Assert.False(Directory.Exists(OutDir) && Directory.GetFiles(OutDir).Length > 0); + } + + [Fact] + public void An_explicit_selection_still_emits_exactly_what_it_names() + { + var outcome = GenCommand.Run( + MetaDir, OutDir, "Acme.Generated", + emitAbstractShapes: false, + generatorNames: ["entity", "payload", "output-parser"], templateRoot: null); + Assert.True(outcome.Ok, string.Join("; ", outcome.LoadErrors)); - // The default suite still includes output-parser, which emits for a responding prompt. Assert.True(File.Exists(Path.Combine(OutDir, "Alpha.response.cs"))); } } diff --git a/server/csharp/MetaObjects.Cli.Tests/GenRefusalExitCodeTests.cs b/server/csharp/MetaObjects.Cli.Tests/GenRefusalExitCodeTests.cs index f69b0a1d0..1ce1356ec 100644 --- a/server/csharp/MetaObjects.Cli.Tests/GenRefusalExitCodeTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/GenRefusalExitCodeTests.cs @@ -45,13 +45,13 @@ public void A_refused_file_fails_the_run_and_baseline_adopt_clears_it() { // A project that predates the committed manifest: output on disk that no longer // matches fresh output, and no `.gen-state` recording it. - Assert.Equal(0, CliProcess.Run(_tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme").ExitCode); + Assert.Equal(0, CliProcess.Run(_tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--generators", "entity").ExitCode); var stale = Path.Combine(OutDir, "Subscriber.g.cs"); var onDisk = "// \n// from an older engine\n"; File.WriteAllText(stale, onDisk); Directory.Delete(Path.Combine(_tmp, ".metaobjects"), recursive: true); - var refusedRun = CliProcess.Run(_tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme"); + var refusedRun = CliProcess.Run(_tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--generators", "entity"); Assert.Equal(1, refusedRun.ExitCode); Assert.Contains("--baseline=adopt", refusedRun.Stderr); @@ -59,7 +59,7 @@ public void A_refused_file_fails_the_run_and_baseline_adopt_clears_it() // The remedy the failure names has to work, or the exit code is just a wall. var adoptRun = CliProcess.Run( - _tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--baseline=adopt"); + _tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--generators", "entity", "--baseline=adopt"); Assert.Equal(0, adoptRun.ExitCode); Assert.Equal(onDisk, File.ReadAllText(stale)); // adopting writes nothing either @@ -70,7 +70,7 @@ public void A_refused_file_fails_the_run_and_baseline_adopt_clears_it() public void An_unknown_baseline_is_a_usage_error() { var run = CliProcess.Run( - _tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--baseline=nonsense"); + _tmp, "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--generators", "entity", "--baseline=nonsense"); Assert.Equal(2, run.ExitCode); Assert.Contains("unknown --baseline", run.Stderr); diff --git a/server/csharp/MetaObjects.Cli.Tests/GenSuiteForTests.cs b/server/csharp/MetaObjects.Cli.Tests/GenSuiteForTests.cs new file mode 100644 index 000000000..ce891cb85 --- /dev/null +++ b/server/csharp/MetaObjects.Cli.Tests/GenSuiteForTests.cs @@ -0,0 +1,31 @@ +namespace MetaObjects.Cli.Tests; + +/// +/// The generator selection the gen/verify MECHANICS tests use. +/// +/// +/// These tests are about the write path, the hash manifest, baselines, column +/// naming, template-spec resolution and drift detection — not about which generators an +/// application should run. They used to get a suite for free from +/// GenCommand.DefaultGeneratorNames, which is gone: codegen is opt-in, and a +/// caller that names no generator now gets a usage error and an empty out dir. +/// So the suite is named here, ONCE, and it is deliberately the nine that used to +/// be the default — these tests' fixtures and assertions were written against exactly +/// that output, and changing what they generate would change what they are testing. +/// This is a test-local convenience, not a default restored by the back door: nothing in +/// the CLI reads it. +/// +internal static class GenSuite +{ + internal static readonly IReadOnlyList Names = + [ + "entity", "names", "db-context", "routes", "filter-allowlist", + "payload", "output-parser", "output-prompt", "extractor", + ]; + + /// `GenCommand.Run` over — the old 4-arg convenience + /// overload, moved into the tests that were its only callers. + internal static GenCommand.Outcome Run( + string metadataDir, string outDir, string ns, bool emitAbstractShapes = false) => + GenCommand.Run(metadataDir, outDir, ns, emitAbstractShapes, Names, templateRoot: null); +} diff --git a/server/csharp/MetaObjects.Cli.Tests/GenTemplateSpecTests.cs b/server/csharp/MetaObjects.Cli.Tests/GenTemplateSpecTests.cs index bd876c895..66cac5473 100644 --- a/server/csharp/MetaObjects.Cli.Tests/GenTemplateSpecTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/GenTemplateSpecTests.cs @@ -44,7 +44,7 @@ public void TemplateSpec_emits_template_output_alongside_default_suite() """); var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated", - emitAbstractShapes: false, generatorNames: null, templateRoot: TemplateRoot, templateSpecPath: SpecPath); + emitAbstractShapes: false, generatorNames: GenSuite.Names, templateRoot: TemplateRoot, templateSpecPath: SpecPath); Assert.True(outcome.Ok, string.Join("; ", outcome.LoadErrors)); var summary = Path.Combine(OutDir, "Widget.summary.txt"); @@ -64,7 +64,7 @@ public void TemplateSpec_bad_ref_yields_clean_error_not_exception() """); var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated", - emitAbstractShapes: false, generatorNames: null, templateRoot: TemplateRoot, templateSpecPath: SpecPath); + emitAbstractShapes: false, generatorNames: GenSuite.Names, templateRoot: TemplateRoot, templateSpecPath: SpecPath); Assert.False(outcome.Ok); Assert.Contains(outcome.LoadErrors, e => e.Contains("template render failed") || e.Contains("unresolved")); @@ -83,7 +83,7 @@ public void TemplateSpec_bad_output_pattern_yields_clean_error_not_exception() """); var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated", - emitAbstractShapes: false, generatorNames: null, templateRoot: TemplateRoot, templateSpecPath: SpecPath); + emitAbstractShapes: false, generatorNames: GenSuite.Names, templateRoot: TemplateRoot, templateSpecPath: SpecPath); Assert.False(outcome.Ok); Assert.Contains(outcome.LoadErrors, e => e.Contains("codegen failed")); @@ -99,7 +99,7 @@ public void TemplateSpec_with_target_is_rejected() """); var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated", - emitAbstractShapes: false, generatorNames: null, templateRoot: TemplateRoot, templateSpecPath: SpecPath); + emitAbstractShapes: false, generatorNames: GenSuite.Names, templateRoot: TemplateRoot, templateSpecPath: SpecPath); Assert.False(outcome.Ok); Assert.Contains(outcome.LoadErrors, e => e.Contains("target")); diff --git a/server/csharp/MetaObjects.Cli.Tests/MetadataDirFallbackTests.cs b/server/csharp/MetaObjects.Cli.Tests/MetadataDirFallbackTests.cs index 2f5346442..17bc8db35 100644 --- a/server/csharp/MetaObjects.Cli.Tests/MetadataDirFallbackTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/MetadataDirFallbackTests.cs @@ -44,7 +44,7 @@ public void Gen_with_no_positional_metadataDir_resolves_the_declared_source_and_ """{ "schema_version": 1, "sources": [ { "path": "model" } ] }"""); var outDir = Path.Combine(_tmp, "generated"); - var (exitCode, stdout, stderr) = CliProcess.Run(_tmp, "gen", "--out", outDir, "--namespace", "Acme.Generated"); + var (exitCode, stdout, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity", "--out", outDir, "--namespace", "Acme.Generated"); Assert.True(exitCode == 0, $"exit={exitCode}\nstdout={stdout}\nstderr={stderr}"); Assert.True(File.Exists(Path.Combine(outDir, "Subscriber.g.cs")), stdout + stderr); @@ -60,7 +60,7 @@ public void Gen_with_no_positional_metadataDir_and_nothing_to_resolve_reports_th // being routed through resolution rather than still failing the old way. Directory.CreateDirectory(_tmp); - var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--out", Path.Combine(_tmp, "generated"), "--namespace", "X"); + var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity", "--out", Path.Combine(_tmp, "generated"), "--namespace", "X"); Assert.Equal(2, exitCode); Assert.Contains("ERR_COLLECTION_NOT_FOUND", stderr); @@ -76,7 +76,7 @@ public void Gen_with_no_outDir_and_nothing_to_resolve_prints_usage_not_the_ladde // confusing on the common first-run case where both are missing at once. Directory.CreateDirectory(_tmp); - var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen"); + var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity"); Assert.Equal(2, exitCode); Assert.Contains("usage: dotnet meta gen", stderr); @@ -99,7 +99,7 @@ public void Gen_with_no_positional_metadataDir_and_multiple_declared_sources_ref """{ "schema_version": 1, "sources": [ { "path": "a" }, { "path": "b" } ] }"""); var outDir = Path.Combine(_tmp, "generated"); - var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--out", outDir, "--namespace", "X"); + var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity", "--out", outDir, "--namespace", "X"); Assert.Equal(2, exitCode); Assert.Contains("2 metadata sources", stderr); @@ -123,7 +123,7 @@ public void Gen_with_no_positional_metadataDir_and_a_single_FILE_source_refuses_ """{ "schema_version": 1, "sources": [ { "path": "vendor/meta.catalog.json" } ] }"""); var outDir = Path.Combine(_tmp, "generated"); - var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--out", outDir, "--namespace", "X"); + var (exitCode, _, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity", "--out", outDir, "--namespace", "X"); Assert.Equal(2, exitCode); Assert.Contains("is a FILE", stderr); @@ -162,7 +162,7 @@ public void Gen_with_no_positional_metadataDir_excludes_pending_drafts() """{ "schema_version": 1, "sources": [ { "path": "model" } ] }"""); var outDir = Path.Combine(_tmp, "generated"); - var (exitCode, stdout, stderr) = CliProcess.Run(_tmp, "gen", "--out", outDir, "--namespace", "Acme.Generated"); + var (exitCode, stdout, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity", "--out", outDir, "--namespace", "Acme.Generated"); Assert.True(exitCode == 0, $"exit={exitCode}\nstdout={stdout}\nstderr={stderr}"); Assert.True(File.Exists(Path.Combine(outDir, "Subscriber.g.cs")), stdout + stderr); @@ -181,7 +181,7 @@ public void Gen_with_an_explicit_positional_metadataDir_is_unaffected() File.WriteAllText(Path.Combine(modelDir, "meta.acme.json"), Metadata); var outDir = Path.Combine(_tmp, "generated"); - var (exitCode, stdout, stderr) = CliProcess.Run(_tmp, "gen", modelDir, "--out", outDir, "--namespace", "Acme.Generated"); + var (exitCode, stdout, stderr) = CliProcess.Run(_tmp, "gen", "--generators", "entity", modelDir, "--out", outDir, "--namespace", "Acme.Generated"); Assert.True(exitCode == 0, $"exit={exitCode}\nstdout={stdout}\nstderr={stderr}"); Assert.True(File.Exists(Path.Combine(outDir, "Subscriber.g.cs")), stdout + stderr); diff --git a/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs b/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs index 98cdf1d89..c5011bf7e 100644 --- a/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/VerifySubverbTests.cs @@ -62,6 +62,9 @@ private VerifyCommand.Options TemplatesOpts(bool templates = true, bool codegen new() { MetadataDir = MetaDir, + // verify --codegen re-runs the SELECTION; there is no default suite to fall + // back on, so the mechanics tests name the one their fixtures were written for. + Generators = GenSuite.Names, TemplatesRoot = TplDir, OutDir = OutDir, // Match the namespace the committed output was generated with, so a @@ -102,6 +105,9 @@ public void Bare_verify_defaults_to_templates_and_emits_the_subverb_note() var opts = new VerifyCommand.Options { MetadataDir = MetaDir, + // verify --codegen re-runs the SELECTION; there is no default suite to fall + // back on, so the mechanics tests name the one their fixtures were written for. + Generators = GenSuite.Names, TemplatesRoot = TplDir, OutDir = OutDir, Templates = false, @@ -131,7 +137,7 @@ public void Db_subverb_is_rejected_exit2_with_message() public void Codegen_clean_committed_output_is_exit0() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); var r = VerifyCommand.RunSubverbs(TemplatesOpts(templates: false, codegen: true)); Assert.Equal(0, r.ExitCode); @@ -144,7 +150,7 @@ public void Codegen_clean_committed_output_is_exit0() public void Codegen_mutated_committed_file_is_nonzero_and_named() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); File.AppendAllText(Path.Combine(OutDir, "Subscriber.g.cs"), "\n// drift\n"); var r = VerifyCommand.RunSubverbs(TemplatesOpts(templates: false, codegen: true)); @@ -156,7 +162,7 @@ public void Codegen_mutated_committed_file_is_nonzero_and_named() public void Codegen_does_not_touch_the_real_out_dir_on_drift() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); File.AppendAllText(Path.Combine(OutDir, "Subscriber.g.cs"), "\n// drift\n"); var before = SnapshotDir(OutDir); @@ -185,6 +191,9 @@ private VerifyCommand.Options CodegenOptsNoNamespace() => new() { MetadataDir = MetaDir, + // verify --codegen re-runs the SELECTION; there is no default suite to fall + // back on, so the mechanics tests name the one their fixtures were written for. + Generators = GenSuite.Names, OutDir = OutDir, Codegen = true, // Namespace intentionally NOT set + NamespaceExplicit defaults to false. @@ -195,7 +204,7 @@ public void Codegen_infers_custom_namespace_from_committed_output_no_flag() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); // Committed output generated with a CUSTOM namespace. - GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); // verify --codegen WITHOUT --namespace → must infer "Acme.Generated" from the // committed files and produce a byte-identical regen → exit 0 (no spurious drift). @@ -208,7 +217,7 @@ public void Codegen_infers_custom_namespace_from_committed_output_no_flag() public void Codegen_inference_still_detects_real_drift_with_custom_namespace() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); // A real hand-edit on top of the custom namespace must still be drift. File.AppendAllText(Path.Combine(OutDir, "Subscriber.g.cs"), "\n// real drift\n"); @@ -221,12 +230,15 @@ public void Codegen_inference_still_detects_real_drift_with_custom_namespace() public void Codegen_explicit_namespace_still_wins_over_inference() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), EntityMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated"); + GenSuite.Run(MetaDir, OutDir, "Acme.Generated"); // Explicit namespace set (matching) → wins, byte-identical regen → exit 0. var opts = new VerifyCommand.Options { MetadataDir = MetaDir, + // verify --codegen re-runs the SELECTION; there is no default suite to fall + // back on, so the mechanics tests name the one their fixtures were written for. + Generators = GenSuite.Names, OutDir = OutDir, Namespace = "Acme.Generated", NamespaceExplicit = true, @@ -272,6 +284,9 @@ public void Combining_templates_and_codegen_aggregates_max_exit() private VerifyCommand.Options ColumnNamingOpts(ColumnNamingStrategy verifyStrategy) => new() { MetadataDir = MetaDir, + // verify --codegen re-runs the SELECTION; there is no default suite to fall + // back on, so the mechanics tests name the one their fixtures were written for. + Generators = GenSuite.Names, OutDir = OutDir, Namespace = "Acme.Generated", NamespaceExplicit = true, @@ -283,7 +298,7 @@ public void Combining_templates_and_codegen_aggregates_max_exit() public void Codegen_with_matching_column_naming_is_clean() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), ColumnNamingMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated", false, null, null, null, ColumnNamingStrategy.SnakeCase); + GenCommand.Run(MetaDir, OutDir, "Acme.Generated", false, GenSuite.Names, null, null, ColumnNamingStrategy.SnakeCase); var r = VerifyCommand.RunSubverbs(ColumnNamingOpts(ColumnNamingStrategy.SnakeCase)); Assert.Equal(0, r.ExitCode); @@ -294,7 +309,7 @@ public void Codegen_with_matching_column_naming_is_clean() public void Codegen_with_mismatched_column_naming_reports_drift() { File.WriteAllText(Path.Combine(MetaDir, "meta.ai.json"), ColumnNamingMetadata); - GenCommand.Run(MetaDir, OutDir, "Acme.Generated", false, null, null, null, ColumnNamingStrategy.SnakeCase); + GenCommand.Run(MetaDir, OutDir, "Acme.Generated", false, GenSuite.Names, null, null, ColumnNamingStrategy.SnakeCase); // The discriminating half: a verify blind to --column-naming (accepting it but // dropping it, or never reading Options.ColumnNaming) would ALSO pass the diff --git a/server/csharp/MetaObjects.Cli.Tests/VerifyTemplateSpecTests.cs b/server/csharp/MetaObjects.Cli.Tests/VerifyTemplateSpecTests.cs index 663e85ef2..41c9e404c 100644 --- a/server/csharp/MetaObjects.Cli.Tests/VerifyTemplateSpecTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/VerifyTemplateSpecTests.cs @@ -56,6 +56,9 @@ public VerifyTemplateSpecTests() private VerifyCommand.Options CodegenOpts() => new() { MetadataDir = MetaDir, + // verify --codegen re-runs the SELECTION; there is no default suite to fall + // back on, so the mechanics tests name the one their fixtures were written for. + Generators = GenSuite.Names, TemplatesRoot = TemplateRoot, TemplateRoot = TemplateRoot, OutDir = OutDir, @@ -67,7 +70,7 @@ public VerifyTemplateSpecTests() private void Gen() => Assert.True( GenCommand.Run(MetaDir, OutDir, "Acme.Generated", emitAbstractShapes: false, - generatorNames: null, templateRoot: TemplateRoot).Ok); + generatorNames: GenSuite.Names, templateRoot: TemplateRoot).Ok); [Fact] public void Gen_auto_discovers_the_conventional_spec() @@ -133,7 +136,7 @@ public void An_explicit_spec_path_overrides_the_discovered_one() Assert.True( GenCommand.Run(MetaDir, OutDir, "Acme.Generated", emitAbstractShapes: false, - generatorNames: null, templateRoot: TemplateRoot, templateSpecPath: other).Ok); + generatorNames: GenSuite.Names, templateRoot: TemplateRoot, templateSpecPath: other).Ok); Assert.True(File.Exists(Path.Combine(OutDir, "Widget.flagged.txt")), "the flag's spec did not run"); Assert.False(File.Exists(Path.Combine(OutDir, "Widget.summary.txt")), "the flag must REPLACE discovery"); @@ -145,7 +148,7 @@ public void A_malformed_discovered_spec_is_a_clean_error() File.WriteAllText(DiscoveredSpec, "{ not json"); var outcome = GenCommand.Run(MetaDir, OutDir, "Acme.Generated", emitAbstractShapes: false, - generatorNames: null, templateRoot: TemplateRoot); + generatorNames: GenSuite.Names, templateRoot: TemplateRoot); // Must fail loudly: silently skipping a broken spec puts gen and verify back // out of agreement, which is the defect this whole change exists to remove. diff --git a/server/csharp/MetaObjects.Cli/GenCommand.cs b/server/csharp/MetaObjects.Cli/GenCommand.cs index 1518adbb9..c3013eedc 100644 --- a/server/csharp/MetaObjects.Cli/GenCommand.cs +++ b/server/csharp/MetaObjects.Cli/GenCommand.cs @@ -30,36 +30,28 @@ public sealed record Outcome(IReadOnlyList LoadErrors, CodegenRunner.Run public const string DefaultNamespace = "Generated"; /// - /// The default generator suite's stable names (ADR-0021 D3). Brought to parity - /// with the Python default (entity / router / filter-allowlist / payload / - /// output-parser / output-prompt / extractor), plus names: the C# suite is - /// entity, names, db-context, routes, - /// filter-allowlist, payload, output-parser, - /// output-prompt, extractor. C# is the best-wired port for the - /// per-object physical-database-names artifact — ColumnNamingStrategy is - /// already threaded to every emit site — so it ships default ON here (program - /// spec §A5); Python has since built the same artifact and also ships it default - /// ON (names_generator in its default suite). - /// The render-helper generator is intentionally NOT in the default suite — - /// it requires --template-root for its build-time drift gate (matching the - /// Python default, which also excludes the render helper). template / - /// callable stay opt-in (config-only / FR-015 niche). Every default name is - /// a registered generator, selectable individually via --generators. + /// The error a run with no generator selection reports. /// - public static readonly IReadOnlyList DefaultGeneratorNames = - ["entity", "names", "db-context", "routes", "filter-allowlist", "payload", "output-parser", "output-prompt", "extractor"]; - - /// The default generator set, built from the registry by stable name. - public static IReadOnlyList DefaultGenerators() => - GeneratorRegistry.Resolve(DefaultGeneratorNames); - - public static Outcome Run(string metadataDir, string outDir, string ns, bool emitAbstractShapes = false) => - Run(metadataDir, outDir, ns, emitAbstractShapes, generatorNames: null, templateRoot: null); + /// + /// There is no default suite. This port used to run NINE generators for a + /// caller who named none — entity, names, db-context, routes, filter-allowlist, + /// payload, output-parser, output-prompt, extractor — which is a shape nobody chose. + /// Java has never had a default set and has been right all along; TypeScript and + /// Python dropped theirs in the same change. + /// Deciding WHICH code an application needs belongs to whoever is building it + /// — increasingly an LLM working in the repo, which is well able to make that call + /// given a truthful catalog and is badly served by a default that pre-empts it. + /// --list is that catalog. + /// + public const string NoGeneratorsSelected = + "gen: no generators selected. Nothing is generated until you choose it — " + + "pass --generators . See the catalog: dotnet meta gen --list"; /// - /// Run codegen selecting generators by stable name. When - /// is null/empty the default suite runs - /// (back-compat). An unknown name (or a render-helper selected without a + /// Run codegen selecting generators by stable name. There is no default suite: + /// a null or empty generates nothing and reports + /// — ADR-0034 Amendment 2 made codegen opt-in. + /// An unknown name (or a render-helper selected without a /// ) surfaces as a load-style error in the /// returned rather than throwing. /// @@ -162,7 +154,11 @@ public static Outcome Run( if (loadErrors.Count > 0) return new Outcome(loadErrors, null); - var names = generatorNames is { Count: > 0 } ? generatorNames : DefaultGeneratorNames; + // No default suite — see NoGeneratorsSelected. A caller that names none gets a + // usage error and an empty out dir, never a shape this CLI picked. + if (generatorNames is not { Count: > 0 }) + return new Outcome([NoGeneratorsSelected], null); + var names = generatorNames; List generators; try { @@ -203,7 +199,7 @@ public static Outcome Run( // throwaway directory and records nothing at all. Baseline = baseline, // C1 — the presence gate: is `names` actually part of THIS resolved suite - // (`names` above — the default suite, or whatever `--generators` selected)? + // (`names` above — whatever `--generators` selected; there is no default)? // Computed the one way GeneratorRegistry.IncludesNames defines, so `gen` and // `verify --codegen` (VerifyCommand.RunCodegenDrift) cannot independently // drift on the answer. diff --git a/server/csharp/MetaObjects.Cli/VerifyCommand.cs b/server/csharp/MetaObjects.Cli/VerifyCommand.cs index b0137a18d..970a86279 100644 --- a/server/csharp/MetaObjects.Cli/VerifyCommand.cs +++ b/server/csharp/MetaObjects.Cli/VerifyCommand.cs @@ -245,9 +245,22 @@ private static Codegen.CodegenDrift.Result RunCodegenDrift(Options opts) string.Join(", ", load.Errors.Select(e => e.Code.ToString())) + ").", }; - var names = opts.Generators is { Count: > 0 } - ? opts.Generators - : GenCommand.DefaultGeneratorNames; + // `verify --codegen` re-runs the SELECTION and compares. With no default suite + // there is nothing to compare against, so it says so rather than silently + // checking nine artifacts this project never generates — which would convict + // every one of them as missing. + if (opts.Generators is not { Count: > 0 }) + return new Codegen.CodegenDrift.Result + { + Clean = true, + Lines = + [ + "verify --codegen: no generators selected, so there is no generated " + + "output to check. Pass --generators (dotnet meta gen --list " + + "is the catalog).", + ], + }; + var names = opts.Generators; IReadOnlyList generators; try { diff --git a/server/csharp/MetaObjects.Codegen.Tests/CodegenDriftTests.cs b/server/csharp/MetaObjects.Codegen.Tests/CodegenDriftTests.cs index 4e4b96b39..669668b32 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/CodegenDriftTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/CodegenDriftTests.cs @@ -37,11 +37,21 @@ public CodegenDriftTests() public void Dispose() { try { Directory.Delete(_tmp, recursive: true); } catch { } } - // Was a hand-copied duplicate of GenCommand.DefaultGeneratorNames, "kept in sync" by - // a source comment only -- nothing asserted it. A generator added to the real list - // and forgotten here is silently never drift-tested: a gate that loses coverage - // fails nothing. Derived instead. - private static readonly IReadOnlyList DefaultNames = GenCommand.DefaultGeneratorNames; + // Was a hand-copied duplicate of the port's default suite, "kept in sync" by a source + // comment only -- nothing asserted it. A generator added to the real list and + // forgotten here is silently never drift-tested: a gate that loses coverage fails + // nothing. So it was derived from that list -- and when opt-in codegen removed the + // default suite, the anchor had to move rather than become a literal again. + // + // It moves UP, not sideways: the whole REGISTRY, which is strictly more coverage than + // the nine that happened to be default. Two are excluded, each because it cannot run + // from a bare fixture rather than because it is uninteresting: + // render-helper -- requires --template-root for its build-time drift gate + // template -- a PRIMITIVE; real use supplies name/walk/template via config + private static readonly IReadOnlyList ExcludedFromDrift = ["render-helper", "template"]; + + private static readonly IReadOnlyList DefaultNames = + [.. GeneratorRegistry.Entries.Keys.Where(n => !ExcludedFromDrift.Contains(n))]; private static IReadOnlyList DefaultGenerators() => GeneratorRegistry.Resolve(DefaultNames); diff --git a/server/csharp/MetaObjects.Codegen.Tests/GeneratorRegistryConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/GeneratorRegistryConformanceTests.cs index 1dc46a03c..7803a2de1 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/GeneratorRegistryConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/GeneratorRegistryConformanceTests.cs @@ -16,10 +16,12 @@ namespace MetaObjects.Codegen.Tests; /// csharp IS in the C# registry, and the C# registry exposes NO name /// whose manifest ports omits csharp (i.e. the two sets are EQUAL); /// (3) tier agreement — every native manifest name is non-neutral in the registry -/// (C# has no neutral generators per the manifest). +/// (C# has no neutral generators per the manifest); +/// (4) layer agreement — every manifest name's layer equals the registry's, +/// and every manifest entry declares one of the six. /// -/// On mismatch this REPORTS the exact diff (extras / missing / tier) so the manifest -/// and registry can be reconciled. It never mutates either. +/// On mismatch this REPORTS the exact diff (extras / missing / tier / layer) so the +/// manifest and registry can be reconciled. It never mutates either. /// public sealed class GeneratorRegistryConformanceTests { @@ -39,7 +41,14 @@ private static string ManifestPath() return Path.Combine(dir, "fixtures", "generator-registry-conformance", "registry.json"); } - private sealed record ManifestEntry(string Name, string Tier, IReadOnlyList Ports); + private sealed record ManifestEntry( + string Name, string Tier, string Layer, IReadOnlyList Ports); + + // The closed set, spelled out rather than read off GeneratorLayer: enumerating the + // enum would make this gate agree with whatever the code says, which is the one + // thing a conformance gate must not do. + private static readonly string[] AllowedLayers = + ["model", "persistence", "api", "client", "docs", "capability"]; private static IReadOnlyList LoadManifest() { @@ -49,9 +58,10 @@ private static IReadOnlyList LoadManifest() foreach (var prop in generators.EnumerateObject()) { var tier = prop.Value.GetProperty("tier").GetString()!; + var layer = prop.Value.TryGetProperty("layer", out var l) ? l.GetString() ?? "" : ""; var ports = prop.Value.GetProperty("ports").EnumerateArray() .Select(p => p.GetString()!).ToList(); - entries.Add(new ManifestEntry(prop.Name, tier, ports)); + entries.Add(new ManifestEntry(prop.Name, tier, layer, ports)); } return entries; } @@ -103,6 +113,45 @@ public void TierAgreement_NativeManifestNames_AreNativeInRegistry() string.Join("\n ", tierMismatches)); } + /// (4a): every manifest name's layer equals the C# registry's. + [Fact] + public void LayerAgreement_ManifestLayers_MatchRegistry() + { + var manifest = LoadManifest(); + + var layerMismatches = manifest + .Where(e => e.Ports.Contains(Port)) + .Where(e => GeneratorRegistry.Entries.TryGetValue(e.Name, out var reg) + && !LayerMatches(e.Layer, reg.Layer)) + .Select(e => + $"{e.Name}: manifest layer=`{e.Layer}` but registry layer=`{GeneratorRegistry.Entries[e.Name].Layer}`") + .ToList(); + + Assert.True( + layerMismatches.Count == 0, + "C# generator registry layer disagrees with the canonical manifest:\n " + + string.Join("\n ", layerMismatches)); + } + + /// (4b): every manifest entry — every port's — declares one of the six. + [Fact] + public void EveryManifestEntry_DeclaresOneOfTheSixLayers() + { + var bad = LoadManifest() + .Where(e => !AllowedLayers.Contains(e.Layer)) + .Select(e => $"{e.Name}=`{e.Layer}`") + .OrderBy(s => s, StringComparer.Ordinal) + .ToList(); + + Assert.True( + bad.Count == 0, + $"manifest entries with a missing or unknown layer (allowed: {string.Join(", ", AllowedLayers)}): " + + string.Join(", ", bad)); + } + + private static bool LayerMatches(string manifestLayer, GeneratorLayer registryLayer) => + manifestLayer == registryLayer.ToString().ToLowerInvariant(); + private static bool TierMatches(string manifestTier, GeneratorTier registryTier) => manifestTier switch { diff --git a/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs b/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs index 1b22e3fbb..03b614dd9 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/NamesGeneratorTests.cs @@ -657,17 +657,21 @@ public void The_RUNTIME_physical_name_accessor_ALSO_refuses_the_divergent_shape( } [Fact] - public void The_default_generator_suite_fails_the_run_on_a_divergent_primary_source() + public void A_run_including_names_fails_on_a_divergent_primary_source() { // R-E's corrected version of the brief's sketched // Assert.Throws(() => GenerateEntity("WeirdBase")): no such // type as GeneratorException exists anywhere in this port, and no per-generator // call reaches the divergent shape (see the test above). D4's guarantee — every // consumption site references the constant unconditionally, divergence is a - // build error — is delivered at the RUN level: "names" is a member of - // GenCommand.DefaultGeneratorNames, so a default `dotnet meta gen` over this - // model fails via NamesGenerator's own InvalidOperationException, caught and - // surfaced by GenCommand.Run as a clean Outcome failure naming both sides. + // build error — is delivered at the RUN level, via NamesGenerator's own + // InvalidOperationException, caught and surfaced by GenCommand.Run as a clean + // Outcome failure naming both sides. + // + // The selection is EXPLICIT now. This used to pass `generatorNames: null` and + // lean on "names is a member of the default suite"; there is no default suite, + // and null is a usage error. Naming the generator under test is the more + // direct statement of the claim anyway. var load = new MetaDataLoader().Load( [new InMemoryStringSource(DivergentBothWritable, id: "gen.json")]); Assert.Empty(load.Errors); @@ -675,7 +679,8 @@ public void The_default_generator_suite_fails_the_run_on_a_divergent_primary_sou var tmp = Path.Combine(Path.GetTempPath(), "moc-names-run-" + Guid.NewGuid().ToString("N")); var outcome = GenCommand.Run( load, outDir: Path.Combine(tmp, "out"), ns: "Acme.Generated", emitAbstractShapes: false, - generatorNames: null, templateRoot: null, templateSpecPath: null, projectRoot: tmp); + generatorNames: ["entity", "names"], templateRoot: null, templateSpecPath: null, + projectRoot: tmp); Assert.False(outcome.Ok); var message = string.Join("\n", outcome.LoadErrors); @@ -684,6 +689,25 @@ public void The_default_generator_suite_fails_the_run_on_a_divergent_primary_sou Assert.Contains("child_table", message); } + [Fact] + public void A_run_that_names_no_generator_is_a_usage_error_and_writes_nothing() + { + // There is no default suite: nine artifacts nobody chose is exactly what opt-in + // codegen exists to stop. A caller naming none gets told what to do. + var load = new MetaDataLoader().Load( + [new InMemoryStringSource(DivergentBothWritable, id: "gen.json")]); + var tmp = Path.Combine(Path.GetTempPath(), "moc-names-none-" + Guid.NewGuid().ToString("N")); + var outDir = Path.Combine(tmp, "out"); + + var outcome = GenCommand.Run( + load, outDir: outDir, ns: "Acme.Generated", emitAbstractShapes: false, + generatorNames: null, templateRoot: null, templateSpecPath: null, projectRoot: tmp); + + Assert.False(outcome.Ok); + Assert.Contains("--generators", string.Join("\n", outcome.LoadErrors)); + Assert.False(Directory.Exists(outDir)); + } + // ------------------------------------------------------------------------- // C1 (Critical) -- the RUN-LEVEL presence gate (GenConfig.IncludeNames). Every // test above proves the ON arm: names IS part of the run, and every consumption diff --git a/server/csharp/MetaObjects.Codegen.Tests/NoMagicPhysicalNamesTests.cs b/server/csharp/MetaObjects.Codegen.Tests/NoMagicPhysicalNamesTests.cs index 62592066e..4f68b80fd 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/NoMagicPhysicalNamesTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/NoMagicPhysicalNamesTests.cs @@ -396,13 +396,20 @@ private static readonly (string Literal, string ShouldUse, Reach Reach, string W // hide a real escape the day one is spelled inside an accessor-shaped string. /// - /// The DEFAULT generator suite — the one `dotnet meta gen` runs — plus callable. - /// callable is opt-in (FR-015 niche) and so OUTSIDE the default set; a stored-proc - /// shape in the model reaches no generator unless it is wired in, and an unreached - /// generator is a gap this gate would otherwise be blind to, not a shape it covers. + /// EVERY registered generator, minus the two that cannot run from a bare fixture. /// + /// + /// This used to be the default suite plus callable — callable + /// added by hand because a stored-proc shape reaches no generator unless it is wired + /// in, and an unreached generator is a gap this gate would be blind to rather than a + /// shape it covers. Opt-in codegen removed the default suite, and that same reasoning + /// says what replaces it: the whole registry, so a NEW generator is covered the day it + /// is registered instead of the day someone remembers this list. + /// render-helper needs a --template-root and template is a + /// primitive with no default walk; neither can emit here. + /// private static readonly IReadOnlyList GeneratorNames = - [.. GenCommand.DefaultGeneratorNames, "callable"]; + [.. GeneratorRegistry.Entries.Keys.Where(n => n is not ("render-helper" or "template"))]; /// Run the generator suite over the fixture. private static IReadOnlyList Generate() diff --git a/server/csharp/MetaObjects.Codegen/CodegenDrift.cs b/server/csharp/MetaObjects.Codegen/CodegenDrift.cs index 072f40013..9b9713761 100644 --- a/server/csharp/MetaObjects.Codegen/CodegenDrift.cs +++ b/server/csharp/MetaObjects.Codegen/CodegenDrift.cs @@ -72,7 +72,8 @@ public sealed record Result /// /// the gen config (provides OutDir = the committed output). /// the loaded model (same object `gen` would use). - /// the generator suite (default suite, or a --generators selection). + /// the generator suite — always a --generators selection; + /// this port has no default set (see GenCommand.NoGeneratorsSelected). public static Result Compute(GenConfig config, MetaRoot root, IReadOnlyList generators) { var committed = Path.GetFullPath(config.OutDir); diff --git a/server/csharp/MetaObjects.Codegen/GeneratorRegistry.cs b/server/csharp/MetaObjects.Codegen/GeneratorRegistry.cs index 9c0ec7155..e83c3ece2 100644 --- a/server/csharp/MetaObjects.Codegen/GeneratorRegistry.cs +++ b/server/csharp/MetaObjects.Codegen/GeneratorRegistry.cs @@ -7,12 +7,13 @@ // the same stable name in every port. This is the discoverability + identity // surface behind `dotnet meta gen --list`. // -// It is ADDITIVE. The existing default suite (GenCommand.DefaultGenerators) -// keeps the same four generators by default — the registry powers `--list`, -// a stable identity, and selection-by-name so the previously-unreachable -// generators (render-helper, extractor, output-prompt, filter-allowlist, -// template) become runnable from the CLI without changing what any generator -// EMITS. +// It is the ONLY door. ADR-0034 Amendment 2 made codegen opt-in and DELETED the +// default suite this note used to name (GenCommand.DefaultGenerators): a run that +// selects no generator generates nothing and says so (GenCommand.NoGeneratorsSelected). +// So the registry powers `--list`, a stable identity, and the selection-by-name every +// run now goes through — including the once-unreachable generators (render-helper, +// extractor, output-prompt, filter-allowlist, template) — without changing what any +// generator EMITS. // // Stable names mirror the TS registry exactly where the concept matches // (cross-port contract): entity, db-context, routes, output-parser, extractor, @@ -33,6 +34,34 @@ public enum GeneratorTier Neutral, } +/// +/// The six layers a generator can belong to — the axis an adopter SELECTS BY, gated +/// cross-port against fixtures/generator-registry-conformance/registry.json +/// exactly as is. +/// +/// +/// Six, not ten. An earlier draft split Capability four ways, each with ONE +/// member — a layer with one member does no grouping work. The first four layers are +/// app-shape decisions a builder makes; Capability holds the ones the MODEL has +/// already made (you declared a template.prompt), which is why they are found by +/// probing a real model rather than by browsing a taxonomy. +/// +public enum GeneratorLayer +{ + /// Entity/DTO/value-object modules and the constants beside them. + Model, + /// Query helpers, DbContext, repositories, table objects. + Persistence, + /// HTTP surface: routes, filter allowlists, validators, wiring. + Api, + /// Browser tier: forms, hooks, grids. + Client, + /// Documentation artifacts (on by default; owned by the docs door). + Docs, + /// Chosen by the model, not by browsing — prompts, parsers, payloads, traces. + Capability, +} + /// /// Extra inputs a factory may need to construct a generator. Today only the /// on-disk template root (required by render-helper's build-time drift @@ -50,6 +79,8 @@ public sealed record GeneratorRegistryEntry public required string Description { get; init; } /// Native = recommended `gen` suite; Neutral = `docs`-owned. public required GeneratorTier Tier { get; init; } + /// The selection axis — see . Gated cross-port. + public required GeneratorLayer Layer { get; init; } /// /// Constructs the generator with sensible defaults. Calling it (even with an /// empty ) must NOT throw — --list @@ -95,6 +126,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "entity", Description = "Per-entity EF Core entity class (the entity module).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Model, Factory = _ => new EntityGenerator(), }, ["db-context"] = new() @@ -102,6 +134,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "db-context", Description = "Single EF Core DbContext binding every generated entity.", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Persistence, Factory = _ => new DbContextGenerator(), }, ["routes"] = new() @@ -109,6 +142,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "routes", Description = "Per-entity ASP.NET Core CRUD route handlers.", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Api, Factory = _ => new RoutesGenerator(), }, ["payload"] = new() @@ -116,6 +150,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "payload", Description = "Per-template strict typed payload record(s) (the prompt/parser/extractor bind type).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = _ => new PayloadGenerator(), }, ["output-parser"] = new() @@ -123,6 +158,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "output-parser", Description = "Per-template tolerant output parser (recover-on-receipt).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = _ => new OutputParserGenerator(), }, ["extractor"] = new() @@ -130,6 +166,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "extractor", Description = "Per-template typed Extract helper (strict payload extraction).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = _ => new ExtractorGenerator(), }, ["output-prompt"] = new() @@ -137,6 +174,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "output-prompt", Description = "Per-template output-format prompt fragment generator.", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = _ => new OutputPromptGenerator(), }, ["render-helper"] = new() @@ -144,6 +182,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "render-helper", Description = "Per-template.output render helper (document/email typed wrappers).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = RenderHelper, Options = "template-root (required when selected)", }, @@ -152,6 +191,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "filter-allowlist", Description = "Per-entity REST filter allowlist (queryable-field guard).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Api, Factory = _ => new FilterAllowlistGenerator(), }, ["names"] = new() @@ -159,6 +199,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "names", Description = "Per-object physical database name constants (table/view name, schema, columns).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Model, Factory = _ => new NamesGenerator(), }, ["template"] = new() @@ -166,6 +207,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "template", Description = "Generic Mustache template primitive (walk + template -> files).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = TemplatePrimitive, Options = "name, walk, template, format? (config-only)", }, @@ -177,6 +219,7 @@ private static IGenerator RenderHelper(GeneratorBuildContext ctx) => Name = "callable", Description = "Per-entity callable wrapper (storedProc / tableFunction FromSqlInterpolated method).", Tier = GeneratorTier.Native, + Layer = GeneratorLayer.Capability, Factory = _ => new CallableGenerator(), }, }; diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index 89066bd13..29fbafb23 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -146,6 +146,23 @@ public enum ErrorCode ERR_SCOPE_PATTERN_INVALID, // Phase-1 metadata-source-resolution: no metadata collection was discovered — no config declaring sources, and no default metaobjects/ directory. ERR_COLLECTION_NOT_FOUND, + + /// + /// FR-043 — .metaobjects/config.json's libraries names a shipped library or + /// layer this build does not have. Refused with the available tokens rather than skipped: + /// skipped, it resurfaces as ERR_UNRESOLVED_SUPER against the adopter's own + /// metadata, which is the wrong place to send someone looking. + /// + ERR_UNKNOWN_LIBRARY, + /// + /// FR-043: a node is declared by BOTH an adopter's own metadata and a shipped library the project opts into — the `meta eject ` copy with the library still in `libraries`. The two merge silently and ASYMMETRICALLY: additions take, deletions do not, because the library still declares what was removed. + /// + /// Raised by the TypeScript SDK's load path; registered in every port so the shared corpus list stays one set. + ERR_LIBRARY_PACKAGE_COLLISION, + /// + /// FR-043: a NEW top-level node is declared into a package a shipped library owns while that library is opted in — a later release of the library may ship a node of that name and merge into it. `overlay: true` on one of the library's OWN nodes is the documented amendment door and is untouched. + /// + ERR_LIBRARY_PACKAGE_NOT_OWNED, // FR5c — multi-file overlay merge produced a conflicting attribute value: // two contributors set the same @attr to different non-empty values. ERR_MERGE_CONFLICT, diff --git a/server/csharp/MetaObjects/Library/EmbeddedLibrary.cs b/server/csharp/MetaObjects/Library/EmbeddedLibrary.cs index 5e8bd2375..f2b095be8 100644 --- a/server/csharp/MetaObjects/Library/EmbeddedLibrary.cs +++ b/server/csharp/MetaObjects/Library/EmbeddedLibrary.cs @@ -20,6 +20,19 @@ public static class EmbeddedLibrary public static readonly IReadOnlyDictionary Content = new Dictionary { - ["ai/llm-call"] = "# library/ai/llm-call.yaml\n# MetaObjects-shipped standard metadata. Adopters opt in via the loader's\n# `libraries: [\"ai\"]` option, then `extends: \"metaobjects::ai::LlmCallBase\"`.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n children:\n - source.rdb: { table: llm_call, role: primary }\n - identity.primary: { name: id, fields: [\"spanId\"] }\n", + ["ai/db"] = "# library/ai/db.yaml — the DB PERSISTENCE layer for metaobjects::ai.\n#\n# Opted into as `\"ai/db\"`, which IMPLIES `\"ai\"`: `LlmCall` is declared in model.yaml and\n# this file only re-opens it, so without the core layer the overlay has no target.\n#\n# `LlmCall` is the concrete, table-backed instance of the abstract envelope. An adopter\n# who wants their OWN table (a different name, extra columns, a different id strategy)\n# extends `LlmCallBase` in their own metadata and never opts into this layer at all.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCall\n overlay: true\n children:\n - source.rdb: { table: llm_call, role: primary }\n", + ["ai/model"] = "# library/ai/model.yaml — the CORE layer: the LLM-call trace envelope.\n#\n# Adopters opt in via `libraries: [\"ai\"]`, then `extends: \"metaobjects::ai::LlmCallBase\"`.\n#\n# This layer declares NO `source.rdb`, so opting into `\"ai\"` alone adds zero tables and\n# zero generated code — the design is present and resolvable, and nothing else happens\n# until the adopter adds `\"ai/db\"`. See library/iam/model.yaml for the full rationale.\n#\n# This file was split out of the former `library/ai/llm-call.yaml`, which shipped the\n# abstract base and a concrete `LlmCall` carrying `source.rdb` together. That was\n# recorded as an accepted wart on the grounds that splitting would change what existing\n# `ai` adopters get; a sweep of the estate found there are none, so it was closed rather\n# than documented (FR-043 Amendment 1).\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n description: The concrete trace row. Its `source.rdb` lives in db.yaml, so opting into the core layer alone declares the shape without proposing a table.\n children:\n - identity.primary: { name: id, fields: [\"spanId\"] }\n", + ["ai/requirements"] = "# library/ai/requirements.yaml — what the LLM-call trace envelope PROMISES.\n#\n# A RETROFIT, not new design: llm-call.yaml landed 2026-06-03 and `requirement.functional`\n# first appears 2026-08-11, so the library could not have carried requirements when it was\n# written. That is why this file is worth reading as a worked example — it shows what\n# declaring the design of something that already exists actually turns up.\n#\n# The entry that earns its keep is `typedIo`, honestly `partial` + `accepted`: the library\n# declares the ENVELOPE and the adopter declares the typed VO columns. Recording that seam\n# in the ledger is where an agent meets it, before adding a fourth trace column.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. An L4 names the OBJECT it is about;\n# the fields that carry it hang off it as an L5 child. Writing the fields at L4 is\n# ERR_REQUIREMENT_L4_NOT_OBJECT, and writing the concerns as SIBLINGS of the L2 leaves the\n# L2 claiming nothing — both of which this file did until the standalone verify gate\n# existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::ai\n children:\n - requirement.functional:\n name: llmTracing\n level: 2\n status: live\n statement: Every call to a language model leaves a row that says what was asked, what came back, what it cost and how long it took.\n counterexample: A spend figure nobody can attribute to a call.\n description: The segment this library covers. Its three children below are the concerns it decomposes into.\n children:\n - requirement.functional:\n name: envelope\n level: 4\n status: live\n statement: A trace row identifies its call and its place in a trace — trace, span, parent span, session, call type, system.\n counterexample: A log line that cannot be joined to the request that produced it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: traceAddressing\n level: 5\n status: live\n statement: The four addressing columns are declared on the base — trace, span, parent span and session.\n counterexample: A row whose place in a trace is inferred from insertion order.\n description: >-\n The member grain exists here so the claim RESOLVES against the fields\n themselves: renaming or dropping one of them dangles this reference and\n fails the build, which naming the object alone would not.\n implementedBy: [LlmCallBase.traceId, LlmCallBase.spanId, LlmCallBase.parentSpanId, LlmCallBase.sessionId]\n\n - requirement.functional:\n name: accounting\n level: 4\n status: live\n statement: A trace row carries the tokens in, the tokens out, and the cost in integer minor units.\n counterexample: A cost stored as a float.\n description: >-\n `field.currency` — integer minor units on the wire, always. Float arithmetic for\n money is forbidden by the cross-port wire contract, and a spend total is exactly\n the sum that exposes it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: tokenAndCostColumns\n level: 5\n status: live\n statement: Tokens in, tokens out and cost are three declared columns, the cost a field.currency.\n counterexample: A cost column declared as a double.\n implementedBy: [LlmCallBase.inputTokens, LlmCallBase.outputTokens, LlmCallBase.costMinor]\n\n - requirement.functional:\n name: typedIo\n level: 4\n status: partial\n disposition: accepted\n statement: The request and response bodies are stored as structured jsonb, not as opaque text.\n counterexample: A prompt stored as a string nobody can query a field out of.\n notes: >-\n The library declares the two columns as generic jsonb with no `@objectRef`,\n because it cannot know the adopter's request/response shape. Typing them is the\n ADOPTER's move: declare an `object.value` and overlay the field with\n `@objectRef` + `@storage: jsonb`. This is the seam ADR-0024 drew, recorded here\n rather than in prose so it is in the ledger an agent reads before adding a\n fourth trace column of its own.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: jsonbBodies\n level: 5\n status: live\n statement: The request and response bodies are declared as jsonb columns on the base.\n counterexample: A prompt stored in a text column.\n description: >-\n `live` where its parent is `partial`, and the split is the point: the\n COLUMNS are shipped and this claim is fully realised; what is outstanding\n is the TYPING of them, which is the parent's gap and the adopter's move.\n implementedBy: [LlmCallBase.llmRequest, LlmCallBase.llmResponse]\n\n - requirement.architectural:\n name: traceRowsCarryTiming\n status: live\n statement: Every trace row records when the call started and how long it took.\n counterexample: A latency figure derived from log timestamps after the fact.\n description: >-\n Architectural, so it propagates down `extends` to every adopter entity deriving\n from LlmCallBase — which is the point: an adopter's own trace table is claimed\n by this requirement for free, and dropping the columns breaks the build.\n implementedBy: [LlmCallBase]\n\n - requirement.architectural:\n name: traceRowsCarryOutcome\n status: live\n statement: Every trace row records how the call ended — a status, a finish reason, and the error detail when there was one.\n counterexample: A failed call indistinguishable from one that never happened.\n implementedBy: [LlmCallBase]\n", + ["iam/db"] = "# library/iam/db.yaml — the DB PERSISTENCE layer for metaobjects::iam.\n#\n# Opted into as `\"iam/db\"`, which IMPLIES `\"iam\"`: this file is nothing but\n# `overlay: true` redeclarations, and an overlay whose target was never declared is\n# ERR_OVERLAY_NO_TARGET.\n#\n# It carries exactly two kinds of child — `source.rdb` and `index.lookup` — and nothing\n# else. The field set, the identities and the relationships all live in model.yaml,\n# because they are the DESIGN; what lives here is where the rows go and which lookups are\n# worth an index. Add a field here and the core layer stops being the whole model, which\n# is the thing the split exists to guarantee.\n#\n# Physical names are `iam_`-prefixed. Two reasons, both real: `user` and `group` are\n# reserved words in Postgres, and an adopter very likely has tables of their own by those\n# names. A library that collides on a table name is a library nobody can adopt.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: User\n overlay: true\n children:\n - source.rdb: { table: iam_user, role: primary }\n\n - object.entity:\n name: GroupType\n overlay: true\n children:\n - source.rdb: { table: iam_group_type, role: primary }\n\n - object.entity:\n name: Group\n overlay: true\n children:\n - source.rdb: { table: iam_group, role: primary }\n # Nesting is walked parent-ward constantly; the FK alone gives no index.\n - index.lookup: { name: ixParent, fields: [parentId] }\n\n - object.entity:\n name: Role\n overlay: true\n children:\n - source.rdb: { table: iam_role, role: primary }\n\n - object.entity:\n name: Permission\n overlay: true\n children:\n - source.rdb: { table: iam_permission, role: primary }\n\n - object.entity:\n name: GroupMember\n overlay: true\n children:\n - source.rdb: { table: iam_group_member, role: primary }\n # The composite PK covers (userId, groupId), so \"who is in this group?\" —\n # the other direction — has no index without this one. Same reasoning for\n # every ixSecond below.\n - index.lookup: { name: ixGroup, fields: [groupId] }\n\n - object.entity:\n name: RolePermission\n overlay: true\n children:\n - source.rdb: { table: iam_role_permission, role: primary }\n - index.lookup: { name: ixPermission, fields: [permissionId] }\n\n - object.entity:\n name: UserRole\n overlay: true\n children:\n - source.rdb: { table: iam_user_role, role: primary }\n - index.lookup: { name: ixRole, fields: [roleId] }\n\n - object.entity:\n name: GroupMemberRole\n overlay: true\n children:\n - source.rdb: { table: iam_group_member_role, role: primary }\n # \"who holds this role in this group?\" — the scoped-grant read.\n - index.lookup: { name: ixGroupRole, fields: [groupId, roleId] }\n", + ["iam/model"] = "# library/iam/model.yaml — the CORE layer: identity and access management.\n#\n# Adopters opt in via `libraries: [\"iam\"]` in .metaobjects/config.json.\n#\n# This layer declares NO `source.rdb`, and that is the whole point of the split. A\n# sourceless object is inert by a contract that already ships: migrate skips an object\n# with no writable source, and codegen emits no route, queries, hooks, grid or form for\n# one (both citing #248 — persistability derives from source presence, never from the\n# object subtype). It still gets a type-only interface, so `extends` and reference work.\n#\n# So `libraries: [\"iam\"]` adds ZERO tables and ZERO generated code. What an adopter gains\n# is the design being present and resolvable: an agent working in the repo knows the\n# capability exists and can draw on it, and nothing else happens until the adopter adds\n# `\"iam/db\"`.\n#\n# Authoring discipline (FR-043 §3.1), so the departures are visible:\n# - `field.uuid` + `generation: uuid` on principals; composite ASSIGNED keys on\n# junctions. Never `increment` — a library cannot know the adopter's id strategy.\n# - Physical names carry the `iam_` prefix (in db.yaml): `user` and `group` are\n# reserved words in Postgres, and an adopter has tables of their own.\n# - No adopter-facing profile data. That arrives by `overlay: true`.\n# - No credentials. See requirements.yaml → `noCredentialsOnUser`.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: IamBase\n abstract: true\n description: Shared shape of every iam principal and definition — a stable uuid plus change timestamps. Junctions do not extend it; they are addressed by their participants.\n children:\n - field.uuid: { name: id, required: true }\n - field.timestamp: { name: createdAt, autoSet: onCreate }\n - field.timestamp: { name: updatedAt, autoSet: onUpdate }\n\n - object.entity:\n name: User\n extends: IamBase\n description: A person or service account that can be granted access. Carries no authentication secret of any kind — see the noCredentialsOnUser requirement.\n children:\n - field.string: { name: username, required: true, maxLength: 64, filterable: true }\n - field.string: { name: email, required: true, maxLength: 254, stringFormat: email, filterable: true }\n - field.string: { name: displayName, maxLength: 120 }\n # NOT `filterable: true`, deliberately. The loader warns when a filterable\n # field is in no identity — filtering on it sequential-scans — and a library\n # must not ship a warning to every adopter. `username` and `email` carry it\n # because they have identity.secondary; `status` does not. An adopter who\n # wants to filter on status overlays `filterable` AND an index together,\n # which is exactly what the layer split is for.\n - field.enum: { name: status, required: true, values: [invited, active, suspended, closed], default: active }\n - field.timestamp: { name: emailVerifiedAt }\n - field.timestamp: { name: lastSeenAt }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqUsername, fields: [username] }\n - identity.secondary: { name: uqEmail, fields: [email] }\n - relationship.association: { name: groups, objectRef: Group, cardinality: many, through: GroupMember }\n - relationship.association: { name: roles, objectRef: Role, cardinality: many, through: UserRole }\n\n - object.entity:\n name: GroupType\n extends: IamBase\n description: What KIND of group this is — a team, a tenant, a project. An entity rather than an enum, because \"which roles may be held in this kind of group\" is data an adopter extends, and an enum's values cannot be extended by overlay.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n - object.entity:\n name: Group\n extends: IamBase\n description: A nestable collection of users, of a declared GroupType. Nesting is by parentId; acyclicity is an invariant the schema cannot express — see the acyclicGroupNesting requirement.\n children:\n - field.uuid: { name: groupTypeId, required: true }\n - field.uuid: { name: parentId }\n - field.string: { name: key, required: true, maxLength: 64 }\n # Not filterable for the same reason as User.status above.\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - identity.reference: { name: fkParent, fields: [parentId], references: Group, onDelete: restrict }\n\n - object.entity:\n name: Role\n extends: IamBase\n description: A reusable bundle of permissions. Code never compares a role NAME to a literal — it asks whether a user holds a permission, and the mapping is data.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - field.uuid: { name: groupTypeId, description: \"When set, this role may be held only within groups of this type; absent means grantable anywhere.\" }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - relationship.association: { name: permissions, objectRef: Permission, cardinality: many, through: RolePermission }\n\n - object.entity:\n name: Permission\n extends: IamBase\n description: \"The assignable unit — a stable : key the application checks against. An entity, not an enum, on ADR-0037's own reasoning: it has its own identity, its own lifecycle, and a junction with real foreign keys.\"\n children:\n - field.string: { name: key, required: true, maxLength: 128, description: \"Stable : key the application checks against.\" }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n # ---- grant surface: every grant is a row, addressed by its participants ----\n #\n # Junctions do NOT extend IamBase: they have no identity of their own, and adding a\n # surrogate uuid to a row whose identity IS its participants invites a duplicate.\n\n - object.entity:\n name: GroupMember\n description: A user's membership of a group.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.timestamp: { name: joinedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n\n - object.entity:\n name: RolePermission\n description: A permission granted by a role.\n children:\n - field.uuid: { name: roleId, required: true }\n - field.uuid: { name: permissionId, required: true }\n - identity.primary: { name: pk, fields: [roleId, permissionId], generation: assigned }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: cascade }\n - identity.reference: { name: fkPermission, fields: [permissionId], references: Permission, onDelete: restrict }\n\n - object.entity:\n name: UserRole\n description: A system-wide grant of a role to a user.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n\n - object.entity:\n name: GroupMemberRole\n description: A grant of a role to a user WITHIN one group. Three foreign keys, so it is not an M:N @through junction (which must declare exactly two identity.reference children); it is read by explicit finders.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n", + ["iam/requirements"] = "# library/iam/requirements.yaml — what this library's design PROMISES.\n#\n# This is what makes iam a library rather than a schema snippet. Without requirements an\n# adopter gets nine tables; with them they get nine tables plus a build that is held to\n# \"no authorization decision is hard-wired to a name\", which no snippet can do.\n#\n# Two reading rules, both load-bearing:\n#\n# `live` here means \"the model AS SHIPPED realises this\" — never \"your application\n# does\". A ledger binds to model nodes; runtime guarantees are the runtime's tests, and\n# this library does not invent a way to point a requirement at code (@verifiedBy was\n# retired for exactly that). Behaviour the model cannot carry ships as `partial` +\n# `disposition: accepted` with a notes sentence naming what the adopter must do.\n#\n# The functional tree roots at L2, not L1. L1 is the adopter's SOLUTION, and a library\n# is by definition a segment of someone else's. Architectural claims ship flat.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. The concerns are CHILDREN of the L2\n# rather than its siblings, and an L4 names the OBJECT it is about while the field that\n# carries it hangs off it as an L5 child. Written flat, the L2 claims nothing in its whole\n# subtree; written at L4, a field reference is ERR_REQUIREMENT_L4_NOT_OBJECT. Both shipped\n# here until the standalone verify gate existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::iam\n children:\n # ---- functional: the L2 segment and the concerns nested under it --------\n - requirement.functional:\n name: accessControl\n level: 2\n status: live\n statement: Who may do what is answered from stored grants, never from a name compared to a literal in code.\n counterexample: A branch that reads `if (user.role === \"admin\")`.\n description: The segment this library covers. The concerns beneath it are what it decomposes into.\n children:\n - requirement.functional:\n name: identity\n level: 4\n status: live\n statement: A person or service account is represented once, addressed by a uuid, and reachable by username or email.\n counterexample: Two rows for the same person because the email changed.\n implementedBy: [User]\n\n - requirement.functional:\n name: grouping\n level: 4\n status: live\n statement: Users are collected into typed, nestable groups, and the kind of group is data rather than a hard-coded set.\n counterexample: A `teamOrTenant` boolean.\n implementedBy: [Group, GroupType, GroupMember]\n\n - requirement.functional:\n name: acyclicGroupNesting\n level: 4\n status: partial\n disposition: accepted\n statement: A group is never its own ancestor.\n counterexample: Two groups each naming the other as parent.\n notes: >-\n The schema cannot express this — a self-referencing FK admits a cycle, and the\n only relational forms that would catch it (a recursive CHECK, a closure table\n maintained by trigger) are DB-specific and would not survive three dialects.\n The adopter enforces it where the write happens. Recorded rather than omitted\n so an agent reading the ledger before adding a parent-setting endpoint sees the\n obligation.\n implementedBy: [Group]\n\n - requirement.functional:\n name: grants\n level: 4\n status: live\n statement: A role is granted to a user either system-wide or scoped to one group, and both are ordinary rows.\n counterexample: A nullable `groupId` on one grant table, where NULL means \"everywhere\".\n description: >-\n Two junctions, not one with a nullable scope. A NULL in a unique key is DISTINCT\n from every other NULL in SQL, so a nullable-scope design lets the same global\n grant be inserted twice; the fix needs a partial index whose expression carries\n a physical column name. Two composite-keyed tables need no escape hatch and\n survive three dialects and five ports unchanged.\n implementedBy: [UserRole, GroupMemberRole]\n\n - requirement.functional:\n name: roleScopedToGroupType\n level: 4\n status: partial\n disposition: accepted\n statement: A role bound to a group type is granted only within groups of that type.\n counterexample: A \"tenant admin\" role granted inside a project group.\n notes: >-\n Expressing this relationally needs the grant row to carry the group's type and\n a composite FK back to (group, type) — three foreign keys deep, unverified\n across five ports' DDL and ORM paths. The adopter checks it at the point of\n grant. The declared half is the L5 child below; the enforcement is not.\n implementedBy: [Role, GroupMemberRole]\n children:\n - requirement.functional:\n name: roleDeclaresItsGroupType\n level: 5\n status: live\n statement: A role declares the group type it is bound to, as a nullable reference.\n counterexample: A role whose intended scope is recoverable only from its name.\n description: >-\n `live` where its parent is `partial`, and the split is grain as much as\n verdict: the DECLARATION is shipped and resolves against the field itself,\n so dropping the column fails the build — while the ENFORCEMENT, which no\n schema here can carry, stays the parent's accepted gap.\n implementedBy: [Role.groupTypeId]\n\n - requirement.functional:\n name: decision\n level: 4\n status: live\n statement: An authorization decision is the question \"does this user hold this permission key\", answered from rows.\n counterexample: A hard-coded list of usernames that bypass a check.\n implementedBy: [Permission, RolePermission]\n\n # ---- architectural: prohibitions in force --------------------------------\n\n - requirement.architectural:\n name: grantsAreRows\n status: live\n statement: A grant exists only as a stored row; nothing is granted by naming, position or convention.\n counterexample: A superuser recognised by username.\n implementedBy: [UserRole, GroupMemberRole, RolePermission, GroupMember]\n\n - requirement.architectural:\n name: noCredentialsOnUser\n status: live\n statement: A user row carries no authentication secret — no password, no hash, no knowledge-based question or answer.\n counterexample: A password or secret-answer column on the user table.\n description: >-\n Authentication is a separate capability with an entity per factor; this library\n is identity and authorization only.\n notes: >-\n This is the one thing every reader of a user table proposes adding, and a real\n legacy model of this shape stored a length-bounded plaintext password and a\n knowledge-based secret pair on the user row. Stating it as a prohibition IN\n FORCE — claimable, and rendered on agent/requirements.md — is what stops an\n agent extending \"the user model\" from re-deriving it on sight. It is\n `architectural`, not `retired`: retired is chartered for a capability built\n here and removed, and this library never built one.\n implementedBy: [User]\n\n - requirement.architectural:\n name: principalDeletionRevokesGrants\n status: live\n statement: Deleting a user or group removes its grants; deleting a role or permission still in use is refused.\n counterexample: A grant row pointing at a user who no longer exists.\n description: The referential rule in one sentence — cascade from a principal, restrict from a definition.\n implementedBy: [GroupMember, UserRole, GroupMemberRole, RolePermission]\n\n - requirement.architectural:\n name: stableIdentifiers\n status: live\n statement: Every principal and definition is addressed by a uuid that never changes; every grant by its participants.\n counterexample: A group referenced by its display name.\n implementedBy: [IamBase]\n", + }; + + /// Library name to the exact text of its library.json manifest. + public static readonly IReadOnlyDictionary Manifests = + new Dictionary + { + ["ai"] = "{\n \"$comment\": \"Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.\",\n \"name\": \"ai\",\n \"kind\": \"feature\",\n \"stability\": \"stable\",\n \"since\": \"0.20.0\",\n \"description\": \"The LLM-call trace envelope: what was asked, what came back, what it cost, how long it took.\",\n \"useWhen\": \"the application calls a language model and someone will ask what it cost or why a call failed\",\n \"packages\": [\"metaobjects::ai\"],\n \"layers\": {\n \"\": { \"refs\": [\"ai/model\", \"ai/requirements\"], \"description\": \"the core model and its requirements — sourceless, so it adds no tables\" },\n \"db\": { \"refs\": [\"ai/db\"], \"description\": \"the concrete llm_call table\" }\n },\n \"generators\": [\n { \"name\": \"trace-helper\", \"anchor\": \"metaobjects::ai::LlmCallBase\" }\n ],\n \"runtime\": {\n \"typescript\": [\"@metaobjectsdev/runtime-ts\"]\n }\n}\n", + ["iam"] = "{\n \"$comment\": \"Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.\",\n \"name\": \"iam\",\n \"kind\": \"feature\",\n \"stability\": \"preview\",\n \"since\": \"1.1.0\",\n \"description\": \"Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group.\",\n \"useWhen\": \"the application has people who log in and things some of them may not do\",\n \"packages\": [\"metaobjects::iam\"],\n \"layers\": {\n \"\": { \"refs\": [\"iam/model\", \"iam/requirements\"], \"description\": \"the core model and its requirements — sourceless, so it adds no tables\" },\n \"db\": { \"refs\": [\"iam/db\"], \"description\": \"nine tables, iam_-prefixed, plus the lookup indexes the composite keys do not cover\" }\n },\n \"generators\": [],\n \"runtime\": {}\n}\n", }; } diff --git a/server/csharp/MetaObjects/Library/LibrarySources.cs b/server/csharp/MetaObjects/Library/LibrarySources.cs index 23f8162e6..48cce9161 100644 --- a/server/csharp/MetaObjects/Library/LibrarySources.cs +++ b/server/csharp/MetaObjects/Library/LibrarySources.cs @@ -1,3 +1,4 @@ +using System.Text.Json; using MetaObjects.Loader; namespace MetaObjects.Library; @@ -19,33 +20,80 @@ namespace MetaObjects.Library; public static class LibrarySources { /// - /// Package to ordered refs, derived from the generated embed so that adding a library file - /// (which regenerates ) needs no edit here. + /// Library name to its manifest's LAYERS: layer token to that layer's ordered refs. + /// The CORE layer's token is the empty string. /// - private static readonly IReadOnlyDictionary> RefsByPackage = - BuildRefsByPackage(); + /// + /// Read from the embedded library.json manifests, not derived from the ref names. + /// This used to be package-granular — every ref under a library came back for a bare + /// "ai" — which under the layered design (FR-043 Amendment 1) would hand an adopter + /// the db layer they did not ask for, and with it a migration proposing tables. + /// + private static readonly IReadOnlyDictionary>> LayersByLibrary = + BuildLayers(); /// Resolved once per process; null means "looked, not present". private static readonly Lazy LibraryDir = new(LibraryDirOnDisk); - private static IReadOnlyDictionary> BuildRefsByPackage() + private static IReadOnlyDictionary>> BuildLayers() { - var map = new Dictionary>(); - foreach (var r in EmbeddedLibrary.Content.Keys.OrderBy(k => k, StringComparer.Ordinal)) + var map = new Dictionary>>(StringComparer.Ordinal); + foreach (var (name, text) in EmbeddedLibrary.Manifests.OrderBy(kv => kv.Key, StringComparer.Ordinal)) { - var slash = r.IndexOf('/'); - if (slash <= 0) continue; - var pkg = r[..slash]; - if (!map.TryGetValue(pkg, out var list)) + using var doc = JsonDocument.Parse(text); + var layers = new Dictionary>(StringComparer.Ordinal); + if (doc.RootElement.TryGetProperty("layers", out var layersEl)) { - list = []; - map[pkg] = list; + foreach (var layer in layersEl.EnumerateObject()) + { + var refs = new List(); + if (layer.Value.TryGetProperty("refs", out var refsEl)) + { + foreach (var r in refsEl.EnumerateArray()) + { + if (r.GetString() is { } s) refs.Add(s); + } + } + layers[layer.Name] = refs; + } } - list.Add(r); + map[name] = layers; } - return map.ToDictionary(kv => kv.Key, kv => (IReadOnlyList)kv.Value); + return map; } + /// + /// Split a selection token into library and layer — "iam" to ("iam", ""), + /// "iam/db" to ("iam", "db"). Only the FIRST separator is meaningful, so a + /// typo stays a typo rather than resolving to a prefix. + /// + public static (string Library, string Layer) SplitToken(string token) + { + var i = token.IndexOf('/'); + return i == -1 ? (token, "") : (token[..i], token[(i + 1)..]); + } + + /// Every selection token this build accepts, sorted — what a config error prints. + /// The prefix every library source id carries. + public const string LibraryFileIdPrefix = "library:"; + + /// + /// The source id a library file loads under, in every build — + /// library:iam/model.yaml. + /// + /// + /// Stable rather than path-derived so a library node's ADR-0009 provenance envelope + /// reads the same from a checkout and from an installed package, carries no absolute + /// path, and cannot be confused with an adopter file sharing a basename. + /// + public static string LibraryFileId(string reference) => $"{LibraryFileIdPrefix}{reference}.yaml"; + + public static IReadOnlyList KnownTokens() => + LayersByLibrary + .SelectMany(kv => kv.Value.Keys.Select(layer => layer.Length == 0 ? kv.Key : $"{kv.Key}/{layer}")) + .OrderBy(t => t, StringComparer.Ordinal) + .ToList(); + /// /// The library package names this build ships, sorted. /// @@ -55,7 +103,7 @@ private static IReadOnlyDictionary> BuildRefsByPac /// validates against this first. /// public static IReadOnlyList KnownPackages() => - RefsByPackage.Keys.OrderBy(k => k, StringComparer.Ordinal).ToList(); + LayersByLibrary.Keys.OrderBy(k => k, StringComparer.Ordinal).ToList(); /// /// Locate the repo-root library/ directory by walking up from this assembly's @@ -85,17 +133,39 @@ public static IReadOnlyList KnownPackages() => /// typed into a config file is the opposite case, and the caller that read it validates /// against before calling this. /// - /// Package names to include (e.g. ["ai"]); null yields none. + /// Selection tokens (e.g. ["iam", "iam/db"]); null yields none. public static List Resolve(IEnumerable? packages) { var outSources = new List(); if (packages is null) return outSources; - var dir = LibraryDir.Value; - foreach (var pkg in packages) + // A token whose LAYER is unknown is dropped whole, not reduced to its core: implying + // the core from an invalid layer would answer a mistyped "iam/database" with an inert + // core and no tables, which is the worst of the available outcomes. + var wanted = packages + .Select(SplitToken) + .Where(t => LayersByLibrary.TryGetValue(t.Library, out var l) && l.ContainsKey(t.Layer)) + .ToList(); + + // Core layers FIRST, across every requested library, so a db layer named before its + // core in the config still parses after it. "iam/db" IMPLIES "iam": a db layer is + // nothing but overlay:true redeclarations, and an overlay whose target was never + // declared is ERR_OVERLAY_NO_TARGET. + var refs = new List(); + var seen = new HashSet(StringComparer.Ordinal); + void Add(string r) { if (seen.Add(r)) refs.Add(r); } + foreach (var (lib, _) in wanted) { - if (!RefsByPackage.TryGetValue(pkg, out var refs)) continue; // unknown — no sources + foreach (var r in LayersByLibrary[lib][""]) Add(r); + } + foreach (var (lib, layer) in wanted) + { + if (layer.Length == 0) continue; + foreach (var r in LayersByLibrary[lib][layer]) Add(r); + } + var dir = LibraryDir.Value; + { foreach (var r in refs) { if (dir is not null) @@ -103,19 +173,20 @@ public static List Resolve(IEnumerable? packages) var path = Path.Combine(dir, r.Replace('/', Path.DirectorySeparatorChar) + ".yaml"); if (File.Exists(path)) { - outSources.Add(new FileSource(path)); + // The SAME id the embedded branch below uses — see LibraryFileId. + outSources.Add(new FileSource(path, LibraryFileId(r))); continue; } } if (!EmbeddedLibrary.Content.TryGetValue(r, out var embedded)) { throw new InvalidOperationException( - $"library ref \"{r}\" (package \"{pkg}\") has no on-disk file and no " + $"library ref \"{r}\" has no on-disk file and no " + "embedded entry — the embedded library class is stale; run " + "scripts/generate-embedded-library.ts"); } outSources.Add(new InMemoryStringSource( - embedded, $"library:{r}.yaml", MetaDataFormat.Yaml)); + embedded, LibraryFileId(r), MetaDataFormat.Yaml)); } } return outSources; diff --git a/server/csharp/MetaObjects/Loader/FileSource.cs b/server/csharp/MetaObjects/Loader/FileSource.cs index 2171a1795..fb25e65fc 100644 --- a/server/csharp/MetaObjects/Loader/FileSource.cs +++ b/server/csharp/MetaObjects/Loader/FileSource.cs @@ -25,6 +25,22 @@ public FileSource(string path) Format = MetaDataFormats.InferFromExtension(path); } + /// + /// A file source with an explicit ID, overriding the file-name default. + /// + /// + /// For a file whose identity in diagnostics should not depend on where it sits on + /// disk — a shipped library's YAML, whose envelope must read the same from a + /// checkout and from an installed package, and must not collide with an adopter + /// file of the same basename. + /// + public FileSource(string path, string id) + { + FilePath = path; + Id = id; + Format = MetaDataFormats.InferFromExtension(path); + } + /// /// Read the file content. BOM stripping is handled by the parser, not here. /// diff --git a/server/csharp/MetaObjects/Parser.cs b/server/csharp/MetaObjects/Parser.cs index daa6a49d4..187cebb92 100644 --- a/server/csharp/MetaObjects/Parser.cs +++ b/server/csharp/MetaObjects/Parser.cs @@ -821,7 +821,17 @@ private static void ParseNodeInto( // `format: "merged"` envelope. Last-writer-wins is preserved for // non-conflicting cases (one side unset, same value, etc.) — those carry // through to the existing ApplyInlineAttrsAndUnknownKeys logic below. - if (fr5cActive && preMergeAttrSnapshot is not null) + // + // FR-043 Amendment 2 — `overlay: true` LICENSES the override. The conflict error + // exists to catch two files that collided without knowing about each other; the + // flag is the author saying "I know about the other declaration and I mean to + // change it". The loader already treats it specially (find-or-throw versus + // create-or-find), so honouring it here makes it mean ONE thing rather than two. + // Per NODE: a nested overlay marks its own ancestors too, and each is judged on + // its own flag. + if (fr5cActive + && preMergeAttrSnapshot is not null + && TryGetBool(nodeData, RESERVED_KEY_OVERLAY) != true) { DetectAttrMergeConflicts(target, nodeData, preMergeAttrSnapshot, newContributorFile, st); } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt index 7ca893839..2c497a076 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistry.kt @@ -30,6 +30,40 @@ enum class GeneratorTier { NEUTRAL, } +/** + * The six layers a generator can belong to — the axis an adopter SELECTS BY, + * mirroring the canonical manifest's `layer` field and gated against it exactly as + * [GeneratorTier] is. + * + * Six, not ten. An earlier draft split [CAPABILITY] four ways, each with ONE member — + * a layer with one member does no grouping work. The first four layers are app-shape + * decisions a builder makes; [CAPABILITY] holds the ones the MODEL has already made + * (you declared a `template.prompt`), which is why they are found by probing a real + * model rather than by browsing a taxonomy. + */ +enum class GeneratorLayer { + /** Entity / DTO / value-object modules and the constants beside them. */ + MODEL, + + /** Query helpers, repositories, table objects, stored-proc bindings. */ + PERSISTENCE, + + /** HTTP surface: routes, filter allowlists, validators, wiring. */ + API, + + /** Browser tier: forms, hooks, grids. */ + CLIENT, + + /** Documentation artifacts (on by default; owned by the docs door). */ + DOCS, + + /** Chosen by the model, not by browsing — prompts, parsers, payloads, traces. */ + CAPABILITY; + + /** The manifest's spelling: lower-case. */ + fun manifestValue(): String = name.lowercase() +} + /** * One registry entry: stable id + one-line description + tier + a refactor-safe * factory. The factory constructs the generator with its no-arg constructor @@ -44,6 +78,8 @@ data class GeneratorInfo( val description: String, /** NATIVE = recommended `gen` suite; NEUTRAL = `docs`-owned. */ val tier: GeneratorTier, + /** The selection axis — see [GeneratorLayer]. Gated cross-port. */ + val layer: GeneratorLayer, /** Constructs the generator with sensible defaults. Calling it must not throw. */ val factory: () -> MultiFileDirectGeneratorBase<*>, ) @@ -62,90 +98,105 @@ val GENERATOR_REGISTRY: Map = linkedMapOf( name = "entity", description = "Per-entity Kotlin data class (the entity module).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.MODEL, factory = ::KotlinEntityGenerator, ), "routes" to GeneratorInfo( name = "routes", description = "Per-entity Spring REST controller (CRUD endpoint surface).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.API, factory = ::KotlinSpringControllerGenerator, ), "repository" to GeneratorInfo( name = "repository", description = "Per-entity Kotlin persistence repository base (row-mapper + CRUD + patch).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.PERSISTENCE, factory = ::KotlinRepositoryGenerator, ), "output-parser" to GeneratorInfo( name = "output-parser", description = "Per-template tolerant output parser (recover-on-receipt).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.CAPABILITY, factory = ::KotlinOutputParserGenerator, ), "output-prompt" to GeneratorInfo( name = "output-prompt", description = "Per-template output-format prompt fragment generator.", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.CAPABILITY, factory = ::KotlinOutputPromptGenerator, ), "render-helper" to GeneratorInfo( name = "render-helper", description = "Per-template.output render helper (document/email typed wrappers).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.CAPABILITY, factory = ::KotlinRenderHelperGenerator, ), "extractor" to GeneratorInfo( name = "extractor", description = "Per-template strict typed extract helper (strict payload extraction).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.CAPABILITY, factory = ::KotlinExtractorGenerator, ), "filter-allowlist" to GeneratorInfo( name = "filter-allowlist", description = "Per-entity REST filter allowlist (queryable-field guard).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.API, factory = ::KotlinFilterAllowlistGenerator, ), "payload" to GeneratorInfo( name = "payload", description = "Per-template payload value object (the strict payload type).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.CAPABILITY, factory = ::KotlinPayloadGenerator, ), "names" to GeneratorInfo( name = "names", description = "Per-object physical database name constants (table/view name, schema, columns).", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.MODEL, factory = ::KotlinNamesGenerator, ), "exposed-table" to GeneratorInfo( name = "exposed-table", description = "Per-entity Kotlin Exposed table object.", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.PERSISTENCE, factory = ::KotlinExposedTableGenerator, ), "relations" to GeneratorInfo( name = "relations", description = "Cross-entity relationship helpers.", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.PERSISTENCE, factory = ::KotlinRelationsGenerator, ), "spring-config" to GeneratorInfo( name = "spring-config", description = "Spring wiring/configuration for the generated surface.", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.API, factory = ::KotlinSpringConfigGenerator, ), "stored-proc" to GeneratorInfo( name = "stored-proc", description = "Stored-procedure binding helpers.", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.PERSISTENCE, factory = ::KotlinStoredProcGenerator, ), "validator" to GeneratorInfo( name = "validator", description = "Per-entity input validator.", tier = GeneratorTier.NATIVE, + layer = GeneratorLayer.API, factory = ::KotlinValidatorGenerator, ), ) diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistryConformanceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistryConformanceTest.kt index 6243efe33..afadfd767 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistryConformanceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/GeneratorRegistryConformanceTest.kt @@ -21,9 +21,11 @@ import kotlin.test.assertTrue * IS in the Kotlin registry, and the Kotlin registry exposes NO name whose * manifest `ports` omits `kotlin` (i.e. the two sets are EQUAL); * (3) tier agreement — every native manifest name is NATIVE in the registry - * (Kotlin has no neutral generators per the manifest). + * (Kotlin has no neutral generators per the manifest); + * (4) layer agreement — every manifest name's `layer` equals the registry's, and + * every manifest entry declares one of the six. * - * On mismatch this REPORTS the exact diff (missing / extra / tier) so the manifest + * On mismatch this REPORTS the exact diff (missing / extra / tier / layer) so the manifest * and registry can be reconciled. It never mutates either. Repo-root resolution * mirrors the sibling Kotlin conformance tests (walk up from `user.dir`). */ @@ -31,6 +33,13 @@ class GeneratorRegistryConformanceTest { private val port = "kotlin" + /** + * The closed set of `layer` values, spelled out rather than derived from + * [GeneratorLayer]. Enumerating the enum would make this gate agree with whatever + * the code says, which is the one thing a conformance gate must not do. + */ + private val allowedLayers = setOf("model", "persistence", "api", "client", "docs", "capability") + private val manifestPath: Path = run { var p: Path? = Path.of(System.getProperty("user.dir")).toAbsolutePath() while (p != null && !Files.exists(p.resolve("fixtures/generator-registry-conformance/registry.json"))) { @@ -43,7 +52,12 @@ class GeneratorRegistryConformanceTest { p!!.resolve("fixtures/generator-registry-conformance/registry.json") } - private data class ManifestEntry(val name: String, val tier: String, val ports: Set) + private data class ManifestEntry( + val name: String, + val tier: String, + val layer: String, + val ports: Set, + ) private fun loadManifest(): List { val root: JsonNode = ObjectMapper().readTree(Files.readString(manifestPath)) @@ -53,6 +67,7 @@ class GeneratorRegistryConformanceTest { ManifestEntry( name = name, tier = node.get("tier").asText(), + layer = node.path("layer").asText(""), ports = node.get("ports").map { it.asText() }.toSet(), ) }.toList() @@ -105,6 +120,39 @@ class GeneratorRegistryConformanceTest { ) } + /** (4a): every manifest name's layer equals the Kotlin registry's. */ + @Test + fun `layers agree with the manifest`() { + val layerMismatches = loadManifest() + .filter { port in it.ports } + .mapNotNull { entry -> + val reg = GENERATOR_REGISTRY[entry.name] ?: return@mapNotNull null + if (reg.layer.manifestValue() != entry.layer) { + "${entry.name}: manifest layer=`${entry.layer}` but registry layer=`${reg.layer.manifestValue()}`" + } else null + } + + assertTrue( + layerMismatches.isEmpty(), + "Kotlin generator registry layer disagrees with the canonical manifest:\n " + + layerMismatches.joinToString("\n "), + ) + } + + /** (4b): every manifest entry — every port's — declares one of the six. */ + @Test + fun `every manifest entry declares one of the six layers`() { + val bad = loadManifest() + .filter { it.layer !in allowedLayers } + .map { "${it.name}='${it.layer}'" } + .sorted() + + assertTrue( + bad.isEmpty(), + "manifest entries with a missing or unknown layer (allowed: ${allowedLayers.sorted()}): $bad", + ) + } + /** Sanity: every registered factory constructs a generator without throwing (powers `--list`). */ @Test fun `every registered factory constructs without throwing`() { diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java index 4bec3710a..88885efa2 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/GeneratorRegistry.java @@ -52,18 +52,52 @@ public enum Tier { NEUTRAL } + /** + * The six layers a generator can belong to — the axis an adopter SELECTS BY, + * mirroring the manifest's {@code layer} field and gated against it exactly as + * {@link Tier} is. + * + *

Six, not ten. An earlier draft split {@code CAPABILITY} four ways, each with + * ONE member — a layer with one member does no grouping work. The first four layers + * are app-shape decisions a builder makes; {@code CAPABILITY} holds the ones the + * MODEL has already made (you declared a {@code template.prompt}), which is why they + * are found by probing a real model rather than by browsing a taxonomy.

+ */ + public enum Layer { + /** Entity / DTO / value-object modules and the constants beside them. */ + MODEL, + /** Query helpers, DbContext, repositories, table objects. */ + PERSISTENCE, + /** HTTP surface: routes, filter allowlists, validators, wiring. */ + API, + /** Browser tier: forms, hooks, grids. */ + CLIENT, + /** Documentation artifacts (on by default; owned by the docs door). */ + DOCS, + /** Chosen by the model, not by browsing — prompts, parsers, payloads, traces. */ + CAPABILITY; + + /** The manifest's spelling: lower-case, hyphen-free. */ + public String manifestValue() { + return name().toLowerCase(java.util.Locale.ROOT); + } + } + /** Immutable metadata for a single registered generator. */ public static final class GeneratorInfo { private final String stableName; private final String classname; private final String description; private final Tier tier; + private final Layer layer; - public GeneratorInfo(String stableName, String classname, String description, Tier tier) { + public GeneratorInfo(String stableName, String classname, String description, + Tier tier, Layer layer) { this.stableName = stableName; this.classname = classname; this.description = description; this.tier = tier; + this.layer = layer; } /** Canonical cross-port stable name (the manifest key). */ @@ -86,9 +120,15 @@ public Tier tier() { return tier; } + /** The generator's layer — the axis an adopter selects by. */ + public Layer layer() { + return layer; + } + @Override public String toString() { - return "GeneratorInfo{" + stableName + " -> " + classname + " (" + tier + ")}"; + return "GeneratorInfo{" + stableName + " -> " + classname + + " (" + tier + ", " + layer + ")}"; } } @@ -100,44 +140,44 @@ private GeneratorRegistry() { private static Map buildRegistry() { Map m = new LinkedHashMap<>(); register(m, "entity", JavaObjectCodeGenerator.class.getName(), - "Per-entity Java model/class (table-backed or value object).", Tier.NATIVE); + "Per-entity Java model/class (table-backed or value object).", Tier.NATIVE, Layer.MODEL); register(m, "routes", SpringControllerGenerator.class.getName(), - "Per-entity Spring @RestController endpoint surface.", Tier.NATIVE); + "Per-entity Spring @RestController endpoint surface.", Tier.NATIVE, Layer.API); register(m, "output-parser", SpringOutputParserGenerator.class.getName(), - "Per-template tolerant output parser (recover-on-receipt).", Tier.NATIVE); + "Per-template tolerant output parser (recover-on-receipt).", Tier.NATIVE, Layer.CAPABILITY); register(m, "output-prompt", SpringOutputPromptGenerator.class.getName(), - "Per-template output-format prompt fragment generator.", Tier.NATIVE); + "Per-template output-format prompt fragment generator.", Tier.NATIVE, Layer.CAPABILITY); register(m, "render-helper", SpringRenderHelperGenerator.class.getName(), - "Per-template.output render helper (document/email typed wrappers).", Tier.NATIVE); + "Per-template.output render helper (document/email typed wrappers).", Tier.NATIVE, Layer.CAPABILITY); register(m, "extractor", ExtractorCodeGenerator.class.getName(), "Per-template strict typed extract helper. FUSED into `entity` on this " - + "port — emitted by JavaObjectCodeGenerator, not separately wirable.", Tier.NATIVE); + + "port — emitted by JavaObjectCodeGenerator, not separately wirable.", Tier.NATIVE, Layer.CAPABILITY); register(m, "template", TemplateScopeGenerator.class.getName(), "Generic Mustache template primitive (walk + template -> files) — the " - + "Maven-wirable declarative form over the conformance-pinned renderer.", Tier.NATIVE); + + "Maven-wirable declarative form over the conformance-pinned renderer.", Tier.NATIVE, Layer.CAPABILITY); register(m, "filter-allowlist", SpringFilterAllowlistGenerator.class.getName(), - "Per-entity REST filter allowlist (queryable-field guard).", Tier.NATIVE); + "Per-entity REST filter allowlist (queryable-field guard).", Tier.NATIVE, Layer.API); register(m, "payload", SpringPayloadGenerator.class.getName(), - "Per-template payload value object (the strict payload type).", Tier.NATIVE); + "Per-template payload value object (the strict payload type).", Tier.NATIVE, Layer.CAPABILITY); register(m, "repository", SpringRepositoryGenerator.class.getName(), - "Per-entity Spring Data repository.", Tier.NATIVE); + "Per-entity Spring Data repository.", Tier.NATIVE, Layer.PERSISTENCE); register(m, "dto", SpringDtoGenerator.class.getName(), - "Per-entity Spring DTO record.", Tier.NATIVE); + "Per-entity Spring DTO record.", Tier.NATIVE, Layer.MODEL); register(m, "value-object", SpringValueObjectGenerator.class.getName(), "Per-value-object Spring record with jakarta constraints — the typed " - + "component the DTO/Patch bind for a field.object @storage:jsonb column.", Tier.NATIVE); + + "component the DTO/Patch bind for a field.object @storage:jsonb column.", Tier.NATIVE, Layer.MODEL); register(m, "trace-helper", LlmTraceHelperGenerator.class.getName(), "Per-entity typed record LLM-trace helper (extract + buildLlmCallRow + persist; " - + "LlmCallBase-derived entities only).", Tier.NATIVE); + + "LlmCallBase-derived entities only).", Tier.NATIVE, Layer.CAPABILITY); register(m, "names", SpringNamesGenerator.class.getName(), "Per-object physical database name constants (table/view/schema/column) " - + "for a hand-written consumer to reference instead of a string literal.", Tier.NATIVE); + + "for a hand-written consumer to reference instead of a string literal.", Tier.NATIVE, Layer.MODEL); return Collections.unmodifiableMap(m); } private static void register(Map m, String stableName, - String classname, String description, Tier tier) { - if (m.put(stableName, new GeneratorInfo(stableName, classname, description, tier)) != null) { + String classname, String description, Tier tier, Layer layer) { + if (m.put(stableName, new GeneratorInfo(stableName, classname, description, tier, layer)) != null) { throw new IllegalStateException("duplicate generator stable name: " + stableName); } } diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/LlmTraceHelperGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/LlmTraceHelperGenerator.java index 7bd0ee586..efa2794cc 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/LlmTraceHelperGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/LlmTraceHelperGenerator.java @@ -5,6 +5,7 @@ import com.metaobjects.generator.GeneratorIOWriter; import com.metaobjects.generator.direct.MultiFileDirectGeneratorBase; import com.metaobjects.generator.util.GeneratorUtil; +import com.metaobjects.library.LibrarySources; import com.metaobjects.loader.MetaDataLoader; import com.metaobjects.object.MetaObject; import com.metaobjects.template.MetaTemplate; @@ -78,8 +79,20 @@ */ public class LlmTraceHelperGenerator extends MultiFileDirectGeneratorBase { - /** The abstract base entity a trace entity must (transitively) extend. */ - public static final String LLM_CALL_BASE = "LlmCallBase"; + /** This generator's cross-port stable name — the key a library manifest declares + * its {@code anchor} under. */ + public static final String STABLE_NAME = "trace-helper"; + + /** + * FQNs of the library nodes a trace entity must (transitively) extend — read from + * the shipped {@code library.json} manifests (FR-043 §6), never hard-coded here. + * + *

What it replaces: {@code "LlmCallBase"} compared against + * {@link MetaData#getShortName()}, which matched ANY adopter entity of that name in + * ANY package while never actually keying on the shipped abstract.

+ */ + public static final java.util.List ANCHOR_FQNS = + java.util.Collections.unmodifiableList(LibrarySources.generatorAnchors(STABLE_NAME)); /** FQN of the runtime extract entry point (emitted as a source FQN string). */ public static final String META_OBJECT_EXTRACT_FQN = @@ -275,11 +288,18 @@ protected void emit(MetaObject entity, MetaDataLoader loader, Path outRoot) { // Resolution helpers // ------------------------------------------------------------------------- - /** Walk the super chain looking for a node short-named {@link #LLM_CALL_BASE}. */ + /** + * Walk the super chain looking for one of {@link #ANCHOR_FQNS}. + * + *

The FULL name, not the short one: within one loader an FQN identifies exactly + * one node (a same-name redeclaration merges), so this is the node-identity compare + * the TypeScript port makes against the resolved anchor node.

+ */ protected static boolean extendsBase(MetaObject entity) { + if (ANCHOR_FQNS.isEmpty()) return false; MetaObject cur = entity.getSuperObject(); while (cur != null) { - if (LLM_CALL_BASE.equals(cur.getShortName())) return true; + if (ANCHOR_FQNS.contains(cur.getName())) return true; cur = cur.getSuperObject(); } return false; diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/GeneratorRegistryConformanceTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/GeneratorRegistryConformanceTest.java index 56f91e47a..4e2f3abba 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/GeneratorRegistryConformanceTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/GeneratorRegistryConformanceTest.java @@ -3,6 +3,7 @@ import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.metaobjects.generator.GeneratorRegistry.GeneratorInfo; +import com.metaobjects.generator.GeneratorRegistry.Layer; import com.metaobjects.generator.GeneratorRegistry.Tier; import org.junit.Test; @@ -22,7 +23,7 @@ * Conformance gate: Java's {@link GeneratorRegistry} stable-name set MUST equal the * {@code java} slice of the canonical cross-port manifest * {@code fixtures/generator-registry-conformance/registry.json} (ADR-0021 D3), and - * every entry's tier MUST agree with the manifest. + * every entry's tier AND layer MUST agree with the manifest. * *

If this fails, the fix is to reconcile the registry and the manifest in the same * change — never edit the manifest just to make a port pass.

@@ -32,6 +33,14 @@ public class GeneratorRegistryConformanceTest { private static final String PORT_ID = "java"; private static final ObjectMapper MAPPER = new ObjectMapper(); + /** + * The closed set of {@code layer} values, spelled out rather than derived from + * {@link Layer}. Enumerating the enum would make this gate agree with whatever the + * code says, which is the one thing a conformance gate must not do. + */ + private static final Set ALLOWED_LAYERS = + Set.of("model", "persistence", "api", "client", "docs", "capability"); + /** * Registered stable names whose {@code classname} is deliberately NOT a wirable * {@link Generator} on this port. PINNED, not exempted — @@ -97,6 +106,18 @@ private static Map manifestSliceForPort(JsonNode manifest) { return slice; } + /** Every manifest entry's {@code layer}, for EVERY port — this gate also checks the + * manifest's own well-formedness, which is not port-scoped. */ + private static Map manifestLayers(JsonNode manifest) { + Map layers = new TreeMap<>(); + Iterator> it = manifest.get("generators").fields(); + while (it.hasNext()) { + Map.Entry e = it.next(); + layers.put(e.getKey(), e.getValue().path("layer").asText("")); + } + return layers; + } + @Test public void registryNameSetEqualsManifestJavaSlice() throws Exception { Map expected = manifestSliceForPort(loadManifest()); @@ -138,6 +159,34 @@ public void registryTiersAgreeWithManifest() throws Exception { } } + @Test + public void registryLayersAgreeWithManifest() throws Exception { + JsonNode manifest = loadManifest(); + Map layers = manifestLayers(manifest); + Map actual = GeneratorRegistry.list(); + + for (Map.Entry e : manifestSliceForPort(manifest).entrySet()) { + GeneratorInfo info = actual.get(e.getKey()); + if (info == null) { + continue; // set-equality test reports the diff; avoid NPE noise here. + } + assertEquals("layer mismatch for stable name '" + e.getKey() + "'", + layers.get(e.getKey()), info.layer().manifestValue()); + } + } + + @Test + public void everyManifestEntryDeclaresOneOfTheSixLayers() throws Exception { + TreeSet bad = new TreeSet<>(); + for (Map.Entry e : manifestLayers(loadManifest()).entrySet()) { + if (!ALLOWED_LAYERS.contains(e.getValue())) { + bad.add(e.getKey() + "='" + e.getValue() + "'"); + } + } + assertTrue("manifest entries with a missing or unknown layer (allowed: " + + new TreeSet<>(ALLOWED_LAYERS) + "): " + bad, bad.isEmpty()); + } + @Test public void everyRegistryEntryIsWellFormed() { for (Map.Entry e : GeneratorRegistry.list().entrySet()) { @@ -148,6 +197,7 @@ public void everyRegistryEntryIsWellFormed() { assertTrue("description must be set for " + e.getKey(), info.description() != null && !info.description().isBlank()); assertTrue("tier must be set for " + e.getKey(), info.tier() != null); + assertTrue("layer must be set for " + e.getKey(), info.layer() != null); } } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTraceHelperCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTraceHelperCompileRunTest.java index 861b3fac2..26d6bdb30 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTraceHelperCompileRunTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/GeneratedTraceHelperCompileRunTest.java @@ -47,33 +47,13 @@ public class GeneratedTraceHelperCompileRunTest { private static final String PKG = "acme::ai"; /** - * Model: LlmCallBase (18 base fields, abstract) + GreetRequest/GreetResponse VOs - * + a concrete GreetingCall extending LlmCallBase with a nested template.prompt + * Model: GreetRequest/GreetResponse VOs + a concrete GreetingCall extending the + * SHIPPED metaobjects::ai::LlmCallBase, with a nested template.prompt * carrying @responseRef + typed voRequest/voResponse object columns. */ private static final String META = "{ \"metadata.root\": {" + " \"package\": \"" + PKG + "\"," + " \"children\": [" - + " { \"object.entity\": { \"name\": \"LlmCallBase\", \"abstract\": true, \"children\": [" - + " { \"field.uuid\": { \"name\": \"traceId\" } }," - + " { \"field.uuid\": { \"name\": \"spanId\" } }," - + " { \"field.uuid\": { \"name\": \"parentSpanId\" } }," - + " { \"field.string\": { \"name\": \"sessionId\" } }," - + " { \"field.string\": { \"name\": \"callType\" } }," - + " { \"field.string\": { \"name\": \"system\" } }," - + " { \"field.string\": { \"name\": \"requestModel\" } }," - + " { \"field.string\": { \"name\": \"responseModel\" } }," - + " { \"field.int\": { \"name\": \"inputTokens\" } }," - + " { \"field.int\": { \"name\": \"outputTokens\" } }," - + " { \"field.currency\": { \"name\": \"costMinor\", \"@currency\": \"USD\" } }," - + " { \"field.int\": { \"name\": \"latencyMs\" } }," - + " { \"field.string\": { \"name\": \"finishReason\" } }," - + " { \"field.string\": { \"name\": \"status\" } }," - + " { \"field.string\": { \"name\": \"errorDetail\" } }," - + " { \"field.timestamp\": { \"name\": \"startedAt\" } }," - + " { \"field.string\": { \"name\": \"llmRequest\", \"@dbColumnType\": \"jsonb\" } }," - + " { \"field.string\": { \"name\": \"llmResponse\", \"@dbColumnType\": \"jsonb\" } }" - + " ]}}," + " { \"object.value\": { \"name\": \"GreetRequest\", \"children\": [" + " { \"field.string\": { \"name\": \"name\", \"@required\": true } }" + " ]}}," @@ -82,7 +62,7 @@ public class GeneratedTraceHelperCompileRunTest { + " { \"field.int\": { \"name\": \"score\" } }" + " ]}}," + " { \"object.entity\": { \"name\": \"GreetingCall\"," - + " \"extends\": \"" + PKG + "::LlmCallBase\", \"children\": [" + + " \"extends\": \"metaobjects::ai::LlmCallBase\", \"children\": [" + " { \"source.rdb\": { \"@table\": \"llm_call\", \"@role\": \"primary\" } }," + " { \"identity.primary\": { \"name\": \"primary\", \"@fields\": [\"spanId\"] } }," + " { \"field.object\": { \"name\": \"voRequest\", \"@column\": \"voRequest\"," @@ -306,6 +286,7 @@ public void responseRefFqnBindsOwnPackageNotABareTailDecoyLoadedFirst() throws E MetaDataLoader loader = new MetaDataLoader( LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, "trace-responseref-fqn"); + loader.setLibraries(java.util.Collections.singletonList("ai")); loader.init(); // Decoy loads FIRST — under the pre-fix bare-tail-fallback bug this would win. loader.load(List.of( @@ -345,6 +326,11 @@ private MetaDataLoader newLoader(String name, String meta) { MetaDataLoader loader = new MetaDataLoader( LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, name); + // The SHIPPED base, through the opt-in. `trace-helper` keys on the `ai` + // manifest's ANCHOR and compares the FULL name (FR-043 §6), so a hand-copied + // base in the fixture's own package no longer matches — and a copy could drift + // from the real one silently, which is the bypass ADR-0024 named. + loader.setLibraries(java.util.Collections.singletonList("ai")); loader.init(); loader.load(List.of(new InMemoryStringSource(meta, name + "/meta.json"))); return loader; diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringAppliesToTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringAppliesToTest.java index 5b8b98cd4..67f9a0fe5 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringAppliesToTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringAppliesToTest.java @@ -82,15 +82,11 @@ public class SpringAppliesToTest { "@responseRef": "SummaryPayload", "@textRef": "responding/prompt" } }, - { "object.entity": { "name": "LlmCallBase", "abstract": true, "children": [ - { "field.uuid": { "name": "traceId" } }, - { "field.string": { "name": "status" } } - ] } }, { "object.value": { "name": "GreetResponse", "children": [ { "field.string": { "name": "greeting", "@required": true } } ] } }, { "object.entity": { "name": "GreetingCall", - "extends": "acme::shop::LlmCallBase", "children": [ + "extends": "metaobjects::ai::LlmCallBase", "children": [ { "source.rdb": { "@table": "llm_call" } }, { "identity.primary": { "name": "pk", "@fields": ["traceId"] } }, { "template.prompt": { @@ -111,9 +107,14 @@ public class SpringAppliesToTest { * needs is registered by loading the template classes, which this path does.) */ private MetaDataLoader loader() { + // The SHIPPED base, via the `libraries` opt-in: `trace-helper` keys on the `ai` + // manifest's ANCHOR now (FR-043 §6) and compares the full name, so a bespoke + // `acme::shop::LlmCallBase` no longer matches — which is the latent bug that + // change removes, and which this fixture used to depend on. MetaDataLoader loader = new MetaDataLoader( LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, "spring-applies-to"); + loader.setLibraries(java.util.Collections.singletonList("ai")); loader.init(); loader.load(List.of(new InMemoryStringSource(FIXTURE, "applies-to/meta.json"))); return loader; @@ -196,7 +197,7 @@ public void traceHelperAppliesToLlmCallWithResponseRef() throws Exception { MetaDataLoader loader = loader(); MetaObject greetingCall = loader.getMetaObjectByName("acme::shop::GreetingCall"); MetaObject author = loader.getMetaObjectByName("acme::shop::Author"); - MetaObject base = loader.getMetaObjectByName("acme::shop::LlmCallBase"); + MetaObject base = loader.getMetaObjectByName("metaobjects::ai::LlmCallBase"); assertTrue("concrete LlmCallBase subclass with prompt @responseRef emits a trace helper", LlmTraceHelperGenerator.appliesTo(greetingCall)); diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/TraceHelperOnShippedLibraryTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/TraceHelperOnShippedLibraryTest.java index 38cdbc669..b7bc76cc0 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/TraceHelperOnShippedLibraryTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/TraceHelperOnShippedLibraryTest.java @@ -138,6 +138,66 @@ public void theShippedBaseFieldsAreExactlyWhatTheRecorderWrites() { new TreeSet<>(baseFields), new TreeSet<>(callFields)); } + /** + * FR-043 §6 — the generator keys on the manifest's ANCHOR, not on a name compiled + * into this class. + * + *

The floor the design names for a port whose plumbing slips is exactly this: + * the port's constant equals the manifest anchor. Here the constant IS the manifest + * read, so the assertion is that the read produced the thing the library declares — + * a manifest whose {@code generators} block was renamed or emptied leaves the + * generator matching nothing, and it should fail here rather than in silence.

+ */ + @Test + public void theAnchorComesFromTheManifest() { + assertEquals("the generator's anchors ARE the manifest's, for its own stable name", + LibrarySources.generatorAnchors(LlmTraceHelperGenerator.STABLE_NAME), + LlmTraceHelperGenerator.ANCHOR_FQNS); + assertTrue("the ai manifest must declare the trace-helper anchor", + LlmTraceHelperGenerator.ANCHOR_FQNS.contains("metaobjects::ai::LlmCallBase")); + } + + /** + * The latent bug the anchor removes: an adopter's own abstract of the same NAME. + * + *

The old predicate compared {@code getShortName()}, so any entity called + * {@code LlmCallBase} in any package triggered a helper — one that writes the + * shipped base's columns, which that entity does not declare.

+ */ + @Test + public void anAdoptersOwnBaseOfTheSameNameDoesNotMatch() { + String meta = "{\"metadata.root\": {" + + " \"package\": \"acme::app\"," + + " \"children\": [" + + " { \"object.value\": { \"name\": \"GreetResponse\", \"children\": [" + + " { \"field.string\": { \"name\": \"greeting\", \"@required\": true } }" + + " ]}}," + + " { \"object.entity\": { \"name\": \"LlmCallBase\", \"abstract\": true, \"children\": [" + + " { \"field.uuid\": { \"name\": \"spanId\" } }" + + " ]}}," + + " { \"object.entity\": { \"name\": \"ImpostorCall\"," + + " \"extends\": \"acme::app::LlmCallBase\", \"children\": [" + + " { \"source.rdb\": { \"@table\": \"impostor\", \"@role\": \"primary\" } }," + + " { \"identity.primary\": { \"name\": \"pk\", \"@fields\": [\"spanId\"] } }," + + " { \"template.prompt\": { \"name\": \"p\"," + + " \"@payloadRef\": \"acme::app::GreetResponse\"," + + " \"@responseRef\": \"acme::app::GreetResponse\" } }" + + " ]}}" + + " ]" + + "}}"; + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), + MetaDataLoader.SUBTYPE_MANUAL, "trace-impostor"); + loader.init(); + loader.load(List.of(new InMemoryStringSource(meta, "trace-impostor/meta.json"))); + + MetaObject impostor = loader.getMetaObjectByName("acme::app::ImpostorCall"); + assertNotNull(impostor); + org.junit.Assert.assertFalse( + "an entity extending an adopter's OWN LlmCallBase must not emit a trace helper", + LlmTraceHelperGenerator.appliesTo(impostor)); + } + @Test public void theLibraryIsNotLoadedUnlessAskedFor() { // The negative arm for the generator path specifically: without the opt-in there is diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/LlmCallTraceRoundTripTest.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/LlmCallTraceRoundTripTest.java index e506cae8b..9953d015b 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/LlmCallTraceRoundTripTest.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/LlmCallTraceRoundTripTest.java @@ -40,7 +40,7 @@ * *

Proves that a typed LLM-call trace persists and reads back through OMDB * against a live Postgres: a trace entity extending the SHIPPED - * {@code metaobjects::ai::LlmCallBase} (loaded from {@code library/ai/llm-call.yaml}) + * {@code metaobjects::ai::LlmCallBase} (loaded from {@code library/ai/model.yaml}) * with an explicit typed {@code voResponse} ({@code field.object} + * {@code @objectRef} + {@code @dbType:jsonb}). Asserts BOTH the raw envelope * (the 18 LlmCallBase fields, raw {@code llmRequest}/{@code llmResponse}) AND the @@ -80,7 +80,7 @@ void typedTraceRoundTripsThroughPostgres() throws Exception { reg.register(() -> Map.of("metaobjects::ai::GreetingResponse", GreetingResponse.class)); ObjectClassRegistry.setGlobal(reg); - // Load the SHIPPED library/ai/llm-call.yaml (LlmCallBase) + the test + // Load the SHIPPED library/ai/{model,db}.yaml (LlmCallBase) + the test // trace entity from one loader (deferred extends resolution merges them). MetaDataLoader loader = loadAiTraceMetadata(); MetaDataLoaderRegistry registry = @@ -199,16 +199,18 @@ void recorderNeverThrowsAndCallsOnError() { // ----------------------------------------------------------------------- /** - * Load the shipped {@code library/ai/llm-call.yaml} (real LlmCallBase) by file + * Load the shipped {@code library/ai/model.yaml} + {@code db.yaml} (real LlmCallBase) by file * URI, plus the test trace-entity resource. Demonstrates loading the shipped * AI library metadata in a Java test (Java has no {@code libraries:} loader * option wired yet, so the library YAML is loaded directly as a source). */ private static MetaDataLoader loadAiTraceMetadata() { - Path libraryYaml = findRepoFile("library/ai/llm-call.yaml"); - URI libUri = URIHelper.toURI("model:file:" + libraryYaml.toAbsolutePath()); + Path libraryModel = findRepoFile("library/ai/model.yaml"); + Path libraryDb = findRepoFile("library/ai/db.yaml"); + URI modelUri = URIHelper.toURI("model:file:" + libraryModel.toAbsolutePath()); + URI dbUri = URIHelper.toURI("model:file:" + libraryDb.toAbsolutePath()); URI entityUri = URIHelper.toURI("model:resource:meta.ai-trace.yaml"); - return MetaDataLoader.fromUris("test-ai-trace", List.of(libUri, entityUri)); + return MetaDataLoader.fromUris("test-ai-trace", List.of(modelUri, dbUri, entityUri)); } /** Walk up from the working dir to locate a repo-relative file. */ diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 35a155e14..dc71bde14 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -309,6 +309,33 @@ public enum ErrorCode { /** Phase-1 metadata-source-resolution: no metadata collection was discovered — no config declaring {@code sources}, and no default {@code metaobjects/} directory. */ ERR_COLLECTION_NOT_FOUND, + /** + * FR-043 — {@code .metaobjects/config.json}'s {@code libraries} names a shipped library + * or layer this build does not have. Refused with the available tokens rather than + * skipped: skipped, it resurfaces as {@code ERR_UNRESOLVED_SUPER} against the adopter's + * own metadata, which is the wrong place to send someone looking. + */ + ERR_UNKNOWN_LIBRARY, + + /** + * FR-043: a node is declared by BOTH an adopter's own metadata and a shipped library + * the project opts into — the {@code meta eject} copy with the library still named in + * {@code libraries}. The two merge silently and ASYMMETRICALLY: additions take, + * deletions do not, because the library still declares what was removed. + * + *

Raised by the TypeScript SDK's load path; registered in every port so the shared + * corpus list stays one set.

+ */ + ERR_LIBRARY_PACKAGE_COLLISION, + + /** + * FR-043: a NEW top-level node is declared into a package a shipped library owns while + * that library is opted in — a later release of the library may ship a node of that + * name and merge into it. {@code overlay: true} on one of the library's OWN nodes is + * the documented amendment door and is untouched. + */ + ERR_LIBRARY_PACKAGE_NOT_OWNED, + /** * FR-016 / ADR-0018: a {@code source.rdb} declares a kind-aware physical-name * alias ({@code @view} / {@code @materializedView} / {@code @proc} / diff --git a/server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java b/server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java index a24d572a8..0847f99c5 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java +++ b/server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java @@ -25,9 +25,22 @@ private EmbeddedLibrary() {} /** Ref to exact file contents, insertion-ordered by ref. */ public static final Map CONTENT; + /** Library name to the exact text of its {@code library.json} manifest. */ + public static final Map MANIFESTS; + static { Map m = new LinkedHashMap<>(); - m.put("ai/llm-call", "# library/ai/llm-call.yaml\n# MetaObjects-shipped standard metadata. Adopters opt in via the loader's\n# `libraries: [\"ai\"]` option, then `extends: \"metaobjects::ai::LlmCallBase\"`.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n children:\n - source.rdb: { table: llm_call, role: primary }\n - identity.primary: { name: id, fields: [\"spanId\"] }\n"); + m.put("ai/db", "# library/ai/db.yaml — the DB PERSISTENCE layer for metaobjects::ai.\n#\n# Opted into as `\"ai/db\"`, which IMPLIES `\"ai\"`: `LlmCall` is declared in model.yaml and\n# this file only re-opens it, so without the core layer the overlay has no target.\n#\n# `LlmCall` is the concrete, table-backed instance of the abstract envelope. An adopter\n# who wants their OWN table (a different name, extra columns, a different id strategy)\n# extends `LlmCallBase` in their own metadata and never opts into this layer at all.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCall\n overlay: true\n children:\n - source.rdb: { table: llm_call, role: primary }\n"); + m.put("ai/model", "# library/ai/model.yaml — the CORE layer: the LLM-call trace envelope.\n#\n# Adopters opt in via `libraries: [\"ai\"]`, then `extends: \"metaobjects::ai::LlmCallBase\"`.\n#\n# This layer declares NO `source.rdb`, so opting into `\"ai\"` alone adds zero tables and\n# zero generated code — the design is present and resolvable, and nothing else happens\n# until the adopter adds `\"ai/db\"`. See library/iam/model.yaml for the full rationale.\n#\n# This file was split out of the former `library/ai/llm-call.yaml`, which shipped the\n# abstract base and a concrete `LlmCall` carrying `source.rdb` together. That was\n# recorded as an accepted wart on the grounds that splitting would change what existing\n# `ai` adopters get; a sweep of the estate found there are none, so it was closed rather\n# than documented (FR-043 Amendment 1).\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n description: The concrete trace row. Its `source.rdb` lives in db.yaml, so opting into the core layer alone declares the shape without proposing a table.\n children:\n - identity.primary: { name: id, fields: [\"spanId\"] }\n"); + m.put("ai/requirements", "# library/ai/requirements.yaml — what the LLM-call trace envelope PROMISES.\n#\n# A RETROFIT, not new design: llm-call.yaml landed 2026-06-03 and `requirement.functional`\n# first appears 2026-08-11, so the library could not have carried requirements when it was\n# written. That is why this file is worth reading as a worked example — it shows what\n# declaring the design of something that already exists actually turns up.\n#\n# The entry that earns its keep is `typedIo`, honestly `partial` + `accepted`: the library\n# declares the ENVELOPE and the adopter declares the typed VO columns. Recording that seam\n# in the ledger is where an agent meets it, before adding a fourth trace column.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. An L4 names the OBJECT it is about;\n# the fields that carry it hang off it as an L5 child. Writing the fields at L4 is\n# ERR_REQUIREMENT_L4_NOT_OBJECT, and writing the concerns as SIBLINGS of the L2 leaves the\n# L2 claiming nothing — both of which this file did until the standalone verify gate\n# existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::ai\n children:\n - requirement.functional:\n name: llmTracing\n level: 2\n status: live\n statement: Every call to a language model leaves a row that says what was asked, what came back, what it cost and how long it took.\n counterexample: A spend figure nobody can attribute to a call.\n description: The segment this library covers. Its three children below are the concerns it decomposes into.\n children:\n - requirement.functional:\n name: envelope\n level: 4\n status: live\n statement: A trace row identifies its call and its place in a trace — trace, span, parent span, session, call type, system.\n counterexample: A log line that cannot be joined to the request that produced it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: traceAddressing\n level: 5\n status: live\n statement: The four addressing columns are declared on the base — trace, span, parent span and session.\n counterexample: A row whose place in a trace is inferred from insertion order.\n description: >-\n The member grain exists here so the claim RESOLVES against the fields\n themselves: renaming or dropping one of them dangles this reference and\n fails the build, which naming the object alone would not.\n implementedBy: [LlmCallBase.traceId, LlmCallBase.spanId, LlmCallBase.parentSpanId, LlmCallBase.sessionId]\n\n - requirement.functional:\n name: accounting\n level: 4\n status: live\n statement: A trace row carries the tokens in, the tokens out, and the cost in integer minor units.\n counterexample: A cost stored as a float.\n description: >-\n `field.currency` — integer minor units on the wire, always. Float arithmetic for\n money is forbidden by the cross-port wire contract, and a spend total is exactly\n the sum that exposes it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: tokenAndCostColumns\n level: 5\n status: live\n statement: Tokens in, tokens out and cost are three declared columns, the cost a field.currency.\n counterexample: A cost column declared as a double.\n implementedBy: [LlmCallBase.inputTokens, LlmCallBase.outputTokens, LlmCallBase.costMinor]\n\n - requirement.functional:\n name: typedIo\n level: 4\n status: partial\n disposition: accepted\n statement: The request and response bodies are stored as structured jsonb, not as opaque text.\n counterexample: A prompt stored as a string nobody can query a field out of.\n notes: >-\n The library declares the two columns as generic jsonb with no `@objectRef`,\n because it cannot know the adopter's request/response shape. Typing them is the\n ADOPTER's move: declare an `object.value` and overlay the field with\n `@objectRef` + `@storage: jsonb`. This is the seam ADR-0024 drew, recorded here\n rather than in prose so it is in the ledger an agent reads before adding a\n fourth trace column of its own.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: jsonbBodies\n level: 5\n status: live\n statement: The request and response bodies are declared as jsonb columns on the base.\n counterexample: A prompt stored in a text column.\n description: >-\n `live` where its parent is `partial`, and the split is the point: the\n COLUMNS are shipped and this claim is fully realised; what is outstanding\n is the TYPING of them, which is the parent's gap and the adopter's move.\n implementedBy: [LlmCallBase.llmRequest, LlmCallBase.llmResponse]\n\n - requirement.architectural:\n name: traceRowsCarryTiming\n status: live\n statement: Every trace row records when the call started and how long it took.\n counterexample: A latency figure derived from log timestamps after the fact.\n description: >-\n Architectural, so it propagates down `extends` to every adopter entity deriving\n from LlmCallBase — which is the point: an adopter's own trace table is claimed\n by this requirement for free, and dropping the columns breaks the build.\n implementedBy: [LlmCallBase]\n\n - requirement.architectural:\n name: traceRowsCarryOutcome\n status: live\n statement: Every trace row records how the call ended — a status, a finish reason, and the error detail when there was one.\n counterexample: A failed call indistinguishable from one that never happened.\n implementedBy: [LlmCallBase]\n"); + m.put("iam/db", "# library/iam/db.yaml — the DB PERSISTENCE layer for metaobjects::iam.\n#\n# Opted into as `\"iam/db\"`, which IMPLIES `\"iam\"`: this file is nothing but\n# `overlay: true` redeclarations, and an overlay whose target was never declared is\n# ERR_OVERLAY_NO_TARGET.\n#\n# It carries exactly two kinds of child — `source.rdb` and `index.lookup` — and nothing\n# else. The field set, the identities and the relationships all live in model.yaml,\n# because they are the DESIGN; what lives here is where the rows go and which lookups are\n# worth an index. Add a field here and the core layer stops being the whole model, which\n# is the thing the split exists to guarantee.\n#\n# Physical names are `iam_`-prefixed. Two reasons, both real: `user` and `group` are\n# reserved words in Postgres, and an adopter very likely has tables of their own by those\n# names. A library that collides on a table name is a library nobody can adopt.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: User\n overlay: true\n children:\n - source.rdb: { table: iam_user, role: primary }\n\n - object.entity:\n name: GroupType\n overlay: true\n children:\n - source.rdb: { table: iam_group_type, role: primary }\n\n - object.entity:\n name: Group\n overlay: true\n children:\n - source.rdb: { table: iam_group, role: primary }\n # Nesting is walked parent-ward constantly; the FK alone gives no index.\n - index.lookup: { name: ixParent, fields: [parentId] }\n\n - object.entity:\n name: Role\n overlay: true\n children:\n - source.rdb: { table: iam_role, role: primary }\n\n - object.entity:\n name: Permission\n overlay: true\n children:\n - source.rdb: { table: iam_permission, role: primary }\n\n - object.entity:\n name: GroupMember\n overlay: true\n children:\n - source.rdb: { table: iam_group_member, role: primary }\n # The composite PK covers (userId, groupId), so \"who is in this group?\" —\n # the other direction — has no index without this one. Same reasoning for\n # every ixSecond below.\n - index.lookup: { name: ixGroup, fields: [groupId] }\n\n - object.entity:\n name: RolePermission\n overlay: true\n children:\n - source.rdb: { table: iam_role_permission, role: primary }\n - index.lookup: { name: ixPermission, fields: [permissionId] }\n\n - object.entity:\n name: UserRole\n overlay: true\n children:\n - source.rdb: { table: iam_user_role, role: primary }\n - index.lookup: { name: ixRole, fields: [roleId] }\n\n - object.entity:\n name: GroupMemberRole\n overlay: true\n children:\n - source.rdb: { table: iam_group_member_role, role: primary }\n # \"who holds this role in this group?\" — the scoped-grant read.\n - index.lookup: { name: ixGroupRole, fields: [groupId, roleId] }\n"); + m.put("iam/model", "# library/iam/model.yaml — the CORE layer: identity and access management.\n#\n# Adopters opt in via `libraries: [\"iam\"]` in .metaobjects/config.json.\n#\n# This layer declares NO `source.rdb`, and that is the whole point of the split. A\n# sourceless object is inert by a contract that already ships: migrate skips an object\n# with no writable source, and codegen emits no route, queries, hooks, grid or form for\n# one (both citing #248 — persistability derives from source presence, never from the\n# object subtype). It still gets a type-only interface, so `extends` and reference work.\n#\n# So `libraries: [\"iam\"]` adds ZERO tables and ZERO generated code. What an adopter gains\n# is the design being present and resolvable: an agent working in the repo knows the\n# capability exists and can draw on it, and nothing else happens until the adopter adds\n# `\"iam/db\"`.\n#\n# Authoring discipline (FR-043 §3.1), so the departures are visible:\n# - `field.uuid` + `generation: uuid` on principals; composite ASSIGNED keys on\n# junctions. Never `increment` — a library cannot know the adopter's id strategy.\n# - Physical names carry the `iam_` prefix (in db.yaml): `user` and `group` are\n# reserved words in Postgres, and an adopter has tables of their own.\n# - No adopter-facing profile data. That arrives by `overlay: true`.\n# - No credentials. See requirements.yaml → `noCredentialsOnUser`.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: IamBase\n abstract: true\n description: Shared shape of every iam principal and definition — a stable uuid plus change timestamps. Junctions do not extend it; they are addressed by their participants.\n children:\n - field.uuid: { name: id, required: true }\n - field.timestamp: { name: createdAt, autoSet: onCreate }\n - field.timestamp: { name: updatedAt, autoSet: onUpdate }\n\n - object.entity:\n name: User\n extends: IamBase\n description: A person or service account that can be granted access. Carries no authentication secret of any kind — see the noCredentialsOnUser requirement.\n children:\n - field.string: { name: username, required: true, maxLength: 64, filterable: true }\n - field.string: { name: email, required: true, maxLength: 254, stringFormat: email, filterable: true }\n - field.string: { name: displayName, maxLength: 120 }\n # NOT `filterable: true`, deliberately. The loader warns when a filterable\n # field is in no identity — filtering on it sequential-scans — and a library\n # must not ship a warning to every adopter. `username` and `email` carry it\n # because they have identity.secondary; `status` does not. An adopter who\n # wants to filter on status overlays `filterable` AND an index together,\n # which is exactly what the layer split is for.\n - field.enum: { name: status, required: true, values: [invited, active, suspended, closed], default: active }\n - field.timestamp: { name: emailVerifiedAt }\n - field.timestamp: { name: lastSeenAt }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqUsername, fields: [username] }\n - identity.secondary: { name: uqEmail, fields: [email] }\n - relationship.association: { name: groups, objectRef: Group, cardinality: many, through: GroupMember }\n - relationship.association: { name: roles, objectRef: Role, cardinality: many, through: UserRole }\n\n - object.entity:\n name: GroupType\n extends: IamBase\n description: What KIND of group this is — a team, a tenant, a project. An entity rather than an enum, because \"which roles may be held in this kind of group\" is data an adopter extends, and an enum's values cannot be extended by overlay.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n - object.entity:\n name: Group\n extends: IamBase\n description: A nestable collection of users, of a declared GroupType. Nesting is by parentId; acyclicity is an invariant the schema cannot express — see the acyclicGroupNesting requirement.\n children:\n - field.uuid: { name: groupTypeId, required: true }\n - field.uuid: { name: parentId }\n - field.string: { name: key, required: true, maxLength: 64 }\n # Not filterable for the same reason as User.status above.\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - identity.reference: { name: fkParent, fields: [parentId], references: Group, onDelete: restrict }\n\n - object.entity:\n name: Role\n extends: IamBase\n description: A reusable bundle of permissions. Code never compares a role NAME to a literal — it asks whether a user holds a permission, and the mapping is data.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - field.uuid: { name: groupTypeId, description: \"When set, this role may be held only within groups of this type; absent means grantable anywhere.\" }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - relationship.association: { name: permissions, objectRef: Permission, cardinality: many, through: RolePermission }\n\n - object.entity:\n name: Permission\n extends: IamBase\n description: \"The assignable unit — a stable : key the application checks against. An entity, not an enum, on ADR-0037's own reasoning: it has its own identity, its own lifecycle, and a junction with real foreign keys.\"\n children:\n - field.string: { name: key, required: true, maxLength: 128, description: \"Stable : key the application checks against.\" }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n # ---- grant surface: every grant is a row, addressed by its participants ----\n #\n # Junctions do NOT extend IamBase: they have no identity of their own, and adding a\n # surrogate uuid to a row whose identity IS its participants invites a duplicate.\n\n - object.entity:\n name: GroupMember\n description: A user's membership of a group.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.timestamp: { name: joinedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n\n - object.entity:\n name: RolePermission\n description: A permission granted by a role.\n children:\n - field.uuid: { name: roleId, required: true }\n - field.uuid: { name: permissionId, required: true }\n - identity.primary: { name: pk, fields: [roleId, permissionId], generation: assigned }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: cascade }\n - identity.reference: { name: fkPermission, fields: [permissionId], references: Permission, onDelete: restrict }\n\n - object.entity:\n name: UserRole\n description: A system-wide grant of a role to a user.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n\n - object.entity:\n name: GroupMemberRole\n description: A grant of a role to a user WITHIN one group. Three foreign keys, so it is not an M:N @through junction (which must declare exactly two identity.reference children); it is read by explicit finders.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n"); + m.put("iam/requirements", "# library/iam/requirements.yaml — what this library's design PROMISES.\n#\n# This is what makes iam a library rather than a schema snippet. Without requirements an\n# adopter gets nine tables; with them they get nine tables plus a build that is held to\n# \"no authorization decision is hard-wired to a name\", which no snippet can do.\n#\n# Two reading rules, both load-bearing:\n#\n# `live` here means \"the model AS SHIPPED realises this\" — never \"your application\n# does\". A ledger binds to model nodes; runtime guarantees are the runtime's tests, and\n# this library does not invent a way to point a requirement at code (@verifiedBy was\n# retired for exactly that). Behaviour the model cannot carry ships as `partial` +\n# `disposition: accepted` with a notes sentence naming what the adopter must do.\n#\n# The functional tree roots at L2, not L1. L1 is the adopter's SOLUTION, and a library\n# is by definition a segment of someone else's. Architectural claims ship flat.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. The concerns are CHILDREN of the L2\n# rather than its siblings, and an L4 names the OBJECT it is about while the field that\n# carries it hangs off it as an L5 child. Written flat, the L2 claims nothing in its whole\n# subtree; written at L4, a field reference is ERR_REQUIREMENT_L4_NOT_OBJECT. Both shipped\n# here until the standalone verify gate existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::iam\n children:\n # ---- functional: the L2 segment and the concerns nested under it --------\n - requirement.functional:\n name: accessControl\n level: 2\n status: live\n statement: Who may do what is answered from stored grants, never from a name compared to a literal in code.\n counterexample: A branch that reads `if (user.role === \"admin\")`.\n description: The segment this library covers. The concerns beneath it are what it decomposes into.\n children:\n - requirement.functional:\n name: identity\n level: 4\n status: live\n statement: A person or service account is represented once, addressed by a uuid, and reachable by username or email.\n counterexample: Two rows for the same person because the email changed.\n implementedBy: [User]\n\n - requirement.functional:\n name: grouping\n level: 4\n status: live\n statement: Users are collected into typed, nestable groups, and the kind of group is data rather than a hard-coded set.\n counterexample: A `teamOrTenant` boolean.\n implementedBy: [Group, GroupType, GroupMember]\n\n - requirement.functional:\n name: acyclicGroupNesting\n level: 4\n status: partial\n disposition: accepted\n statement: A group is never its own ancestor.\n counterexample: Two groups each naming the other as parent.\n notes: >-\n The schema cannot express this — a self-referencing FK admits a cycle, and the\n only relational forms that would catch it (a recursive CHECK, a closure table\n maintained by trigger) are DB-specific and would not survive three dialects.\n The adopter enforces it where the write happens. Recorded rather than omitted\n so an agent reading the ledger before adding a parent-setting endpoint sees the\n obligation.\n implementedBy: [Group]\n\n - requirement.functional:\n name: grants\n level: 4\n status: live\n statement: A role is granted to a user either system-wide or scoped to one group, and both are ordinary rows.\n counterexample: A nullable `groupId` on one grant table, where NULL means \"everywhere\".\n description: >-\n Two junctions, not one with a nullable scope. A NULL in a unique key is DISTINCT\n from every other NULL in SQL, so a nullable-scope design lets the same global\n grant be inserted twice; the fix needs a partial index whose expression carries\n a physical column name. Two composite-keyed tables need no escape hatch and\n survive three dialects and five ports unchanged.\n implementedBy: [UserRole, GroupMemberRole]\n\n - requirement.functional:\n name: roleScopedToGroupType\n level: 4\n status: partial\n disposition: accepted\n statement: A role bound to a group type is granted only within groups of that type.\n counterexample: A \"tenant admin\" role granted inside a project group.\n notes: >-\n Expressing this relationally needs the grant row to carry the group's type and\n a composite FK back to (group, type) — three foreign keys deep, unverified\n across five ports' DDL and ORM paths. The adopter checks it at the point of\n grant. The declared half is the L5 child below; the enforcement is not.\n implementedBy: [Role, GroupMemberRole]\n children:\n - requirement.functional:\n name: roleDeclaresItsGroupType\n level: 5\n status: live\n statement: A role declares the group type it is bound to, as a nullable reference.\n counterexample: A role whose intended scope is recoverable only from its name.\n description: >-\n `live` where its parent is `partial`, and the split is grain as much as\n verdict: the DECLARATION is shipped and resolves against the field itself,\n so dropping the column fails the build — while the ENFORCEMENT, which no\n schema here can carry, stays the parent's accepted gap.\n implementedBy: [Role.groupTypeId]\n\n - requirement.functional:\n name: decision\n level: 4\n status: live\n statement: An authorization decision is the question \"does this user hold this permission key\", answered from rows.\n counterexample: A hard-coded list of usernames that bypass a check.\n implementedBy: [Permission, RolePermission]\n\n # ---- architectural: prohibitions in force --------------------------------\n\n - requirement.architectural:\n name: grantsAreRows\n status: live\n statement: A grant exists only as a stored row; nothing is granted by naming, position or convention.\n counterexample: A superuser recognised by username.\n implementedBy: [UserRole, GroupMemberRole, RolePermission, GroupMember]\n\n - requirement.architectural:\n name: noCredentialsOnUser\n status: live\n statement: A user row carries no authentication secret — no password, no hash, no knowledge-based question or answer.\n counterexample: A password or secret-answer column on the user table.\n description: >-\n Authentication is a separate capability with an entity per factor; this library\n is identity and authorization only.\n notes: >-\n This is the one thing every reader of a user table proposes adding, and a real\n legacy model of this shape stored a length-bounded plaintext password and a\n knowledge-based secret pair on the user row. Stating it as a prohibition IN\n FORCE — claimable, and rendered on agent/requirements.md — is what stops an\n agent extending \"the user model\" from re-deriving it on sight. It is\n `architectural`, not `retired`: retired is chartered for a capability built\n here and removed, and this library never built one.\n implementedBy: [User]\n\n - requirement.architectural:\n name: principalDeletionRevokesGrants\n status: live\n statement: Deleting a user or group removes its grants; deleting a role or permission still in use is refused.\n counterexample: A grant row pointing at a user who no longer exists.\n description: The referential rule in one sentence — cascade from a principal, restrict from a definition.\n implementedBy: [GroupMember, UserRole, GroupMemberRole, RolePermission]\n\n - requirement.architectural:\n name: stableIdentifiers\n status: live\n statement: Every principal and definition is addressed by a uuid that never changes; every grant by its participants.\n counterexample: A group referenced by its display name.\n implementedBy: [IamBase]\n"); CONTENT = Collections.unmodifiableMap(m); + + Map n = new LinkedHashMap<>(); + n.put("ai", "{\n \"$comment\": \"Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.\",\n \"name\": \"ai\",\n \"kind\": \"feature\",\n \"stability\": \"stable\",\n \"since\": \"0.20.0\",\n \"description\": \"The LLM-call trace envelope: what was asked, what came back, what it cost, how long it took.\",\n \"useWhen\": \"the application calls a language model and someone will ask what it cost or why a call failed\",\n \"packages\": [\"metaobjects::ai\"],\n \"layers\": {\n \"\": { \"refs\": [\"ai/model\", \"ai/requirements\"], \"description\": \"the core model and its requirements — sourceless, so it adds no tables\" },\n \"db\": { \"refs\": [\"ai/db\"], \"description\": \"the concrete llm_call table\" }\n },\n \"generators\": [\n { \"name\": \"trace-helper\", \"anchor\": \"metaobjects::ai::LlmCallBase\" }\n ],\n \"runtime\": {\n \"typescript\": [\"@metaobjectsdev/runtime-ts\"]\n }\n}\n"); + n.put("iam", "{\n \"$comment\": \"Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.\",\n \"name\": \"iam\",\n \"kind\": \"feature\",\n \"stability\": \"preview\",\n \"since\": \"1.1.0\",\n \"description\": \"Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group.\",\n \"useWhen\": \"the application has people who log in and things some of them may not do\",\n \"packages\": [\"metaobjects::iam\"],\n \"layers\": {\n \"\": { \"refs\": [\"iam/model\", \"iam/requirements\"], \"description\": \"the core model and its requirements — sourceless, so it adds no tables\" },\n \"db\": { \"refs\": [\"iam/db\"], \"description\": \"nine tables, iam_-prefixed, plus the lookup indexes the composite keys do not cover\" }\n },\n \"generators\": [],\n \"runtime\": {}\n}\n"); + MANIFESTS = Collections.unmodifiableMap(n); } } diff --git a/server/java/metadata/src/main/java/com/metaobjects/library/LibrarySources.java b/server/java/metadata/src/main/java/com/metaobjects/library/LibrarySources.java index 17f46333c..f2077c05e 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/library/LibrarySources.java +++ b/server/java/metadata/src/main/java/com/metaobjects/library/LibrarySources.java @@ -30,24 +30,134 @@ public final class LibrarySources { private LibrarySources() {} - /** Package to ordered refs, derived from the generated embed so that adding a library - * file (which regenerates {@link EmbeddedLibrary}) needs no edit here. */ - private static final Map> REFS_BY_PACKAGE = buildRefsByPackage(); + /** + * Library name to its manifest's LAYERS: layer token to that layer's ordered refs. The + * CORE layer's token is the empty string. + * + *

Read from the embedded {@code library.json} manifests, not derived from the ref + * names. This used to be package-granular — every ref under a library came back for a + * bare {@code "ai"} — which under the layered design (FR-043 Amendment 1) would hand an + * adopter the db layer they did not ask for, and with it a migration proposing tables.

+ */ + private static final Map>> LAYERS_BY_LIBRARY = buildLayers(); /** Resolved once per process; {@code null} value means "looked, not present". */ private static volatile Path cachedDir; private static volatile boolean dirResolved; - private static Map> buildRefsByPackage() { - Map> map = new LinkedHashMap<>(); - for (String ref : new TreeSet<>(EmbeddedLibrary.CONTENT.keySet())) { - int slash = ref.indexOf('/'); - if (slash <= 0) continue; - map.computeIfAbsent(ref.substring(0, slash), k -> new ArrayList<>()).add(ref); + private static Map>> buildLayers() { + Map>> map = new LinkedHashMap<>(); + for (String name : new TreeSet<>(EmbeddedLibrary.MANIFESTS.keySet())) { + // Hand-parsed rather than pulled through Jackson: this module is the metadata + // core and does not depend on a JSON binder, and the shape read here is four + // keys deep in a file this repo generates. A binder would be a dependency + // added for a manifest we also write. + map.put(name, parseLayers(EmbeddedLibrary.MANIFESTS.get(name))); } return map; } + /** The {@code "layers"} object of a manifest: token to refs, in declaration order. */ + private static Map> parseLayers(String manifestJson) { + Map> layers = new LinkedHashMap<>(); + java.util.regex.Matcher block = java.util.regex.Pattern + .compile("\"layers\"\\s*:\\s*\\{(.*?)\\n \\}", java.util.regex.Pattern.DOTALL) + .matcher(manifestJson); + if (!block.find()) return layers; + java.util.regex.Matcher entry = java.util.regex.Pattern + .compile("\"([^\"]*)\"\\s*:\\s*\\{[^}]*?\"refs\"\\s*:\\s*\\[([^\\]]*)\\]", java.util.regex.Pattern.DOTALL) + .matcher(block.group(1)); + while (entry.find()) { + List refs = new ArrayList<>(); + java.util.regex.Matcher ref = java.util.regex.Pattern.compile("\"([^\"]+)\"").matcher(entry.group(2)); + while (ref.find()) refs.add(ref.group(1)); + layers.put(entry.group(1), refs); + } + return layers; + } + + /** + * The FQNs a generator's ANCHOR declarations name, across every shipped manifest + * (FR-043 §6). + * + *

An anchor is the library node a generator keys on. Reading it here is what + * retires a hard-coded entity name in the generator: {@code LlmTraceHelperGenerator} + * compared a short name, so ANY adopter entity called {@code LlmCallBase}, in any + * package, triggered it — and the shipped abstract was never actually what matched.

+ * + *

Hand-parsed for the reason {@link #parseLayers} is: this module is the metadata + * core and does not depend on a JSON binder, and the file is one this repo + * generates. Each object in the {@code "generators"} array is read for its own + * {@code name} and {@code anchor}, so key ORDER inside it does not matter.

+ * + * @param generatorName the cross-port stable name, e.g. {@code "trace-helper"} + * @return the anchor FQNs, in manifest order; empty when none declares one + */ + public static List generatorAnchors(String generatorName) { + List out = new ArrayList<>(); + for (String name : new TreeSet<>(EmbeddedLibrary.MANIFESTS.keySet())) { + String manifest = EmbeddedLibrary.MANIFESTS.get(name); + java.util.regex.Matcher block = java.util.regex.Pattern + .compile("\"generators\"\\s*:\\s*\\[(.*?)\\]", java.util.regex.Pattern.DOTALL) + .matcher(manifest); + if (!block.find()) continue; + java.util.regex.Matcher obj = java.util.regex.Pattern + .compile("\\{([^}]*)\\}").matcher(block.group(1)); + while (obj.find()) { + String body = obj.group(1); + String declared = manifestField(body, "name"); + String anchor = manifestField(body, "anchor"); + if (generatorName.equals(declared) && anchor != null) out.add(anchor); + } + } + return out; + } + + /** One {@code "key": "value"} string field out of a flat JSON object body. */ + private static String manifestField(String objectBody, String key) { + java.util.regex.Matcher m = java.util.regex.Pattern + .compile("\"" + key + "\"\\s*:\\s*\"([^\"]*)\"").matcher(objectBody); + return m.find() ? m.group(1) : null; + } + + /** The prefix every library source id carries. */ + public static final String LIBRARY_FILE_ID_PREFIX = "library:"; + + /** + * The source id a library file loads under, in every build — + * {@code library:iam/model.yaml}. + * + *

Stable rather than path-derived so a library node's ADR-0009 provenance envelope + * reads the same from a checkout and from an installed jar, carries no absolute path, + * and cannot be confused with an adopter file sharing a basename.

+ * + * @param ref the path under {@code library/} minus {@code .yaml} + * @return the stable source id + */ + public static String libraryFileId(String ref) { + return LIBRARY_FILE_ID_PREFIX + ref + ".yaml"; + } + + /** Split a selection token into {@code [library, layer]} — {@code "iam"} to + * {@code ["iam", ""]}, {@code "iam/db"} to {@code ["iam", "db"]}. Only the FIRST + * separator is meaningful, so a typo stays a typo rather than resolving to a prefix. */ + public static String[] splitToken(String token) { + int i = token.indexOf('/'); + return i == -1 ? new String[] { token, "" } + : new String[] { token.substring(0, i), token.substring(i + 1) }; + } + + /** Every selection token this build accepts, sorted — what a config error prints. */ + public static List knownTokens() { + TreeSet out = new TreeSet<>(); + for (Map.Entry>> e : LAYERS_BY_LIBRARY.entrySet()) { + for (String layer : e.getValue().keySet()) { + out.add(layer.isEmpty() ? e.getKey() : e.getKey() + "/" + layer); + } + } + return new ArrayList<>(out); + } + /** * The library package names this build ships, sorted. * @@ -59,7 +169,7 @@ private static Map> buildRefsByPackage() { * @return the shipped package names, sorted */ public static List knownPackages() { - return new ArrayList<>(new TreeSet<>(REFS_BY_PACKAGE.keySet())); + return new ArrayList<>(new TreeSet<>(LAYERS_BY_LIBRARY.keySet())); } /** @@ -117,28 +227,57 @@ public static List librarySources(List packages) { List out = new ArrayList<>(); if (packages == null) return out; - Path dir = getLibraryDir(); - for (String pkg : packages) { - List refs = REFS_BY_PACKAGE.get(pkg); - if (refs == null) continue; // unknown package — no sources + // A token whose LAYER is unknown is dropped whole, not reduced to its core: implying + // the core from an invalid layer would answer a mistyped "iam/database" with an inert + // core and no tables, which is the worst of the available outcomes. + List wanted = new ArrayList<>(); + for (String token : packages) { + String[] parts = splitToken(token); + Map> layers = LAYERS_BY_LIBRARY.get(parts[0]); + if (layers != null && layers.containsKey(parts[1])) wanted.add(parts); + } + // Core layers FIRST, across every requested library, so a db layer named before its + // core still parses after it. "iam/db" IMPLIES "iam": a db layer is nothing but + // overlay:true redeclarations, and an overlay whose target was never declared is + // ERR_OVERLAY_NO_TARGET. + List refs = new ArrayList<>(); + java.util.Set seen = new java.util.LinkedHashSet<>(); + for (String[] parts : wanted) { + for (String ref : LAYERS_BY_LIBRARY.get(parts[0]).get("")) { + if (seen.add(ref)) refs.add(ref); + } + } + for (String[] parts : wanted) { + if (parts[1].isEmpty()) continue; + for (String ref : LAYERS_BY_LIBRARY.get(parts[0]).get(parts[1])) { + if (seen.add(ref)) refs.add(ref); + } + } + + Path dir = getLibraryDir(); + { for (String ref : refs) { if (dir != null) { Path path = dir.resolve(ref + ".yaml"); if (Files.isRegularFile(path)) { - out.add(new FileSource(path)); + // The SAME id the embedded branch below uses. A path-derived id + // would make a library node's error envelope differ between a + // checkout and an installed jar, and would collide with an adopter + // file of the same basename. + out.add(new FileSource(path, libraryFileId(ref))); continue; } } String embedded = EmbeddedLibrary.CONTENT.get(ref); if (embedded == null) { throw new IllegalStateException( - "library ref \"" + ref + "\" (package \"" + pkg + "\") has no on-disk file " + "library ref \"" + ref + "\" has no on-disk file " + "and no embedded entry — the embedded library class is stale; run " + "scripts/generate-embedded-library.ts"); } out.add(new InMemoryStringSource( - embedded, "library:" + ref + ".yaml", MetaDataSource.MetaDataFormat.YAML)); + embedded, libraryFileId(ref), MetaDataSource.MetaDataFormat.YAML)); } } return out; diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/FileSource.java b/server/java/metadata/src/main/java/com/metaobjects/loader/FileSource.java index 9b1d03ea3..f9f566712 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/FileSource.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/FileSource.java @@ -29,6 +29,8 @@ public final class FileSource implements MetaDataSource { private final Path path; private final MetaDataFormat format; + /** Explicit id, or {@code null} to derive it from the file name. */ + private final String id; /** * Constructs a file source with format inferred from the extension. @@ -39,6 +41,21 @@ public FileSource(Path path) { this(path, inferFormat(path)); } + /** + * Constructs a file source with an explicit ID, overriding the file-name default. + * + *

For a file whose identity in diagnostics should not depend on where it sits on + * disk — a shipped library's YAML, whose envelope must read the same from a checkout + * and from an installed jar, and must not collide with an adopter file of the same + * basename.

+ * + * @param path the filesystem path; must not be {@code null} + * @param id the source id to report; must not be {@code null} + */ + public FileSource(Path path, String id) { + this(path, inferFormat(path), id); + } + /** * Constructs a file source with an explicit format (overrides extension inference). * @@ -46,13 +63,25 @@ public FileSource(Path path) { * @param format the document format; must not be {@code null} */ public FileSource(Path path, MetaDataFormat format) { + this(path, format, null); + } + + /** + * Constructs a file source with an explicit format AND id. + * + * @param path the filesystem path; must not be {@code null} + * @param format the document format; must not be {@code null} + * @param id the source id to report, or {@code null} to use the file name + */ + public FileSource(Path path, MetaDataFormat format, String id) { this.path = Objects.requireNonNull(path, "path"); this.format = Objects.requireNonNull(format, "format"); + this.id = id; } @Override public String getId() { - return path.getFileName().toString(); + return id != null ? id : path.getFileName().toString(); } @Override diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java b/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java index d4ecd0aab..65498ad83 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/parser/json/CanonicalJsonParser.java @@ -924,7 +924,17 @@ private void processNode(MetaData parent, String type, String subType, // Detect attr conflicts up-front. The base-parser's parseInlineAttribute // is last-writer-wins (the existing attr child is deleted and replaced), // so we must compare BEFORE that replacement happens. - detectAttrMergeConflicts(md, body, preMergeAttrSnapshot, preMergeSource); + // + // FR-043 Amendment 2 — `overlay: true` LICENSES the override. The conflict + // error exists to catch two files that collided without knowing about each + // other; the flag is the author saying "I know about the other declaration + // and I mean to change it". The loader already treats it specially + // (find-or-throw versus create-or-find), so honouring it here makes it mean + // ONE thing rather than two. Per NODE: a nested overlay marks its own + // ancestors too, and each is judged on its own flag. + if (!Boolean.TRUE.equals(isOverlay)) { + detectAttrMergeConflicts(md, body, preMergeAttrSnapshot, preMergeSource); + } } // FR5a / ADR-0009 — tag the node with its JsonSource provenance. diff --git a/server/python/scripts/generate_embedded_library.py b/server/python/scripts/generate_embedded_library.py index 5b5f52bee..d86f2fbf0 100644 --- a/server/python/scripts/generate_embedded_library.py +++ b/server/python/scripts/generate_embedded_library.py @@ -52,10 +52,43 @@ def main() -> int: if not entries: raise SystemExit(f"no *.yaml found under {library_dir} — refusing to emit an empty library") - body = "".join(f" {ref!r}: {text!r},\n" for ref, text in entries) - out_path.write_text(HEADER + body + "}\n", encoding="utf-8") + # The MANIFESTS ride the same embed, in their own map keyed by library NAME. + # + # Their own map rather than another entry in the ref map, because a ref is + # "path under library/ minus .yaml" and every reader appends ``.yaml`` back. Folding + # a ``.json`` into that would mean a special case at every call site in four ports. + manifests = [] + missing = [] + for d in sorted(p for p in library_dir.iterdir() if p.is_dir()): + manifest = d / "library.json" + if manifest.is_file(): + manifests.append((d.name, manifest.read_text(encoding="utf-8"))) + else: + missing.append(d.name) + + # A library directory with no manifest is a build error, not a silently-skipped one: + # the manifest declares the library's LAYERS, and without it ``libraries: ["iam/db"]`` + # resolves to nothing and the adopter gets an empty tree with no explanation. + if missing: + raise SystemExit(f"library director{'y' if len(missing) == 1 else 'ies'} with no library.json: {', '.join(missing)}") - print(f"wrote {out_path.relative_to(root)} ({len(entries)} ref(s): {', '.join(r for r, _ in entries)})") + body = "".join(f" {ref!r}: {text!r},\n" for ref, text in entries) + manifest_body = "".join(f" {name!r}: {text!r},\n" for name, text in manifests) + out_path.write_text( + HEADER + + body + + "}\n\n" + + "#: Library NAME -> the exact text of its ``library.json`` manifest.\n" + + "EMBEDDED_LIBRARY_MANIFESTS: dict[str, str] = {\n" + + manifest_body + + "}\n", + encoding="utf-8", + ) + + print( + f"wrote {out_path.relative_to(root)} ({len(entries)} ref(s): " + f"{', '.join(r for r, _ in entries)}; {len(manifests)} manifest(s))" + ) return 0 diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index 7155b9b5e..a107fc37a 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -53,7 +53,12 @@ from metaobjects import MetaDataLoader from metaobjects.errors import ParseError -from metaobjects.config.dependencies import Collection, imported_from, refuse_unowned_packages +from metaobjects.config.dependencies import ( + Collection, + imported_from, + refuse_library_package_misuse, + refuse_unowned_packages, +) from metaobjects.loader.meta_data_loader import LoadResult from metaobjects.loader.sources import FileSource from metaobjects.meta.core.object.meta_object import MetaObject @@ -119,22 +124,29 @@ def _pkg_of(node: MetaData) -> str: return "" if i == -1 else key[:i] -def _default_generators() -> list[Generator]: - """The default codegen suite — the no-config generators every project gets. +#: The error a run with no generator selection reports. +#: +#: There is no default suite. This port used to run EIGHT generators for a caller who +#: named none — entity, router, filter-allowlist, names, payload, output-parser, +#: output-prompt, extractor — which is a shape nobody chose. Java has never had a +#: default set and has been right all along; TypeScript and C# dropped theirs in the +#: same change. +#: +#: Deciding WHICH code an application needs belongs to whoever is building it — +#: increasingly an LLM working in the repo, which is well able to make that call given a +#: truthful catalog and is badly served by a default that pre-empts it. ``--list`` is +#: that catalog. +NO_GENERATORS_SELECTED = ( + "gen: no generators selected. Nothing is generated until you choose it — " + "pass --generators . See the catalog: metaobjects gen --list" +) - ``template_generator`` is excluded: it requires a caller-supplied text - provider + Mustache template and is not a zero-config per-entity emitter. - """ - return [ - entity_model(), - router_generator(), - filter_allowlist_generator(), - names_generator(), - payload_vo_generator(), - output_parser_generator(), - output_prompt_generator(), - extractor_generator(), - ] + +class NoGeneratorsSelectedError(ValueError): + """Raised when a codegen run names no generator. Carries the usage message.""" + + def __init__(self) -> None: + super().__init__(NO_GENERATORS_SELECTED) def _resolve_providers(specs: list[str] | None) -> tuple[list[object], list[str]]: @@ -338,6 +350,11 @@ def _load_collection_result( result = loader.load([*lib_sources, *sources]) if not result.errors: refuse_unowned_packages(result.root, collection.imported_packages, collection.imported_nodes) + # FR-043 §3.4 / §3.5 — the same rule for a shipped LIBRARY's package, where the + # two ways to get it wrong are opposite: a node the library also declares (an + # ejected copy, still opted in) and one it does not (a new node in someone + # else's package). + refuse_library_package_misuse(result.root, libraries) return result @@ -507,7 +524,11 @@ def _run_suite( # verify path (those pass gen_state_dir=None and record nothing at all). baseline=baseline, ) - suite = generators if generators is not None else _default_generators() + # No default suite — see NO_GENERATORS_SELECTED. A caller that names none gets a + # usage error and an empty out dir, never a shape this CLI picked. + if not generators: + raise NoGeneratorsSelectedError + suite = generators result = run_gen(config, root, generators=suite, entity_filter=entity_filter, select=select) for warning in result.warnings: print(f"warning: {warning}") @@ -885,6 +906,11 @@ def _cmd_gen(args: argparse.Namespace) -> int: for msg in gen_errors: print(f" {msg}", file=sys.stderr) return 1 + else: + # Codegen is opt-in: refuse here, at the door, rather than letting the run reach + # the generator loop and fail with an empty out dir half-created. + print(f"error: {NO_GENERATORS_SELECTED}", file=sys.stderr) + return 2 # SP-1: declarative Mustache generators from a JSON template-spec. Their output # is format-agnostic (text/markdown/csv/json/xml/html), so they run as a SECOND, @@ -1380,12 +1406,35 @@ def _verify_codegen(args: argparse.Namespace) -> int: print(spec_err, file=sys.stderr) return 1 + # `verify --codegen` re-runs the SELECTION and compares. Codegen is opt-in, so the + # selection has to be named — with none, there is no generated output to check, and + # saying so beats silently regenerating a suite this project never asked for and + # convicting every file of being missing. + selection: list[Generator] | None = None + if getattr(args, "generators", None): + selection, gen_errors = _resolve_generators(args.generators) + if gen_errors: + print("error: invalid --generators selection:", file=sys.stderr) + for msg in gen_errors: + print(f" {msg}", file=sys.stderr) + return 1 + elif not spec_gens: + print( + "verify --codegen: no generators selected, so there is no generated output " + "to check. Pass --generators naming the same suite `gen` ran " + "(metaobjects gen --list is the catalog).", + ) + return 0 + with tempfile.TemporaryDirectory() as tmp: entities = _parse_entities(getattr(args, "entities", None)) - written, errors = _generate( - args.metadata_dir, tmp, None, entities, strict=strict, providers=providers, - column_naming=column_naming, - ) + written: list[str] = [] + errors: list[str] = [] + if selection is not None: + written, errors = _generate( + args.metadata_dir, tmp, selection, entities, strict=strict, + providers=providers, column_naming=column_naming, + ) if errors: print("error: failed to load metadata:", file=sys.stderr) for msg in errors: @@ -1458,6 +1507,22 @@ def _verify_codegen_neutral_fallback(args: argparse.Namespace) -> int: if not providers_ok: return 1 + # Same rule as _verify_codegen: the selection must be named, because there is no + # default suite to regenerate and diff against. + if not getattr(args, "generators", None): + print( + "verify --codegen: no generators selected, so there is no generated output " + "to check. Pass --generators naming the same suite `gen` ran " + "(metaobjects gen --list is the catalog).", + ) + return 0 + selection, gen_errors = _resolve_generators(args.generators) + if gen_errors: + print("error: invalid --generators selection:", file=sys.stderr) + for msg in gen_errors: + print(f" {msg}", file=sys.stderr) + return 1 + with tempfile.TemporaryDirectory() as tmp: entities = _parse_entities(getattr(args, "entities", None)) root, load_errors = _load_root_from_collection(collection, strict=strict, providers=providers) @@ -1475,7 +1540,7 @@ def _verify_codegen_neutral_fallback(args: argparse.Namespace) -> int: return 2 _run_suite( - root, tmp, None, entities, gen_state_dir=None, column_naming=column_naming, + root, tmp, selection, entities, gen_state_dir=None, column_naming=column_naming, select=collection.in_scope, ) expected = _relative_set(Path(tmp)) @@ -1903,9 +1968,9 @@ def _build_parser() -> argparse.ArgumentParser: "--generators", default=None, help=( - "comma-separated STABLE generator names to run (e.g. entity,routes). " - "Resolved via the registry; omit to run the default suite. " - "See `gen --list`." + "REQUIRED — comma-separated STABLE generator names to run " + "(e.g. entity,routes). Codegen is opt-in: there is no default suite, so a " + "run that names none is a usage error. See `gen --list` for the catalog." ), ) gen.add_argument( @@ -2134,6 +2199,15 @@ def _build_parser() -> argparse.ArgumentParser: default=None, help="verify only this named target from the config (default: every target)", ) + verify.add_argument( + "--generators", + default=None, + help=( + "comma-separated STABLE generator names — must MATCH the `gen` that " + "produced --out. `verify --codegen` re-runs the selection and diffs; with " + "none named there is nothing to check and it says so." + ), + ) verify.set_defaults(func=_cmd_verify) agent_docs = sub.add_parser( diff --git a/server/python/src/metaobjects/codegen/generator_registry.py b/server/python/src/metaobjects/codegen/generator_registry.py index ba716b9e9..11d7949cb 100644 --- a/server/python/src/metaobjects/codegen/generator_registry.py +++ b/server/python/src/metaobjects/codegen/generator_registry.py @@ -6,9 +6,11 @@ in every port. This module is the discoverability + identity surface behind ``metaobjects gen --list`` and the ``--generators a,b`` selection path. -It is ADDITIVE. The default suite in ``cli.py`` (``_default_generators``) and the -``run_gen(..., generators=[...])`` factory-array path keep working unchanged — the -registry powers ``--list`` and stable identity; it does not replace those paths. +It is the ONLY name-based door. ADR-0034 Amendment 2 made codegen opt-in and DELETED +the default suite this note used to name (``cli._default_generators``): a run that +selects no generator generates nothing and raises ``NoGeneratorsSelectedError``. The +``run_gen(..., generators=[...])`` factory-array path remains for in-process callers, +where ``generators`` is likewise required. The registry's name set is conformance-tested for SET EQUALITY against the Python slice of the canonical manifest @@ -45,10 +47,27 @@ GeneratorTier = str # "native" | "neutral" +#: The six layers a generator can belong to — the axis an adopter SELECTS BY, gated +#: cross-port against the manifest exactly as ``tier`` is. Six, not ten: an earlier +#: draft split ``capability`` four ways, each with ONE member, and a layer with one +#: member does no grouping work. ``capability`` holds the generators the MODEL has +#: already chosen (you declared a ``template.prompt``), which is why they are found +#: by probing a real model rather than by browsing a taxonomy. +GENERATOR_LAYERS: tuple[str, ...] = ( + "model", + "persistence", + "api", + "client", + "docs", + "capability", +) + +GeneratorLayer = str # one of GENERATOR_LAYERS + @dataclass(frozen=True) class GeneratorEntry: - """A registry entry: stable name + one-line description + tier + factory.""" + """A registry entry: stable name + one-line description + tier + layer + factory.""" #: Stable, cross-port-consistent id. Equals the registry map key. name: str @@ -56,6 +75,8 @@ class GeneratorEntry: description: str #: "native" = recommended ``metaobjects gen`` suite; "neutral" = ``meta docs``-owned. tier: GeneratorTier + #: The selection axis — one of :data:`GENERATOR_LAYERS`. Gated cross-port. + layer: GeneratorLayer #: Constructs the generator with sensible defaults. Calling it must not throw. factory: Callable[[], Generator] @@ -89,72 +110,84 @@ def _render_helper_default() -> Generator: #: Stable name -> GeneratorEntry. The 11 native generators whose manifest `ports` -#: include `python` (ADR-0021 D3). Set-equality conformance-tested vs the manifest. +#: include `python` (ADR-0021 D3). Set equality, tier AND layer are conformance-tested +#: against the manifest. GENERATOR_REGISTRY: dict[str, GeneratorEntry] = { "entity": GeneratorEntry( name="entity", description="Per-entity model/class — the entity module (table-backed or value object).", tier="native", + layer="model", factory=entity_model, ), "routes": GeneratorEntry( name="routes", description="Per-entity REST endpoint surface (controllers / routes / router).", tier="native", + layer="api", factory=router_generator, ), "output-parser": GeneratorEntry( name="output-parser", description="Per-template tolerant output parser (recover-on-receipt).", tier="native", + layer="capability", factory=output_parser_generator, ), "output-prompt": GeneratorEntry( name="output-prompt", description="Per-template output-format prompt fragment generator.", tier="native", + layer="capability", factory=output_prompt_generator, ), "render-helper": GeneratorEntry( name="render-helper", description="Per-template.output render helper (document/email typed wrappers).", tier="native", + layer="capability", factory=_render_helper_default, ), "extractor": GeneratorEntry( name="extractor", description="Per-template strict typed extract helper (strict payload extraction).", tier="native", + layer="capability", factory=extractor_generator, ), "template": GeneratorEntry( name="template", description="Generic Mustache template primitive (walk + template -> files).", tier="native", + layer="capability", factory=_template_primitive, ), "filter-allowlist": GeneratorEntry( name="filter-allowlist", description="Per-entity REST filter allowlist (queryable-field guard).", tier="native", + layer="api", factory=filter_allowlist_generator, ), "names": GeneratorEntry( name="names", description="Per-entity physical database name constants (table/view name, schema, column names).", tier="native", + layer="model", factory=names_generator, ), "payload": GeneratorEntry( name="payload", description="Per-template payload value object (the strict payload type).", tier="native", + layer="capability", factory=payload_vo_generator, ), "trace-helper": GeneratorEntry( name="trace-helper", description="Per-entity typed record LLM-trace helper (extract + buildLlmCallRow + persist).", tier="native", + layer="capability", factory=trace_helper_generator, ), } diff --git a/server/python/src/metaobjects/codegen/generators/trace_helper_generator.py b/server/python/src/metaobjects/codegen/generators/trace_helper_generator.py index 633e61d44..b6dcb727a 100644 --- a/server/python/src/metaobjects/codegen/generators/trace_helper_generator.py +++ b/server/python/src/metaobjects/codegen/generators/trace_helper_generator.py @@ -54,13 +54,31 @@ from metaobjects.meta.template import template_constants as tc from metaobjects.meta.template.meta_template import MetaTemplate from metaobjects.shared.base_types import TYPE_TEMPLATE +from metaobjects.library.library_sources import LIBRARY_MANIFESTS from metaobjects.shared.separators import PACKAGE_SEP _GENERATOR_NAME = "trace-helper" -#: The abstract base entity a trace entity must (transitively) ``extends``. -#: Cross-port constant — mirrors TS ``LLM_CALL_BASE`` / Java ``LLM_CALL_BASE``. -LLM_CALL_BASE = "LlmCallBase" +def _anchor_fqns() -> frozenset[str]: + """FQNs of the library nodes this generator keys on, read from the shipped + ``library.json`` manifests (FR-043 §6) rather than hard-coded here. + + What it replaces: ``LLM_CALL_BASE = "LlmCallBase"``, compared against + ``MetaData.name`` (the SHORT name) anywhere in the super chain — so any adopter + entity of that name, in any package, triggered a helper that writes the shipped + base's columns. An anchor is fully qualified and is resolved to a node below. + """ + out: set[str] = set() + for manifest in LIBRARY_MANIFESTS.values(): + for gen in manifest.get("generators", []): + if gen.get("name") == _GENERATOR_NAME and gen.get("anchor"): + out.add(gen["anchor"]) + return frozenset(out) + + +#: The anchor FQNs, resolved once per process from the embedded manifests. +#: Mirrors TS ``anchorFqns()`` / Java ``LlmTraceHelperGenerator.ANCHOR_FQNS``. +ANCHOR_FQNS = _anchor_fqns() def _pkg_of(node: MetaData) -> str: @@ -88,15 +106,18 @@ def _snake_case(name: str) -> str: def _extends_base(entity: MetaObject) -> bool: - """Walk the resolved super chain looking for a node SHORT-named - :data:`LLM_CALL_BASE`. ``MetaData.name`` holds the short name only (the package - lives on ``MetaData.package``), so a plain ``name`` compare is the short-name - test — mirrors the Java ``getShortName()`` walk and the TS ``superResolved`` - walk.""" + """Walk the resolved super chain looking for one of :data:`ANCHOR_FQNS`. + + The FULLY-QUALIFIED name, via ``resolution_key()``: within one loaded root an FQN + identifies exactly one node (a same-name redeclaration merges), so this is the + node-identity compare the TypeScript port makes against the resolved anchor node, + and the Java port makes against ``getName()``.""" + if not ANCHOR_FQNS: + return False cur = entity.super_data visited: set[int] = set() while cur is not None and id(cur) not in visited: - if cur.name == LLM_CALL_BASE: + if cur.resolution_key() in ANCHOR_FQNS: return True visited.add(id(cur)) cur = cur.super_data @@ -125,7 +146,7 @@ def render_trace_helper(entity: MetaObject, root: MetaData) -> str | None: """Render one ``record_.py`` for a concrete trace ``object.entity``. Returns ``None`` when the entity is not a trace-helper target (abstract, not - ``LlmCallBase``-derived, no nested ``template.prompt``, or that prompt carries + derived from a library anchor, no nested ``template.prompt``, or that prompt carries neither ``@responseRef`` nor ``@payloadRef``). Raises ``ValueError`` when the prompt's ``@responseRef`` does not resolve to an diff --git a/server/python/src/metaobjects/config/dependencies.py b/server/python/src/metaobjects/config/dependencies.py index d9fa100ed..ad5237822 100644 --- a/server/python/src/metaobjects/config/dependencies.py +++ b/server/python/src/metaobjects/config/dependencies.py @@ -515,6 +515,88 @@ def refuse_unowned_packages( ) +def refuse_library_package_misuse( + root: MetaData, + selection: Sequence[str] | None, +) -> None: + """FR-043 §3.4 / §3.5 — refuse the two ways an adopter's own file lands in a + shipped library's package while that library is opted in. + + Both are SILENT otherwise, and they fail in opposite directions: + + the EJECTED COPY + ``meta eject `` hands you the library's YAML to own, and the next step it + prints is to remove the library from ``libraries``. Skip that and both trees + load: the copy merges into the shipped node, so ADDITIONS take and DELETIONS do + not, because the library still declares what you removed + (``ERR_LIBRARY_PACKAGE_COLLISION``). + + the NEW NODE + something of your own declared into ``metaobjects::``, where a later + release of the library may ship a node of that name and merge into it + (``ERR_LIBRARY_PACKAGE_NOT_OWNED``). + + An ``overlay: true`` redeclaration is the documented adaptation door (§3.4) and is + deliberately untouched — ``is_merge`` is the loader's own record that the flag was + honoured, so this cannot mistake the two. Mirrors the TS + ``refuseLibraryPackageMisuse`` (``sdk/src/memory.ts``); called AFTER the loader's + own errors for the same reason its sibling above is. + + No-op for a project that opts into no library. + """ + if not selection: + return + from metaobjects.library.library_sources import ( + LIBRARY_MANIFESTS, + is_library_file_id, + split_layer_token, + ) + + owner: dict[str, str] = {} + for token in selection: + library = split_layer_token(token)[0] + for pkg in LIBRARY_MANIFESTS.get(library, {}).get("packages", []): + owner[pkg] = library + if not owner: + return + + # ADR-0039 SANCTIONED own-accessor case: a root-level scan, exactly as the + # dependency refusal above — `MetaRoot` has no super, and the question is "what did + # this tree declare at the top level". + for node in root.own_children(): + key = node.resolution_key() + library = owner.get(package_of_resolution_key(key)) + if library is None: + continue + files = getattr(node.source, "files", None) or [] + if not any(not is_library_file_id(f) for f in files): + continue # the library's own node, untouched + if getattr(node, "is_merge", False): + continue # a marked overlay — the documented door + if any(is_library_file_id(f) for f in files): + raise ParseError( + f'"{key}" is declared by your own metadata AND by the shipped library ' + f'"{library}", which this project opts into. The two merge silently and ' + "asymmetrically: additions in your copy take effect and DELETIONS do not, " + "because the library still declares what you removed. Remove " + f'"{library}" from \'libraries\' in .metaobjects/config.json — you own the ' + "metadata now — or delete your copy and amend the library with " + "'overlay: true' on the nodes you want to change.", + code=ErrorCode.ERR_LIBRARY_PACKAGE_COLLISION, + ) + raise ParseError( + f'"{key}" is declared here, but the package ' + f'"{package_of_resolution_key(key)}" belongs to the shipped library ' + f'"{library}", which this project opts into. A later release of that library ' + "may ship a node of this name and merge into yours. Declare it in a package " + "this project owns and 'extends' the library's node if it needs its shape; if " + "it was meant to AMEND a library node, give it that node's name and " + f"'overlay: true'; if you want this design outright, run 'meta eject {library}' " + f'and remove "{library}" from \'libraries\'.', + code=ErrorCode.ERR_LIBRARY_PACKAGE_NOT_OWNED, + ) + + @dataclass(frozen=True) class Collection: """Everything the FR-023-aware source ladder resolved for one project: diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index 6c64b4952..5902dad73 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -127,6 +127,15 @@ class ErrorCode(str, Enum): ERR_COLLECTION_NOT_FOUND = "ERR_COLLECTION_NOT_FOUND" # FR-023 — a declared dependency's transport could not locate a directory holding # metaobjects.pkg.json. + # FR-043: .metaobjects/config.json's `libraries` names a shipped library or layer + # this build does not have. Refused with the available tokens rather than skipped — + # skipped, it resurfaces as ERR_UNRESOLVED_SUPER against the adopter's own metadata. + ERR_UNKNOWN_LIBRARY = "ERR_UNKNOWN_LIBRARY" + # FR-043: a node is declared by BOTH an adopter's own metadata and a shipped library the project opts into — the `meta eject ` copy with the library still in `libraries`. The two merge silently and ASYMMETRICALLY: additions take, deletions do not, because the library still declares what was removed. + # Raised by the TypeScript SDK's load path; registered in every port so the shared corpus list stays one set. + ERR_LIBRARY_PACKAGE_COLLISION = "ERR_LIBRARY_PACKAGE_COLLISION" + # FR-043: a NEW top-level node is declared into a package a shipped library owns while that library is opted in — a later release of the library may ship a node of that name and merge into it. `overlay: true` on one of the library's OWN nodes is the documented amendment door and is untouched. + ERR_LIBRARY_PACKAGE_NOT_OWNED = "ERR_LIBRARY_PACKAGE_NOT_OWNED" ERR_DEPENDENCY_UNRESOLVED = "ERR_DEPENDENCY_UNRESOLVED" # FR-023 — a dependency's metaobjects.pkg.json fails its schema, names a different # dependency, points at a missing/hash-mismatched artifact, or the artifact does not diff --git a/server/python/src/metaobjects/library/embedded_library.py b/server/python/src/metaobjects/library/embedded_library.py index be6ef5c46..3c2757040 100644 --- a/server/python/src/metaobjects/library/embedded_library.py +++ b/server/python/src/metaobjects/library/embedded_library.py @@ -7,5 +7,16 @@ # Keys are refs: path under library/ minus the .yaml extension. EMBEDDED_LIBRARY: dict[str, str] = { - 'ai/llm-call': '# library/ai/llm-call.yaml\n# MetaObjects-shipped standard metadata. Adopters opt in via the loader\'s\n# `libraries: ["ai"]` option, then `extends: "metaobjects::ai::LlmCallBase"`.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n children:\n - source.rdb: { table: llm_call, role: primary }\n - identity.primary: { name: id, fields: ["spanId"] }\n', + 'ai/db': '# library/ai/db.yaml — the DB PERSISTENCE layer for metaobjects::ai.\n#\n# Opted into as `"ai/db"`, which IMPLIES `"ai"`: `LlmCall` is declared in model.yaml and\n# this file only re-opens it, so without the core layer the overlay has no target.\n#\n# `LlmCall` is the concrete, table-backed instance of the abstract envelope. An adopter\n# who wants their OWN table (a different name, extra columns, a different id strategy)\n# extends `LlmCallBase` in their own metadata and never opts into this layer at all.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCall\n overlay: true\n children:\n - source.rdb: { table: llm_call, role: primary }\n', + 'ai/model': '# library/ai/model.yaml — the CORE layer: the LLM-call trace envelope.\n#\n# Adopters opt in via `libraries: ["ai"]`, then `extends: "metaobjects::ai::LlmCallBase"`.\n#\n# This layer declares NO `source.rdb`, so opting into `"ai"` alone adds zero tables and\n# zero generated code — the design is present and resolvable, and nothing else happens\n# until the adopter adds `"ai/db"`. See library/iam/model.yaml for the full rationale.\n#\n# This file was split out of the former `library/ai/llm-call.yaml`, which shipped the\n# abstract base and a concrete `LlmCall` carrying `source.rdb` together. That was\n# recorded as an accepted wart on the grounds that splitting would change what existing\n# `ai` adopters get; a sweep of the estate found there are none, so it was closed rather\n# than documented (FR-043 Amendment 1).\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n description: The concrete trace row. Its `source.rdb` lives in db.yaml, so opting into the core layer alone declares the shape without proposing a table.\n children:\n - identity.primary: { name: id, fields: ["spanId"] }\n', + 'ai/requirements': "# library/ai/requirements.yaml — what the LLM-call trace envelope PROMISES.\n#\n# A RETROFIT, not new design: llm-call.yaml landed 2026-06-03 and `requirement.functional`\n# first appears 2026-08-11, so the library could not have carried requirements when it was\n# written. That is why this file is worth reading as a worked example — it shows what\n# declaring the design of something that already exists actually turns up.\n#\n# The entry that earns its keep is `typedIo`, honestly `partial` + `accepted`: the library\n# declares the ENVELOPE and the adopter declares the typed VO columns. Recording that seam\n# in the ledger is where an agent meets it, before adding a fourth trace column.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. An L4 names the OBJECT it is about;\n# the fields that carry it hang off it as an L5 child. Writing the fields at L4 is\n# ERR_REQUIREMENT_L4_NOT_OBJECT, and writing the concerns as SIBLINGS of the L2 leaves the\n# L2 claiming nothing — both of which this file did until the standalone verify gate\n# existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::ai\n children:\n - requirement.functional:\n name: llmTracing\n level: 2\n status: live\n statement: Every call to a language model leaves a row that says what was asked, what came back, what it cost and how long it took.\n counterexample: A spend figure nobody can attribute to a call.\n description: The segment this library covers. Its three children below are the concerns it decomposes into.\n children:\n - requirement.functional:\n name: envelope\n level: 4\n status: live\n statement: A trace row identifies its call and its place in a trace — trace, span, parent span, session, call type, system.\n counterexample: A log line that cannot be joined to the request that produced it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: traceAddressing\n level: 5\n status: live\n statement: The four addressing columns are declared on the base — trace, span, parent span and session.\n counterexample: A row whose place in a trace is inferred from insertion order.\n description: >-\n The member grain exists here so the claim RESOLVES against the fields\n themselves: renaming or dropping one of them dangles this reference and\n fails the build, which naming the object alone would not.\n implementedBy: [LlmCallBase.traceId, LlmCallBase.spanId, LlmCallBase.parentSpanId, LlmCallBase.sessionId]\n\n - requirement.functional:\n name: accounting\n level: 4\n status: live\n statement: A trace row carries the tokens in, the tokens out, and the cost in integer minor units.\n counterexample: A cost stored as a float.\n description: >-\n `field.currency` — integer minor units on the wire, always. Float arithmetic for\n money is forbidden by the cross-port wire contract, and a spend total is exactly\n the sum that exposes it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: tokenAndCostColumns\n level: 5\n status: live\n statement: Tokens in, tokens out and cost are three declared columns, the cost a field.currency.\n counterexample: A cost column declared as a double.\n implementedBy: [LlmCallBase.inputTokens, LlmCallBase.outputTokens, LlmCallBase.costMinor]\n\n - requirement.functional:\n name: typedIo\n level: 4\n status: partial\n disposition: accepted\n statement: The request and response bodies are stored as structured jsonb, not as opaque text.\n counterexample: A prompt stored as a string nobody can query a field out of.\n notes: >-\n The library declares the two columns as generic jsonb with no `@objectRef`,\n because it cannot know the adopter's request/response shape. Typing them is the\n ADOPTER's move: declare an `object.value` and overlay the field with\n `@objectRef` + `@storage: jsonb`. This is the seam ADR-0024 drew, recorded here\n rather than in prose so it is in the ledger an agent reads before adding a\n fourth trace column of its own.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: jsonbBodies\n level: 5\n status: live\n statement: The request and response bodies are declared as jsonb columns on the base.\n counterexample: A prompt stored in a text column.\n description: >-\n `live` where its parent is `partial`, and the split is the point: the\n COLUMNS are shipped and this claim is fully realised; what is outstanding\n is the TYPING of them, which is the parent's gap and the adopter's move.\n implementedBy: [LlmCallBase.llmRequest, LlmCallBase.llmResponse]\n\n - requirement.architectural:\n name: traceRowsCarryTiming\n status: live\n statement: Every trace row records when the call started and how long it took.\n counterexample: A latency figure derived from log timestamps after the fact.\n description: >-\n Architectural, so it propagates down `extends` to every adopter entity deriving\n from LlmCallBase — which is the point: an adopter's own trace table is claimed\n by this requirement for free, and dropping the columns breaks the build.\n implementedBy: [LlmCallBase]\n\n - requirement.architectural:\n name: traceRowsCarryOutcome\n status: live\n statement: Every trace row records how the call ended — a status, a finish reason, and the error detail when there was one.\n counterexample: A failed call indistinguishable from one that never happened.\n implementedBy: [LlmCallBase]\n", + 'iam/db': '# library/iam/db.yaml — the DB PERSISTENCE layer for metaobjects::iam.\n#\n# Opted into as `"iam/db"`, which IMPLIES `"iam"`: this file is nothing but\n# `overlay: true` redeclarations, and an overlay whose target was never declared is\n# ERR_OVERLAY_NO_TARGET.\n#\n# It carries exactly two kinds of child — `source.rdb` and `index.lookup` — and nothing\n# else. The field set, the identities and the relationships all live in model.yaml,\n# because they are the DESIGN; what lives here is where the rows go and which lookups are\n# worth an index. Add a field here and the core layer stops being the whole model, which\n# is the thing the split exists to guarantee.\n#\n# Physical names are `iam_`-prefixed. Two reasons, both real: `user` and `group` are\n# reserved words in Postgres, and an adopter very likely has tables of their own by those\n# names. A library that collides on a table name is a library nobody can adopt.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: User\n overlay: true\n children:\n - source.rdb: { table: iam_user, role: primary }\n\n - object.entity:\n name: GroupType\n overlay: true\n children:\n - source.rdb: { table: iam_group_type, role: primary }\n\n - object.entity:\n name: Group\n overlay: true\n children:\n - source.rdb: { table: iam_group, role: primary }\n # Nesting is walked parent-ward constantly; the FK alone gives no index.\n - index.lookup: { name: ixParent, fields: [parentId] }\n\n - object.entity:\n name: Role\n overlay: true\n children:\n - source.rdb: { table: iam_role, role: primary }\n\n - object.entity:\n name: Permission\n overlay: true\n children:\n - source.rdb: { table: iam_permission, role: primary }\n\n - object.entity:\n name: GroupMember\n overlay: true\n children:\n - source.rdb: { table: iam_group_member, role: primary }\n # The composite PK covers (userId, groupId), so "who is in this group?" —\n # the other direction — has no index without this one. Same reasoning for\n # every ixSecond below.\n - index.lookup: { name: ixGroup, fields: [groupId] }\n\n - object.entity:\n name: RolePermission\n overlay: true\n children:\n - source.rdb: { table: iam_role_permission, role: primary }\n - index.lookup: { name: ixPermission, fields: [permissionId] }\n\n - object.entity:\n name: UserRole\n overlay: true\n children:\n - source.rdb: { table: iam_user_role, role: primary }\n - index.lookup: { name: ixRole, fields: [roleId] }\n\n - object.entity:\n name: GroupMemberRole\n overlay: true\n children:\n - source.rdb: { table: iam_group_member_role, role: primary }\n # "who holds this role in this group?" — the scoped-grant read.\n - index.lookup: { name: ixGroupRole, fields: [groupId, roleId] }\n', + 'iam/model': '# library/iam/model.yaml — the CORE layer: identity and access management.\n#\n# Adopters opt in via `libraries: ["iam"]` in .metaobjects/config.json.\n#\n# This layer declares NO `source.rdb`, and that is the whole point of the split. A\n# sourceless object is inert by a contract that already ships: migrate skips an object\n# with no writable source, and codegen emits no route, queries, hooks, grid or form for\n# one (both citing #248 — persistability derives from source presence, never from the\n# object subtype). It still gets a type-only interface, so `extends` and reference work.\n#\n# So `libraries: ["iam"]` adds ZERO tables and ZERO generated code. What an adopter gains\n# is the design being present and resolvable: an agent working in the repo knows the\n# capability exists and can draw on it, and nothing else happens until the adopter adds\n# `"iam/db"`.\n#\n# Authoring discipline (FR-043 §3.1), so the departures are visible:\n# - `field.uuid` + `generation: uuid` on principals; composite ASSIGNED keys on\n# junctions. Never `increment` — a library cannot know the adopter\'s id strategy.\n# - Physical names carry the `iam_` prefix (in db.yaml): `user` and `group` are\n# reserved words in Postgres, and an adopter has tables of their own.\n# - No adopter-facing profile data. That arrives by `overlay: true`.\n# - No credentials. See requirements.yaml → `noCredentialsOnUser`.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: IamBase\n abstract: true\n description: Shared shape of every iam principal and definition — a stable uuid plus change timestamps. Junctions do not extend it; they are addressed by their participants.\n children:\n - field.uuid: { name: id, required: true }\n - field.timestamp: { name: createdAt, autoSet: onCreate }\n - field.timestamp: { name: updatedAt, autoSet: onUpdate }\n\n - object.entity:\n name: User\n extends: IamBase\n description: A person or service account that can be granted access. Carries no authentication secret of any kind — see the noCredentialsOnUser requirement.\n children:\n - field.string: { name: username, required: true, maxLength: 64, filterable: true }\n - field.string: { name: email, required: true, maxLength: 254, stringFormat: email, filterable: true }\n - field.string: { name: displayName, maxLength: 120 }\n # NOT `filterable: true`, deliberately. The loader warns when a filterable\n # field is in no identity — filtering on it sequential-scans — and a library\n # must not ship a warning to every adopter. `username` and `email` carry it\n # because they have identity.secondary; `status` does not. An adopter who\n # wants to filter on status overlays `filterable` AND an index together,\n # which is exactly what the layer split is for.\n - field.enum: { name: status, required: true, values: [invited, active, suspended, closed], default: active }\n - field.timestamp: { name: emailVerifiedAt }\n - field.timestamp: { name: lastSeenAt }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqUsername, fields: [username] }\n - identity.secondary: { name: uqEmail, fields: [email] }\n - relationship.association: { name: groups, objectRef: Group, cardinality: many, through: GroupMember }\n - relationship.association: { name: roles, objectRef: Role, cardinality: many, through: UserRole }\n\n - object.entity:\n name: GroupType\n extends: IamBase\n description: What KIND of group this is — a team, a tenant, a project. An entity rather than an enum, because "which roles may be held in this kind of group" is data an adopter extends, and an enum\'s values cannot be extended by overlay.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n - object.entity:\n name: Group\n extends: IamBase\n description: A nestable collection of users, of a declared GroupType. Nesting is by parentId; acyclicity is an invariant the schema cannot express — see the acyclicGroupNesting requirement.\n children:\n - field.uuid: { name: groupTypeId, required: true }\n - field.uuid: { name: parentId }\n - field.string: { name: key, required: true, maxLength: 64 }\n # Not filterable for the same reason as User.status above.\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - identity.reference: { name: fkParent, fields: [parentId], references: Group, onDelete: restrict }\n\n - object.entity:\n name: Role\n extends: IamBase\n description: A reusable bundle of permissions. Code never compares a role NAME to a literal — it asks whether a user holds a permission, and the mapping is data.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - field.uuid: { name: groupTypeId, description: "When set, this role may be held only within groups of this type; absent means grantable anywhere." }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - relationship.association: { name: permissions, objectRef: Permission, cardinality: many, through: RolePermission }\n\n - object.entity:\n name: Permission\n extends: IamBase\n description: "The assignable unit — a stable : key the application checks against. An entity, not an enum, on ADR-0037\'s own reasoning: it has its own identity, its own lifecycle, and a junction with real foreign keys."\n children:\n - field.string: { name: key, required: true, maxLength: 128, description: "Stable : key the application checks against." }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n # ---- grant surface: every grant is a row, addressed by its participants ----\n #\n # Junctions do NOT extend IamBase: they have no identity of their own, and adding a\n # surrogate uuid to a row whose identity IS its participants invites a duplicate.\n\n - object.entity:\n name: GroupMember\n description: A user\'s membership of a group.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.timestamp: { name: joinedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n\n - object.entity:\n name: RolePermission\n description: A permission granted by a role.\n children:\n - field.uuid: { name: roleId, required: true }\n - field.uuid: { name: permissionId, required: true }\n - identity.primary: { name: pk, fields: [roleId, permissionId], generation: assigned }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: cascade }\n - identity.reference: { name: fkPermission, fields: [permissionId], references: Permission, onDelete: restrict }\n\n - object.entity:\n name: UserRole\n description: A system-wide grant of a role to a user.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n\n - object.entity:\n name: GroupMemberRole\n description: A grant of a role to a user WITHIN one group. Three foreign keys, so it is not an M:N @through junction (which must declare exactly two identity.reference children); it is read by explicit finders.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n', + 'iam/requirements': '# library/iam/requirements.yaml — what this library\'s design PROMISES.\n#\n# This is what makes iam a library rather than a schema snippet. Without requirements an\n# adopter gets nine tables; with them they get nine tables plus a build that is held to\n# "no authorization decision is hard-wired to a name", which no snippet can do.\n#\n# Two reading rules, both load-bearing:\n#\n# `live` here means "the model AS SHIPPED realises this" — never "your application\n# does". A ledger binds to model nodes; runtime guarantees are the runtime\'s tests, and\n# this library does not invent a way to point a requirement at code (@verifiedBy was\n# retired for exactly that). Behaviour the model cannot carry ships as `partial` +\n# `disposition: accepted` with a notes sentence naming what the adopter must do.\n#\n# The functional tree roots at L2, not L1. L1 is the adopter\'s SOLUTION, and a library\n# is by definition a segment of someone else\'s. Architectural claims ship flat.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. The concerns are CHILDREN of the L2\n# rather than its siblings, and an L4 names the OBJECT it is about while the field that\n# carries it hangs off it as an L5 child. Written flat, the L2 claims nothing in its whole\n# subtree; written at L4, a field reference is ERR_REQUIREMENT_L4_NOT_OBJECT. Both shipped\n# here until the standalone verify gate existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::iam\n children:\n # ---- functional: the L2 segment and the concerns nested under it --------\n - requirement.functional:\n name: accessControl\n level: 2\n status: live\n statement: Who may do what is answered from stored grants, never from a name compared to a literal in code.\n counterexample: A branch that reads `if (user.role === "admin")`.\n description: The segment this library covers. The concerns beneath it are what it decomposes into.\n children:\n - requirement.functional:\n name: identity\n level: 4\n status: live\n statement: A person or service account is represented once, addressed by a uuid, and reachable by username or email.\n counterexample: Two rows for the same person because the email changed.\n implementedBy: [User]\n\n - requirement.functional:\n name: grouping\n level: 4\n status: live\n statement: Users are collected into typed, nestable groups, and the kind of group is data rather than a hard-coded set.\n counterexample: A `teamOrTenant` boolean.\n implementedBy: [Group, GroupType, GroupMember]\n\n - requirement.functional:\n name: acyclicGroupNesting\n level: 4\n status: partial\n disposition: accepted\n statement: A group is never its own ancestor.\n counterexample: Two groups each naming the other as parent.\n notes: >-\n The schema cannot express this — a self-referencing FK admits a cycle, and the\n only relational forms that would catch it (a recursive CHECK, a closure table\n maintained by trigger) are DB-specific and would not survive three dialects.\n The adopter enforces it where the write happens. Recorded rather than omitted\n so an agent reading the ledger before adding a parent-setting endpoint sees the\n obligation.\n implementedBy: [Group]\n\n - requirement.functional:\n name: grants\n level: 4\n status: live\n statement: A role is granted to a user either system-wide or scoped to one group, and both are ordinary rows.\n counterexample: A nullable `groupId` on one grant table, where NULL means "everywhere".\n description: >-\n Two junctions, not one with a nullable scope. A NULL in a unique key is DISTINCT\n from every other NULL in SQL, so a nullable-scope design lets the same global\n grant be inserted twice; the fix needs a partial index whose expression carries\n a physical column name. Two composite-keyed tables need no escape hatch and\n survive three dialects and five ports unchanged.\n implementedBy: [UserRole, GroupMemberRole]\n\n - requirement.functional:\n name: roleScopedToGroupType\n level: 4\n status: partial\n disposition: accepted\n statement: A role bound to a group type is granted only within groups of that type.\n counterexample: A "tenant admin" role granted inside a project group.\n notes: >-\n Expressing this relationally needs the grant row to carry the group\'s type and\n a composite FK back to (group, type) — three foreign keys deep, unverified\n across five ports\' DDL and ORM paths. The adopter checks it at the point of\n grant. The declared half is the L5 child below; the enforcement is not.\n implementedBy: [Role, GroupMemberRole]\n children:\n - requirement.functional:\n name: roleDeclaresItsGroupType\n level: 5\n status: live\n statement: A role declares the group type it is bound to, as a nullable reference.\n counterexample: A role whose intended scope is recoverable only from its name.\n description: >-\n `live` where its parent is `partial`, and the split is grain as much as\n verdict: the DECLARATION is shipped and resolves against the field itself,\n so dropping the column fails the build — while the ENFORCEMENT, which no\n schema here can carry, stays the parent\'s accepted gap.\n implementedBy: [Role.groupTypeId]\n\n - requirement.functional:\n name: decision\n level: 4\n status: live\n statement: An authorization decision is the question "does this user hold this permission key", answered from rows.\n counterexample: A hard-coded list of usernames that bypass a check.\n implementedBy: [Permission, RolePermission]\n\n # ---- architectural: prohibitions in force --------------------------------\n\n - requirement.architectural:\n name: grantsAreRows\n status: live\n statement: A grant exists only as a stored row; nothing is granted by naming, position or convention.\n counterexample: A superuser recognised by username.\n implementedBy: [UserRole, GroupMemberRole, RolePermission, GroupMember]\n\n - requirement.architectural:\n name: noCredentialsOnUser\n status: live\n statement: A user row carries no authentication secret — no password, no hash, no knowledge-based question or answer.\n counterexample: A password or secret-answer column on the user table.\n description: >-\n Authentication is a separate capability with an entity per factor; this library\n is identity and authorization only.\n notes: >-\n This is the one thing every reader of a user table proposes adding, and a real\n legacy model of this shape stored a length-bounded plaintext password and a\n knowledge-based secret pair on the user row. Stating it as a prohibition IN\n FORCE — claimable, and rendered on agent/requirements.md — is what stops an\n agent extending "the user model" from re-deriving it on sight. It is\n `architectural`, not `retired`: retired is chartered for a capability built\n here and removed, and this library never built one.\n implementedBy: [User]\n\n - requirement.architectural:\n name: principalDeletionRevokesGrants\n status: live\n statement: Deleting a user or group removes its grants; deleting a role or permission still in use is refused.\n counterexample: A grant row pointing at a user who no longer exists.\n description: The referential rule in one sentence — cascade from a principal, restrict from a definition.\n implementedBy: [GroupMember, UserRole, GroupMemberRole, RolePermission]\n\n - requirement.architectural:\n name: stableIdentifiers\n status: live\n statement: Every principal and definition is addressed by a uuid that never changes; every grant by its participants.\n counterexample: A group referenced by its display name.\n implementedBy: [IamBase]\n', +} + +#: Library NAME -> the exact text of its ``library.json`` manifest. +EMBEDDED_LIBRARY_MANIFESTS: dict[str, str] = { + 'ai': '{\n "$comment": "Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library\'s own nodes, and `name` against the last package segment.",\n "name": "ai",\n "kind": "feature",\n "stability": "stable",\n "since": "0.20.0",\n "description": "The LLM-call trace envelope: what was asked, what came back, what it cost, how long it took.",\n "useWhen": "the application calls a language model and someone will ask what it cost or why a call failed",\n "packages": ["metaobjects::ai"],\n "layers": {\n "": { "refs": ["ai/model", "ai/requirements"], "description": "the core model and its requirements — sourceless, so it adds no tables" },\n "db": { "refs": ["ai/db"], "description": "the concrete llm_call table" }\n },\n "generators": [\n { "name": "trace-helper", "anchor": "metaobjects::ai::LlmCallBase" }\n ],\n "runtime": {\n "typescript": ["@metaobjectsdev/runtime-ts"]\n }\n}\n', + 'iam': '{\n "$comment": "Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library\'s own nodes, and `name` against the last package segment.",\n "name": "iam",\n "kind": "feature",\n "stability": "preview",\n "since": "1.1.0",\n "description": "Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group.",\n "useWhen": "the application has people who log in and things some of them may not do",\n "packages": ["metaobjects::iam"],\n "layers": {\n "": { "refs": ["iam/model", "iam/requirements"], "description": "the core model and its requirements — sourceless, so it adds no tables" },\n "db": { "refs": ["iam/db"], "description": "nine tables, iam_-prefixed, plus the lookup indexes the composite keys do not cover" }\n },\n "generators": [],\n "runtime": {}\n}\n', } diff --git a/server/python/src/metaobjects/library/library_sources.py b/server/python/src/metaobjects/library/library_sources.py index 0f6e98f58..54be13dea 100644 --- a/server/python/src/metaobjects/library/library_sources.py +++ b/server/python/src/metaobjects/library/library_sources.py @@ -12,6 +12,7 @@ from __future__ import annotations +import json from functools import lru_cache from pathlib import Path @@ -22,26 +23,82 @@ MetaDataSource, ) -from .embedded_library import EMBEDDED_LIBRARY +from .embedded_library import EMBEDDED_LIBRARY, EMBEDDED_LIBRARY_MANIFESTS -# Package -> ordered refs, derived from the generated module so that adding a library -# file (which regenerates EMBEDDED_LIBRARY) needs no edit here. -_REFS_BY_PACKAGE: dict[str, list[str]] = {} -for _ref in sorted(EMBEDDED_LIBRARY): - _pkg = _ref.split("/")[0] - if _pkg: - _REFS_BY_PACKAGE.setdefault(_pkg, []).append(_ref) +#: Library name -> its manifest's LAYERS: layer token -> that layer's ordered refs. The +#: CORE layer's token is the empty string. +#: +#: Read from the embedded ``library.json`` manifests, not derived from the ref names. +#: This used to be package-granular — every ref under a library came back for a bare +#: ``"ai"`` — which under the layered design (FR-043 Amendment 1) would hand an adopter +#: the db layer they did not ask for, and with it a migration proposing tables. +_LAYERS_BY_LIBRARY: dict[str, dict[str, list[str]]] = { + _name: { + _layer: list(_spec.get("refs", [])) + for _layer, _spec in json.loads(_text).get("layers", {}).items() + } + for _name, _text in sorted(EMBEDDED_LIBRARY_MANIFESTS.items()) +} + +#: Every shipped library's parsed manifest, keyed by name. +LIBRARY_MANIFESTS: dict[str, dict] = { + _name: json.loads(_text) for _name, _text in sorted(EMBEDDED_LIBRARY_MANIFESTS.items()) +} + + +#: The prefix every library source id carries — the discriminator for "did a shipped +#: library contribute this file", and the reason the id is stable rather than derived +#: from a path (see :func:`library_file_id`). +LIBRARY_FILE_ID_PREFIX = "library:" + + +def library_file_id(ref: str) -> str: + """The source id a library file loads under, in every build — + ``library:iam/model.yaml``. + + Stable rather than path-derived so a library node's ADR-0009 provenance envelope + reads the same from a checkout and from an installed wheel, carries no absolute + path, and cannot be confused with an adopter file sharing a basename.""" + return f"{LIBRARY_FILE_ID_PREFIX}{ref}.yaml" + + +def is_library_file_id(source_id: str) -> bool: + """True when a source id names a file a shipped library contributed.""" + return source_id.startswith(LIBRARY_FILE_ID_PREFIX) + + +def split_layer_token(token: str) -> tuple[str, str]: + """Split a selection token into ``(library, layer)``. + + ``"iam"`` -> ``("iam", "")``; ``"iam/db"`` -> ``("iam", "db")``. Only the FIRST + separator is meaningful, so a typo stays a typo rather than resolving to a prefix. + """ + library, sep, layer = token.partition("/") + return (library, layer if sep else "") def known_packages() -> list[str]: - """The shipped library package names, sorted. + """The shipped library names, sorted. :func:`library_sources` deliberately skips an unrecognised name (see there), so a typo would otherwise surface only as ``ERR_UNRESOLVED_SUPER`` against the adopter's own metadata — the wrong place to look. Callers that took the name from a human (the CLI reading a config file) validate against this first. """ - return sorted(_REFS_BY_PACKAGE) + return sorted(_LAYERS_BY_LIBRARY) + + +def known_tokens() -> list[str]: + """Every selection token this build accepts, sorted — what a config error prints. + + TOKENS, not library names, so an adopter who typed ``iam/database`` is shown + ``iam/db`` rather than only the half they got right. + """ + return sorted( + name if layer == "" else f"{name}/{layer}" + for name, layers in _LAYERS_BY_LIBRARY.items() + for layer in layers + ) @lru_cache(maxsize=1) @@ -60,11 +117,18 @@ def _library_dir_on_disk() -> Path | None: def library_sources(packages: list[str]) -> list[MetaDataSource]: """Sources for the requested shipped-library packages. + Layer-granular: a token is ```` or ``/``, and the CORE + layer is the bare name. ``"iam/db"`` IMPLIES ``"iam"`` — a db layer is nothing but + ``overlay: true`` redeclarations, and an overlay whose target was never declared is + ``ERR_OVERLAY_NO_TARGET``, so implying it is the only coherent reading. + Args: - packages: package names to include, e.g. ``["ai"]``. An unrecognised name + packages: selection tokens, e.g. ``["iam", "iam/db"]``. An unrecognised token contributes no sources rather than raising — a consumer asking for a - package this version does not ship should not fail to load its own - metadata. + library this version does not ship should not fail to load its own + metadata. A token whose LAYER is unknown is dropped WHOLE rather than + reduced to its core: implying the core from an invalid layer would answer a + mistyped ``iam/database`` with an inert core and no tables. Raises: ValueError: a known ref has neither an on-disk file nor an embedded entry, @@ -73,27 +137,55 @@ def library_sources(packages: list[str]) -> list[MetaDataSource]: directory = _library_dir_on_disk() out: list[MetaDataSource] = [] - for package in packages: - for ref in _REFS_BY_PACKAGE.get(package, []): - if directory is not None: - path = directory / f"{ref}.yaml" - if path.is_file(): - out.append(FileSource(path, format=MetaDataFormat.YAML)) - continue - - embedded = EMBEDDED_LIBRARY.get(ref) - if embedded is None: - raise ValueError( - f'library ref "{ref}" (package "{package}") has no on-disk file and no ' - "embedded entry — the embedded library module is stale; run " - "scripts/generate_embedded_library.py" - ) - out.append( - InMemoryStringSource( - embedded, - id=f"library:{ref}.yaml", - format=MetaDataFormat.YAML, + wanted = [ + (lib, layer) + for lib, layer in (split_layer_token(t) for t in packages) + if layer in _LAYERS_BY_LIBRARY.get(lib, {}) + ] + + # Core layers FIRST, across every requested library, so a db layer named before its + # core in the config still parses after it. + refs: list[str] = [] + seen: set[str] = set() + for lib, _ in wanted: + for ref in _LAYERS_BY_LIBRARY[lib][""]: + if ref not in seen: + seen.add(ref) + refs.append(ref) + for lib, layer in wanted: + if layer == "": + continue + for ref in _LAYERS_BY_LIBRARY[lib][layer]: + if ref not in seen: + seen.add(ref) + refs.append(ref) + + for ref in refs: + if directory is not None: + path = directory / f"{ref}.yaml" + if path.is_file(): + # The SAME id the embedded branch uses: a path-derived id would make a + # library node's error envelope differ between a checkout and an + # installed wheel, and would collide with an adopter file of the same + # basename. + out.append( + FileSource(path, id=library_file_id(ref), format=MetaDataFormat.YAML) ) + continue + + embedded = EMBEDDED_LIBRARY.get(ref) + if embedded is None: + raise ValueError( + f'library ref "{ref}" has no on-disk file and no ' + "embedded entry — the embedded library module is stale; run " + "scripts/generate_embedded_library.py" + ) + out.append( + InMemoryStringSource( + embedded, + id=library_file_id(ref), + format=MetaDataFormat.YAML, ) + ) return out diff --git a/server/python/src/metaobjects/loader/merge.py b/server/python/src/metaobjects/loader/merge.py index 15c83dc4a..95a61dd23 100644 --- a/server/python/src/metaobjects/loader/merge.py +++ b/server/python/src/metaobjects/loader/merge.py @@ -235,6 +235,11 @@ def _apply_overlay( ) ) return + # The loader's record that this node was amended BY an `overlay: true` declaration — + # mirrors the TS parser's `target.setIsMerge(true)`. Read by the FR-043 library guard + # to tell an intentional amendment from an ejected copy that landed in the same + # package; both merge, and only the flag separates them. + tc.is_merge = True _merge_into(tc, node, errors, warnings, envelope_warnings, None) @@ -382,7 +387,8 @@ def _merge_into( """Merge *src*'s own attrs/children into *target* in place. FR5c — runs three diagnostics around the merge: - 1. ``ERR_MERGE_CONFLICT`` on conflicting @-attrs (before the write). + 1. ``ERR_MERGE_CONFLICT`` on conflicting @-attrs (before the write), unless + *src* is marked ``overlay: true`` — see FR-043 Amendment 2 below. 2. ``MergedSource`` upgrade when the merge produced semantic change. 3. ``WARN_DUPLICATE_DECLARATION`` when no semantic change occurred AND the contributor file is new. @@ -398,7 +404,16 @@ def _merge_into( pre_canonical: Optional[str] = None if fr5c_active: pre_canonical = canonical_serialize(target) - _detect_attr_merge_conflicts(target, src, errors) + # FR-043 Amendment 2 — ``overlay: true`` LICENSES the override. The conflict + # error exists to catch two files that collided without knowing about each + # other; the flag is the author saying "I know about the other declaration and + # I mean to change it". The loader already treats it specially (find-or-throw + # versus create-or-find), so honouring it here makes it mean ONE thing rather + # than two. Per NODE, not per file: a nested overlay marks its own ancestors + # too, and each is judged on its own flag — same rule as the TS parser's + # ``nodeData[RESERVED_KEY_OVERLAY]`` check. + if not getattr(src, "is_overlay", False): + _detect_attr_merge_conflicts(target, src, errors) # ADR-0039 sanctioned own: overlay/merge — accumulate each file's own attrs + # own children (declared-here layers) into the merged tree. diff --git a/server/python/src/metaobjects/meta/meta_data.py b/server/python/src/metaobjects/meta/meta_data.py index 18c1e8df0..c8f2e7469 100644 --- a/server/python/src/metaobjects/meta/meta_data.py +++ b/server/python/src/metaobjects/meta/meta_data.py @@ -30,6 +30,12 @@ def __init__(self, type_: str, sub_type: str, name: str) -> None: self.super_data: Optional[MetaData] = None self.is_abstract = False self.is_overlay = False + # Set on a node an `overlay: true` declaration MERGED INTO (never on the + # overlay, which is discarded once applied). The loader's own record that the + # flag was honoured — read by the FR-043 library guard to tell an intentional + # amendment from a copy that landed in the same package, a distinction no + # comparison of the merged trees could make. Mirrors TS `MetaData.isMerge`. + self.is_merge = False self.is_array = False self.parent: Optional[MetaData] = None self._attr_nodes: dict[str, MetaData] = {} # name -> MetaAttr instance diff --git a/server/python/tests/codegen/gen_suite.py b/server/python/tests/codegen/gen_suite.py new file mode 100644 index 000000000..96776a5b0 --- /dev/null +++ b/server/python/tests/codegen/gen_suite.py @@ -0,0 +1,16 @@ +"""The generator selection the CLI mechanics tests use. + +These tests are about the write path, the hash manifest, baselines, column naming, +staleness nudges, provider loading and codegen-drift detection — not about which +generators an application should run. They used to get a suite for free from +``cli._default_generators``, which is gone: codegen is opt-in, and a run that names no +generator is a usage error with an empty out dir. + +So the suite is named here, ONCE, and it is deliberately the eight that used to be the +default — these tests' fixtures and assertions were written against exactly that output, +and changing what they generate would change what they are testing. This is a test-local +convenience, not a default restored by the back door: nothing in ``metaobjects`` reads it. +""" + +#: Stable generator names, comma-joined for ``--generators``. +GEN_SUITE = "entity,routes,filter-allowlist,names,payload,output-parser,output-prompt,extractor" diff --git a/server/python/tests/codegen/test_cli.py b/server/python/tests/codegen/test_cli.py index 2ef157779..d62dff0ed 100644 --- a/server/python/tests/codegen/test_cli.py +++ b/server/python/tests/codegen/test_cli.py @@ -12,6 +12,7 @@ from pathlib import Path from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE FIXTURE = ( Path(__file__).parents[4] @@ -33,7 +34,7 @@ def _meta_dir(tmp_path: Path) -> str: def test_gen_writes_files(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - rc = main(["gen", meta_dir, "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc == 0 written = list(out.rglob("*.py")) assert written, "gen wrote no files" @@ -46,7 +47,7 @@ def test_gen_entities_allowlist_emits_only_named(tmp_path: Path) -> None: loaded, so references resolve); the others are not written.""" meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - rc = main(["gen", meta_dir, "--out", str(out), "--entities", "Program,Week"]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--entities", "Program,Week"]) assert rc == 0 assert (out / "Program.py").exists() assert (out / "Week.py").exists() @@ -61,34 +62,34 @@ def test_verify_entities_allowlist_in_sync(tmp_path: Path) -> None: (without the filter it would flag the un-emitted entities as missing).""" meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out), "--entities", "Program,Week"]) == 0 - assert main(["verify", meta_dir, "--out", str(out), "--entities", "Program,Week"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--entities", "Program,Week"]) == 0 + assert main(["verify", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--entities", "Program,Week"]) == 0 def test_verify_in_sync_returns_zero(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 # Freshly generated → verify must report no drift. - assert main(["verify", meta_dir, "--out", str(out)]) == 0 + assert main(["verify", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 def test_verify_detects_drift(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 # Mutate a generated file → verify must detect codegen drift. target = out / "Program.py" target.write_text(target.read_text() + "\n# hand-edited drift\n") - assert main(["verify", meta_dir, "--out", str(out)]) != 0 + assert main(["verify", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) != 0 def test_verify_detects_missing_file(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 (out / "Program.py").unlink() - assert main(["verify", meta_dir, "--out", str(out)]) != 0 + assert main(["verify", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) != 0 def test_gen_load_error_returns_nonzero(tmp_path: Path) -> None: @@ -96,7 +97,7 @@ def test_gen_load_error_returns_nonzero(tmp_path: Path) -> None: bad.mkdir() (bad / "broken.json").write_text("{ not valid json") out = tmp_path / "out" - assert main(["gen", str(bad), "--out", str(out)]) != 0 + assert main(["gen", "--generators", GEN_SUITE, str(bad), "--out", str(out)]) != 0 _TEMPLATE_CORPUS = Path(__file__).parents[4] / "fixtures" / "template-codegen-conformance" @@ -110,7 +111,7 @@ def test_template_spec_output_gets_no_package_init(tmp_path: Path) -> None: out = tmp_path / "out" rc = main( [ - "gen", + "gen", "--generators", GEN_SUITE, str(_TEMPLATE_CORPUS / "metadata"), "--out", str(out), @@ -140,7 +141,7 @@ def test_template_spec_bad_ref_clean_error(tmp_path: Path, capsys) -> None: out = tmp_path / "out" rc = main( [ - "gen", + "gen", "--generators", GEN_SUITE, str(_TEMPLATE_CORPUS / "metadata"), "--out", str(out), @@ -191,7 +192,7 @@ def test_gen_auto_discovers_template_spec(tmp_path: Path) -> None: """`gen` with NO --template-spec picks up /template-spec.json.""" root = _spec_project(tmp_path) out = root / "out" - rc = main(["gen", str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) + rc = main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) assert rc == 0 assert (out / "Product.txt").exists(), "discovered spec's perEntity output missing" assert (out / "shop" / "_package.txt").exists() @@ -202,7 +203,7 @@ def test_gen_without_spec_file_emits_no_template_output(tmp_path: Path) -> None: """No spec file ⇒ today's behaviour exactly: the default suite only.""" root = _spec_project(tmp_path, spec_name=None) out = root / "out" - rc = main(["gen", str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) + rc = main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) assert rc == 0 assert not (out / "Product.txt").exists() assert not (out / "_model.txt").exists() @@ -214,13 +215,13 @@ def test_verify_codegen_sees_the_discovered_template_spec(tmp_path: Path) -> Non list, never saw the spec, and convicted every spec-emitted file as `extra:`.""" root = _spec_project(tmp_path) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 # Non-vacuous: without discovery `gen` emits none of these, so a verify that # "passes" would only be agreeing that nothing exists. assert (out / "Product.txt").exists() assert (out / "_model.txt").exists() - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates-root", str(root / "templates")]) assert rc == 0, "verify --codegen convicted output that `gen` had just written" @@ -231,10 +232,10 @@ def test_verify_codegen_still_catches_a_missing_template_file(tmp_path: Path) -> spec-generated file must still be reported as drift.""" root = _spec_project(tmp_path) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 (out / "Product.txt").unlink() - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates-root", str(root / "templates")]) assert rc == 1, "verify went blind — a deleted template-spec file is still drift" @@ -243,10 +244,10 @@ def test_verify_codegen_catches_a_stale_template_file(tmp_path: Path) -> None: """Same guard, the other direction: edited content is drift, not just absence.""" root = _spec_project(tmp_path) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 (out / "Product.txt").write_text("stale\n") - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates-root", str(root / "templates")]) assert rc == 1 @@ -270,10 +271,10 @@ def test_verify_codegen_clean_for_a_py_emitting_spec(tmp_path: Path) -> None: '"scope": "perEntity", "outputPattern": "{name}Service.py"}]}' ) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 assert (out / "ProductService.py").exists() - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates-root", str(root / "templates")]) assert rc == 0, "the reported `extra:` false-conviction is back" @@ -285,12 +286,12 @@ def test_verify_codegen_ignores_a_file_it_never_wrote(tmp_path: Path) -> None: — otherwise the broadened glob turns a clean project red.""" root = _spec_project(tmp_path) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 (out / "NOTES.md").write_text("hand-written, not ours\n") (out / "sub").mkdir() (out / "sub" / "stray.txt").write_text("also not ours\n") - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates-root", str(root / "templates")]) assert rc == 0, "the gate convicted a file it never wrote" @@ -299,11 +300,11 @@ def test_verify_codegen_ignores_pycache(tmp_path: Path) -> None: """Interpreter droppings are never artifacts, manifest or not.""" root = _spec_project(tmp_path) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 (out / "__pycache__").mkdir() (out / "__pycache__" / "Product.cpython-312.pyc").write_bytes(b"\x00\x01binary") - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates-root", str(root / "templates")]) assert rc == 0 @@ -317,7 +318,7 @@ def test_explicit_template_spec_flag_overrides_discovery(tmp_path: Path) -> None '"scope": "perEntity", "outputPattern": "{name}.flagged.txt"}]}' ) out = root / "out" - rc = main(["gen", str(root / "meta"), "--out", str(out), + rc = main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates"), "--template-spec", str(other)]) assert rc == 0 assert (out / "Product.flagged.txt").exists(), "the flag's spec did not run" @@ -331,7 +332,7 @@ def test_malformed_discovered_spec_is_a_clean_error(tmp_path: Path, capsys) -> N root = _spec_project(tmp_path) (root / "template-spec.json").write_text("{ not json") out = root / "out" - rc = main(["gen", str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) + rc = main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) assert rc == 1 err = capsys.readouterr().err assert "error:" in err @@ -358,7 +359,7 @@ def test_gen_manifest_keys_are_project_relative(tmp_path: Path) -> None: back), and a unit test of any one of them passes while the other two disagree.""" meta_dir = _meta_dir(tmp_path) out = tmp_path / "build" / "gen" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 manifest = json.loads( (tmp_path / ".metaobjects" / ".gen-state" / ".hashes.json").read_text() @@ -375,8 +376,8 @@ def test_two_out_dirs_write_disjoint_manifest_entries(tmp_path: Path) -> None: re-key they were one entry, and whichever ran last decided whether the OTHER run's file still counted as ours.""" meta_dir = _meta_dir(tmp_path) - assert main(["gen", meta_dir, "--out", str(tmp_path / "a")]) == 0 - assert main(["gen", meta_dir, "--out", str(tmp_path / "b")]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(tmp_path / "a")]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(tmp_path / "b")]) == 0 manifest = json.loads( (tmp_path / ".metaobjects" / ".gen-state" / ".hashes.json").read_text() @@ -395,7 +396,7 @@ def test_verify_codegen_still_convicts_output_it_did_write(tmp_path: Path) -> No the gate kept printing a clean verdict. This is the assertion that can see it.""" meta_dir = Path(_meta_dir(tmp_path)) out = tmp_path / "build" / "gen" - assert main(["gen", str(meta_dir), "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, str(meta_dir), "--out", str(out)]) == 0 assert (out / "Program.py").exists() # Remove the metadata: a regen no longer emits Program.py, but it IS committed and it @@ -403,7 +404,7 @@ def test_verify_codegen_still_convicts_output_it_did_write(tmp_path: Path) -> No (meta_dir / "meta.fitness.json").write_text( '{"metadata.root": {"package": "fitness", "children": []}}' ) - assert main(["verify", "--codegen", str(meta_dir), "--out", str(out)]) == 1 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, str(meta_dir), "--out", str(out)]) == 1 def test_gen_baseline_adopt_records_without_writing(tmp_path: Path, capsys) -> None: @@ -417,12 +418,12 @@ def test_gen_baseline_adopt_records_without_writing(tmp_path: Path, capsys) -> N """ meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 stale = out / "Program.py" stale.write_text("# @generated by metaobjects\n# from an older engine\n") shutil.rmtree(tmp_path / ".metaobjects", ignore_errors=True) - assert main(["gen", meta_dir, "--out", str(out), "--baseline=adopt"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--baseline=adopt"]) == 0 assert stale.read_text() == "# @generated by metaobjects\n# from an older engine\n" assert (tmp_path / ".metaobjects" / ".gen-state" / ".hashes.json").exists() @@ -439,14 +440,14 @@ def test_gen_exits_non_zero_when_a_file_is_refused(tmp_path: Path) -> None: """ meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 (out / "Program.py").write_text("# @generated by metaobjects\n# from an older engine\n") shutil.rmtree(tmp_path / ".metaobjects", ignore_errors=True) - assert main(["gen", meta_dir, "--out", str(out)]) == 1 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 1 # …and the remedy clears it, which is what makes the failure actionable rather than a wall. - assert main(["gen", meta_dir, "--out", str(out), "--baseline=adopt"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--baseline=adopt"]) == 0 def test_prompts_and_templates_root_are_the_same_flag(tmp_path: Path) -> None: @@ -460,11 +461,11 @@ def test_prompts_and_templates_root_are_the_same_flag(tmp_path: Path) -> None: vacuous success.""" root = _spec_project(tmp_path) out = root / "out" - assert main(["gen", str(root / "meta"), "--out", str(out), + assert main(["gen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--templates", str(root / "templates")]) == 0 for spelling in ("--prompts", "--templates-root"): - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), spelling, str(root / "templates")]) assert rc == 0, f"verify rejected output `gen` had just written, via {spelling}" @@ -473,6 +474,6 @@ def test_prompts_and_templates_root_are_the_same_flag(tmp_path: Path) -> None: # also hold for a parser that accepted the flag and ignored its value. (The --templates # gate is the wrong probe here: this fixture declares no template.* nodes, so that gate is # vacuously clean whatever the directory says.) - rc = main(["verify", "--codegen", str(root / "meta"), "--out", str(out), + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, str(root / "meta"), "--out", str(out), "--prompts", str(root / "no-such-dir")]) assert rc != 0, "--prompts value was ignored — a missing dir still regenerated the spec files" diff --git a/server/python/tests/codegen/test_cli_config_gen.py b/server/python/tests/codegen/test_cli_config_gen.py index d694553ac..51a46b822 100644 --- a/server/python/tests/codegen/test_cli_config_gen.py +++ b/server/python/tests/codegen/test_cli_config_gen.py @@ -6,6 +6,7 @@ from pathlib import Path from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE FITNESS = ( Path(__file__).parents[4] @@ -41,7 +42,7 @@ def _project(tmp_path: Path, config_text: str, meta_subdir: str = "metaobjects") def test_gen_no_args_runs_all_targets_via_config_flag(tmp_path: Path) -> None: cfg = _project(tmp_path, TWO_TARGETS) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 0 assert (tmp_path / "gen/models/Program.py").exists() assert (tmp_path / "gen/models/Week.py").exists() @@ -62,7 +63,7 @@ def test_gen_no_args_discovers_config_in_cwd(tmp_path: Path, monkeypatch) -> Non def test_gen_target_scopes_to_one(tmp_path: Path) -> None: cfg = _project(tmp_path, TWO_TARGETS) - rc = main(["gen", "--config", str(cfg), "--target", "models"]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg), "--target", "models"]) assert rc == 0 assert (tmp_path / "gen/models/Program.py").exists() assert not (tmp_path / "gen/other").exists() @@ -70,7 +71,7 @@ def test_gen_target_scopes_to_one(tmp_path: Path) -> None: def test_gen_unknown_target_errors(tmp_path: Path, capsys) -> None: cfg = _project(tmp_path, TWO_TARGETS) - rc = main(["gen", "--config", str(cfg), "--target", "nope"]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg), "--target", "nope"]) assert rc == 1 assert "unknown --target" in capsys.readouterr().err @@ -97,7 +98,7 @@ def test_gen_missing_config_errors(tmp_path: Path, monkeypatch, capsys) -> None: def test_gen_cross_target_duplicate_output_path_guard(tmp_path: Path, capsys) -> None: cfg = _project(tmp_path, DUP_TARGETS) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 1 assert "duplicate output path across targets" in capsys.readouterr().err @@ -107,7 +108,7 @@ def test_verify_dup_targets_config_rejected(tmp_path: Path, capsys) -> None: gen (verify is symmetric with gen): the DUP_TARGETS config — two targets emit the same Program.py into the same outDir — is rejected with exit 1.""" cfg = _project(tmp_path, DUP_TARGETS) - rc = main(["verify", "--codegen", "--config", str(cfg)]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 1 assert "duplicate output path across targets" in capsys.readouterr().err @@ -130,7 +131,7 @@ def test_gen_cross_target_shared_outdir_disjoint_entities_not_flagged(tmp_path: auto-emitted package-marker __init__.py is byte-identical across both targets and must not trip the cross-target duplicate-output-path guard.""" cfg = _project(tmp_path, SHARED_OUTDIR_DISJOINT_ENTITIES) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 0 assert (tmp_path / "shared/Program.py").exists() assert (tmp_path / "shared/Week.py").exists() @@ -203,7 +204,7 @@ def test_gen_resolves_provider_config_relative_without_pythonpath(tmp_path: Path ) assert str(tmp_path) not in sys.path # precondition: not already importable try: - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) finally: if str(tmp_path) in sys.path: sys.path.remove(str(tmp_path)) @@ -219,7 +220,7 @@ def test_gen_flag_path_ignores_config_when_present(tmp_path: Path) -> None: out = tmp_path / "flagout" meta = tmp_path / "metaobjects" # created by _project # Flag path: metadata_dir + --out present => config ignored, normal gen. - rc = main(["gen", str(meta), "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, str(meta), "--out", str(out)]) assert rc == 0 assert (out / "Program.py").exists() @@ -231,7 +232,7 @@ def test_gen_missing_metadata_dir_errors_cleanly(tmp_path: Path, capsys) -> None fix) — `iterdir()` raises OSError on a directory that never existed; nothing on the `_load_root` path caught it.""" missing = tmp_path / "does" / "not" / "exist" - rc = main(["gen", str(missing), "--out", str(tmp_path / "out")]) + rc = main(["gen", "--generators", GEN_SUITE, str(missing), "--out", str(tmp_path / "out")]) assert rc == 1 err = capsys.readouterr().err assert "error: failed to load metadata" in err @@ -242,7 +243,7 @@ def test_gen_metadata_dir_is_a_file_errors_cleanly(tmp_path: Path, capsys) -> No to a plain file rather than a directory.""" not_a_dir = tmp_path / "somefile.txt" not_a_dir.write_text("not a directory") - rc = main(["gen", str(not_a_dir), "--out", str(tmp_path / "out")]) + rc = main(["gen", "--generators", GEN_SUITE, str(not_a_dir), "--out", str(tmp_path / "out")]) assert rc == 1 err = capsys.readouterr().err assert "error: failed to load metadata" in err diff --git a/server/python/tests/codegen/test_cli_config_verify.py b/server/python/tests/codegen/test_cli_config_verify.py index 74418bb65..13165d29f 100644 --- a/server/python/tests/codegen/test_cli_config_verify.py +++ b/server/python/tests/codegen/test_cli_config_verify.py @@ -5,6 +5,7 @@ from pathlib import Path from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE FITNESS = ( Path(__file__).parents[4] @@ -50,9 +51,9 @@ def _project(tmp_path: Path, config_text: str = TWO_TARGETS) -> Path: def test_verify_codegen_no_args_in_sync(tmp_path: Path) -> None: cfg = _project(tmp_path) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 # Fresh gen → no drift across every target. - assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 def test_verify_codegen_bare_defaults_to_codegen(tmp_path: Path, monkeypatch) -> None: @@ -64,10 +65,10 @@ def test_verify_codegen_bare_defaults_to_codegen(tmp_path: Path, monkeypatch) -> def test_verify_codegen_detects_drift_in_one_target(tmp_path: Path, capsys) -> None: cfg = _project(tmp_path) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 target = tmp_path / "gen/other/Node.py" target.write_text(target.read_text() + "\n# hand-edited drift\n") - rc = main(["verify", "--codegen", "--config", str(cfg)]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 1 err = capsys.readouterr().err assert "[other]" in err and "drifted" in err @@ -75,12 +76,12 @@ def test_verify_codegen_detects_drift_in_one_target(tmp_path: Path, capsys) -> N def test_verify_codegen_target_scopes(tmp_path: Path) -> None: cfg = _project(tmp_path) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 # Drift in `other`, but scope verify to `models` → clean. target = tmp_path / "gen/other/Node.py" target.write_text(target.read_text() + "\n# drift\n") - assert main(["verify", "--codegen", "--config", str(cfg), "--target", "models"]) == 0 - assert main(["verify", "--codegen", "--config", str(cfg), "--target", "other"]) == 1 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg), "--target", "models"]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg), "--target", "other"]) == 1 def test_verify_flag_path_still_works_with_config_present(tmp_path: Path) -> None: @@ -88,15 +89,15 @@ def test_verify_flag_path_still_works_with_config_present(tmp_path: Path) -> Non _project(tmp_path) meta = tmp_path / "metaobjects" out = tmp_path / "flagout" - assert main(["gen", str(meta), "--out", str(out)]) == 0 - assert main(["verify", str(meta), "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, str(meta), "--out", str(out)]) == 0 + assert main(["verify", "--generators", GEN_SUITE, str(meta), "--out", str(out)]) == 0 def test_verify_templates_config_mode_requires_metadata_dir(tmp_path: Path) -> None: """`verify --templates` is not config-driven — the guard returns exit 2 when no positional metadata_dir is given (config mode / --templates only drives --codegen).""" - assert main(["verify", "--templates"]) == 2 + assert main(["verify", "--generators", GEN_SUITE, "--templates"]) == 2 def test_verify_codegen_shared_outdir_disjoint_entities_in_sync(tmp_path: Path) -> None: @@ -104,21 +105,21 @@ def test_verify_codegen_shared_outdir_disjoint_entities_in_sync(tmp_path: Path) verify --codegen must NOT report the co-resident target's files as false `extra` drift. The diff is union-of-co-resident-regen vs the shared dir.""" cfg = _project(tmp_path, SHARED_OUTDIR) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 assert (tmp_path / "shared/Program.py").exists() assert (tmp_path / "shared/Week.py").exists() # Bug repro: this exits 1 today with a false `extra` on both targets. - assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 def test_verify_codegen_shared_outdir_detects_real_drift(tmp_path: Path, capsys) -> None: """Real drift is still detected under a shared outDir: hand-editing one co-resident target's file flags it as `drifted` (labeled for the shared unit).""" cfg = _project(tmp_path, SHARED_OUTDIR) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 target = tmp_path / "shared/Week.py" target.write_text(target.read_text() + "\n# hand-edited drift\n") - rc = main(["verify", "--codegen", "--config", str(cfg)]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 1 err = capsys.readouterr().err assert "drifted" in err and "Week.py" in err @@ -146,10 +147,10 @@ def test_verify_codegen_shared_outdir_detects_stale_extra(tmp_path: Path, capsys been generated at all, so the test was pinning the pre-jurisdiction behaviour where the gate convicted every stranger in the directory.""" cfg = _project(tmp_path, SHARED_OUTDIR) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 (tmp_path / "shared/Orphan.py").write_text("# we wrote this before; regen no longer emits it\n") _record_as_written(tmp_path, "Orphan.py") - rc = main(["verify", "--codegen", "--config", str(cfg)]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 1 err = capsys.readouterr().err assert "extra" in err and "Orphan.py" in err @@ -160,9 +161,9 @@ def test_verify_codegen_shared_outdir_ignores_a_stranger(tmp_path: Path) -> None to convict. `outDir` is a directory, not a namespace this tool owns — convicting strangers is what failed projects with zero drift (the TS gate's 0.24.3 ruling).""" cfg = _project(tmp_path, SHARED_OUTDIR) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 (tmp_path / "shared/HAND_WRITTEN.md").write_text("mine, not yours\n") - assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 def test_verify_target_scoping_widens_to_shared_outdir(tmp_path: Path, capsys) -> None: @@ -170,16 +171,16 @@ def test_verify_target_scoping_widens_to_shared_outdir(tmp_path: Path, capsys) - as a unit: no false positive on a clean tree (with a widening note), and a drift in target b's file IS caught because the shared dir is verified together.""" cfg = _project(tmp_path, SHARED_OUTDIR) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 # Clean tree: --target a widens to cover the shared dir (b co-resident) → no false positive. - rc = main(["verify", "--codegen", "--config", str(cfg), "--target", "a"]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg), "--target", "a"]) assert rc == 0 note = capsys.readouterr().err assert "note:" in note and "shares an outDir" in note and "b" in note # Drift target b's file; --target a still catches it (shared dir verified as a unit). week = tmp_path / "shared/Week.py" week.write_text(week.read_text() + "\n# drift\n") - rc = main(["verify", "--codegen", "--config", str(cfg), "--target", "a"]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg), "--target", "a"]) assert rc == 1 assert "Week.py" in capsys.readouterr().err @@ -210,13 +211,13 @@ def test_verify_codegen_no_args_no_yaml_falls_back_to_neutral_config( # regenerates the full default suite), so `gen` must run the full suite # too or the diff reports the un-emitted generators as spurious drift — # same constraint the flag-mode docstring notes for --entities. - assert main(["gen", "--out", "gen/models"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--out", "gen/models"]) == 0 # Fresh gen -> no drift, via the same fallback rung. - assert main(["verify", "--codegen", "--out", "gen/models"]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--out", "gen/models"]) == 0 # Drift the committed output; the fallback rung must still catch it. program = tmp_path / "gen/models/Program.py" program.write_text(program.read_text() + "\n# drift\n") - assert main(["verify", "--codegen", "--out", "gen/models"]) == 1 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--out", "gen/models"]) == 1 def test_verify_codegen_neutral_fallback_threads_column_naming( @@ -239,19 +240,19 @@ def test_verify_codegen_neutral_fallback_threads_column_naming( ) monkeypatch.chdir(tmp_path) - assert main(["gen", "--out", "gen/models", "--column-naming", "snake_case"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--out", "gen/models", "--column-naming", "snake_case"]) == 0 capsys.readouterr() # Matching strategy -> clean. rc_clean = main( - ["verify", "--codegen", "--out", "gen/models", "--column-naming", "snake_case"] + ["verify", "--codegen", "--generators", GEN_SUITE, "--out", "gen/models", "--column-naming", "snake_case"] ) assert rc_clean == 0, capsys.readouterr().err # Mismatched strategy -> the discriminating half: proves the flag is actually # read at this rung, not merely accepted and dropped. rc_drift = main( - ["verify", "--codegen", "--out", "gen/models", "--column-naming", "literal"] + ["verify", "--codegen", "--generators", GEN_SUITE, "--out", "gen/models", "--column-naming", "literal"] ) err = capsys.readouterr().err assert rc_drift == 1 @@ -272,7 +273,7 @@ def test_config_mode_refuses_template_spec_instead_of_ignoring_it( '{"generators": [{"name": "s", "template": "entity", ' '"scope": "perEntity", "outputPattern": "{name}.txt"}]}' ) - rc = main(["gen", "--config", str(cfg), "--template-spec", str(spec)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg), "--template-spec", str(spec)]) assert rc == 2 err = capsys.readouterr().err assert "--template-spec is not supported in declarative-config mode" in err @@ -288,8 +289,8 @@ def test_config_mode_ignores_a_discovered_spec(tmp_path: Path) -> None: '{"generators": [{"name": "s", "template": "entity", ' '"scope": "perEntity", "outputPattern": "{name}.txt"}]}' ) - assert main(["gen", "--config", str(cfg)]) == 0 - assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 # --- the `extra` verdict at the two rungs the flag-mode test cannot reach ------------ @@ -319,7 +320,7 @@ def test_neutral_fallback_convicts_output_it_did_write( '{"schema_version": 1, "sources": [{"path": "model"}]}' ) monkeypatch.chdir(tmp_path) - assert main(["gen", "--out", "gen/models"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--out", "gen/models"]) == 0 assert (tmp_path / "gen/models/Program.py").exists() # A regen no longer emits Program.py, but it IS committed and IS in the manifest — @@ -327,7 +328,7 @@ def test_neutral_fallback_convicts_output_it_did_write( (model / "meta.fitness.json").write_text( '{"metadata.root": {"package": "fitness", "children": []}}' ) - assert main(["verify", "--codegen", "--out", "gen/models"]) == 1 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--out", "gen/models"]) == 1 def test_config_mode_convicts_output_it_did_write(tmp_path: Path) -> None: @@ -335,7 +336,7 @@ def test_config_mode_convicts_output_it_did_write(tmp_path: Path) -> None: prefix matters most, because every target has its own outDir and all of them share ONE manifest.""" cfg = _project(tmp_path) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 assert (tmp_path / "gen/models/Program.py").exists() # Narrow `models` to one entity: Week.py stays committed and recorded while a fresh @@ -343,4 +344,4 @@ def test_config_mode_convicts_output_it_did_write(tmp_path: Path) -> None: cfg.write_text( TWO_TARGETS.replace("entities: [Program, Week]", "entities: [Program]") ) - assert main(["verify", "--codegen", "--config", str(cfg)]) == 1 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 1 diff --git a/server/python/tests/codegen/test_cli_dependencies.py b/server/python/tests/codegen/test_cli_dependencies.py index b0b0d75c3..be311c0e3 100644 --- a/server/python/tests/codegen/test_cli_dependencies.py +++ b/server/python/tests/codegen/test_cli_dependencies.py @@ -15,6 +15,7 @@ from pathlib import Path from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE _CORPUS_ARTIFACT = ( Path(__file__).resolve().parents[4] @@ -89,7 +90,7 @@ def _consumer( def test_gen_excludes_the_dependencys_entities_by_default(tmp_path: Path) -> None: cfg = _consumer(tmp_path) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 0 assert (tmp_path / "gen" / "order_names.py").exists() assert not (tmp_path / "gen" / "customer_names.py").exists() @@ -97,7 +98,7 @@ def test_gen_excludes_the_dependencys_entities_by_default(tmp_path: Path) -> Non def test_gen_scope_include_naming_the_package_opts_it_in(tmp_path: Path) -> None: cfg = _consumer(tmp_path, scope_include=["acme::common::**"]) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 0 assert (tmp_path / "gen" / "order_names.py").exists() assert (tmp_path / "gen" / "customer_names.py").exists() @@ -107,7 +108,7 @@ def test_gen_a_wildcard_scope_include_does_not_opt_the_package_in(tmp_path: Path # `acme::**` REACHES the package's nodes without NAMING it literally # (`explicitly_includes`) — the default exclusion still applies. cfg = _consumer(tmp_path, scope_include=["acme::**"]) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 0 assert (tmp_path / "gen" / "order_names.py").exists() assert not (tmp_path / "gen" / "customer_names.py").exists() @@ -117,7 +118,7 @@ def test_gen_target_naming_an_excluded_import_refuses_with_exit_2( tmp_path: Path, capsys ) -> None: cfg = _consumer(tmp_path, target_entities=["Customer"]) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 2 err = capsys.readouterr().err assert "'Customer'" in err @@ -130,7 +131,7 @@ def test_gen_target_naming_an_excluded_import_is_fine_once_scope_includes_it( tmp_path: Path, ) -> None: cfg = _consumer(tmp_path, scope_include=["acme::common::**"], target_entities=["Customer"]) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc == 0 assert (tmp_path / "gen" / "customer_names.py").exists() assert not (tmp_path / "gen" / "order_names.py").exists() @@ -138,16 +139,16 @@ def test_gen_target_naming_an_excluded_import_is_fine_once_scope_includes_it( def test_verify_codegen_shares_the_selection_with_gen(tmp_path: Path) -> None: cfg = _consumer(tmp_path) - assert main(["gen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 # Fresh gen, same selection → verify --codegen must see no drift, and must # NOT regenerate the excluded customer_names.py and report it "missing". - assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 def test_verify_codegen_shares_the_selection_after_scope_widens(tmp_path: Path) -> None: cfg = _consumer(tmp_path, scope_include=["acme::common::**"]) - assert main(["gen", "--config", str(cfg)]) == 0 - assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, "--config", str(cfg)]) == 0 def test_gen_a_stale_snapshot_is_refused_before_generation(tmp_path: Path, capsys) -> None: @@ -161,7 +162,7 @@ def test_gen_a_stale_snapshot_is_refused_before_generation(tmp_path: Path, capsy data[0] ^= 0xFF # one bit-flipped byte -> the lock's pinned sha256 no longer matches artifact.write_bytes(bytes(data)) - rc = main(["gen", "--config", str(cfg)]) + rc = main(["gen", "--generators", GEN_SUITE, "--config", str(cfg)]) assert rc != 0 err = capsys.readouterr().err # `_resolve_metadata_location_or_print_error` prints `str(exc)` — a diff --git a/server/python/tests/codegen/test_cli_providers.py b/server/python/tests/codegen/test_cli_providers.py index 879694110..8416f0d3a 100644 --- a/server/python/tests/codegen/test_cli_providers.py +++ b/server/python/tests/codegen/test_cli_providers.py @@ -14,6 +14,8 @@ from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE + # A provider module the test writes to disk + imports. Registers the custom # ``validator.geocheck`` subtype (a validator, so it exercises provider-threading # without depending on how codegen maps a novel field's physical type). @@ -78,7 +80,7 @@ def _install_provider_module(tmp_path: Path, name: str) -> None: def test_gen_without_provider_fails_on_custom_subtype(tmp_path: Path) -> None: """Baseline: the custom subtype is unknown without the provider.""" - rc = main(["gen", _project(tmp_path), "--out", str(tmp_path / "out")]) + rc = main(["gen", "--generators", GEN_SUITE, _project(tmp_path), "--out", str(tmp_path / "out")]) assert rc != 0 @@ -89,7 +91,7 @@ def test_gen_with_provider_loads_custom_subtype(tmp_path: Path) -> None: try: rc = main( [ - "gen", + "gen", "--generators", GEN_SUITE, meta_dir, "--out", str(tmp_path / "out"), @@ -111,14 +113,14 @@ def test_verify_with_provider_loads_custom_subtype(tmp_path: Path) -> None: try: # generate committed output first (with the provider), then verify no drift. gen_rc = main( - ["gen", meta_dir, "--out", str(out), "--provider", "geo_prov_ver:geo_provider"] + ["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--provider", "geo_prov_ver:geo_provider"] ) assert gen_rc == 0 rc = main( [ "verify", meta_dir, - "--codegen", + "--codegen", "--generators", GEN_SUITE, "--out", str(out), "--provider", @@ -135,7 +137,7 @@ def test_bad_provider_spec_reports_error(tmp_path: Path) -> None: """A malformed --provider spec fails cleanly (not 'module:symbol').""" rc = main( [ - "gen", + "gen", "--generators", GEN_SUITE, _project(tmp_path), "--out", str(tmp_path / "out"), diff --git a/server/python/tests/codegen/test_cli_registry.py b/server/python/tests/codegen/test_cli_registry.py index 5baa89d40..d830d9253 100644 --- a/server/python/tests/codegen/test_cli_registry.py +++ b/server/python/tests/codegen/test_cli_registry.py @@ -11,6 +11,7 @@ from metaobjects.cli import main from metaobjects.codegen.generator_registry import GENERATOR_REGISTRY, list_generators +from tests.codegen.gen_suite import GEN_SUITE FIXTURE = ( Path(__file__).parents[4] @@ -29,7 +30,7 @@ def _meta_dir(tmp_path: Path) -> str: def test_gen_list_prints_all_and_exits_zero(capsys, tmp_path: Path) -> None: - rc = main(["gen", "--list"]) + rc = main(["gen", "--generators", GEN_SUITE, "--list"]) assert rc == 0 out = capsys.readouterr().out # Every registered stable name appears in the listing. @@ -44,7 +45,7 @@ def test_gen_list_prints_all_and_exits_zero(capsys, tmp_path: Path) -> None: def test_gen_list_does_not_run_codegen(capsys, tmp_path: Path) -> None: out = tmp_path / "out" - rc = main(["gen", "--list", "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, "--list", "--out", str(out)]) assert rc == 0 # --list must NOT write any generated files even when --out is given. assert not out.exists() or not list(out.rglob("*.py")) diff --git a/server/python/tests/codegen/test_cli_staleness_nudge.py b/server/python/tests/codegen/test_cli_staleness_nudge.py index 3e8573f1f..dfcb6d3c0 100644 --- a/server/python/tests/codegen/test_cli_staleness_nudge.py +++ b/server/python/tests/codegen/test_cli_staleness_nudge.py @@ -11,6 +11,7 @@ from pathlib import Path from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE FIXTURE = ( Path(__file__).parents[4] @@ -43,7 +44,7 @@ def test_gen_nudges_on_stale_manifest(tmp_path, capsys, monkeypatch) -> None: monkeypatch.chdir(tmp_path) _write_manifest(tmp_path, "0.0.1-old") - rc = main(["gen", meta_dir, "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc == 0 # advisory: never changes the exit code err = capsys.readouterr().err assert "0.0.1-old" in err @@ -56,12 +57,12 @@ def test_verify_nudges_on_stale_manifest(tmp_path, capsys, monkeypatch) -> None: monkeypatch.chdir(tmp_path) # Generate AND verify under the same cwd so verify --codegen is in-sync (no # DRIFT) — isolating the nudge from the codegen exit code. - main(["gen", meta_dir, "--out", str(out)]) + main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) capsys.readouterr() # drain _write_manifest(tmp_path, "0.0.1-old") - rc = main(["verify", meta_dir, "--codegen", "--out", str(out)]) + rc = main(["verify", meta_dir, "--codegen", "--generators", GEN_SUITE, "--out", str(out)]) assert rc == 0 err = capsys.readouterr().err assert "0.0.1-old" in err @@ -73,7 +74,7 @@ def test_gen_silent_when_no_manifest(tmp_path, capsys, monkeypatch) -> None: out = tmp_path / "out" monkeypatch.chdir(tmp_path) # no manifest in cwd - rc = main(["gen", meta_dir, "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc == 0 err = capsys.readouterr().err assert "npx meta agent-docs" not in err @@ -87,7 +88,7 @@ def test_gen_silent_on_corrupt_manifest(tmp_path, capsys, monkeypatch) -> None: p.parent.mkdir(parents=True, exist_ok=True) p.write_text("{ this is not valid json ") # corrupt → silently ignored - rc = main(["gen", meta_dir, "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc == 0 err = capsys.readouterr().err assert "npx meta agent-docs" not in err diff --git a/server/python/tests/codegen/test_cli_verify_strict.py b/server/python/tests/codegen/test_cli_verify_strict.py index 2e58480dc..af9f648b8 100644 --- a/server/python/tests/codegen/test_cli_verify_strict.py +++ b/server/python/tests/codegen/test_cli_verify_strict.py @@ -11,6 +11,7 @@ from pathlib import Path from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE # Shared cross-port fixture (also asserted by the TS CLI verify-strict test): # a registered field.string carrying one undeclared own @attr. @@ -60,7 +61,7 @@ def test_verify_codegen_fails_on_undeclared_attr_by_default( ) -> None: meta_dir = _meta_dir(tmp_path, _MADE_UP) out = tmp_path / "out" - rc = main(["verify", "--codegen", meta_dir, "--out", str(out)]) + rc = main(["verify", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc != 0 err = capsys.readouterr().err assert "ERR_UNKNOWN_ATTR" in err @@ -75,8 +76,8 @@ def test_verify_codegen_passes_with_lax(tmp_path: Path) -> None: # Lax load tolerates the undeclared attr; gen-to-temp + diff (vs --out) is # the codegen drift result, NOT a load failure. First gen (already lax), # then verify --lax against the committed output. - assert main(["gen", meta_dir, "--out", str(out)]) == 0 - assert main(["verify", "--lax", "--codegen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 + assert main(["verify", "--lax", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 # --- verify --templates ---------------------------------------------------- @@ -116,7 +117,7 @@ def test_verify_templates_fails_on_undeclared_attr_by_default( troot = tmp_path / "templates" (troot / "pages").mkdir(parents=True) (troot / "pages" / "welcome.mustache").write_text("Hello {{name}}") - rc = main(["verify", "--templates", meta_dir, "--templates-root", str(troot)]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", str(troot)]) assert rc != 0 err = capsys.readouterr().err assert "ERR_UNKNOWN_ATTR" in err @@ -129,11 +130,11 @@ def test_gen_stays_lax_by_default(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path, _MADE_UP) out = tmp_path / "out" # gen tolerates the undeclared attr (no strict default for gen). - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 def test_verify_clean_metadata_passes_under_strict(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path, _CLEAN) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 - assert main(["verify", "--codegen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 diff --git a/server/python/tests/codegen/test_cli_verify_subverbs.py b/server/python/tests/codegen/test_cli_verify_subverbs.py index eb7a40d16..70ccc384a 100644 --- a/server/python/tests/codegen/test_cli_verify_subverbs.py +++ b/server/python/tests/codegen/test_cli_verify_subverbs.py @@ -23,6 +23,7 @@ import pytest from metaobjects.cli import main +from tests.codegen.gen_suite import GEN_SUITE FITNESS = ( Path(__file__).parents[4] @@ -135,17 +136,17 @@ def _meta_dir_with(tmp_path: Path, meta_json: str) -> str: def test_codegen_in_sync_returns_zero(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 - assert main(["verify", "--codegen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 def test_codegen_detects_drift(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 target = out / "Program.py" target.write_text(target.read_text() + "\n# hand-edited drift\n") - assert main(["verify", "--codegen", meta_dir, "--out", str(out)]) != 0 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) != 0 # --- 2. --templates = render-verify drift ----------------------------------- @@ -154,14 +155,14 @@ def test_codegen_detects_drift(tmp_path: Path) -> None: def test_templates_clean_returns_zero(tmp_path: Path) -> None: meta_dir = _meta_dir_with(tmp_path, _META_CLEAN) troot = _templates_dir(tmp_path, "Hello {{name}}") - rc = main(["verify", "--templates", meta_dir, "--templates-root", troot]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", troot]) assert rc == 0 def test_templates_field_not_on_payload_is_drift(tmp_path: Path, capsys) -> None: meta_dir = _meta_dir_with(tmp_path, _META_CLEAN) troot = _templates_dir(tmp_path, "Hello {{missing}}") - rc = main(["verify", "--templates", meta_dir, "--templates-root", troot]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", troot]) assert rc != 0 err = capsys.readouterr().err # Names the offending field + the template. @@ -174,7 +175,7 @@ def test_templates_unresolvable_ref_is_drift(tmp_path: Path) -> None: # Empty templates root → pages/welcome.mustache does not exist. troot = tmp_path / "empty_templates" troot.mkdir() - rc = main(["verify", "--templates", meta_dir, "--templates-root", str(troot)]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", str(troot)]) assert rc != 0 @@ -183,7 +184,7 @@ def test_templates_email_clean_returns_zero(tmp_path: Path) -> None: # fields is clean (#193 — email parts drift-checked like a document body). meta_dir = _meta_dir_with(tmp_path, _META_EMAIL) troot = _email_templates_dir(tmp_path, "Hello {{name}}", "

Hi {{name}}

") - rc = main(["verify", "--templates", meta_dir, "--templates-root", troot]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", troot]) assert rc == 0 @@ -192,7 +193,7 @@ def test_templates_email_body_drift_is_caught(tmp_path: Path, capsys) -> None: # per-port bug was skipping @kind=email templates entirely. meta_dir = _meta_dir_with(tmp_path, _META_EMAIL) troot = _email_templates_dir(tmp_path, "Hello {{name}}", "

Hi {{missing}}

") - rc = main(["verify", "--templates", meta_dir, "--templates-root", troot]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", troot]) assert rc != 0 err = capsys.readouterr().err assert "missing" in err @@ -234,7 +235,7 @@ def test_templates_prompt_required_tag_missing_is_drift(tmp_path: Path, capsys) # enforced neither — a divergence from TS/Java/C#. meta_dir = _meta_dir_with(tmp_path, _META_PROMPT_TAGS) troot = _templates_dir(tmp_path, "Hello {{name}}") # no … tag - rc = main(["verify", "--templates", meta_dir, "--templates-root", troot]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", troot]) assert rc != 0 err = capsys.readouterr().err assert "ERR_OUTPUT_TAG_MISSING" in err @@ -244,7 +245,7 @@ def test_templates_prompt_required_tag_missing_is_drift(tmp_path: Path, capsys) def test_templates_prompt_required_tag_present_passes(tmp_path: Path) -> None: meta_dir = _meta_dir_with(tmp_path, _META_PROMPT_TAGS) troot = _templates_dir(tmp_path, "Hello {{name}} {{name}}") - rc = main(["verify", "--templates", meta_dir, "--templates-root", troot]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", meta_dir, "--templates-root", troot]) assert rc == 0 @@ -254,8 +255,8 @@ def test_templates_prompt_required_tag_present_passes(tmp_path: Path) -> None: def test_bare_verify_is_codegen_backcompat(tmp_path: Path, capsys) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 - rc = main(["verify", meta_dir, "--out", str(out)]) + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 + rc = main(["verify", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc == 0 note = capsys.readouterr().err + capsys.readouterr().out # A one-line note advertising the explicit subverbs is printed. @@ -265,9 +266,9 @@ def test_bare_verify_is_codegen_backcompat(tmp_path: Path, capsys) -> None: def test_bare_verify_detects_codegen_drift(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 (out / "Program.py").unlink() - assert main(["verify", meta_dir, "--out", str(out)]) != 0 + assert main(["verify", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) != 0 # --- 4. --db is rejected in the Python port (exit 2) ------------------------ @@ -275,7 +276,7 @@ def test_bare_verify_detects_codegen_drift(tmp_path: Path) -> None: def test_db_is_rejected_exit_2(tmp_path: Path, capsys) -> None: meta_dir = _meta_dir(tmp_path) - rc = main(["verify", "--db", "postgres://x", meta_dir]) + rc = main(["verify", "--generators", GEN_SUITE, "--db", "postgres://x", meta_dir]) assert rc == 2 err = capsys.readouterr().err assert "not supported" in err.lower() @@ -289,7 +290,7 @@ def test_invalid_flag_exit_2() -> None: import pytest with pytest.raises(SystemExit) as exc: - main(["verify", "--bogus", "x"]) + main(["verify", "--generators", GEN_SUITE, "--bogus", "x"]) assert exc.value.code == 2 @@ -299,13 +300,13 @@ def test_invalid_flag_exit_2() -> None: def test_combined_codegen_and_templates_aggregates_exit(tmp_path: Path) -> None: meta_dir = _meta_dir_with(tmp_path, _META_CLEAN) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out)]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) == 0 troot = _templates_dir(tmp_path, "Hello {{missing}}") # codegen clean, templates drift → aggregate non-zero. rc = main( [ "verify", - "--codegen", + "--codegen", "--generators", GEN_SUITE, "--templates", meta_dir, "--out", @@ -420,5 +421,5 @@ def test_templates_payload_ref_fqn_collision_binds_correct_package_not_load_firs # first (never the FQN's actual target unless it happened to be first), # whose field tree lacks `target_field` — spurious ERR_VAR_NOT_ON_PAYLOAD # drift. The fix must report CLEAN (exit 0) regardless of load order. - rc = main(["verify", "--templates", str(d), "--templates-root", str(troot)]) + rc = main(["verify", "--generators", GEN_SUITE, "--templates", str(d), "--templates-root", str(troot)]) assert rc == 0 diff --git a/server/python/tests/codegen/test_gen_state_keying.py b/server/python/tests/codegen/test_gen_state_keying.py index 1ad6e41aa..47defff1f 100644 --- a/server/python/tests/codegen/test_gen_state_keying.py +++ b/server/python/tests/codegen/test_gen_state_keying.py @@ -33,6 +33,7 @@ decide_and_write, read_generated_hash, ) +from tests.codegen.gen_suite import GEN_SUITE def _manifest(gen_state: Path) -> dict[str, str]: @@ -167,7 +168,7 @@ def test_a_project_reached_through_a_symlink_keeps_its_jurisdiction( from metaobjects.cli import main monkeypatch.chdir(tmp_path) - assert main(["gen", str(link / "metaobjects"), "--out", str(link / "gen")]) == 0 + assert main(["gen", "--generators", GEN_SUITE, str(link / "metaobjects"), "--out", str(link / "gen")]) == 0 manifest = json.loads( (real / ".metaobjects" / ".gen-state" / ".hashes.json").read_text(encoding="utf-8") @@ -179,4 +180,4 @@ def test_a_project_reached_through_a_symlink_keeps_its_jurisdiction( # And the gate still convicts stale output reached through the link. meta_file.write_text('{"metadata.root": {"package": "fitness", "children": []}}') - assert main(["verify", "--codegen", str(link / "metaobjects"), "--out", str(link / "gen")]) == 1 + assert main(["verify", "--codegen", "--generators", GEN_SUITE, str(link / "metaobjects"), "--out", str(link / "gen")]) == 1 diff --git a/server/python/tests/codegen/test_names_generator.py b/server/python/tests/codegen/test_names_generator.py index a792f3a42..f6f59b557 100644 --- a/server/python/tests/codegen/test_names_generator.py +++ b/server/python/tests/codegen/test_names_generator.py @@ -398,6 +398,7 @@ def test_the_artifact_agrees_with_the_runtime() -> None: / "canonical" / "meta.fitness.json" ) +from tests.codegen.gen_suite import GEN_SUITE def _meta_dir(tmp_path: Path) -> str: @@ -410,7 +411,7 @@ def _meta_dir(tmp_path: Path) -> str: def test_cli_column_naming_flag_reaches_the_generator(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - rc = main(["gen", meta_dir, "--out", str(out), "--column-naming", "snake_case"]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--column-naming", "snake_case"]) assert rc == 0 content = (out / "program_names.py").read_text() # priceCents has no explicit @column -> strategy-derived. @@ -422,7 +423,7 @@ def test_cli_column_naming_flag_reaches_the_generator(tmp_path: Path) -> None: def test_cli_column_naming_defaults_to_literal(tmp_path: Path) -> None: meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - rc = main(["gen", meta_dir, "--out", str(out)]) + rc = main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out)]) assert rc == 0 content = (out / "program_names.py").read_text() assert 'PROGRAM_PRICE_CENTS_COLUMN: Final[str] = "priceCents"' in content @@ -439,11 +440,11 @@ def test_verify_codegen_with_matching_column_naming_is_clean(tmp_path: Path, cap """ meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out), "--column-naming", "snake_case"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--column-naming", "snake_case"]) == 0 capsys.readouterr() # discard `gen`'s own stdout rc = main( - ["verify", meta_dir, "--codegen", "--out", str(out), "--column-naming", "snake_case"] + ["verify", meta_dir, "--codegen", "--generators", GEN_SUITE, "--out", str(out), "--column-naming", "snake_case"] ) captured = capsys.readouterr() assert rc == 0, captured.err @@ -458,11 +459,11 @@ def test_verify_codegen_with_mismatched_column_naming_reports_drift(tmp_path: Pa """ meta_dir = _meta_dir(tmp_path) out = tmp_path / "out" - assert main(["gen", meta_dir, "--out", str(out), "--column-naming", "snake_case"]) == 0 + assert main(["gen", "--generators", GEN_SUITE, meta_dir, "--out", str(out), "--column-naming", "snake_case"]) == 0 capsys.readouterr() rc = main( - ["verify", meta_dir, "--codegen", "--out", str(out), "--column-naming", "literal"] + ["verify", meta_dir, "--codegen", "--generators", GEN_SUITE, "--out", str(out), "--column-naming", "literal"] ) captured = capsys.readouterr() assert rc == 1 diff --git a/server/python/tests/codegen/test_trace_helper_generator.py b/server/python/tests/codegen/test_trace_helper_generator.py index e56abd6a5..f1d2d5062 100644 --- a/server/python/tests/codegen/test_trace_helper_generator.py +++ b/server/python/tests/codegen/test_trace_helper_generator.py @@ -67,10 +67,14 @@ def _value_object(name: str, fields: list[MetaField]) -> MetaObject: def _llm_call_base() -> MetaObject: - """A minimal abstract ``LlmCallBase`` — only the fields ``build_llm_call_row`` - reads via the input matter at codegen time; the abstract flag + name drive the - ``_extends_base`` short-name walk.""" + """A minimal abstract stand-in for the shipped ``metaobjects::ai::LlmCallBase``. + + The PACKAGE is load-bearing now: ``_extends_base`` compares ``resolution_key()`` + against the anchor FQNs the ``ai`` manifest declares (FR-043 §6), not the short + name, so a base built here without it is an adopter's own entity that happens to + share a name — which is exactly what no longer matches, on purpose.""" base = MetaObject(TYPE_OBJECT, "entity", "LlmCallBase") + base.package = "metaobjects::ai" base.is_abstract = True return base @@ -101,7 +105,7 @@ def _prompt( def _greeting_call(base: MetaObject, prompt: MetaTemplate | None = None) -> MetaObject: entity = MetaObject(TYPE_OBJECT, "entity", "GreetingCall") - entity.super_data = base # resolved super chain → _extends_base sees LlmCallBase + entity.super_data = base # resolved super chain → _extends_base sees the anchor FQN source = MetaSource(TYPE_SOURCE, SOURCE_SUBTYPE_RDB, "primary") source.set_attr("table", "llm_call") @@ -337,3 +341,42 @@ def test_generated_record_reports_error_on_lost_required(tmp_path, monkeypatch) assert result.status == "error" assert result.error_detail is not None assert "greeting" in result.error_detail + + +# --------------------------------------------------------------------------- +# FR-043 §6 — the anchor comes from the manifest, not from a constant here +# --------------------------------------------------------------------------- + + +def test_the_anchor_comes_from_the_shipped_manifest() -> None: + """The floor the design names for a port: the port's constant equals the manifest + anchor. Here the constant IS the manifest read, so this asserts the read produced + what the library declares — an emptied or renamed ``generators`` block would + otherwise leave the generator matching nothing, in silence.""" + from metaobjects.codegen.generators.trace_helper_generator import ANCHOR_FQNS + from metaobjects.library.library_sources import LIBRARY_MANIFESTS + + declared = { + gen["anchor"] + for manifest in LIBRARY_MANIFESTS.values() + for gen in manifest.get("generators", []) + if gen.get("name") == "trace-helper" and gen.get("anchor") + } + assert ANCHOR_FQNS == frozenset(declared) + assert "metaobjects::ai::LlmCallBase" in ANCHOR_FQNS + + +def test_an_adopters_own_base_of_the_same_name_does_not_match() -> None: + """The latent bug the anchor removes: the old walk compared the SHORT name, so any + entity called ``LlmCallBase`` in any package emitted a helper writing the shipped + base's columns — which that entity does not declare.""" + impostor = MetaObject(TYPE_OBJECT, "entity", "LlmCallBase") + impostor.package = "acme::app" + impostor.is_abstract = True + + entity = _greeting_call(impostor) + root = MetaRoot(TYPE_METADATA, SUBTYPE_ROOT, "test") + for child in (impostor, entity, _greeting_response()): + root.add_child(child) + + assert render_trace_helper(entity, root) is None diff --git a/server/python/tests/config/test_library_package_guard.py b/server/python/tests/config/test_library_package_guard.py new file mode 100644 index 000000000..e16be6ec9 --- /dev/null +++ b/server/python/tests/config/test_library_package_guard.py @@ -0,0 +1,112 @@ +"""FR-043 §3.4 / §3.5 — an adopter's own file in a shipped library's package. + +Mirrors the TypeScript ``sdk/test/library-package-guard.test.ts``. Two failures, +opposite in shape and both silent before this guard: the ejected copy still named in +``libraries`` (additions take, deletions do not) and a new node declared into a package +the library owns. The ``overlay: true`` door stays open — ``is_merge`` is what tells an +amendment from a copy, a distinction no comparison of the merged trees could make. +""" + +from __future__ import annotations + +import pytest + +from metaobjects import MetaDataLoader +from metaobjects.config.dependencies import refuse_library_package_misuse +from metaobjects.errors import ErrorCode, ParseError +from metaobjects.library import library_sources +from metaobjects.loader.sources.meta_data_source import ( + InMemoryStringSource, + MetaDataFormat, +) + +COPY_OF_A_SHIPPED_NODE = """ +metadata: + package: metaobjects::iam + children: + - object.entity: + name: User + children: + - field.string: { name: nickname } +""" + +OVERLAY_OF_A_SHIPPED_NODE = """ +metadata: + package: metaobjects::iam + children: + - object.entity: + name: User + overlay: true + children: + - field.string: { name: nickname } +""" + +A_NEW_NODE_IN_THE_LIBRARYS_PACKAGE = """ +metadata: + package: metaobjects::iam + children: + - object.entity: + name: ApiKey + children: + - field.uuid: { name: id } + - identity.primary: { name: pk, fields: [id] } +""" + +MY_OWN_PACKAGE = """ +metadata: + package: acme::app + children: + - object.entity: + name: Account + extends: metaobjects::iam::User + children: + - identity.primary: { name: pk, fields: [id] } +""" + + +def _load(yaml: str, libraries: list[str] | None = None): + selection = ["iam"] if libraries is None else libraries + sources = [*library_sources(selection), InMemoryStringSource(yaml, "mine.yaml", MetaDataFormat.YAML)] + result = MetaDataLoader(strict=True).load(sources) + assert not result.errors, [str(e) for e in result.errors] + refuse_library_package_misuse(result.root, selection) + return result.root + + +def test_an_ejected_copy_that_is_still_opted_in_is_refused() -> None: + with pytest.raises(ParseError) as caught: + _load(COPY_OF_A_SHIPPED_NODE) + assert caught.value.code is ErrorCode.ERR_LIBRARY_PACKAGE_COLLISION + # The message must say WHY silence would be worse: the merge is asymmetric. + assert "DELETIONS" in str(caught.value) + assert "metaobjects::iam::User" in str(caught.value) + + +def test_a_new_node_in_the_librarys_package_is_refused() -> None: + with pytest.raises(ParseError) as caught: + _load(A_NEW_NODE_IN_THE_LIBRARYS_PACKAGE) + assert caught.value.code is ErrorCode.ERR_LIBRARY_PACKAGE_NOT_OWNED + assert "metaobjects::iam::ApiKey" in str(caught.value) + + +def test_an_overlay_amendment_is_the_documented_door() -> None: + assert _load(OVERLAY_OF_A_SHIPPED_NODE) is not None + + +def test_your_own_package_extending_a_library_node_is_untouched() -> None: + assert _load(MY_OWN_PACKAGE) is not None + + +def test_with_the_library_not_opted_in_the_guard_says_nothing() -> None: + """Which is the state ``meta eject`` leaves you in once the library is removed from + ``libraries`` — refusing it there would make the ejection door unusable.""" + sources = [InMemoryStringSource(COPY_OF_A_SHIPPED_NODE, "mine.yaml", MetaDataFormat.YAML)] + result = MetaDataLoader(strict=True).load(sources) + refuse_library_package_misuse(result.root, []) + + +def test_the_librarys_own_layers_do_not_trip_it() -> None: + """``iam/db`` is nothing but ``overlay: true`` redeclarations of ``iam``'s own nodes, + from library files. A guard keyed on "two files contributed" would fire on each.""" + root = _load(MY_OWN_PACKAGE, ["iam", "iam/db"]) + assert any(c.name == "User" for c in root.own_children()) diff --git a/server/python/tests/conformance/test_generator_registry_conformance.py b/server/python/tests/conformance/test_generator_registry_conformance.py index 34056633b..4b60bdb2d 100644 --- a/server/python/tests/conformance/test_generator_registry_conformance.py +++ b/server/python/tests/conformance/test_generator_registry_conformance.py @@ -4,7 +4,7 @@ single cross-port source of truth for generator stable names. This test asserts that the Python ``GENERATOR_REGISTRY`` exposes EXACTLY the set of stable names whose manifest ``ports`` array includes ``python`` — no extras, none missing — -and that every Python entry's tier agrees with the manifest. +and that every Python entry's tier AND layer agree with the manifest. If this fails, the registry and the manifest disagree. The manifest is canonical: do NOT edit it to make this pass — fix the Python registry (or report the diff). @@ -75,6 +75,36 @@ def test_registry_tiers_agree_with_manifest() -> None: assert not mismatches, "tier disagreement vs manifest:\n " + "\n ".join(mismatches) +def test_registry_layers_agree_with_manifest() -> None: + py_slice = _manifest_python_slice() + mismatches = [] + for name, spec in py_slice.items(): + if name not in GENERATOR_REGISTRY: + continue # name-set test reports this + expected_layer = spec["layer"] + actual_layer = GENERATOR_REGISTRY[name].layer + if actual_layer != expected_layer: + mismatches.append( + f"{name}: registry layer={actual_layer!r} != manifest layer={expected_layer!r}" + ) + assert not mismatches, "layer disagreement vs manifest:\n " + "\n ".join(mismatches) + + +def test_every_manifest_entry_declares_one_of_the_six_layers() -> None: + # The closed set, spelled out rather than imported from the code: importing the + # port's own tuple would make the gate agree with whatever the code says. + allowed = {"model", "persistence", "api", "client", "docs", "capability"} + manifest = json.loads(_find_manifest().read_text()) + bad = sorted( + f"{name}={spec.get('layer')!r}" + for name, spec in manifest["generators"].items() + if spec.get("layer") not in allowed + ) + assert not bad, ( + f"manifest entries with a missing or unknown layer (allowed: {sorted(allowed)}): {bad}" + ) + + def test_registry_python_slice_is_all_native() -> None: # The Python slice of the manifest is entirely tier: native. assert all(e.tier == "native" for e in GENERATOR_REGISTRY.values()), ( diff --git a/server/python/tests/integration/test_llm_call_trace.py b/server/python/tests/integration/test_llm_call_trace.py index 43cf67fe5..76de446d0 100644 --- a/server/python/tests/integration/test_llm_call_trace.py +++ b/server/python/tests/integration/test_llm_call_trace.py @@ -2,7 +2,7 @@ Proves a typed LLM-call trace persists + reads back through the ObjectManager runtime against a live Postgres: a trace entity extending the SHIPPED -``metaobjects::ai::LlmCallBase`` (library/ai/llm-call.yaml) plus a typed +``metaobjects::ai::LlmCallBase`` (library/ai/model.yaml) plus a typed ``voResponse`` (field.object + @objectRef + @storage:jsonb). Asserts BOTH the raw envelope (the 18 base fields, raw llmRequest/llmResponse jsonb) AND the typed voResponse jsonb round-trip. @@ -76,7 +76,8 @@ def test_typed_trace_round_trips_through_postgres() -> None: # through the metadata-driven ObjectManager runtime. loader = MetaDataLoader(strict=True) result = loader.load([ - FileSource(_repo_file("library/ai/llm-call.yaml")), + FileSource(_repo_file("library/ai/model.yaml")), + FileSource(_repo_file("library/ai/db.yaml")), FileSource(Path(__file__).parent / "meta_ai_trace.yaml"), ]) assert not result.errors, f"metadata failed to load: {result.errors}" diff --git a/server/python/tests/loader/test_shipped_library_ai.py b/server/python/tests/loader/test_shipped_library_ai.py index 061449e19..9282f40a9 100644 --- a/server/python/tests/loader/test_shipped_library_ai.py +++ b/server/python/tests/loader/test_shipped_library_ai.py @@ -21,6 +21,8 @@ from metaobjects import LoadResult, MetaDataLoader, load_directory from metaobjects.errors import ErrorCode from metaobjects.library import library_sources +from metaobjects.loader.sources import FileSource +from metaobjects.library.library_sources import known_tokens from metaobjects.library.embedded_library import EMBEDDED_LIBRARY from metaobjects.runtime import LlmCallInput from metaobjects.meta.core.object.meta_object import MetaObject @@ -152,10 +154,19 @@ def test_a_checkout_serves_the_canonical_file_not_the_embed(self) -> None: if _repo_root() is None: pytest.skip("no repo-root library/ (installed layout)") - sources = library_sources(["ai"]) + # Every token, so the count covers the whole embed. `["ai"]` alone is the CORE + # layer only since FR-043 Amendment 1 — a bare name no longer means "every ref + # under this library", which is exactly the behaviour change this file's + # sibling tests pin. + sources = library_sources(known_tokens()) assert len(sources) == len(EMBEDDED_LIBRARY) - assert all("library:" not in s.id for s in sources), "expected on-disk FileSource in a checkout" + # The SOURCE KIND, not its id: since FR-043 a library file carries the same + # stable `library:.yaml` id in both branches, so that a library node's + # error envelope reads identically from a checkout and from an installed + # wheel. Which branch served it is a question about the object, not its label. + assert all(isinstance(s, FileSource) for s in sources), "expected on-disk FileSource in a checkout" + assert all(s.id.startswith("library:") for s in sources), "and the stable id in both branches" def test_the_loader_prepends_library_sources(adopter_dir: Path) -> None: diff --git a/server/python/tests/unit/test_llm_recorder.py b/server/python/tests/unit/test_llm_recorder.py index 87b5cb42c..c21c1d7ca 100644 --- a/server/python/tests/unit/test_llm_recorder.py +++ b/server/python/tests/unit/test_llm_recorder.py @@ -3,7 +3,7 @@ No database required. Covers: - the CONTRACT gate: ``build_llm_call_row`` writes exactly the field set the SHIPPED ``metaobjects::ai::LlmCallBase`` declares (loaded from - ``library/ai/llm-call.yaml``) — a divergence becomes a build failure; + ``library/ai/model.yaml``) — a divergence becomes a build failure; - ``NullLlmCallRecorder`` is a no-op; - ``ObjectManagerLlmCallRecorder`` NEVER raises on a write failure and routes the error to ``on_error`` (telemetry must not break the app). @@ -54,7 +54,10 @@ def _sample_input() -> LlmCallInput: def test_build_row_matches_shipped_llm_call_base_field_set() -> None: """The base row's keys == the shipped LlmCallBase concrete entity's fields.""" loader = MetaDataLoader(strict=True) - result = loader.load([FileSource(_repo_file("library/ai/llm-call.yaml"))]) + result = loader.load([ + FileSource(_repo_file("library/ai/model.yaml")), + FileSource(_repo_file("library/ai/db.yaml")), + ]) assert not result.errors, f"shipped library failed to load: {result.errors}" llm_call = None @@ -62,7 +65,7 @@ def test_build_row_matches_shipped_llm_call_base_field_set() -> None: if obj.name.endswith("LlmCall") and not obj.name.endswith("LlmCallBase"): llm_call = obj break - assert llm_call is not None, "concrete LlmCall entity not found in library/ai/llm-call.yaml" + assert llm_call is not None, "concrete LlmCall entity not found in library/ai/{model,db}.yaml" shipped_fields = {f.name for f in llm_call.fields()} row_keys = set(build_llm_call_row(_sample_input()).keys()) diff --git a/server/typescript/packages/cli/README.md b/server/typescript/packages/cli/README.md index 4e0005e0d..9415606b0 100644 --- a/server/typescript/packages/cli/README.md +++ b/server/typescript/packages/cli/README.md @@ -57,12 +57,20 @@ Run schema ops from the compiled binary: ```bash # 1. Scaffold metaobjects/ + .metaobjects/ + codegen/generators/ + metaobjects.config.ts +# Codegen is OPT-IN: the scaffolded selection is empty. meta init # 2. Author entity metadata $EDITOR metaobjects/meta.myapp.json # see .metaobjects/AGENTS.md for format -# 3. Generate TS code (config-driven via metaobjects.config.ts) +# 3. Choose your generators. --probe runs each one against YOUR model and reports +# how many files it would emit, so you can see what your metadata already asks for. +meta gen --list --probe + +# 4. Take the ones you want. Prints the import, the entry to wire, and what to install. +meta eject entity queries routes barrel + +# 5. Generate TS code (config-driven via metaobjects.config.ts) meta gen # 4. Diff metadata against your DB and emit migration SQL @@ -86,9 +94,9 @@ Running `meta` with no arguments prints a concise status line (whether a `metaob ### `meta init` -Scaffolds `metaobjects/` (visible entity declarations, with a placeholder `meta.common.json`), `.metaobjects/` (hidden tool state: `config.json`, `AGENTS.md`, `CLAUDE.md`, `.gitignore`, `.gen-state/`), the **owned codegen generators** at `codegen/generators/{entity,queries,routes,barrel}.ts`, and `metaobjects.config.ts` at the repo root. +Scaffolds `metaobjects/` (visible entity declarations, with a placeholder `meta.common.json`), `.metaobjects/` (hidden tool state: `config.json`, `AGENTS.md`, `CLAUDE.md`, `.gitignore`, `.gen-state/`), an **empty** `codegen/generators/` plus its `tsconfig.codegen.json`, and `metaobjects.config.ts` at the repo root. -The generators are copied from the codegen reference templates and are **yours to edit** (ADR-0034 scaffold-and-own); the scaffolded `metaobjects.config.ts` imports them locally, and `meta gen` runs from those local copies — not from the package. Each generator file is written only if absent, so re-running with `--force` never clobbers a hand-edited generator. +**It copies no generators and declares no dependencies.** Codegen is opt-in (ADR-0034 Amendment 2): the scaffolded config carries `generators: []` and a comment pointing at the catalog. `meta gen --list --probe` shows what each generator would emit for your model; `meta eject ...` copies the ones you choose into `codegen/generators/` — **yours to edit** — and prints the import line, the entry to add, the install command and any config keys those generators read. `meta gen` runs your local copies, never the package's. Flags: - `--force` — overwrite scaffold files (memory records preserved) @@ -161,7 +169,7 @@ Flags: Two config files, by design: -**`metaobjects.config.ts`** (at repo root) — generator wiring and codegen knobs, type-checked TS. The generators are imported from the **owned local copies** that `meta init` scaffolded into `codegen/generators/` (ADR-0034 scaffold-and-own), not from the package: +**`metaobjects.config.ts`** (at repo root) — generator wiring and codegen knobs, type-checked TS. It starts with `generators: []`; each import below appears when you `meta eject` that generator, which copies it into `codegen/generators/` (ADR-0034 scaffold-and-own) and prints the line to add: ```ts import { defineConfig } from "@metaobjectsdev/cli"; @@ -193,7 +201,7 @@ concern — declare named **targets** and point generators at them with `target` ```ts import { defineConfig } from "@metaobjectsdev/cli"; -// Owned generators scaffolded by `meta init` (ADR-0034 scaffold-and-own). +// Owned generators — copied in by `meta eject` (ADR-0034 scaffold-and-own). import { entityFile } from "./codegen/generators/entity"; import { queriesFile } from "./codegen/generators/queries"; import { routesFile } from "./codegen/generators/routes"; diff --git a/server/typescript/packages/cli/src/commands/eject.ts b/server/typescript/packages/cli/src/commands/eject.ts index 1994c2d11..6642fd6aa 100644 --- a/server/typescript/packages/cli/src/commands/eject.ts +++ b/server/typescript/packages/cli/src/commands/eject.ts @@ -1,8 +1,19 @@ -// FR-040 §4.2(a) — `meta eject ` takes ownership of any reference-template -// generator, in any package, at any time after `meta init`. ADR-0034 scaffold-and-own -// has `init` copy five of them eagerly (entity, queries, routes, barrel, names); this is the -// SAME copy operation, generalised to every ejectable name and callable on demand — for -// a generator you skipped at init time, or one a package gained since. +// FR-040 §4.2(a) — `meta eject ...` takes ownership of any reference-template +// generator, in any package, at any time after `meta init`. +// +// Under opt-in codegen this is THE copy door: `meta init` scaffolds the layout and an +// empty selection, so every generator an adopter runs arrives through here. It takes +// MANY names, because a real selection is several — choosing a client tier is +// `form hooks grid` — and three separate invocations produce three separate install +// lines for the same package. The consolidated install set is the point. +// +// It REPORTS; it never edits `metaobjects.config.ts` or `package.json`. ADR-0034 §3(c) +// already rules out a parameterized add/configure surface; the config is user-owned +// TypeScript with comments, and automated mutation of a real user config is the +// persistently fragile step even for well-resourced teams (Nuxt's `nuxi module add` +// config-array edit has regressed across at least four filed issues). The primary +// consumer performs two file edits and one install trivially when told exactly what +// they are. `meta gen` plus `tsc` is the audit. import { mkdir, writeFile, stat, readFile } from "node:fs/promises"; import { join } from "node:path"; import { cliVersion } from "../lib/version.js"; @@ -13,6 +24,13 @@ import { parseEjectArgs } from "../lib/args.js"; import { log } from "../lib/log.js"; import { declaredDependencyNames, readPackageManifest } from "../lib/package-manifest.js"; import { compareOwnedCopy, type OwnedComparison } from "../lib/owned-copy.js"; +import { composeCatalog } from "../lib/catalog.js"; +import { installSetFor, type InstallSet } from "../lib/install-set.js"; +import { emitStructured, type OutputFormat } from "../lib/format.js"; +import { + ejectLibrary, ejectableLibraryNames, isLibraryName, libraryStaleness, + type LibraryEjectResult, +} from "../lib/library-eject.js"; // Mirrors `OWNED_GENERATORS_DIR` in init.ts's `writeOwnedGenerators` — same directory, // same never-clobber-without-consent contract. Kept as its own local constant rather @@ -291,11 +309,209 @@ async function listOutput(cwd: string): Promise { ); lines.push(""); } + // FR-043 — the shipped libraries, and how far this project's copies have drifted. + lines.push("Shipped libraries (copy one into your own sources and own it):"); + for (const name of ejectableLibraryNames()) lines.push(` ${name}`); + lines.push(""); + const staleness = await libraryStaleness(cwd); + if (staleness.length > 0) { + lines.push("Libraries this project has already ejected:"); + for (const row of staleness) { + if (row.verdict === "unreadable") { + lines.push(` ${row.library} [ejected — NOT COMPARED: ${row.reason}]`); + } else if (row.verdict === "identical") { + lines.push(` ${row.library} [ejected — identical to the shipped library]`); + } else { + lines.push( + ` ${row.library} [ejected — DIFFERS: ${row.changed} node(s) changed, ` + + `${row.upstreamOnly} only upstream, ${row.localOnly} only yours]`, + ); + } + lines.push(` ${row.files.join(", ")}`); + if (row.stillOptedIn) { + lines.push( + ` STILL OPTED IN — remove "${row.library}" from \`libraries\`; the shipped ` + + "tree and your copy both load, and deletions in yours do nothing.", + ); + } + } + lines.push(""); + lines.push( + " Compared through the canonical serializer in OWN mode, so re-indentation and " + + "key order never show up — only a declaration that actually changed. Nodes are " + + "matched by NAME, so renaming the package (which you may) does not read as drift.", + ); + lines.push( + " \"only upstream\" — the library gained or you deleted. \"only yours\" — you added.", + ); + lines.push(""); + } + lines.push("Run: meta eject "); return lines.join("\n"); } -export async function ejectCommand(args: string[], cwd: string): Promise { +// --------------------------------------------------------------------------- +// the command +// --------------------------------------------------------------------------- + +/** One row of the `--format json` payload. */ +interface EjectedRow { + name: string; + path: string; + status: EjectResult["status"]; + /** The two edits an adopter makes in `metaobjects.config.ts`. */ + wire: { import: string; entry: string }; + requires: readonly string[]; +} + +interface EjectPayload { + ejected: EjectedRow[]; + install: InstallSet; + /** Config keys the ejected generators read — what to set beside `generators`. */ + config: { keys: string[] }; + /** FR-043 — libraries ejected in this call. Absent when none were. */ + libraries?: LibraryEjectResult[]; +} + +/** The text report for one ejected library. */ +function reportLibrary(result: LibraryEjectResult): void { + const created = result.files.filter((f) => f.status !== "preserved"); + const kept = result.files.filter((f) => f.status === "preserved"); + if (created.length > 0) { + log.info( + `Ejected library "${result.library}" -> ${result.root}:`, + ); + for (const f of created) log.info(` ${f.path} [${f.status}]`); + } + if (kept.length > 0) { + // Never clobber without consent — the same contract the generator path keeps, and + // the stakes are higher here: these files are the adopter's MODEL. + for (const f of kept) { + log.info(` ${f.path} already exists — left untouched (--force replaces it).`); + } + } + log.info( + `You own this metadata now (FR-043 §3.4): rename the package, delete what you do ` + + `not need, change anything. Nothing regenerates it.`, + ); + if (result.stillOptedIn.length > 0) { + // The one step that makes the eject complete. Said here AND stamped in every file, + // because the failure it prevents is silent in a way an adopter cannot diagnose. + log.info( + `NEXT: remove ${result.stillOptedIn.map((t) => `"${t}"`).join(", ")} from ` + + `\`libraries\` in .metaobjects/config.json. Left there, the shipped tree and your ` + + `copy BOTH load: additions take effect and deletions do NOT, because the library ` + + `still declares what you removed. The loader refuses it outright ` + + `(ERR_LIBRARY_PACKAGE_COLLISION) — your next command will fail until you do.`, + ); + } + log.info("`meta eject --list` reports how far your copy has drifted from the shipped one."); +} + +/** + * Validate EVERY name before writing ANY file. + * + * A partial eject is the worst outcome available here: a non-zero exit over a repo + * that is half-changed, where re-running the fixed command then reports the + * already-copied half as "preserved" and the adopter cannot tell what happened. + * Returns the unknown names, or an empty array. + */ +function unknownNames(names: readonly string[]): string[] { + return names.filter((n) => resolveSource(n) === undefined && !isLibraryName(n)); +} + +/** Print the per-name text report — every branch below predates this command taking + * more than one name, and each was written against a real incident. */ +function reportOne(result: EjectResult, name: string): void { + if (result.status === "preserved") { + // "already exists — left untouched" was the whole message, and it answered the + // question nobody has. What an owner needs to know is whether their copy still + // matches what this CLI ships — the only way an owned generator's staleness is + // ever observable, since no gate compares the two. + if (result.comparison?.verdict === "identical") { + log.info( + `${result.path} already exists and is IDENTICAL to the ${result.packageName} ` + + "reference template — nothing to do.", + ); + } else if (result.comparison?.verdict === "reformatted") { + // Worth its own branch: this is the state a project that formats what it owns + // is in permanently, and calling it DIFFERS taught every one of them to ignore + // the line that is supposed to warn them. + log.info( + `${result.path} already exists and has the SAME CONTENT as the ` + + `${result.packageName} reference template, in your own formatting — ` + + "nothing to do.", + ); + } else { + log.info( + `${result.path} already exists and DIFFERS from the ${result.packageName} ` + + `reference template (${result.comparison?.referenceOnly ?? 0} line(s) behind it, ` + + `${result.comparison?.localOnly ?? 0} line(s) of your own) — left untouched.`, + ); + log.info( + " Formatting is not counted: both files are re-formatted and their lines " + + "sorted before comparing, so re-wrapping and import order never show up.", + ); + log.info( + " That difference is either YOUR customization or upstream having moved on. " + + "See which, before deciding:", + ); + log.info(` diff -u node_modules/${result.packageName}/src/reference/${name}.ts ${result.path}`); + log.info( + " To take upstream changes AND keep your customization, three-way merge them — " + + "`git merge-file --diff3 `. " + + "`--force` does NOT merge: it replaces the file and your customization with it.", + ); + } + } else if (result.status === "replaced") { + // Never let a --force over a modified file be silent: this is the step that + // destroys an adopter's customization, and the file's own header is often the + // only record that the customization was deliberate. + log.info( + `Ejected "${name}" -> ${result.path}, REPLACING the file that was there` + + (result.comparison?.verdict === "differs" + ? ` and DISCARDING ${result.comparison.localOnly} line(s) it had that the reference does not.` + : result.comparison?.verdict === "reformatted" + ? " (its content was already the reference's — only your formatting is gone)." + : " (it was already identical to the reference)."), + ); + } else { + log.info(`Ejected "${name}" -> ${result.path}. You own it now (ADR-0034 scaffold-and-own).`); + } + + // REPLACE, never "paste". A generator reaches `generators: [...]` under ONE binding, + // so a reader told to "paste" gets a duplicate identifier at best — and at worst + // deletes nothing, keeps `formFile()` in the array bound to the PACKAGE import, and + // silently runs the packaged generator while editing the ejected file. That failure + // is invisible and is the exact one ejecting exists to prevent. + // + // But eject reads no config, so it cannot know WHICH of the three states this project + // is in, and stating one of them as fact is wrong in the other two. Name the goal, + // then the three branches; the reader knows which one they are looking at. + log.info(`In metaobjects.config.ts, "${result.exportName}" must resolve to this file:`); + log.info(` ${result.importLine}`); + log.info( + ` - If it is imported from "${result.packageName}", REPLACE that import with the ` + + "line above. Adding a second one leaves `generators` bound to the PACKAGED " + + "generator, and your edits to this file do nothing.", + ); + log.info( + " - If it is already imported from ./codegen/generators/, it points here already " + + "— nothing to change.", + ); + log.info( + ` - If ${result.exportName}() is not in \`generators\` yet, add the import above ` + + "AND the entry.", + ); + for (const line of result.dependencyNotes) log.info(line); +} + +export async function ejectCommand( + args: string[], + cwd: string, + fmt: OutputFormat = "text", +): Promise { let flags; try { flags = parseEjectArgs(args); @@ -309,98 +525,96 @@ export async function ejectCommand(args: string[], cwd: string): Promise return 0; } - if (flags.name === undefined) { - log.error("meta eject requires a generator name, or --list to see what's ejectable."); + if (flags.names.length === 0) { + log.error( + "meta eject requires at least one generator name, or --list to see what's ejectable. " + + "`meta gen --list --format json --probe` is the catalog, with a file count per " + + "generator for your own model.", + ); + return 2; + } + + // All-or-nothing on the names, BEFORE any write — see unknownNames(). + const unknown = unknownNames(flags.names); + if (unknown.length > 0) { + log.error( + `unknown name(s): ${unknown.join(", ")}. Nothing was ejected. ` + + `Ejectable generators: ${ejectableNames().join(", ")}. ` + + `Shipped libraries: ${ejectableLibraryNames().join(", ")}. ` + + "Run `meta eject --list` to see them grouped.", + ); return 2; } + const catalog = composeCatalog(); + const rows: EjectedRow[] = []; + const libraries: LibraryEjectResult[] = []; try { - const result = await ejectGenerator({ cwd, name: flags.name, force: flags.force }); - if (result.status === "preserved") { - // "already exists — left untouched" was the whole message, and it answered the - // question nobody has. What an owner needs to know is whether their copy still - // matches what this CLI ships — the only way an owned generator's staleness is - // ever observable, since no gate compares the two. - if (result.comparison?.verdict === "identical") { - log.info( - `${result.path} already exists and is IDENTICAL to the ${result.packageName} ` + - "reference template — nothing to do.", - ); - } else if (result.comparison?.verdict === "reformatted") { - // Worth its own branch: this is the state a project that formats what it owns - // is in permanently, and calling it DIFFERS taught every one of them to ignore - // the line that is supposed to warn them. - log.info( - `${result.path} already exists and has the SAME CONTENT as the ` + - `${result.packageName} reference template, in your own formatting — ` + - "nothing to do.", - ); - } else { - log.info( - `${result.path} already exists and DIFFERS from the ${result.packageName} ` + - `reference template (${result.comparison?.referenceOnly ?? 0} line(s) behind it, ` + - `${result.comparison?.localOnly ?? 0} line(s) of your own) — left untouched.`, - ); - log.info( - " Formatting is not counted: both files are re-formatted and their lines " + - "sorted before comparing, so re-wrapping and import order never show up.", - ); - log.info( - " That difference is either YOUR customization or upstream having moved on. " + - "See which, before deciding:", - ); - log.info(` diff -u node_modules/${result.packageName}/src/reference/${flags.name}.ts ${result.path}`); - log.info( - " To take upstream changes AND keep your customization, three-way merge them — " + - "`git merge-file --diff3 `. " + - "`--force` does NOT merge: it replaces the file and your customization with it.", - ); + for (const name of flags.names) { + if (isLibraryName(name)) { + const lib = await ejectLibrary({ cwd, name, force: flags.force }); + libraries.push(lib); + if (fmt === "text") reportLibrary(lib); + continue; } - } else if (result.status === "replaced") { - // Never let a --force over a modified file be silent: this is the step that - // destroys an adopter's customization, and the file's own header is often the - // only record that the customization was deliberate. - log.info( - `Ejected "${flags.name}" -> ${result.path}, REPLACING the file that was there` + - (result.comparison?.verdict === "differs" - ? ` and DISCARDING ${result.comparison.localOnly} line(s) it had that the reference does not.` - : result.comparison?.verdict === "reformatted" - ? " (its content was already the reference's — only your formatting is gone)." - : " (it was already identical to the reference)."), - ); - } else { - log.info(`Ejected "${flags.name}" -> ${result.path}. You own it now (ADR-0034 scaffold-and-own).`); + const result = await ejectGenerator({ cwd, name, force: flags.force }); + rows.push({ + name, + path: result.path, + status: result.status, + wire: { import: result.importLine, entry: `${result.exportName}()` }, + requires: catalog[name]?.requires ?? [], + }); + if (fmt === "text") reportOne(result, name); } - // REPLACE, never "paste". A generator reaches `generators: [...]` under ONE binding, - // so a reader told to "paste" gets a duplicate identifier at best — and at worst - // deletes nothing, keeps `formFile()` in the array bound to the PACKAGE import, and - // silently runs the packaged generator while editing the ejected file. That failure - // is invisible and is the exact one ejecting exists to prevent. - // - // But eject reads no config, so it cannot know WHICH of the three states this project - // is in, and stating one of them as fact is wrong in the other two — including for the - // five `meta init` scaffolds, whose config already imports from ./codegen/generators/, - // which is precisely the `meta eject --force` re-sync case. Name the goal, then - // the three branches; the reader knows which one they are looking at. - log.info(`In metaobjects.config.ts, "${result.exportName}" must resolve to this file:`); - log.info(` ${result.importLine}`); - log.info( - ` - If it is imported from "${result.packageName}", REPLACE that import with the ` + - "line above. Adding a second one leaves `generators` bound to the PACKAGED " + - "generator, and your edits to this file do nothing.", - ); - log.info( - " - If it is already imported from ./codegen/generators/ (what `meta init` " + - "scaffolds), it points here already — nothing to change.", - ); - log.info( - ` - If ${result.exportName}() is not in \`generators\` yet, add the import above ` + - "AND the entry.", - ); - for (const line of result.dependencyNotes) log.info(line); - return 0; } catch (err) { log.error((err as Error).message); return 1; } + + // ONE install set for the whole call, not one per name: ejecting `hooks` and `grid` + // needs @metaobjectsdev/codegen-ts-tanstack once, and an adopter handed the same + // package twice reasonably wonders which line to run. + const entries = flags.names + .filter((n) => !isLibraryName(n)) + .map((n) => catalog[n]) + .filter((e) => e !== undefined); + const install = installSetFor(entries); + const configKeys = [...new Set(entries.flatMap((e) => e.configKeys ?? []))].sort(); + + if (fmt === "text") { + if (install.command !== "") { + log.info(""); + log.info("Install what the ejected generators and their output need:"); + log.info(` ${install.command}`); + } + if (configKeys.length > 0) { + log.info( + `These generators read config: ${configKeys.join(", ")} — set them in ` + + "metaobjects.config.ts beside `generators`.", + ); + } + // Every requires edge the selection does not itself satisfy. `meta gen` warns + // about this too, but saying it HERE is what stops the adopter wiring a broken + // pair in the first place. + const chosen = new Set(flags.names); + const missing = [...new Set(rows.flatMap((r) => r.requires).filter((d) => !chosen.has(d)))].sort(); + if (missing.length > 0) { + log.info( + `Also needed: ${missing.join(", ")} — the code you just ejected imports modules ` + + `${missing.length === 1 ? "that generator emits" : "those generators emit"}. ` + + `Eject ${missing.length === 1 ? "it" : "them"} too, or keep your own.`, + ); + } + } else { + const payload: EjectPayload = { + ejected: rows, + install, + config: { keys: configKeys }, + ...(libraries.length > 0 ? { libraries } : {}), + }; + emitStructured(payload, fmt); + } + + return 0; } diff --git a/server/typescript/packages/cli/src/commands/gen.ts b/server/typescript/packages/cli/src/commands/gen.ts index f94279b4f..0feed96bc 100644 --- a/server/typescript/packages/cli/src/commands/gen.ts +++ b/server/typescript/packages/cli/src/commands/gen.ts @@ -15,12 +15,18 @@ import { type AdvisoryFindingRow, type AdvisorySection, } from "../lib/advisory.js"; import { loadMemory, resolveCollection } from "@metaobjectsdev/sdk"; -import { runGen, listGenerators } from "@metaobjectsdev/codegen-ts"; +import { runGen } from "@metaobjectsdev/codegen-ts"; import type { WriteStatus } from "@metaobjectsdev/codegen-ts"; import { packageOfResolutionKey } from "@metaobjectsdev/metadata"; import type { MetaRoot } from "@metaobjectsdev/metadata"; import type { Collection } from "@metaobjectsdev/sdk"; import { reportLoadError } from "../lib/load-error.js"; +import { + buildCatalogListing, renderCatalogText, wiredGeneratorNames, ownedGeneratorNames, + declaredDepsOf, +} from "../lib/catalog-listing.js"; +import { emitStructured } from "../lib/format.js"; +import { composeCatalog } from "../lib/catalog.js"; /** * Print a load failure with everything the loader's ADR-0009 envelope carried — the stable @@ -56,10 +62,16 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat try { flags = parseGenArgs(args); } catch (err) { log.error((err as Error).message); return 2; } - // ADR-0021 D3 — `meta gen --list`: print the stable-name generator registry - // and exit 0 WITHOUT running codegen (no config/metadata required). + // ADR-0021 D3 — `meta gen --list`: print the generator CATALOG and exit 0 WITHOUT + // running codegen. No project is required (like `meta types`): "what can this engine + // do" is a question about the installed engine. `--probe` adds "...and what would + // each emit for MY model", which does need one. if (flags.list) { - return listGeneratorsCommand(); + return listCatalogCommand(cwd, fmt, flags.probe); + } + if (flags.probe) { + log.error("--probe is only meaningful with --list. Run: meta gen --list --probe"); + return 2; } const cliConfig = resolveGenConfig(flags); @@ -153,6 +165,9 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat config: forgeConfig, metadata, projectRoot, + // The COMPOSED catalog, so the requires / api-framework gates can see the react + // and tanstack entries too. codegen-ts only knows its own slice. + catalog: composeCatalog(), baseline: flags.baseline, // --dry-run must actually preview. This was previously passed only to the // display object below, so a "preview" run wrote every file. @@ -166,6 +181,10 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat // snapshot artifact), so sharedModelFile() defaults its `files` selection // to exactly what this project itself declares. sourceFiles: genCollection.ownFiles, + // FR-043 §6 — the shipped-library selection. Always passed, including as `[]`: + // that is what tells the post-selection audit "this project opted into none", + // which is a different statement from a programmatic caller that never said. + libraries: genCollection.libraries, ...(cliConfig.entities.length > 0 ? { entityFilter: cliConfig.entities } : {}), }); } catch (err) { @@ -175,6 +194,21 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat for (const w of result.warnings) { log.warn(w); } + // A first-run POINTER, not a warning: an empty selection is the designed state of a + // fresh `meta init`, so calling it a problem would make every new project start with + // one. It says what to do next and disappears the moment anything is wired. + if ((forgeConfig.generators?.length ?? 0) === 0) { + log.info( + "\nNothing is generated until you choose it — `generators: []` is what `meta init` " + + "scaffolds, by design.\n" + + " meta gen --list --probe the catalog, with how many files each generator " + + "would emit for YOUR model\n" + + " meta eject ... take the ones you want; it prints the import, the " + + "entry to wire, and what to install\n" + + "(`meta docs` needs none of this — documentation is on by default.)", + ); + } + // result.files[].path is the absolute full path from decideAndWrite. With // per-target output, show each path relative to the project root so files in // different targets are distinguishable. @@ -311,34 +345,100 @@ function runAntiPatternScan( } /** - * `meta gen --list` — print the stable-name generator registry (ADR-0021 D3). + * `meta gen --list` — the generator catalog (ADR-0021 D3; opt-in-codegen design §D3). * - * Generators are grouped by tier: the recommended native `meta gen` suite - * first, then neutral artifacts (owned by `meta docs` per D1). Each line is - * ` — ` plus an options summary and, for neutral - * entries, a note pointing at the canonical door. Exits 0; no codegen runs. + * Codegen is opt-in, so this is the door: the tool describes what it can do and the + * builder decides what the app needs. Rows are grouped by `layer`, the axis you select + * by; `--format json|toon` emits the same catalog as ONE document (nothing else on + * stdout, per `meta types`' purity rule). + * + * With `--probe`, every catalog generator is constructed and dry-run against this + * project's real model, so the `capability` layer stops being a list of labels and + * becomes `output-parser: 3, callable: 0, requirement-tests: 7` — information that + * cannot go stale, because it does not describe the generators, it runs them. */ -function listGeneratorsCommand(): number { - const entries = listGenerators(); - const native = entries.filter((e) => e.tier === "native"); - const neutral = entries.filter((e) => e.tier === "neutral"); - const width = Math.max(...entries.map((e) => e.name.length)); +async function listCatalogCommand( + cwd: string, + fmt: OutputFormat, + probe: boolean, +): Promise { + let opts: Parameters[0] = {}; - const lines: string[] = []; - lines.push("Available generators (select by stable name):"); - lines.push(""); - lines.push("Native (recommended `meta gen` suite):"); - for (const e of native) { - lines.push(` ${e.name.padEnd(width)} — ${e.description}`); - if (e.options) lines.push(` ${" ".repeat(width)} options: ${e.options}`); - } - lines.push(""); - lines.push("Neutral (owned by `meta docs`; not part of the native suite):"); - for (const e of neutral) { - lines.push(` ${e.name.padEnd(width)} — ${e.description}`); - if (e.note) lines.push(` ${" ".repeat(width)} ${e.note}`); + if (probe) { + // A probe reports what YOUR model would produce, so a project is mandatory here + // even though it is optional for a bare --list. Failing loudly beats a listing of + // zeros that reads like "none of these apply to you". + let collection; + try { + collection = await resolveCollection(cwd); + } catch (err) { + log.error( + `meta gen --list --probe needs a project to probe: ${(err as Error).message}\n` + + "Run `meta gen --list` (no --probe) for the catalog on its own.", + ); + return 2; + } + const projectRoot = resolveGenConfigDir(cwd, collection.configDir); + const genCollection = await resolveGenCollection(collection, projectRoot); + + let forgeConfig; + try { + forgeConfig = await loadMetaobjectsConfig(projectRoot); + } catch (err) { + log.error((err as Error).message); + return 2; + } + + let metadata; + try { + metadata = await loadMemory(genCollection.configDir, { + ...collectionLoadOptions(genCollection), + ...loadMemoryOptionsFrom(forgeConfig), + }); + } catch (err) { + reportLoadError(log, "failed to load metadata", err); + return 2; + } + + opts = { + project: { + projectRoot, + config: forgeConfig, + wiredNames: wiredGeneratorNames(forgeConfig), + ownedNames: ownedGeneratorNames(projectRoot), + declaredDeps: declaredDepsOf(projectRoot), + // FR-043 — the library rows read the selection from the COLLECTION, which is + // where `libraries` lives now (`.metaobjects/config.json`), not from the + // codegen config it was moved out of. + libraries: genCollection.libraries, + }, + probe: { metadata, scope: genCollection.inScope }, + }; + } else { + // No probe: still report `wired` / `owned` when a project happens to be here, since + // both are free. A directory with no config is not an error — the catalog is a fact + // about the installed engine. + try { + const collection = await resolveCollection(cwd); + const projectRoot = resolveGenConfigDir(cwd, collection.configDir); + const forgeConfig = await loadMetaobjectsConfig(projectRoot); + opts = { + project: { + projectRoot, + config: forgeConfig, + wiredNames: wiredGeneratorNames(forgeConfig), + ownedNames: ownedGeneratorNames(projectRoot), + declaredDeps: declaredDepsOf(projectRoot), + libraries: collection.libraries, + }, + }; + } catch { + // No project here — the catalog stands on its own. + } } - log.info(lines.join("\n")); + const rows = await buildCatalogListing(opts); + if (fmt === "text") log.info(renderCatalogText(rows, probe)); + else emitStructured(rows, fmt); return 0; } diff --git a/server/typescript/packages/cli/src/commands/init.ts b/server/typescript/packages/cli/src/commands/init.ts index c623634ed..cc7076354 100644 --- a/server/typescript/packages/cli/src/commands/init.ts +++ b/server/typescript/packages/cli/src/commands/init.ts @@ -2,7 +2,6 @@ import { mkdir, writeFile, readFile, readdir, stat, rm } from "node:fs/promises" import { join } from "node:path"; import { dirname } from "node:path"; import { existsSync as existsSyncWrap, readFileSync as readFileSyncWrap } from "node:fs"; -import { createRequire } from "node:module"; import { DEFAULT_CONFIG, ConfigSchema, saveConfig, PACKAGE_MANIFEST_FILE, DEFAULT_METADATA_DIR, DEFAULT_METAOBJECTS_DIR, DEPS_DIR, LOCK_FILE } from "@metaobjectsdev/sdk"; import { assemble, resolveAgentContextRoot, planScaffold, @@ -13,39 +12,23 @@ import { reportIgnoredScaffold } from "../lib/ignored-scaffold-check.js"; import { parseInitArgs } from "../lib/args.js"; import { log } from "../lib/log.js"; import { cliVersion } from "../lib/version.js"; -import { declaredDependencyNames, type PackageManifest } from "../lib/package-manifest.js"; import { findWranglerConfig, parseWranglerConfig } from "@metaobjectsdev/migrate-ts"; -import { DEFAULT_DOCS_DIR, readReferenceTemplate, type ReferenceGeneratorName } from "@metaobjectsdev/codegen-ts"; +import { DEFAULT_DOCS_DIR } from "@metaobjectsdev/codegen-ts"; // ADR-0034 scaffold-and-own — `meta init` copies the codegen reference templates into // the consumer's repo so they OWN them; metaobjects.config.ts imports them locally. const OWNED_GENERATORS_DIR = "codegen/generators"; -// The FIVE reference generators `meta init` copies EAGERLY — deliberately an explicit -// literal, not derived from @metaobjectsdev/codegen-ts's REFERENCE_GENERATOR_NAMES (the -// full list of everything `meta eject` can copy). Looping over that array unconditionally -// used to mean init scaffolded whatever it contained: when a later task registered -// "routes-hono" there, init started writing an unwired Hono generator nothing in the -// scaffolded config imports, on every fresh project, silently. This constant is the -// scaffolded metaobjects.config.ts's import list (buildMetaobjectsConfigBody below) made -// explicit and checkable — anything else is eject-on-demand via `meta eject `, which -// exists exactly so eager copying isn't the only way to take ownership of a template. + +// The scaffolded config's outDir, as a named constant so the config template and +// anything derived from it cannot drift. // -// "names" joined this list by spec §A5's ruling, not by falling out of the general rule -// above: TypeScript is opt-in by construction under ADR-0034 (meta gen runs the adopter's -// copy, so a packaged change to a generator can never reach anyone who has ejected), so -// the honest maximum for the names artifact is every NEW `meta init` getting it and every -// existing project adding one config line by hand. -export const SCAFFOLDED_GENERATOR_NAMES: readonly ReferenceGeneratorName[] = ["entity", "queries", "routes", "barrel", "names"]; - -// The scaffolded config's outDir + dbImport, as named constants so the throwing-stub -// path below is DERIVED from the same values the config template embeds rather than -// duplicated as a second literal that could drift from it. +// `dbImport` is deliberately NOT here any more. It existed only because the scaffold +// wired `routesFile()`, whose output emits `import { db } from …`; with nothing wired +// there is no such import, so the throwing `src/db.ts` stub that made it resolve has +// gone too. `dbImport` is now a `configKey` on the `routes` catalog entry — reported by +// `meta gen --list` and by `meta eject routes`, to the adopter who actually chose it. const SCAFFOLD_OUT_DIR = "src/generated"; -const SCAFFOLD_DB_IMPORT = "../db"; -// "src/generated" + "../db" -> "src/db" -> "src/db.ts" (dbImport resolves relative -// to outDir, same as the module specifier a generated route file emits). -const DB_STUB_REL_PATH = `${join(SCAFFOLD_OUT_DIR, SCAFFOLD_DB_IMPORT)}.ts`; const META_COMMON_JSON = JSON.stringify( { @@ -107,122 +90,49 @@ dist/ function buildMetaobjectsConfigBody(dialect: "sqlite" | "postgres" | "d1" = "sqlite"): string { return `import { defineConfig } from "@metaobjectsdev/cli"; -// Owned codegen generators (ADR-0034 scaffold-and-own). \`meta init\` copied these -// reference templates into ./codegen/generators/ — they are YOURS to edit, and -// \`meta gen\` runs from these local copies, not from the package. Read each file's -// header doc-block for what it emits and how to customize it. -import { entityFile } from "./codegen/generators/entity.js"; -import { queriesFile } from "./codegen/generators/queries.js"; -import { routesFile } from "./codegen/generators/routes.js"; -import { namesFile } from "./codegen/generators/names.js"; -import { barrel } from "./codegen/generators/barrel.js"; export default defineConfig({ - outDir: "${SCAFFOLD_OUT_DIR}", - extStyle: "js", // ".js"-extensioned relative imports — correct for Node ESM and \`tsc\` with - // nodenext, which is what a fresh project has. BUNDLERS DISAGREE: this fails - // outright under Turbopack (even between two generated files, so the whole - // generated tree goes unresolvable), while Vite and esbuild accept it and - // webpack needs \`resolve.extensionAlias\`. If a generated import fails to - // resolve, set "none" and retest — do not assume this line covers your bundler. - dbImport: "${SCAFFOLD_DB_IMPORT}", // routesFile() below emits \`import { db } from …\` — meta init - // scaffolded ${DB_STUB_REL_PATH} as a THROWING STUB (types clean, no - // driver chosen) so meta gen and tsc pass; replace it with your real - // Drizzle connection before running the app. - // (queriesFile() takes db as a parameter and never reads this.) - dialect: "${dialect}", - apiPrefix: "", // set to "/api" if your routes mount under /api - generators: [ - entityFile(), - queriesFile(), - routesFile(), - namesFile(), // Names — physical table/column constants (spec §A1/§A5) - barrel(), - ], + outDir: "${SCAFFOLD_OUT_DIR}", + dialect: "${dialect}", + + // NOTHING IS GENERATED UNTIL YOU CHOOSE IT. + // + // MetaObjects does not decide which code your application needs — you do, or the + // agent working in this repo does. The catalog is: + // + // meta gen --list --format json --probe + // + // \`--probe\` runs every generator against YOUR model and reports how many files each + // would emit, so you can see what your metadata is already asking for. Group by + // \`layer\`: model / persistence / api / client / docs / capability. + // + // Then take the ones you want: + // + // meta eject entity queries routes barrel + // + // That copies each generator into ./codegen/generators/ — YOURS to edit, and what + // \`meta gen\` runs — and prints the import line to add here, the entry to add below, + // what to install, and any config keys those generators read (\`dbImport\`, + // \`apiPrefix\`, \`extStyle\`, …). Add them here as you go. + generators: [], + docs: { - outDir: "${DEFAULT_DOCS_DIR}", // every surface lands here (run: meta docs). + outDir: "${DEFAULT_DOCS_DIR}", // every surface lands here (run: meta docs). // A SUB-directory on purpose: these pages are regenerated // and overwritten, and docs/ itself is usually yours. - layout: "flat", // or "package" for multi-package models + layout: "flat", // or "package" for multi-package models // surfaces defaults to ["model", "api", "requirements", "agent"] — all four. - // Deliberately NOT narrowed here: the requirements and agent surfaces emit ZERO - // files for a project that has nothing for them to describe, and the always-on - // agent context points at the agent/ pages by name. Scaffolding a narrower list - // turned that pointer into a pointer at nothing. Narrow it yourself if you want - // fewer docs — but note it says what your docs ARE, not which of them you commit - // (a page you do not commit is exempt from verify --docs once it is git-ignored). + // Documentation is the one thing that IS on by default, because the requirements + // and agent surfaces emit ZERO files for a project with nothing to describe, and + // the always-on agent context points at the agent/ pages by name. }, }); `; } -// The throwing-stub scaffolded at `dbImport`'s resolved path (DB_STUB_REL_PATH, -// "src/db.ts" by default). It exists so `meta gen` and a fresh project's FIRST -// `tsc` both succeed with no driver chosen and no dependency added — deliberately -// NOT a real connection. Every generated route only ever passes `db` straight -// through to `mountCrudRoutes(...)`; nothing reads a property off it at import -// time, so a value typed `unknown` (not `any`) satisfies every call site while -// making a genuine runtime use (mountCrudRoutes calling `db.select()` etc.) throw -// immediately with an actionable message instead of failing to resolve at all. -// Built as an array of plain single-quoted lines (not a template literal) so the -// backticks and quotes inside the comment/message need no escaping. -const DB_STUB_BODY = [ - "// `meta init` scaffolded this file because the generated Fastify routes", - '// `import { db } from "../db.js"` (see `dbImport` in metaobjects.config.ts) —', - "// a module that has to exist for `meta gen` and `tsc` to succeed. MetaObjects", - "// cannot fill it in for real without choosing a database driver on your", - "// behalf (better-sqlite3 vs @libsql/client vs pg vs postgres.js) and adding a", - "// dependency you may not want, so this is a STUB, not a connection.", - "//", - "// It type-checks and satisfies every generated import, but throws the first", - "// time anything actually touches `db` at runtime. Replace the export below", - "// with your real Drizzle connection, e.g.:", - "//", - '// import { drizzle } from "drizzle-orm/better-sqlite3";', - '// import Database from "better-sqlite3";', - '// export const db = drizzle(new Database("dev.sqlite"));', - "//", - "// (swap the driver import for your dialect — see", - "// https://github.com/metaobjectsdev/metaobjects/blob/main/docs/recipes/wiring-generated-queries.md", - "// for SQLite/libsql, Cloudflare D1, Postgres and multi-tenant setups.)", - "", - "const UNWIRED_MESSAGE =", - ' "src/db.ts is still the scaffolded stub meta init wrote — it cannot choose " +', - " \"a database driver for you. Replace 'export const db = ...' below with \" +", - ' "your real Drizzle connection, e.g.:\\n\\n" +', - " \" import { drizzle } from 'drizzle-orm/better-sqlite3';\\n\" +", - " \" import Database from 'better-sqlite3';\\n\" +", - " \" export const db = drizzle(new Database('dev.sqlite'));\\n\";", - "", - "function unwired(): never {", - " throw new Error(UNWIRED_MESSAGE);", - "}", - "", - "/**", - " * Stand-in for your real Drizzle database connection. Generated code only", - " * ever passes `db` straight through to `mountCrudRoutes(...)` — it never", - " * reads a property off it at import time — so this typechecks everywhere", - " * `db` is used, and throws the message above the first time anything really", - " * touches it.", - " */", - "export const db: unknown = new Proxy({}, { get: unwired });", - "", -].join("\n"); - -// Printed only when the stub was ACTUALLY written this run. It is gated on having just -// written the scaffolded config (see the db-stub block in `init`), so a re-run in a -// project that keeps its own config writes nothing — and a block claiming otherwise is -// the same "asserting things about its own scaffold that aren't true" defect the rest of -// this file was corrected for. -const DB_STUB_NOTE = `Also scaffolded ${DB_STUB_REL_PATH}: a THROWING STUB standing in for your database -connection, so the generated routes' \`db\` import resolves and the first tsc is clean. -Replace it with a real connection before running the app — until you do, the first -request that touches \`db\` throws with instructions. -`; - const SCAFFOLD_SUMMARY = ` Initialized metaobjects/ + .metaobjects/ + metaobjects.config.ts -Codegen generators copied to codegen/generators/ — they're YOURS to edit (ADR-0034 scaffold-and-own). +codegen/generators/ is EMPTY on purpose: no code is generated until you choose it. `; const NEXT_STEPS = ` @@ -230,10 +140,14 @@ Next steps: 0. Everything here is ESM — package.json needs "type": "module" (init sets it unless the project has CommonJS sources; without it the first tsc fails). 1. Author entities under metaobjects/ (start from the scaffolded meta.common.json) - 2. meta gen # generate idiomatic TypeScript from your entities - meta gen --dry-run # ...preview without writing - 3. meta docs # neutral model + API docs - 4. Create your tables: meta migrate --from-db --db file:dev.sqlite --dialect sqlite --slug init --apply + 2. meta gen --list --probe # the catalog: every generator, grouped by layer, with + # how many files each would emit for YOUR model + 3. meta eject ... # take the ones you want — copies them into + # codegen/generators/ (yours to edit) and prints the + # import, the entry to wire, and what to install + 4. meta gen # generate from exactly what you wired + 5. meta docs # neutral model + API docs (on by default) + 6. Create your tables: meta migrate --from-db --db file:dev.sqlite --dialect sqlite --slug init --apply Ship in later sub-projects: meta ingest (propose entities from existing code), meta serve (local viewer), meta install-hooks (MCP server + Claude Code hooks). @@ -461,29 +375,19 @@ async function wireRootMemory(cwd: string, result: InitResult, dryRun = false): } /** - * ADR-0034 — copy the codegen reference templates into the consumer's repo at - * `codegen/generators/.ts` so they own them. Each file is written only if absent, - * so a re-run with --force never clobbers a hand-edited generator. The scaffolded - * metaobjects.config.ts imports these local copies (not the package `/generators` export). + * Scaffold the owned-codegen TIER: the directory and its tsconfig, and nothing in it. + * + * `meta init` used to copy five reference generators here eagerly and wire all five in + * the scaffolded config. Codegen is opt-in now, so it copies NONE: the directory exists + * because it is part of the layout ADR-0034 promises (and `tsconfig.codegen.json` + * covers it the moment something lands), and `meta eject ...` is the door. * - * Copies SCAFFOLDED_GENERATOR_NAMES only — the five the scaffolded config actually - * wires — not every name @metaobjectsdev/codegen-ts happens to register. Anything else - * (routes-hono, and any UI-tier template from codegen-ts-react/-tanstack) is reached with - * `meta eject `, not by eager copying. + * The empty directory is not a placeholder for a decision deferred — it is the decision. + * A scaffold that wires a suite pre-empts the one judgement this design exists to leave + * to whoever is building the app. */ async function writeOwnedGenerators(opts: InitOptions, result: InitResult): Promise { - const dir = join(opts.cwd, OWNED_GENERATORS_DIR); - await mkdir(dir, { recursive: true }); - for (const name of SCAFFOLDED_GENERATOR_NAMES) { - const rel = `${OWNED_GENERATORS_DIR}/${name}.ts`; - const abs = join(dir, `${name}.ts`); - if (await fileExists(abs)) { - result.preserved.push(rel); - continue; - } - await writeFile(abs, readReferenceTemplate(name), "utf8"); - result.created.push(rel); - } + await mkdir(join(opts.cwd, OWNED_GENERATORS_DIR), { recursive: true }); await writeCodegenTsconfig(opts, result); } @@ -676,9 +580,8 @@ export async function init(opts: InitOptions): Promise { ".metaobjects/.gitignore", ); result.created.push(".metaobjects/AGENTS.md", ".metaobjects/CLAUDE.md", ".claude/skills/metaobjects-*", AGENT_CONTEXT_MANIFEST_PATH); - for (const name of SCAFFOLDED_GENERATOR_NAMES) result.created.push(`${OWNED_GENERATORS_DIR}/${name}.ts`); - result.created.push(CODEGEN_TSCONFIG_REL); - result.created.push("metaobjects.config.ts", DB_STUB_REL_PATH, ".gitignore"); + result.created.push(OWNED_GENERATORS_DIR, CODEGEN_TSCONFIG_REL); + result.created.push("metaobjects.config.ts", ".gitignore"); return result; } @@ -716,60 +619,17 @@ export async function init(opts: InitOptions): Promise { await writeAgentContext(opts, result); - // ADR-0034 — scaffold the OWNED codegen generators that metaobjects.config.ts imports - // locally. Done before the config so the import targets exist on first `meta gen`. + // ADR-0034 — the owned-codegen TIER: the directory and its tsconfig. Nothing is + // copied into it; `meta eject ...` is the door (see writeOwnedGenerators). await writeOwnedGenerators(opts, result); // Scaffold metaobjects.config.ts at the project root. Never overwrite if it exists. const forgeConfigPath = join(opts.cwd, "metaobjects.config.ts"); - const wroteScaffoldedConfig = !(await fileExists(forgeConfigPath)); - if (wroteScaffoldedConfig) { + if (!(await fileExists(forgeConfigPath))) { await writeFile(forgeConfigPath, buildMetaobjectsConfigBody(opts.d1 ? "d1" : "sqlite"), "utf8"); result.created.push("metaobjects.config.ts"); } - // Scaffold the `dbImport` throwing stub at DB_STUB_REL_PATH ("src/db.ts" by - // default) — ONLY if absent, so a re-run never clobbers a user's real db module - // (same "write once" precedent as writeOwnedGenerators above). Without this, the - // scaffolded config declares `dbImport: "../db"` pointing at a module `meta init` - // never creates, and a fresh project's FIRST `tsc` fails to resolve it. - // - // Gated on having just WRITTEN that config, not merely on the stub being absent. - // DB_STUB_REL_PATH is derived from SCAFFOLD_OUT_DIR + SCAFFOLD_DB_IMPORT — the - // scaffold's own constants — so it describes where the SCAFFOLDED config points and - // nowhere else. Re-running `meta init` in a project that already has a config with - // its own `outDir`/`dbImport` preserves that config (above) and would otherwise still - // drop a src/db.ts that nothing in the project references: a stray file, in the - // adopter's application source, answering a question they had already answered. - const dbStubPath = join(opts.cwd, DB_STUB_REL_PATH); - const dbStubExists = await fileExists(dbStubPath); - if (wroteScaffoldedConfig && !dbStubExists) { - await mkdir(dirname(dbStubPath), { recursive: true }); - await writeFile(dbStubPath, DB_STUB_BODY, "utf8"); - result.created.push(DB_STUB_REL_PATH); - } else if (dbStubExists) { - // "preserved" means a file we would have written was left alone. Skipping because - // this project keeps its own config is not preservation — there is nothing there. - result.preserved.push(DB_STUB_REL_PATH); - } else { - // Neither written nor preserved: this project keeps its own config AND has no stub. - // Usually correct — its `dbImport` points at a real module somewhere else. But it is - // ALSO what a scaffolded project looks like after someone deletes or moves src/db.ts, - // and `meta init --force` cannot tell those apart: `wroteScaffoldedConfig` is only - // "no config existed", so it is false for the config init itself wrote. Silently - // doing nothing there leaves the scaffolded `dbImport: "../db"` and the generated - // routes' `import { db } from "../db.js"` pointing at nothing, and the adopter meets - // it as a TS2307 from `tsc` with no word from the command that could have said so. - // Say it here rather than writing: dropping a file into a project that owns its - // config is what the branch above deliberately refuses. - result.warnings.push( - `${DB_STUB_REL_PATH} was not scaffolded — this project has its own ` + - "metaobjects.config.ts, so init leaves the database module to it. If that config's " + - `\`dbImport\` resolves to ${DB_STUB_REL_PATH}, create it or the generated routes will ` + - "not resolve.", - ); - } - // Scaffold a minimal root .gitignore ONLY when the project has none — never // clobber a user's existing one (they may have their own rules). const rootGitignorePath = join(opts.cwd, ".gitignore"); @@ -877,195 +737,29 @@ async function prepareManifestForScaffold(cwd: string, result: InitResult): Prom } } - const added = addScaffoldDevDependencies(pkg); - const addedRuntime = addScaffoldRuntimeDependencies(pkg); - if (moduleSystemNote !== undefined || added.length > 0 || addedRuntime.length > 0) { + // NO DEPENDENCIES ARE ADDED. `meta init` used to declare five packages here — + // codegen-ts and metadata for the generators it copied, drizzle-orm / zod / fastify + // for the code those generators would write — and every one of them was a + // consequence of the scaffold WIRING a suite. It wires nothing now, so declaring + // anything would be declaring a dependency on code this project may never generate. + // + // The need did not vanish, it MOVED to the moment a generator is chosen: + // `meta eject ...` reports the exact install set for what you took, with + // third-party ranges read from the runtime package's own peerDependencies (see + // lib/install-set.ts), and `meta gen --list --format json` carries the same set per + // entry before you commit to anything. + if (moduleSystemNote !== undefined) { // Preserve the file's existing indentation rather than reformatting someone's manifest. const indent = /\n(\s+)"/.exec(raw)?.[1] ?? " "; await writeFile(pkgPath, `${JSON.stringify(pkg, null, indent)}\n`, "utf8"); - } - if (moduleSystemNote !== undefined) result.warnings.push(moduleSystemNote); - if (added.length > 0) { - result.warnings.push( - `added ${added.join(" + ")} to devDependencies — the scaffolded ` + - "codegen/generators/ are YOUR source now (ADR-0034) and import them. " + - "Run your package manager's install before `meta gen`.", - ); - } - if (addedRuntime.length > 0) { - result.warnings.push( - `added ${addedRuntime.join(" + ")} to dependencies — the code \`meta gen\` writes ` + - "imports them, so `npx tsc` reports TS2307 on the generated files until they are " + - "installed. Run your package manager's install before `meta gen`.", - ); - } - // Never degrade in silence. The first version of this returned `{}` on any failure to - // read runtime-ts's peer ranges and skipped the three packages that come from them — - // which is exactly what happened against the published package, and the run looked - // successful while leaving the project with the six TS2307s this code exists to - // prevent. A fallback that cannot be observed is indistinguishable from a bug. - const undeclarable = SCAFFOLD_OUTPUT_PEERS.filter( - (name) => !declaredDependencyNames(pkg as PackageManifest).has(name), - ); - if (undeclarable.length > 0) { - result.warnings.push( - `could NOT declare ${undeclarable.join(" + ")} — the version ranges are read from ` + - "@metaobjectsdev/runtime-ts's own peerDependencies and that manifest could not be " + - "read. Generated code imports them, so add them to dependencies by hand or " + - "`npx tsc` will report TS2307 on files `meta gen` writes.", - ); + result.warnings.push(moduleSystemNote); } } -/** - * ADR-0034 scaffold-and-own hands the project real source files under - * `codegen/generators/`, and those files import `@metaobjectsdev/codegen-ts` and - * `@metaobjectsdev/metadata`. Installing `@metaobjectsdev/cli` alone does not put - * either of them where the project can resolve them, so the scaffold arrived - * un-typecheckable: ten TS2307s on files `meta init` had just written. - * - * Declaring them is the honest fix — they are dependencies of code that now lives in - * the adopter's repo. Deliberately NOT declared: `ts-poet`. The scaffolded templates - * import the ts-poet combinators via @metaobjectsdev/codegen-ts (re-exported from its - * own ts-poet instance) precisely so that the Code objects they compose share ONE - * ts-poet copy with the engine's render* primitives — a project-local ts-poet is the - * second physical copy that split the class identity under a globally-installed / - * linked CLI (duplicate imports in generated files, TS2300 on first tsc; see the - * gen-split-tree gate). Only ever ADDS a missing key: an existing pin is the user's. - * Returns what it added so the caller can tell them to install. - */ -function addScaffoldDevDependencies(pkg: Record): string[] { - const version = cliVersion(); - const wanted: Record = { - "@metaobjectsdev/codegen-ts": `^${version}`, - "@metaobjectsdev/metadata": `^${version}`, - }; - const dev = (pkg.devDependencies ?? {}) as Record; - // "Already declared" spans all four dependency fields — the shared rule, so this - // cannot drift from what `meta eject` means by the same words. It used to ask only - // dependencies + devDependencies, which meant a project declaring codegen-ts as a - // PEER dependency (correct for a library whose consumer supplies the version) got it - // added to devDependencies as well: the same package pinned twice in one manifest, - // and a second physical copy is the class-identity split this repo has been bitten - // by twice. - const declared = declaredDependencyNames(pkg as PackageManifest); - const added: string[] = []; - for (const [name, range] of Object.entries(wanted)) { - if (declared.has(name)) continue; - dev[name] = range; - added.push(name); - } - if (added.length > 0) { - pkg.devDependencies = Object.fromEntries(Object.entries(dev).sort(([a], [b]) => a.localeCompare(b))); - } - return added; -} -/** - * The third-party packages the scaffolded suite's GENERATED OUTPUT imports, each named - * with the artifact that imports it. This is the set, not the ranges — see - * `scaffoldRuntimeDependencies` for where those come from. - * - * `@metaobjectsdev/runtime-ts` is deliberately not here: its range is the CLI's own - * version, like the two build-time packages above, not a peer range read off itself. - */ -const SCAFFOLD_OUTPUT_PEERS = [ - "drizzle-orm", // .ts (the table + column builders) and .queries.ts - "zod", // .ts — the Insert/Update schemas - "fastify", // .routes.ts — `import type { FastifyInstance }` -] as const; -/** - * The runtime dependencies of the code `meta gen` will WRITE, for `dependencies`. - * - * `addScaffoldDevDependencies` above fixed this defect one layer in: the generator - * SOURCES under `codegen/generators/` import `@metaobjectsdev/codegen-ts` and - * `@metaobjectsdev/metadata`, nothing declared them, and the scaffold arrived - * un-typecheckable. The same argument reaches one layer further out and was not - * followed there. Generated `.ts` / `.queries.ts` / `.routes.ts` import - * drizzle-orm, zod, fastify and `@metaobjectsdev/runtime-ts/drizzle-fastify` — five - * specifiers, none declared — so `npx tsc`, which is the next step `meta gen` itself - * prints, reported NINE TS2307s on a brand-new project that had done nothing wrong. - * npm hides four of the five by hoisting them out of the CLI's own tree; pnpm's strict - * layout, which is the point of testing both, shows all five. - * - * `dependencies`, not `devDependencies`: generated routes and queries are application - * source that runs in production. The two build-time packages stay where they are. - * - * The RANGES are read from `@metaobjectsdev/runtime-ts`'s own `peerDependencies` rather - * than written here. That package already declares the versions its helpers are built - * against, bounded above (the peer-range gate enforces the bound), so a second copy of - * those ranges in the scaffolder is a second thing to keep in step — and the failure - * mode of drift is an adopter installing a major nothing was tested against. If a range - * cannot be read, the package is SKIPPED rather than guessed at, and `meta gen` still - * type-checks for anyone whose manifest already declares it. - */ -function scaffoldRuntimeDependencies(): Record { - const wanted: Record = { - "@metaobjectsdev/runtime-ts": `^${cliVersion()}`, - }; - const peers = runtimeTsPeerRanges(); - for (const name of SCAFFOLD_OUTPUT_PEERS) { - const range = peers[name]; - if (range !== undefined) wanted[name] = range; - } - return wanted; -} -/** - * `@metaobjectsdev/runtime-ts`'s declared peer ranges, or `{}` if they cannot be read. - * - * Resolves the package's ENTRY and walks up to the nearest package.json, rather than - * asking for `"@metaobjectsdev/runtime-ts/package.json"` directly. That direct form is - * the obvious one and it is WRONG: a package's `exports` map gates every subpath, and - * runtime-ts's exports are `.` / `./drivers` / `./fastify` / `./drizzle-fastify` / - * `./hono` — no `./package.json`. Under real Node it throws - * ERR_PACKAGE_PATH_NOT_EXPORTED. It appeared to work here only because the workspace - * runs this under bun against a symlinked source tree; against the published package it - * failed on the first try, silently, and `meta init` declared one package instead of - * four. Resolving the entry is never gated — `.` is the one subpath every package - * exports — so this form works under npm, pnpm, bun and a linked checkout alike. - */ -function runtimeTsPeerRanges(): Record { - try { - const req = createRequire(import.meta.url); - let dir = dirname(req.resolve("@metaobjectsdev/runtime-ts")); - // The entry sits under dist/; the manifest is at the package root above it. - for (let hops = 0; hops < 8; hops++) { - const candidate = join(dir, "package.json"); - if (existsSyncWrap(candidate)) { - const manifest = JSON.parse(readFileSyncWrap(candidate, "utf8")) as PackageManifest & { name?: string }; - // Guard against stopping at a nested manifest that is not the package itself. - if (manifest.name === "@metaobjectsdev/runtime-ts") { - return (manifest.peerDependencies ?? {}) as Record; - } - } - const up = dirname(dir); - if (up === dir) break; - dir = up; - } - return {}; - } catch { - return {}; - } -} -/** Adds any missing `scaffoldRuntimeDependencies()` to `dependencies`. Only ever ADDS a - * missing key — an existing pin, in any of the four dependency fields, is the user's. */ -function addScaffoldRuntimeDependencies(pkg: Record): string[] { - const deps = (pkg.dependencies ?? {}) as Record; - const declared = declaredDependencyNames(pkg as PackageManifest); - const added: string[] = []; - for (const [name, range] of Object.entries(scaffoldRuntimeDependencies())) { - if (declared.has(name)) continue; - deps[name] = range; - added.push(name); - } - if (added.length > 0) { - pkg.dependencies = Object.fromEntries(Object.entries(deps).sort(([a], [b]) => a.localeCompare(b))); - } - return added; -} /** True when the project has hand-written CommonJS at the root (excluding tooling dirs). */ async function hasCommonJsSources(cwd: string): Promise { @@ -1111,15 +805,13 @@ function buildD1MigrateBlock(cwd: string): Record { /** * The post-init message. * - * @param dbStubWritten whether THIS run wrote {@link DB_STUB_REL_PATH}. Required rather - * than defaulted, because a default would silently restore the bug this parameter - * exists to close: the note used to be part of one static string and so claimed the - * stub on every run, including the `meta init --force` in a project keeping its own - * `metaobjects.config.ts`, where the stub is deliberately not written. Callers pass - * `result.created.includes(DB_STUB_REL_PATH)` — the same list the message describes. + * Takes no arguments now. It used to take `dbStubWritten`, because the scaffold wrote a + * throwing `src/db.ts` on some runs and not others and a static string claimed it on + * every one. With nothing wired there is no `dbImport` and no stub, so the message is + * the same on every path again. */ -export function nextStepsBlock(dbStubWritten: boolean): string { - return SCAFFOLD_SUMMARY + (dbStubWritten ? DB_STUB_NOTE : "") + NEXT_STEPS; +export function nextStepsBlock(): string { + return SCAFFOLD_SUMMARY + NEXT_STEPS; } async function dirExists(p: string): Promise { @@ -1195,7 +887,7 @@ export async function initCommand(args: string[], cwd: string): Promise } for (const w of result.warnings) log.warn(w); } else { - log.info(nextStepsBlock(result.created.includes(DB_STUB_REL_PATH))); + log.info(nextStepsBlock()); // Surface any scaffold warnings (e.g. the #77 monorepo-subdir agent-context // discovery warning) — these are otherwise dropped on the normal init path. for (const w of result.warnings) log.warn(w); diff --git a/server/typescript/packages/cli/src/commands/verify.ts b/server/typescript/packages/cli/src/commands/verify.ts index 69f1a1633..b78d3c8e0 100644 --- a/server/typescript/packages/cli/src/commands/verify.ts +++ b/server/typescript/packages/cli/src/commands/verify.ts @@ -14,7 +14,9 @@ import { emitStructured, type OutputFormat } from "../lib/format.js"; import { antiPatternRows, missingBaseUrlRows, removedPropRows, ranSection, skippedSection, warnCapped, type AdvisoryDiagnosticRow, type AdvisoryFindingRow, type AdvisorySection, + libraryPrefixRows, } from "../lib/advisory.js"; +import { scanForUnprovenancedLibraryPrefix } from "../lib/library-prefix-advisory.js"; import { warnIfAgentContextStale } from "../lib/agent-context-staleness.js"; import { warnIfManifestIgnored } from "../lib/manifest-ignored-check.js"; import { scanSourceForAntiPatterns } from "../lib/anti-patterns.js"; @@ -382,7 +384,7 @@ export async function verifyCommand( // model. Warnings ONLY — never changes the exit code (bias to under-flagging). // Suppressed with --no-antipatterns or META_NO_ANTIPATTERNS=1 for the rare // noisy project (both opt-outs work on `meta verify` and `meta gen`). - runAntiPatternAdvisory(); + await runAntiPatternAdvisory(); const exitCode = Math.max( templateExit, @@ -646,15 +648,20 @@ export async function verifyCommand( say( `meta verify — requirements: ${s.total} entries (${s.functional} functional, ` + `${s.architectural} architectural) — ${parts.join(", ")}; ` + - `${s.entitiesClaimed}/${s.entitiesTotal} entities claimed, ` + - `counted over ${collection.files.length} metadata file(s)` + - // FR-023 — only when there ARE dependencies. A project that declares none - // must print the sentence it printed before dependencies existed, to the - // byte: "0 from dependencies" is noise on every existing project, and this - // line is a SURFACE the no-dependency guarantee covers like any other. - (collection.dependencies.length > 0 - ? `, ${collection.dependencies.length} from dependencies.` - : `.`), + // FR-043 §5.4 — a project whose every requirement came from a shipped library + // is not measured, and says so rather than printing a ratio. Silence would be + // worse than either: a missing clause reads as a clean coverage result. + (s.entitiesTotal === undefined + ? `coverage: not measured (no project-authored requirements).` + : `${s.entitiesClaimed}/${s.entitiesTotal} entities claimed, ` + + `counted over ${collection.files.length} metadata file(s)` + + // FR-023 — only when there ARE dependencies. A project that declares none + // must print the sentence it printed before dependencies existed, to the + // byte: "0 from dependencies" is noise on every existing project, and this + // line is a SURFACE the no-dependency guarantee covers like any other. + (collection.dependencies.length > 0 + ? `, ${collection.dependencies.length} from dependencies.` + : `.`)), ); if (s.undecided > 0) { say( @@ -674,8 +681,12 @@ export async function verifyCommand( .filter((k) => (s.byStatus[k] ?? 0) > 0) .map((k) => ({ status: k, count: s.byStatus[k] ?? 0 })), undecided: s.undecided, - entitiesClaimed: s.entitiesClaimed, - entitiesTotal: s.entitiesTotal, + // Absent when coverage was not measured (FR-043 §5.4), never zeroed: a reader + // cannot tell `0/0 claimed` from "not measured", and the two mean opposite + // things. + ...(s.entitiesTotal === undefined + ? {} + : { entitiesClaimed: s.entitiesClaimed ?? 0, entitiesTotal: s.entitiesTotal }), metadataFiles: collection.files.length, }; } @@ -777,7 +788,7 @@ export async function verifyCommand( // Records its result EITHER WAY — a skip carries its reason rather than looking // like a clean scan. Warnings only; nothing here reaches the exit code (the // scanner's own header: bias to under-flagging, never a non-zero exit). - function runAntiPatternAdvisory(): void { + async function runAntiPatternAdvisory(): Promise { if (flags.noAntipatterns) { antiPatternSection = skippedSection("suppressed by --no-antipatterns"); return; @@ -816,10 +827,23 @@ export async function verifyCommand( // advisory only — never breaks verify. } + // FR-043 §3.5 — `metaobjects::` in this project's own metadata with no ejection + // provenance. Advisory because with the library NOT opted in nothing is broken; the + // opted-in case is refused at load instead. + let libraryPrefix: Awaited> = []; + try { + libraryPrefix = await scanForUnprovenancedLibraryPrefix( + root, collection.ownFiles, collection.configDir, + ); + } catch { + // Same discipline as its siblings: an advisory scan never breaks verify. + } + antiPatternSection = ranSection([ ...antiPatternRows(findings), ...missingBaseUrlRows(baseUrl), ...removedPropRows(removedProps), + ...libraryPrefixRows(libraryPrefix), ]); if (findings.length > 0) { log.warn( @@ -842,6 +866,13 @@ export async function verifyCommand( ); warnCapped(removedProps.map((f) => ` ${f.message}`), flags.limit, { structured }); } + if (libraryPrefix.length > 0) { + log.warn( + `meta verify — ${libraryPrefix.length} node(s) declared under "metaobjects::" with ` + + `no ejection provenance (advisory — does not fail the build):`, + ); + warnCapped(libraryPrefix.map((f) => ` ${f.message}`), flags.limit, { structured }); + } } // -- template (prompt / output) drift -------------------------------------- @@ -1572,8 +1603,9 @@ interface RequirementCounts { architectural: number; byStatus: { status: string; count: number }[]; undecided: number; - entitiesClaimed: number; - entitiesTotal: number; + /** Absent when coverage was not measured — FR-043 §5.4. */ + entitiesClaimed?: number; + entitiesTotal?: number; /** How many metadata files the two entity counts were taken over. */ metadataFiles: number; } diff --git a/server/typescript/packages/cli/src/index.ts b/server/typescript/packages/cli/src/index.ts index 0d76b4378..1db115cb8 100644 --- a/server/typescript/packages/cli/src/index.ts +++ b/server/typescript/packages/cli/src/index.ts @@ -17,7 +17,7 @@ const VERSION = cliVersion(); * human text. It is named ONCE and used by both the warning below and the help * text above, so the two cannot drift apart. */ -const FORMAT_AWARE_COMMANDS: readonly string[] = ["gen", "verify", "migrate", "types", "deps"]; +const FORMAT_AWARE_COMMANDS: readonly string[] = ["gen", "verify", "migrate", "types", "deps", "eject"]; const HELP_TEXT = `meta — MetaObjects CLI (v${VERSION}) @@ -58,8 +58,14 @@ GLOBAL OPTIONS: GEN FLAGS: --dry-run Compute and print, don't write - --list Print the generator registry (name, tier, options) and exit — - no config or metadata required + --list Print the generator CATALOG (name, layer, tier, what it emits, + what to install) and exit. Codegen is opt-in: nothing runs until + you wire it, and this is where you choose. No config or metadata + required — add --format json for the same catalog, machine-readable. + --probe With --list: construct every catalog generator and dry-run it + against YOUR model, reporting how many files each would emit. + A count per generator beats any category label — and it + cannot go stale, because it runs the generators. Needs a project. --baseline First-time-on-existing-file behavior. Default: refuse a file that cannot be proved to be generated output. "adopt" records the files @@ -156,8 +162,14 @@ USAGE: FLAGS: --dry-run Compute and print, don't write - --list Print the generator registry (name, tier, options) and exit — - no config or metadata required + --list Print the generator CATALOG (name, layer, tier, what it emits, + what to install) and exit. Codegen is opt-in: nothing runs until + you wire it, and this is where you choose. No config or metadata + required — add --format json for the same catalog, machine-readable. + --probe With --list: construct every catalog generator and dry-run it + against YOUR model, reporting how many files each would emit. + A count per generator beats any category label — and it cannot go + stale, because it runs the generators. Needs a project. --baseline First-time-on-existing-file behavior. Default: refuse a file that cannot be proved to be generated output. "adopt" records the files @@ -183,23 +195,33 @@ limit on the machine-readable result. NOTE: outDir, dialect, dbImport, extStyle are read from metaobjects.config.ts `, - eject: `meta eject — copy a reference generator into your repo so you own it + eject: `meta eject — copy reference generators into your repo so you own them USAGE: - meta eject Copy generator into codegen/generators/.ts + meta eject ... Copy each generator into codegen/generators/.ts meta eject --list List every ejectable generator name, grouped by package +Codegen is opt-in — \`meta init\` wires nothing — so this is the door every generator +you run comes through. Take several at once: the install set is CONSOLIDATED, so +\`meta eject form hooks grid\` prints one install line, not three. An unknown name +refuses the whole call and copies nothing. + FLAGS: --list List ejectable generators instead of copying one --force Overwrite an already-ejected file (default: never clobber) + --format Output format (global flag). The structured form carries + each file's wire lines, the consolidated install set and the + config keys the ejected generators read. --help, -h Print this help -\`meta init\` copies five generators (entity, queries, routes, barrel, names) into -codegen/generators/ automatically (ADR-0034 scaffold-and-own). \`meta eject\` is -the same operation for ANY generator — one you skipped at init time, a UI-tier -generator like form/hooks/grid, or one a package gains later. It prints the -import line to paste into metaobjects.config.ts, and it never overwrites a -file you already own unless you pass --force. +\`meta init\` copies NO generators — codegen is opt-in, and codegen/generators/ +starts empty on purpose. Everything you run comes through here (ADR-0034 +scaffold-and-own): the file is YOURS to edit, and \`meta gen\` runs your copy, not +the package's. It prints the import line and the entry to add in +metaobjects.config.ts, and never overwrites a file you already own without --force. + +Find names with \`meta gen --list\` — add --probe to see how many files each would +emit for your model. `, deps: `meta deps — sync a declared metadata dependency's committed snapshot @@ -584,7 +606,7 @@ export async function run(argv: string[]): Promise { } case "eject": { const { ejectCommand } = await import("./commands/eject.js"); - return ejectCommand(rest, cwd); + return ejectCommand(rest, cwd, fmt); } case "deps": { const { depsCommand } = await import("./commands/deps.js"); diff --git a/server/typescript/packages/cli/src/lib/advisory.ts b/server/typescript/packages/cli/src/lib/advisory.ts index 895164496..cc2c0a85a 100644 --- a/server/typescript/packages/cli/src/lib/advisory.ts +++ b/server/typescript/packages/cli/src/lib/advisory.ts @@ -150,6 +150,19 @@ export function removedPropRows(findings: readonly RemovedPropFinding[]): Adviso })); } +/** FR-043 §3.5 findings in the same row shape. */ +export function libraryPrefixRows( + findings: readonly { file: string; fqn: string; message: string }[], +): AdvisoryFindingRow[] { + return findings.map((f) => ({ + file: f.file, + line: 0, + rule: "library-prefix-unprovenanced", + construct: f.fqn, + message: f.message, + })); +} + // --------------------------------------------------------------------------- // text output // --------------------------------------------------------------------------- diff --git a/server/typescript/packages/cli/src/lib/args.ts b/server/typescript/packages/cli/src/lib/args.ts index d9864e3b3..39d16642f 100644 --- a/server/typescript/packages/cli/src/lib/args.ts +++ b/server/typescript/packages/cli/src/lib/args.ts @@ -107,6 +107,12 @@ export interface GenFlags { /** ADR-0021 D3 — print the stable-name generator registry and exit without * running codegen. */ list: boolean; + /** + * `--list` only: construct every catalog generator and dry-run it against THIS + * project's model, reporting a real file count per generator. Needs a project; + * plain `--list` deliberately does not (it describes the installed engine). + */ + probe: boolean; /** Suppress the advisory anti-pattern (verify-as-teacher) pass. */ noAntipatterns: boolean; /** @@ -122,6 +128,7 @@ export const GEN_OPTIONS = { "dry-run": { type: "boolean", default: false }, "baseline": { type: "string" }, "list": { type: "boolean", default: false }, + "probe": { type: "boolean", default: false }, "no-antipatterns": { type: "boolean", default: false }, "limit": { type: "string" }, } as const; @@ -149,6 +156,7 @@ export function parseGenArgs(argv: string[]): GenFlags { entities: positionals, baseline: (baselineRaw as "default" | "fresh" | "adopt" | undefined) ?? "default", list: !!values.list, + probe: !!values.probe, noAntipatterns: !!values["no-antipatterns"], // Throws on a bad value; the command layer reports it and exits 2, exactly as // it does for --baseline above. @@ -602,8 +610,14 @@ export function parseMigrateArgs(argv: string[]): MigrateFlags { // --------------------------------------------------------------------------- export interface EjectFlags { - /** The generator name to eject; undefined when only --list was given. */ - name: string | undefined; + /** + * The generator names to eject, in the order given. Empty when only --list was given. + * + * MANY, not one: choosing a client tier means `form hooks grid`, and three separate + * invocations produce three separate install lines for the same package. The + * consolidated install set is the point of taking them together. + */ + names: string[]; list: boolean; /** Overwrite an already-ejected file; default false — eject never clobbers. */ force: boolean; @@ -623,12 +637,22 @@ export function parseEjectArgs(argv: string[]): EjectFlags { allowPositionals: true, }); - if (positionals.length > 1) { - throw new Error(`meta eject takes at most one generator name; got: ${positionals.join(", ")}`); + // `Set.add` returns the SET, which is always truthy — so the tempting + // `positionals.filter((n) => !seen.add(n))` never reports anything. + const seen = new Set(); + const duplicates: string[] = []; + for (const n of positionals) { + if (seen.has(n)) duplicates.push(n); + else seen.add(n); + } + if (duplicates.length > 0) { + throw new Error( + `meta eject: repeated generator name(s): ${[...new Set(duplicates)].join(", ")}`, + ); } return { - name: positionals[0], + names: positionals, list: !!values.list, force: !!values.force, }; diff --git a/server/typescript/packages/cli/src/lib/catalog-listing.ts b/server/typescript/packages/cli/src/lib/catalog-listing.ts new file mode 100644 index 000000000..d482da886 --- /dev/null +++ b/server/typescript/packages/cli/src/lib/catalog-listing.ts @@ -0,0 +1,369 @@ +// `meta gen --list` — the catalog as one machine-readable document. +// +// The governing rule (opt-in-codegen design §1): stop hard-coding selection decisions +// into the CLI. The tool's job is to describe what it can do, accurately, and to make +// each choice cheap to act on. Deciding WHICH generators an application needs belongs +// to whoever is building it — increasingly an LLM working in the repo, which is well +// able to make that call given a truthful catalog and is badly served by a default that +// pre-empts it. +// +// Two questions this answers, kept apart on purpose (§D5): +// +// APPLICABILITY — "would this emit anything for MY model?" — is `--probe`, which +// cannot drift, because it does not describe the generators, it RUNS them. +// COMPATIBILITY — "what does this need in order to work?" — is declared +// (`requires`, `runtimePeers`) and therefore has to be gated separately. +// +// Conflating the two is how a catalog goes quietly wrong as frameworks are added. + +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { + runGen, + type GeneratorRegistryEntry, + type Layer, + type MetaobjectsGenConfig, +} from "@metaobjectsdev/codegen-ts"; +import * as coreTpl from "@metaobjectsdev/codegen-ts"; +import * as reactTpl from "@metaobjectsdev/codegen-ts-react"; +import * as tanstackTpl from "@metaobjectsdev/codegen-ts-tanstack"; +import { listCatalog, packageOf } from "./catalog.js"; +import { + buildLibraryRows, renderLibraryText, + type LibraryCatalogRow, type LibraryProjectContext, +} from "./library-listing.js"; +import { installSetFor } from "./install-set.js"; +import { readPackageManifest, declaredDependencyNames } from "./package-manifest.js"; + +/** + * One `--list` row. The cross-port subset is name / layer / tier / description. + * + * `kind` is the discriminator: FR-043 adds `kind: "library"` rows to THIS table rather + * than a parallel one, because an agent about to model a capability should meet the + * library that already declares it in the list it was already reading. + */ +export type CatalogRow = GeneratorCatalogRow | LibraryCatalogRow; + +export type { LibraryCatalogRow }; + +export interface GeneratorCatalogRow { + name: string; + kind: "generator"; + layer: Layer; + framework?: string; + tier: "native" | "neutral"; + package: string; + description: string; + /** From the reference template's `use-when:` header. Absent for a non-ejectable entry. */ + useWhen?: string; + /** From the reference template's `emits:` header. Absent for a non-ejectable entry. */ + emits?: string; + requires?: readonly string[]; + configKeys?: readonly string[]; + install: { dev: string[]; runtime: string[] }; + source: { + kind: "reference-template" | "package-only"; + ejectable: boolean; + /** Whether this project already owns a copy. `null` with no project. */ + owned: boolean | null; + }; + project?: { + /** Wired in this project's `generators: [...]`. */ + wired: boolean; + /** This generator's framework is a declared dependency. `null` when neutral or unknown. */ + frameworkDetected: boolean | null; + /** Files this generator WOULD emit for this model. `null` unless --probe. */ + wouldEmit: number | null; + /** Why `wouldEmit` is null despite --probe. Absent when it is a number. */ + probeError?: string; + }; +} + +/** Where an ejected copy lands — mirrors `eject.ts`'s constant of the same name. */ +const OWNED_GENERATORS_DIR = "codegen/generators"; + +/** The npm package a framework token names, for `frameworkDetected`. */ +const FRAMEWORK_PACKAGE: Record = { + fastify: "fastify", + hono: "hono", + react: "react", + tanstack: "@tanstack/react-query", +}; + +// --------------------------------------------------------------------------- +// reference-template headers +// --------------------------------------------------------------------------- + +/** The three packages that ship reference templates, and their roots. */ +const TEMPLATE_ROOTS: ReadonlyArray string]> = [ + [coreTpl.REFERENCE_GENERATOR_NAMES, coreTpl.resolveReferenceRoot], + [reactTpl.REFERENCE_GENERATOR_NAMES, reactTpl.resolveReferenceRoot], + [tanstackTpl.REFERENCE_GENERATOR_NAMES, tanstackTpl.resolveReferenceRoot], +]; + +/** + * A `// : ` header line and its indented continuations, unwrapped. + * + * Read from the template rather than restated in the registry entry, for the same + * reason `eject` reads its import line from the header: the template is the thing an + * adopter opens, so a second copy of its own summary in the registry is a second thing + * to keep in step. A test asserts every ejectable entry's row actually carries these. + */ +function headerFacet(source: string, facet: string): string | undefined { + const lines = source.split("\n"); + const start = lines.findIndex((l) => l.startsWith(`// ${facet}:`)); + if (start === -1) return undefined; + const parts = [lines[start]!.slice(`// ${facet}:`.length).trim()]; + for (let i = start + 1; i < lines.length; i++) { + const line = lines[i]!; + // A continuation is a comment line indented past the facet column and NOT itself a + // new `facet:` line. Anything else ends the block. + if (!/^\/\/ {3,}\S/.test(line) || /^\/\/ {3,}[a-z-]+: /.test(line)) break; + parts.push(line.replace(/^\/\/\s+/, "").trim()); + } + return parts.join(" ").trim() || undefined; +} + +function readTemplate(name: string): string | undefined { + for (const [names, root] of TEMPLATE_ROOTS) { + if (!names.includes(name)) continue; + try { + return readFileSync(join(root(), `${name}.ts`), "utf8"); + } catch { + return undefined; + } + } + return undefined; +} + +// --------------------------------------------------------------------------- +// the probe +// --------------------------------------------------------------------------- + +/** + * The project facts a `--list` row can carry. All optional: `--list` with no project at + * all is the documented shape (`meta types` behaves the same way), because "what CAN + * this engine do" is a question about the installed engine, not about a repo. + */ +export interface CatalogProject { + projectRoot: string; + config: MetaobjectsGenConfig; + /** Stable names of the wired generators, as far as they can be identified. */ + wiredNames: ReadonlySet; + /** Owned copies present under `codegen/generators/`. */ + ownedNames: ReadonlySet; + /** Declared dependency names from package.json, for `frameworkDetected`. */ + declaredDeps: ReadonlySet | undefined; + /** FR-043 — the project's `libraries` selection, for the `kind: "library"` rows. + * Defaults to none, so a caller that predates libraries reports "not opted in" + * rather than crashing. */ + libraries?: readonly string[]; +} + +export interface CatalogListingOpts { + project?: CatalogProject; + /** Run every generator in memory and report a real file count. Needs `project` + `metadata`. */ + probe?: { metadata: MetaData; scope?: (fqn: string) => boolean }; +} + +/** + * Dry-run ONE generator and count what it would emit. + * + * One `runGen` per generator rather than one run over all of them, for isolation: a + * generator that throws (or one whose real use requires options `--list` cannot supply, + * like `template` and `shared-model`) must report itself and leave every other row + * intact. `dryRun` touches nothing on disk. + */ +async function probeOne( + entry: GeneratorRegistryEntry, + project: CatalogProject, + metadata: MetaData, + scope: ((fqn: string) => boolean) | undefined, +): Promise<{ count: number } | { error: string }> { + try { + const result = await runGen({ + config: { ...project.config, generators: [entry.factory()] }, + metadata, + projectRoot: project.projectRoot, + dryRun: true, + ...(scope !== undefined ? { scope } : {}), + }); + return { count: result.files.length }; + } catch (err) { + return { error: (err as Error).message }; + } +} + +// --------------------------------------------------------------------------- +// the listing +// --------------------------------------------------------------------------- + +export async function buildCatalogListing(opts: CatalogListingOpts = {}): Promise { + const rows: CatalogRow[] = []; + + for (const entry of listCatalog()) { + const template = entry.ejectable ? readTemplate(entry.name) : undefined; + const install = installSetFor([entry]); + const pkg = packageOf(entry.name) ?? ""; + + const row: GeneratorCatalogRow = { + name: entry.name, + kind: entry.kind, + layer: entry.layer, + ...(entry.framework !== undefined ? { framework: entry.framework } : {}), + tier: entry.tier, + package: pkg, + description: entry.description, + ...(template !== undefined && headerFacet(template, "use-when") !== undefined + ? { useWhen: headerFacet(template, "use-when")! } + : {}), + ...(template !== undefined && headerFacet(template, "emits") !== undefined + ? { emits: headerFacet(template, "emits")! } + : {}), + ...(entry.requires !== undefined ? { requires: entry.requires } : {}), + ...(entry.configKeys !== undefined ? { configKeys: entry.configKeys } : {}), + install: { dev: install.dev, runtime: install.runtime }, + source: { + kind: entry.ejectable ? "reference-template" : "package-only", + ejectable: entry.ejectable, + owned: opts.project === undefined ? null : opts.project.ownedNames.has(entry.name), + }, + }; + + if (opts.project !== undefined) { + const project = opts.project; + const fw = entry.framework; + const frameworkDetected = + fw === undefined || project.declaredDeps === undefined + ? null + : project.declaredDeps.has(FRAMEWORK_PACKAGE[fw] ?? fw); + + let wouldEmit: number | null = null; + let probeError: string | undefined; + if (opts.probe !== undefined) { + const outcome = await probeOne(entry, project, opts.probe.metadata, opts.probe.scope); + if ("count" in outcome) wouldEmit = outcome.count; + else probeError = outcome.error; + } + + row.project = { + wired: project.wiredNames.has(entry.name), + frameworkDetected, + wouldEmit, + ...(probeError !== undefined ? { probeError } : {}), + }; + } + + rows.push(row); + } + + // FR-043 — the library rows, after the generators. Same table, same `--probe`. + const libraryCtx: LibraryProjectContext | undefined = + opts.project === undefined + ? undefined + : { + libraries: opts.project.libraries ?? [], + wiredNames: opts.project.wiredNames, + ...(opts.probe !== undefined ? { metadata: opts.probe.metadata } : {}), + }; + rows.push(...(await buildLibraryRows(libraryCtx))); + + return rows; +} + +/** + * The stable names wired in a config. + * + * `generators: [...]` holds constructed Generator objects (and, since ADR-0021 #1, + * bare stable-name strings). A constructed generator carries its own kebab-case `name`, + * which IS the stable name for every catalog entry — the registry's factory is what + * produced it. A generator whose name is not a catalog key is an owned or third-party + * one and simply contributes nothing here. + */ +export function wiredGeneratorNames(config: MetaobjectsGenConfig): Set { + const names = new Set(); + for (const spec of config.generators ?? []) { + if (typeof spec === "string") names.add(spec); + else if (typeof spec?.name === "string") names.add(spec.name); + } + return names; +} + +/** Owned copies present under `codegen/generators/` in this project. */ +export function ownedGeneratorNames(projectRoot: string): Set { + const owned = new Set(); + for (const entry of listCatalog()) { + if (!entry.ejectable) continue; + try { + readFileSync(join(projectRoot, OWNED_GENERATORS_DIR, `${entry.name}.ts`), "utf8"); + owned.add(entry.name); + } catch { + // absent — not owned + } + } + return owned; +} + +/** Declared dependency names, or undefined when there is no readable manifest. */ +export function declaredDepsOf(projectRoot: string): Set | undefined { + const pkg = readPackageManifest(projectRoot); + return pkg === undefined ? undefined : declaredDependencyNames(pkg); +} + +// --------------------------------------------------------------------------- +// text rendering +// --------------------------------------------------------------------------- + +const LAYER_BLURB: Record = { + model: "the entity modules and the constants beside them", + persistence: "how rows are read and written", + api: "the HTTP surface — pick ONE framework", + client: "the browser tier — form + hooks + grid compose, they do not conflict", + docs: "on by default; the canonical door is `meta docs`", + capability: "chosen by your MODEL, not by browsing — run --probe", +}; + +/** The human rendering: grouped by layer, because layer is the axis you select by. */ +export function renderCatalogText(rows: CatalogRow[], probed: boolean): string { + const lines: string[] = []; + lines.push("Generator catalog — nothing runs until you wire it in `generators: [...]`."); + lines.push(""); + + const generators = rows.filter((r): r is GeneratorCatalogRow => r.kind === "generator"); + const width = Math.max(...generators.map((r) => r.name.length)); + let layer: string | undefined; + for (const r of generators) { + if (r.layer !== layer) { + layer = r.layer; + lines.push(`${layer} — ${LAYER_BLURB[r.layer]}`); + } + const marks: string[] = []; + if (r.framework !== undefined) marks.push(r.framework); + if (r.tier === "neutral") marks.push("neutral"); + if (r.project?.wired) marks.push("WIRED"); + if (r.source.owned) marks.push("owned"); + if (probed && r.project?.wouldEmit !== null && r.project?.wouldEmit !== undefined) { + marks.push(`would emit ${r.project.wouldEmit}`); + } + const suffix = marks.length > 0 ? ` [${marks.join(", ")}]` : ""; + lines.push(` ${r.name.padEnd(width)} — ${r.description}${suffix}`); + if (r.requires !== undefined && r.requires.length > 0) { + lines.push(` ${" ".repeat(width)} requires: ${r.requires.join(", ")}`); + } + } + + // The library section sits between the generators and the footer: the footer's + // `meta eject` line applies to both kinds, and putting libraries after it buries + // the one row an agent modelling a capability most needs to see. + lines.push(...renderLibraryText(rows.filter((r): r is LibraryCatalogRow => r.kind === "library"))); + + lines.push(""); + lines.push("`meta eject ` copies a generator into codegen/generators/ and prints"); + lines.push("the import to add, the entry to wire, and what to install."); + if (!probed) { + lines.push("Add --probe to see how many files each would emit for YOUR model."); + } + lines.push("`meta gen --list --format json` is the same catalog, machine-readable."); + return lines.join("\n"); +} diff --git a/server/typescript/packages/cli/src/lib/catalog.ts b/server/typescript/packages/cli/src/lib/catalog.ts new file mode 100644 index 000000000..e5f476cce --- /dev/null +++ b/server/typescript/packages/cli/src/lib/catalog.ts @@ -0,0 +1,93 @@ +// The TypeScript port's generator CATALOG — the one place all three registry slices +// are visible at once (ADR-0021 D3; opt-in-codegen design §D2a). +// +// `codegen-ts` owns the entry type and its own entries, but cannot import +// `codegen-ts-react` / `codegen-ts-tanstack` (dependency direction), so each of those +// exports its own slice and the CLI unions them here — exactly how `eject.ts` already +// composes its `SOURCES` table over the same three packages. +// +// Before this existed, `meta gen --list` read the codegen-ts registry and +// `meta eject --list` read the reference-template lists, so an agent asking "what can I +// turn on" got two different answers. After it, they are one table with two views. +// +// (The Angular generators stay out until ADR-0048's promotion bar is met — a +// source-only package is not a catalog entry.) + +import { + generatorRegistry, + GENERATOR_LAYERS, + type GeneratorRegistryEntry, +} from "@metaobjectsdev/codegen-ts"; +import { reactGeneratorRegistry } from "@metaobjectsdev/codegen-ts-react"; +import { tanstackGeneratorRegistry } from "@metaobjectsdev/codegen-ts-tanstack"; + +/** The slices, in install-boundary order: the engine, then its framework packages. */ +const SLICES: ReadonlyArray]> = [ + ["@metaobjectsdev/codegen-ts", generatorRegistry], + ["@metaobjectsdev/codegen-ts-react", reactGeneratorRegistry], + ["@metaobjectsdev/codegen-ts-tanstack", tanstackGeneratorRegistry], +]; + +/** + * Which npm package a catalog entry comes from. + * + * This is the fact composition ADDS: a slice cannot know its own package name without + * hard-coding it in every entry, and the package is precisely what `meta eject` has to + * name in an install command. Undefined for a name no slice registers. + */ +export function packageOf(name: string): string | undefined { + return SLICES.find(([, slice]) => name in slice)?.[0]; +} + +/** Every `@metaobjectsdev` package that contributes catalog entries. */ +export function catalogPackages(): string[] { + return SLICES.map(([pkg]) => pkg); +} + +/** + * The composed catalog, keyed by stable name. + * + * Throws on a name registered by two packages. A stable name identifies ONE generator + * across the whole catalog — that is the entire content of ADR-0021 D3 — so a collision + * is a build-time defect, not something to resolve by precedence. Silently letting the + * last slice win would make `meta gen --list` describe one generator and `meta gen` run + * another. + */ +export function composeCatalog(): Record { + const out: Record = {}; + const from: Record = {}; + for (const [pkg, slice] of SLICES) { + for (const [name, entry] of Object.entries(slice)) { + if (name in out) { + throw new Error( + `generator "${name}" is registered by both ${from[name]} and ${pkg} — a stable ` + + "name identifies ONE generator across the whole catalog (ADR-0021 D3). Rename " + + "one of them, or delete the duplicate registration.", + ); + } + out[name] = entry; + from[name] = pkg; + } + } + return out; +} + +/** + * Every catalog entry, grouped by layer in {@link GENERATOR_LAYERS} order and + * alphabetical within a layer. + * + * Layer order, not tier order: `layer` is the axis an adopter selects by, so it is the + * axis the listing is organized along. `tier` still travels on each row. + */ +export function listCatalog(): GeneratorRegistryEntry[] { + return Object.values(composeCatalog()).sort( + (a, b) => + GENERATOR_LAYERS.indexOf(a.layer) - GENERATOR_LAYERS.indexOf(b.layer) || + a.name.localeCompare(b.name), + ); +} + +/** Resolve one catalog entry by stable name, or undefined. */ +export function catalogEntry(name: string): GeneratorRegistryEntry | undefined { + return composeCatalog()[name]; +} diff --git a/server/typescript/packages/cli/src/lib/collection-load-options.ts b/server/typescript/packages/cli/src/lib/collection-load-options.ts index c9602de5f..629503f8b 100644 --- a/server/typescript/packages/cli/src/lib/collection-load-options.ts +++ b/server/typescript/packages/cli/src/lib/collection-load-options.ts @@ -26,11 +26,17 @@ import type { Collection, LoadMemoryOptions } from "@metaobjectsdev/sdk"; */ export function collectionLoadOptions( collection: Collection, -): Required> { +): Required> { return { files: collection.files, fileIds: collection.fileIds, importedPackages: collection.importedPackages, importedNodes: collection.importedNodes, + // FR-043 — the shipped-library selection moved here from metaobjects.config.ts, so + // it now rides the same helper as everything else the collection contributes. That + // is the point of the move as much as the neutrality is: `libraries` used to be + // threaded by `loadMemoryOptionsFrom`, a SECOND helper, and #333 is on record as the + // bug where one of the two reached every command and the other reached none. + libraries: collection.libraries, }; } diff --git a/server/typescript/packages/cli/src/lib/install-set.ts b/server/typescript/packages/cli/src/lib/install-set.ts new file mode 100644 index 000000000..953ffd004 --- /dev/null +++ b/server/typescript/packages/cli/src/lib/install-set.ts @@ -0,0 +1,116 @@ +// What a catalog entry costs to install — the one derivation behind both +// `meta gen --list --format json` and `meta eject --format json`. +// +// Two kinds of package, resolved two different ways: +// +// dev the `@metaobjectsdev/codegen-*` package the generator lives in. Its range +// is the CLI's own version: the engine and the CLI ship in lockstep, so a +// generator from a different minor is a version mismatch, not a choice. +// runtime the `@metaobjectsdev` runtime the EMITTED code imports, plus the +// third-party packages it imports. The runtime's range is the CLI's version +// for the same reason; the third-party RANGES are read from that runtime +// package's own `peerDependencies` and are never written here. +// +// That last rule is the point of this module. A second copy of a peer range in the CLI +// is a second thing to keep in step, and the failure mode of drift is an adopter +// installing a major nothing was tested against — the ERESOLVE trap FR-040 §4.4 +// documented, closed by construction rather than by a warning. If a range cannot be +// read, the package is reported WITHOUT one rather than guessed at. + +import { createRequire } from "node:module"; +import { existsSync, readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import type { GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; +import { cliVersion } from "./version.js"; +import { packageOf } from "./catalog.js"; +import type { PackageManifest } from "./package-manifest.js"; + +/** + * A package's declared peer ranges, or `{}` if they cannot be read. + * + * Resolves the package's ENTRY and walks up to the nearest manifest, rather than asking + * for `"/package.json"` directly. The direct form is the obvious one and it is + * WRONG: a package's `exports` map gates every subpath, and `@metaobjectsdev/runtime-ts` + * exports `.` / `./drivers` / `./fastify` / `./drizzle-fastify` / `./hono` and no + * `./package.json`, so under real Node it throws ERR_PACKAGE_PATH_NOT_EXPORTED. It + * appeared to work in-repo only because bun resolves a symlinked source tree; against + * the published package it failed silently on the first try. Resolving the entry is + * never gated — `.` is the one subpath every package exports. + */ +export function peerRangesOf(packageName: string): Record { + try { + const req = createRequire(import.meta.url); + let dir = dirname(req.resolve(packageName)); + // The entry sits under dist/; the manifest is at the package root above it. + for (let hops = 0; hops < 8; hops++) { + const candidate = join(dir, "package.json"); + if (existsSync(candidate)) { + const manifest = JSON.parse(readFileSync(candidate, "utf8")) as PackageManifest & { + name?: string; + }; + // Guard against stopping at a nested manifest that is not the package itself. + if (manifest.name === packageName) { + return (manifest.peerDependencies ?? {}) as Record; + } + } + const up = dirname(dir); + if (up === dir) break; + dir = up; + } + return {}; + } catch { + return {}; + } +} + +export interface InstallSet { + /** Build-time packages, `name@range`. */ + dev: string[]; + /** Application-runtime packages, `name@range` (or bare `name` if no range is known). */ + runtime: string[]; + /** A paste-ready shell line, or "" when there is nothing to install. */ + command: string; +} + +function spec(name: string, range: string | undefined): string { + return range === undefined ? name : `${name}@${range}`; +} + +/** + * The consolidated install set for a group of catalog entries. + * + * Consolidated, not per-entry: ejecting `hooks` and `grid` needs + * `@metaobjectsdev/codegen-ts-tanstack` ONCE, and an adopter handed the same package + * twice reasonably wonders which one to run. + */ +export function installSetFor(entries: readonly GeneratorRegistryEntry[]): InstallSet { + const version = cliVersion(); + const dev = new Set(); + const runtime = new Set(); + + for (const entry of entries) { + const pkg = packageOf(entry.name); + if (pkg !== undefined) dev.add(spec(pkg, `^${version}`)); + + for (const rt of entry.runtimePackages ?? []) runtime.add(spec(rt, `^${version}`)); + + if (entry.runtimePeers !== undefined && entry.runtimePeers.length > 0) { + // Ranges come from whichever runtime package declares these as peers — the union + // over this generator's runtimes, since a generator emitting against two of them + // may take a peer from either. A generator with third-party peers but no runtime + // package of its own has none to read, so its peers are named UNPINNED: honest, + // and better than a made-up bound. + const ranges: Record = {}; + for (const rt of entry.runtimePackages ?? []) Object.assign(ranges, peerRangesOf(rt)); + for (const peer of entry.runtimePeers) runtime.add(spec(peer, ranges[peer])); + } + } + + const devList = [...dev].sort(); + const runtimeList = [...runtime].sort(); + const parts: string[] = []; + if (devList.length > 0) parts.push(`npm i -D ${devList.join(" ")}`); + if (runtimeList.length > 0) parts.push(`npm i ${runtimeList.join(" ")}`); + + return { dev: devList, runtime: runtimeList, command: parts.join(" && ") }; +} diff --git a/server/typescript/packages/cli/src/lib/library-eject.ts b/server/typescript/packages/cli/src/lib/library-eject.ts new file mode 100644 index 000000000..1829e909a --- /dev/null +++ b/server/typescript/packages/cli/src/lib/library-eject.ts @@ -0,0 +1,298 @@ +// `meta eject ` — FR-043 §3.4, the copy door for a shipped library's METADATA. +// +// ADR-0034 ruled that a reference GENERATOR is copied into the adopter's repo because +// the adopter owns their code. §3.4 extends the same ruling to declared design, and +// makes it the EXPECTED mode rather than the fallback: a library is first a reference — +// something to copy, rename, and cut down. Using one in place, tracking upstream, is the +// deliberate minority choice. +// +// Two things follow, and both are here rather than in the generator path: +// +// The destination is `Collection.sourceRoots[0]` — the project's first DECLARED +// source root, resolved through `resolveCollection()`. Never `metaobjects/`: that +// string is the default value of `sources` and nothing else (CLAUDE.md names the five +// sites allowed to spell it, and this is not one of them). +// +// Every file carries a provenance header. It is what makes the copy's origin legible +// a year later, what `--list` reads to report staleness against the shipped tree, and +// what carries the ONE instruction that makes the eject complete: remove the library +// from `libraries`, or both trees load and merge (ERR_LIBRARY_PACKAGE_COLLISION). +import { mkdir, readFile, stat, writeFile } from "node:fs/promises"; +import { basename, join } from "node:path"; +import { MetaDataLoader, PACKAGE_SEPARATOR, canonicalSerialize, type MetaData } from "@metaobjectsdev/metadata"; +import { + libraryManifests, libraryRefSource, libraryRefs, librarySources, + knownLibraryTokens, splitLayerToken, +} from "@metaobjectsdev/metadata/library"; +import { resolveCollection } from "@metaobjectsdev/sdk"; +import { cliVersion } from "./version.js"; + +/** The first line of every ejected file — the marker `--list` scans for. */ +export const EJECT_MARKER = "# meta-eject:"; + +/** True when `name` is a shipped library rather than a generator. */ +export function isLibraryName(name: string): boolean { + return name in libraryManifests(); +} + +/** Every shipped library name, sorted. */ +export function ejectableLibraryNames(): string[] { + return Object.keys(libraryManifests()).sort(); +} + +/** + * Where an ejected ref lands: `meta...yaml`. + * + * Not the library's own basename. `model.yaml` / `db.yaml` / `requirements.yaml` are + * meaningful inside `library/iam/` and meaningless in a directory holding an + * application's whole model — and three libraries ejected into one root would collide + * outright. The `meta..<...>` shape is the file-naming convention this project + * documents. + */ +export function ejectedFileName(library: string, ref: string): string { + return `meta.${library}.${basename(ref)}.yaml`; +} + +function header(library: string, ref: string): string { + return [ + `${EJECT_MARKER} library=${library} ref=${ref} cli=${cliVersion()}`, + `# Copied from the MetaObjects shipped library "${library}". YOU OWN THIS FILE: rename`, + `# the package, delete what you do not need, change anything. Nothing regenerates it,`, + `# and no gate compares it to the shipped tree except \`meta eject --list\`.`, + `#`, + `# ONE STEP REMAINS: remove "${library}" (and any "${library}/") from`, + `# \`libraries\` in .metaobjects/config.json. Left there, the shipped tree and this`, + `# copy BOTH load and merge — additions take effect and deletions do not, because the`, + `# library still declares what you removed. The loader refuses that outright`, + `# (ERR_LIBRARY_PACKAGE_COLLISION) rather than letting it run.`, + "", + ].join("\n"); +} + +export interface EjectedLibraryFile { + ref: string; + /** Path relative to the project root. */ + path: string; + status: "created" | "preserved" | "replaced"; +} + +export interface LibraryEjectResult { + library: string; + /** Absolute destination directory. */ + root: string; + files: EjectedLibraryFile[]; + /** The tokens for this library the project currently declares — what to remove. */ + stillOptedIn: readonly string[]; +} + +async function fileExists(p: string): Promise { + try { + return (await stat(p)).isFile(); + } catch { + return false; + } +} + +export async function ejectLibrary(opts: { + cwd: string; + name: string; + force?: boolean; +}): Promise { + const manifest = libraryManifests()[opts.name]; + if (manifest === undefined) { + throw new Error( + `unknown library "${opts.name}". Shipped libraries: ${ejectableLibraryNames().join(", ")}.`, + ); + } + const collection = await resolveCollection(opts.cwd); + const root = collection.sourceRoots[0]; + if (root === undefined) { + throw new Error( + `this project declares no metadata source root, so there is nowhere to put ` + + `"${opts.name}". Declare "sources" in .metaobjects/config.json, or run 'meta init'.`, + ); + } + + await mkdir(root, { recursive: true }); + const files: EjectedLibraryFile[] = []; + for (const ref of libraryRefs(opts.name)) { + const text = header(opts.name, ref) + (await libraryRefSource(ref).read()); + const name = ejectedFileName(opts.name, ref); + const abs = join(root, name); + const existed = await fileExists(abs); + if (existed && opts.force !== true) { + files.push({ ref, path: name, status: "preserved" }); + continue; + } + await writeFile(abs, text, "utf8"); + files.push({ ref, path: name, status: existed ? "replaced" : "created" }); + } + + return { + library: opts.name, + root, + files, + stillOptedIn: collection.libraries.filter((t) => splitLayerToken(t)[0] === opts.name), + }; +} + +// --------------------------------------------------------------------------- +// staleness — `meta eject --list` +// --------------------------------------------------------------------------- + +export interface LibraryStaleness { + library: string; + /** Files in this project carrying the eject marker for this library. */ + files: string[]; + verdict: "identical" | "differs" | "unreadable"; + /** Nodes present in both whose own-mode serialization differs. */ + changed: number; + /** Nodes the SHIPPED tree has that the ejected copy does not — upstream moved. */ + upstreamOnly: number; + /** Nodes the ejected copy has that the shipped tree does not — your additions. */ + localOnly: number; + /** Why the comparison could not be made, when `verdict` is "unreadable". */ + reason?: string; + /** Still named in `libraries` — the ejection is not finished. */ + stillOptedIn: boolean; +} + +/** The longest package prefix every node shares — the library's own root package. + * + * Segment-wise, never by string prefix: `acme::identity` and `acme::identityhub` share + * six characters and no package at all. */ +function commonPackagePrefix(packages: readonly string[]): string { + if (packages.length === 0) return ""; + let common = packages[0]!.split(PACKAGE_SEPARATOR); + for (const pkg of packages.slice(1)) { + const segments = pkg.split(PACKAGE_SEPARATOR); + let i = 0; + while (i < common.length && i < segments.length && common[i] === segments[i]) i++; + common = common.slice(0, i); + } + return common.join(PACKAGE_SEPARATOR); +} + +/** Root-level nodes keyed by their package path RELATIVE to the library's own root + * package, canonically serialized in OWN mode. + * + * Relative rather than by resolution key, because §3.4 says an adopter who ejects "may + * rename the package freely" — keying on the FQN would report every node of a renamed + * copy as both upstream-only and local-only, which is the least useful answer + * available. Relative rather than by BARE NAME, because a library free to use two + * packages is free to declare `User` in both, and one key for two declarations means + * the one serialized last silently wins: a real divergence in the other is reported as + * `identical`. The relative path is what survives the rename — moving the root leaves + * every suffix below it exactly where it was — so it distinguishes without giving that + * edit anything to catch on. A single-package library yields the bare names it always + * did. Own mode, because the question is what each file DECLARES; the effective tree + * would fold a shipped base's fields into a subtype and report a difference that is not + * in either file. */ +export function ownNodesByName(root: MetaData): Map { + const nodes = [...root.children()]; + const packageOf = (node: MetaData) => node.package ?? node.fileDefaultPackage ?? ""; + // The ROOT package, neutralized in every node's serialization — for the same reason + // the keying is relative to it. A rename reaches the `package` key AND every `extends` + // inside the node, so comparing the raw text would report a renamed copy as wholly + // changed: the loudest possible answer to the one edit §3.4 explicitly invites. The + // root rather than each node's own package, so that a deeper node's reference back up + // to the root package is neutralized in it too. + const rootPackage = commonPackagePrefix(nodes.map(packageOf)); + + const out = new Map(); + for (const node of nodes) { + const pkg = packageOf(node); + const suffix = + rootPackage === "" ? pkg + : pkg === rootPackage ? "" + : pkg.slice(rootPackage.length + PACKAGE_SEPARATOR.length); + const key = suffix === "" ? node.name : `${suffix}${PACKAGE_SEPARATOR}${node.name}`; + const text = canonicalSerialize(node); + out.set(key, rootPackage === "" ? text : text.replaceAll(rootPackage, "")); + } + return out; +} + +/** Every project file carrying an eject marker, grouped by library. */ +async function ejectedFilesByLibrary(files: readonly string[]): Promise> { + const out = new Map(); + for (const file of files) { + let head: string; + try { + head = (await readFile(file, "utf8")).slice(0, 400); + } catch { + continue; + } + const match = new RegExp(`^${EJECT_MARKER} library=(\\S+)`, "m").exec(head); + const library = match?.[1]; + if (library === undefined) continue; + out.set(library, [...(out.get(library) ?? []), file]); + } + return out; +} + +/** + * Per-library staleness for the copies THIS project owns. + * + * The same gap `--list` closes for generators, one level up: an ejected copy can sit any + * number of releases behind the tree it came from and nothing compares the two. The + * comparison is structural, through the canonical serializer, so re-indentation and key + * order never show up — the only thing that can be reported is a declaration that + * actually changed. + */ +export async function libraryStaleness(cwd: string): Promise { + let collection; + try { + collection = await resolveCollection(cwd); + } catch { + return []; // no project — nothing is ejected here + } + const byLibrary = await ejectedFilesByLibrary(collection.ownFiles); + const selected = new Set(collection.libraries.map((t) => splitLayerToken(t)[0])); + + const rows: LibraryStaleness[] = []; + for (const [library, files] of [...byLibrary].sort(([a], [b]) => a.localeCompare(b))) { + const row: LibraryStaleness = { + library, + files: files.map((f) => f.slice(cwd.length + 1)), + verdict: "identical", + changed: 0, + upstreamOnly: 0, + localOnly: 0, + stillOptedIn: selected.has(library), + }; + if (!(library in libraryManifests())) { + rows.push({ ...row, verdict: "unreadable", reason: `this build ships no library "${library}"` }); + continue; + } + + // Both trees WHOLE, never file by file: a db layer is nothing but `overlay: true` + // redeclarations, so loading one alone is ERR_OVERLAY_NO_TARGET by construction. + const tokens = knownLibraryTokens().filter((t) => splitLayerToken(t)[0] === library); + const shipped = await new MetaDataLoader({ strict: true }).load(librarySources(tokens)); + const { FileSource } = await import("@metaobjectsdev/metadata/core"); + const local = await new MetaDataLoader({ strict: true }).load( + files.map((f) => new FileSource(f)), + ); + if (local.errors.length > 0) { + rows.push({ + ...row, + verdict: "unreadable", + reason: `the ejected copy does not load on its own: ${local.errors[0]!.message}`, + }); + continue; + } + + const up = ownNodesByName(shipped.root); + const mine = ownNodesByName(local.root); + for (const [name, text] of up) { + if (!mine.has(name)) row.upstreamOnly++; + else if (mine.get(name) !== text) row.changed++; + } + for (const name of mine.keys()) if (!up.has(name)) row.localOnly++; + row.verdict = + row.changed + row.upstreamOnly + row.localOnly === 0 ? "identical" : "differs"; + rows.push(row); + } + return rows; +} diff --git a/server/typescript/packages/cli/src/lib/library-listing.ts b/server/typescript/packages/cli/src/lib/library-listing.ts new file mode 100644 index 000000000..66d77e99f --- /dev/null +++ b/server/typescript/packages/cli/src/lib/library-listing.ts @@ -0,0 +1,258 @@ +// `meta gen --list` — the LIBRARY rows (FR-043 §4). +// +// It is the codegen catalog, with `kind: "library"` rows in the same table rather than +// a parallel one: one door, one namespace, one `--probe`, one skill procedure. An agent +// about to model "users and permissions" should meet `iam` in the list it was already +// reading, not in a second command it has to know exists. +// +// THE ROW DESCRIBES THE BOX; THE PROJECT BLOCK DESCRIBES YOUR SHELF. `provides` counts +// what the library ships across ALL its layers — what you would get if you took every +// layer — while `project.tablesAdded` counts what YOUR selection actually put into the +// model. Those are different numbers whenever an adopter takes the core layer alone, +// which Amendment 1 makes the common case: `libraries: ["iam"]` provides nine entities +// and adds zero tables. +// +// Every fact here is COMPUTED — from the manifest, or by loading the library and +// counting. Nothing is a second copy of a sentence written somewhere else, which is the +// rule §4 states as "every manifest fact is resolved, not trusted". +import { + MetaDataLoader, + TYPE_OBJECT, + TYPE_REQUIREMENT, + OBJECT_SUBTYPE_ENTITY, + isWritableSource, + type MetaData, +} from "@metaobjectsdev/metadata"; +import { + libraryManifests, + librarySources, + knownLibraryTokens, + splitLayerToken, + type LibraryManifest, +} from "@metaobjectsdev/metadata/library"; +import { SERVER_LANGS } from "@metaobjectsdev/sdk"; + +/** One `--list` row for a shipped library. */ +export interface LibraryCatalogRow { + name: string; + kind: "library"; + /** The manifest's own `kind` — what SORT of library this is (`feature`, `nfr`). A + * second axis from the row's `kind`, which is the catalog discriminator. */ + libraryKind: string; + stability: string; + /** The language ports this library is reachable from. Every port embeds the whole + * `library/` tree from one generator script, so this is all of them — and a test + * resolves that against each port's embedded module rather than trusting it. */ + ports: readonly string[]; + description: string; + useWhen?: string; + packages: readonly string[]; + /** Selection tokens, core first — `["iam", "iam/db"]`. The bare name IS the core + * layer, and a layer token implies it. This is the field an adopter acts on: under + * Amendment 1 you do not opt into a library, you opt into its layers. */ + layers: ReadonlyArray<{ token: string; description?: string }>; + /** What is in the box, across every layer. */ + provides: { + entities: number; + abstracts: number; + requirements: number; + generators: readonly string[]; + }; + project?: LibraryProjectFacts; +} + +export interface LibraryProjectFacts { + /** Does this project's `libraries` name this library at all? */ + optedIn: boolean; + /** The tokens this project selected, in config order. Empty when not opted in. */ + selectedLayers: readonly string[]; + /** Project entities whose `extends` chain reaches a node this library owns — the + * adoption that opting out would break. `null` unless the model was loaded. */ + extendedBy: readonly string[] | null; + /** Tables this library's SELECTED layers contribute to the loaded model. Zero for a + * core-only selection, which is the inertness promise expressed as a number. */ + tablesAdded: number | null; + /** Requirement entries the library put into this project's ledger. */ + requirementsAdded: number | null; + /** Generators the manifest says this library implies, which this project has not + * wired. Not an error: the library is metadata, and wiring is the adopter's call. */ + impliedGeneratorsNotWired: readonly string[]; +} + +/** What a `--list` caller knows about the project, for the library rows. */ +export interface LibraryProjectContext { + /** The project's `libraries` selection, verbatim. */ + libraries: readonly string[]; + /** Stable names wired in `generators: [...]`. */ + wiredNames: ReadonlySet; + /** The loaded model, when one was loaded (`--probe`). */ + metadata?: MetaData; +} + +/** Every selection token belonging to one library, core first. */ +function tokensOf(name: string): string[] { + return knownLibraryTokens().filter((t) => splitLayerToken(t)[0] === name); +} + +/** Walk every node, since a requirement nests to any depth and an object does not. */ +function walk(node: MetaData, visit: (n: MetaData) => void): void { + for (const child of node.children()) { + visit(child); + walk(child, visit); + } +} + +function effectivePackage(node: MetaData): string { + return node.package ?? node.fileDefaultPackage ?? ""; +} + +/** + * Load one library with ALL of its layers and count what it ships. + * + * The FULL selection, not the core: `provides` answers "what is in this library", + * and an adopter reading it is deciding whether to opt in at all. What their current + * selection actually added is the project block's job. + */ +async function providesOf(name: string, manifest: LibraryManifest): Promise { + const result = await new MetaDataLoader({ strict: true }).load(librarySources(tokensOf(name))); + let entities = 0; + let abstracts = 0; + let requirements = 0; + walk(result.root, (n) => { + if (n.type === TYPE_REQUIREMENT) requirements++; + else if (n.type === TYPE_OBJECT && n.subType === OBJECT_SUBTYPE_ENTITY) { + if (n.isAbstract) abstracts++; + else entities++; + } + }); + return { + entities, + abstracts, + requirements, + generators: (manifest.generators ?? []).map((g) => g.name), + }; +} + +/** + * The project facts for one library. + * + * `extendedBy` and the two counts read the LOADED model rather than the library, + * because that is the question being answered: not "what does iam contain" but "what + * is iam doing in this repo". Without `--probe` there is no model and they are null — + * never zero, which would read as "nothing", the opposite of "not measured". + */ +function projectFactsFor( + manifest: LibraryManifest, + ctx: LibraryProjectContext, +): LibraryProjectFacts { + const name = manifest.name; + const selectedLayers = ctx.libraries.filter((t) => splitLayerToken(t)[0] === name); + const impliedGeneratorsNotWired = (manifest.generators ?? []) + .map((g) => g.name) + .filter((g) => !ctx.wiredNames.has(g)); + + const facts: LibraryProjectFacts = { + optedIn: selectedLayers.length > 0, + selectedLayers, + extendedBy: null, + tablesAdded: null, + requirementsAdded: null, + impliedGeneratorsNotWired, + }; + if (ctx.metadata === undefined) return facts; + + const owned = new Set(manifest.packages ?? []); + const extendedBy: string[] = []; + let tablesAdded = 0; + let requirementsAdded = 0; + walk(ctx.metadata, (n) => { + const mine = owned.has(effectivePackage(n)); + if (n.type === TYPE_REQUIREMENT && mine) requirementsAdded++; + if (n.type !== TYPE_OBJECT) return; + if (mine) { + // A writable source is what migrate keys a CREATE TABLE off (#248 — + // persistability derives from source presence, never from the subtype), so + // counting them here and counting tables there cannot drift apart. + if (n.children().some(isWritableSource)) tablesAdded++; + return; + } + // A project entity reaching this library through `extends`, at any depth: the + // adoption an adopter would break by removing the library from `libraries`. + for (let sup = n.superData; sup !== undefined; sup = sup.superData) { + if (owned.has(effectivePackage(sup))) { + extendedBy.push(`${effectivePackage(n)}::${n.name}`); + return; + } + } + }); + + facts.extendedBy = extendedBy.sort(); + facts.tablesAdded = tablesAdded; + facts.requirementsAdded = requirementsAdded; + return facts; +} + +/** Every shipped library, as `--list` rows, in name order. */ +export async function buildLibraryRows( + ctx?: LibraryProjectContext, +): Promise { + const rows: LibraryCatalogRow[] = []; + for (const [name, manifest] of Object.entries(libraryManifests()).sort(([a], [b]) => + a.localeCompare(b), + )) { + const layers = tokensOf(name).map((token) => { + const layer = (manifest.layers ?? {})[splitLayerToken(token)[1]]; + return layer?.description === undefined + ? { token } + : { token, description: layer.description }; + }); + rows.push({ + name, + kind: "library", + libraryKind: manifest.kind ?? "feature", + stability: manifest.stability ?? "preview", + ports: SERVER_LANGS, + description: manifest.description ?? "", + ...(manifest.useWhen !== undefined ? { useWhen: manifest.useWhen } : {}), + packages: manifest.packages ?? [], + layers, + provides: await providesOf(name, manifest), + ...(ctx === undefined ? {} : { project: projectFactsFor(manifest, ctx) }), + }); + } + return rows; +} + +function plural(n: number, one: string, many: string): string { + return `${n} ${n === 1 ? one : many}`; +} + +/** The human rendering — one section, after the generator layers. */ +export function renderLibraryText(rows: readonly LibraryCatalogRow[]): string[] { + if (rows.length === 0) return []; + const lines: string[] = []; + lines.push(""); + lines.push("libraries — declared design you opt into, a LAYER at a time; the core layer adds no tables"); + for (const r of rows) { + const marks: string[] = [r.stability]; + if (r.project?.optedIn) marks.push(`opted in: ${r.project.selectedLayers.join(", ")}`); + if (r.project?.tablesAdded) marks.push(`${plural(r.project.tablesAdded, "table", "tables")} added`); + lines.push(` ${r.name} — ${r.description} [${marks.join(", ")}]`); + if (r.useWhen !== undefined) lines.push(` use when: ${r.useWhen}`); + lines.push( + ` provides: ${plural(r.provides.entities, "entity", "entities")}, ` + + `${r.provides.abstracts} abstract, ` + + `${plural(r.provides.requirements, "requirement", "requirements")}`, + ); + lines.push(` layers: ${r.layers.map((l) => l.token).join(", ")}`); + if (r.project !== null && r.project?.impliedGeneratorsNotWired.length) { + lines.push( + ` implies: ${r.project.impliedGeneratorsNotWired.join(", ")} (not wired)`, + ); + } + } + lines.push(""); + lines.push('Opt in with `"libraries": ["iam", "iam/db"]` in .metaobjects/config.json —'); + lines.push("the bare name is the core layer, and a layer token implies it."); + return lines; +} diff --git a/server/typescript/packages/cli/src/lib/library-prefix-advisory.ts b/server/typescript/packages/cli/src/lib/library-prefix-advisory.ts new file mode 100644 index 000000000..65fe77d79 --- /dev/null +++ b/server/typescript/packages/cli/src/lib/library-prefix-advisory.ts @@ -0,0 +1,78 @@ +// FR-043 §3.5 — `metaobjects::` in an adopter's own metadata, with no ejection provenance. +// +// The loader already REFUSES a node in a library's package while that library is opted +// in (ERR_LIBRARY_PACKAGE_COLLISION / ERR_LIBRARY_PACKAGE_NOT_OWNED). This is the other +// half, and it is advisory on purpose: with the library NOT opted in there is nothing +// broken to refuse — the nodes are simply the adopter's, sitting under a prefix this +// project ships libraries into. +// +// Two ways to arrive there, and the advice is the same for both: +// +// a hand-copy, rather than `meta eject ` — no provenance header, so nothing +// can tell later where it came from or how far it has drifted; +// a package名 chosen freely that happens to start `metaobjects::` — which will +// collide the day a library ships into it. +// +// A file carrying the eject marker is exempt: that IS the provenance, and `meta eject +// --list` reports its staleness. +import { readFile } from "node:fs/promises"; +import { EJECT_MARKER } from "./library-eject.js"; +import type { MetaData } from "@metaobjectsdev/metadata"; + +/** The prefix every shipped library declares into. */ +export const LIBRARY_PACKAGE_PREFIX = "metaobjects::"; + +export interface LibraryPrefixFinding { + file: string; + fqn: string; + message: string; +} + +/** + * Root nodes under `metaobjects::` whose file carries no ejection provenance. + * + * Reads the loaded tree for the nodes and the FILES only for the marker — the marker is + * `meta eject`'s own bookkeeping, not metadata, so there is nothing in the model to read + * it from. + */ +export async function scanForUnprovenancedLibraryPrefix( + root: MetaData, + ownFiles: readonly string[], + projectRoot: string, +): Promise { + const suspect = root + .children() + .filter((n) => n.resolutionKey().startsWith(LIBRARY_PACKAGE_PREFIX)); + if (suspect.length === 0) return []; + + // Which of this project's own files carry the marker — read once, not per node. + const provenanced = new Set(); + for (const file of ownFiles) { + try { + if ((await readFile(file, "utf8")).slice(0, 400).includes(EJECT_MARKER)) { + provenanced.add(file.slice(projectRoot.length + 1)); + } + } catch { + // unreadable — it contributes no exemption, which is the safe direction + } + } + + const out: LibraryPrefixFinding[] = []; + for (const node of suspect) { + const files = "files" in node.source ? node.source.files : []; + // A node the SHIPPED library contributed is not the adopter's — `library:` ids say so. + if (files.some((f) => f.startsWith("library:"))) continue; + if (files.some((f) => provenanced.has(f) || [...provenanced].some((p) => p.endsWith(f)))) continue; + out.push({ + file: files[0] ?? "(unknown)", + fqn: node.resolutionKey(), + message: + `${node.resolutionKey()} is declared under "${LIBRARY_PACKAGE_PREFIX}", which is where ` + + `MetaObjects ships its libraries, and no file declaring it carries an ejection header. ` + + `If it was copied from a library, re-copy it with 'meta eject ' so its origin ` + + `and its drift stay legible; if it is your own, move it to a package you own — a later ` + + `release shipping a node of that name would merge into yours.`, + }); + } + return out; +} diff --git a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts index c3bd9cf22..5b5ee3532 100644 --- a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts +++ b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts @@ -249,11 +249,10 @@ function rewriteImportSpecifiers(source: string, aliasMap: Record | undefined, -): { providers?: readonly MetaDataTypeProvider[]; libraries?: readonly string[] } { + cfg: Pick | undefined, +): { providers?: readonly MetaDataTypeProvider[] } { return { ...(cfg?.providers !== undefined ? { providers: cfg.providers } : {}), - ...(cfg?.libraries !== undefined ? { libraries: cfg.libraries } : {}), }; } @@ -518,24 +517,19 @@ export async function loadMetaobjectsConfig(projectRoot: string): Promise 0) { - const { knownLibraryPackages } = await import("@metaobjectsdev/metadata/library"); - const available = knownLibraryPackages(); - const unknown = cfg.libraries.filter((n) => !available.includes(n)); - if (unknown.length > 0) { - throw new Error( - `metaobjects.config.ts at ${fullPath}: 'libraries' names unknown package(s) ` + - `${JSON.stringify(unknown)}; available: ${JSON.stringify(available)}.`, - ); - } + // `libraries` used to be validated here. It moved to `.metaobjects/config.json` + // (FR-043 Amendment 1), and so did its validation — see `assertKnownLibraries`, + // called from `resolveCollection`'s readers. A config still carrying the old key + // gets a pointed error rather than silence, because silence is what turns a moved + // key into "my library stopped loading and nothing said why". + if ("libraries" in (cfg as unknown as Record)) { + throw new Error( + `metaobjects.config.ts at ${fullPath}: 'libraries' moved to ` + + `.metaobjects/config.json (FR-043). Which designs a project adopts is a fact ` + + `about the PROJECT, not about how one port generates code from it, and that ` + + `file is the port-neutral one every port already reads. Move the array across ` + + `verbatim — the tokens are unchanged.`, + ); } return cfg; } finally { diff --git a/server/typescript/packages/cli/src/lib/requirement-check.ts b/server/typescript/packages/cli/src/lib/requirement-check.ts index 6c396a2a9..35952360e 100644 --- a/server/typescript/packages/cli/src/lib/requirement-check.ts +++ b/server/typescript/packages/cli/src/lib/requirement-check.ts @@ -42,6 +42,10 @@ import { type MetaRequirement, type RequirementStatus, } from "@metaobjectsdev/metadata"; +// FR-043 §5.4 — the provenance key for coverage activation. A node-only subpath, which +// is why it is a second import rather than a name on the barrel above: the library +// module reaches the filesystem and the root barrel stays browser-safe. +import { libraryPackages } from "@metaobjectsdev/metadata/library"; export type Severity = "error" | "warn"; @@ -82,8 +86,11 @@ export interface RequirementSummary { undecided: number; /** deferred entries naming no ticket, so nobody will be reminded. */ deferredUntracked: number; - entitiesTotal: number; - entitiesClaimed: number; + /** ABSENT when coverage was not measured — FR-043 §5.4. A number here is a + * ratio the project is held to; absence is the honest reading of "this project + * has authored no requirement of its own, so it asked to be held to none". */ + entitiesTotal?: number; + entitiesClaimed?: number; } /** Severity of the object-coverage gate. Promotion to `"error"` is a one-line @@ -255,16 +262,61 @@ export interface RequirementScan { * Threaded through so BOTH `coverableEntities` call sites (the gate and the * summary) inherit the same narrowing — see `coverableEntities`. */ readonly coverable?: (fqn: string) => boolean; + /** FR-043 §5.4 — whether object coverage applies at all on this run. See + * {@link projectAuthoredRequirements}. Read by BOTH the gate and the summary from + * this one scan, for the same reason `claimedObjects` is shared: a summary that + * printed a ratio the gate had not enforced would be a measurement nobody could + * reconcile with the diagnostics beneath it. */ + readonly measureCoverage: boolean; +} + +/** + * Did the ADOPTER author any of these requirements? + * + * FR-043 §5.4, and the rule exists because `checkRequirements` activates object + * coverage on the presence of any requirement at all. A library ships its own ledger + * — `iam` ships eleven entries — so without this, opting into a library would switch + * the unclaimed-entity gate on across a project that has never written a requirement, + * and report every entity in it. The gate would then be measuring the LIBRARY's + * decision to ship a ledger rather than anything the adopter did. + * + * Provenance is the library's declared PACKAGE. That is a manifest fact the standalone + * library gate resolves against the library loaded alone, whereas a node's source id + * differs between the on-disk dev layout (an absolute path) and the embedded one + * (`library:.yaml`) — a rule keyed on source would hold in this repo and stop + * holding in an installed build. + * + * An adopter OVERLAYING a library requirement (§5.5) is deliberately not authoring one: + * the overlay merges into the library's node, in the library's package, and disagreeing + * with a shipped verdict is a statement about the library's design rather than about + * what the adopter's own model is for. Treating it as activation would mean correcting + * one library entry silently acquired a coverage gate over the whole estate. + */ +function projectAuthoredRequirements(addressed: readonly AddressedRequirement[]): boolean { + const libPkgs = libraryPackages(); + return addressed.some((r) => { + const pkg = r.node.package ?? r.node.fileDefaultPackage ?? ""; + return !libPkgs.has(pkg); + }); } export function scanRequirements( root: MetaData, - opts?: { coverable?: (fqn: string) => boolean }, + opts?: { + coverable?: (fqn: string) => boolean; + /** Force coverage on or off instead of deriving it. The one caller that legitimately + * knows better is the gate over a SHIPPED library loaded standalone: there the + * library IS the project under test, and "every entity this library ships is claimed + * by its own ledger" is exactly what is being asserted — so the derivation would + * switch the check off precisely where it is the point. */ + measureCoverage?: boolean; + }, ): RequirementScan { const addressed = collectAddressedRequirements(root); return { addressed, claimedObjects: claimedObjectKeys(root, addressed.map((r) => r.node)), + measureCoverage: opts?.measureCoverage ?? projectAuthoredRequirements(addressed), // `exactOptionalPropertyTypes` — an omitted key, never an explicit `undefined`. ...(opts?.coverable !== undefined ? { coverable: opts.coverable } : {}), }; @@ -350,7 +402,7 @@ function coverableEntities(root: MetaData, coverable?: (fqn: string) => boolean) */ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRequirements(root)): Diagnostic[] { const out: Diagnostic[] = []; - const { addressed, claimedObjects, coverable } = scan; + const { addressed, claimedObjects, coverable, measureCoverage } = scan; if (addressed.length === 0) return out; // opt-in by declaration — no requirements, nothing to say for (const { node: req, path: reqPath } of addressed) { @@ -564,7 +616,9 @@ export function checkRequirements(root: MetaData, scan: RequirementScan = scanRe // // So a green run means "every entity is claimed by something", not "every node is // described". The stronger reading would be false. - for (const ent of coverableEntities(root, coverable)) { + // ADOPTER-AUTHORED ONLY. FR-043 §5.4 — see `projectAuthoredRequirements`. A library's + // ledger is counted and checked, but it cannot volunteer you for coverage. + for (const ent of measureCoverage ? coverableEntities(root, coverable) : []) { const key = ent.resolutionKey(); if (!claimedObjects.has(key)) { out.push({ @@ -601,8 +655,8 @@ export function summariseRequirements( byStatus: {}, undecided: 0, deferredUntracked: 0, - entitiesTotal: 0, - entitiesClaimed: 0, + // The coverage pair is filled in below only when the run measures coverage — + // `exactOptionalPropertyTypes`, so the keys are absent rather than undefined. }; for (const req of reqs) { @@ -621,10 +675,16 @@ export function summariseRequirements( // Both sides of the ratio come from the SAME scan the gate read, so the printed // summary cannot disagree with the diagnostics printed beneath it — previously // the same helper, now literally the same result. - const claimed = scan.claimedObjects; - for (const ent of coverableEntities(root, scan.coverable)) { - summary.entitiesTotal++; - if (claimed.has(ent.resolutionKey())) summary.entitiesClaimed++; + if (scan.measureCoverage) { + const claimed = scan.claimedObjects; + let total = 0; + let claimedCount = 0; + for (const ent of coverableEntities(root, scan.coverable)) { + total++; + if (claimed.has(ent.resolutionKey())) claimedCount++; + } + summary.entitiesTotal = total; + summary.entitiesClaimed = claimedCount; } return summary; diff --git a/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap b/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap index 80da2b40b..6c76fc241 100644 --- a/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap +++ b/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap @@ -30,7 +30,7 @@ COMMANDS: GLOBAL OPTIONS: --cwd , -C Run as if launched from (default: current directory) --format Output format (default: toon on non-TTY, text on TTY). - Honored by gen, verify, migrate, types, deps; every other + Honored by gen, verify, migrate, types, deps, eject; every other command prints text and says so if you pass it. \`types\` is the one exception to the default: it prints TEXT unless you ask for a format, on a TTY or not, because its @@ -40,8 +40,14 @@ GLOBAL OPTIONS: GEN FLAGS: --dry-run Compute and print, don't write - --list Print the generator registry (name, tier, options) and exit — - no config or metadata required + --list Print the generator CATALOG (name, layer, tier, what it emits, + what to install) and exit. Codegen is opt-in: nothing runs until + you wire it, and this is where you choose. No config or metadata + required — add --format json for the same catalog, machine-readable. + --probe With --list: construct every catalog generator and dry-run it + against YOUR model, reporting how many files each would emit. + A count per generator beats any category label — and it + cannot go stale, because it runs the generators. Needs a project. --baseline First-time-on-existing-file behavior. Default: refuse a file that cannot be proved to be generated output. "adopt" records the files diff --git a/server/typescript/packages/cli/test/catalog-conformance.test.ts b/server/typescript/packages/cli/test/catalog-conformance.test.ts new file mode 100644 index 000000000..9e4cf2726 --- /dev/null +++ b/server/typescript/packages/cli/test/catalog-conformance.test.ts @@ -0,0 +1,145 @@ +// Conformance gate (ADR-0021 D3): the COMPOSED TypeScript catalog must equal the +// canonical cross-port manifest's `typescript` slice, both ways. +// +// This assertion cannot live in `codegen-ts`. That package registers one of three +// slices and cannot import its own dependents, so it can only check "no rogue names" +// (which it does, in test/golden/generator-registry-conformance.test.ts). The CLI is the +// only package that sees all three, so set equality — the half that catches a MISSING +// registration — belongs here. +// +// If this fails, the manifest and the composed catalog DISAGREE: report the diff; do +// NOT mutate the manifest to force a pass (it is reconciled cross-port, not per-port). + +import { describe, it, expect } from "bun:test"; +import { existsSync, readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { GENERATOR_LAYERS, stableNameIndex } from "@metaobjectsdev/codegen-ts"; +import { composeCatalog, listCatalog, packageOf, catalogPackages } from "../src/lib/catalog.js"; + +const PORT = "typescript" as const; + +// Walk UP until a directory holds BOTH fixtures/ and server/ — the repo root. +// (Same strategy as every other conformance test; no hard-coded absolute paths.) +function findRepoRoot(start: string): string { + let dir = start; + for (;;) { + if (existsSync(join(dir, "fixtures")) && existsSync(join(dir, "server"))) return dir; + const parent = dirname(dir); + if (parent === dir) { + throw new Error("Could not locate repo root (dir containing fixtures/ and server/)"); + } + dir = parent; + } +} + +interface ManifestEntry { + concept: string; + tier: "native" | "neutral"; + layer: string; + note?: string; + ports: string[]; +} + +const manifestPath = join( + findRepoRoot(import.meta.dir), + "fixtures", + "generator-registry-conformance", + "registry.json", +); +const manifest = JSON.parse(readFileSync(manifestPath, "utf-8")) as { + ports: string[]; + generators: Record; +}; + +const expectedNames = new Set( + Object.entries(manifest.generators) + .filter(([, e]) => e.ports.includes(PORT)) + .map(([n]) => n), +); + +describe("the composed TS catalog conforms to the canonical manifest", () => { + it(`composed catalog == manifest's ${PORT} slice (no rogue, no missing)`, () => { + const actualNames = new Set(Object.keys(composeCatalog())); + const extra = [...actualNames].filter((n) => !expectedNames.has(n)).sort(); + const missing = [...expectedNames].filter((n) => !actualNames.has(n)).sort(); + + const message = [ + `The composed TypeScript catalog disagrees with the canonical manifest.`, + ` extra (registered by a slice, but the manifest's ${PORT} omits it): [${extra.join(", ")}]`, + ` missing (manifest expects ${PORT}, no slice registers it): [${missing.join(", ")}]`, + ` slices composed: ${catalogPackages().join(", ")}`, + ` manifest: ${manifestPath}`, + ].join("\n"); + + expect({ extra, missing }, message).toEqual({ extra: [], missing: [] }); + }); + + it("tier and layer agree with the manifest, entry by entry", () => { + const catalog = composeCatalog(); + const disagreements: string[] = []; + for (const name of expectedNames) { + const entry = catalog[name]; + if (entry === undefined) continue; // reported by the set-equality test above + const m = manifest.generators[name]!; + if (entry.tier !== m.tier) disagreements.push(`${name}.tier: ${entry.tier} != ${m.tier}`); + if (entry.layer !== m.layer) disagreements.push(`${name}.layer: ${entry.layer} != ${m.layer}`); + } + expect(disagreements).toEqual([]); + }); + + it("every catalog entry is attributable to exactly one package", () => { + for (const name of Object.keys(composeCatalog())) { + expect(packageOf(name), `packageOf(${name})`).toBeDefined(); + } + }); + + it("composition refuses a duplicate stable name", () => { + // The live slices must not collide; the throw path is what makes that a build + // failure rather than last-slice-wins. + expect(() => composeCatalog()).not.toThrow(); + }); + + it("listCatalog() is grouped by layer, in the declared layer order", () => { + const seen: string[] = []; + for (const e of listCatalog()) { + if (seen[seen.length - 1] !== e.layer) seen.push(e.layer); + } + // Each layer appears exactly once (i.e. the list is grouped, not interleaved) and + // in GENERATOR_LAYERS order. + expect(new Set(seen).size).toBe(seen.length); + expect(seen).toEqual(GENERATOR_LAYERS.filter((l) => seen.includes(l))); + }); + + it("every catalog entry resolves back from the generator its factory builds", () => { + // The runner sees CONSTRUCTED generators, whose `name` is the implementation name + // (`routes-file`, `tanstack-grid-hook`), not the catalog key (`routes`, + // `grid-hook`). Both post-selection gates map back through stableNameIndex(), and + // an entry missing from that index is skipped in SILENCE — indistinguishable from + // somebody's own generator. So the index has to be total, and that is asserted + // here rather than left to the two gates to fail vaguely. + const catalog = composeCatalog(); + const index = stableNameIndex(catalog); + const unreachable: string[] = []; + for (const [stable, entry] of Object.entries(catalog)) { + const implName = entry.factory().name; + if (index.get(implName) !== stable) { + unreachable.push(`${stable}: its factory builds "${implName}", which resolves to ${String(index.get(implName))}`); + } + } + expect( + unreachable, + "A catalog entry whose constructed generator does not map back to it. The\n" + + "requires and api-framework gates would skip it silently:\n " + unreachable.join("\n "), + ).toEqual([]); + }); + + it("every one of the six layers has at least one member", () => { + const populated = new Set(listCatalog().map((e) => e.layer)); + // A layer with no member in ANY port would be dead vocabulary; a layer with no + // member in THIS port is legitimate, so this asserts against the manifest, not the + // TS catalog. + const manifestLayers = new Set(Object.values(manifest.generators).map((e) => e.layer)); + expect([...GENERATOR_LAYERS].filter((l) => !manifestLayers.has(l))).toEqual([]); + expect(populated.size).toBeGreaterThan(0); + }); +}); diff --git a/server/typescript/packages/cli/test/catalog-declarations-resolved.test.ts b/server/typescript/packages/cli/test/catalog-declarations-resolved.test.ts new file mode 100644 index 000000000..870766587 --- /dev/null +++ b/server/typescript/packages/cli/test/catalog-declarations-resolved.test.ts @@ -0,0 +1,277 @@ +// Every compatibility declaration in the catalog is RESOLVED, not trusted. +// +// This is the `@implementedBy` doctrine applied to the catalog. A declaration is a +// promise someone has to remember to keep, and "someone remembers" scales badly across +// five ports and a growing framework set — so `requires` and `runtimePeers` are checked +// against what the generators ACTUALLY EMIT, by running them. +// +// Note what this does NOT do. It never asks whether a generator applies to a model — +// that is `--probe`, which cannot drift because it runs the generators rather than +// describing them. This file is the other half: what a generator NEEDS in order to +// work, which is declared and therefore has to be gated. Conflating the two is how a +// catalog goes quietly wrong as frameworks are added (design §D5). +// +// The gates are deliberately SUBSET checks (declared ⊇ actual), not equality. Over- +// declaring is a documentation defect an adopter can shrug off; under-declaring hands +// them a file that does not compile. + +import { describe, test, expect, beforeAll } from "bun:test"; +import { mkdtempSync, rmSync, readFileSync, readdirSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, relative, dirname, resolve, sep } from "node:path"; +import { loadMemory } from "@metaobjectsdev/sdk"; +import { runGen, type MetaobjectsGenConfig } from "@metaobjectsdev/codegen-ts"; +import * as coreTpl from "@metaobjectsdev/codegen-ts"; +import * as reactTpl from "@metaobjectsdev/codegen-ts-react"; +import * as tanstackTpl from "@metaobjectsdev/codegen-ts-tanstack"; +import { composeCatalog } from "../src/lib/catalog.js"; + +const FIXTURE = join(import.meta.dir, "fixtures", "catalog-probe"); +const OUT_DIR = "src/generated"; + +/** Emitted files for one generator: project-relative path → contents. */ +type Emission = Map; + +/** Every generator's emission, keyed by stable name. Empty when it could not run. */ +const emissions = new Map(); +/** Which generator emitted a given project-relative path (first writer wins). */ +const owners = new Map(); + +/** A path with its extension removed — the form an import specifier can be matched to. + * Generated code imports `./Customer.js`; the file on disk is `Customer.ts`. */ +function stem(p: string): string { + return p.replace(/\.[cm]?[jt]sx?$/, ""); +} + +function walkFiles(root: string, dir = root): string[] { + const out: string[] = []; + for (const entry of readdirSync(dir)) { + const abs = join(dir, entry); + if (statSync(abs).isDirectory()) out.push(...walkFiles(root, abs)); + else out.push(relative(root, abs)); + } + return out; +} + +/** Every import/export specifier in a source file. */ +function specifiersOf(source: string): string[] { + const found: string[] = []; + for (const m of source.matchAll(/(?:^|\n)\s*(?:import|export)[\s\S]{0,400}?from\s+["']([^"']+)["']/g)) { + if (m[1] !== undefined) found.push(m[1]); + } + for (const m of source.matchAll(/\bimport\(\s*["']([^"']+)["']\s*\)/g)) { + if (m[1] !== undefined) found.push(m[1]); + } + return found; +} + +/** The npm PACKAGE a bare specifier names — `zod/v4` → `zod`, `@scope/x/y` → `@scope/x`. */ +function packageOfSpecifier(spec: string): string { + const parts = spec.split("/"); + return spec.startsWith("@") ? parts.slice(0, 2).join("/") : parts[0]!; +} + +beforeAll(async () => { + const metadata = await loadMemory(FIXTURE, {}); + const config: MetaobjectsGenConfig = { + outDir: OUT_DIR, + extStyle: "js", + dialect: "sqlite", + dbImport: "../db", + generators: [], + }; + + for (const [name, entry] of Object.entries(composeCatalog())) { + const root = mkdtempSync(join(tmpdir(), `catalog-emit-${name}-`)); + try { + // A REAL write, not a dry run: this gate reads the emitted CONTENT, and a + // dry run reports paths only. + await runGen({ + config: { ...config, generators: [entry.factory()] }, + metadata, + projectRoot: root, + genStateDir: join(root, ".gen-state"), + }); + const emission: Emission = new Map(); + for (const rel of walkFiles(root)) { + if (rel.split(sep)[0] === ".gen-state") continue; + emission.set(rel, readFileSync(join(root, rel), "utf8")); + if (!owners.has(rel)) owners.set(rel, name); + } + emissions.set(name, emission); + } catch { + // A generator that cannot run from a bare fixture (render-helper needs a + // template root, shared-model needs a `files` selection) contributes nothing + // and is reported by the coverage test below rather than silently skipped. + emissions.set(name, new Map()); + } finally { + rmSync(root, { recursive: true, force: true }); + } + } +}); + +describe("catalog declarations are resolved against what the generators emit", () => { + test("the harness actually ran — most generators emitted something", () => { + const produced = [...emissions.entries()].filter(([, e]) => e.size > 0).map(([n]) => n); + // If this ever collapses to a handful, every subset check below is vacuously + // green and the whole file stops gating anything. + expect(produced.length).toBeGreaterThan(12); + }); + + test("exactly these generators contribute NO evidence — the list may only shrink", () => { + // A generator that emits nothing here makes every subset check below vacuously + // green for it, so its declarations are UNVERIFIED. That is a real coverage hole + // and it is pinned rather than left implicit: a corpus that quietly loses coverage + // fails nothing. + // + // Why each is here, and what it would take to close it: + // render-helper needs an on-disk template root for its build-time drift gate + // shared-model needs a `files` selection `meta gen` supplies at run time + // trace-helper needs an entity extending metaobjects::ai::LlmCallBase + // docs, api-docs emit under a docs config this bare fixture does not carry + // template a PRIMITIVE — registered with a no-op walk, by construction + // requirement-tests emits stubs, which this harness writes but reads no imports + // from beyond `bun:test` (a builtin, deliberately skipped) + const silent = [...emissions.entries()].filter(([, e]) => e.size === 0).map(([n]) => n).sort(); + expect(silent).toEqual([ + "api-docs", + "docs", + "render-helper", + "shared-model", + "template", + "trace-helper", + ]); + }); + + test("runtimePeers ⊇ the third-party packages the emitted files import", () => { + const catalog = composeCatalog(); + const undeclared: string[] = []; + + for (const [name, emission] of emissions) { + const declared = new Set(catalog[name]!.runtimePeers ?? []); + const seen = new Set(); + for (const source of emission.values()) { + for (const spec of specifiersOf(source)) { + // `node:` and `bun:` are runtime builtins, not packages an adopter installs. + // `bun:test` in particular is what a requirement-tests STUB imports, and + // telling anyone to `npm i bun:test` would be nonsense. + if (spec.startsWith(".") || spec.startsWith("node:") || spec.startsWith("bun:")) continue; + const pkg = packageOfSpecifier(spec); + // `@metaobjectsdev/*` is `runtimePackage`, not a third-party peer, and is + // asserted separately below. + if (pkg.startsWith("@metaobjectsdev/")) continue; + if (!declared.has(pkg)) seen.add(pkg); + } + } + for (const pkg of [...seen].sort()) undeclared.push(`${name} emits an import of "${pkg}"`); + } + + expect( + undeclared, + "A generator's output imports a third-party package its catalog entry does not\n" + + "declare, so `meta eject` will not tell an adopter to install it and their first\n" + + "tsc reports TS2307. Add each to that entry's `runtimePeers`:\n " + + undeclared.join("\n "), + ).toEqual([]); + }); + + test("runtimePackages ⊇ the @metaobjectsdev runtimes the emitted code imports", () => { + const catalog = composeCatalog(); + const undeclared: string[] = []; + + for (const [name, emission] of emissions) { + const declared = new Set(catalog[name]!.runtimePackages ?? []); + const seen = new Set(); + for (const source of emission.values()) { + for (const spec of specifiersOf(source)) { + const pkg = packageOfSpecifier(spec); + if (!pkg.startsWith("@metaobjectsdev/")) continue; + if (!declared.has(pkg)) seen.add(pkg); + } + } + for (const pkg of [...seen].sort()) { + undeclared.push(`${name} emits an import of "${pkg}" but does not declare it in runtimePackages`); + } + } + + expect( + undeclared, + "Generated code imports a MetaObjects runtime the catalog does not name, so an\n" + + "adopter is never told to install it:\n " + undeclared.join("\n "), + ).toEqual([]); + }); + + test("requires ⊇ the generators whose emitted paths this generator's output imports", () => { + const catalog = composeCatalog(); + const undeclared: string[] = []; + + for (const [name, emission] of emissions) { + const declared = new Set(catalog[name]!.requires ?? []); + const needed = new Set(); + for (const [rel, source] of emission) { + for (const spec of specifiersOf(source)) { + if (!spec.startsWith(".")) continue; + const target = stem(resolve("/", dirname(rel), spec).slice(1)); + for (const [ownedPath, owner] of owners) { + // A relative import that resolves to nothing any generator emits is a + // reference to the adopter's OWN code (routes' `dbImport`, for one) — + // config, not a generator dependency, so it is deliberately not a + // `requires` edge. + if (stem(ownedPath) !== target) continue; + if (owner !== name && !declared.has(owner)) needed.add(owner); + } + } + } + for (const dep of [...needed].sort()) { + undeclared.push(`${name} imports a module "${dep}" emits, but does not declare requires: [..., "${dep}"]`); + } + } + + expect( + undeclared, + "A generator depends on another generator's output without saying so, so the\n" + + "requires gate cannot warn an adopter who wires one without the other:\n " + + undeclared.join("\n "), + ).toEqual([]); + }); + + test("every ejectable entry has a template, and every template has an entry", () => { + const catalog = composeCatalog(); + const templates = new Set([ + ...coreTpl.REFERENCE_GENERATOR_NAMES, + ...reactTpl.REFERENCE_GENERATOR_NAMES, + ...tanstackTpl.REFERENCE_GENERATOR_NAMES, + ]); + + expect( + [...templates].filter((n) => !(n in catalog)).sort(), + "a reference template nothing in the catalog can reach", + ).toEqual([]); + expect( + Object.values(catalog).filter((e) => e.ejectable && !templates.has(e.name)).map((e) => e.name).sort(), + "an entry claiming to be ejectable with no template to copy", + ).toEqual([]); + expect( + Object.values(catalog).filter((e) => !e.ejectable && templates.has(e.name)).map((e) => e.name).sort(), + "a template that exists but whose entry says it cannot be ejected", + ).toEqual([]); + }); + + test("every ejectable entry's template header carries the facets `--list` reports", () => { + const roots: ReadonlyArray string]> = [ + [coreTpl.REFERENCE_GENERATOR_NAMES, coreTpl.resolveReferenceRoot], + [reactTpl.REFERENCE_GENERATOR_NAMES, reactTpl.resolveReferenceRoot], + [tanstackTpl.REFERENCE_GENERATOR_NAMES, tanstackTpl.resolveReferenceRoot], + ]; + const missing: string[] = []; + for (const [names, root] of roots) { + for (const name of names) { + const source = readFileSync(join(root(), `${name}.ts`), "utf8"); + for (const facet of ["use-when", "emits"]) { + if (!source.includes(`// ${facet}:`)) missing.push(`${name} has no "${facet}:" header line`); + } + } + } + expect(missing, missing.join("\n ")).toEqual([]); + }); +}); diff --git a/server/typescript/packages/cli/test/catalog-gates.test.ts b/server/typescript/packages/cli/test/catalog-gates.test.ts new file mode 100644 index 000000000..70cc60ebd --- /dev/null +++ b/server/typescript/packages/cli/test/catalog-gates.test.ts @@ -0,0 +1,160 @@ +// The post-selection audits `meta gen` runs over a wired suite. +// +// In the CLI rather than in codegen-ts, because both need the COMPOSED catalog: three +// of the four cases below involve `form` / `hooks` / `grid-hook`, which live in the +// react and tanstack slices that codegen-ts cannot import. + +import { describe, test, expect } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadMemory } from "@metaobjectsdev/sdk"; +import { runGen, type Generator, type MetaobjectsGenConfig } from "@metaobjectsdev/codegen-ts"; +import { formFile } from "@metaobjectsdev/codegen-ts-react"; +import { tanstackQuery, tanstackGrid, tanstackGridHook } from "@metaobjectsdev/codegen-ts-tanstack"; +import { composeCatalog } from "../src/lib/catalog.js"; + +const FIXTURE = join(import.meta.dir, "fixtures", "catalog-probe"); + +async function warningsFor( + generators: Generator[], + libraries?: string[], +): Promise { + const metadata = await loadMemory(FIXTURE, libraries === undefined ? {} : { libraries }); + const root = mkdtempSync(join(tmpdir(), "catalog-gates-")); + const config: MetaobjectsGenConfig = { + outDir: "src/generated", + extStyle: "js", + dialect: "sqlite", + dbImport: "../db", + generators, + }; + try { + const result = await runGen({ + config, + metadata, + projectRoot: root, + dryRun: true, + catalog: composeCatalog(), + ...(libraries === undefined ? {} : { libraries }), + }); + return result.warnings; + } finally { + rmSync(root, { recursive: true, force: true }); + } +} + +/** Construct a catalog generator by stable name — the same factory `--list` calls. */ +function gen(name: string): Generator { + const entry = composeCatalog()[name]; + if (entry === undefined) throw new Error(`no catalog entry "${name}"`); + return entry.factory(); +} + +describe("the requires gate", () => { + test("warns, naming what is missing and how to get it", async () => { + const warnings = (await warningsFor([tanstackGridHook()])).join("\n"); + expect(warnings).toContain('"grid-hook" is wired'); + // grid-hook declares requires: [entity, grid, hooks] — all three unwired here. + for (const dep of ["entity", "grid", "hooks"]) { + expect(warnings, `names the missing "${dep}"`).toContain(`"${dep}"`); + } + expect(warnings).toContain("meta eject"); + }); + + test("says nothing once the requirement is wired", async () => { + const warnings = await warningsFor([ + gen("entity"), tanstackGrid(), tanstackQuery(), tanstackGridHook(), + ]); + expect(warnings.filter((w) => w.includes("is wired but"))).toEqual([]); + }); + + test("is silent for a generator the catalog does not know", async () => { + // An owned or third-party generator has no declaration to check, so it must not be + // reported as depending on nothing — the gate skips it rather than guessing. + const mine: Generator = { name: "my-own-thing", generate: () => [] }; + const warnings = await warningsFor([mine]); + expect(warnings.filter((w) => w.includes("is wired but"))).toEqual([]); + }); + + test("warns rather than failing — a hand-written half is legitimate", async () => { + // The run still completes and reports files; the gate never touches the exit path. + const metadata = await loadMemory(FIXTURE, {}); + const root = mkdtempSync(join(tmpdir(), "catalog-gates-")); + try { + const result = await runGen({ + config: { + outDir: "src/generated", extStyle: "js", dialect: "sqlite", dbImport: "../db", + generators: [tanstackGridHook()], + }, + metadata, + projectRoot: root, + dryRun: true, + catalog: composeCatalog(), + }); + expect(result.conflicts).toEqual([]); + expect(result.files.length).toBeGreaterThan(0); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); + +describe("the api-framework advisory", () => { + test("warns when two api-layer generators bring different frameworks", async () => { + const warnings = (await warningsFor([gen("entity"), gen("routes"), gen("routes-hono")])).join("\n"); + expect(warnings).toContain("two api-layer frameworks"); + expect(warnings).toContain("fastify"); + expect(warnings).toContain("hono"); + // The point of the message: nothing is broken, so say so rather than implying it. + expect(warnings).toContain("DIFFERENT paths"); + }); + + test("is silent for ONE api framework", async () => { + const warnings = await warningsFor([gen("entity"), gen("routes")]); + expect(warnings.filter((w) => w.includes("api-layer frameworks"))).toEqual([]); + }); + + test("does NOT fire on form + hooks + grid — client is a composition, not a conflict", async () => { + // @metaobjectsdev/tanstack peers on react, so this is the documented, normal client + // selection. An earlier draft's "one framework per layer" rule would have forbidden + // the single most common one. + const warnings = await warningsFor([ + gen("entity"), formFile(), tanstackQuery(), tanstackGrid(), + ]); + expect(warnings.filter((w) => w.includes("frameworks"))).toEqual([]); + }); +}); + +describe("the library gate (FR-043 §6)", () => { + test("opted in, not wired: the design is in the model and its code is not generated", async () => { + const warnings = (await warningsFor([gen("entity")], ["ai"])).join("\n"); + expect(warnings).toContain('library "ai" is opted in and implies "trace-helper"'); + }); + + test("wired, not opted in: it will match nothing, and says so", async () => { + // The failure this removes is a SILENT one: a wired generator emitting zero files + // reads exactly like "my model has no trace entities yet". + const warnings = (await warningsFor([gen("entity"), gen("trace-helper")], [])).join("\n"); + expect(warnings).toContain('"trace-helper" is wired'); + expect(warnings).toContain('library "ai"'); + expect(warnings).toContain("emit"); + }); + + test("both halves together: silence", async () => { + const warnings = await warningsFor([gen("entity"), gen("trace-helper")], ["ai"]); + expect(warnings.filter((w) => w.includes("library"))).toEqual([]); + }); + + test("neither half: silence — a library nobody mentioned is not a finding", async () => { + const warnings = await warningsFor([gen("entity")], []); + expect(warnings.filter((w) => w.includes("library"))).toEqual([]); + }); + + test("a caller that never threads `libraries` gets no library warnings at all", async () => { + // Undefined is "nobody told me", which is a different statement from `[]`. Claiming + // "you opted into nothing" on a programmatic run would be a claim we cannot support. + const warnings = await warningsFor([gen("entity"), gen("trace-helper")]); + expect(warnings.filter((w) => w.includes("opted in"))).toEqual([]); + }); +}); diff --git a/server/typescript/packages/cli/test/catalog-listing.test.ts b/server/typescript/packages/cli/test/catalog-listing.test.ts new file mode 100644 index 000000000..70803fd45 --- /dev/null +++ b/server/typescript/packages/cli/test/catalog-listing.test.ts @@ -0,0 +1,305 @@ +// `meta gen --list` — the catalog, and `--probe`, the half that runs the generators. +// +// Two separate claims, kept apart because they fail for different reasons: +// - the LISTING describes the installed engine and needs no project at all; +// - the PROBE answers "what would this emit for MY model" by dry-running every +// catalog generator, so it needs one and says so when there isn't one. + +import { describe, test, expect, beforeEach, afterEach } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadMemory } from "@metaobjectsdev/sdk"; +import type { MetaobjectsGenConfig } from "@metaobjectsdev/codegen-ts"; +import { genCommand } from "../src/commands/gen.js"; +import { + buildCatalogListing, wiredGeneratorNames, + type GeneratorCatalogRow, type LibraryCatalogRow, +} from "../src/lib/catalog-listing.js"; +import { composeCatalog } from "../src/lib/catalog.js"; + +const FIXTURE = join(import.meta.dir, "fixtures", "catalog-probe"); + +let logged: string[]; +let erred: string[]; +const origLog = console.log; +const origErr = console.error; + +beforeEach(() => { + logged = []; + erred = []; + console.log = (...a: unknown[]) => { logged.push(a.join(" ")); }; + console.error = (...a: unknown[]) => { erred.push(a.join(" ")); }; +}); +afterEach(() => { + console.log = origLog; + console.error = origErr; +}); + +const baseConfig: MetaobjectsGenConfig = { + outDir: "src/generated", + extStyle: "js", + dialect: "sqlite", + // `routes` reads this — it emits `import { db } from …` and needs the module to + // import from. It is a declared `configKey` on that entry precisely so a selection + // knows to set it; the probe is where an adopter finds out they have not. + dbImport: "../db", + generators: [], +}; + +async function allProbeRows(config: MetaobjectsGenConfig = baseConfig, libraries: string[] = []) { + const metadata = await loadMemory(FIXTURE, { libraries }); + const tmp = mkdtempSync(join(tmpdir(), "catalog-probe-")); + try { + return await buildCatalogListing({ + project: { + projectRoot: tmp, + config, + wiredNames: wiredGeneratorNames(config), + ownedNames: new Set(), + declaredDeps: undefined, + libraries, + }, + probe: { metadata }, + }); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } +} + +/** The GENERATOR rows alone — the catalog is one table with two kinds in it since + * FR-043, and every assertion below is about the generator half. */ +async function probeRows(config: MetaobjectsGenConfig = baseConfig) { + return (await allProbeRows(config)).filter((r): r is GeneratorCatalogRow => r.kind === "generator"); +} + +describe("meta gen --list — the catalog", () => { + test("--format json emits ONE document and every row is well-formed", async () => { + const tmp = mkdtempSync(join(tmpdir(), "meta-catalog-")); + try { + const code = await genCommand(["--list"], tmp, "json"); + expect(code).toBe(0); + const rows = JSON.parse(logged.join("\n")) as Array>; + const generators = rows.filter((r) => r.kind === "generator"); + const libraries = rows.filter((r) => r.kind === "library"); + // ONE table, two kinds (FR-043 §4) — and nothing else in it. + expect(generators.length + libraries.length).toBe(rows.length); + expect(generators.length).toBe(Object.keys(composeCatalog()).length); + expect(libraries.length).toBeGreaterThan(0); + for (const r of generators) { + expect(typeof r.layer, String(r.name)).toBe("string"); + expect(String(r.package), String(r.name)).toStartWith("@metaobjectsdev/"); + expect(typeof r.description, String(r.name)).toBe("string"); + } + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + test("needs no project at all — an empty directory is not an error", async () => { + const tmp = mkdtempSync(join(tmpdir(), "meta-catalog-")); + try { + expect(await genCommand(["--list"], tmp, "json")).toBe(0); + expect(erred.join("\n")).not.toContain("not found"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + test("an ejectable entry carries the facets from its own reference-template header", async () => { + const tmp = mkdtempSync(join(tmpdir(), "meta-catalog-")); + try { + await genCommand(["--list"], tmp, "json"); + const rows = JSON.parse(logged.join("\n")) as Array>; + const entity = rows.find((r) => r.name === "entity")!; + // Read from the template, never restated in the registry — so the row cannot + // describe the generator differently from the file an adopter opens. + expect(String(entity.useWhen)).toContain("entity-module generator"); + expect(String(entity.emits)).toContain(".ts"); + expect((entity.source as Record).ejectable).toBe(true); + + // A package-only entry has no template to read, and says so rather than + // inventing a summary. + const template = rows.find((r) => r.name === "template")!; + expect(template.useWhen).toBeUndefined(); + expect((template.source as Record).kind).toBe("package-only"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + test("--probe without a project is a usage error, not a listing of zeros", async () => { + const tmp = mkdtempSync(join(tmpdir(), "meta-catalog-")); + try { + expect(await genCommand(["--list", "--probe"], tmp, "json")).toBe(2); + expect(erred.join("\n")).toContain("--probe"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + test("--probe without --list is refused rather than silently ignored", async () => { + const tmp = mkdtempSync(join(tmpdir(), "meta-catalog-")); + try { + expect(await genCommand(["--probe"], tmp, "text")).toBe(2); + expect(erred.join("\n")).toContain("only meaningful with --list"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); +}); + +describe("--probe — what would this emit for MY model", () => { + test("reports a real count for the generators this model asks for", async () => { + const rows = await probeRows(); + const by = (n: string) => rows.find((r) => r.name === n)!.project!; + + // Two concrete sourced entities (+ the shared enums module the entity generator + // emits once) — the exact number is the engine's business; that it is non-zero and + // came from RUNNING the generator is this gate's. + expect(by("entity").wouldEmit).toBeGreaterThan(0); + expect(by("queries").wouldEmit).toBeGreaterThan(0); + expect(by("routes").wouldEmit).toBeGreaterThan(0); + + // The capability tier, which is the whole reason --probe exists: these are not + // chosen by browsing a taxonomy, they are chosen because the model declared them. + expect(by("output-parser").wouldEmit).toBeGreaterThan(0); // a responding template.prompt + expect(by("output-prompt").wouldEmit).toBeGreaterThan(0); // ...and its format fragment + expect(by("prompt-render").wouldEmit).toBeGreaterThan(0); // ...and its render helper + expect(by("callable").wouldEmit).toBeGreaterThan(0); // a storedProc projection + expect(by("requirement-tests").wouldEmit).toBeGreaterThan(0); // a requirement.functional + + // The client tier keys off a declared layout.dataGrid. + expect(by("grid").wouldEmit).toBeGreaterThan(0); + }); + + test("a count of ZERO is a real answer, not a failure", async () => { + const rows = await probeRows(); + // No entity in this model extends LlmCallBase, so the trace helper genuinely has + // nothing to emit. Zero with no probeError is the honest report, and it is the + // signal an agent uses to leave the generator unwired. + const trace = rows.find((r) => r.name === "trace-helper")!.project!; + expect(trace.wouldEmit).toBe(0); + expect(trace.probeError).toBeUndefined(); + }); + + test("every generator is probed — none is silently skipped", async () => { + const rows = await probeRows(); + for (const r of rows) { + const p = r.project!; + const answered = typeof p.wouldEmit === "number" || typeof p.probeError === "string"; + expect(answered, `${r.name} reported neither a count nor a reason`).toBe(true); + } + }); + + test("a generator that cannot be probed reports WHY, and leaves the rest intact", async () => { + // Two shipped generators genuinely cannot run from a bare `--probe`: + // `render-helper` needs an on-disk template root for its drift gate, and + // `shared-model` needs a `files` selection `meta gen` supplies at run time. Both + // report their own message. This is the whole reason each generator gets its own + // isolated dry-run rather than one run over the suite — a suite-wide run would + // take the entire listing down with them. + const rows = await probeRows(); + for (const name of ["render-helper", "shared-model"]) { + const p = rows.find((r) => r.name === name)!.project!; + expect(p.wouldEmit, `${name} should not report a count`).toBeNull(); + expect(String(p.probeError), `${name} should say why`).toContain(name); + } + // ...and everything else still answered. + expect(rows.filter((r) => typeof r.project!.wouldEmit === "number").length) + .toBeGreaterThan(rows.length - 4); + }); + + test("a config that omits a generator's configKey shows up as that generator's probeError", async () => { + // The catalog says `routes` reads `dbImport`. Dropping it must surface on the + // ROUTES row and nowhere else — that is what makes the probe an answer to "can I + // turn this on" rather than a yes/no about the whole project. + const { dbImport: _omitted, ...withoutDbImport } = baseConfig; + const rows = await probeRows(withoutDbImport); + const routes = rows.find((r) => r.name === "routes")!.project!; + expect(routes.wouldEmit).toBeNull(); + expect(String(routes.probeError)).toContain("dbImport"); + expect(rows.find((r) => r.name === "entity")!.project!.wouldEmit).toBeGreaterThan(0); + }); + + test("`wired` reflects the config's own generator list", async () => { + const rows = await probeRows(); + expect(rows.find((r) => r.name === "entity")!.project!.wired).toBe(false); + }); +}); + +// --------------------------------------------------------------------------- +// FR-043 §4 — the library rows +// --------------------------------------------------------------------------- + +async function libraryRows(libraries: string[] = []): Promise { + return (await allProbeRows(baseConfig, libraries)).filter( + (r): r is LibraryCatalogRow => r.kind === "library", + ); +} + +describe("meta gen --list — the library rows", () => { + test("a library describes what is IN THE BOX, across every layer", async () => { + const iam = (await libraryRows()).find((r) => r.name === "iam")!; + expect(iam.libraryKind).toBe("feature"); + expect(iam.stability).toBe("preview"); + expect(iam.packages).toEqual(["metaobjects::iam"]); + // `provides` is the whole library, not the selection — an adopter reading it is + // deciding whether to opt in at all. + expect(iam.provides.entities).toBe(9); + expect(iam.provides.abstracts).toBeGreaterThan(0); + expect(iam.provides.requirements).toBeGreaterThan(0); + // The layer tokens are the field an adopter ACTS on: under Amendment 1 you do not + // opt into a library, you opt into its layers. + expect(iam.layers.map((l) => l.token)).toEqual(["iam", "iam/db"]); + expect(iam.ports).toContain("java"); + }); + + test("not opted in: the project block says so without pretending to measure", async () => { + const iam = (await libraryRows([])).find((r) => r.name === "iam")!; + expect(iam.project!.optedIn).toBe(false); + expect(iam.project!.selectedLayers).toEqual([]); + // Probed, with no library in the model: nothing extends it, nothing was added. + expect(iam.project!.tablesAdded).toBe(0); + expect(iam.project!.requirementsAdded).toBe(0); + expect(iam.project!.extendedBy).toEqual([]); + }); + + test("the CORE layer adds requirements and NO tables — the inertness promise, as a number", async () => { + const iam = (await libraryRows(["iam"])).find((r) => r.name === "iam")!; + expect(iam.project!.optedIn).toBe(true); + expect(iam.project!.selectedLayers).toEqual(["iam"]); + expect(iam.project!.tablesAdded).toBe(0); + expect(iam.project!.requirementsAdded).toBe(iam.provides.requirements); + }); + + test("...and the db layer is what puts tables on the table", async () => { + const iam = (await libraryRows(["iam", "iam/db"])).find((r) => r.name === "iam")!; + expect(iam.project!.selectedLayers).toEqual(["iam", "iam/db"]); + expect(iam.project!.tablesAdded).toBe(9); + }); + + test("an implied generator nobody wired is reported, not enforced", async () => { + // `ai` declares `trace-helper`; the fixture config wires nothing. A library is + // metadata — wiring the generator it implies stays the adopter's call, so this is + // a fact on the row rather than a warning here. + const ai = (await libraryRows(["ai"])).find((r) => r.name === "ai")!; + expect(ai.project!.impliedGeneratorsNotWired).toEqual(["trace-helper"]); + expect(ai.provides.generators).toEqual(["trace-helper"]); + }); + + test("`extendedBy` names the project entities that would break if you opted out", async () => { + // The catalog-probe fixture has no entity extending a library base, so the + // interesting arm is the one that finds one — see gen-libraries.test.ts for the + // end-to-end path. Here: the field exists and is a real answer, not a null. + const ai = (await libraryRows(["ai"])).find((r) => r.name === "ai")!; + expect(Array.isArray(ai.project!.extendedBy)).toBe(true); + }); + + test("with no project there is no project block at all", async () => { + const { buildLibraryRows } = await import("../src/lib/library-listing.js"); + const rows = await buildLibraryRows(); + expect(rows.length).toBeGreaterThan(0); + for (const r of rows) expect(r.project).toBeUndefined(); + }); +}); diff --git a/server/typescript/packages/cli/test/eject-library.test.ts b/server/typescript/packages/cli/test/eject-library.test.ts new file mode 100644 index 000000000..9037bc53f --- /dev/null +++ b/server/typescript/packages/cli/test/eject-library.test.ts @@ -0,0 +1,216 @@ +// FR-043 §3.4 — `meta eject `: the copy door for declared design. +// +// ADR-0034 ruled that a reference GENERATOR is copied into the adopter's repo because +// the adopter owns their code; §3.4 extends that to metadata and makes it the EXPECTED +// mode. So the assertions here are about ownership rather than about copying: the files +// land in the project's OWN declared source root, they carry the provenance that makes +// their origin legible later, and they tell the adopter the one step that finishes the +// job — which the loader then enforces. +import { describe, test, expect } from "bun:test"; +import { mkdtempSync, mkdirSync, rmSync, readFileSync, writeFileSync, existsSync, readdirSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ejectCommand } from "../src/commands/eject.js"; +import { ejectLibrary, libraryStaleness, EJECT_MARKER } from "../src/lib/library-eject.js"; + +/** A project whose declared source root is deliberately NOT named `metaobjects/` — + * that string is the default value of `sources`, and eject must resolve the root + * through the collection rather than assuming it. */ +function project(libraries: string[] = ["iam"]): string { + const root = mkdtempSync(join(tmpdir(), "eject-lib-")); + mkdirSync(join(root, ".metaobjects")); + mkdirSync(join(root, "model")); + writeFileSync( + join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, sources: [{ path: "model" }], libraries }, null, 2), + ); + writeFileSync( + join(root, "model", "mine.yaml"), + "metadata:\n package: acme::app\n children:\n - object.value:\n name: Ping\n children:\n - field.string: { name: at }\n", + ); + return root; +} + +describe("meta eject ", () => { + test("copies every layer into the project's own DECLARED source root", async () => { + const root = project(); + try { + const result = await ejectLibrary({ cwd: root, name: "iam" }); + expect(result.root).toBe(join(root, "model")); + // Every ref, core and db — you eject the library, not a layer of it. + expect(result.files.map((f) => f.ref).sort()).toEqual([ + "iam/db", "iam/model", "iam/requirements", + ]); + for (const f of result.files) { + expect(f.status).toBe("created"); + expect(existsSync(join(root, "model", f.path))).toBe(true); + } + // Named for the concept, not for the library's internal filenames: three + // libraries ejected into one root would otherwise collide on `model.yaml`. + expect(readdirSync(join(root, "model")).sort()).toEqual([ + "meta.iam.db.yaml", "meta.iam.model.yaml", "meta.iam.requirements.yaml", "mine.yaml", + ]); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("every file carries provenance AND the one step that finishes the eject", async () => { + const root = project(); + try { + await ejectLibrary({ cwd: root, name: "iam" }); + const text = readFileSync(join(root, "model", "meta.iam.model.yaml"), "utf8"); + expect(text.startsWith(`${EJECT_MARKER} library=iam ref=iam/model`)).toBe(true); + expect(text).toContain("YOU OWN THIS FILE"); + expect(text).toContain("ERR_LIBRARY_PACKAGE_COLLISION"); + // And the metadata itself is intact under the header. + expect(text).toContain("metadata:"); + expect(text).toContain("name: User"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("never clobbers without --force", async () => { + const root = project(); + try { + await ejectLibrary({ cwd: root, name: "iam" }); + writeFileSync(join(root, "model", "meta.iam.model.yaml"), "# mine now\n"); + const again = await ejectLibrary({ cwd: root, name: "iam" }); + expect(again.files.find((f) => f.ref === "iam/model")!.status).toBe("preserved"); + expect(readFileSync(join(root, "model", "meta.iam.model.yaml"), "utf8")).toBe("# mine now\n"); + + const forced = await ejectLibrary({ cwd: root, name: "iam", force: true }); + expect(forced.files.find((f) => f.ref === "iam/model")!.status).toBe("replaced"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("reports what is still opted in — the step that makes the copy usable", async () => { + const root = project(["iam", "iam/db"]); + try { + const result = await ejectLibrary({ cwd: root, name: "iam" }); + expect(result.stillOptedIn).toEqual(["iam", "iam/db"]); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("a library name is accepted by the command beside generator names", async () => { + const root = project([]); + try { + expect(await ejectCommand(["iam"], root, "json")).toBe(0); + expect(existsSync(join(root, "model", "meta.iam.model.yaml"))).toBe(true); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); + +describe("meta eject --list — library staleness", () => { + test("a fresh eject is identical to the shipped library", async () => { + const root = project([]); + try { + await ejectLibrary({ cwd: root, name: "iam" }); + const rows = await libraryStaleness(root); + expect(rows.length).toBe(1); + expect(rows[0]!.library).toBe("iam"); + expect(rows[0]!.verdict).toBe("identical"); + expect(rows[0]!.stillOptedIn).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("a changed node, a deleted one and an added one are counted apart", async () => { + const root = project([]); + try { + await ejectLibrary({ cwd: root, name: "iam" }); + // Delete one node outright, add one of my own, and change a third — the three + // things an owner actually does to a copy. + const model = join(root, "model", "meta.iam.model.yaml"); + const text = readFileSync(model, "utf8"); + writeFileSync( + model, + `${text}\n - object.entity:\n name: ApiKey\n children:\n - field.uuid: { name: id }\n`, + ); + const rows = await libraryStaleness(root); + expect(rows[0]!.verdict).toBe("differs"); + expect(rows[0]!.localOnly).toBe(1); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("renaming the package is NOT drift — nodes are matched by name", async () => { + // §3.4 says an adopter who ejects may rename the package freely. Keying the + // comparison on the FQN would report every node as both gone and new. + const root = project([]); + try { + await ejectLibrary({ cwd: root, name: "iam" }); + for (const f of ["meta.iam.model.yaml", "meta.iam.db.yaml", "meta.iam.requirements.yaml"]) { + const p = join(root, "model", f); + writeFileSync(p, readFileSync(p, "utf8").replaceAll("metaobjects::iam", "acme::iam")); + } + const rows = await libraryStaleness(root); + expect(rows[0]!.verdict).toBe("identical"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("a project that ejected nothing reports nothing", async () => { + const root = project([]); + try { + expect(await libraryStaleness(root)).toEqual([]); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); + +describe("the `metaobjects::` prefix advisory (FR-043 §3.5)", () => { + test("an ejected file is exempt — its header IS the provenance", async () => { + const { scanForUnprovenancedLibraryPrefix } = await import( + "../src/lib/library-prefix-advisory.js" + ); + const { loadMemory } = await import("@metaobjectsdev/sdk"); + const { resolveCollection } = await import("@metaobjectsdev/sdk"); + const root = project([]); + try { + await ejectLibrary({ cwd: root, name: "iam" }); + const collection = await resolveCollection(root); + const model = await loadMemory(root, { strict: true }); + expect( + await scanForUnprovenancedLibraryPrefix(model, collection.ownFiles, collection.configDir), + ).toEqual([]); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("a hand-copied node under the prefix is flagged, with the eject door named", async () => { + const { scanForUnprovenancedLibraryPrefix } = await import( + "../src/lib/library-prefix-advisory.js" + ); + const { loadMemory, resolveCollection } = await import("@metaobjectsdev/sdk"); + const root = project([]); + try { + writeFileSync( + join(root, "model", "copied.yaml"), + "metadata:\n package: metaobjects::iam\n children:\n - object.value:\n name: Whatever\n children:\n - field.string: { name: x }\n", + ); + const collection = await resolveCollection(root); + const model = await loadMemory(root, { strict: true }); + const findings = await scanForUnprovenancedLibraryPrefix( + model, collection.ownFiles, collection.configDir, + ); + expect(findings.length).toBe(1); + expect(findings[0]!.fqn).toBe("metaobjects::iam::Whatever"); + expect(findings[0]!.message).toContain("meta eject"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/cli/test/eject-multi.test.ts b/server/typescript/packages/cli/test/eject-multi.test.ts new file mode 100644 index 000000000..74b53b77c --- /dev/null +++ b/server/typescript/packages/cli/test/eject-multi.test.ts @@ -0,0 +1,212 @@ +// `meta eject ...` — many names, one consolidated install set, one document. +// +// The single-name behaviour (preserve / differs / replace, and the three-branch wiring +// message) is covered by eject.test.ts and is unchanged. This file covers what taking +// several names at once adds, and the one failure mode it introduces. + +import { describe, test, expect, beforeEach, afterEach } from "bun:test"; +import { mkdtempSync, rmSync, existsSync, readFileSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ejectCommand } from "../src/commands/eject.js"; +import { cliVersion } from "../src/lib/version.js"; + +let logged: string[]; +let erred: string[]; +const origLog = console.log; +const origErr = console.error; + +beforeEach(() => { + logged = []; + erred = []; + console.log = (...a: unknown[]) => { logged.push(a.join(" ")); }; + console.error = (...a: unknown[]) => { erred.push(a.join(" ")); }; +}); +afterEach(() => { + console.log = origLog; + console.error = origErr; +}); + +function tmp(): string { + return mkdtempSync(join(tmpdir(), "meta-eject-multi-")); +} + +interface EjectPayload { + ejected: Array<{ + name: string; + path: string; + status: string; + wire: { import: string; entry: string }; + requires: string[]; + }>; + install: { dev: string[]; runtime: string[]; command: string }; + config: { keys: string[] }; +} + +function payload(): EjectPayload { + return JSON.parse(logged.join("\n")) as EjectPayload; +} + +describe("meta eject takes many names", () => { + test("ejects each one, in the order given", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["entity", "queries", "routes"], dir, "text")).toBe(0); + for (const n of ["entity", "queries", "routes"]) { + expect(existsSync(join(dir, "codegen/generators", `${n}.ts`)), n).toBe(true); + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("an unknown name refuses the WHOLE call and copies nothing", async () => { + // A partial eject is the worst outcome available: a non-zero exit over a repo that + // is half-changed, where re-running the fixed command reports the already-copied + // half as "preserved" and the adopter cannot tell what happened. + const dir = tmp(); + try { + expect(await ejectCommand(["entity", "nonesuch"], dir, "text")).toBe(2); + expect(existsSync(join(dir, "codegen/generators/entity.ts"))).toBe(false); + expect(erred.join("\n")).toContain("Nothing was ejected"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("a repeated name is a usage error, not a silent second write", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["entity", "entity"], dir, "text")).toBe(2); + expect(erred.join("\n")).toContain("repeated"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("no names at all points at the catalog rather than just refusing", async () => { + const dir = tmp(); + try { + expect(await ejectCommand([], dir, "text")).toBe(2); + expect(erred.join("\n")).toContain("meta gen --list"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe("--format json", () => { + test("emits ONE document with the wire lines per file", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["hooks"], dir, "json")).toBe(0); + const p = payload(); + expect(p.ejected.length).toBe(1); + expect(p.ejected[0]!.name).toBe("hooks"); + expect(p.ejected[0]!.status).toBe("created"); + // The exported symbol does NOT follow the file name — hooks.ts exports + // tanstackQuery — so the entry is read from the template's own header. + expect(p.ejected[0]!.wire.entry).toBe("tanstackQuery()"); + expect(p.ejected[0]!.wire.import).toContain("./codegen/generators/hooks.js"); + expect(p.ejected[0]!.requires).toContain("entity"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("the install set is CONSOLIDATED, not per name", async () => { + const dir = tmp(); + try { + await ejectCommand(["hooks", "grid"], dir, "json"); + const p = payload(); + const tanstack = p.install.dev.filter((d) => + d.startsWith("@metaobjectsdev/codegen-ts-tanstack@")); + expect(tanstack.length, "the shared codegen package appears once").toBe(1); + expect(tanstack[0]).toBe(`@metaobjectsdev/codegen-ts-tanstack@^${cliVersion()}`); + // Both generators' third-party peers, unioned. + expect(p.install.runtime.some((r) => r.startsWith("@tanstack/react-query"))).toBe(true); + expect(p.install.runtime.some((r) => r.startsWith("@tanstack/react-table"))).toBe(true); + expect(p.install.command).toContain("npm i -D"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("third-party ranges come from the runtime package's own peerDependencies", async () => { + const dir = tmp(); + try { + await ejectCommand(["routes"], dir, "json"); + const p = payload(); + const fastify = p.install.runtime.find((r) => r.startsWith("fastify")); + // A RANGE, not a bare name and not the CLI's version: read off + // @metaobjectsdev/runtime-ts, which is what routes' emitted code imports. + expect(fastify, "fastify is in the runtime install set").toBeDefined(); + expect(fastify).toContain("@"); + expect(fastify).not.toBe(`fastify@^${cliVersion()}`); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("reports the config keys the ejected generators read", async () => { + const dir = tmp(); + try { + await ejectCommand(["routes"], dir, "json"); + expect(payload().config.keys).toContain("dbImport"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("nothing but the document reaches stdout", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity"], dir, "json"); + // Would throw on any prose line — the `meta types` stdout-purity rule. + expect(() => JSON.parse(logged.join("\n"))).not.toThrow(); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe("text output", () => { + test("names the requires edges the selection does not itself satisfy", async () => { + const dir = tmp(); + try { + await ejectCommand(["grid-hook"], dir, "text"); + const out = logged.join("\n"); + expect(out).toContain("Also needed:"); + for (const dep of ["entity", "grid", "hooks"]) { + expect(out, `names "${dep}"`).toContain(dep); + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("says nothing about requires once the selection covers them", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity", "grid", "hooks", "grid-hook"], dir, "text"); + expect(logged.join("\n")).not.toContain("Also needed:"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("still never clobbers an owned copy without --force", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity"], dir, "text"); + const owned = join(dir, "codegen/generators/entity.ts"); + writeFileSync(owned, "// mine\n"); + await ejectCommand(["entity", "queries"], dir, "text"); + expect(readFileSync(owned, "utf8")).toBe("// mine\n"); + // ...and the OTHER name in the same call still landed. + expect(existsSync(join(dir, "codegen/generators/queries.ts"))).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/cli/test/fixtures/catalog-probe/.metaobjects/config.json b/server/typescript/packages/cli/test/fixtures/catalog-probe/.metaobjects/config.json new file mode 100644 index 000000000..2194141f4 --- /dev/null +++ b/server/typescript/packages/cli/test/fixtures/catalog-probe/.metaobjects/config.json @@ -0,0 +1,10 @@ +{ + "schema_version": 1, + "pending_in_git": true, + "confidence_thresholds": { + "pending_promote": 0.8, + "drift_warn": 0.7 + }, + "sources": [], + "extract": {} +} diff --git a/server/typescript/packages/cli/test/fixtures/catalog-probe/metaobjects/meta.shop.yaml b/server/typescript/packages/cli/test/fixtures/catalog-probe/metaobjects/meta.shop.yaml new file mode 100644 index 000000000..cf524f07d --- /dev/null +++ b/server/typescript/packages/cli/test/fixtures/catalog-probe/metaobjects/meta.shop.yaml @@ -0,0 +1,105 @@ +# The catalog probe's model. +# +# `--probe` constructs EVERY catalog generator and dry-runs it, so this fixture has to +# make each one either emit or deliberately not emit. It is therefore not a realistic +# app — it is one declaration per generator gate, which is exactly what the probe's own +# gate needs to distinguish "emits 0 because nothing declared it" from "emits 0 because +# it is broken". +# +# Deliberately its OWN fixture rather than a reach into fixtures/conformance/: a corpus +# fixture exists to pin a loader behaviour and may change shape for reasons that have +# nothing to do with this gate. +metadata: + package: catalog::shop + children: + # Two entities with a relationship — entity / queries / routes / names / barrel / + # api-docs / form / hooks all key off concrete sourced objects. + - object.entity: + name: Customer + children: + - source.rdb: { table: customer, role: primary } + - field.long: { name: id, required: true } + - field.string: { name: email, required: true, maxLength: 254, filterable: true } + - field.string: { name: displayName, maxLength: 120 } + - field.boolean: { name: active, filterable: true } + - identity.primary: { name: pk, fields: [id], generation: increment } + - identity.secondary: { name: uqEmail, fields: [email] } + # grid / grid-hook emit only for an entity that DECLARES a data grid. The + # `filter` PRESET is load-bearing here and not decoration: the grid-hook only + # imports the columns module when a preset exists, so without one the + # grid-hook → grid dependency edge the requires gate is supposed to derive + # would never appear, and the gate would certify a declaration of `[]`. + - layout.dataGrid: + name: default + columns: [email, displayName, active] + defaultSortField: email + pageSize: 25 + filterable: true + filter: + active: true + + - object.entity: + name: Order + children: + - source.rdb: { table: shop_order, role: primary } + - field.long: { name: id, required: true } + - field.long: { name: customerId, required: true } + - field.int: { name: quantity } + - identity.primary: { name: pk, fields: [id], generation: increment } + - identity.reference: { name: fkCustomer, fields: [customerId], references: Customer, onDelete: cascade } + - index.lookup: { name: ixCustomer, fields: [customerId] } + + # A proc-kind projection — the `callable` generator's gate. + - object.projection: + name: OrderTotals + children: + - source.rdb: { kind: storedProc, proc: fn_order_totals } + - field.long: { name: customerId } + - field.double: { name: total } + + # The prompt tier: payload / response value objects, a RESPONDING template.prompt + # (one carrying @responseRef — since 0.24.0 the whole inbound tier keys off it) and + # an outbound template.output. + - object.value: + name: RestockRequest + children: + - field.string: { name: sku, required: true } + + - object.value: + name: RestockAdvice + children: + - field.string: { name: recommendation, required: true } + - field.int: { name: units, required: true } + + - template.prompt: + name: RestockPrompt + payloadRef: catalog::shop::RestockRequest + responseRef: catalog::shop::RestockAdvice + textRef: shop/restock + format: text + responseFormat: json + promptStyle: inline + + - object.value: + name: OrderReceipt + children: + - field.string: { name: customerName } + - field.string: { name: summary } + + - template.output: + name: ReceiptDocument + kind: document + payloadRef: catalog::shop::OrderReceipt + textRef: shop/receipt + format: html + + # One functional claim — the `requirement-tests` generator's gate. Level 4 + # (OBJECT) deliberately: the generator's default filter is functional claims at or + # below the link floor, so a level-2 claim would probe as 0 and look like a bug. + - requirement.functional: + name: EveryOrderNamesItsCustomer + level: 4 + status: live + statement: An order always names the customer who placed it. + counterexample: An order row whose customer reference is absent. + implementedBy: [Order.customerId] diff --git a/server/typescript/packages/cli/test/help-and-exit.test.ts b/server/typescript/packages/cli/test/help-and-exit.test.ts index f1a4a40ab..a47f6c0f1 100644 --- a/server/typescript/packages/cli/test/help-and-exit.test.ts +++ b/server/typescript/packages/cli/test/help-and-exit.test.ts @@ -1,6 +1,5 @@ import { test, expect, spyOn } from "bun:test"; import { run } from "../src/index.js"; -import { SCAFFOLDED_GENERATOR_NAMES } from "../src/commands/init.js"; // These assert EXIT CODES (help/usage handling), not performance — so they must // not be timing-sensitive. `run()` lazily imports each command's (sometimes heavy: @@ -39,16 +38,14 @@ test("bare meta (no args) exits 0", async () => { expect(await run([])).toBe(0); }, HELP_TIMEOUT_MS); -// Finding 1 (fix round 1): `meta eject --help` used to name a fixed count of `meta -// init`'s eagerly-scaffolded generators — a literal that fell out of sync the moment -// Task 4 grew the scaffold set from four to five (adding "names"), telling an adopter -// the names generator was eject-only when it had already been scaffolded and wired for -// them. Deriving the expected text from SCAFFOLDED_GENERATOR_NAMES itself — rather than -// hardcoding "five" here, which would just move the same bug one level down — means the -// next person who extends the scaffold set cannot leave this string behind unnoticed. -const COUNT_WORDS = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"]; - -test("eject --help enumerates every scaffolded generator name and states the count", async () => { +// `meta eject --help` used to name a fixed COUNT of the generators `meta init` +// scaffolded eagerly — a literal that fell out of sync the moment the scaffold set grew +// from four to five, telling an adopter the names generator was eject-only when it had +// already been wired for them. The whole class of defect is gone: init scaffolds NONE, +// so there is no count to keep in step. What has to be true instead is that the help +// does not still claim otherwise — a stale sentence promising five generators would send +// a fresh adopter looking for files that are not there. +test("eject --help says init scaffolds nothing, and points at the catalog", async () => { const lines: string[] = []; const spy = spyOn(console, "log").mockImplementation((...args: unknown[]) => { lines.push(args.map(String).join(" ")); @@ -62,9 +59,9 @@ test("eject --help enumerates every scaffolded generator name and states the cou expect(exitCode).toBe(0); const helpText = lines.join("\n"); - const countWord = COUNT_WORDS[SCAFFOLDED_GENERATOR_NAMES.length]; - expect(countWord).toBeDefined(); - // Pins the count AND the exact, ordered enumeration in one assertion — a transposed - // or dropped name fails this even if the bare count happens to still read right. - expect(helpText).toContain(`${countWord} generators (${SCAFFOLDED_GENERATOR_NAMES.join(", ")})`); + expect(helpText).toContain("copies NO generators"); + expect(helpText).toContain("meta gen --list"); + // The retired claim, in either of the two spellings it ever had. + expect(helpText).not.toContain("copies five generators"); + expect(helpText).not.toContain("copies four generators"); }, HELP_TIMEOUT_MS); diff --git a/server/typescript/packages/cli/test/init-esm-package-type.test.ts b/server/typescript/packages/cli/test/init-esm-package-type.test.ts index 4bedf6293..60083a1e7 100644 --- a/server/typescript/packages/cli/test/init-esm-package-type.test.ts +++ b/server/typescript/packages/cli/test/init-esm-package-type.test.ts @@ -109,24 +109,26 @@ describe("meta init — ESM package type", () => { }); }); -describe("meta init — the scaffold's own dependencies", () => { - test("declares what codegen/generators/ imports, so the scaffold typechecks", async () => { - // ADR-0034 puts real source in the adopter's repo; installing only the CLI left - // those files with unresolvable imports (10x TS2307 on files init had just written). +describe("meta init — the scaffold declares no dependencies", () => { + // ADR-0034 puts real source in the adopter's repo, and init used to declare what that + // source imported (@metaobjectsdev/codegen-ts + /metadata) plus what the code it would + // GENERATE imported (drizzle-orm, zod, fastify). Both sets existed only because init + // WIRED a suite. Under opt-in codegen it wires none, so declaring anything would be a + // dependency on code this project may never produce. + // + // The obligation moved to `meta eject`, which reports the install set for exactly the + // generators you take — covered end-to-end in scaffold-output-imports-declared.test.ts, + // which ejects a selection, applies its install set and then checks the generated + // output against the manifest. + test("adds nothing to dependencies or devDependencies", async () => { writeFileSync(join(dir, "package.json"), JSON.stringify({ name: "x", type: "commonjs" }, null, 2) + "\n"); - const result = await init({ cwd: dir, quiet: true }); - const dev = readPkg().devDependencies as Record; - // ts-poet is deliberately absent: the scaffolded templates import its combinators - // via @metaobjectsdev/codegen-ts so generated-code composition shares ONE ts-poet - // instance with the engine (see the gen-split-tree gate), and a project-local - // ts-poet is the second physical copy that used to split it. - expect(Object.keys(dev).sort()).toEqual( - ["@metaobjectsdev/codegen-ts", "@metaobjectsdev/metadata"], - ); - expect(result.warnings.join("\n")).toContain("Run your package manager's install"); + await init({ cwd: dir, quiet: true }); + const pkg = readPkg(); + expect(pkg.devDependencies).toBeUndefined(); + expect(pkg.dependencies).toBeUndefined(); }); - test("never overwrites a pin the project already chose", async () => { + test("still leaves a pin the project already chose exactly as it was", async () => { writeFileSync(join(dir, "package.json"), JSON.stringify({ name: "x", type: "commonjs", devDependencies: { "ts-poet": "6.0.0" }, @@ -136,7 +138,8 @@ describe("meta init — the scaffold's own dependencies", () => { const pkg = readPkg(); expect((pkg.devDependencies as Record)["ts-poet"]).toBe("6.0.0"); expect((pkg.dependencies as Record)["@metaobjectsdev/codegen-ts"]).toBe("0.1.0"); - // …and still adds only the genuinely-missing one. - expect((pkg.devDependencies as Record)["@metaobjectsdev/metadata"]).toBeDefined(); + // ...and adds nothing beside them. + expect(Object.keys(pkg.devDependencies as Record)).toEqual(["ts-poet"]); + expect(Object.keys(pkg.dependencies as Record)).toEqual(["@metaobjectsdev/codegen-ts"]); }); }); diff --git a/server/typescript/packages/cli/test/init.test.ts b/server/typescript/packages/cli/test/init.test.ts index 2f9e8ff98..0c927e874 100644 --- a/server/typescript/packages/cli/test/init.test.ts +++ b/server/typescript/packages/cli/test/init.test.ts @@ -58,7 +58,7 @@ afterEach(() => { describe("init() — next-steps message (S1)", () => { test("presents `meta gen` and `meta docs` as working steps, not as unshipped 'later sub-projects'", () => { - const block = nextStepsBlock(true); + const block = nextStepsBlock(); // gen + docs work TODAY — they must be shown as actionable next steps. expect(block).toContain("meta gen"); expect(block).toContain("meta docs"); @@ -68,19 +68,17 @@ describe("init() — next-steps message (S1)", () => { expect(block).not.toMatch(/later sub-projects[\s\S]*meta docs\b/); }); - // The block used to be ONE static string, so it claimed the db stub on every run — - // including `meta init --force` in a project that keeps its own metaobjects.config.ts, - // where the stub is deliberately not written. Asserted on the message rather than on - // the run, because it is the message that was lying. - test("claims the src/db.ts stub only when this run actually wrote it", () => { - expect(nextStepsBlock(true)).toContain("src/db.ts"); - expect(nextStepsBlock(false)).not.toContain("src/db.ts"); - // Everything else is unconditional — a run that skipped the stub still gets the - // scaffold summary and the numbered steps. - for (const written of [true, false]) { - expect(nextStepsBlock(written)).toContain("codegen/generators/"); - expect(nextStepsBlock(written)).toContain("meta gen"); - } + // The block once took a `dbStubWritten` flag, because the scaffold wrote a throwing + // `src/db.ts` on some runs and not others while a static string claimed it on every + // one. Nothing is wired now, so there is no `dbImport` and no stub — the message is + // the same on every path, and must not mention a file init no longer writes. + test("mentions no db stub, and routes the reader through the catalog", () => { + const block = nextStepsBlock(); + expect(block).not.toContain("src/db.ts"); + expect(block).toContain("codegen/generators/"); + // The two commands that replace the wired-suite scaffold. + expect(block).toContain("meta gen --list"); + expect(block).toContain("meta eject"); }); }); @@ -206,78 +204,77 @@ describe("init() — happy path", () => { }); }); -// FR-040 fix round 1, Finding 1 — writeOwnedGenerators() used to loop over ALL of -// @metaobjectsdev/codegen-ts's REFERENCE_GENERATOR_NAMES unconditionally, so when an -// earlier task on this branch registered "routes-hono" there, `meta init` silently -// started scaffolding a fifth, unwired file: nothing in the scaffolded -// metaobjects.config.ts imports routesFileHono. This pins the exact set — a future -// template registered in REFERENCE_GENERATOR_NAMES must NOT silently join init's eager -// scaffold; it stays reachable only via `meta eject ` until a human decides -// otherwise. +// ADR-0034 Amendment 2 — `meta init` copies NO generators. // -// "names" is the one deliberate exception, added by spec §A5: TypeScript is opt-in by -// construction under ADR-0034 (meta gen runs the adopter's copy, so a packaged change -// can't reach anyone who has ejected), and §A5 rules that the honest maximum for existing -// projects is a one-line config addition while every NEW `meta init` gets it for free. -// That is the human decision this comment says the pin defers to — for "names" only. The -// pin still guards everything else (routes-hono stays eject-only below). -describe("init() — owned generator scaffold set (FR-040 fix round 1, Finding 1)", () => { - test("copies exactly the five generators the scaffolded config wires — not routes-hono", async () => { +// This block used to pin the exact five it scaffolded, and the pin was load-bearing: +// looping over the full REFERENCE_GENERATOR_NAMES array had once made init silently +// start writing an unwired `routes-hono.ts` into every fresh project. Opt-in codegen +// removes the whole class — there is no eager set to drift — so what is pinned now is +// that the directory is EMPTY and the door still works. +describe("init() — the owned-codegen tier is empty on purpose", () => { + test("creates codegen/generators/ and copies nothing into it", async () => { const result = await init({ cwd }); const dir = join(cwd, "codegen", "generators"); - const files = readdirSync(dir).sort(); - expect(files).toEqual(["barrel.ts", "entity.ts", "names.ts", "queries.ts", "routes.ts"]); - expect(existsSync(join(dir, "routes-hono.ts"))).toBe(false); + expect(existsSync(dir), "the directory is part of the promised layout").toBe(true); + expect(readdirSync(dir)).toEqual([]); + expect(result.created.filter((p) => p.startsWith("codegen/generators/"))).toEqual([]); + }); - for (const rel of [ - "codegen/generators/entity.ts", - "codegen/generators/queries.ts", - "codegen/generators/routes.ts", - "codegen/generators/barrel.ts", - "codegen/generators/names.ts", - ]) { - expect(result.created).toContain(rel); - } - expect(result.created).not.toContain("codegen/generators/routes-hono.ts"); + test("the scaffolded config wires nothing and points at the catalog", async () => { + await init({ cwd }); + const configSrc = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); + expect(configSrc).toContain("generators: []"); + // No IMPORT of an owned generator. The comment block deliberately NAMES the + // directory (it is where `meta eject` puts things), so the assertion is on the + // import statement, not on the string appearing anywhere in the file. + expect(configSrc).not.toMatch(/^import .* from "\.\/codegen\/generators\//m); + expect(configSrc).toContain("meta gen --list"); + expect(configSrc).toContain("meta eject"); }); - test("--print-only forecasts the same five-file set, not routes-hono", async () => { + test("--print-only forecasts the same: the tier, not files in it", async () => { const result = await init({ cwd, printOnly: true }); - expect(result.created).toContain("codegen/generators/entity.ts"); - expect(result.created).toContain("codegen/generators/queries.ts"); - expect(result.created).toContain("codegen/generators/routes.ts"); - expect(result.created).toContain("codegen/generators/barrel.ts"); - expect(result.created).toContain("codegen/generators/names.ts"); - expect(result.created).not.toContain("codegen/generators/routes-hono.ts"); + expect(result.created).toContain("codegen/generators"); + expect(result.created.filter((p) => p.startsWith("codegen/generators/"))).toEqual([]); }); - test("routes-hono is still reachable via `meta eject` — not scaffolded eagerly, but not missing", async () => { + test("every generator is reachable through `meta eject`", async () => { await init({ cwd }); - // Not written by init... - expect(existsSync(join(cwd, "codegen", "generators", "routes-hono.ts"))).toBe(false); - // ...but eject can still copy it on demand (proves it wasn't deregistered, only - // moved off the eager path). const { ejectGenerator } = await import("../src/commands/eject.js"); - const ejectResult = await ejectGenerator({ cwd, name: "routes-hono" }); - expect(ejectResult.status).toBe("created"); - expect(existsSync(join(cwd, "codegen", "generators", "routes-hono.ts"))).toBe(true); + for (const name of ["entity", "names", "routes-hono"]) { + const r = await ejectGenerator({ cwd, name }); + expect(r.status, name).toBe("created"); + expect(existsSync(join(cwd, "codegen", "generators", `${name}.ts`)), name).toBe(true); + } }); -}); -// Spec §A5 — the one deliberate exception to the pin above: every new `meta init` gets -// the names generator scaffolded AND wired, because that's the only route a packaged -// change to it can reach an ADR-0034 adopter through at all. -describe("init() — names generator is scaffolded and wired (spec §A5)", () => { - test("scaffolds codegen/generators/names.ts", async () => { - const dir = await init({ cwd }).then(() => join(cwd, "codegen", "generators")); - expect(existsSync(join(dir, "names.ts"))).toBe(true); + test("adds no dependencies to package.json — nothing is wired, so nothing is needed", async () => { + writeFileSync(join(cwd, "package.json"), JSON.stringify({ name: "x", version: "0.0.0" })); + await init({ cwd }); + const pkg = JSON.parse(readFileSync(join(cwd, "package.json"), "utf8")) as { + dependencies?: Record; + devDependencies?: Record; + }; + const declared = Object.keys({ ...pkg.dependencies, ...pkg.devDependencies }); + for (const d of ["drizzle-orm", "zod", "fastify", "@metaobjectsdev/codegen-ts", + "@metaobjectsdev/metadata", "@metaobjectsdev/runtime-ts"]) { + expect(declared, `must not declare ${d}`).not.toContain(d); + } }); - test("the scaffolded metaobjects.config.ts imports and wires namesFile()", async () => { + test("still sets `type: module` — the ESM rule is unchanged", async () => { + writeFileSync(join(cwd, "package.json"), JSON.stringify({ name: "x", version: "0.0.0" })); await init({ cwd }); - const configSrc = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); - expect(configSrc).toContain('import { namesFile } from "./codegen/generators/names.js";'); - expect(configSrc).toContain("namesFile()"); + const pkg = JSON.parse(readFileSync(join(cwd, "package.json"), "utf8")) as { type?: string }; + expect(pkg.type).toBe("module"); + }); + + test("scaffolds no src/db.ts — there is no dbImport to resolve", async () => { + await init({ cwd }); + expect(existsSync(join(cwd, "src", "db.ts"))).toBe(false); + // No dbImport KEY. The comment block names it among the config keys `meta eject` + // will tell you about when you take `routes`, which is the point. + expect(readFileSync(join(cwd, "metaobjects.config.ts"), "utf8")).not.toMatch(/^\s*dbImport:/m); }); }); @@ -570,231 +567,105 @@ describe("init --d1", () => { const code = await initCommand(["--d1"], cwd); expect(code).toBe(0); const configTs = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); - expect(configTs).toContain('dialect: "d1"'); + expect(configTs).toContain('dialect: "d1"'); }); test("scaffolds metaobjects.config.ts with dialect = 'sqlite' when --d1 is not passed", async () => { const code = await initCommand([], cwd); expect(code).toBe(0); const configTs = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); - expect(configTs).toContain('dialect: "sqlite"'); - }); - - test("scaffolds metaobjects.config.ts with outDir = 'src/generated' (not the ambiguous src/db)", async () => { - const code = await initCommand([], cwd); - expect(code).toBe(0); - const configTs = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); - expect(configTs).toContain('outDir: "src/generated"'); - // "./src/db" as outDir collides with the user-created src/db.ts that the - // generated routes import via dbImport "../db" — keep them distinct. - expect(configTs).not.toContain('"./src/db"'); + expect(configTs).toContain('dialect: "sqlite"'); }); -}); -describe("init() — scaffolded config.ts honesty", () => { - // A cold adoption probe found `dbImport: "../db"` scaffolded pointing at a file - // `init` never creates, with nothing in the config saying so. - // - // NOTE: an earlier draft of this fix commented `dbImport` out entirely. That - // regresses the default scaffold — verified by running `meta gen` against it: - // the default `generators` array wires routesFile() (Fastify), whose emitted - // routes DO `import { db } from …` (server/typescript/packages/codegen-ts/src/ - // templates/routes-file.ts), and the runner demands dbImport at that point of - // use (`runner.ts`'s dbImportUndeclaredFor), throwing `codegen config is - // missing dbImport` on the very first `meta gen` after a fresh `meta init`. - // `queriesFile` genuinely takes `db` as a parameter and never reads dbImport — - // but routesFile (what `init` actually scaffolds) does. So dbImport must stay - // ACTIVE; the fix is telling the user what to do about the file it names. - test("dbImport carries a comment naming the file the user must create", async () => { + test("scaffolds metaobjects.config.ts with outDir = 'src/generated'", async () => { const code = await initCommand([], cwd); expect(code).toBe(0); const configTs = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); - expect(configTs).toMatch(/dbImport:\s*"\.\.\/db",/); - expect(configTs).toContain("src/db.ts"); - }); - - test("meta gen still succeeds against the scaffolded config for an entity with a source.rdb", async () => { - // Regression pin for the near-miss above: the scaffold's dbImport must stay - // functional (routesFile() genuinely needs it), not just present-with-a-comment. - // Runs `gen`, so it needs an in-package project dir — see mkGenProjectDir. - const dir = mkGenProjectDir("dbimport-gen-"); - try { - expect(await initCommand([], dir)).toBe(0); - writeFileSync( - join(dir, "metaobjects", "meta.common.json"), - PROBE_ENTITY, - ); - const { genCommand } = await import("../src/commands/gen.js"); - expect(await genCommand([], dir)).toBe(0); - const routes = readFileSync(join(dir, "src", "generated", "Author.routes.ts"), "utf8"); - expect(routes).toContain('import { db } from "../db.js"'); - } finally { - rmSync(dir, { recursive: true, force: true }); - } + expect(configTs).toContain('outDir: "src/generated"'); }); }); -// Task 15 — closes the residue Task 9 (above) made discoverable but did not -// eliminate: `meta init` declared `dbImport: "../db"` but never created the -// module, so a fresh project's FIRST `tsc` failed to resolve it. The fix is a -// scaffolded THROWING STUB: it types clean and satisfies every generated -// import, choosing no driver and adding no dependency, but throws a clear, -// actionable error the first time anything actually touches `db` at runtime. -describe("init() — dbImport throwing stub (Task 15)", () => { - test("scaffolds src/db.ts as a throwing stub, typed without `any`, that exports `db`", async () => { +// A cold adoption probe once found `dbImport: "../db"` scaffolded pointing at a file +// `init` never created, and the fix at the time was a throwing `src/db.ts` stub plus a +// comment — because the scaffold WIRED routesFile(), whose emitted routes do +// `import { db } from …`. +// +// Opt-in codegen removes the premise rather than the symptom. Nothing is wired, so +// nothing emits that import, so there is no `dbImport` to point anywhere and no stub to +// scaffold. `dbImport` became a declared `configKey` on the `routes` catalog entry: +// `meta eject routes` reports it, to the adopter who actually chose routes. What these +// tests pin now is that BOTH artifacts are gone and the config-key path works. +describe("init() — no dbImport, no db stub, and routes still works once chosen", () => { + test("neither the key nor the stub is scaffolded", async () => { const result = await init({ cwd }); - expect(result.created).toContain("src/db.ts"); - const body = readFileSync(join(cwd, "src", "db.ts"), "utf8"); - // Zero live imports — no driver chosen, no dependency added. (Driver names - // may appear in the comment's illustrative example line; that's the point.) - expect(body).not.toMatch(/^import /m); - expect(body).not.toMatch(/\bany\b/); - expect(body).toContain("export const db: unknown"); - // Actually throws on first real use, rather than silently no-op-ing. - expect(body).toContain("new Proxy("); - expect(body).toContain("throw new Error("); - }); - - test("the thrown message names the file and shows a concrete replacement line", async () => { - await init({ cwd }); - const body = readFileSync(join(cwd, "src", "db.ts"), "utf8"); - expect(body).toContain("src/db.ts"); - expect(body).toContain("export const db = drizzle("); - }); - - test("does NOT clobber an existing src/db.ts on a re-run with --force", async () => { - await init({ cwd }); - const realDb = 'import { drizzle } from "drizzle-orm/better-sqlite3";\nexport const db = drizzle({} as never);\n'; - writeFileSync(join(cwd, "src", "db.ts"), realDb, "utf8"); - - const result = await init({ cwd, force: true }); - - expect(result.preserved).toContain("src/db.ts"); expect(result.created).not.toContain("src/db.ts"); - expect(readFileSync(join(cwd, "src", "db.ts"), "utf8")).toBe(realDb); - }); - - // Fix round 2. DB_STUB_REL_PATH is derived from the SCAFFOLD's own outDir/dbImport, - // so it describes where the SCAFFOLDED config points and nowhere else. Re-running - // init in a project that already has its own config preserves that config — and must - // not then drop a src/db.ts answering a question the adopter already answered - // differently. A stray unreferenced file in application source is exactly the - // unilateral host-project touch FR-040 §4.4 lists as a defect. - test("writes no db stub when the project keeps its own config", async () => { - writeFileSync( - join(cwd, "metaobjects.config.ts"), - [ - `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, - `export default defineConfig({ outDir: "packages/db/src/generated",`, - ` dbImport: "../../conn", dialect: "sqlite", generators: [] });`, - ].join("\n"), - "utf8", - ); - - const result = await init({ cwd, force: true }); - expect(existsSync(join(cwd, "src", "db.ts"))).toBe(false); - expect(result.created).not.toContain("src/db.ts"); - // Nor may it CLAIM to have preserved one — there is no such file to preserve. - expect(result.preserved).not.toContain("src/db.ts"); - // ...but it must not be SILENT about it either — see the test below. - expect(result.warnings.join("\n")).toContain("src/db.ts"); - }); - - // Fix round 3. The rule above ("keeps its own config ⇒ write nothing") is right, but - // `wroteScaffoldedConfig` is only "no metaobjects.config.ts existed" — so it is FALSE - // for the config `meta init` itself just wrote. A scaffolded project whose src/db.ts - // is later deleted or moved therefore hits the same branch: init writes nothing, and - // used to report nothing either, while the scaffolded `dbImport: "../db"` and the - // generated routes' `import { db } from "../db.js"` still point at it. The adopter met - // that as a TS2307 from `tsc`, with no word from the command that could have said so. - test("warns rather than going silent when a scaffolded project has lost its src/db.ts", async () => { - await init({ cwd }); - rmSync(join(cwd, "src", "db.ts")); - - const result = await init({ cwd, force: true }); + const configTs = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); + expect(configTs).not.toMatch(/^\s*dbImport:/m); + }); - // Still no write — a project owning its config owns its database module. - expect(existsSync(join(cwd, "src", "db.ts"))).toBe(false); + test("--print-only forecasts no db stub either", async () => { + const result = await init({ cwd, printOnly: true }); expect(result.created).not.toContain("src/db.ts"); - expect(result.preserved).not.toContain("src/db.ts"); - // But the run has to SAY so, naming the path and what breaks. - const warned = result.warnings.find((w) => w.includes("src/db.ts")); - expect(warned).toBeDefined(); - expect(warned).toContain("dbImport"); }); - test("dry run (--print) reports src/db.ts as a would-be-created file", async () => { - const result = await init({ cwd, printOnly: true }); - expect(result.created).toContain("src/db.ts"); - expect(existsSync(join(cwd, "src", "db.ts"))).toBe(false); + test("`meta eject routes` reports dbImport as the key to set", async () => { + await init({ cwd }); + const { ejectCommand } = await import("../src/commands/eject.js"); + const lines: string[] = []; + const origLog = console.log; + console.log = (...a: unknown[]) => { lines.push(a.join(" ")); }; + try { + expect(await ejectCommand(["routes"], cwd, "json")).toBe(0); + } finally { + console.log = origLog; + } + const payload = JSON.parse(lines.join("\n")) as { config: { keys: string[] } }; + expect(payload.config.keys).toContain("dbImport"); }); - // The headline gate: the documented sequence — init, author an entity with a - // source.rdb child, gen, tsc — must all succeed with no unresolved-module - // error. Mirrors the "meta gen still succeeds..." regression pin above, one - // step further: it actually type-checks the generated output + the scaffolded - // stub with the real TypeScript compiler this repo depends on, under the same - // nodenext options a stock `tsc --init` project resolves relative imports - // with. Like the pin above it runs `gen`, so the project dir is in-package — - // see mkGenProjectDir for why the OS tmpdir is wrong for both of them. - test("end to end: init -> author a source.rdb entity -> gen -> tsc resolves dbImport with no unresolved-module error", async () => { - const dir = mkGenProjectDir("dbstub-tsc-"); + test("a project that wires routes and sets dbImport generates exactly as before", async () => { + // The regression pin the old block carried, moved to the path that now reaches it: + // routes is CHOSEN, dbImport is SET, and the emitted import resolves. + const dir = mkGenProjectDir("dbimport-gen-"); try { expect(await initCommand([], dir)).toBe(0); - writeFileSync( - join(dir, "metaobjects", "meta.common.json"), - PROBE_ENTITY, - ); + writeFileSync(join(dir, "metaobjects", "meta.common.json"), PROBE_ENTITY); + + const { ejectCommand } = await import("../src/commands/eject.js"); + const origLog = console.log; + console.log = () => {}; + try { + expect(await ejectCommand(["entity", "routes"], dir, "text")).toBe(0); + } finally { + console.log = origLog; + } + + const configPath = join(dir, "metaobjects.config.ts"); + writeFileSync(configPath, [ + 'import { defineConfig } from "@metaobjectsdev/cli";', + 'import { entityFile } from "./codegen/generators/entity.js";', + 'import { routesFile } from "./codegen/generators/routes.js";', + "export default defineConfig({", + ' outDir: "src/generated",', + ' dialect: "sqlite",', + ' extStyle: "js",', + ' dbImport: "../db",', + " generators: [entityFile(), routesFile()],", + "});", + "", + ].join("\n")); + const { genCommand } = await import("../src/commands/gen.js"); expect(await genCommand([], dir)).toBe(0); - - const generatedDir = join(dir, "src", "generated"); - const rootFiles = [ - join(generatedDir, "Author.ts"), - join(generatedDir, "Author.queries.ts"), - join(generatedDir, "Author.routes.ts"), - join(dir, "src", "db.ts"), - ]; - const program = ts.createProgram(rootFiles, { - noEmit: true, - strict: true, - skipLibCheck: true, - target: ts.ScriptTarget.ES2022, - module: ts.ModuleKind.NodeNext, - moduleResolution: ts.ModuleResolutionKind.NodeNext, - }); - const diagnostics = ts.getPreEmitDiagnostics(program); - // TS2307 = "Cannot find module" — the exact class of error the missing - // src/db.ts used to produce. Scoped to this one code, rather than asserting - // zero diagnostics overall, so this gate stays about the dbImport defect it - // exists to catch and not about every diagnostic a real compiler could ever - // emit here. (This repo's own `cli` devDependency on `fastify` used to skew - // against `runtime-ts`'s peer range and surface as an unrelated structural - // TS2740 under this exact compile — fixed by aligning the devDependency to - // the peer range, FR-040 fix round 1 Item 4; confirmed empirically that this - // compile now produces zero diagnostics of any code, not just none at 2307.) - const unresolvedModules = diagnostics - .filter((d) => d.code === 2307) - .map((d) => ts.flattenDiagnosticMessageText(d.messageText, "\n")); - expect(unresolvedModules).toEqual([]); + const routes = readFileSync(join(dir, "src", "generated", "Author.routes.ts"), "utf8"); + expect(routes).toContain('import { db } from "../db.js"'); } finally { rmSync(dir, { recursive: true, force: true }); } - }, 30_000); + }); }); -// `meta gen` loads metaobjects.config.ts and ./codegen/** through jiti, which -// TRANSPILES WITHOUT TYPECHECKING — so a generator can import a symbol the engine no -// longer exports and every gate stays green until the import is evaluated, which for -// an unwired generator may be never. A project's app tsconfig covers src/ and tests/ -// and not this tier, so without a tsconfig of its own NOTHING compiles the code the -// build depends on. -// -// Reported from an estate that found two real defects the moment one existed: a -// generator importing `CODEGEN_ATTR_EMIT_ROUTES` (retired with the `@emit*` family, -// invisible because that generator was not wired), and a WIRED generator carrying -// five type errors including `ownFields()` on a node with no such method. describe("meta init scaffolds a tsconfig for the tier it just handed you", () => { test("writes tsconfig.codegen.json and reports it", async () => { const result = await init({ cwd }); diff --git a/server/typescript/packages/cli/test/integration/gen-libraries.test.ts b/server/typescript/packages/cli/test/integration/gen-libraries.test.ts index b4d7955d3..3c77bd958 100644 --- a/server/typescript/packages/cli/test/integration/gen-libraries.test.ts +++ b/server/typescript/packages/cli/test/integration/gen-libraries.test.ts @@ -1,6 +1,9 @@ /** * #333 — a project opts into a MetaObjects-shipped library with `libraries` in - * `metaobjects.config.ts`, and `extends: "metaobjects::ai::LlmCallBase"` resolves. + * `.metaobjects/config.json`, and `extends: "metaobjects::ai::LlmCallBase"` resolves. + * + * The key moved there from `metaobjects.config.ts` in FR-043: which designs a project + * adopts is a fact about the PROJECT, not about how one port generates code from it. * * `librarySources` was reachable only from `MetaDataLoader.fromDirectory`, which the CLI * does not use, so a generator that consumes a library was registered FOR the command @@ -43,7 +46,16 @@ function setup(libraries: string[] | undefined): { root: string; outDir: string mkdirSync(WORKSPACE_TMP, { recursive: true }); const root = mkdtempSync(join(WORKSPACE_TMP, "forge-libraries-")); mkdirSync(join(root, "metaobjects"), { recursive: true }); + mkdirSync(join(root, ".metaobjects"), { recursive: true }); writeFileSync(join(root, "metaobjects", "trace.json"), MODEL, "utf8"); + writeFileSync( + join(root, ".metaobjects", "config.json"), + `${JSON.stringify( + { schema_version: 1, sources: [], ...(libraries === undefined ? {} : { libraries }) }, + null, + 2, + )}\n`, + ); const outDir = join(root, "generated"); writeFileSync( join(root, "metaobjects.config.ts"), @@ -54,7 +66,7 @@ export default defineConfig({ dialect: "postgres", dbImport: "~/db", extStyle: "none", -${libraries === undefined ? "" : ` libraries: ${JSON.stringify(libraries)},\n`} generators: ["entity"], + generators: ["entity"], }); `, ); diff --git a/server/typescript/packages/cli/test/library-eject-staleness-packages.test.ts b/server/typescript/packages/cli/test/library-eject-staleness-packages.test.ts new file mode 100644 index 000000000..d4be401c5 --- /dev/null +++ b/server/typescript/packages/cli/test/library-eject-staleness-packages.test.ts @@ -0,0 +1,95 @@ +// FR-043 §3.4 — `meta eject --list` staleness across a MULTI-PACKAGE library. +// +// The staleness comparison keys root-level nodes by NAME rather than by resolution key, +// because §3.4 invites an adopter who ejects to "rename the package freely" — keying on +// the FQN would report every node of a renamed copy as both upstream-only and local-only. +// +// The bare name alone is not enough to be a key. A library that ships two root-level +// nodes with the SAME bare name in DIFFERENT packages collapses both onto one entry, and +// whichever is serialized last silently wins — so a real divergence in the OTHER one is +// reported as "identical", which is the worst answer a staleness check can give. Today's +// shipped libraries each use a single package, so nothing exercises it; these tests do, +// so the second-package library cannot land the misreport with it. +// +// The key is therefore the node's package PATH RELATIVE to the library's own root +// package, which survives the one edit §3.4 invites: renaming `metaobjects::iam` to +// `acme::identity` moves the root and leaves the `admin` suffix exactly where it was. +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource, type MetaData } from "@metaobjectsdev/metadata"; +import { ownNodesByName } from "../src/lib/library-eject.js"; + +interface Extra { + /** An added field on the root-package `User`. */ + base?: string; + /** An added field on the `admin`-package `User`. */ + admin?: string; +} + +function object(pkg: string, field: string, extra?: string): InMemoryStringSource { + return new InMemoryStringSource( + `metadata:\n package: ${pkg}\n children:\n` + + ` - object.value:\n name: User\n children:\n` + + ` - field.string: { name: ${field} }\n` + + (extra === undefined ? "" : ` - field.string: { name: ${extra} }\n`), + { id: `library:${pkg}.yaml`, format: "yaml" }, + ); +} + +/** One root-level node per package, so the two share a bare name and nothing else. */ +async function load(root: string, extra: Extra = {}): Promise { + const result = await new MetaDataLoader({ strict: true }).load([ + object(root, "email", extra.base), + object(`${root}::admin`, "scope", extra.admin), + ]); + expect(result.errors).toEqual([]); + return result.root; +} + +/** What `libraryStaleness` does with the two maps, in the same three counters. */ +function compare(up: Map, mine: Map) { + let changed = 0, upstreamOnly = 0, localOnly = 0; + for (const [key, text] of up) { + if (!mine.has(key)) upstreamOnly++; + else if (mine.get(key) !== text) changed++; + } + for (const key of mine.keys()) if (!up.has(key)) localOnly++; + return { changed, upstreamOnly, localOnly }; +} + +describe("eject staleness across a multi-package library", () => { + test("two same-named nodes in different packages stay DISTINCT", async () => { + const keys = [...ownNodesByName(await load("metaobjects::iam")).keys()].sort(); + // Not one entry. `admin::User` is a different declaration from `User` and the + // comparison has to be able to say so. + expect(keys).toEqual(["User", "admin::User"]); + }); + + test("a renamed package still compares as identical — §3.4's invited edit", async () => { + const up = ownNodesByName(await load("metaobjects::iam")); + const mine = ownNodesByName(await load("acme::identity")); + expect(compare(up, mine)).toEqual({ changed: 0, upstreamOnly: 0, localOnly: 0 }); + }); + + test("an edit to the ROOT-package node is reported, not swallowed by its namesake", async () => { + const up = ownNodesByName(await load("metaobjects::iam")); + const mine = ownNodesByName(await load("acme::identity", { base: "nickname" })); + // Keyed by bare name, the `admin` copy overwrites this one on BOTH sides and the + // edit reads as `identical` — a staleness check answering the one question it + // exists to answer with the wrong answer. + expect(compare(up, mine)).toEqual({ changed: 1, upstreamOnly: 0, localOnly: 0 }); + }); + + test("an edit to the DEEPER package is reported too", async () => { + const up = ownNodesByName(await load("metaobjects::iam")); + const mine = ownNodesByName(await load("acme::identity", { admin: "grantedBy" })); + expect(compare(up, mine)).toEqual({ changed: 1, upstreamOnly: 0, localOnly: 0 }); + }); + + test("an edit to BOTH is reported as two", async () => { + const up = ownNodesByName(await load("metaobjects::iam")); + const mine = ownNodesByName( + await load("acme::identity", { base: "nickname", admin: "grantedBy" }), + ); + expect(compare(up, mine)).toEqual({ changed: 2, upstreamOnly: 0, localOnly: 0 }); + }); +}); diff --git a/server/typescript/packages/cli/test/library-manifest-resolved.test.ts b/server/typescript/packages/cli/test/library-manifest-resolved.test.ts new file mode 100644 index 000000000..8759cad79 --- /dev/null +++ b/server/typescript/packages/cli/test/library-manifest-resolved.test.ts @@ -0,0 +1,141 @@ +// FR-043 §4 — "every manifest fact is RESOLVED, not trusted". +// +// `library.json` is the record `meta gen --list` renders, and every field in it is a +// claim about something else: a package the library declares, a ref that is embedded, +// a generator the registry registers, a node the library ships. A manifest is the +// easiest kind of file to leave behind — nothing in a normal run reads it against +// reality — so each claim is checked against the thing it claims here. +// +// The one rule that is NOT about a single library is the last: a library name and a +// generator stable name live in ONE namespace, because they are rows in one table and +// `meta eject ` takes either. +import { describe, test, expect } from "bun:test"; +import { existsSync, readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { MetaDataLoader, type MetaData } from "@metaobjectsdev/metadata"; +import { + libraryManifests, librarySources, knownLibraryTokens, splitLayerToken, +} from "@metaobjectsdev/metadata/library"; +import { SERVER_LANGS } from "@metaobjectsdev/sdk"; +import { composeCatalog } from "../src/lib/catalog.js"; + +const MANIFESTS = libraryManifests(); +const NAMES = Object.keys(MANIFESTS).sort(); + +function findRepoRoot(start: string): string { + let dir = start; + for (;;) { + if (existsSync(join(dir, "library")) && existsSync(join(dir, "server"))) return dir; + const parent = dirname(dir); + if (parent === dir) throw new Error("no repo root (a dir holding library/ and server/)"); + dir = parent; + } +} + +/** Load one library with every layer it declares. */ +async function loadWhole(name: string): Promise { + const tokens = knownLibraryTokens().filter((t) => splitLayerToken(t)[0] === name); + const result = await new MetaDataLoader({ strict: true }).load(librarySources(tokens)); + expect(result.errors, `${name} loads clean`).toEqual([]); + return result.root; +} + +function packagesDeclaredIn(root: MetaData): Set { + const out = new Set(); + const walk = (n: MetaData): void => { + for (const c of n.children()) { + const pkg = c.package ?? c.fileDefaultPackage ?? ""; + if (pkg !== "") out.add(pkg); + walk(c); + } + }; + walk(root); + return out; +} + +function namesIn(root: MetaData): Set { + const out = new Set(); + const walk = (n: MetaData): void => { + for (const c of n.children()) { + out.add(c.name); + const pkg = c.package ?? c.fileDefaultPackage ?? ""; + if (pkg !== "") out.add(`${pkg}::${c.name}`); + walk(c); + } + }; + walk(root); + return out; +} + +describe("every library manifest fact is resolved against the thing it claims", () => { + test("there is at least one manifest to check", () => { + expect(NAMES.length).toBeGreaterThan(0); + }); + + for (const name of NAMES) { + const manifest = MANIFESTS[name]!; + + test(`${name}: the manifest key, the \`name\` field and the package's last segment agree`, () => { + expect(manifest.name).toBe(name); + for (const pkg of manifest.packages ?? []) { + expect(pkg.split("::").pop(), `${pkg} should end in the library's own name`).toBe(name); + } + }); + + test(`${name}: \`packages\` is exactly what the library declares — both ways`, async () => { + const declared = packagesDeclaredIn(await loadWhole(name)); + const claimed = new Set(manifest.packages ?? []); + // Both directions: a package claimed but empty is a promise the library does not + // keep, and a package declared but unclaimed escapes the ownership refusal (§3.5) + // that reads this list. + expect([...declared].filter((p) => !claimed.has(p)).sort(), "declared but not claimed").toEqual([]); + expect([...claimed].filter((p) => !declared.has(p)).sort(), "claimed but not declared").toEqual([]); + }); + + test(`${name}: every implied generator is a real stable name, and its anchor a real node`, async () => { + const catalog = composeCatalog(); + const root = await loadWhole(name); + const nodes = namesIn(root); + for (const g of manifest.generators ?? []) { + expect(catalog[g.name], `${name} implies generator "${g.name}", which nothing registers`).toBeDefined(); + if (g.anchor === undefined) continue; + // The anchor is what retires the generator's hard-coded entity name, so it has + // to resolve in the library it is declared by — not merely look plausible. + expect(nodes.has(g.anchor), `${name}'s anchor "${g.anchor}" is not a node it ships`).toBe(true); + } + }); + + test(`${name}: every layer's refs are embedded in EVERY port`, () => { + // The row says `ports: [typescript, java, kotlin, csharp, python]`, and the basis + // for that claim is that one generator script writes the embed for all of them. + // Checked rather than asserted, because "the library is reachable from your port" + // is the single fact a polyglot adopter acts on. Kotlin has no embed of its own — + // it runs on the JVM and reads Java's, which is why it is mapped onto that file. + const root = findRepoRoot(import.meta.dir); + const EMBEDS: Record = { + typescript: "server/typescript/packages/metadata/src/library/embedded-library.generated.ts", + java: "server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java", + kotlin: "server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java", + csharp: "server/csharp/MetaObjects/Library/EmbeddedLibrary.cs", + python: "server/python/src/metaobjects/library/embedded_library.py", + }; + expect(Object.keys(EMBEDS).sort(), "a port with no embed mapping").toEqual([...SERVER_LANGS].sort()); + + const refs = Object.values(manifest.layers ?? {}).flatMap((l) => [...l.refs]); + expect(refs.length, `${name} declares no refs at all`).toBeGreaterThan(0); + for (const port of SERVER_LANGS) { + const text = readFileSync(join(root, EMBEDS[port]!), "utf8"); + for (const ref of refs) { + expect(text.includes(`"${ref}"`), `${port} does not embed ${ref}`).toBe(true); + } + } + }); + } + + test("a library name and a generator stable name share ONE namespace", () => { + // They are rows in one table, and `meta eject ` takes either. A collision + // would make the catalog ambiguous in exactly the place an agent acts on it. + const collisions = NAMES.filter((n) => n in composeCatalog()); + expect(collisions, `library names that are also generator names: ${collisions.join(", ")}`).toEqual([]); + }); +}); diff --git a/server/typescript/packages/cli/test/scaffold-output-imports-declared.test.ts b/server/typescript/packages/cli/test/scaffold-output-imports-declared.test.ts index 20bc914d0..03cd12693 100644 --- a/server/typescript/packages/cli/test/scaffold-output-imports-declared.test.ts +++ b/server/typescript/packages/cli/test/scaffold-output-imports-declared.test.ts @@ -1,4 +1,5 @@ -// Everything `meta gen` WRITES must be resolvable from what `meta init` DECLARED. +// Everything `meta gen` WRITES must be resolvable from what the toolchain TOLD YOU to +// install. // // It was not. A brand-new project — `npm init -y`, install the CLI, `meta init`, // `meta gen`, then `npx tsc`, which is the next step `meta gen` itself prints — @@ -8,24 +9,26 @@ // CLI's own dependency tree; pnpm's strict layout shows all five, which is why the // release smoke test runs both. // -// `addScaffoldDevDependencies` had already fixed this defect ONE LAYER IN — the -// generator sources under `codegen/generators/` import `@metaobjectsdev/codegen-ts` and -// `@metaobjectsdev/metadata`, and its own docstring records the scaffold arriving -// un-typecheckable with ten TS2307s. The identical argument reaches the generated -// output and nobody followed it there. +// **Who is asked has changed; the question has not.** `meta init` used to wire a suite +// and therefore had to declare that suite's dependencies. Codegen is opt-in now: init +// wires nothing, declares nothing, and generates nothing, so asking init the question +// would pass vacuously. The obligation MOVED to `meta eject`, which reports the install +// set for exactly the generators you took — so that is what this test now applies and +// then checks the generated output against. // -// Why no existing test could see it: every test that runs `gen` scaffolds into -// `test/fixtures/__tmp__/`, deliberately, so that node resolution walks up into cli's -// OWN node_modules — where fastify, drizzle-orm and zod all sit as devDependencies. -// `mkGenProjectDir`'s comment says so outright. So the generated imports resolved in -// every test for a reason that has nothing to do with the adopter's manifest. This test -// runs in the same place and asks a different question: not "does it resolve here" but -// "did the scaffolder DECLARE it", which is answerable from the manifest alone and is -// exactly what a strict installer will enforce. +// Why no ordinary `gen` test could see the original defect, and still cannot: every +// test that runs `gen` scaffolds into `test/fixtures/__tmp__/`, deliberately, so node +// resolution walks up into cli's OWN node_modules — where fastify, drizzle-orm and zod +// all sit as devDependencies. `mkGenProjectDir`'s comment says so outright. So the +// generated imports resolve in every test for a reason that has nothing to do with the +// adopter's manifest. This test runs in the same place and asks a different question: +// not "does it resolve here" but "was it DECLARED", which is answerable from the +// manifest alone and is exactly what a strict installer enforces. import { describe, test, expect } from "bun:test"; import { mkdtempSync, mkdirSync, rmSync, readFileSync, writeFileSync, readdirSync } from "node:fs"; import { join } from "node:path"; import { initCommand } from "../src/commands/init.js"; +import { ejectCommand } from "../src/commands/eject.js"; import { declaredDependencyNames, type PackageManifest } from "../src/lib/package-manifest.js"; const PROBE_ENTITY = JSON.stringify({ @@ -45,6 +48,9 @@ const PROBE_ENTITY = JSON.stringify({ }, }, null, 2); +/** The selection this test adopts — the server-side shape `meta init` used to wire. */ +const SELECTION = ["entity", "queries", "routes", "barrel"] as const; + /** Bare package specifiers imported by a generated file — not relative, not `node:`. * A subpath resolves to its package (`@scope/pkg/sub` -> `@scope/pkg`), which is what * a manifest declares. */ @@ -59,19 +65,44 @@ function bareImports(source: string): Set { return found; } +/** `name@range` -> `[name, range]`, tolerating a scoped name's leading `@`. */ +function splitSpec(spec: string): [string, string] { + const at = spec.lastIndexOf("@"); + return at <= 0 ? [spec, "*"] : [spec.slice(0, at), spec.slice(at + 1)]; +} + +/** Apply an eject payload's install set to the project's package.json — what an adopter + * does when they run the printed `install.command`. */ +function applyInstallSet( + dir: string, + install: { dev: string[]; runtime: string[] }, +): void { + const path = join(dir, "package.json"); + const pkg = JSON.parse(readFileSync(path, "utf8")) as Record; + const deps = (pkg.dependencies ?? {}) as Record; + const devDeps = (pkg.devDependencies ?? {}) as Record; + for (const spec of install.runtime) { const [n, r] = splitSpec(spec); deps[n] = r; } + for (const spec of install.dev) { const [n, r] = splitSpec(spec); devDeps[n] = r; } + pkg.dependencies = deps; + pkg.devDependencies = devDeps; + writeFileSync(path, `${JSON.stringify(pkg, null, 2)}\n`); +} + /** * THE STARTING MANIFEST IS A DIMENSION, and leaving it fixed is what let this ship broken. * - * The declarations were made inside the function that fixes the module system, BELOW its - * early returns — so they happened only on the one path where `"type"` had to be changed. - * A project that already said `"type": "module"` got no declarations and no warning, and - * the guard's own comment read "nothing to do, nothing to say". That is the adopter who - * set their project up correctly, and under a strict installer their first `tsc` reported - * TS2307 on all five files `meta init` had just written. + * The declarations used to be made inside the function that fixes the module system, + * BELOW its early returns — so they happened only on the one path where `"type"` had to + * be changed. A project that already said `"type": "module"` got no declarations and no + * warning, and the guard's own comment read "nothing to do, nothing to say". That is the + * adopter who set their project up correctly, and under a strict installer their first + * `tsc` reported TS2307 on all five files `meta init` had just written. * - * This test could not see it because it used exactly one shape — `npm init -y`'s + * The old test could not see it because it used exactly one shape — `npm init -y`'s * `"type": "commonjs"` — which is the path that worked. A cold adoption probe on * Kysely + Express + pnpm found it in about ten minutes, on rc.4, the day the fix shipped. + * The dimension is kept even though the install set no longer travels through that + * function: the lesson is about fixing one shape, not about that one function. */ const STARTING_MANIFESTS = [ // The `npm init -y` shape, which is the documented first step. @@ -82,13 +113,33 @@ const STARTING_MANIFESTS = [ { label: "no type field", type: undefined }, ]; -describe("meta init declares what meta gen's output imports", () => { +function scaffoldedConfig(): string { + return [ + 'import { defineConfig } from "@metaobjectsdev/cli";', + 'import { entityFile } from "./codegen/generators/entity.js";', + 'import { queriesFile } from "./codegen/generators/queries.js";', + 'import { routesFile } from "./codegen/generators/routes.js";', + 'import { barrel } from "./codegen/generators/barrel.js";', + "export default defineConfig({", + ' outDir: "src/generated",', + ' dialect: "sqlite",', + ' extStyle: "js",', + ' dbImport: "../db",', + " generators: [entityFile(), queriesFile(), routesFile(), barrel()],", + "});", + "", + ].join("\n"); +} + +describe("meta eject declares what meta gen's output imports", () => { for (const manifest of STARTING_MANIFESTS) { test(`every bare specifier in the generated files is a declared dependency — ${manifest.label}`, async () => { - // In-package, so `gen` can resolve the scaffolded generators' own imports. + // In-package, so `gen` can resolve the ejected generators' own imports. const root = join(import.meta.dirname, "fixtures", "__tmp__"); mkdirSync(root, { recursive: true }); const dir = mkdtempSync(join(root, "scaffold-deps-")); + const origLog = console.log; + const logged: string[] = []; try { writeFileSync( join(dir, "package.json"), @@ -102,6 +153,17 @@ describe("meta init declares what meta gen's output imports", () => { ); expect(await initCommand([], dir)).toBe(0); writeFileSync(join(dir, "metaobjects", "meta.common.json"), PROBE_ENTITY); + + // Take a selection, and do what its install set says. + console.log = (...a: unknown[]) => { logged.push(a.join(" ")); }; + expect(await ejectCommand([...SELECTION], dir, "json")).toBe(0); + console.log = origLog; + const payload = JSON.parse(logged.join("\n")) as { + install: { dev: string[]; runtime: string[] }; + }; + applyInstallSet(dir, payload.install); + writeFileSync(join(dir, "metaobjects.config.ts"), scaffoldedConfig()); + const { genCommand } = await import("../src/commands/gen.js"); expect(await genCommand([], dir)).toBe(0); @@ -129,8 +191,30 @@ describe("meta init declares what meta gen's output imports", () => { expect([...all]).toContain(required); } } finally { + console.log = origLog; rmSync(dir, { recursive: true, force: true }); } }); } + + test("`meta init` alone declares NOTHING — it wires nothing, so it needs nothing", async () => { + const root = join(import.meta.dirname, "fixtures", "__tmp__"); + mkdirSync(root, { recursive: true }); + const dir = mkdtempSync(join(root, "scaffold-nodeps-")); + try { + writeFileSync(join(dir, "package.json"), `${JSON.stringify({ name: "probe", version: "1.0.0" }, null, 2)}\n`); + expect(await initCommand([], dir)).toBe(0); + const declared = declaredDependencyNames( + JSON.parse(readFileSync(join(dir, "package.json"), "utf8")) as PackageManifest, + ); + // The five init used to add. Declaring any of them now would be declaring a + // dependency on code this project may never generate. + for (const d of ["drizzle-orm", "zod", "fastify", + "@metaobjectsdev/codegen-ts", "@metaobjectsdev/metadata"]) { + expect([...declared], `must not declare ${d}`).not.toContain(d); + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); }); diff --git a/server/typescript/packages/cli/test/shipped-library-verify.test.ts b/server/typescript/packages/cli/test/shipped-library-verify.test.ts new file mode 100644 index 000000000..009024ed9 --- /dev/null +++ b/server/typescript/packages/cli/test/shipped-library-verify.test.ts @@ -0,0 +1,86 @@ +// FR-043 §8 item 2 — every SHIPPED library passes `meta verify` standalone. +// +// This is the gate the library's own ledger is held to, and it is separate from the +// load test in `metadata/test/library-load.test.ts` for one reason: loading clean and +// VERIFYING clean are different claims. A library whose requirements load fine can +// still ship `ERR_REQUIREMENT_L4_NOT_OBJECT` — and it did, in both libraries, until +// this test existed. Every adopter who opts in runs `meta verify` over metadata they +// did not write and cannot fix, so a finding here is a finding in every adopter's +// build. +// +// STANDALONE, with no project files at all (`files: []`), because that is the claim +// §4 makes: "every @implementedBy resolving WITHIN the library standalone — a +// library's ledger must be self-contained". A library that only verifies alongside +// an adopter's model has a hidden dependency on it. +// +// EVERY TOKEN, derived from the manifests rather than listed here. A new library, or +// a new layer of an existing one, is gated the day it is embedded; a hard-coded list +// would gate the two that exist and silently skip the third. +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "@metaobjectsdev/metadata"; +import { knownLibraryTokens, librarySources } from "@metaobjectsdev/metadata/library"; +import { + checkRequirements, summariseRequirements, scanRequirements, +} from "../src/lib/requirement-check.js"; +import { lintRequirements } from "../src/lib/requirement-lint.js"; + +const TOKENS = knownLibraryTokens(); + +describe("every shipped library verifies standalone (FR-043 §8 item 2)", () => { + test("there is something to gate", () => { + // A guard on the derivation itself: if `knownLibraryTokens()` ever returned an + // empty list — a stale embed, a renamed manifest key — every loop below would + // pass by iterating nothing. + expect(TOKENS.length).toBeGreaterThan(0); + expect(TOKENS).toContain("iam"); + }); + + for (const token of TOKENS) { + test(`\`${token}\` loads with no errors and no warnings`, async () => { + const result = await new MetaDataLoader({ strict: true }).load(librarySources([token])); + expect(result.errors, `${token} errors`).toEqual([]); + // Warnings matter as much as errors here: `WARN_LEGACY [filterable-without-index]` + // on one library field would print in every adopter's run forever, and nobody but + // this repo can silence it. + expect(result.warnings, `${token} warnings`).toEqual([]); + }); + + test(`\`${token}\` passes the requirement gate, with nothing left to rule on`, async () => { + const result = await new MetaDataLoader({ strict: true }).load(librarySources([token])); + const root = result.root; + // Coverage is MEASURED here even though FR-043 §5.4 switches it off for an + // adopter with no requirements of their own: standalone, the library IS the + // project under test, and "every entity this library ships is claimed by its + // own ledger" is exactly what the library owes. Without the override this gate + // would silently stop checking the thing it was written for. + const scan = scanRequirements(root, { measureCoverage: true }); + const diags = checkRequirements(root, scan); + expect( + diags.map((d) => `[${d.severity}] ${d.code} @${d.path ?? "-"}: ${d.message}`), + `${token} requirement gate`, + ).toEqual([]); + + const summary = summariseRequirements(root, scan); + expect(summary, `${token} ships requirements`).toBeDefined(); + // §5.4 — "A library must not ship unruled gaps; the standalone gate asserts it." + // A `partial` an adopter cannot act on and nobody has ruled on is worse than no + // entry: it reads as an open question the adopter is expected to answer. + expect(summary!.undecided, `${token} unruled gaps`).toBe(0); + // Every entity the library ships is claimed by the library's own ledger. + expect( + `${summary!.entitiesClaimed}/${summary!.entitiesTotal}`, + `${token} entities claimed`, + ).toBe(`${summary!.entitiesTotal}/${summary!.entitiesTotal}`); + }); + + test(`\`${token}\` is clean under the authoring lint`, async () => { + const result = await new MetaDataLoader({ strict: true }).load(librarySources([token])); + // Advisory for an adopter, mandatory here: the lint's findings are all about + // prose an adopter cannot edit. + expect( + lintRequirements(result.root).map((d) => `${d.code} @${d.path ?? "-"}: ${d.message}`), + `${token} authoring lint`, + ).toEqual([]); + }); + } +}); diff --git a/server/typescript/packages/cli/test/skill-catalog-grounding.test.ts b/server/typescript/packages/cli/test/skill-catalog-grounding.test.ts new file mode 100644 index 000000000..5c88b4bd2 --- /dev/null +++ b/server/typescript/packages/cli/test/skill-catalog-grounding.test.ts @@ -0,0 +1,213 @@ +// The codegen skill's prose is grounded in the LIVE catalog. +// +// The design's rule is that the skill teaches a PROCEDURE, not a list: recipes name +// layers, members come from `meta gen --list`. A list in prose goes stale the day a +// generator is added or renamed, and the reader has no way to tell — which is the same +// failure the `--probe` design exists to remove one level down. +// +// So two things are checked, and only two, because a broader "every backticked token +// must be real" sweep over prose collides with ordinary English: +// +// 1. Every generator name the skill hands to a COMMAND — `meta eject a b c`, +// `--generators a,b,c` — is a real stable name. These are the tokens a reader will +// literally type, so a renamed generator breaks them concretely. +// 2. Every layer the skill names is one of the six. +// +// The authority for (1) is the cross-port MANIFEST, not the composed TypeScript catalog: +// this skill carries a reference fragment per port, and the C# fragment legitimately +// names `db-context`, which no TypeScript slice registers. + +import { describe, test, expect } from "bun:test"; +import { readFileSync, readdirSync, statSync } from "node:fs"; +import { join } from "node:path"; +import { GENERATOR_LAYERS } from "@metaobjectsdev/codegen-ts"; +import { knownLibraryTokens } from "@metaobjectsdev/metadata/library"; + +const REPO_ROOT = join(import.meta.dir, "../../../../.."); +const SKILL_ROOT = join(REPO_ROOT, "agent-context/skills/metaobjects-codegen"); +/** FR-043 §4 puts the mirror of the library step here, so its tokens are gated too. */ +const AUTHORING_ROOT = join(REPO_ROOT, "agent-context/skills/metaobjects-authoring"); + +/** Every stable name, in every port — the manifest, not one port's slice. */ +const MANIFEST_NAMES: ReadonlySet = new Set( + Object.keys( + ( + JSON.parse( + readFileSync( + join(REPO_ROOT, "fixtures/generator-registry-conformance/registry.json"), + "utf8", + ), + ) as { generators: Record } + ).generators, + ), +); + +function markdownFiles(dir: string): string[] { + const out: string[] = []; + for (const entry of readdirSync(dir)) { + const abs = join(dir, entry); + if (statSync(abs).isDirectory()) out.push(...markdownFiles(abs)); + else if (entry.endsWith(".md")) out.push(abs); + } + return out; +} + +/** Placeholders a command example legitimately uses in place of a real name. */ +const PLACEHOLDERS = new Set([ + "", "", "...", "", "", "", "", + "a", "b", "c", "...", +]); + +interface Named { + file: string; + token: string; + line: string; +} + +/** Generator names handed to `meta eject` / `--generators` anywhere in the skill. */ +function namesInCommands(files: readonly string[]): Named[] { + const found: Named[] = []; + for (const file of files) { + for (const line of readFileSync(file, "utf8").split("\n")) { + // `meta eject entity queries routes` — bare tokens up to a trailing shell + // comment, a backtick, or a table cell boundary. + for (const m of line.matchAll(/\b(?:meta|metaobjects|dotnet meta)\s+eject\s+([^`\n|#]*)/g)) { + const toks = (m[1] ?? "").trim().split(/\s+/); + for (let i = 0; i < toks.length; i++) { + const tok = toks[i]!; + if (tok.startsWith("--")) { + // Skip the flag AND its value: `--format json` must not read `json` as a + // generator name. A `--flag=value` form carries its own value already. + if (!tok.includes("=")) i++; + continue; + } + if (tok === "" || PLACEHOLDERS.has(tok)) continue; + found.push({ file, token: tok, line: line.trim() }); + } + } + // `--generators entity,routes` / `--generators ` + for (const m of line.matchAll(/--generators[= ]+([A-Za-z0-9,._<>-]+)/g)) { + const arg = (m[1] ?? "").trim(); + if (PLACEHOLDERS.has(arg)) continue; + for (const tok of arg.split(",")) { + if (tok === "" || PLACEHOLDERS.has(tok)) continue; + found.push({ file, token: tok, line: line.trim() }); + } + } + } + } + return found; +} + +describe("the codegen skill is grounded in the live catalog", () => { + const files = markdownFiles(SKILL_ROOT); + + test("the skill was actually found and read", () => { + // Guards every assertion below from passing over an empty file list — the way a + // grounding gate silently stops gating. + expect(files.length).toBeGreaterThan(3); + }); + + test("every generator name the skill puts in a command is a real stable name", () => { + const unknown = namesInCommands(files) + .filter((n) => !MANIFEST_NAMES.has(n.token)) + .map((n) => `${n.file.slice(REPO_ROOT.length + 1)}: "${n.token}" — ${n.line}`); + + expect( + unknown, + "The skill tells a reader to type a generator name the manifest does not have.\n" + + "Either the generator was renamed or removed, or the example is a typo:\n " + + unknown.join("\n "), + ).toEqual([]); + }); + + test("the skill's command examples name at least a few real generators", () => { + // Otherwise the test above is vacuous: a skill that names none passes it trivially. + const named = new Set(namesInCommands(files).map((n) => n.token)); + expect([...named].length).toBeGreaterThan(2); + }); + + test("every layer the skill names is one of the six", () => { + // Precise about WHERE a layer is named: the first cell of a row in a table whose + // header names `layer`. A looser "any backticked word near the word layer" sweep + // was tried and flagged `framework`, `requires` and the English word `interface` — + // a gate that cries wolf gets ignored, and then it gates nothing. + const layers = new Set(GENERATOR_LAYERS); + const offenders: string[] = []; + + for (const file of files) { + const lines = readFileSync(file, "utf8").split("\n"); + let inLayerTable = false; + for (const line of lines) { + if (!line.trimStart().startsWith("|")) { + inLayerTable = false; + continue; + } + const cells = line.split("|").map((c) => c.trim()); + // A header row naming `layer` in its first cell opens the table. + if (/^`?layers?`?$/i.test(cells[1] ?? "")) { + inLayerTable = true; + continue; + } + if (!inLayerTable) continue; + const first = cells[1] ?? ""; + if (/^-+$/.test(first) || first === "") continue; // the separator row + const m = /^`([a-z][a-z-]*)`$/.exec(first); + if (m === null) continue; + if (layers.has(m[1]!)) continue; + offenders.push(`${file.slice(REPO_ROOT.length + 1)}: \`${m[1]}\` — ${line.trim()}`); + } + } + + expect( + offenders, + `A layer token that is not one of the six (${[...layers].join(", ")}):\n ` + + offenders.join("\n "), + ).toEqual([]); + }); + + test("every library token the skill tells a reader to configure is real", () => { + // STRUCTURAL, like the two checks above: a token inside a `libraries` ARRAY, which + // is a config value a reader copies, not prose. The same rule the layer check + // follows — a sweep over backticked words near the word "library" would flag + // ordinary English and then get ignored. + const tokens = new Set(knownLibraryTokens()); + const offenders: string[] = []; + for (const file of [...files, ...markdownFiles(AUTHORING_ROOT)]) { + for (const line of readFileSync(file, "utf8").split("\n")) { + const m = /"?libraries"?\s*:\s*\[([^\]]*)\]/.exec(line); + if (m === null) continue; + for (const raw of m[1]!.split(",")) { + const token = raw.trim().replace(/^["'`]|["'`]$/g, ""); + if (token === "" || tokens.has(token)) continue; + offenders.push(`${file.slice(REPO_ROOT.length + 1)}: "${token}" — ${line.trim()}`); + } + } + } + expect( + offenders, + `A \`libraries\` example naming a token this build does not ship ` + + `(${[...tokens].join(", ")}):\n ` + offenders.join("\n "), + ).toEqual([]); + }); + + test("the library step is actually in both skills", () => { + // The inverse: the check above passes trivially on prose that mentions no library. + // FR-043 §4 puts the step in the codegen procedure AND its mirror where authoring + // teaches declaring an entity, so both are asserted. + for (const root of [SKILL_ROOT, AUTHORING_ROOT]) { + const prose = markdownFiles(root).map((f) => readFileSync(f, "utf8")).join("\n"); + expect(prose, `${root} teaches the library rows`).toContain('kind: "library"'); + } + }); + + test("the skill actually documents the six layers", () => { + // The inverse of the test above, which passes trivially on a skill that mentions no + // layer at all. The selection procedure rests on grouping BY layer, so the words + // have to be there. + const prose = files.map((f) => readFileSync(f, "utf8")).join("\n"); + for (const layer of GENERATOR_LAYERS) { + expect(prose, `the skill names the "${layer}" layer`).toContain(`\`${layer}\``); + } + }); +}); diff --git a/server/typescript/packages/cli/test/unit/args-gen.test.ts b/server/typescript/packages/cli/test/unit/args-gen.test.ts index 348ffd3c2..6128d5dbe 100644 --- a/server/typescript/packages/cli/test/unit/args-gen.test.ts +++ b/server/typescript/packages/cli/test/unit/args-gen.test.ts @@ -8,7 +8,7 @@ describe("parseGenArgs", () => { dryRun: false, entities: [], baseline: "default", - list: false, + list: false, probe: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT, }); diff --git a/server/typescript/packages/cli/test/unit/config-resolver.test.ts b/server/typescript/packages/cli/test/unit/config-resolver.test.ts index ad930d3f4..ab6751cfe 100644 --- a/server/typescript/packages/cli/test/unit/config-resolver.test.ts +++ b/server/typescript/packages/cli/test/unit/config-resolver.test.ts @@ -22,13 +22,13 @@ describe("resolveGenConfig", () => { // outDir/dialect/dbImport/extStyle. Only dryRun + entities come from flags. test("passes dryRun and entities through", () => { - const resolved = resolveGenConfig({ dryRun: true, entities: ["User", "Post"], baseline: "default", list: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT }); + const resolved = resolveGenConfig({ dryRun: true, entities: ["User", "Post"], baseline: "default", list: false, probe: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT }); expect(resolved.dryRun).toBe(true); expect(resolved.entities).toEqual(["User", "Post"]); }); test("defaults: dryRun false, entities empty", () => { - const resolved = resolveGenConfig({ dryRun: false, entities: [], baseline: "default", list: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT }); + const resolved = resolveGenConfig({ dryRun: false, entities: [], baseline: "default", list: false, probe: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT }); expect(resolved.dryRun).toBe(false); expect(resolved.entities).toEqual([]); }); diff --git a/server/typescript/packages/cli/test/unit/init-refresh-docs.test.ts b/server/typescript/packages/cli/test/unit/init-refresh-docs.test.ts index 0fec41db5..4f7824f89 100644 --- a/server/typescript/packages/cli/test/unit/init-refresh-docs.test.ts +++ b/server/typescript/packages/cli/test/unit/init-refresh-docs.test.ts @@ -123,9 +123,11 @@ describe("metaobjects.config.ts wiring still scaffolded", () => { await init({ cwd }); const configTs = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); expect(configTs).toContain("defineConfig"); - // ADR-0034 — the scaffolded config imports the OWNED local generators, never the - // deprecated package `/generators` export. - expect(configTs).toContain('from "./codegen/generators/entity.js"'); + // ADR-0034 Amendment 2 — the scaffolded config wires nothing and imports nothing. + // What it must NOT do is reach for the deprecated package `/generators` export, + // which is the one import path 1.0 removed. + expect(configTs).toContain("generators: []"); + expect(configTs).not.toMatch(/^import .* from "\.\/codegen\/generators\//m); expect(configTs).not.toContain("@metaobjectsdev/codegen-ts/generators"); }); }); diff --git a/server/typescript/packages/cli/test/unit/init-scaffold-config.test.ts b/server/typescript/packages/cli/test/unit/init-scaffold-config.test.ts index d878ea477..6665bd08b 100644 --- a/server/typescript/packages/cli/test/unit/init-scaffold-config.test.ts +++ b/server/typescript/packages/cli/test/unit/init-scaffold-config.test.ts @@ -14,10 +14,11 @@ describe("meta init scaffolds metaobjects.config.ts", () => { expect(existsSync(join(tmp, "metaobjects.config.ts"))).toBe(true); const body = readFileSync(join(tmp, "metaobjects.config.ts"), "utf-8"); expect(body).toContain(`import { defineConfig } from "@metaobjectsdev/cli"`); - expect(body).toContain(`entityFile()`); - expect(body).toContain(`queriesFile()`); - expect(body).toContain(`routesFile()`); - expect(body).toContain(`barrel()`); + // Codegen is opt-in: the scaffolded selection is EMPTY, and the config's job is to + // say so and point at the catalog rather than to wire a shape nobody chose. + expect(body).toContain("generators: []"); + expect(body).toContain("meta gen --list"); + expect(body).toContain("meta eject"); expect(result.created).toContain("metaobjects.config.ts"); }); @@ -42,7 +43,7 @@ describe("meta init scaffolds metaobjects.config.ts", () => { // `requirements` and `agent` for every scaffolded project and defeating exactly // that. Asserted as an ABSENCE because the correct scaffold says nothing here. expect(body).not.toMatch(/^\s*surfaces:\s*\[/m); - expect(nextStepsBlock(true)).toContain("meta docs"); + expect(nextStepsBlock()).toContain("meta docs"); }); test("does not overwrite an existing metaobjects.config.ts on subsequent runs", async () => { @@ -69,47 +70,48 @@ describe("meta init scaffolds metaobjects.config.ts", () => { }); }); -// ADR-0034 scaffold-and-own — `meta init` copies the codegen reference templates into -// the consumer repo (codegen/generators/*.ts), and the scaffolded config imports those -// OWNED local copies instead of the deprecated package `/generators` export. -describe("meta init scaffolds OWNED codegen generators (ADR-0034)", () => { - const GENERATORS = ["entity", "queries", "routes", "barrel"] as const; - - test("writes codegen/generators/{entity,queries,routes,barrel}.ts", async () => { +// ADR-0034 Amendment 2 — `meta init` copies NOTHING into codegen/generators/. +// +// The directory and its tsconfig are scaffolded because they are the LAYOUT the +// scaffold-and-own contract promises; what lands in them is the adopter's choice, made +// through `meta eject`. Ownership itself is unchanged: an ejected file is theirs, and +// `meta gen` runs their copy. +describe("meta init scaffolds the owned-codegen tier, empty (ADR-0034 Amendment 2)", () => { + test("creates the directory and its tsconfig, and copies no generator into it", async () => { const result = await init({ cwd: tmp, quiet: true }); - for (const name of GENERATORS) { - const rel = `codegen/generators/${name}.ts`; - expect(existsSync(join(tmp, rel))).toBe(true); - expect(result.created).toContain(rel); + expect(existsSync(join(tmp, "codegen/generators"))).toBe(true); + expect(existsSync(join(tmp, "tsconfig.codegen.json"))).toBe(true); + expect(result.created).toContain("tsconfig.codegen.json"); + for (const name of ["entity", "queries", "routes", "barrel", "names"]) { + expect(existsSync(join(tmp, `codegen/generators/${name}.ts`)), name).toBe(false); } + expect(result.created.filter((p) => p.startsWith("codegen/generators/"))).toEqual([]); }); - test("owned generators are the copyable reference templates (REFERENCE TEMPLATE header)", async () => { - await init({ cwd: tmp, quiet: true }); - const entity = readFileSync(join(tmp, "codegen/generators/entity.ts"), "utf-8"); - expect(entity).toContain("REFERENCE TEMPLATE"); - // They import the stable engine, never the deprecated `/generators` export. - expect(entity).toContain('from "@metaobjectsdev/codegen-ts"'); - expect(entity).not.toContain("@metaobjectsdev/codegen-ts/generators"); - }); - - test("the scaffolded config imports each owned generator locally", async () => { + test("the scaffolded config imports no generator at all", async () => { await init({ cwd: tmp, quiet: true }); const body = readFileSync(join(tmp, "metaobjects.config.ts"), "utf-8"); - expect(body).toContain('import { entityFile } from "./codegen/generators/entity.js"'); - expect(body).toContain('import { queriesFile } from "./codegen/generators/queries.js"'); - expect(body).toContain('import { routesFile } from "./codegen/generators/routes.js"'); - expect(body).toContain('import { barrel } from "./codegen/generators/barrel.js"'); + expect(body).not.toMatch(/^import .* from "\.\/codegen\/generators\//m); expect(body).not.toContain("@metaobjectsdev/codegen-ts/generators"); }); - test("re-init with --force preserves a hand-edited owned generator", async () => { + test("an ejected generator is the copyable reference template, and re-init preserves it", async () => { await init({ cwd: tmp, quiet: true }); + + const { ejectGenerator } = await import("../../src/commands/eject.js"); + await ejectGenerator({ cwd: tmp, name: "entity" }); const entityPath = join(tmp, "codegen/generators/entity.ts"); - const edited = readFileSync(entityPath, "utf-8") + "\n// HAND-EDIT-SENTINEL\n"; - writeFileSync(entityPath, edited); - const result = await init({ cwd: tmp, quiet: true, force: true }); + const entity = readFileSync(entityPath, "utf-8"); + expect(entity).toContain("REFERENCE TEMPLATE"); + // It imports the stable engine, never the deprecated `/generators` export. + expect(entity).toContain('from "@metaobjectsdev/codegen-ts"'); + expect(entity).not.toContain("@metaobjectsdev/codegen-ts/generators"); + + // A hand edit survives `meta init --force` — init has no business in this + // directory at all now, which is a stronger guarantee than the copier's old + // preserve-if-present rule. + writeFileSync(entityPath, entity + "\n// HAND-EDIT-SENTINEL\n"); + await init({ cwd: tmp, quiet: true, force: true }); expect(readFileSync(entityPath, "utf-8")).toContain("HAND-EDIT-SENTINEL"); - expect(result.preserved).toContain("codegen/generators/entity.ts"); }); }); diff --git a/server/typescript/packages/cli/test/unit/requirement-coverage-activation.test.ts b/server/typescript/packages/cli/test/unit/requirement-coverage-activation.test.ts new file mode 100644 index 000000000..f40e9444c --- /dev/null +++ b/server/typescript/packages/cli/test/unit/requirement-coverage-activation.test.ts @@ -0,0 +1,180 @@ +// FR-043 §5.4 — object coverage activates on ADOPTER-authored requirements only. +// +// The problem this rule exists for: `checkRequirements` early-returns only when the +// tree has ZERO requirements, so a library shipping its own ledger would switch the +// unclaimed-entity gate ON for every entity in the adopting project. A project that +// has never written a requirement would opt into `iam`, gain eleven entries it did +// not author, and be told its own ninety entities are unclaimed. The gate would be +// measuring the library's decision to ship a ledger, not the adopter's. +// +// So: library requirements are always COUNTED, always gate-checked for their own +// integrity, and always claim what they claim — they simply do not volunteer the +// adopter for coverage. Write one requirement of your own and coverage turns on, +// over the library's entities too, which by then are claimed by the library's own +// ledger and add no warnings. +// +// Provenance is the library's own declared PACKAGE, resolved from the embedded +// manifests rather than from a source-file path: `packages` is a manifest fact the +// standalone gate already resolves against the library loaded alone, whereas a +// source id differs between the on-disk dev layout (an absolute path) and the +// embedded one (`library:.yaml`) — a rule keyed on that would hold in this +// repo and silently stop holding in an installed build. +import { describe, test, expect } from "bun:test"; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadMemory } from "@metaobjectsdev/sdk"; +import { + checkRequirements, summariseRequirements, scanRequirements, + WARN_REQUIREMENT_OBJECT_UNCLAIMED, +} from "../../src/lib/requirement-check.js"; + +/** Two entities, one of which no requirement below ever claims. */ +const MODEL = ` +metadata: + package: acme::shop + children: + - object.entity: + name: Order + children: + - source.rdb: { table: orders } + - field.uuid: { name: id } + - identity.primary: { name: pk, fields: [id] } + - object.entity: + name: Invoice + children: + - source.rdb: { table: invoices } + - field.uuid: { name: id } + - identity.primary: { name: pk, fields: [id] } +`; + +/** One requirement the ADOPTER wrote, claiming one of the two entities. */ +const OWN_CAPS = ` +metadata: + package: acme::caps + children: + - requirement.functional: + name: ordering + level: 4 + status: live + statement: A customer's order is recorded before it is paid for. + counterexample: A payment against an order that was never stored. + implementedBy: [acme::shop::Order] +`; + +/** An adopter DISAGREEING with a library requirement (§5.5) — an overlay of the + * library's own node, in the library's own package. */ +const OVERLAY_CAPS = ` +metadata: + package: metaobjects::iam + children: + - requirement.architectural: + name: noCredentialsOnUser + overlay: true + status: partial + disposition: accepted + notes: We store a password hash on User because our auth stack predates this library. +`; + +interface Run { + unclaimed: string[]; + measured: boolean; + entitiesTotal?: number; + total: number; +} + +async function run(files: Record, libraries?: string[]): Promise { + const dir = mkdtempSync(join(tmpdir(), "req-cover-")); + try { + mkdirSync(join(dir, "metaobjects")); + for (const [name, text] of Object.entries(files)) { + writeFileSync(join(dir, "metaobjects", name), text); + } + const root = await loadMemory(dir, { + strict: true, + ...(libraries === undefined ? {} : { libraries }), + }); + const scan = scanRequirements(root); + const summary = summariseRequirements(root, scan); + return { + unclaimed: checkRequirements(root, scan) + .filter((d) => d.code === WARN_REQUIREMENT_OBJECT_UNCLAIMED) + .map((d) => d.message), + measured: scan.measureCoverage, + ...(summary?.entitiesTotal === undefined ? {} : { entitiesTotal: summary.entitiesTotal }), + total: summary?.total ?? 0, + }; + } finally { + rmSync(dir, { recursive: true, force: true }); + } +} + +describe("object coverage activates on adopter-authored requirements only", () => { + test("a project that opted into a library and wrote none of its own is not measured", async () => { + const r = await run({ "meta.shop.yaml": MODEL }, ["iam"]); + // The library's ledger is COUNTED — it is in the tree and it is checked. + expect(r.total).toBeGreaterThan(0); + // ...but it does not volunteer the adopter's entities for coverage. + expect(r.measured).toBe(false); + expect(r.unclaimed).toEqual([]); + // And the summary says so rather than printing a ratio nobody asked to be held to. + expect(r.entitiesTotal).toBeUndefined(); + }); + + test("one requirement of your own switches it on — over the library's entities too", async () => { + const r = await run({ "meta.shop.yaml": MODEL, "meta.caps.yaml": OWN_CAPS }, ["iam"]); + expect(r.measured).toBe(true); + // `Invoice` is the adopter's own unclaimed entity — the warning they can act on. + expect(r.unclaimed.join("\n")).toContain("acme::shop::Invoice"); + expect(r.unclaimed.length).toBe(1); + // The denominator GREW to include the library's nine, and every one of them is + // claimed by the library's own ledger, so the gate stays quiet about them. That + // is the property that makes activation safe: turning coverage on must not hand + // the adopter a pile of warnings about metadata they did not write. + expect(r.entitiesTotal).toBe(11); + }); + + test("no library, no change: a project's own ledger measures coverage exactly as before", async () => { + const r = await run({ "meta.shop.yaml": MODEL, "meta.caps.yaml": OWN_CAPS }); + expect(r.measured).toBe(true); + expect(r.entitiesTotal).toBe(2); + expect(r.unclaimed.length).toBe(1); + }); + + test("overlaying a library requirement is not authoring one (§5.5)", async () => { + // Disagreeing with a shipped claim is a verdict on the LIBRARY's design, not a + // statement about what the adopter's own model is for — and it merges into the + // library's node, in the library's package. Treating it as activation would mean + // an adopter who corrected one library entry silently acquired a coverage gate + // over their whole estate. + const r = await run( + { "meta.shop.yaml": MODEL, "meta.overlay.yaml": OVERLAY_CAPS }, + ["iam"], + ); + expect(r.measured).toBe(false); + expect(r.unclaimed).toEqual([]); + }); + + test("`measureCoverage` can be forced on, for a caller that IS the library", async () => { + // The standalone library gate (`shipped-library-verify.test.ts`) loads a library + // with no project around it, so the derivation would switch coverage off exactly + // where it is the thing being tested. + const dir = mkdtempSync(join(tmpdir(), "req-cover-lib-")); + try { + mkdirSync(join(dir, "metaobjects")); + writeFileSync(join(dir, "metaobjects", "meta.shop.yaml"), MODEL); + const root = await loadMemory(dir, { strict: true, libraries: ["iam"] }); + const scan = scanRequirements(root, { measureCoverage: true }); + expect(scan.measureCoverage).toBe(true); + const diags = checkRequirements(root, scan).filter( + (d) => d.code === WARN_REQUIREMENT_OBJECT_UNCLAIMED, + ); + // Both of the adopter's entities are now reported — which is the forced + // behaviour, and the reason the override exists rather than being the default. + expect(diags.length).toBe(2); + expect(summariseRequirements(root, scan)!.entitiesTotal).toBe(11); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/codegen-ts-react/src/generator-registry.ts b/server/typescript/packages/codegen-ts-react/src/generator-registry.ts new file mode 100644 index 000000000..7b6e123a0 --- /dev/null +++ b/server/typescript/packages/codegen-ts-react/src/generator-registry.ts @@ -0,0 +1,38 @@ +// ADR-0021 D3 — this package's SLICE of the stable-name generator registry. +// +// `codegen-ts` owns the entry type and the framework-neutral + server-side entries, but +// it cannot import this package (dependency direction), so the React client-tier +// generator registers itself here and the CLI unions the three slices +// (`cli/src/lib/catalog.ts`). Before this existed, `meta gen --list` and +// `meta eject --list` read two different tables and gave an agent two different answers +// to "what can I turn on". + +import type { GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; +import { formFile } from "./form-file.js"; +import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; + +/** True iff this package ships a copyable reference template for `name`. */ +function ejectable(name: string): boolean { + return (REFERENCE_GENERATOR_NAMES as readonly string[]).includes(name); +} + +export const reactGeneratorRegistry: Record = { + form: { + name: "form", + kind: "generator", + layer: "client", + description: "Per-entity React form component over the generated Zod schema.", + tier: "native", + factory: () => formFile(), + options: "filter?, target?", + // `react`, not "the client framework": `@metaobjectsdev/tanstack` declares `react` + // as a peer, so form (react) + hooks/grid (tanstack) is the intended composition, + // NOT a conflict. Framework exclusivity is advisory and only on the `api` layer. + framework: "react", + requires: ["entity"], + runtimePackages: ["@metaobjectsdev/react"], + runtimePeers: ["react", "react-hook-form"], + configKeys: ["extStyle", "clientDirective"], + ejectable: ejectable("form"), + }, +}; diff --git a/server/typescript/packages/codegen-ts-react/src/index.ts b/server/typescript/packages/codegen-ts-react/src/index.ts index a2be7ce43..246234f64 100644 --- a/server/typescript/packages/codegen-ts-react/src/index.ts +++ b/server/typescript/packages/codegen-ts-react/src/index.ts @@ -1,6 +1,10 @@ // Public API surface for @metaobjectsdev/codegen-ts-react. export { formFile, type FormFileOpts } from "./form-file.js"; +// ADR-0021 D3 — this package's slice of the stable-name generator registry. The CLI +// unions the three slices into the catalog behind `meta gen --list`. +export { reactGeneratorRegistry } from "./generator-registry.js"; + // FR-040 §4.2(b) — public so an owned generator composes the engine rather than // forking it. Signature is stable API: (entity, ctx) => string. export { renderFormFile } from "./templates/form-file.js"; diff --git a/server/typescript/packages/codegen-ts-tanstack/src/generator-registry.ts b/server/typescript/packages/codegen-ts-tanstack/src/generator-registry.ts new file mode 100644 index 000000000..a16ea1e1b --- /dev/null +++ b/server/typescript/packages/codegen-ts-tanstack/src/generator-registry.ts @@ -0,0 +1,67 @@ +// ADR-0021 D3 — this package's SLICE of the stable-name generator registry. +// See the react package's copy for why the catalog is composed rather than central. + +import type { GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; +import { tanstackQuery } from "./tanstack-query.js"; +import { tanstackGrid } from "./tanstack-grid.js"; +import { tanstackGridHook } from "./tanstack-grid-hook.js"; +import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; + +/** True iff this package ships a copyable reference template for `name`. */ +function ejectable(name: string): boolean { + return (REFERENCE_GENERATOR_NAMES as readonly string[]).includes(name); +} + +export const tanstackGeneratorRegistry: Record = { + hooks: { + name: "hooks", + kind: "generator", + layer: "client", + description: + "Per-entity TanStack Query hooks (useEntity / useEntities / useCreate / useUpdate / useDelete).", + tier: "native", + factory: () => tanstackQuery(), + options: "filter?, target?", + framework: "tanstack", + requires: ["entity"], + runtimePackages: ["@metaobjectsdev/runtime-web", "@metaobjectsdev/tanstack"], + runtimePeers: ["@tanstack/react-query"], + configKeys: ["extStyle", "clientDirective"], + ejectable: ejectable("hooks"), + }, + grid: { + name: "grid", + kind: "generator", + layer: "client", + description: + "Per-entity TanStack Table column definitions, from a layout.dataGrid declaration.", + tier: "native", + factory: () => tanstackGrid(), + options: "filter?, tphSubtypeGrids?, target?", + framework: "tanstack", + requires: ["entity"], + runtimePeers: ["@tanstack/react-table"], + configKeys: ["extStyle", "clientDirective"], + ejectable: ejectable("grid"), + }, + "grid-hook": { + name: "grid-hook", + kind: "generator", + layer: "client", + description: + "Per-entity server-driven grid state hook (sort/filter/page) over the generated columns.", + tier: "native", + factory: () => tanstackGridHook(), + options: "filter?, tphSubtypeGrids?, target?", + framework: "tanstack", + // `grid` is here because the emitted hook imports the columns module's filter + // PRESET constants — an edge that only exists when a layout.dataGrid declares a + // `filter`. Derived, not assumed: the probe fixture declares one precisely so this + // edge is visible to the gate rather than left to prose. + requires: ["entity", "grid", "hooks"], + runtimePackages: ["@metaobjectsdev/runtime-web", "@metaobjectsdev/tanstack"], + runtimePeers: ["@tanstack/react-query", "@tanstack/react-table", "react"], + configKeys: ["extStyle", "clientDirective"], + ejectable: ejectable("grid-hook"), + }, +}; diff --git a/server/typescript/packages/codegen-ts-tanstack/src/index.ts b/server/typescript/packages/codegen-ts-tanstack/src/index.ts index 22be5a594..a42208bce 100644 --- a/server/typescript/packages/codegen-ts-tanstack/src/index.ts +++ b/server/typescript/packages/codegen-ts-tanstack/src/index.ts @@ -3,6 +3,10 @@ export { tanstackQuery, type TanstackQueryOpts } from "./tanstack-query.js"; export { tanstackGrid, type TanstackGridOpts } from "./tanstack-grid.js"; export { tanstackGridHook, type TanstackGridHookOpts } from "./tanstack-grid-hook.js"; +// ADR-0021 D3 — this package's slice of the stable-name generator registry. The CLI +// unions the three slices into the catalog behind `meta gen --list`. +export { tanstackGeneratorRegistry } from "./generator-registry.js"; + // FR-040 §4.2(b) — public so an owned generator composes the engine rather than // forking it. Signatures are stable API: (entity, ctx) => string. export { renderHooksFile } from "./templates/hooks-file.js"; diff --git a/server/typescript/packages/codegen-ts/src/catalog-gates.ts b/server/typescript/packages/codegen-ts/src/catalog-gates.ts new file mode 100644 index 000000000..1d6b087ab --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/catalog-gates.ts @@ -0,0 +1,222 @@ +// The post-selection audits `meta gen` runs over the wired suite. +// +// Codegen is opt-in, so the selection is the adopter's (increasingly their agent's). +// That makes `meta gen` the place the selection is CHECKED — not by refusing it, but by +// naming the ways a legal selection still surprises you: +// +// 1. a generator whose output imports a module nothing in the run emits; +// 2. two `api`-layer generators bringing two different HTTP frameworks; +// 3. (FR-043) a library opted into whose implied generator is not wired, or a +// generator wired whose library is not opted into — the two halves of one choice. +// +// Both are WARNINGS, self-extinguishing, never a build failure. An adopter may +// legitimately have hand-written the other half, and serving Node and edge from one +// model is a real thing people do. Following the house rule: a gate that cries wolf +// gets deleted, and a gate nobody can satisfy is worse than no gate. + +import type { Generator } from "./generator.js"; +import type { GeneratorRegistryEntry } from "./generator-registry.js"; + +/** + * Map a CONSTRUCTED generator back to its catalog stable name. + * + * These are two different names and conflating them is the trap here: the catalog key + * is `routes` / `grid-hook`, while the object a factory returns calls itself + * `routes-file` / `tanstack-grid-hook`. A gate keyed on `g.name` therefore matches + * nothing at all — silently, since a name the catalog does not know is legitimately + * skipped as somebody's own generator. + * + * DERIVED by constructing every catalog factory and reading the name back, not kept as + * a table: a table would be a third spelling to maintain, and the failure mode of it + * going stale is exactly the silent no-op above. The factories are trivial (that is a + * registry invariant `--list` already depends on), and the result is memoized per + * catalog object. + * + * An implementation name produced by two entries is dropped rather than guessed at. + */ +const STABLE_NAME_CACHE = new WeakMap>(); + +export function stableNameIndex( + catalog: Record, +): ReadonlyMap { + const cached = STABLE_NAME_CACHE.get(catalog); + if (cached !== undefined) return cached; + + const index = new Map(); + const ambiguous = new Set(); + for (const [stable, entry] of Object.entries(catalog)) { + let implName: string; + try { + implName = entry.factory().name; + } catch { + continue; // a factory that cannot construct contributes nothing + } + if (index.has(implName) && index.get(implName) !== stable) ambiguous.add(implName); + index.set(implName, stable); + // The stable name maps to itself too, so a generator that already calls itself by + // its stable name (and an ejected copy that renamed itself to one) still resolves. + if (!index.has(stable)) index.set(stable, stable); + } + for (const name of ambiguous) index.delete(name); + + STABLE_NAME_CACHE.set(catalog, index); + return index; +} + +/** + * Warn for each wired generator whose `requires` are not also wired. + * + * A generator whose name resolves to no catalog entry is an owned or third-party one + * with nothing declared, and is skipped in silence — there is no declaration to check + * it against. + */ +export function warnUnsatisfiedRequires( + generators: readonly Generator[], + catalog: Record, + warn: (message: string) => void, +): void { + const index = stableNameIndex(catalog); + const stableOf = (g: Generator): string | undefined => index.get(g.name); + const wired = new Set(generators.map(stableOf).filter((n): n is string => n !== undefined)); + + for (const g of generators) { + const stable = stableOf(g); + if (stable === undefined) continue; + const entry = catalog[stable]; + if (entry?.requires === undefined) continue; + const missing = entry.requires.filter((dep) => !wired.has(dep)); + if (missing.length === 0) continue; + + const list = missing.map((m) => `"${m}"`).join(", "); + warn( + `"${stable}" is wired but ${list} ${missing.length === 1 ? "is" : "are"} not. ` + + `The code "${stable}" emits imports modules ${list} would have emitted, so ` + + `tsc will report an unresolved import. Wire ${missing.length === 1 ? "it" : "them"} ` + + `(\`meta eject ${missing.join(" ")}\` copies the reference generator${missing.length === 1 ? "" : "s"}), ` + + `or keep your own hand-written module${missing.length === 1 ? "" : "s"} at ${missing.length === 1 ? "that path" : "those paths"}.`, + ); + } +} + +/** + * Warn when two wired `api`-layer generators declare DIFFERENT frameworks. + * + * Verified before it was written: `routes` emits `.routes.ts` and `routes-hono` + * emits `.routes.hono.ts`. Different paths — so wiring both does not trip the + * runner's conflicting-output-path error, does not fail `tsc`, and silently produces two + * complete HTTP surfaces over the same entities. Worth saying; not worth refusing. + * + * **`api` only, and there is deliberately no general per-layer rule.** The `client` + * layer disproves one: `@metaobjectsdev/tanstack` declares `react` as a peer, so + * `form` (react) + `hooks`/`grid` (tanstack) is the documented, normal composition. An + * earlier draft of the design carried "at most one framework per api and per client + * layer"; that rule would have forbidden the single most common client selection. + */ +export function warnMixedApiFrameworks( + generators: readonly Generator[], + catalog: Record, + warn: (message: string) => void, +): void { + const index = stableNameIndex(catalog); + const byFramework = new Map(); + for (const g of generators) { + const stable = index.get(g.name); + if (stable === undefined) continue; + const entry = catalog[stable]; + if (entry?.layer !== "api" || entry.framework === undefined) continue; + const names = byFramework.get(entry.framework) ?? []; + names.push(stable); + byFramework.set(entry.framework, names); + } + if (byFramework.size < 2) return; + + const described = [...byFramework.entries()] + .sort(([a], [b]) => a.localeCompare(b)) + .map(([fw, names]) => `${names.sort().map((n) => `"${n}"`).join(" + ")} (${fw})`) + .join(" and "); + + warn( + `two api-layer frameworks are wired — ${described}. They emit to DIFFERENT paths, ` + + "so nothing conflicts and nothing fails; this run produces two complete HTTP " + + "surfaces over the same entities. That is legitimate when you are migrating " + + "between them or serving Node and edge from one model — otherwise wire one.", + ); +} + +/** + * Warn on the two halves of a library/generator selection that do not agree. + * + * A library ships metadata; a generator that keys on that metadata is a separate + * choice, and FR-043 §6 deliberately wires neither for you. What it does instead is + * refuse to let either half be silently half-done: + * + * - **opted in, not wired** — the design is in your model and the code it implies is + * not being generated. + * - **wired, not opted in** — the generator's anchor is a node only that library + * declares, so it will match nothing and emit zero files. Without this line that is + * indistinguishable from "my model has no trace entities yet". + * + * Both self-extinguish: wire it, or opt in, or remove one. Neither ever fails a build — + * an adopter who ejected the library and owns the metadata is in a legitimate state + * this cannot see, and a gate that cries wolf gets switched off. + * + * `optedIn` is the project's selection TOKENS (`["ai", "ai/db"]`); a layer token names + * the same library as its bare form. + */ +export function warnLibrarySelectionMismatch( + generators: readonly Generator[], + catalog: Record, + manifests: Readonly }>>, + optedIn: readonly string[], + warn: (message: string) => void, +): void { + const index = stableNameIndex(catalog); + const wired = new Set( + generators.map((g) => index.get(g.name)).filter((n): n is string => n !== undefined), + ); + const selected = new Set(optedIn.map((t) => t.split("/")[0]!)); + + for (const [library, manifest] of Object.entries(manifests)) { + const implied = (manifest.generators ?? []).map((g) => g.name); + if (implied.length === 0) continue; + + if (selected.has(library)) { + const notWired = implied.filter((n) => !wired.has(n)); + if (notWired.length > 0) { + warn( + `library "${library}" is opted in and implies ${notWired.map((n) => `"${n}"`).join(", ")}, ` + + `which ${notWired.length === 1 ? "is" : "are"} not wired. The design is in your model; ` + + `the code it implies is not being generated. Wire it, or ignore this if you are ` + + `taking the metadata alone.`, + ); + } + continue; + } + + // Not opted in: name any of its generators that ARE wired. Checked per library + // rather than per generator so a name two libraries imply is not reported twice + // when one of them is selected. + const orphaned = implied.filter( + (n) => wired.has(n) && !impliedByAnySelected(n, manifests, selected), + ); + if (orphaned.length > 0) { + warn( + `${orphaned.map((n) => `"${n}"`).join(", ")} ${orphaned.length === 1 ? "is" : "are"} wired, ` + + `but library "${library}" — which declares the node ${orphaned.length === 1 ? "it keys" : "they key"} ` + + `on — is not opted in, so ${orphaned.length === 1 ? "it will" : "they will"} match nothing and emit ` + + `no files. Add "${library}" to \`libraries\` in .metaobjects/config.json, or unwire ` + + `${orphaned.length === 1 ? "it" : "them"}.`, + ); + } + } +} + +function impliedByAnySelected( + generator: string, + manifests: Readonly }>>, + selected: ReadonlySet, +): boolean { + return [...selected].some((lib) => + (manifests[lib]?.generators ?? []).some((g) => g.name === generator), + ); +} diff --git a/server/typescript/packages/codegen-ts/src/generator-registry.ts b/server/typescript/packages/codegen-ts/src/generator-registry.ts index 0b1959c2e..d5bd28cd9 100644 --- a/server/typescript/packages/codegen-ts/src/generator-registry.ts +++ b/server/typescript/packages/codegen-ts/src/generator-registry.ts @@ -10,6 +10,13 @@ // factory-array config keeps working unchanged — the registry powers `--list` // and a stable identity, it does not replace the config path. // +// **This is a SLICE, not the whole catalog.** `codegen-ts` cannot import +// `codegen-ts-react` / `codegen-ts-tanstack` (dependency direction), so those two +// packages export their own slices and the CLI unions all three +// (`cli/src/lib/catalog.ts`). Set equality against the cross-port manifest is a +// property of the COMPOSED catalog and is asserted there; this package's own +// conformance test can only check one direction — no rogue names. +// // Tiering (ADR-0020 / ADR-0021 D1): // - "native" — the recommended Tier-1 `meta gen` suite (idiomatic emission). // - "neutral" — Tier-2 artifacts owned by the neutral docs engine. `docs` and @@ -18,6 +25,8 @@ // door for documentation is `meta docs` (D1). import type { Generator } from "./generator.js"; +import type { MetaobjectsGenConfig } from "./metaobjects-config.js"; +import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; // The four ADR-0034 ownable generators are no longer exported from ./generators/index.js // (1.0 removed them — see that file's header). They remain the engine's internal composers // and the stable-name registry still constructs them, so import them from their own modules. @@ -40,13 +49,55 @@ import { templateGenerator, traceHelperFile, sharedModelFile, + requirementTests, } from "./generators/index.js"; export type GeneratorTier = "native" | "neutral"; +/** + * The six layers a generator can belong to — the axis an adopter SELECTS BY. + * + * Cross-port: `layer` is gated by every port's registry-conformance test against + * `fixtures/generator-registry-conformance/registry.json`, exactly as `tier` is, so a + * polyglot agent groups the catalog by the same words everywhere. + * + * Six, not ten. The first four are app-shape decisions a builder makes; `docs` is on by + * default (`meta docs`); `capability` holds the ones the MODEL has already made — nobody + * picks `prompt-render` by browsing a taxonomy, they pick it because they declared a + * `template.prompt`, which `meta gen --list --probe` reports with a real file count. A + * layer with one member does no grouping work, so do not split `capability` to tidy it. + */ +export const GENERATOR_LAYERS = [ + "model", + "persistence", + "api", + "client", + "docs", + "capability", +] as const; +export type Layer = (typeof GENERATOR_LAYERS)[number]; + +/** + * The framework a generator's OUTPUT targets. Absent = framework-neutral. + * + * Exclusivity is an ADVISORY and only on the `api` layer (spec §8a): `routes` and + * `routes-hono` emit to different paths, so wiring both is legal and silently produces + * two HTTP surfaces — worth a warning, never an error, because migrating between them is + * legitimate. There is NO general per-layer rule: `@metaobjectsdev/tanstack` peers on + * `react`, so `form` (react) + `hooks`/`grid` (tanstack) is the intended composition. + */ +export type GeneratorFramework = "fastify" | "hono" | "react" | "tanstack"; + export interface GeneratorRegistryEntry { /** Stable, cross-port-consistent id. Equals the registry map key. */ name: string; + /** + * Catalog discriminator. Today every entry is a generator; FR-043 adds + * `kind: "library"` rows to the same table rather than building a parallel one. + */ + kind: "generator"; + /** The selection axis — see {@link GENERATOR_LAYERS}. Gated cross-port. */ + layer: Layer; /** One-line (no newline) human description for `--list`. */ description: string; /** "native" = recommended `meta gen` suite; "neutral" = `meta docs`-owned. */ @@ -55,10 +106,55 @@ export interface GeneratorRegistryEntry { factory: () => Generator; /** Optional one-line options summary for `--list`. */ options?: string; + /** The framework this generator's OUTPUT targets. Absent = framework-neutral. */ + framework?: GeneratorFramework; + /** + * Stable names whose emitted output THIS generator's output imports. + * + * Resolved, not trusted: a gate dry-runs every generator, resolves each emitted + * RELATIVE import back to whichever generator emits that path, and asserts the result + * is a subset of this list — so a generator that quietly starts depending on `entity` + * cannot ship claiming it depends on nothing. + */ + requires?: readonly string[]; + /** + * The `@metaobjectsdev` runtime packages the EMITTED code imports. + * + * PLURAL, against the design's first sketch, because the emitted code is: an + * `output-parser` module imports `@metaobjectsdev/metadata`, `/render` AND + * `/runtime-ts`, and a `hooks` module imports `/runtime-web` and `/tanstack`. A + * singular field would have forced two of the three to go undeclared, which is the + * exact under-declaration the gate exists to catch. Resolved, not trusted — the + * emitted files are read back and every `@metaobjectsdev` import must appear here. + */ + runtimePackages?: readonly string[]; + /** + * Third-party packages the EMITTED code imports. + * + * Per generator rather than derived from {@link runtimePackages}, because + * `@metaobjectsdev/runtime-ts`'s peers are a union (drizzle-orm, fastify, hono, kysely, + * zod) — deriving per package would tell someone ejecting `entity` to install both + * Fastify and Hono. Version RANGES are read from the runtime package's own + * `peerDependencies`, never written here. Gated the same way `requires` is. + */ + runtimePeers?: readonly string[]; + /** + * Config keys this generator reads. Typed against the real config so a renamed key + * breaks the build instead of going quietly stale — the same "resolved, not trusted" + * doctrine as `requires`, obtained here for free from the type system. + */ + configKeys?: readonly (keyof MetaobjectsGenConfig)[]; + /** True iff this package ships a `src/reference/.ts` for `meta eject`. */ + ejectable: boolean; /** Optional note — used to point neutral entries at their canonical door. */ note?: string; } +/** True iff this package ships a copyable reference template for `name`. */ +function ejectable(name: string): boolean { + return (REFERENCE_GENERATOR_NAMES as readonly string[]).includes(name); +} + // The `template` generator is a PRIMITIVE: callers supply { name, walk, // template }. For registry identity + `--list` we expose a no-op default so the // factory constructs a valid Generator without throwing; real use passes opts @@ -73,139 +169,244 @@ function templatePrimitive(): Generator { } export const generatorRegistry: Record = { - // ----- Tier-1 native suite (idiomatic per-port emission) ----------------- + // ----- model ------------------------------------------------------------ entity: { name: "entity", + kind: "generator", + layer: "model", description: "Per-entity Drizzle table + typed model module (the entity module).", tier: "native", factory: () => entityFile(), options: "filter?, target?", + requires: [], + runtimePackages: ["@metaobjectsdev/runtime-ts"], + runtimePeers: ["drizzle-orm", "zod"], + configKeys: ["dialect", "extStyle", "outputLayout", "columnNamingStrategy", + "pluralizeCollections", "collectionNameOverrides", "providedEnumModule", "apiPrefix"], + ejectable: ejectable("entity"), + }, + names: { + name: "names", + kind: "generator", + layer: "model", + description: "Per-entity physical database name constants (table/view, schema, columns).", + tier: "native", + factory: () => namesFile(), + options: "filter?, target?", + configKeys: ["columnNamingStrategy"], + ejectable: ejectable("names"), }, + barrel: { + name: "barrel", + kind: "generator", + layer: "model", + description: "Single index.ts re-exporting every generated entity module.", + tier: "native", + factory: () => barrel(), + options: "target?", + requires: ["entity"], + configKeys: ["extStyle"], + ejectable: ejectable("barrel"), + }, + + // ----- persistence ------------------------------------------------------ queries: { name: "queries", + kind: "generator", + layer: "persistence", description: "Per-entity typed query helpers (findById/create/...).", tier: "native", factory: () => queriesFile(), options: "filter?, target?", + requires: ["entity"], + runtimePeers: ["drizzle-orm"], + configKeys: ["dialect", "extStyle", "pluralizeCollections", "collectionNameOverrides"], + ejectable: ejectable("queries"), }, - callable: { - name: "callable", - description: "Per-entity callable/service surface wrapping the query helpers.", - tier: "native", - factory: () => callableFile(), - options: "filter?, target?", - }, + + // ----- api -------------------------------------------------------------- routes: { name: "routes", + kind: "generator", + layer: "api", description: "Per-entity Fastify CRUD routes (drizzle-fastify mountCrudRoutes).", tier: "native", factory: () => routesFile(), options: "filter?, target?", + framework: "fastify", + requires: ["entity"], + runtimePackages: ["@metaobjectsdev/runtime-ts"], + runtimePeers: ["fastify"], + configKeys: ["dbImport", "apiPrefix", "dialect", "extStyle", "outputLayout", "columnNamingStrategy"], + ejectable: ejectable("routes"), }, "routes-hono": { name: "routes-hono", + kind: "generator", + layer: "api", description: "Per-entity Hono CRUD routes (runtime-ts/hono mountCrudRoutes).", tier: "native", factory: () => routesFileHono(), options: "filter?, target?", + framework: "hono", + requires: ["entity"], + runtimePackages: ["@metaobjectsdev/runtime-ts"], + runtimePeers: ["hono"], + configKeys: ["dbImport", "apiPrefix", "dialect", "extStyle"], + ejectable: ejectable("routes-hono"), }, - barrel: { - name: "barrel", - description: "Single index.ts re-exporting every generated entity module.", + + // ----- docs ------------------------------------------------------------- + "api-docs": { + name: "api-docs", + kind: "generator", + layer: "docs", + description: + "Per-entity/template SDK API reference (the generated code's API, human + agent forms).", tier: "native", - factory: () => barrel(), - options: "target?", + factory: () => apiDocsFile(), + options: "filter?, target?", + ejectable: ejectable("api-docs"), }, - names: { - name: "names", - description: "Per-entity physical database name constants (table/view, schema, columns).", + docs: { + name: "docs", + kind: "generator", + layer: "docs", + description: "Neutral per-entity / per-template Markdown documentation pages.", + tier: "neutral", + factory: () => docsFile(), + ejectable: ejectable("docs"), + note: "neutral artifact — use `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended `meta gen` native suite.", + }, + "mermaid-er": { + name: "mermaid-er", + kind: "generator", + layer: "docs", + description: "Mermaid ER diagram of the entity/relationship model.", + tier: "neutral", + factory: () => mermaidErDiagram(), + ejectable: ejectable("mermaid-er"), + note: "neutral artifact owned by the docs engine (ADR-0020); surfaced via `meta docs`, not the recommended `meta gen` native suite.", + }, + + // ----- capability ------------------------------------------------------- + // Not chosen by browsing this list — chosen because the MODEL already asks for + // them. `meta gen --list --probe` reports a real file count per entry. + callable: { + name: "callable", + kind: "generator", + layer: "capability", + description: "Per-entity callable/service surface wrapping the query helpers.", tier: "native", - factory: () => namesFile(), + factory: () => callableFile(), options: "filter?, target?", + requires: ["entity"], + runtimePeers: ["drizzle-orm"], + configKeys: ["columnNamingStrategy"], + ejectable: ejectable("callable"), }, "prompt-render": { name: "prompt-render", + kind: "generator", + layer: "capability", description: "Per-template prompt-render helper over the render engine.", tier: "native", factory: () => promptRender(), options: "filter?, target?", + runtimePackages: ["@metaobjectsdev/render"], + ejectable: ejectable("prompt-render"), }, "output-parser": { name: "output-parser", + kind: "generator", + layer: "capability", description: "Per-template tolerant output parser (recover-on-receipt).", tier: "native", factory: () => outputParser(), options: "filter?, target?", + runtimePackages: ["@metaobjectsdev/metadata", "@metaobjectsdev/render", "@metaobjectsdev/runtime-ts"], + runtimePeers: ["zod"], + ejectable: ejectable("output-parser"), }, extractor: { name: "extractor", + kind: "generator", + layer: "capability", description: "Per-template typed extract helper (strict payload extraction).", tier: "native", factory: () => extractor(), options: "filter?, target?", + requires: ["entity", "output-parser"], + runtimePackages: ["@metaobjectsdev/metadata", "@metaobjectsdev/render"], + ejectable: ejectable("extractor"), }, "output-prompt": { name: "output-prompt", + kind: "generator", + layer: "capability", description: "Per-template output-format prompt fragment generator.", tier: "native", factory: () => outputPrompt(), options: "filter?, target?", + runtimePackages: ["@metaobjectsdev/render"], + ejectable: ejectable("output-prompt"), }, "render-helper": { name: "render-helper", + kind: "generator", + layer: "capability", description: "Per-template.output render helper (document/email typed wrappers).", tier: "native", factory: () => renderHelper(), options: "filter?, target?", + runtimePackages: ["@metaobjectsdev/render"], + configKeys: ["extStyle"], + ejectable: ejectable("render-helper"), }, template: { name: "template", + kind: "generator", + layer: "capability", description: "Generic Mustache template primitive (walk + template → files).", tier: "native", factory: () => templatePrimitive(), options: "name, walk, template, format?, filter?, provider?, target?", + ejectable: ejectable("template"), }, - "api-docs": { - name: "api-docs", - description: - "Per-entity/template SDK API reference (the generated code's API, human + agent forms).", - tier: "native", - factory: () => apiDocsFile(), - options: "filter?, target?", - }, - "trace-helper": { name: "trace-helper", - description: "Per-entity typed record/call trace helpers (extract + buildLlmCallRow + persist; LlmCallBase-derived entities only).", + kind: "generator", + layer: "capability", + description: + "Per-entity typed record/call trace helpers (extract + buildLlmCallRow + persist; LlmCallBase-derived entities only).", tier: "native", factory: () => traceHelperFile(), options: "outDir?, target?", + ejectable: ejectable("trace-helper"), + }, + "requirement-tests": { + name: "requirement-tests", + kind: "generator", + layer: "capability", + description: "Per-requirement test stub, one per requirement.functional claim in the ledger.", + tier: "native", + factory: () => requirementTests(), + options: "filter?, target?", + ejectable: ejectable("requirement-tests"), }, "shared-model": { name: "shared-model", - description: "FR-023: a publisher's flattened shared-model artifact + manifest for a consumer's `meta deps sync`.", + kind: "generator", + layer: "capability", + description: + "FR-023: a publisher's flattened shared-model artifact + manifest for a consumer's `meta deps sync`.", tier: "native", // `name`/`include` are required at run time (an empty include matches // everything, so a placeholder here constructs without throwing — real use // always supplies both, same as `template`'s templatePrimitive() above). factory: () => sharedModelFile({ name: "shared-model", include: [] }), options: "name, include, exclude?, files?, version?, target?", - }, - - // ----- Tier-2 neutral (owned by the `meta docs` engine — D1 / ADR-0020) --- - docs: { - name: "docs", - description: "Neutral per-entity / per-template Markdown documentation pages.", - tier: "neutral", - factory: () => docsFile(), - note: "neutral artifact — use `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended `meta gen` native suite.", - }, - "mermaid-er": { - name: "mermaid-er", - description: "Mermaid ER diagram of the entity/relationship model.", - tier: "neutral", - factory: () => mermaidErDiagram(), - note: "neutral artifact owned by the docs engine (ADR-0020); surfaced via `meta docs`, not the recommended `meta gen` native suite.", + ejectable: ejectable("shared-model"), }, }; diff --git a/server/typescript/packages/codegen-ts/src/generator.ts b/server/typescript/packages/codegen-ts/src/generator.ts index 954865736..738cd0042 100644 --- a/server/typescript/packages/codegen-ts/src/generator.ts +++ b/server/typescript/packages/codegen-ts/src/generator.ts @@ -1,4 +1,5 @@ import type { MetaObject, MetaRoot, TypeRegistry } from "@metaobjectsdev/metadata"; +import type { LibraryManifest } from "@metaobjectsdev/metadata/library"; import type { RenderContext } from "./render-context.js"; import type { ResolvedGenConfig } from "./metaobjects-config.js"; import type { OrphanPolicy } from "./reconcile-orphans.js"; @@ -65,6 +66,22 @@ export interface GenContext { * own `files` option does). */ sourceFiles?: readonly string[]; + /** + * FR-043 §6 — the manifests of the libraries this project opted into, filled by the + * runner from `RunGenOpts.libraries`. + * + * What a generator reads it FOR is the `anchor`: the library node it keys on. That + * retires the hard-coded entity name a library-aware generator would otherwise carry + * (`trace-helper` compared `.name === "LlmCallBase"`, so ANY adopter entity of that + * name in ANY package triggered it) in favour of resolving the manifest's anchor to a + * node and comparing by node identity. + * + * Undefined means the caller never said — a programmatic `runGen()` rather than + * `meta gen`. A generator reading it should fall back to every SHIPPED manifest's + * anchors, which is still FQN-anchored and still not a bare name. An empty array is + * the opposite and is meaningful: the caller looked, and this project opted into none. + */ + libraries?: readonly LibraryManifest[]; warn: (msg: string) => void; } diff --git a/server/typescript/packages/codegen-ts/src/generators/trace-helper-file.ts b/server/typescript/packages/codegen-ts/src/generators/trace-helper-file.ts index d8f0727bf..c6ad9f880 100644 --- a/server/typescript/packages/codegen-ts/src/generators/trace-helper-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/trace-helper-file.ts @@ -28,9 +28,11 @@ import { TEMPLATE_ATTR_TEXT_REF, } from "@metaobjectsdev/metadata"; import { responseFormatOf } from "../templates/find-inbound.js"; -import type { MetaObject } from "@metaobjectsdev/metadata"; +import type { MetaData, MetaObject } from "@metaobjectsdev/metadata"; +import { libraryManifests } from "@metaobjectsdev/metadata/library"; import { type EmittedFile, + type GenContext, type Generator, type GeneratorFactory, perEntity, @@ -39,8 +41,40 @@ import { generatePayloadInterfacesBatch } from "../payload-codegen.js"; import { GENERATED_HEADER } from "../constants.js"; import { tphDiscriminatorPin } from "../templates/zod-validators.js"; -/** Short name of the shipped abstract base every trace entity extends. */ -const LLM_CALL_BASE = "LlmCallBase"; +/** This generator's stable name — the key a library manifest declares its anchor under. */ +const STABLE_NAME = "trace-helper"; + +/** + * The FQNs this generator keys on, from the LIBRARY MANIFESTS rather than a constant + * here (FR-043 §6). + * + * What it replaces: `const LLM_CALL_BASE = "LlmCallBase"`, compared against `.name` + * anywhere in the super chain — so any adopter entity called `LlmCallBase`, in any + * package, triggered the generator. An anchor is a fully-qualified node the library + * declares, and it is resolved to a node and compared by identity below. + * + * `ctx.libraries` undefined means the caller never said which libraries are selected + * (a programmatic `runGen()`); every shipped manifest's anchor is then a candidate, + * which is still FQN-anchored. An EMPTY array is the opposite and is honoured: the + * caller looked, this project opted into none, and the generator matches nothing. + */ +function anchorFqns(ctx: GenContext): string[] { + const manifests = ctx.libraries ?? Object.values(libraryManifests()); + const out: string[] = []; + for (const manifest of manifests) { + for (const g of manifest.generators ?? []) { + if (g.name === STABLE_NAME && g.anchor !== undefined) out.push(g.anchor); + } + } + return out; +} + +/** Resolve each anchor FQN to the node it names, skipping any the model does not hold. */ +function anchorNodes(ctx: GenContext): MetaData[] { + const wanted = new Set(anchorFqns(ctx)); + if (wanted.size === 0) return []; + return ctx.loadedRoot.children().filter((n) => wanted.has(n.resolutionKey())); +} export interface TraceHelperOpts { /** Output directory prefix relative to the target's outDir. Default: "" (root). */ @@ -49,12 +83,17 @@ export interface TraceHelperOpts { target?: string; } -/** Walk the super chain looking for a node named LLM_CALL_BASE. */ -function extendsBase(obj: MetaObject): boolean { - let cur = obj.superResolved; - while (cur !== undefined) { - if (cur.name === LLM_CALL_BASE) return true; - cur = cur.superResolved; +/** + * Walk the super chain looking for one of the anchor NODES. + * + * Node identity, not `.name` and not even the FQN string: the anchors were resolved + * against this run's own loaded root, so an entity whose chain reaches one reaches + * exactly the node the library declared. + */ +function extendsAnchor(obj: MetaObject, anchors: readonly MetaData[]): boolean { + if (anchors.length === 0) return false; + for (let cur = obj.superResolved; cur !== undefined; cur = cur.superResolved) { + if (anchors.includes(cur)) return true; } return false; } @@ -66,12 +105,16 @@ function pascal(s: string): string { export const traceHelperFile = function traceHelperFile(opts?: TraceHelperOpts): Generator { const dirPrefix = opts?.outDir ? `${opts.outDir.replace(/\/$/, "")}/` : ""; + // Resolved once per run, on first use: `perEntity` calls back per entity and the + // anchor set is a property of the run, not of the entity. + let anchors: MetaData[] | undefined; const generator: Generator = { - name: "trace-helper", + name: STABLE_NAME, generate: perEntity((entity, ctx) => { - // Only concrete entities derived from LlmCallBase. + anchors ??= anchorNodes(ctx); + // Only concrete entities derived from a library's declared anchor. if (entity.isAbstract) return []; - if (!extendsBase(entity)) return []; + if (!extendsAnchor(entity, anchors)) return []; // Find the nested template.prompt. // ADR-0039: resolving — a concrete trace entity may inherit its diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index ec0d398d5..7545a6d82 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -38,8 +38,22 @@ export { generatorRegistry, listGenerators, getGenerator, + GENERATOR_LAYERS, } from "./generator-registry.js"; -export type { GeneratorRegistryEntry, GeneratorTier } from "./generator-registry.js"; +export type { + GeneratorRegistryEntry, + GeneratorTier, + GeneratorFramework, + Layer, +} from "./generator-registry.js"; + +// The post-selection audits `meta gen` runs over a wired suite, and the impl-name → +// stable-name resolver both of them go through. Public so the CLI can gate it. +export { + warnUnsatisfiedRequires, + warnMixedApiFrameworks, + stableNameIndex, +} from "./catalog-gates.js"; export type { MetaobjectsGenConfig, NormalizedMetaobjectsGenConfig, ResolvedGenConfig, Dialect, ExtStyle, ColumnNamingStrategy, MetaDataTypeProvider, GeneratorSpec, DocsConfig, ResolvedDocsConfig, DocsSurface, ApiSurface, VerifyConfig } from "./metaobjects-config.js"; export { defineConfig, normalizeConfig, resolveGenerators, resolveDocsConfig } from "./metaobjects-config.js"; diff --git a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts index 7b3c65594..452c4359c 100644 --- a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts +++ b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts @@ -198,21 +198,6 @@ export interface MetaobjectsGenConfig extends Omit; /** * Output scope — an object is generated only when this predicate returns true * for its fully-qualified name (`obj.resolutionKey()`, `::`). @@ -118,6 +132,16 @@ export interface RunGenOpts { * option explicitly) can omit this with no behavior change elsewhere. */ sourceFiles?: readonly string[]; + /** + * FR-043 §6 — the project's shipped-library selection TOKENS (`Collection.libraries`, + * e.g. `["ai", "ai/db"]`), resolved to manifests and threaded onto + * `GenContext.libraries`, and read by the post-selection audit. + * + * Omitted means the caller never said; `[]` means it looked and this project opted + * into none. The two differ for a generator keying on a library anchor — see + * `GenContext.libraries`. + */ + libraries?: readonly string[]; } export interface RunGenResult { @@ -386,6 +410,16 @@ export async function runGen(opts: RunGenOpts): Promise { // 2. Resolve targets + entity-module target. const config = normalizeConfig(opts.config); + // FR-043 §6 — the opted-in manifests, resolved once per run. A token names its + // library whether or not it carries a layer, and an unknown name contributes nothing + // (the config reader is what refuses a typo, by name — see `knownLibraryTokens`). + const selectedManifests = + opts.libraries === undefined + ? undefined + : [...new Set(opts.libraries.map((t) => splitLayerToken(t)[0]))] + .map((name) => libraryManifests()[name]) + .filter((m): m is NonNullable => m !== undefined); + // FR-023 §4.3 — the run's composed registry, threaded onto every GenContext. // Mirrors sdk's `loadMemory` exactly (`defaultLoadMemoryProviders` = core // providers, then the project's own `config.providers` appended) so @@ -528,6 +562,20 @@ export async function runGen(opts: RunGenOpts): Promise { warnMissingPromptGenerators(root, config.generators, (m) => warnings.push(m)); warnUnmarkedUiGenerators(config.generators, (m) => warnings.push(m)); + // Codegen is opt-in, so `meta gen` is the post-selection audit: the two ways a legal + // selection still surprises you. Warnings only — see catalog-gates.ts. + const catalog = opts.catalog ?? generatorRegistry; + warnUnsatisfiedRequires(config.generators, catalog, (m) => warnings.push(m)); + warnMixedApiFrameworks(config.generators, catalog, (m) => warnings.push(m)); + // FR-043 §6 — only when the caller actually told us what was selected. A programmatic + // caller that never threads `libraries` gets no library warnings at all, rather than + // "you opted into nothing", which would be a claim we cannot support. + if (opts.libraries !== undefined) { + warnLibrarySelectionMismatch( + config.generators, catalog, libraryManifests(), opts.libraries, (m) => warnings.push(m), + ); + } + // Names is opt-in on TypeScript and an existing project gets no signal that // it exists. Fires ONCE, on the first gen after crossing the release that made it the // doctrine — see shouldNoteNamesArtifactAbsent for why it is keyed on the engine stamp @@ -644,6 +692,7 @@ export async function runGen(opts: RunGenOpts): Promise { ...(projectRoot !== undefined && { projectRoot }), registry, ...(opts.sourceFiles !== undefined && { sourceFiles: opts.sourceFiles }), + ...(selectedManifests !== undefined && { libraries: selectedManifests }), warn: (msg) => warnings.push(`[${generator.name}] ${msg}`), }; diff --git a/server/typescript/packages/codegen-ts/test/ai-trace-sti.test.ts b/server/typescript/packages/codegen-ts/test/ai-trace-sti.test.ts index 6a46d9571..fbc692a98 100644 --- a/server/typescript/packages/codegen-ts/test/ai-trace-sti.test.ts +++ b/server/typescript/packages/codegen-ts/test/ai-trace-sti.test.ts @@ -15,12 +15,7 @@ const STI_MODEL = JSON.stringify({ "metadata.root": { package: "t::ai", children { "object.value": { name: "ClassifyRes", children: [{ "field.string": { name: "label", "@required": true } }] } }, { "object.value": { name: "SummarizeReq", children: [{ "field.string": { name: "doc" } }] } }, { "object.value": { name: "SummarizeRes", children: [{ "field.string": { name: "summary", "@required": true } }] } }, - { "object.entity": { name: "LlmCallBase", abstract: true, children: [ - { "field.uuid": { name: "spanId" } }, - { "field.string": { name: "callType" } }, - { "field.string": { name: "status" } }, - ] } }, - { "object.entity": { name: "PromptTrace", extends: "LlmCallBase", "@discriminator": "callType", children: [ + { "object.entity": { name: "PromptTrace", extends: "metaobjects::ai::LlmCallBase", "@discriminator": "callType", children: [ { "source.rdb": { "@table": "prompt_llm_call", "@role": "primary" } }, { "identity.primary": { "name": "id", "@fields": ["spanId"] } }, ] } }, @@ -40,7 +35,11 @@ async function genTrace(): Promise<{ classify: string; summarize: string }> { const tmp = mkdtempSync(join(tmpdir(), "ai1c-out-")); const dir = mkdtempSync(join(tmpdir(), "ai1c-model-")); writeFileSync(join(dir, "m.json"), STI_MODEL); - const loaded = await MetaDataLoader.fromDirectory(dir); + // The REAL shipped base, not a bespoke one named the same: `trace-helper` keys on + // the `ai` manifest's anchor and compares by node identity (FR-043 §6), so a fixture + // declaring its own `LlmCallBase` is exactly the bypass ADR-0024 warned about — it + // proved nothing about the path an adopter follows, and now it does not even fire. + const loaded = await MetaDataLoader.fromDirectory(dir, { libraries: ["ai"] }); rmSync(dir, { recursive: true, force: true }); expect(loaded.errors).toEqual([]); const out = await runGen({ @@ -94,7 +93,11 @@ describe("ai-trace #1c — STI table collapse", () => { test("N trace subtypes collapse to one prompt_llm_call table", async () => { const dir = mkdtempSync(join(tmpdir(), "ai1c-schema-")); writeFileSync(join(dir, "m.json"), STI_MODEL); - const loaded = await MetaDataLoader.fromDirectory(dir); + // The REAL shipped base, not a bespoke one named the same: `trace-helper` keys on + // the `ai` manifest's anchor and compares by node identity (FR-043 §6), so a fixture + // declaring its own `LlmCallBase` is exactly the bypass ADR-0024 warned about — it + // proved nothing about the path an adopter follows, and now it does not even fire. + const loaded = await MetaDataLoader.fromDirectory(dir, { libraries: ["ai"] }); rmSync(dir, { recursive: true, force: true }); expect(loaded.errors).toEqual([]); diff --git a/server/typescript/packages/codegen-ts/test/generator-registry.test.ts b/server/typescript/packages/codegen-ts/test/generator-registry.test.ts index 6f0d86fa4..3faef0fb5 100644 --- a/server/typescript/packages/codegen-ts/test/generator-registry.test.ts +++ b/server/typescript/packages/codegen-ts/test/generator-registry.test.ts @@ -3,8 +3,10 @@ import { generatorRegistry, listGenerators, getGenerator, + GENERATOR_LAYERS, type GeneratorRegistryEntry, } from "../src/generator-registry.js"; +import { REFERENCE_GENERATOR_NAMES } from "../src/reference-templates.js"; // ADR-0021 D3 — stable-name generator registry. The stable names are the // cross-port contract; this test pins them + asserts each entry carries a @@ -29,6 +31,7 @@ const EXPECTED_NATIVE = [ "template", "api-docs", "trace-helper", + "requirement-tests", "shared-model", ] as const; @@ -108,4 +111,45 @@ describe("generator-registry (ADR-0021 D3)", () => { // embedded its own literal. expect(gen?.emitsNames).toBe(true); }); + + // ----- the catalog facets (opt-in codegen design §D2b) -------------------- + + test("every entry declares kind, a known layer, and ejectable", () => { + for (const [id, entry] of Object.entries(generatorRegistry)) { + expect(entry.kind, `${id}.kind`).toBe("generator"); + expect(GENERATOR_LAYERS, `${id}.layer`).toContain(entry.layer); + expect(typeof entry.ejectable, `${id}.ejectable`).toBe("boolean"); + } + }); + + test("ejectable is DERIVED from the shipped reference templates, never hand-kept", () => { + const templates = new Set(REFERENCE_GENERATOR_NAMES); + for (const [id, entry] of Object.entries(generatorRegistry)) { + expect(entry.ejectable, `${id}.ejectable`).toBe(templates.has(id)); + } + // ...and every template this package ships has an entry to be reached through. + for (const name of templates) { + expect(Object.keys(generatorRegistry), `template ${name} has a registry entry`).toContain(name); + } + }); + + test("requires only names entries this slice actually contains", () => { + // codegen-ts is self-contained: nothing here may require a react/tanstack name, + // because this package cannot see those slices to resolve one. + for (const [id, entry] of Object.entries(generatorRegistry)) { + for (const dep of entry.requires ?? []) { + expect(Object.keys(generatorRegistry), `${id} requires ${dep}`).toContain(dep); + } + } + }); + + test("no entry declares a framework — every codegen-ts generator but routes is neutral", () => { + // The two `api`-layer route generators are the only framework-bearing entries in + // this package; `client` (react/tanstack) lives in the other two slices. + const framed = Object.values(generatorRegistry) + .filter((e) => e.framework !== undefined) + .map((e) => `${e.name}=${e.framework}`) + .sort(); + expect(framed).toEqual(["routes-hono=hono", "routes=fastify"]); + }); }); diff --git a/server/typescript/packages/codegen-ts/test/generators/trace-helper-anchor.test.ts b/server/typescript/packages/codegen-ts/test/generators/trace-helper-anchor.test.ts new file mode 100644 index 000000000..af908c0cd --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/generators/trace-helper-anchor.test.ts @@ -0,0 +1,121 @@ +// FR-043 §6 — `trace-helper` keys on a library ANCHOR, not on a hard-coded name. +// +// What this replaces: `const LLM_CALL_BASE = "LlmCallBase"`, compared against `.name` +// anywhere in an entity's super chain. Any adopter entity called `LlmCallBase`, in any +// package, triggered the generator — and the shipped abstract it was meant to key on +// was never actually the thing being matched, which is how the whole codegen suite came +// to test this generator against bespoke bases wearing that name (ADR-0024's complaint, +// on this port). +// +// The anchor is declared in `library/ai/library.json` and RESOLVED: to a node in the +// run's own loaded root, compared by node identity. +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { librarySources, libraryManifests } from "@metaobjectsdev/metadata/library"; +import { traceHelperFile } from "../../src/generators/trace-helper-file.js"; +import type { GenContext } from "../../src/generator.js"; + +/** An entity extending `base`, with the prompt + VO columns the helper needs. */ +function model(base: string, extras: unknown[] = []): string { + return JSON.stringify({ + "metadata.root": { + package: "app::ops", + children: [ + ...extras, + { "object.value": { name: "AskVO", children: [{ "field.string": { name: "question" } }] } }, + { "object.value": { name: "AnswerVO", children: [{ "field.string": { name: "answer" } }] } }, + { + "object.entity": { + name: "ApiCall", + extends: base, + children: [ + { "source.rdb": { "@table": "api_call", "@role": "primary" } }, + { "identity.primary": { name: "id", "@fields": ["traceId"] } }, + { + "template.prompt": { + name: "AskPrompt", + "@textRef": "p/ask", + "@payloadRef": "AskVO", + "@responseRef": "AnswerVO", + "@format": "json", + }, + }, + ], + }, + }, + ], + }, + }); +} + +/** The adopter's OWN abstract, named exactly like the shipped one. */ +const IMPOSTOR = { + "object.entity": { + name: "LlmCallBase", + abstract: true, + children: [ + { "field.uuid": { name: "traceId" } }, + { "field.string": { name: "callType" } }, + { "field.string": { name: "llmRequest", "@dbColumnType": "jsonb" } }, + ], + }, +}; + +async function emitCount( + doc: string, + opts: { withLibrary: boolean; libraries?: GenContext["libraries"] }, +): Promise { + const res = await new MetaDataLoader().load([ + ...(opts.withLibrary ? librarySources(["ai"]) : []), + new InMemoryStringSource(doc, { id: "meta.json", format: "json" }), + ]); + expect(res.errors).toEqual([]); + const ctx = { + entities: res.root.objects(), + loadedRoot: res.root, + matches: () => true, + config: { outDir: "/tmp/out", dialect: "postgres" } as never, + ...(opts.libraries !== undefined ? { libraries: opts.libraries } : {}), + warn: () => {}, + } as GenContext; + return (await traceHelperFile().generate(ctx)).length; +} + +const AI = [libraryManifests()["ai"]!]; + +describe("trace-helper keys on the library anchor (FR-043 §6)", () => { + test("fires for an entity extending the SHIPPED base", async () => { + const count = await emitCount(model("metaobjects::ai::LlmCallBase"), { + withLibrary: true, + libraries: AI, + }); + expect(count).toBe(1); + }); + + test("does NOT fire for an adopter's own abstract of the same NAME", async () => { + // The latent bug, pinned. `app::ops::LlmCallBase` is a different node in a + // different package, and the helper it would have emitted writes columns that + // entity does not declare. + const count = await emitCount(model("LlmCallBase", [IMPOSTOR]), { + withLibrary: true, + libraries: AI, + }); + expect(count).toBe(0); + }); + + test("an EMPTY selection is honoured: the caller looked, and nothing is opted in", async () => { + const count = await emitCount(model("metaobjects::ai::LlmCallBase"), { + withLibrary: true, + libraries: [], + }); + expect(count).toBe(0); + }); + + test("no selection at all falls back to every shipped anchor — still FQN, never a bare name", async () => { + // A programmatic `runGen()` that never threads `libraries`. The fallback keeps + // that caller working without reintroducing the name match: the impostor arm + // below is the half that proves it. + expect(await emitCount(model("metaobjects::ai::LlmCallBase"), { withLibrary: true })).toBe(1); + expect(await emitCount(model("LlmCallBase", [IMPOSTOR]), { withLibrary: true })).toBe(0); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/generators/trace-helper-response-format.test.ts b/server/typescript/packages/codegen-ts/test/generators/trace-helper-response-format.test.ts index 3f95cc008..fed4d1c0c 100644 --- a/server/typescript/packages/codegen-ts/test/generators/trace-helper-response-format.test.ts +++ b/server/typescript/packages/codegen-ts/test/generators/trace-helper-response-format.test.ts @@ -9,6 +9,7 @@ // read that yields Format.JSON; only reading @responseFormat yields Format.XML. import { describe, test, expect } from "bun:test"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { librarySources } from "@metaobjectsdev/metadata/library"; import { traceHelperFile } from "../../src/generators/trace-helper-file.js"; import type { GenContext } from "../../src/generator.js"; @@ -29,21 +30,10 @@ async function loadRoot(promptAttrs: Record) { children: [{ "field.string": { name: "answer" } }], }, }, - { - "object.entity": { - name: "LlmCallBase", - abstract: true, - children: [ - { "field.uuid": { name: "traceId" } }, - { "field.string": { name: "callType" } }, - { "field.string": { name: "llmRequest", "@dbColumnType": "jsonb" } }, - ], - }, - }, { "object.entity": { name: "ApiCall", - extends: "LlmCallBase", + extends: "metaobjects::ai::LlmCallBase", children: [ { "source.rdb": { "@table": "api_call", "@role": "primary" } }, { "identity.primary": { name: "id", "@fields": ["traceId"] } }, @@ -61,7 +51,10 @@ async function loadRoot(promptAttrs: Record) { ], }, }; + // The shipped base, not a local one wearing its name: `trace-helper` keys on the + // `ai` manifest's anchor and compares by node identity (FR-043 §6). const res = await new MetaDataLoader().load([ + ...librarySources(["ai"]), new InMemoryStringSource(JSON.stringify(doc), { id: "meta.json", format: "json" }), ]); expect(res.errors).toEqual([]); diff --git a/server/typescript/packages/codegen-ts/test/golden/generator-registry-conformance.test.ts b/server/typescript/packages/codegen-ts/test/golden/generator-registry-conformance.test.ts index 30baf1052..3332a0db8 100644 --- a/server/typescript/packages/codegen-ts/test/golden/generator-registry-conformance.test.ts +++ b/server/typescript/packages/codegen-ts/test/golden/generator-registry-conformance.test.ts @@ -5,11 +5,19 @@ // The manifest is the single source of truth. For THIS port (`typescript`) the // README contract is: // 1. Every stable name the TS registry exposes appears in the manifest. -// 2. Presence both ways — every manifest entry whose `ports` includes -// `typescript` IS in the TS registry, and the TS registry exposes NO name -// whose `ports` omits `typescript`. (Set equality catches both at once.) -// 3. Tier agreement — a manifest name marked `tier: "neutral"` is flagged +// 2. Tier agreement — a manifest name marked `tier: "neutral"` is flagged // neutral in the TS registry; native manifest names are NOT neutral. +// 3. Layer agreement — a manifest name's `layer` equals the TS registry's. +// 4. Every manifest entry declares one of the six layers. +// +// PRESENCE BOTH WAYS IS ASSERTED ELSEWHERE. `codegen-ts` is one SLICE of the +// TypeScript catalog: `form` lives in `codegen-ts-react` and `hooks`/`grid`/ +// `grid-hook` in `codegen-ts-tanstack`, and this package cannot import its own +// dependents to see them. Set equality against the manifest's `typescript` slice is +// therefore a property of the COMPOSED catalog and is asserted in +// `packages/cli/test/catalog-conformance.test.ts`, which is the only place all three +// slices are visible at once. What is checkable here is the one direction that does +// not need them: no rogue names. // // If this test fails, the manifest and the TS registry DISAGREE: report the diff; // do NOT mutate the manifest to force a pass (the manifest is reconciled @@ -45,10 +53,17 @@ function findRepoRoot(start: string): string { interface ManifestEntry { concept: string; tier: "native" | "neutral"; + layer: string; note?: string; ports: string[]; } +// The closed set, mirrored from GENERATOR_LAYERS. Spelled out here rather than +// imported so this gate fails if the CODE's union and the MANIFEST's values ever +// diverge from the six the design ruled — importing the union would make the test +// agree with whatever the code says. +const LAYERS = ["model", "persistence", "api", "client", "docs", "capability"] as const; + interface Manifest { ports: string[]; generators: Record; @@ -78,32 +93,39 @@ describe("generator registry — conforms to canonical stable-name manifest (ADR expect(manifest.ports).toContain(PORT); }); - it(`TS registry names == manifest's ${PORT} slice (no rogue, no missing)`, () => { + it(`no rogue names — every name codegen-ts registers is a ${PORT} name in the manifest`, () => { const extraInRegistry = [...actualNames] .filter((n) => !expectedNames.has(n)) .sort(); - const missingFromRegistry = [...expectedNames] - .filter((n) => !actualNames.has(n)) - .sort(); const message = [ - `TS generator registry disagrees with the canonical manifest for port "${PORT}".`, - ` extra in registry (name registered but manifest's ${PORT} omits it): [${extraInRegistry.join(", ")}]`, - ` missing from registry (manifest expects ${PORT} but not registered): [${missingFromRegistry.join(", ")}]`, + `The codegen-ts registry exposes names the canonical manifest does not give to "${PORT}".`, + ` extra in registry: [${extraInRegistry.join(", ")}]`, ` manifest: ${manifestPath}`, + " (The other direction — every manifest typescript name IS registered — is asserted", + " on the COMPOSED catalog in packages/cli/test/catalog-conformance.test.ts, because", + " codegen-ts is one slice of three.)", ].join("\n"); - // Assert the symmetric difference is empty (actionable on failure). + expect(extraInRegistry, message).toEqual([]); + }); + + it("every manifest entry declares one of the six layers", () => { + const bad = Object.entries(manifest.generators) + .filter(([, e]) => !(LAYERS as readonly string[]).includes(e.layer)) + .map(([n, e]) => `${n}=${String(e.layer)}`) + .sort(); expect( - { extraInRegistry, missingFromRegistry }, - message, - ).toEqual({ extraInRegistry: [], missingFromRegistry: [] }); + bad, + `entries with a missing or unknown layer (allowed: ${LAYERS.join(", ")}): ${bad.join(", ")}`, + ).toEqual([]); }); - // Tier agreement — only over names present in BOTH sets. + // Tier + layer agreement — only over names present in BOTH sets. const sharedNames = [...expectedNames].filter((n) => actualNames.has(n)).sort(); for (const name of sharedNames) { const manifestTier = manifest.generators[name]!.tier; + const manifestLayer = manifest.generators[name]!.layer; it(`tier agreement: "${name}" is ${manifestTier} in both manifest and TS registry`, () => { const registryTier = generatorRegistry[name]!.tier; if (manifestTier === "neutral") { @@ -112,5 +134,11 @@ describe("generator registry — conforms to canonical stable-name manifest (ADR expect(registryTier).not.toBe("neutral"); } }); + it(`layer agreement: "${name}" is ${manifestLayer} in both manifest and TS registry`, () => { + // Compared as plain strings: the manifest is the source of truth and its value is + // untyped here on purpose, so narrowing it to the code's union would make the + // gate agree with whatever the code says. + expect(String(generatorRegistry[name]!.layer)).toBe(manifestLayer); + }); } }); diff --git a/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts b/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts index eaa12cde9..6e3ca1fdc 100644 --- a/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts +++ b/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts @@ -91,6 +91,20 @@ function genConfig(outDir: string) { }); } +/** + * The warnings this file cares about — everything except the api-framework advisory. + * + * These tests wire BOTH route generators deliberately: the question under test is + * whether a sourceless object gets DB-bound artifacts, and asking it of both HTTP + * surfaces at once is the point. That is exactly the shape `warnMixedApiFrameworks` + * exists to remark on, so the advisory fires here and is correct to. Filtering it by + * substring rather than relaxing the assertion to "some warnings" keeps every OTHER + * unexpected warning failing these tests, which is what `toEqual([])` was buying. + */ +function unrelatedWarnings(warnings: readonly string[]): string[] { + return warnings.filter((w) => !w.includes("two api-layer frameworks are wired")); +} + // Task 3 — the ADR-0034 scaffold-and-own reference templates (src/reference/*) // must gate queries/routes emission on the same hasAnyRdbSource signal as the // engine generators above. `entityFile` is reused unchanged (its table-vs-shape @@ -111,7 +125,7 @@ describe("#248 R2 — sourceless objects get no DB-bound artifacts", () => { test("Order (sourced) gets entity+queries+routes+hono; Money (value) and Ghost (sourceless entity) get neither", async () => { const root = await loadRoot([ORDER, MONEY, GHOST]); const out = await runGen({ config: genConfig(tmp), metadata: root }); - expect(out.warnings).toEqual([]); + expect(unrelatedWarnings(out.warnings)).toEqual([]); // runGen reports files.path as absolute (outDir-joined) paths. const paths = new Set(out.files.map((f) => f.path)); @@ -145,7 +159,7 @@ describe("#248 R2 — sourceless objects get no DB-bound artifacts", () => { test("reference (scaffold-and-own) generators: same gating as the engine — Order gets queries+routes; Money/Ghost get neither", async () => { const root = await loadRoot([ORDER, MONEY, GHOST]); const out = await runGen({ config: refGenConfig(tmp), metadata: root }); - expect(out.warnings).toEqual([]); + expect(unrelatedWarnings(out.warnings)).toEqual([]); const paths = new Set(out.files.map((f) => f.path)); const at = (name: string) => join(tmp, name); @@ -235,7 +249,7 @@ describe("#248 R2 — sourceless objects get no DB-bound artifacts", () => { ]); const out = await runGen({ config: genConfig(tmp), metadata: root }); - expect(out.warnings).toEqual([]); + expect(unrelatedWarnings(out.warnings)).toEqual([]); const paths = new Set(out.files.map((f) => f.path)); expect(paths.has(join(tmp, "ProgramSummary.ts"))).toBe(true); expect(paths.has(join(tmp, "ProgramSummary.queries.ts"))).toBe(true); diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index 913b010d4..8cc39d23f 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -260,6 +260,16 @@ export const ERROR_CODES = [ // Phase-1 metadata-source-resolution — no metadata collection was discovered: // no config declaring sources, and no default metaobjects/ directory. "ERR_COLLECTION_NOT_FOUND", + // FR-043 — `.metaobjects/config.json`'s `libraries` names a shipped library or layer + // this build does not have. A HUMAN typed it, so it is refused with the available + // tokens rather than skipped: skipped, it resurfaces as ERR_UNRESOLVED_SUPER against + // the adopter's own metadata, which is the wrong place to send someone looking. + "ERR_UNKNOWN_LIBRARY", + // FR-043 — a node is declared by BOTH an adopter's own metadata and a shipped library the project opts into — the `meta eject ` copy with the library still in `libraries`. The two merge silently and ASYMMETRICALLY: additions take, deletions do not, because the library still declares what was removed. + // Raised by the TypeScript SDK's load path; registered in every port so the shared corpus list stays one set. + "ERR_LIBRARY_PACKAGE_COLLISION", + // FR-043 — a NEW top-level node is declared into a package a shipped library owns while that library is opted in — a later release of the library may ship a node of that name and merge into it. `overlay: true` on one of the library's OWN nodes is the documented amendment door and is untouched. + "ERR_LIBRARY_PACKAGE_NOT_OWNED", // FR-023 — a declared dependency's transport could not locate a directory holding // metaobjects.pkg.json. "ERR_DEPENDENCY_UNRESOLVED", diff --git a/server/typescript/packages/metadata/src/library/embedded-library.generated.ts b/server/typescript/packages/metadata/src/library/embedded-library.generated.ts index 6f28f4712..50b0f8049 100644 --- a/server/typescript/packages/metadata/src/library/embedded-library.generated.ts +++ b/server/typescript/packages/metadata/src/library/embedded-library.generated.ts @@ -6,5 +6,16 @@ // wherever the on-disk library/ directory is unavailable. // Keys are refs: path under library/ minus the .yaml extension. export const EMBEDDED_LIBRARY: Record = { - "ai/llm-call": "# library/ai/llm-call.yaml\n# MetaObjects-shipped standard metadata. Adopters opt in via the loader's\n# `libraries: [\"ai\"]` option, then `extends: \"metaobjects::ai::LlmCallBase\"`.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n children:\n - source.rdb: { table: llm_call, role: primary }\n - identity.primary: { name: id, fields: [\"spanId\"] }\n", + "ai/db": "# library/ai/db.yaml — the DB PERSISTENCE layer for metaobjects::ai.\n#\n# Opted into as `\"ai/db\"`, which IMPLIES `\"ai\"`: `LlmCall` is declared in model.yaml and\n# this file only re-opens it, so without the core layer the overlay has no target.\n#\n# `LlmCall` is the concrete, table-backed instance of the abstract envelope. An adopter\n# who wants their OWN table (a different name, extra columns, a different id strategy)\n# extends `LlmCallBase` in their own metadata and never opts into this layer at all.\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCall\n overlay: true\n children:\n - source.rdb: { table: llm_call, role: primary }\n", + "ai/model": "# library/ai/model.yaml — the CORE layer: the LLM-call trace envelope.\n#\n# Adopters opt in via `libraries: [\"ai\"]`, then `extends: \"metaobjects::ai::LlmCallBase\"`.\n#\n# This layer declares NO `source.rdb`, so opting into `\"ai\"` alone adds zero tables and\n# zero generated code — the design is present and resolvable, and nothing else happens\n# until the adopter adds `\"ai/db\"`. See library/iam/model.yaml for the full rationale.\n#\n# This file was split out of the former `library/ai/llm-call.yaml`, which shipped the\n# abstract base and a concrete `LlmCall` carrying `source.rdb` together. That was\n# recorded as an accepted wart on the grounds that splitting would change what existing\n# `ai` adopters get; a sweep of the estate found there are none, so it was closed rather\n# than documented (FR-043 Amendment 1).\nmetadata:\n package: metaobjects::ai\n children:\n - object.entity:\n name: LlmCallBase\n abstract: true\n children:\n - field.uuid: { name: traceId }\n - field.uuid: { name: spanId }\n - field.uuid: { name: parentSpanId }\n - field.string: { name: sessionId }\n - field.string: { name: callType }\n - field.string: { name: system }\n - field.string: { name: requestModel }\n - field.string: { name: responseModel }\n - field.int: { name: inputTokens }\n - field.int: { name: outputTokens }\n - field.currency: { name: costMinor, currency: USD }\n - field.int: { name: latencyMs }\n - field.string: { name: finishReason }\n - field.string: { name: status }\n - field.string: { name: errorDetail }\n - field.timestamp: { name: startedAt }\n - field.string: { name: llmRequest, dbColumnType: jsonb } # generic jsonb (no objectRef)\n - field.string: { name: llmResponse, dbColumnType: jsonb }\n - object.entity:\n name: LlmCall\n extends: metaobjects::ai::LlmCallBase\n description: The concrete trace row. Its `source.rdb` lives in db.yaml, so opting into the core layer alone declares the shape without proposing a table.\n children:\n - identity.primary: { name: id, fields: [\"spanId\"] }\n", + "ai/requirements": "# library/ai/requirements.yaml — what the LLM-call trace envelope PROMISES.\n#\n# A RETROFIT, not new design: llm-call.yaml landed 2026-06-03 and `requirement.functional`\n# first appears 2026-08-11, so the library could not have carried requirements when it was\n# written. That is why this file is worth reading as a worked example — it shows what\n# declaring the design of something that already exists actually turns up.\n#\n# The entry that earns its keep is `typedIo`, honestly `partial` + `accepted`: the library\n# declares the ENVELOPE and the adopter declares the typed VO columns. Recording that seam\n# in the ledger is where an agent meets it, before adding a fourth trace column.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. An L4 names the OBJECT it is about;\n# the fields that carry it hang off it as an L5 child. Writing the fields at L4 is\n# ERR_REQUIREMENT_L4_NOT_OBJECT, and writing the concerns as SIBLINGS of the L2 leaves the\n# L2 claiming nothing — both of which this file did until the standalone verify gate\n# existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::ai\n children:\n - requirement.functional:\n name: llmTracing\n level: 2\n status: live\n statement: Every call to a language model leaves a row that says what was asked, what came back, what it cost and how long it took.\n counterexample: A spend figure nobody can attribute to a call.\n description: The segment this library covers. Its three children below are the concerns it decomposes into.\n children:\n - requirement.functional:\n name: envelope\n level: 4\n status: live\n statement: A trace row identifies its call and its place in a trace — trace, span, parent span, session, call type, system.\n counterexample: A log line that cannot be joined to the request that produced it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: traceAddressing\n level: 5\n status: live\n statement: The four addressing columns are declared on the base — trace, span, parent span and session.\n counterexample: A row whose place in a trace is inferred from insertion order.\n description: >-\n The member grain exists here so the claim RESOLVES against the fields\n themselves: renaming or dropping one of them dangles this reference and\n fails the build, which naming the object alone would not.\n implementedBy: [LlmCallBase.traceId, LlmCallBase.spanId, LlmCallBase.parentSpanId, LlmCallBase.sessionId]\n\n - requirement.functional:\n name: accounting\n level: 4\n status: live\n statement: A trace row carries the tokens in, the tokens out, and the cost in integer minor units.\n counterexample: A cost stored as a float.\n description: >-\n `field.currency` — integer minor units on the wire, always. Float arithmetic for\n money is forbidden by the cross-port wire contract, and a spend total is exactly\n the sum that exposes it.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: tokenAndCostColumns\n level: 5\n status: live\n statement: Tokens in, tokens out and cost are three declared columns, the cost a field.currency.\n counterexample: A cost column declared as a double.\n implementedBy: [LlmCallBase.inputTokens, LlmCallBase.outputTokens, LlmCallBase.costMinor]\n\n - requirement.functional:\n name: typedIo\n level: 4\n status: partial\n disposition: accepted\n statement: The request and response bodies are stored as structured jsonb, not as opaque text.\n counterexample: A prompt stored as a string nobody can query a field out of.\n notes: >-\n The library declares the two columns as generic jsonb with no `@objectRef`,\n because it cannot know the adopter's request/response shape. Typing them is the\n ADOPTER's move: declare an `object.value` and overlay the field with\n `@objectRef` + `@storage: jsonb`. This is the seam ADR-0024 drew, recorded here\n rather than in prose so it is in the ledger an agent reads before adding a\n fourth trace column of its own.\n implementedBy: [LlmCallBase]\n children:\n - requirement.functional:\n name: jsonbBodies\n level: 5\n status: live\n statement: The request and response bodies are declared as jsonb columns on the base.\n counterexample: A prompt stored in a text column.\n description: >-\n `live` where its parent is `partial`, and the split is the point: the\n COLUMNS are shipped and this claim is fully realised; what is outstanding\n is the TYPING of them, which is the parent's gap and the adopter's move.\n implementedBy: [LlmCallBase.llmRequest, LlmCallBase.llmResponse]\n\n - requirement.architectural:\n name: traceRowsCarryTiming\n status: live\n statement: Every trace row records when the call started and how long it took.\n counterexample: A latency figure derived from log timestamps after the fact.\n description: >-\n Architectural, so it propagates down `extends` to every adopter entity deriving\n from LlmCallBase — which is the point: an adopter's own trace table is claimed\n by this requirement for free, and dropping the columns breaks the build.\n implementedBy: [LlmCallBase]\n\n - requirement.architectural:\n name: traceRowsCarryOutcome\n status: live\n statement: Every trace row records how the call ended — a status, a finish reason, and the error detail when there was one.\n counterexample: A failed call indistinguishable from one that never happened.\n implementedBy: [LlmCallBase]\n", + "iam/db": "# library/iam/db.yaml — the DB PERSISTENCE layer for metaobjects::iam.\n#\n# Opted into as `\"iam/db\"`, which IMPLIES `\"iam\"`: this file is nothing but\n# `overlay: true` redeclarations, and an overlay whose target was never declared is\n# ERR_OVERLAY_NO_TARGET.\n#\n# It carries exactly two kinds of child — `source.rdb` and `index.lookup` — and nothing\n# else. The field set, the identities and the relationships all live in model.yaml,\n# because they are the DESIGN; what lives here is where the rows go and which lookups are\n# worth an index. Add a field here and the core layer stops being the whole model, which\n# is the thing the split exists to guarantee.\n#\n# Physical names are `iam_`-prefixed. Two reasons, both real: `user` and `group` are\n# reserved words in Postgres, and an adopter very likely has tables of their own by those\n# names. A library that collides on a table name is a library nobody can adopt.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: User\n overlay: true\n children:\n - source.rdb: { table: iam_user, role: primary }\n\n - object.entity:\n name: GroupType\n overlay: true\n children:\n - source.rdb: { table: iam_group_type, role: primary }\n\n - object.entity:\n name: Group\n overlay: true\n children:\n - source.rdb: { table: iam_group, role: primary }\n # Nesting is walked parent-ward constantly; the FK alone gives no index.\n - index.lookup: { name: ixParent, fields: [parentId] }\n\n - object.entity:\n name: Role\n overlay: true\n children:\n - source.rdb: { table: iam_role, role: primary }\n\n - object.entity:\n name: Permission\n overlay: true\n children:\n - source.rdb: { table: iam_permission, role: primary }\n\n - object.entity:\n name: GroupMember\n overlay: true\n children:\n - source.rdb: { table: iam_group_member, role: primary }\n # The composite PK covers (userId, groupId), so \"who is in this group?\" —\n # the other direction — has no index without this one. Same reasoning for\n # every ixSecond below.\n - index.lookup: { name: ixGroup, fields: [groupId] }\n\n - object.entity:\n name: RolePermission\n overlay: true\n children:\n - source.rdb: { table: iam_role_permission, role: primary }\n - index.lookup: { name: ixPermission, fields: [permissionId] }\n\n - object.entity:\n name: UserRole\n overlay: true\n children:\n - source.rdb: { table: iam_user_role, role: primary }\n - index.lookup: { name: ixRole, fields: [roleId] }\n\n - object.entity:\n name: GroupMemberRole\n overlay: true\n children:\n - source.rdb: { table: iam_group_member_role, role: primary }\n # \"who holds this role in this group?\" — the scoped-grant read.\n - index.lookup: { name: ixGroupRole, fields: [groupId, roleId] }\n", + "iam/model": "# library/iam/model.yaml — the CORE layer: identity and access management.\n#\n# Adopters opt in via `libraries: [\"iam\"]` in .metaobjects/config.json.\n#\n# This layer declares NO `source.rdb`, and that is the whole point of the split. A\n# sourceless object is inert by a contract that already ships: migrate skips an object\n# with no writable source, and codegen emits no route, queries, hooks, grid or form for\n# one (both citing #248 — persistability derives from source presence, never from the\n# object subtype). It still gets a type-only interface, so `extends` and reference work.\n#\n# So `libraries: [\"iam\"]` adds ZERO tables and ZERO generated code. What an adopter gains\n# is the design being present and resolvable: an agent working in the repo knows the\n# capability exists and can draw on it, and nothing else happens until the adopter adds\n# `\"iam/db\"`.\n#\n# Authoring discipline (FR-043 §3.1), so the departures are visible:\n# - `field.uuid` + `generation: uuid` on principals; composite ASSIGNED keys on\n# junctions. Never `increment` — a library cannot know the adopter's id strategy.\n# - Physical names carry the `iam_` prefix (in db.yaml): `user` and `group` are\n# reserved words in Postgres, and an adopter has tables of their own.\n# - No adopter-facing profile data. That arrives by `overlay: true`.\n# - No credentials. See requirements.yaml → `noCredentialsOnUser`.\nmetadata:\n package: metaobjects::iam\n children:\n - object.entity:\n name: IamBase\n abstract: true\n description: Shared shape of every iam principal and definition — a stable uuid plus change timestamps. Junctions do not extend it; they are addressed by their participants.\n children:\n - field.uuid: { name: id, required: true }\n - field.timestamp: { name: createdAt, autoSet: onCreate }\n - field.timestamp: { name: updatedAt, autoSet: onUpdate }\n\n - object.entity:\n name: User\n extends: IamBase\n description: A person or service account that can be granted access. Carries no authentication secret of any kind — see the noCredentialsOnUser requirement.\n children:\n - field.string: { name: username, required: true, maxLength: 64, filterable: true }\n - field.string: { name: email, required: true, maxLength: 254, stringFormat: email, filterable: true }\n - field.string: { name: displayName, maxLength: 120 }\n # NOT `filterable: true`, deliberately. The loader warns when a filterable\n # field is in no identity — filtering on it sequential-scans — and a library\n # must not ship a warning to every adopter. `username` and `email` carry it\n # because they have identity.secondary; `status` does not. An adopter who\n # wants to filter on status overlays `filterable` AND an index together,\n # which is exactly what the layer split is for.\n - field.enum: { name: status, required: true, values: [invited, active, suspended, closed], default: active }\n - field.timestamp: { name: emailVerifiedAt }\n - field.timestamp: { name: lastSeenAt }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqUsername, fields: [username] }\n - identity.secondary: { name: uqEmail, fields: [email] }\n - relationship.association: { name: groups, objectRef: Group, cardinality: many, through: GroupMember }\n - relationship.association: { name: roles, objectRef: Role, cardinality: many, through: UserRole }\n\n - object.entity:\n name: GroupType\n extends: IamBase\n description: What KIND of group this is — a team, a tenant, a project. An entity rather than an enum, because \"which roles may be held in this kind of group\" is data an adopter extends, and an enum's values cannot be extended by overlay.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n - object.entity:\n name: Group\n extends: IamBase\n description: A nestable collection of users, of a declared GroupType. Nesting is by parentId; acyclicity is an invariant the schema cannot express — see the acyclicGroupNesting requirement.\n children:\n - field.uuid: { name: groupTypeId, required: true }\n - field.uuid: { name: parentId }\n - field.string: { name: key, required: true, maxLength: 64 }\n # Not filterable for the same reason as User.status above.\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - identity.reference: { name: fkParent, fields: [parentId], references: Group, onDelete: restrict }\n\n - object.entity:\n name: Role\n extends: IamBase\n description: A reusable bundle of permissions. Code never compares a role NAME to a literal — it asks whether a user holds a permission, and the mapping is data.\n children:\n - field.string: { name: key, required: true, maxLength: 64 }\n - field.string: { name: name, required: true, maxLength: 120 }\n - field.string: { name: description, maxLength: 500 }\n - field.uuid: { name: groupTypeId, description: \"When set, this role may be held only within groups of this type; absent means grantable anywhere.\" }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n - identity.reference: { name: fkGroupType, fields: [groupTypeId], references: GroupType, onDelete: restrict }\n - relationship.association: { name: permissions, objectRef: Permission, cardinality: many, through: RolePermission }\n\n - object.entity:\n name: Permission\n extends: IamBase\n description: \"The assignable unit — a stable : key the application checks against. An entity, not an enum, on ADR-0037's own reasoning: it has its own identity, its own lifecycle, and a junction with real foreign keys.\"\n children:\n - field.string: { name: key, required: true, maxLength: 128, description: \"Stable : key the application checks against.\" }\n - field.string: { name: description, maxLength: 500 }\n - identity.primary: { name: pk, fields: [id], generation: uuid }\n - identity.secondary: { name: uqKey, fields: [key] }\n\n # ---- grant surface: every grant is a row, addressed by its participants ----\n #\n # Junctions do NOT extend IamBase: they have no identity of their own, and adding a\n # surrogate uuid to a row whose identity IS its participants invites a duplicate.\n\n - object.entity:\n name: GroupMember\n description: A user's membership of a group.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.timestamp: { name: joinedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n\n - object.entity:\n name: RolePermission\n description: A permission granted by a role.\n children:\n - field.uuid: { name: roleId, required: true }\n - field.uuid: { name: permissionId, required: true }\n - identity.primary: { name: pk, fields: [roleId, permissionId], generation: assigned }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: cascade }\n - identity.reference: { name: fkPermission, fields: [permissionId], references: Permission, onDelete: restrict }\n\n - object.entity:\n name: UserRole\n description: A system-wide grant of a role to a user.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n\n - object.entity:\n name: GroupMemberRole\n description: A grant of a role to a user WITHIN one group. Three foreign keys, so it is not an M:N @through junction (which must declare exactly two identity.reference children); it is read by explicit finders.\n children:\n - field.uuid: { name: userId, required: true }\n - field.uuid: { name: groupId, required: true }\n - field.uuid: { name: roleId, required: true }\n - field.timestamp: { name: grantedAt, autoSet: onCreate }\n - identity.primary: { name: pk, fields: [userId, groupId, roleId], generation: assigned }\n - identity.reference: { name: fkUser, fields: [userId], references: User, onDelete: cascade }\n - identity.reference: { name: fkGroup, fields: [groupId], references: Group, onDelete: cascade }\n - identity.reference: { name: fkRole, fields: [roleId], references: Role, onDelete: restrict }\n", + "iam/requirements": "# library/iam/requirements.yaml — what this library's design PROMISES.\n#\n# This is what makes iam a library rather than a schema snippet. Without requirements an\n# adopter gets nine tables; with them they get nine tables plus a build that is held to\n# \"no authorization decision is hard-wired to a name\", which no snippet can do.\n#\n# Two reading rules, both load-bearing:\n#\n# `live` here means \"the model AS SHIPPED realises this\" — never \"your application\n# does\". A ledger binds to model nodes; runtime guarantees are the runtime's tests, and\n# this library does not invent a way to point a requirement at code (@verifiedBy was\n# retired for exactly that). Behaviour the model cannot carry ships as `partial` +\n# `disposition: accepted` with a notes sentence naming what the adopter must do.\n#\n# The functional tree roots at L2, not L1. L1 is the adopter's SOLUTION, and a library\n# is by definition a segment of someone else's. Architectural claims ship flat.\n#\n# HIERARCHY IS NESTING, and the L4/L5 split is grain. The concerns are CHILDREN of the L2\n# rather than its siblings, and an L4 names the OBJECT it is about while the field that\n# carries it hangs off it as an L5 child. Written flat, the L2 claims nothing in its whole\n# subtree; written at L4, a field reference is ERR_REQUIREMENT_L4_NOT_OBJECT. Both shipped\n# here until the standalone verify gate existed (`cli/test/shipped-library-verify.test.ts`).\nmetadata:\n package: metaobjects::iam\n children:\n # ---- functional: the L2 segment and the concerns nested under it --------\n - requirement.functional:\n name: accessControl\n level: 2\n status: live\n statement: Who may do what is answered from stored grants, never from a name compared to a literal in code.\n counterexample: A branch that reads `if (user.role === \"admin\")`.\n description: The segment this library covers. The concerns beneath it are what it decomposes into.\n children:\n - requirement.functional:\n name: identity\n level: 4\n status: live\n statement: A person or service account is represented once, addressed by a uuid, and reachable by username or email.\n counterexample: Two rows for the same person because the email changed.\n implementedBy: [User]\n\n - requirement.functional:\n name: grouping\n level: 4\n status: live\n statement: Users are collected into typed, nestable groups, and the kind of group is data rather than a hard-coded set.\n counterexample: A `teamOrTenant` boolean.\n implementedBy: [Group, GroupType, GroupMember]\n\n - requirement.functional:\n name: acyclicGroupNesting\n level: 4\n status: partial\n disposition: accepted\n statement: A group is never its own ancestor.\n counterexample: Two groups each naming the other as parent.\n notes: >-\n The schema cannot express this — a self-referencing FK admits a cycle, and the\n only relational forms that would catch it (a recursive CHECK, a closure table\n maintained by trigger) are DB-specific and would not survive three dialects.\n The adopter enforces it where the write happens. Recorded rather than omitted\n so an agent reading the ledger before adding a parent-setting endpoint sees the\n obligation.\n implementedBy: [Group]\n\n - requirement.functional:\n name: grants\n level: 4\n status: live\n statement: A role is granted to a user either system-wide or scoped to one group, and both are ordinary rows.\n counterexample: A nullable `groupId` on one grant table, where NULL means \"everywhere\".\n description: >-\n Two junctions, not one with a nullable scope. A NULL in a unique key is DISTINCT\n from every other NULL in SQL, so a nullable-scope design lets the same global\n grant be inserted twice; the fix needs a partial index whose expression carries\n a physical column name. Two composite-keyed tables need no escape hatch and\n survive three dialects and five ports unchanged.\n implementedBy: [UserRole, GroupMemberRole]\n\n - requirement.functional:\n name: roleScopedToGroupType\n level: 4\n status: partial\n disposition: accepted\n statement: A role bound to a group type is granted only within groups of that type.\n counterexample: A \"tenant admin\" role granted inside a project group.\n notes: >-\n Expressing this relationally needs the grant row to carry the group's type and\n a composite FK back to (group, type) — three foreign keys deep, unverified\n across five ports' DDL and ORM paths. The adopter checks it at the point of\n grant. The declared half is the L5 child below; the enforcement is not.\n implementedBy: [Role, GroupMemberRole]\n children:\n - requirement.functional:\n name: roleDeclaresItsGroupType\n level: 5\n status: live\n statement: A role declares the group type it is bound to, as a nullable reference.\n counterexample: A role whose intended scope is recoverable only from its name.\n description: >-\n `live` where its parent is `partial`, and the split is grain as much as\n verdict: the DECLARATION is shipped and resolves against the field itself,\n so dropping the column fails the build — while the ENFORCEMENT, which no\n schema here can carry, stays the parent's accepted gap.\n implementedBy: [Role.groupTypeId]\n\n - requirement.functional:\n name: decision\n level: 4\n status: live\n statement: An authorization decision is the question \"does this user hold this permission key\", answered from rows.\n counterexample: A hard-coded list of usernames that bypass a check.\n implementedBy: [Permission, RolePermission]\n\n # ---- architectural: prohibitions in force --------------------------------\n\n - requirement.architectural:\n name: grantsAreRows\n status: live\n statement: A grant exists only as a stored row; nothing is granted by naming, position or convention.\n counterexample: A superuser recognised by username.\n implementedBy: [UserRole, GroupMemberRole, RolePermission, GroupMember]\n\n - requirement.architectural:\n name: noCredentialsOnUser\n status: live\n statement: A user row carries no authentication secret — no password, no hash, no knowledge-based question or answer.\n counterexample: A password or secret-answer column on the user table.\n description: >-\n Authentication is a separate capability with an entity per factor; this library\n is identity and authorization only.\n notes: >-\n This is the one thing every reader of a user table proposes adding, and a real\n legacy model of this shape stored a length-bounded plaintext password and a\n knowledge-based secret pair on the user row. Stating it as a prohibition IN\n FORCE — claimable, and rendered on agent/requirements.md — is what stops an\n agent extending \"the user model\" from re-deriving it on sight. It is\n `architectural`, not `retired`: retired is chartered for a capability built\n here and removed, and this library never built one.\n implementedBy: [User]\n\n - requirement.architectural:\n name: principalDeletionRevokesGrants\n status: live\n statement: Deleting a user or group removes its grants; deleting a role or permission still in use is refused.\n counterexample: A grant row pointing at a user who no longer exists.\n description: The referential rule in one sentence — cascade from a principal, restrict from a definition.\n implementedBy: [GroupMember, UserRole, GroupMemberRole, RolePermission]\n\n - requirement.architectural:\n name: stableIdentifiers\n status: live\n statement: Every principal and definition is addressed by a uuid that never changes; every grant by its participants.\n counterexample: A group referenced by its display name.\n implementedBy: [IamBase]\n", +}; + +/** Library NAME -> the exact text of its `library.json` manifest. */ +export const EMBEDDED_LIBRARY_MANIFESTS: Record = { + "ai": "{\n \"$comment\": \"Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.\",\n \"name\": \"ai\",\n \"kind\": \"feature\",\n \"stability\": \"stable\",\n \"since\": \"0.20.0\",\n \"description\": \"The LLM-call trace envelope: what was asked, what came back, what it cost, how long it took.\",\n \"useWhen\": \"the application calls a language model and someone will ask what it cost or why a call failed\",\n \"packages\": [\"metaobjects::ai\"],\n \"layers\": {\n \"\": { \"refs\": [\"ai/model\", \"ai/requirements\"], \"description\": \"the core model and its requirements — sourceless, so it adds no tables\" },\n \"db\": { \"refs\": [\"ai/db\"], \"description\": \"the concrete llm_call table\" }\n },\n \"generators\": [\n { \"name\": \"trace-helper\", \"anchor\": \"metaobjects::ai::LlmCallBase\" }\n ],\n \"runtime\": {\n \"typescript\": [\"@metaobjectsdev/runtime-ts\"]\n }\n}\n", + "iam": "{\n \"$comment\": \"Library manifest (FR-043 §4). Embedded beside the YAML in every port. Every fact here is RESOLVED by a test, never trusted: `packages` against the library loaded standalone, `layers[].refs` against the embedded set, `generators[].name` against the generator registry, `generators[].anchor` against the library's own nodes, and `name` against the last package segment.\",\n \"name\": \"iam\",\n \"kind\": \"feature\",\n \"stability\": \"preview\",\n \"since\": \"1.1.0\",\n \"description\": \"Users, nestable typed groups, roles as permission bundles, grants global or scoped to a group.\",\n \"useWhen\": \"the application has people who log in and things some of them may not do\",\n \"packages\": [\"metaobjects::iam\"],\n \"layers\": {\n \"\": { \"refs\": [\"iam/model\", \"iam/requirements\"], \"description\": \"the core model and its requirements — sourceless, so it adds no tables\" },\n \"db\": { \"refs\": [\"iam/db\"], \"description\": \"nine tables, iam_-prefixed, plus the lookup indexes the composite keys do not cover\" }\n },\n \"generators\": [],\n \"runtime\": {}\n}\n", }; diff --git a/server/typescript/packages/metadata/src/library/library-sources.ts b/server/typescript/packages/metadata/src/library/library-sources.ts index c19876989..a044567b7 100644 --- a/server/typescript/packages/metadata/src/library/library-sources.ts +++ b/server/typescript/packages/metadata/src/library/library-sources.ts @@ -11,20 +11,57 @@ import { fileURLToPath } from "node:url"; import { FileSource } from "../loader/sources/file-source.js"; import { InMemoryStringSource } from "../loader/meta-data-source.js"; import type { MetaDataSource } from "../loader/meta-data-source.js"; -import { EMBEDDED_LIBRARY } from "./embedded-library.generated.js"; - -// Package → ordered refs, derived from the generated embedded module so adding a -// library file (which regenerates EMBEDDED_LIBRARY) needs no edit here. -const REFS_BY_PACKAGE: Readonly> = (() => { - const map: Record = {}; - for (const ref of Object.keys(EMBEDDED_LIBRARY).sort()) { - const pkg = ref.split("/")[0]; - if (pkg === undefined || pkg === "") continue; - (map[pkg] ??= []).push(ref); +import { EMBEDDED_LIBRARY, EMBEDDED_LIBRARY_MANIFESTS } from "./embedded-library.generated.js"; + +/** One layer of a library, as its manifest declares it. */ +export interface LibraryLayer { + /** Refs (path under `library/` minus `.yaml`) this layer contributes, in order. */ + readonly refs: readonly string[]; + readonly description?: string; +} + +/** A library's `library.json`, parsed. Only the fields this module reads are typed; + * the catalog reads the rest off the same text. */ +export interface LibraryManifest { + readonly name: string; + readonly kind?: string; + readonly stability?: string; + readonly since?: string; + readonly description?: string; + readonly useWhen?: string; + readonly packages?: readonly string[]; + /** Layer token → layer. The CORE layer's token is the empty string. */ + readonly layers?: Readonly>; + readonly generators?: ReadonlyArray<{ readonly name: string; readonly anchor?: string }>; + readonly runtime?: Readonly>; +} + +const MANIFESTS: Readonly> = (() => { + const out: Record = {}; + for (const [name, text] of Object.entries(EMBEDDED_LIBRARY_MANIFESTS)) { + out[name] = JSON.parse(text) as LibraryManifest; } - return map; + return out; })(); +/** Every shipped library's parsed manifest, keyed by name. */ +export function libraryManifests(): Readonly> { + return MANIFESTS; +} + +/** + * Split a selection token into `[library, layer]` — `"iam"` → `["iam", ""]`, + * `"iam/db"` → `["iam", "db"]`. + * + * Path-like, so `libraries` stays `string[]` and no config schema moves. Only ONE + * separator is meaningful; anything after a second is part of the layer token, which + * keeps a typo failing loudly rather than resolving to a prefix. + */ +export function splitLayerToken(token: string): [string, string] { + const i = token.indexOf("/"); + return i === -1 ? [token, ""] : [token.slice(0, i), token.slice(i + 1)]; +} + /** * Locate the repo-root `library/` directory by walking up from this module's * location until a directory contains BOTH `library/` and `server/` (the two @@ -62,50 +99,163 @@ function getLibraryDir(): string | undefined { * available (Python's `project_config` draws the same line, in the same place). */ export function knownLibraryPackages(): string[] { - return Object.keys(REFS_BY_PACKAGE).sort(); + return Object.keys(MANIFESTS).sort(); +} + +/** + * Every package name a shipped library OWNS, across every library and layer. + * + * The provenance key for FR-043 §5.4 — object coverage activates on adopter-authored + * requirements only, and "adopter-authored" means "declared outside every library + * package". It reads the manifests rather than node source ids deliberately: `packages` + * is a manifest fact the standalone gate resolves against the library loaded alone, + * while a source id differs between the on-disk dev layout (an absolute path) and the + * embedded one (`library:.yaml`), so a rule keyed on that would hold here and stop + * holding in an installed build. + */ +/** + * The source id a library file loads under, in EVERY build — `library:iam/model.yaml`. + * + * Stable rather than path-derived so a library node's ADR-0009 provenance envelope reads + * the same from a checkout and from an installed package, carries no absolute path, and + * cannot be confused with an adopter file that happens to share a basename. The + * `library:` prefix is the discriminator {@link isLibraryFileId} reads. + */ +export function libraryFileId(ref: string): string { + return `${LIBRARY_FILE_ID_PREFIX}${ref}.yaml`; +} + +/** The prefix every library source id carries. */ +export const LIBRARY_FILE_ID_PREFIX = "library:"; + +/** True when a source id names a file a shipped library contributed. */ +export function isLibraryFileId(id: string): boolean { + return id.startsWith(LIBRARY_FILE_ID_PREFIX); +} + +export function libraryPackages(): ReadonlySet { + const out = new Set(); + for (const manifest of Object.values(MANIFESTS)) { + for (const pkg of manifest.packages ?? []) out.add(pkg); + } + return out; +} + +/** + * Every selection token this build accepts, sorted — `["ai", "ai/db", "iam", "iam/db"]`. + * + * What a config error message should print, so an adopter who typed `iam/database` is + * shown the layer they meant rather than only the library they got right. + */ +export function knownLibraryTokens(): string[] { + const out: string[] = []; + for (const [name, manifest] of Object.entries(MANIFESTS)) { + for (const layer of Object.keys(manifest.layers ?? { "": { refs: [] } })) { + out.push(layer === "" ? name : `${name}/${layer}`); + } + } + return out.sort(); } /** - * Returns a list of `MetaDataSource` instances for the requested library packages. + * `MetaDataSource` instances for the requested library selection. + * + * **Layer-granular.** A token is `` or `/`; the CORE layer is + * the bare name. This used to be package-granular — every ref under a library came back + * for a bare `"iam"` — which under the layered design would have handed an adopter the + * db and ui layers they did not ask for, and with them a migration proposing nine tables. + * + * **`"iam/db"` IMPLIES `"iam"`**, and the implication is not a convenience: a db layer is + * nothing but `overlay: true` redeclarations, and an overlay whose target was never + * declared is `ERR_OVERLAY_NO_TARGET`. Resolving the layer without its core would produce + * exactly that error, so implying it is the only coherent reading. + * + * Refs are de-duplicated and returned in a stable order — core first, then each requested + * layer in the manifest's own order — because an overlay must be parsed after its base + * even though ADR-0055 applies overlays in a deferred pass. * - * - Recognized packages: `"ai"` (others contribute no sources). - * - Per ref: if the on-disk `library/.yaml` exists, returns a `FileSource`; - * otherwise falls back to an `InMemoryStringSource` built from the embedded content. + * An unrecognised token contributes nothing and is skipped silently: that is right for a + * programmatic caller asking for something a given version may not ship. A name a HUMAN + * typed is a different case and is refused by the config readers, which call + * {@link knownLibraryTokens} to say what is available. * - * @param packages - Package names to include (e.g. `["ai"]`). + * @param selection - Tokens, e.g. `["iam", "iam/db"]`. */ -export function librarySources(packages: string[]): MetaDataSource[] { +export function librarySources(selection: string[]): MetaDataSource[] { + const refs: string[] = []; + const seen = new Set(); + + const add = (ref: string): void => { + if (seen.has(ref)) return; + seen.add(ref); + refs.push(ref); + }; + + // Core layers first, across every requested library, so a db layer named before its + // core in the config still parses after it. + // + // A token whose LAYER is unknown is dropped whole, not reduced to its core. The core is + // implied by a VALID layer token; implying it from an invalid one would answer a + // mistyped `iam/database` with an inert core and no tables — "I asked for the db layer + // and got nothing" with no diagnostic, which is the worst of the available outcomes. + const wanted = selection + .map(splitLayerToken) + .filter(([lib, layer]) => lib in MANIFESTS && (MANIFESTS[lib]!.layers ?? {})[layer] !== undefined); + for (const [lib] of wanted) { + for (const ref of MANIFESTS[lib]!.layers?.[""]?.refs ?? []) add(ref); + } + for (const [lib, layer] of wanted) { + if (layer === "") continue; + for (const ref of MANIFESTS[lib]!.layers?.[layer]?.refs ?? []) add(ref); + } + + return refs.map(libraryRefSource); +} + +/** + * One library ref as a source — on-disk first, embedded otherwise. + * + * Factored out of {@link librarySources} because `meta eject ` needs the TEXT + * of one ref and must resolve it exactly the way a load does: an adopter ejecting from a + * checkout must get the file they can see, and from an installed package the embedded + * copy, with no third rule to keep in step. + */ +export function libraryRefSource(ref: string): MetaDataSource { const dir = getLibraryDir(); - const out: MetaDataSource[] = []; - - for (const pkg of packages) { - const refs = REFS_BY_PACKAGE[pkg]; - if (refs === undefined) continue; // unknown package — no sources - - for (const ref of refs) { - if (dir !== undefined) { - const path = join(dir, `${ref}.yaml`); - if (existsSync(path)) { - out.push(new FileSource(path)); - continue; - } - } - const embedded = EMBEDDED_LIBRARY[ref]; - if (embedded !== undefined) { - out.push( - new InMemoryStringSource(embedded, { - id: `library:${ref}.yaml`, - format: "yaml", - }), - ); - } else { - throw new Error( - `library ref "${ref}" (package "${pkg}") has no on-disk file and no embedded entry — ` + - `the embedded library module is stale; run scripts/generate-embedded-library.ts`, - ); - } + if (dir !== undefined) { + const path = join(dir, `${ref}.yaml`); + if (existsSync(path)) { + // The SAME id the embedded branch below uses, deliberately. A `FileSource` + // defaults its id to the file's BASENAME, which would make a library node's error + // envelope read `model.yaml` in a checkout and `library:iam/model.yaml` in an + // installed build — and would collide outright with an adopter file of that name. + // One stable id makes the two builds report identically and gives anything asking + // "did a library declare this node" an unambiguous answer. + return new FileSource(path, { id: libraryFileId(ref) }); } } - return out; + const embedded = EMBEDDED_LIBRARY[ref]; + if (embedded === undefined) { + throw new Error( + `library ref "${ref}" has no on-disk file and no embedded entry — ` + + `the embedded library module is stale; run scripts/generate-embedded-library.ts`, + ); + } + return new InMemoryStringSource(embedded, { id: libraryFileId(ref), format: "yaml" }); +} + +/** Every ref one library contributes, core layer first — what `meta eject` copies. */ +export function libraryRefs(name: string): string[] { + const layers = MANIFESTS[name]?.layers ?? {}; + const refs: string[] = []; + const seen = new Set(); + for (const token of ["", ...Object.keys(layers).filter((k) => k !== "")]) { + for (const ref of layers[token]?.refs ?? []) { + if (seen.has(ref)) continue; + seen.add(ref); + refs.push(ref); + } + } + return refs; } diff --git a/server/typescript/packages/metadata/src/parser-core.ts b/server/typescript/packages/metadata/src/parser-core.ts index eeb93c67c..3844794d3 100644 --- a/server/typescript/packages/metadata/src/parser-core.ts +++ b/server/typescript/packages/metadata/src/parser-core.ts @@ -945,7 +945,18 @@ function parseNodeInto( // "merged"` envelope. Last-writer-wins is preserved for non-conflicting // cases (one side unset, same value, etc.) — those carry through to the // existing applyInlineAttrsAndUnknownKeys logic below. - if (fr5cActive && preMergeAttrSnapshot !== undefined) { + // + // FR-043 Amendment 2 — `overlay: true` LICENSES the override. The conflict error + // exists to catch two files that collided without knowing about each other; the flag + // is the author saying "I know about the other declaration and I mean to change it". + // The loader already treats the flag specially (find-or-throw versus create-or-find), + // so honouring it here makes it mean ONE thing rather than two. An unmarked + // redeclaration still merges and still errors, which is the case FR5c was written for. + if ( + fr5cActive && + preMergeAttrSnapshot !== undefined && + nodeData[RESERVED_KEY_OVERLAY] !== true + ) { detectAttrMergeConflicts( target, nodeData, @@ -1037,7 +1048,9 @@ function parseNodeInto( * non-empty value. If so, emit ERR_MERGE_CONFLICT with a `format: "merged"` * envelope naming both contributors. The merge itself proceeds (existing * last-writer-wins) so the loader sees one canonical tree; the error - * surfaces the conflict so a consumer can fix the metadata. */ + * surfaces the conflict so a consumer can fix the metadata. + * + * NOT called for an `overlay: true` declaration — see the call site. */ function detectAttrMergeConflicts( target: MetaData, nodeData: Record, diff --git a/server/typescript/packages/metadata/test/embedded-library.test.ts b/server/typescript/packages/metadata/test/embedded-library.test.ts index 88a8b4088..e196afafe 100644 --- a/server/typescript/packages/metadata/test/embedded-library.test.ts +++ b/server/typescript/packages/metadata/test/embedded-library.test.ts @@ -8,7 +8,8 @@ // the .yaml suffix). // 2. EXACT COVERAGE — the embedded map keys are exactly the canonical set // (no missing, no extra). -// 3. Content sanity — the known "ai/llm-call" entry contains expected text. +// 3. Content sanity — the known refs contain expected text, and every library +// directory contributes a manifest. import { describe, test, expect } from "bun:test"; import { readdirSync, readFileSync, existsSync } from "node:fs"; @@ -56,7 +57,9 @@ describe("EMBEDDED_LIBRARY — exact coverage", () => { }); describe("EMBEDDED_LIBRARY — content sanity", () => { - test("ai/llm-call entry contains the LlmCallBase definition", () => { - expect(EMBEDDED_LIBRARY["ai/llm-call"]).toContain("LlmCallBase"); + test("ai/model entry contains the LlmCallBase definition", () => { + // Was `ai/llm-call`, before FR-043 Amendment 1 split every library into a sourceless + // CORE layer and a db layer. The abstract envelope is core; the table is not. + expect(EMBEDDED_LIBRARY["ai/model"]).toContain("LlmCallBase"); }); }); diff --git a/server/typescript/packages/metadata/test/library-load.test.ts b/server/typescript/packages/metadata/test/library-load.test.ts index 95a7d4c20..207d80a3c 100644 --- a/server/typescript/packages/metadata/test/library-load.test.ts +++ b/server/typescript/packages/metadata/test/library-load.test.ts @@ -2,20 +2,139 @@ import { describe, test, expect } from "bun:test"; import { mkdtempSync, writeFileSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { librarySources } from "../src/library/library-sources.js"; +import { + librarySources, + knownLibraryPackages, + knownLibraryTokens, + libraryManifests, + splitLayerToken, +} from "../src/library/library-sources.js"; import { MetaDataLoader } from "../src/index.js"; -describe("librarySources", () => { - test("returns a source for the ai package whose content mentions LlmCallBase", async () => { +async function textOf(sources: readonly { read: () => Promise }[]): Promise { + return (await Promise.all(sources.map((s) => s.read()))).join("\n"); +} + +describe("librarySources — layer selection (FR-043 Amendment 1)", () => { + test("a bare library name resolves its CORE layer only", async () => { const sources = librarySources(["ai"]); - expect(sources.length).toBe(1); - // read() returns Promise per the MetaDataSource contract. - const text = await sources[0]!.read(); + const text = await textOf(sources); + expect(text).toContain("LlmCallBase"); + // The core layer is SOURCELESS — that is the whole point of the split. A bare name + // must not drag the db layer in, or opting into a design would propose tables. + expect(text).not.toContain("table: llm_call"); + }); + + test("a layer token adds that layer AND implies the core", async () => { + const text = await textOf(librarySources(["ai/db"])); + // The core comes too: a db layer is nothing but `overlay: true` redeclarations, and + // an overlay whose target was never declared is ERR_OVERLAY_NO_TARGET. Implying the + // core is the only coherent reading, not a convenience. expect(text).toContain("LlmCallBase"); + expect(text).toContain("table: llm_call"); + }); + + test("naming both the core and a layer resolves each ref ONCE", async () => { + const both = librarySources(["ai", "ai/db"]); + const layerOnly = librarySources(["ai/db"]); + expect(both.length).toBe(layerOnly.length); }); - test("unknown package yields no sources", () => { + test("the core is ordered BEFORE its layers, however the tokens are written", async () => { + const sources = librarySources(["ai/db", "ai"]); + const first = await sources[0]!.read(); + // ADR-0055 applies overlays in a deferred pass, so this is belt rather than brace — + // but a base parsed after its overlay is the shape #160's retired partition existed + // to work around, and there is no reason to reintroduce the ordering question. + expect(first).toContain("LlmCallBase"); + }); + + test("an unknown library, and an unknown LAYER of a known library, yield no sources", () => { expect(librarySources(["does-not-exist"]).length).toBe(0); + expect(librarySources(["ai/does-not-exist"]).length).toBe(0); + }); + + test("splitLayerToken splits on the FIRST separator only", () => { + expect(splitLayerToken("iam")).toEqual(["iam", ""]); + expect(splitLayerToken("iam/db")).toEqual(["iam", "db"]); + // Not a prefix match: a typo stays a typo rather than resolving to something. + expect(splitLayerToken("iam/db/extra")).toEqual(["iam", "db/extra"]); + }); +}); + +describe("library manifests", () => { + test("every shipped library has a manifest, and the names agree", () => { + const manifests = libraryManifests(); + expect(knownLibraryPackages()).toEqual(Object.keys(manifests).sort()); + for (const [name, m] of Object.entries(manifests)) { + // `name` is RESOLVED against the key, and both against the last package segment — + // a manifest that names itself differently from its directory would resolve + // layers under one name and packages under another. + expect(m.name, `${name}.name`).toBe(name); + for (const pkg of m.packages ?? []) { + expect(pkg.split("::").pop(), `${name} package ${pkg}`).toBe(name); + } + } + }); + + test("every layer's refs are really embedded", async () => { + for (const [name, m] of Object.entries(libraryManifests())) { + for (const [layer, spec] of Object.entries(m.layers ?? {})) { + const token = layer === "" ? name : `${name}/${layer}`; + const sources = librarySources([token]); + // A ref the manifest declares but the embed does not carry makes + // `librarySources` throw; getting here at all is the assertion. + expect(sources.length, `${token} resolves`).toBeGreaterThan(0); + expect(spec.refs.length, `${token} declares refs`).toBeGreaterThan(0); + } + } + }); + + test("knownLibraryTokens lists every library AND every layer", () => { + const tokens = knownLibraryTokens(); + // What a config error prints. Listing only library names would show an adopter who + // typed `iam/database` nothing about the layer they meant. + expect(tokens).toContain("ai"); + expect(tokens).toContain("ai/db"); + expect(tokens).toContain("iam"); + expect(tokens).toContain("iam/db"); + expect([...tokens].sort()).toEqual(tokens); + }); +}); + +describe("the CORE layer of every shipped library is INERT", () => { + test("no core layer declares a source, in any library", async () => { + // FR-043 §8 item 2b — the inertness promise, RESOLVED rather than trusted. + // + // A sourceless object generates nothing and migrates to nothing (migrate skips an + // object with no writable source; codegen emits no route, queries, hooks, grid or + // form for one, both citing #248). That is what makes `libraries: ["iam"]` add zero + // tables. A single `source.rdb` slipped into a core layer breaks it SILENTLY — the + // adopter's next `meta migrate` simply proposes a table. + for (const name of knownLibraryPackages()) { + const result = await new MetaDataLoader({ strict: true }).load(librarySources([name])); + expect(result.errors, `${name} core loads clean`).toEqual([]); + const sourced = result.root + .objects() + .filter((o) => o.children().some((c) => c.type === "source")) + .map((o) => o.name); + expect(sourced, `${name} core declares no source`).toEqual([]); + } + }); + + test("...and the db layer is what adds them", async () => { + // The inverse, so the test above cannot pass by the library shipping nothing at all. + const result = await new MetaDataLoader({ strict: true }).load(librarySources(["iam/db"])); + expect(result.errors).toEqual([]); + const sourced = result.root + .objects() + .filter((o) => o.children().some((c) => c.type === "source")) + .map((o) => o.name) + .sort(); + expect(sourced).toEqual([ + "Group", "GroupMember", "GroupMemberRole", "GroupType", + "Permission", "Role", "RolePermission", "User", "UserRole", + ]); }); }); diff --git a/server/typescript/packages/runtime-ts/test/llm-recorder-contract.test.ts b/server/typescript/packages/runtime-ts/test/llm-recorder-contract.test.ts index 5d1f8a3ac..0cddebad6 100644 --- a/server/typescript/packages/runtime-ts/test/llm-recorder-contract.test.ts +++ b/server/typescript/packages/runtime-ts/test/llm-recorder-contract.test.ts @@ -2,7 +2,7 @@ // effective field set. // // This is the regression guard for the headline bug — recordLlmCall once wrote a -// `voResponse` key that the shipped abstract base (`library/ai/llm-call.yaml`, +// `voResponse` key that the shipped abstract base (`library/ai/model.yaml`, // `metaobjects::ai::LlmCallBase`) does not declare, so the documented adoption // path (`extends metaobjects::ai::LlmCallBase` → generated `record`) // threw `Unknown field 'voResponse'`. By loading the REAL shipped base via the diff --git a/server/typescript/packages/sdk/src/collection.ts b/server/typescript/packages/sdk/src/collection.ts index b4faa9d0a..8958debfe 100644 --- a/server/typescript/packages/sdk/src/collection.ts +++ b/server/typescript/packages/sdk/src/collection.ts @@ -72,6 +72,10 @@ export interface Collection { * config's declaration order), each verified against `.metaobjects/deps.lock.json` * before this resolves. Empty for a project that declares none. */ readonly dependencies: readonly ResolvedDependency[]; + /** FR-043 — the shipped-library selection this project declares, as authored. + * Path-like tokens (`"iam"`, `"iam/db"`). Empty for a project that names none, + * which is every project that has not asked for one. */ + readonly libraries: readonly string[]; /** FR-023 — the `FileSource` id each file loads under, for the dependency * ARTIFACTS only (`dep:/`); own files are absent and keep the * default `basename(path)`. Threaded to `loadMemory` so every node an artifact @@ -249,6 +253,7 @@ export async function resolveCollection( let scopeSpec: Config["scope"]; let migrateSpec: string[] | undefined; let dependencySpecs: readonly DependencySpec[] = []; + let libraries: readonly string[] = []; if (hasConfig) { // No try/catch here: a config.json that EXISTS but fails to load @@ -262,6 +267,30 @@ export async function resolveCollection( scopeSpec = cfg.scope; migrateSpec = cfg.migrate?.scope; dependencySpecs = cfg.dependencies; + libraries = cfg.libraries; + // An unknown token is a hard config error naming the valid ones, while + // `librarySources` keeps skipping one silently for a programmatic caller. The two + // are deliberately different: an API caller asking for a library this version does + // not ship should still be able to load its own metadata, but a token a HUMAN typed + // into a config file is a mistake worth failing on — skipped, it resurfaces later as + // ERR_UNRESOLVED_SUPER pointing at the adopter's own metadata, which is the wrong + // place to send someone looking. + // + // The available list is TOKENS, not library names, so an adopter who typed + // `iam/database` is shown `iam/db` rather than only the half they got right. + if (libraries.length > 0) { + const { knownLibraryTokens } = await import("@metaobjectsdev/metadata/library"); + const available = knownLibraryTokens(); + const unknown = libraries.filter((n) => !available.includes(n)); + if (unknown.length > 0) { + throw new ParseError( + `.metaobjects/config.json in ${configDir}: 'libraries' names unknown ` + + `librar${unknown.length === 1 ? "y" : "ies"} ${JSON.stringify(unknown)}; ` + + `available: ${JSON.stringify(available)}.`, + { code: "ERR_UNKNOWN_LIBRARY", source: { format: "code", caller: "resolveCollection" } }, + ); + } + } } // Only the DEFAULT is allowed to be absent — an explicitly declared source @@ -346,6 +375,7 @@ export async function resolveCollection( ownFiles, sources: [...dependencySources, ...ownSources], dependencies, + libraries, fileIds: new Map(dependencies.map((d) => [d.artifactPath, d.sourceId])), importedPackages: [...importedPackages].sort(), importedNodes, diff --git a/server/typescript/packages/sdk/src/config.ts b/server/typescript/packages/sdk/src/config.ts index 4c70a24d9..7b312bdf2 100644 --- a/server/typescript/packages/sdk/src/config.ts +++ b/server/typescript/packages/sdk/src/config.ts @@ -149,6 +149,22 @@ export const ConfigSchema = z.object({ .refine((a) => new Set(a.map((d) => d.name)).size === a.length, { message: "dependencies: names must be unique", }), + /** + * MetaObjects-shipped libraries this project opts into (FR-043). + * + * Path-like tokens: `"iam"` is the core layer, `"iam/db"` adds its persistence layer + * and IMPLIES the core. A library's CORE layer declares no `source.rdb`, so naming one + * adds zero tables and zero generated code — the design is present and resolvable, and + * nothing else happens until a layer is named. + * + * Declared HERE, beside `dependencies`, and no longer in `metaobjects.config.ts` + * (FR-043 Amendment 1, §12 Q4). Two reasons: which designs a project adopts is a fact + * about the PROJECT, not about how one port generates code from it, and this file is + * the port-neutral one every port already reads. Moved outright with no dual-read, + * because a sweep of the estate found zero configs using the old key — there is no + * installed base to be compatible with. + */ + libraries: z.array(z.string()).default([]), /** Output filter applied across every command — see `./scope.ts`. Absent * means "everything" (no filtering), matching `Scope`'s own contract. */ scope: ScopeSchema.optional(), diff --git a/server/typescript/packages/sdk/src/memory.ts b/server/typescript/packages/sdk/src/memory.ts index b38c7ede9..f0d5dd174 100644 --- a/server/typescript/packages/sdk/src/memory.ts +++ b/server/typescript/packages/sdk/src/memory.ts @@ -6,10 +6,18 @@ import { packageOfResolutionKey, ParseError, TYPE_OBJECT, + type ErrorSource, type MetaDataTypeProvider, type MetaRoot, } from "@metaobjectsdev/metadata"; import { FileSource } from "@metaobjectsdev/metadata/core"; +// FR-043 — a node-only subpath (it reaches the filesystem), which is why it is imported +// separately from the browser-safe barrel above. Static here rather than the dynamic +// import the library SOURCES use: this module already resolves it on every load that +// names a library, and the guard runs after the load either way. +import { + isLibraryFileId, libraryManifests, splitLayerToken, +} from "@metaobjectsdev/metadata/library"; import { resolveCollection } from "./collection.js"; /** @@ -223,10 +231,98 @@ export async function loadMemory( // overlay flag, so this walk cannot tell an overlay from a new declaration. // Only an UNFLAGGED new declaration survives to here. refuseUnownedPackages(result.root, importedPackages, importedNodes); + // FR-043 §3.4 / §3.5 — the same rule for a shipped LIBRARY's package, where the two + // ways to get it wrong are opposite: a node the library also declares (an ejected + // copy, still opted in) and one it does not (a new node in someone else's package). + refuseLibraryPackageMisuse(result.root, options?.libraries); return result.root; } +/** Every source file that contributed to a node, across the envelope variants that + * name files at all (`code` and `database` name none). */ +function contributingFiles(source: ErrorSource): readonly string[] { + return "files" in source ? source.files : []; +} + +/** + * FR-043 — refuse the two ways an adopter's own file lands in a shipped library's + * package while that library is opted in. + * + * Both are SILENT today, and they fail in opposite directions: + * + * **The ejected copy.** `meta eject iam` hands you the library's YAML to own, and the + * next step it prints is to remove `iam` from `libraries`. Skip that and both trees + * load: the copy merges into the shipped node, so ADDITIONS take and DELETIONS do not + * — you delete a field from your copy and it is still there, because the library still + * declares it. Nothing says so. That is `ERR_LIBRARY_PACKAGE_COLLISION`. + * + * **The new node.** Declaring something of your own into `metaobjects::iam` makes the + * library's package yours to break: the next release of the library may ship a node of + * that name and merge into it. Own a package and `extends`, or say `overlay: true` and + * mean it. + * + * An `overlay: true` redeclaration is the documented adaptation door (§3.4) and is + * deliberately untouched — `isMerge` is the loader's own record that the flag was + * honoured, so this cannot mistake the two. + * + * No-op for a project that opts into no library, which is every project today. + */ +function refuseLibraryPackageMisuse( + root: MetaRoot, + selection: readonly string[] | undefined, +): void { + if (selection === undefined || selection.length === 0) return; + const manifests = libraryManifests(); + const owner = new Map(); + for (const token of selection) { + const library = splitLayerToken(token)[0]; + for (const pkg of manifests[library]?.packages ?? []) owner.set(pkg, library); + } + if (owner.size === 0) return; + + // ADR-0039 SANCTIONED own-accessor case: a root-level scan, exactly as + // `refuseUnownedPackages` does — `MetaRoot` has no super, and the question is + // "what did this tree declare at the top level". + for (const node of root.ownChildren()) { + const key = node.resolutionKey(); + const library = owner.get(packageOfResolutionKey(key)); + if (library === undefined) continue; + + const files = contributingFiles(node.source); + if (!files.some((f) => !isLibraryFileId(f))) continue; // library's own, untouched + if (node.isMerge) continue; // a marked overlay — the door + + const collision = files.some(isLibraryFileId); + throw new ParseError( + collision + ? `"${key}" is declared by your own metadata AND by the shipped library ` + + `"${library}", which this project opts into. The two merge silently: ` + + `additions in your copy take effect and DELETIONS do not, because the library ` + + `still declares what you removed.` + : `"${key}" is declared here, but the package "${packageOfResolutionKey(key)}" ` + + `belongs to the shipped library "${library}", which this project opts into. ` + + `The next release of that library may ship a node of this name and merge into ` + + `yours.`, + { + code: collision ? "ERR_LIBRARY_PACKAGE_COLLISION" : "ERR_LIBRARY_PACKAGE_NOT_OWNED", + source: node.source, + node: { type: node.type, subtype: node.subType, name: node.name, fqn: key }, + suggestions: collision + ? [ + `Remove "${library}" from 'libraries' in .metaobjects/config.json — you own the metadata now, which is what 'meta eject ${library}' told you to do.`, + `Or delete your copy and keep tracking the library, amending it with 'overlay: true' on the nodes you want to change.`, + ] + : [ + `Declare it in a package this project owns, and 'extends' the library's node if it needs its shape.`, + `If it was meant to AMEND a library node, give it that node's name and 'overlay: true'.`, + `If you want to own this design outright, run 'meta eject ${library}' and remove "${library}" from 'libraries'.`, + ], + }, + ); + } +} + /** * FR-023 §11.5 — a consumer may not declare a NEW top-level node into a package * one of its dependencies owns. diff --git a/server/typescript/packages/sdk/test/library-package-guard.test.ts b/server/typescript/packages/sdk/test/library-package-guard.test.ts new file mode 100644 index 000000000..b5a342693 --- /dev/null +++ b/server/typescript/packages/sdk/test/library-package-guard.test.ts @@ -0,0 +1,128 @@ +// FR-043 §3.4 / §3.5 — an adopter's own file in a shipped library's package, while that +// library is opted in. +// +// Two failures, opposite in shape and both SILENT before this guard: +// +// the EJECTED COPY — `meta eject iam` hands you the metadata and tells you to drop +// `iam` from `libraries`. Skip that and both trees load and merge: additions take, +// DELETIONS do not, because the library still declares what you removed. +// the NEW NODE — something of your own declared into `metaobjects::iam`, where the +// next release of the library may ship a node of that name and merge into it. +// +// The `overlay: true` door stays open: that is the documented way to amend a shipped +// node while tracking upstream, and the loader's own `isMerge` is what tells the two +// apart — a distinction no comparison of the merged trees could make. +import { describe, test, expect } from "bun:test"; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadMemory } from "../src/memory.js"; + +/** An adopter file, loaded with `iam` opted in. Returns the thrown error, or undefined. */ +async function loadWith(yaml: string, libraries: string[] = ["iam"]): Promise { + const dir = mkdtempSync(join(tmpdir(), "lib-guard-")); + try { + mkdirSync(join(dir, "metaobjects")); + writeFileSync(join(dir, "metaobjects", "mine.yaml"), yaml); + await loadMemory(dir, { strict: true, libraries }); + return undefined; + } catch (err) { + return err as Error; + } finally { + rmSync(dir, { recursive: true, force: true }); + } +} + +const COPY_OF_A_SHIPPED_NODE = ` +metadata: + package: metaobjects::iam + children: + - object.entity: + name: User + children: + - field.string: { name: nickname } +`; + +const OVERLAY_OF_A_SHIPPED_NODE = ` +metadata: + package: metaobjects::iam + children: + - object.entity: + name: User + overlay: true + children: + - field.string: { name: nickname } +`; + +const A_NEW_NODE_IN_THE_LIBRARYS_PACKAGE = ` +metadata: + package: metaobjects::iam + children: + - object.entity: + name: ApiKey + children: + - field.uuid: { name: id } + - identity.primary: { name: pk, fields: [id] } +`; + +const MY_OWN_PACKAGE = ` +metadata: + package: acme::app + children: + - object.entity: + name: Account + extends: metaobjects::iam::User + children: + - identity.primary: { name: pk, fields: [id] } +`; + +describe("a shipped library's package, while it is opted in", () => { + test("an ejected copy that is still opted in is refused, by name", async () => { + const err = await loadWith(COPY_OF_A_SHIPPED_NODE); + expect(err).toBeDefined(); + expect((err as { code?: string }).code).toBe("ERR_LIBRARY_PACKAGE_COLLISION"); + // The message has to say WHY silence would be worse: the merge is not a no-op, it + // is asymmetric. + expect(err!.message).toContain("metaobjects::iam::User"); + expect(err!.message).toContain("DELETIONS"); + // And the fix is the step `meta eject` already printed. + expect((err as { suggestions?: string[] }).suggestions?.[0]) + .toContain("from 'libraries'"); + }); + + test("a NEW node in the library's package is refused as not yours to declare", async () => { + const err = await loadWith(A_NEW_NODE_IN_THE_LIBRARYS_PACKAGE); + expect(err).toBeDefined(); + expect((err as { code?: string }).code).toBe("ERR_LIBRARY_PACKAGE_NOT_OWNED"); + expect(err!.message).toContain("metaobjects::iam::ApiKey"); + }); + + test("an `overlay: true` amendment is the documented door and stays open", async () => { + expect(await loadWith(OVERLAY_OF_A_SHIPPED_NODE)).toBeUndefined(); + }); + + test("your own package extending a library node is untouched", async () => { + expect(await loadWith(MY_OWN_PACKAGE)).toBeUndefined(); + }); + + test("with the library NOT opted in, the guard says nothing", async () => { + // The copy is then just your metadata — which is exactly the state `meta eject` + // leaves you in once you remove the library from `libraries`. Refusing it there + // would make the ejection door unusable. + expect(await loadWith(COPY_OF_A_SHIPPED_NODE, [])).toBeUndefined(); + }); + + test("the library's own layers do not trip it", async () => { + // `iam/db` is nothing but `overlay: true` redeclarations of `iam`'s own nodes, from + // library files. A guard keyed on "two files contributed" would fire on every one. + const dir = mkdtempSync(join(tmpdir(), "lib-guard-")); + try { + mkdirSync(join(dir, "metaobjects")); + writeFileSync(join(dir, "metaobjects", "mine.yaml"), MY_OWN_PACKAGE); + const root = await loadMemory(dir, { strict: true, libraries: ["iam", "iam/db"] }); + expect(root.children().some((c) => c.name === "User")).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/spec/decisions/ADR-0034-codegen-scaffold-and-own.md b/spec/decisions/ADR-0034-codegen-scaffold-and-own.md index 98366d216..c5f24d438 100644 --- a/spec/decisions/ADR-0034-codegen-scaffold-and-own.md +++ b/spec/decisions/ADR-0034-codegen-scaffold-and-own.md @@ -103,3 +103,52 @@ component library — code you copy into your project and own." - Open: whether to hard-remove the package generator export or deprecate it; the exact compiled-port mechanism per language; whether templates live in-package as assets or in a dedicated scaffold package. Tracked in the design doc. + +## Amendments + +### Amendment 1 (FR-040, 2026-08) — `meta eject` is the copy operation, generalised + +Decision 2 left the copy to "plain file operations" by a human or Claude, with +`meta init` doing an eager copy of a chosen few. FR-040 made the copy a first-class +command: `meta eject ` copies ANY reference template, from any package, at any +time after `init`, reporting the import line to wire and the packages the copied file +needs. The doctrine is unchanged — the adopter owns the file, and `meta gen` runs their +copy — but "adding a starting point is adding a documented file" now also means it is +reachable by name. + +### Amendment 2 (2026-09-13) — codegen is OPT-IN; `init` scaffolds an empty selection + +**Superseded:** Decision 2's "`meta init` scaffolds a sensible default generator set + +local-import config for a running start", and the consequence "First-run still works +(init scaffolds defaults), so the pivot costs no quick-start." + +**Replaced by:** `meta init` scaffolds the **layout and an empty documented selection** +— `codegen/generators/` (empty), `tsconfig.codegen.json`, and a config whose +`generators: []` carries a comment pointing at the catalog. `meta eject ...` +(Amendment 1, now taking many names) is the copy door. The catalog is the composed +stable-name registry behind `meta gen --list`, with `--probe` reporting how many files +each generator would emit for the adopter's own model. C# and Python dropped their +default suites in the same change; Java never had one. + +**Why.** A default suite is a selection decision hard-coded into the CLI, and Decision 2 +already ruled that such decisions "cannot be captured in CLI flags" because they are +judgment over infinite per-project variation. It then made one anyway, for the +first-run case. The consequence was measurable: a new project got five generators, five +dependencies declared for them, and a throwing `src/db.ts` stub to make one of those +generators' emitted import resolve — none of it chosen, and the count was growing. +Deciding what an application needs belongs to whoever is building it; increasingly that +is an LLM in the repo, which is well able to make the call given a truthful catalog and +is badly served by a default that pre-empts it. + +**Everything else in ADR-0034 stands** — the engine/template split, scaffold-and-own +ownership, no interview, and "adding a starting point is adding a documented file, +never a CLI change". + +**Compatibility.** This is a PATCH. No existing project changes by one byte: an adopter +already has their owned copies on disk and their selection committed in their own +config, `meta gen` keeps running exactly that list, and re-running `init` never +clobbers a file that exists. `docs/compatibility-policy.md` is narrowed in the same +change — the scaffold-and-own promise is the LAYOUT and the INTERFACES, not which +generators a fresh scaffold happens to wire. + +Design: `docs/superpowers/specs/2026-09-12-opt-in-codegen-and-generator-catalog-design.md`. diff --git a/spec/roadmap.md b/spec/roadmap.md index aa28cd5c2..7055af369 100644 --- a/spec/roadmap.md +++ b/spec/roadmap.md @@ -53,6 +53,7 @@ under **Shipped**; planned FRs under **Planned** + the **Release plan**. ✅ shi | FR-038 | Requirement-derived test stubs (inverts `@verifiedBy`) | 🟢 the vocabulary retirement shipped `0.24.0`; the stub generator remains. Generate the test from the requirement so the link is structural, not a name the author picks — an audit of one 19-name ledger found 4 names that did not verify their claim. **The `@verifiedBy` retirement was breaking and rode the SAME pre-1.0 MINOR as FR-037's R1/R2, not a second one — SHIPPED in `0.24.0`** (`@verifiedBy`, `@supersededBy`, and `@status: abandoned|superseded` all deregistered). The stub generator is additive and remains → 1.1. Design: `docs/superpowers/specs/2026-08-15-fr-038-requirement-derived-test-stubs-design.md` | 1.0 · 1.1 | — | | FR-041 | Public A/B drift benchmark — coding agents with vs without MetaObjects, pre-registered, friction-first | 📋 **design settled 2026-09-12, unbuilt** — the "proving the value" work; it is what licenses the claims FR-042 §4 withholds. Revised after an adversarial two-reviewer design review (spec §14): the scored task set is now held out from the friction pass (the draft tuned Arm B on the tasks it would later be scored on, with no symmetric loop for the control), Arm A is derived from Arm B's generated output so the seeds differ only in the model and the gate, `n` is calibrated from a pilot instead of asserted (one identical config measured 25/51/28 turns at temperature 0 — 41% CV), H2 is time-to-**correct** rather than time-to-done, the primary is analysed intention-to-treat so an arm cannot win by not finishing, and escaped defects are reported split by whether `meta verify` already covers the invariant class (it covers four of five, so H1 is partly definitional for those). First deliverable is the Phase 0b friction log. Design: `docs/superpowers/specs/2026-09-11-fr-041-drift-ab-benchmark-design.md` | 1.x | — | | FR-042 | First-touch positioning — one typed model, two verbs (README, llms, sites) | 🚧 in progress — pitch **locked** 2026-09-12 (two verbs: Generate + Verify; requirements fold into Verify; H1 model-first). Implementing across the four first-touch surfaces. Design: `docs/superpowers/specs/2026-09-11-fr-042-first-touch-positioning-design.md` | — | — | +| FR-043 | **Libraries** — reusable declared design: model metadata + the requirements that make it checkable (+ an implied generator selection), opted into by name | 🟢 **design approved 2026-09-13, un-deferred** — proposed as a SIXTH pillar. Not greenfield: `library/ai/llm-call.yaml` already ships one (opt-in via the loader's `libraries: ["ai"]`, embedded per port under an `embedded-library drift` gate, with `trace-helper` codegen beside it). Generalises it — requirements as a component, discovery through the codegen catalog's `kind: "library"`, declared package→generator coupling replacing `trace-helper`'s hard-coded `LlmCallBase`, `overlay: true` as the adaptation door and `meta eject` for repackaging. **No new metamodel vocabulary; `metamodelVersion` does not move.** Second library `iam` (users, typed nestable groups, roles as permission bundles, global and group-scoped grants) ships `stability: preview`; its model is verified to load clean under strict. Phase 2: third-party authoring over FR-023's deferred transports. Design: `docs/superpowers/specs/2026-09-13-fr-043-feature-and-nfr-packages-design.md` | 1.1 | — | _(FR-001 was the original metamodel foundation — pre-dates the FR-numbered tracking.)_ _(FR-032 was developed under the working number "FR-026" — see commit history; renumbered to avoid the FR-026=Forms collision. Design: `docs/superpowers/specs/2026-06-13-fr-032-canonical-fqn-refs-design.md`, ADR-0032.)_