From 54211826cb4a410657803edd0b15d2fc74b95e29 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 22:32:19 +0000 Subject: [PATCH 01/17] fix(canon): stop carrying one translation element onto several texts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every in-place edit of a page was refused, while a full rewrite worked: refusing to write unit …: 1 element id(s) are used more than once c540fbf0-… held by [Texts$Translation ×8] `GRANT VIEW ON PAGE` failed, `ALTER PAGE … INSERT` failed, and `CREATE OR REPLACE PAGE` did not — it rewrites the unit rather than patching it. So the page looked correct and only UPDATES were blocked, which is why it took enabling a second language to surface at all. CarryTranslations pairs a rebuilt text to its stored translations by SOURCE STRING when the two documents' text paths differ, and mergeText appended the stored element verbatim — deliberately, because keeping the stored $ID is what lets no-op elision fire. When several rebuilt texts share one source string (eight copies of the literal '{1}' on a page is entirely ordinary) they all resolve to the same stored set, and every one got the same element, id included. The first use keeps the stored id; each further copy gets a deterministic derived one. Derived rather than random so the same inputs give the same bytes, and the visit order is sorted rather than map order — otherwise which text keeps the stored id varies per run and the document churns. Re-identifying a copy is safe here in a way that deduplicating ids in general is not, and that distinction is the argument for doing it at all. An $ID is a pointer target, and rewriting one means finding every reference to it (ADR-0008) — which is exactly why the write-time guard refuses rather than repairs. Nothing references a Texts$Translation: it is a leaf child of a Texts$Text with four keys and no identity anything resolves by, so there are no references to miss. The guard in duplicates.go recorded that the cause of the reported case could not be established — two explanations proposed, both withdrawn. This is it. That comment is corrected, and the guard stays: it is cheap, it is the only thing between a write and an unopenable project, and nothing says Texts$Translation was the only way to get here. Verified end to end on a real 11.13 project with de_DE enabled and three widgets sharing a caption. Pre-fix: one id used 3x, and the next ALTER PAGE is refused with the reporting project's message verbatim. Fixed: 27 distinct ids for 27 elements, the ALTER PAGE succeeds, and mx check is 0 errors. Three controls, because the fix trades against the property the verbatim append existed for. The German translation still arrives, so this is not a "fix" that stopped carrying anything. A second identical run still reports `Unchanged page` with the same sha and mtime, so elision still fires. And stubbing reuseSafeID reproduces the duplicate. Reported as CapTrackV2 FINDINGS §30 and §17 (one root cause). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../skills/fix-issue/findings/modelsdk.jsonl | 1 + modelsdk/canon/duplicates.go | 25 ++- modelsdk/canon/translations.go | 125 ++++++++++++- .../canon/translations_duplicate_id_test.go | 177 ++++++++++++++++++ 4 files changed, 317 insertions(+), 11 deletions(-) create mode 100644 modelsdk/canon/translations_duplicate_id_test.go diff --git a/.claude/skills/fix-issue/findings/modelsdk.jsonl b/.claude/skills/fix-issue/findings/modelsdk.jsonl index f35fd504f..55a49ec6e 100644 --- a/.claude/skills/fix-issue/findings/modelsdk.jsonl +++ b/.claude/skills/fix-issue/findings/modelsdk.jsonl @@ -14,3 +14,4 @@ {"area": "modelsdk/widgets", "date": "2026-08-24", "symptom": "**CE0463 on every page carrying a File Uploader**, whatever is on it — an action-free one with nothing but its `datasource:` fails identically. The widget the #956 action-slot work was *about*, and the one it never exercised", "cause": "**Two independent causes, and each alone still fails.** (A) The widget XML's `defaultType` — the KIND of a default, `defaultType=\"CallNanoflow\"` on the create* actions, `defaultType=\"Association\"` on the datasources — was never parsed, so all six landed as `DefaultType: \"None\"`. It is part of the widget DEFINITION, exactly like the stale `onChange` of #716. (B) `uploadMode`'s declared default never reached the visibility evaluator, because a primitive mapping carries the XML default in `PropertyMapping.Value` while `widgetValueMap` read only `.Default` (populated for selections alone). With the condition unknown, `hiddenUnnamedProperties` correctly refuses to guess — so `associatedImages` was not pruned, the one `datasource:` clause fanned out into BOTH datasource properties, and the pruned one carried a value. File Uploader is the only widget in a stock 11.13.0 app that declares any `defaultType` (6 properties; every other bundled widget declares 0), which is why one widget was unwritable while the rest were fine", "file": "(A) `modelsdk/widgets/mpk/mpk.go` + `sdk/widgets/mpk/mpk.go` (`PropertyDef.DefaultType`, `xml:\"defaultType,attr\"`), `modelsdk/widgets/augment.go` + `sdk/widgets/augment.go` (`defaultTypeOf`); (B) `mdl/executor/validate_widgets.go` (`widgetValueMap`)", "insight": "**`mx update-widgets` on a COPY is the oracle when there is no Studio Pro** — and when one IS reachable, `ped_check_errors` over the written documents is the real verdict, with the same page written by the pre-fix binary as the control (it reports the CE0463 text and a `/layoutCall/arguments/0/widgets/0/widgets/0` location; the fixed 17 documents report \"No errors found.\"). `pg_read_page` then shows the editor's own view — on the pre-fix page it lists BOTH `associatedFiles` and `associatedImages` under `uploadMode: \"files\"`, which is the fanout, and on the fixed one only the active half. It is Studio Pro's own normalizer: run it, confirm it clears the error, then diff its output against yours — a 2,650-path document diff whose *only* differences are the cause. Diff by property NAME, never by index: the two dumps order top-level keys differently and a positional flatten reports ~60 spurious differences that hide the 6 real ones. Widget properties are keyed by `TypePointer` into `Type.ObjectType.PropertyTypes`, so resolve the pointer to its `PropertyKey` first or the diff is unreadable. Two more findings from the same measurement, both still open: mxcli writes widget-action `ParameterMappings` with list marker **3** where Studio Pro writes **2** (and omits the sibling `Variable`/`OutputMappings` keys) — accepted by both mxbuild 11.13 **and Studio Pro**, which resolves every mapping to the right parameter (checked live over PED), so it is a diff-churn item rather than a correctness one; and the visibility extractor misses a `hidePropertiesIn` call inside a **ternary branch** (`cond ? hide([…]) : hide([…])`), as opposed to the ternary *array element* #233 fixed — so `createImageAction` carries no uploadMode rule, MDL-WIDGET10 cannot warn, and wiring it in files mode is a CE0463 the script author gets no help with. Repro `mdl-examples/bug-tests/956-fileuploader-six-action-slots.mdl`. Issue #956", "refs": ["#233", "#716", "#956"], "ce": ["CE0463"]} {"area": "modelsdk/meta", "date": "2026-08-28", "symptom": "`GRANT` on an entity that extends **System.Image** fails the build with **CE1613** \"The selected association 'System.Thumbnail_Image' no longer exists\" at the access rule — and `describe entity` renders that rule without any broken member, so nothing inside mxcli shows it", "cause": "The virtual System module carries entities that exist only in the **runtime's** metamodel, not in the System module Studio Pro shows. They arrived with the block marked \"extracted from MDP (Phase 3)\". `Thumbnail_Image` is owned by BOTH ends, so System.Image counted as an owner through the CHILD side and every specialization inherited an access-rule member for it", "file": "`modelsdk/meta/system_module.go` (`RuntimeOnly`, `ModelerSystemEntities`, `ModelerSystemAssociations`), consumed by `mdl/backend/modelsdk/system_module_read.go`; the inheritance walk that reads it is `mdl/executor/cmd_security_write.go` (`inheritedAssociations`)", "insight": "**The legacy engine is a free control here**: its System list (`sdk/mpr/system_module.go`) never had the MDP block, so the same script under `--engine legacy` writes a clean rule — which pins the defect to the data rather than to the GRANT path reading it. Establish what the modeler actually has by MEASURING, not by reasoning about names: one microflow per candidate retrieving from `System.`, then one `mx check`. All eleven MDP rows came back CE1613 while the controls (Image, FileDocument, Session) came back clean, so the probe discriminates. Flag rather than delete — the runtime rows were harvested deliberately and a storage backend speaking to the runtime metamodel wants them — and DERIVE the association filter from the entity flag (an association is runtime-only exactly when one of its ends is), because two hand-kept lists drift and the one that falls behind writes a dangling by-name reference. Unrelated pre-existing gap found alongside: `extends ` is never reference-validated, for System or for a same-module name.", "ce": ["CE1613"]} {"area": "modelsdk/canon", "date": "2026-08-28", "symptom": "`mx check` fails with **`InvalidOperationException: … Duplicate Guid in unit page 'M.P'. Object types: Translation, Translation`** and the project will not LOAD at all — and every later edit of that page reports the same thing", "cause": "Two elements in one unit share an `$ID`. \"Guid\" in Mendix's message is the element `$ID`, not a `GUID` property — a `Texts$Translation` stores exactly four keys (`$ID`, `$Type`, `LanguageCode`, `Text`) and has no GUID of its own", "file": "`modelsdk/canon/duplicates.go` (`DuplicateElementIDs`, `DuplicateElementIDError`), wired at all four write choke points: `modelsdk/mpr/writer_core.go` (`insertUnit`, `updateUnit`) and `sdk/mpr/writer_units.go` (same two)", "insight": "**A guard is worth shipping without a root cause.** The reporter lost ~1h to one of these, proposed two causes, tested both, and withdrew both (ako/mxcli-captrack #2) — the trigger is still unknown. Refusing at the write turns an unopenable project into one message, and breaks the thing that made it expensive: the corruption is **STICKY**, so once a unit carries duplicate ids every later edit inherits them and each subsequent write looks like the culprit. Restore before re-testing, or the second experiment measures the first one's damage. Check BEFORE `canon.Reconcile`, not after — an elided write is not a safe one, only one that did not happen this time. Two controls are mandatory and both are cheap: **pointers are not duplicates** (an element's id is referenced by primitive properties like `ParentPointer` all over a normal document — counting those refuses every write), and a **normal translated page** with two translations differing only in id must still be accepted. False-positive control run at scale: **0 flagged out of 33,645 units across 90 projects**. Do NOT \"repair\" by deduplicating — an `$ID` is a pointer target, so choosing which element keeps its identity silently re-points references (ADR-0008).", "refs": ["#2"]} +{"area": "modelsdk", "date": "2026-09-02", "symptom": "Every in-place edit of a page is refused with `refusing to write unit \u2026: 1 element id(s) are used more than once \u2026 held by [Texts$Translation \u00d78]` \u2014 `GRANT VIEW ON PAGE`, `ALTER PAGE \u2026 INSERT` \u2014 while a full `CREATE OR REPLACE PAGE` still works, so the page looks correct and only UPDATES are blocked. Surfaces after a second language is enabled.", "cause": "`canon.CarryTranslations` pairs a rebuilt text to its stored translations BY SOURCE STRING when the two documents' text paths differ, and `mergeText` appended the stored `Texts$Translation` element **verbatim** \u2014 deliberately, because keeping the stored `$ID` is what lets no-op elision fire. When several rebuilt texts share one source string (eight copies of the literal `'{1}'` on a page is ordinary), all of them resolve to the SAME stored set and every one got the same element, id included. `reuseSafeID` now gives the first use the stored id and derives a fresh deterministic one (SHA-256 of stored id + containment path + language) for each further copy; the visit order is sorted rather than map order, or which text keeps the stored id would vary per run and the document would churn.", "file": "`modelsdk/canon/translations.go` (`reuseSafeID`, `derivedID`, `elementIDs`, `sortedPaths`, `mergeText`), `modelsdk/canon/duplicates.go` (comment corrected \u2014 it recorded the cause as unestablished)", "insight": "**Re-identifying a copy is safe here in a way that deduplicating ids in general is not, and that distinction is the whole argument.** An `$ID` is a pointer target and rewriting one means finding every reference (ADR-0008) \u2014 which is exactly why `duplicates.go` refuses rather than repairs. Nothing references a `Texts$Translation`: it is a leaf child of a `Texts$Text` with four keys and no identity anything resolves by, so there are no references to miss. Only the COPIES are re-identified; the first use keeps the stored id, so an unchanged document still compares equal. **Verify elision explicitly after touching this** \u2014 the fix trades against the exact property the verbatim append existed for: measured, a second identical run still reports `Unchanged page` with the same sha and mtime. Controls, end-to-end on a real 11.13 project with de_DE enabled and three widgets sharing a caption: the pre-fix binary writes one id used 3\u00d7 and the next `ALTER PAGE` is refused with the reporter's message verbatim; the fixed binary writes 27 distinct ids for 27 elements, the `ALTER PAGE` succeeds, the German translation survives (the control against a 'fix' that just stops carrying), and `mx check` is 0 errors. Reported as CapTrackV2 FINDINGS \u00a730/\u00a717."} diff --git a/modelsdk/canon/duplicates.go b/modelsdk/canon/duplicates.go index 949e986c3..54c480db8 100644 --- a/modelsdk/canon/duplicates.go +++ b/modelsdk/canon/duplicates.go @@ -21,13 +21,24 @@ import ( // the identity Mendix is complaining about is the 16-byte $ID. Verified against // stored translations on 11.13. // -// This is a WRITE-TIME guard with no root cause behind it, and that is -// deliberate. The reporter (ako/mxcli-captrack #2) lost about an hour to one of -// these and could not establish what produced it: two candidate explanations -// were proposed during the build, both were tested, and both were withdrawn. -// What was established is that the corruption is STICKY — once a unit carries -// duplicate ids, every later edit inherits them, so each subsequent write looks -// like the culprit and the second experiment measures the first one's damage. +// This was a WRITE-TIME guard with no root cause behind it, and that is what +// made it worth having: the reporter (ako/mxcli-captrack #2) lost about an hour +// to one of these and could not establish what produced it — two candidate +// explanations were proposed during the build, both tested, both withdrawn. The +// corruption is STICKY, which is why: once a unit carries duplicate ids, every +// later edit inherits them, so each subsequent write looks like the culprit and +// the second experiment measures the first one's damage. +// +// **The cause of the reported case is now known**, and fixed at source in +// CarryTranslations (see reuseSafeID). Carrying translations by source string +// handed ONE stored Texts$Translation element to every rebuilt text sharing that +// string — eight copies of the literal '{1}' on one page — and appending it +// verbatim put its $ID on all eight. Reproduced end to end and fixed +// (CapTrackV2 §30/§17). +// +// The guard stays, and not only for the case it caught. It is cheap, it is the +// only thing standing between a write and an unopenable project, and nothing +// says Texts$Translation was the only way to get here. // // A guard is worth having anyway, and arguably worth more without a root cause // than with one: it converts an unopenable project and an hour of archaeology diff --git a/modelsdk/canon/translations.go b/modelsdk/canon/translations.go index 931aa10a9..ed854cc06 100644 --- a/modelsdk/canon/translations.go +++ b/modelsdk/canon/translations.go @@ -3,6 +3,7 @@ package canon import ( + "crypto/sha256" "sort" "strconv" @@ -95,8 +96,16 @@ func CarryTranslations(contents, stored []byte) []byte { return contents } + // Every element id already in the rebuilt document, so a carried element that + // would collide with one gets a fresh id instead. See reuseSafeID. + used := elementIDs(newDoc) + changed := false - for path, text := range newByPath { + // Sorted, not map order: which text is visited first decides which one keeps + // a stored element's id, and a run-to-run difference there would change the + // bytes and stop no-op elision from ever firing. + for _, path := range sortedPaths(newByPath) { + text := newByPath[path] var want map[string]bson.D if byPath { want = textTranslationElems(storedByPath[path]) @@ -106,7 +115,7 @@ func CarryTranslations(contents, stored []byte) []byte { if len(want) == 0 { continue } - if mergeText(text, want) { + if mergeText(text, want, path, used) { changed = true } } @@ -226,7 +235,11 @@ func storedTranslationSets(doc bson.D) map[string]*translationSet { // mergeText appends the languages a rebuilt text is missing, reporting whether // anything was added. A language the rebuild wrote is never overwritten: the // statement is the authority for what it says. -func mergeText(text bson.D, want map[string]bson.D) bool { +// +// used carries every element id already in the document and is updated as ids +// are taken, so the same stored element handed to two texts does not put one id +// on both of them (see reuseSafeID). +func mergeText(text bson.D, want map[string]bson.D, path string, used map[string]bool) bool { have := textTranslations(text) items, ok := docLookup(text, "Items").(bson.A) if !ok { @@ -237,7 +250,7 @@ func mergeText(text bson.D, want map[string]bson.D) bool { if _, exists := have[lang]; exists { continue } - items = append(items, want[lang]) + items = append(items, reuseSafeID(want[lang], path, lang, used)) added = true } if !added { @@ -328,3 +341,107 @@ func docLookup(d bson.D, key string) any { } return nil } + +// reuseSafeID returns the stored translation element to append, with a fresh +// element id if the stored one is already taken in this document. +// +// The by-source branch can hand ONE stored element to several rebuilt texts — +// eight copies of the literal '{1}' on a page is entirely ordinary — and +// appending it verbatim each time put the same $ID on all eight. A unit whose +// document holds two elements with the same $ID cannot be opened: Studio Pro +// refuses the whole project, and mxcli's own write-time guard (duplicates.go) +// refuses the write. That guard is what surfaced this, and its comment records +// that the cause could not be established at the time — this is it, for the +// Texts$Translation case it was reported against (CapTrackV2 §30/§17). +// +// Re-identifying a copy is safe HERE in a way that deduplicating ids in general +// is not, and the distinction is the whole argument for doing it: +// +// - An $ID is a pointer target, and rewriting one means finding every +// reference to it (ADR-0008). Nothing references a Texts$Translation: it is +// a leaf child of a Texts$Text, holding four keys and no identity of its own +// that anything resolves by. So there are no references to miss. +// - The FIRST use keeps the stored id, so an unchanged document still compares +// equal and no-op elision fires. Only the copies — elements that never +// existed on disk under that id — are re-identified. +// +// The fresh id is derived rather than random, so the same inputs give the same +// bytes on every run; a random id would make the document differ from itself and +// write on every execution. +func reuseSafeID(elem bson.D, path, lang string, used map[string]bool) bson.D { + id, ok := docLookup(elem, "$ID").(bson.Binary) + if !ok { + return elem // no id to collide; leave it exactly as it was + } + key := string(id.Data) + if !used[key] { + used[key] = true + return elem + } + next := derivedID(id, path, lang, used) + out := make(bson.D, len(elem)) + copy(out, elem) + for i := range out { + if out[i].Key == "$ID" { + out[i].Value = next + } + } + used[string(next.Data)] = true + return out +} + +// derivedID builds a deterministic 16-byte id from the element being copied and +// where the copy is going. Two different destinations therefore get two +// different ids, and the same run twice gets the same ones. +// +// The loop is a collision guard rather than an expectation: a SHA-256 prefix +// colliding with an id already in the document is not a case anyone will see, +// but the failure it would cause is precisely the one this function exists to +// remove, so it is cheaper to rule out than to reason about. +func derivedID(from bson.Binary, path, lang string, used map[string]bool) bson.Binary { + for n := 0; ; n++ { + h := sha256.New() + h.Write(from.Data) + h.Write([]byte(path)) + h.Write([]byte(lang)) + h.Write([]byte(strconv.Itoa(n))) + out := bson.Binary{Subtype: from.Subtype, Data: h.Sum(nil)[:16]} + if !used[string(out.Data)] { + return out + } + } +} + +// elementIDs collects every $ID in a document, so a carried element can be told +// whether the id it wants is already spoken for. +func elementIDs(doc bson.D) map[string]bool { + out := map[string]bool{} + var walk func(v any) + walk = func(v any) { + switch n := v.(type) { + case bson.D: + if b, ok := docLookup(n, "$ID").(bson.Binary); ok { + out[string(b.Data)] = true + } + for _, e := range n { + walk(e.Value) + } + case bson.A: + for _, e := range n { + walk(e) + } + } + } + walk(doc) + return out +} + +// sortedPaths keeps the visit order stable across runs. +func sortedPaths(m map[string]bson.D) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + sort.Strings(out) + return out +} diff --git a/modelsdk/canon/translations_duplicate_id_test.go b/modelsdk/canon/translations_duplicate_id_test.go new file mode 100644 index 000000000..4aeace8fb --- /dev/null +++ b/modelsdk/canon/translations_duplicate_id_test.go @@ -0,0 +1,177 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// CapTrackV2 FINDINGS §30/§17 — carrying translations gave several elements ONE +// element id, and mxcli's own duplicate-GUID guard then refused every in-place +// edit of the document: +// +// refusing to write unit …: 1 element id(s) are used more than once +// c540fbf0-… held by [Texts$Translation ×8] +// +// Eight translations of the literal '{1}' on one page shared an id. `GRANT VIEW +// ON PAGE …` failed, and so did `ALTER PAGE … INSERT` (§17, the same root +// cause); a full `CREATE OR REPLACE PAGE` still worked because it rewrites the +// unit rather than patching it. So the page looked correct and only UPDATES were +// blocked — which is why it took enabling a second language to surface at all. +// +// The mechanism is in the source-string pairing. `matchBySource` hands back the +// stored translation set for a source string, and mergeText appends those stored +// elements VERBATIM — deliberately, because keeping the stored $ID is what lets +// no-op elision fire. When two rebuilt texts share a source string (eight copies +// of '{1}' is entirely ordinary), both resolve to the SAME stored set, and both +// get the same element, id included. +// +// The positional branch cannot do this: each path maps to its own stored text. +// Only the by-source branch can hand one element to many. + +// twoTextDoc builds a document with two Texts$Text elements at distinct paths, +// each carrying the languages given. +func twoTextDoc(t *testing.T, extraPath bool, first, second []bson.D) []byte { + t.Helper() + mk := func(id byte, items []bson.D) bson.D { + arr := bson.A{int32(3)} + for _, it := range items { + arr = append(arr, it) + } + return bson.D{ + {Key: "$Type", Value: "Texts$Text"}, + {Key: "$ID", Value: bin(id)}, + {Key: "Items", Value: arr}, + } + } + doc := bson.D{ + {Key: "$Type", Value: "Forms$Form"}, + {Key: "$ID", Value: bin(1)}, + {Key: "Name", Value: "Page"}, + {Key: "Title", Value: mk(10, first)}, + {Key: "Caption", Value: mk(11, second)}, + } + if extraPath { + // A third text makes the path SETS differ, which is what sends + // CarryTranslations down the by-source branch instead of the positional + // one. Without it the two documents pair by path and each text gets its + // own stored element. + doc = append(doc, bson.E{Key: "Footer", Value: mk(12, first)}) + } + return marshal(t, doc) +} + +// collectTranslationIDs returns every Texts$Translation $ID in the document, in +// document order, so duplicates are visible. +func collectTranslationIDs(t *testing.T, raw []byte) []string { + t.Helper() + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + var out []string + var walk func(v any) + walk = func(v any) { + switch n := v.(type) { + case bson.D: + if ty, _ := docLookup(n, "$Type").(string); ty == "Texts$Translation" { + if b, ok := docLookup(n, "$ID").(bson.Binary); ok { + out = append(out, string(b.Data)) + } + } + for _, e := range n { + walk(e.Value) + } + case bson.A: + for _, e := range n { + walk(e) + } + } + } + walk(doc) + return out +} + +// The reported case: two texts with the SAME source string, carried from a +// stored document that has a translation for it. Both must end up with a +// translation, and the two must not share an element id. +func TestCarryTranslations_DoesNotReuseOneElementIDTwice(t *testing.T) { + de := tr(20, "de_DE", "Übersetzt") + en := tr(21, "en_US", "{1}") + stored := twoTextDoc(t, true, []bson.D{en, de}, []bson.D{en, de}) + // The rebuild carries one language, and a different path set, so the carry + // runs by source string. + contents := twoTextDoc(t, false, []bson.D{tr(30, "en_US", "{1}")}, []bson.D{tr(31, "en_US", "{1}")}) + + out := CarryTranslations(contents, stored) + + ids := collectTranslationIDs(t, out) + seen := map[string]int{} + for _, id := range ids { + seen[id]++ + } + for id, n := range seen { + if n > 1 { + t.Errorf("element id %q is used %d times — mxcli's own duplicate-GUID guard "+ + "refuses this document, blocking every in-place edit of it (§30)", id, n) + } + } + // CONTROL within the same test: the carry must still have happened. A fix + // that stopped carrying anything would satisfy the assertion above and + // silently reintroduce the translation loss this function exists to prevent. + if len(ids) != 4 { + t.Errorf("got %d translations, want 4 (two texts × en_US + carried de_DE): %v", len(ids), ids) + } +} + +// CONTROL: one text, one stored set — the ordinary case — must still keep the +// STORED id. That is what makes an unchanged document compare equal, so no-op +// elision can fire; minting a fresh id for every carry would write on every run. +func TestCarryTranslations_SingleUseKeepsTheStoredID(t *testing.T) { + de := tr(20, "de_DE", "Übersetzt") + stored := twoTextDoc(t, true, + []bson.D{tr(21, "en_US", "one"), de}, + []bson.D{tr(22, "en_US", "two"), tr(23, "de_DE", "Zwei")}) + contents := twoTextDoc(t, false, + []bson.D{tr(30, "en_US", "one")}, + []bson.D{tr(31, "en_US", "two")}) + + ids := collectTranslationIDs(t, CarryTranslations(contents, stored)) + + want := string(bin(20).Data) + found := false + for _, id := range ids { + if id == want { + found = true + } + } + if !found { + t.Errorf("the carried translation did not keep its stored id; "+ + "an unchanged document then never equals itself and every run writes. ids=%v", ids) + } +} + +// The same inputs must produce the same bytes, or elision cannot fire and the +// document churns on every run. Map iteration order decides which text is +// visited first, so this is not automatic. +func TestCarryTranslations_IsDeterministic(t *testing.T) { + de := tr(20, "de_DE", "Übersetzt") + en := tr(21, "en_US", "{1}") + stored := twoTextDoc(t, true, []bson.D{en, de}, []bson.D{en, de}) + + var first []byte + for i := 0; i < 20; i++ { + contents := twoTextDoc(t, false, + []bson.D{tr(30, "en_US", "{1}")}, []bson.D{tr(31, "en_US", "{1}")}) + out := CarryTranslations(contents, stored) + if first == nil { + first = out + continue + } + if string(out) != string(first) { + t.Fatalf("run %d produced different bytes — the document churns on every write", i) + } + } +} From af26ca646f278d10804edf761bf35b19b1baa889 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 22:42:37 +0000 Subject: [PATCH 02/17] fix(microflows): sort on an inherited attribute using its declaring entity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sorting a list on an attribute the entity INHERITS was not expressible in either direction: retrieve $Users from App.AppUser sort by Name asc; -> mxcli check and exec both pass, then mxbuild: [CE1613] "The selected attribute 'App.AppUser.Name' no longer exists." retrieve $Users from App.AppUser sort by System.User.Name asc; -> mxcli refuses: "sort by attribute 'System.User.Name' does not belong to entity 'App.AppUser'" So the spelling that wrote was wrong and the spelling that was right was refused. Mendix resolves a sort reference against the entity that DECLARES the attribute; mxcli qualified a bare name with the entity being retrieved, and treated any other entity in a qualified name as foreign — inferring association traversal steps, finding none, and refusing. An ancestor is not a traversal: the attribute is on the object already. Both halves now consult the generalization chain. The interesting part is that neither needed new machinery: flowBuilder has carried resolveAttributeInEntityHierarchy and entityIsSubtypeOf all along, and this path simply did not call them. The tell was in the report — reading the same attribute worked, because the page builder has walked the chain for years (declaringEntityFor). When one path resolves a name and its sibling does not, the resolver usually already exists. Verified end-to-end on a real 11.13 project with `extends System.User`. Pre-fix: stores App.AppUser.Name, mx check reports CE1613. Fixed: stores System.User.Name, mx check 0 errors, and the qualified spelling that was refused now executes. Two controls, both of which a careless fix would break: an attribute the entity declares ITSELF is still qualified with that entity, and an attribute on an unrelated entity is still refused rather than passed through to become a CE1613 at the far end of a build. Reported as CapTrackV2 FINDINGS §13. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../captrack-13-sort-inherited-attribute.mdl | 56 ++++++ .../cmd_microflows_builder_actions.go | 27 ++- ...icroflows_sort_inherited_attribute_test.go | 161 ++++++++++++++++++ 4 files changed, 243 insertions(+), 2 deletions(-) create mode 100644 mdl-examples/bug-tests/captrack-13-sort-inherited-attribute.mdl create mode 100644 mdl/executor/cmd_microflows_sort_inherited_attribute_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 599e15f98..e0062d04d 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -505,3 +505,4 @@ {"area": "mdl/executor", "date": "2026-09-03", "symptom": "describe -> exec of a mapping silently drops OriginalValue (the sample parsed from the JSON structure's snippet), and reformats the structure's snippet from one line to multi-line. No build error either way — pure diff churn against a Studio Pro original.", "cause": "OriginalValue was written empty on every element (import carried whatever the caller set, which was nothing; export hardcoded \"\"), on the strength of #882's measurement over TWO mappings a blank app ships. describe pretty-prints the snippet and exec stored the pretty form.", "file": "mdl/executor/mapping_original_value.go, mdl/executor/cmd_jsonstructures.go (sameJSONContent), model/types.go (ExportMappingElement.OriginalValue), mdl/backend/modelsdk/mapping_write.go, sdk/mpr/writer_export_mapping.go", "insight": "**Neither global default was right, and measuring the SPLIT is what showed it.** Across 3,042 value elements whose structure carries a sample, 2,322 (76%) store it and 720 do not — but the split is PER DOCUMENT: 145 mappings carry it on every element, 107 on none, 2 mixed. So which one a mapping gets is a property of how and when it was authored, not something derivable. Always-copy is wrong for 107 mappings; always-empty (the old behaviour) is wrong for 145. **A REWRITE does not have to choose** — it knows what was stored, so it carries it (guard-don't-drop, ADR-0005), matching stored to rebuilt by JsonPath because names and order can change while the schema binding cannot. That leaves #882's actual decision intact: a NEWLY authored mapping still writes empty, which is what that issue was about. The general lesson: when a measurement says 'always X' from a small sample and a wider one says 'sometimes X', check whether the split is per-document before picking a default — a per-document split usually means the answer is 'preserve', not 'choose'. The snippet half is the same shape: keep the stored formatting when the JSON is semantically equal (compare decoded values, not strings), so describe -> exec is a no-op instead of a reformat.", "refs": ["ako/mxcli#379", "ako/mxcli#882"]} {"area": "mdl/executor", "date": "2026-09-03", "symptom": "Implementing a new document type from a corpus census alone produces a document that builds at 0 errors and still differs from Studio Pro's in five places — Path, the typed-array marker, an empty mandatory list, PrimitiveType, and a dropped authored field.", "cause": "The census was 36 marketplace-module collections. A module author and someone building an app by hand exercise different parts of a document, so a census over shipped modules misses whatever only hand-authoring sets, and averages away anything the modules happen not to use.", "file": "mdl/backend/modelsdk/messagedefinition_write.go, mdl/executor/cmd_messagedefinitions.go, testdata/TestApp.OrderMessageDefinitions.bson", "insight": "**One hand-authored reference document is worth more than a large census of marketplace modules.** ako/TestApp's OrderMessageDefinitions found five things a 36-collection / 4,686-element census had not: (1) Path is a chain of ORIGINAL names, not exposed ones, and an ASSOCIATION contributes TWO segments — `Order|OrderLine_Order|OrderLine|Amount` — confirmed afterwards at 4,707/4,707 once we knew to look; (2) typed-array marker is 2, the codec defaults to 3; (3) every element serializes Children even when empty (the bare [2], same MandatoryLists rule as a rule document's Flows); (4) PrimitiveType is MAPPED not passed through — Long->Integer, AutoNumber->Integer, Enumeration->String, 279 corpus elements a pass-through gets wrong; (5) Example is author-set — empty in 4,686/4,686 of the corpus, set in TestApp, so hardcoding it empty silently drops the one that exists. **The round-trip test is what finds these**: read a REAL stored document into the semantic model, re-encode, diff against the STORED BYTES. Do not diff against a re-encoding of the decoded original — a lazily-decoded element that was never marked dirty encodes as an empty document, so that baseline passes by comparing nothing to nothing. **A hand-authored document also tends to carry natural controls**: this one uses the same association in both directions, which is exactly the control the cardinality rule needed. **After a CREATE that resolves a folder, invalidate the hierarchy cache** — the cached hierarchy predates the new folder, so a later lookup by module fails and CREATE OR MODIFY writes a DUPLICATE (CE0122). The update branch gets this free from applyDocumentFolder; a create branch has to say it.", "refs": ["ako/mxcli#272"], "ce": ["CE0122", "CE1613"]} {"area": "mdl/executor", "cause": "Two DataTypes$ sub-documents the writer never emitted, both on Microflows$CallExternalAction. (1) edmReturnTypeToKind mapped only EDM primitives and returned \"\" for anything else -- documented in-code as \"Complex / collection / entity-typed returns aren't yet mapped\" -- so an action returning an ENTITY got no VariableDataType at all. (2) ExternalActionParameterMapping.ParameterType was never written, though generated/metamodel declares it WITHOUT omitempty. Separately, mdl/catalog/builder_external.go catalogued only entities whose Source is Rest$ODataRemoteEntitySource, skipping every Rest$ODataEntityTypeSource -- the derived, abstract, contained and action parameter/return types that have no entity set.", "ce": ["CE7252", "CE7269", "CE0117", "CE7251"], "date": "2026-09-03", "file": "mdl/executor/cmd_microflows_builder_calls.go (resolveExternalActionReturnKind, resolveExternalActionParameterKinds, edmBareTypeName); sdk/mpr/writer_microflow_actions.go + mdl/backend/modelsdk/microflow_external_action_write.go (both writers); sdk/microflows/microflows_actions.go (ResultEntity, ParameterDataType/ParameterEntity); mdl/catalog/builder_external.go (isODataEntitySource); new validator mdl/executor/validate_external_action_calls.go", "insight": "**Read the CE code out of Mendix's own assemblies before theorising about it.** `strings Mendix.Modeler.Texts.dll | grep CE7252` gives the symbol, the English text AND the LOCATION comment -- here CallExternalAction.cs for both codes, which settles in one command that the entity import can never fix them and that the reporter was pulling the wrong lever. Same technique found CE7253 and the CE7251 constraint (Mendix's `call external action` takes OData ACTIONS only, not Functions, and an unbound action needs an in the EntityContainer or it is not callable at all). **A missing mandatory sub-document is the recurring shape here**: generated/metamodel omitting `omitempty` on a pointer property is the tell, and the same fix pattern applied twice in one bug. **The reporter's evidence was an artifact of OUR tool**: contract_entities.UsedByExternalEntity is an mxcli catalog column filled by joining external_entities on RemoteName, so while that table skipped type-sourced entities the column was structurally always empty for exactly the entities in question -- it read as 'not linked' whether or not the import had worked. When a report cites one of our own derived columns as evidence, verify the column can be non-empty for that case before believing it. **Verification without a fixture**: no $metadata in the repo declared an action, so the contract was served from `python3 -m http.server` on 127.0.0.1 and MetadataUrl pointed at it -- a local HTTP contract makes the whole consumed-OData path testable end to end. Controls: reverting the return resolver reproduces CE7269 verbatim; before the ParameterType fix, ANY parameter of ANY type produced CE7252 + one CE0117 per argument; after, 0 errors on all three shapes. Repro mdl-examples/bug-tests/odata-1020-external-action-types.mdl", "refs": ["mendixlabs/mxcli#1020"], "symptom": "CE7252 \"The parameters for remote action '' have changed\" and CE7269 \"The return type for remote action '' has changed\" persist after CREATE OR MODIFY EXTERNAL ENTITIES, which reports success and changes nothing. A SQL query over CATALOG.contract_entities shows UsedByExternalEntity empty for the action's parameter/response entities while entity-set entities populate it, which reads as a broken link between the imported entity and the contract."} +{"area": "mdl/executor", "date": "2026-09-02", "symptom": "`retrieve $L from Mod.Sub sort by asc` passes `mxcli check` AND `exec`, then mxbuild fails **CE1613** \"The selected attribute 'Mod.Sub.Attr' no longer exists.\" \u2014 and the correct spelling, `sort by Base.Entity.Attr`, is REFUSED by mxcli with \"sort by attribute '\u2026' does not belong to entity '\u2026'\". Reading the same attribute works fine. The list operation `SORT($L, Attr ASC)` writes the same broken reference.", "cause": "The sort path qualified a bare attribute name with the entity being RETRIEVED (`entityQN + \".\" + attr`), but Mendix resolves a sort reference against the entity that DECLARES the attribute. For the qualified form it treated any entity other than the retrieved one as foreign, tried to infer association traversal steps, found none, and refused \u2014 an ancestor is not a traversal, the attribute is on the object already. Both halves now consult the generalization chain: `resolveAttributeInEntityHierarchy` for the bare name, `entityIsSubtypeOf` for the qualified one.", "file": "`mdl/executor/cmd_microflows_builder_actions.go` (the SORT BY block in `addRetrieveAction`)", "insight": "**Both helpers already existed on `flowBuilder` and this path simply did not call them** \u2014 the fix is two call sites, not new machinery. The tell was in the report: reading the attribute worked while sorting on it did not, and the page builder has walked the chain for years (`declaringEntityFor`). When one path resolves a name and a sibling path does not, look for the existing resolver before writing one. Controls: an attribute the entity declares itself must still be qualified with that entity (a fix that always walked to the base breaks every ordinary sort), and an attribute on an unrelated entity must still be refused rather than passed through to become a CE1613 at the far end of a build. Verified end-to-end on a real 11.13 project with `extends System.User`: pre-fix writes `App.AppUser.Name` and mx check reports CE1613; fixed writes `System.User.Name` and mx check is 0 errors, and the qualified spelling that was refused now executes. Reported as CapTrackV2 FINDINGS \u00a713."} diff --git a/mdl-examples/bug-tests/captrack-13-sort-inherited-attribute.mdl b/mdl-examples/bug-tests/captrack-13-sort-inherited-attribute.mdl new file mode 100644 index 000000000..c034d4b99 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-13-sort-inherited-attribute.mdl @@ -0,0 +1,56 @@ +-- CapTrackV2 FINDINGS §13 — sorting on an INHERITED attribute was not +-- expressible in either direction. +-- +-- retrieve $Users from App.AppUser sort by Name asc; +-- -> mxcli check and exec both passed, then mxbuild: +-- [CE1613] "The selected attribute 'App.AppUser.Name' no longer exists." +-- mxcli qualified with the LIST's entity; Mendix resolves a sort +-- reference against the DECLARING one. +-- +-- retrieve $Users from App.AppUser sort by System.User.Name asc; +-- -> mxcli refused: "sort by attribute 'System.User.Name' does not belong +-- to entity 'App.AppUser'" — which is the reference Mendix wants. +-- +-- So the spelling that wrote was wrong and the spelling that was right was +-- refused. Reading the same attribute always worked (`textbox (Attribute: +-- Name)`), because the page builder walks the generalization chain; this path +-- did not, though flowBuilder has carried resolveAttributeInEntityHierarchy and +-- entityIsSubtypeOf all along. +-- +-- Verify: +-- mxcli exec captrack-13-sort-inherited-attribute.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe microflow SortInh.MF_SortInherited" +-- -- must emit: sort by System.User.Name asc +-- mx check -p app.mpr -- 0 errors; before the fix, CE1613 + +create module SortInh; +/ +create persistent entity SortInh.AppUser extends System.User ( + FullName: string(200) +); +/ + +-- The reported case: a BARE inherited name. +create microflow SortInh.MF_SortInherited () +returns list of SortInh.AppUser as $Users +begin + retrieve $Users from SortInh.AppUser sort by Name asc; +end; +/ + +-- The other half: naming the declaring entity outright, which was refused. +create microflow SortInh.MF_SortQualified () +returns list of SortInh.AppUser as $Users +begin + retrieve $Users from SortInh.AppUser sort by System.User.Name asc; +end; +/ + +-- CONTROL: an attribute the entity declares ITSELF must still be qualified with +-- it. A fix that always walked to the base would break every ordinary sort. +create microflow SortInh.MF_SortOwn () +returns list of SortInh.AppUser as $Users +begin + retrieve $Users from SortInh.AppUser sort by FullName asc; +end; +/ diff --git a/mdl/executor/cmd_microflows_builder_actions.go b/mdl/executor/cmd_microflows_builder_actions.go index 294b93036..a4956d894 100644 --- a/mdl/executor/cmd_microflows_builder_actions.go +++ b/mdl/executor/cmd_microflows_builder_actions.go @@ -997,7 +997,25 @@ func (fb *flowBuilder) addRetrieveAction(s *ast.RetrieveStmt) model.ID { attrPath := col.Attribute var entityRefSteps []microflows.EntityRefStep if !strings.Contains(attrPath, ".") { - attrPath = entityQN + "." + attrPath + // Qualify with the entity that DECLARES the attribute, which is + // not always the one being retrieved. Mendix resolves a sort + // reference against the declaring entity, so qualifying an + // INHERITED name with the list's own entity produced CE1613 "The + // selected attribute … no longer exists" — after `mxcli check` + // and `exec` both passed (CapTrackV2 §13). Reading the same + // attribute already worked, because the page builder walks the + // chain; this path did not, though flowBuilder has carried + // resolveAttributeInEntityHierarchy all along. + // + // Falling back to the plain qualification keeps the behaviour + // for a unit test with no backend, and for an attribute the + // model does not know about — which is the checker's business to + // report, not this function's to guess at. + if declared, ok := fb.resolveAttributeInEntityHierarchy(entityQN, attrPath); ok { + attrPath = declared + } else { + attrPath = entityQN + "." + attrPath + } } else { // Validate that qualified attribute path belongs to the retrieved entity // Expected format: Module.Entity.Attribute @@ -1005,7 +1023,12 @@ func (fb *flowBuilder) addRetrieveAction(s *ast.RetrieveStmt) model.ID { if len(parts) >= 3 { // Extract entity from attribute path (first two parts) attrEntityQN := parts[0] + "." + parts[1] - if attrEntityQN != entityQN { + // An ANCESTOR is not a foreign entity: naming the declaring + // entity outright is the reference Mendix wants, and it was + // refused as not belonging (CapTrackV2 §13). It takes no + // EntityRefSteps either — inheritance is not a traversal, + // the attribute is on the object already. + if attrEntityQN != entityQN && !fb.entityIsSubtypeOf(entityQN, attrEntityQN) { entityRefSteps = fb.inferSortEntityRefSteps(entityQN, attrPath) if len(entityRefSteps) == 0 { fb.addError("sort by attribute '%s' does not belong to entity '%s'", col.Attribute, entityQN) diff --git a/mdl/executor/cmd_microflows_sort_inherited_attribute_test.go b/mdl/executor/cmd_microflows_sort_inherited_attribute_test.go new file mode 100644 index 000000000..37af9274b --- /dev/null +++ b/mdl/executor/cmd_microflows_sort_inherited_attribute_test.go @@ -0,0 +1,161 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// CapTrackV2 FINDINGS §13 — sorting a list on an attribute the entity INHERITS +// is not expressible, in either direction: +// +// RETRIEVE $U FROM CapTrack.CapTrackUser SORT BY Name ASC; +// -> mxcli check and exec both pass, then mxbuild fails +// CE1613 "The selected attribute 'CapTrack.CapTrackUser.Name' no longer +// exists." — mxcli qualifies with the LIST's entity, mxbuild wants the +// DECLARING one. +// +// RETRIEVE … SORT BY System.User.Name ASC; +// -> mxcli refuses: "sort by attribute 'System.User.Name' does not belong to +// entity 'CapTrack.CapTrackUser'". +// +// So the spelling that writes is wrong and the spelling that is right is +// refused. Reading the same attribute works — `TEXTBOX (Attribute: Name)` +// resolves — which is the tell: the page builder walks the generalization chain +// (`declaringEntityFor`) and this path did not, though `flowBuilder` has carried +// `resolveAttributeInEntityHierarchy` and `entityIsSubtypeOf` all along. + +// inheritedSortBackend is an entity that inherits `Name` from a base in another +// module, and declares `FullName` itself. +func inheritedSortBackend() *mock.MockBackend { + appModuleID := model.ID("synthetic-app-module") + baseModuleID := model.ID("synthetic-base-module") + return &mock.MockBackend{ + GetModuleByNameFunc: func(name string) (*model.Module, error) { + switch name { + case "SyntheticApp": + return &model.Module{BaseElement: model.BaseElement{ID: appModuleID}, Name: name}, nil + case "SyntheticBase": + return &model.Module{BaseElement: model.BaseElement{ID: baseModuleID}, Name: name}, nil + } + return nil, nil + }, + GetDomainModelFunc: func(id model.ID) (*domainmodel.DomainModel, error) { + switch id { + case appModuleID: + return &domainmodel.DomainModel{ + ContainerID: appModuleID, + Entities: []*domainmodel.Entity{{ + Name: "AppUser", + GeneralizationRef: "SyntheticBase.User", + Attributes: []*domainmodel.Attribute{ + {Name: "FullName", Type: &domainmodel.StringAttributeType{}}, + }, + }}, + }, nil + case baseModuleID: + return &domainmodel.DomainModel{ + ContainerID: baseModuleID, + Entities: []*domainmodel.Entity{{ + Name: "User", + Attributes: []*domainmodel.Attribute{ + {Name: "Name", Type: &domainmodel.StringAttributeType{}}, + }, + }}, + }, nil + } + return nil, nil + }, + } +} + +// sortAttrOf builds the retrieve and returns the stored sort attribute paths and +// any build errors. +func sortAttrOf(t *testing.T, attr string) (paths []string, errs []string) { + t.Helper() + fb := &flowBuilder{backend: inheritedSortBackend(), spacing: 100} + fb.addRetrieveAction(&ast.RetrieveStmt{ + Variable: "Users", + Source: ast.QualifiedName{Module: "SyntheticApp", Name: "AppUser"}, + SortColumns: []ast.SortColumnDef{{Attribute: attr, Order: "ASC"}}, + }) + for _, obj := range fb.objects { + act, ok := obj.(*microflows.ActionActivity) + if !ok { + continue + } + ra, ok := act.Action.(*microflows.RetrieveAction) + if !ok { + continue + } + ds, ok := ra.Source.(*microflows.DatabaseRetrieveSource) + if !ok { + continue + } + for _, s := range ds.Sorting { + paths = append(paths, s.AttributeQualifiedName) + } + } + return paths, fb.errors +} + +// A bare inherited name must be stored against the DECLARING entity, which is +// what mxbuild resolves. +func TestSortBy_BareInheritedAttributeUsesTheDeclaringEntity(t *testing.T) { + paths, errs := sortAttrOf(t, "Name") + if len(errs) > 0 { + t.Fatalf("unexpected errors: %v", errs) + } + if len(paths) != 1 { + t.Fatalf("got %d sort columns, want 1: %v", len(paths), paths) + } + if paths[0] != "SyntheticBase.User.Name" { + t.Errorf("stored %q, want %q — the list's own entity does not declare Name, "+ + "and mxbuild rejects that reference with CE1613", paths[0], "SyntheticBase.User.Name") + } +} + +// CONTROL 1: an attribute the entity declares ITSELF is still qualified with it. +// A fix that always walked to the base would break every ordinary sort. +func TestSortBy_OwnAttributeIsUnchanged(t *testing.T) { + paths, errs := sortAttrOf(t, "FullName") + if len(errs) > 0 { + t.Fatalf("unexpected errors: %v", errs) + } + if len(paths) != 1 || paths[0] != "SyntheticApp.AppUser.FullName" { + t.Errorf("stored %v, want [SyntheticApp.AppUser.FullName]", paths) + } +} + +// The other half of §13: naming the declaring entity explicitly is the CORRECT +// reference and was refused outright. +func TestSortBy_QualifiedAncestorAttributeIsAccepted(t *testing.T) { + paths, errs := sortAttrOf(t, "SyntheticBase.User.Name") + if len(errs) > 0 { + t.Fatalf("naming the declaring entity is the reference mxbuild wants, "+ + "and it was refused: %v", errs) + } + if len(paths) != 1 || paths[0] != "SyntheticBase.User.Name" { + t.Errorf("stored %v, want [SyntheticBase.User.Name]", paths) + } +} + +// CONTROL 2: an attribute on an entity that is NOT in the chain and not reachable +// by association is still refused. Accepting anything qualified would turn a +// diagnosable mistake into a CE1613 at the far end of a build. +func TestSortBy_UnrelatedEntityIsStillRefused(t *testing.T) { + _, errs := sortAttrOf(t, "SyntheticBase.Other.Name") + if len(errs) == 0 { + t.Fatal("an attribute of an unrelated entity must still be refused") + } + if !strings.Contains(strings.Join(errs, " "), "does not belong to entity") { + t.Errorf("unexpected message: %v", errs) + } +} From d57f9e88a1ea7b2b6958ab8def25334bf980274c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 05:55:43 +0000 Subject: [PATCH 03/17] feat(associations): SQL referential actions, and the message PREVENT needs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An association written with DELETE_BEHAVIOR PREVENT produced an app whose RUNTIME WOULD NOT START. Not a build error — `mx check` reports 0 errors either way: ERROR - M2EE: An error occurred while initializing the Runtime: None.get java.util.NoSuchElementException: None.get at …SchemeFactory$.…$setDeleteBehavior(SchemeFactory.scala:515) mxcli wrote ChildDeleteBehavior "DeleteMeIfNoReferences" with a NULL ChildErrorMessage, and MDL had no syntax for the message at all. The field is CONDITIONAL in Studio Pro's own dialog — it appears only once that third radio button is selected — which is why it went unnoticed, and why a screenshot of the association properties nearly falsified the (correct) diagnosis. A census of 47,789 units across 122 projects found 423 delete behaviours and NOT ONE using this one, so no reference existed anywhere until one was authored for this fix. That reference (ako/TestApp, Mappings.Order_Customer) pins the shape: "ChildDeleteBehavior": "DeleteMeIfNoReferences", "ChildErrorMessage": { "$Type": "Texts$Text", "Items": [ 3, { "$Type": "Texts$Translation", "LanguageCode": "en_US", "Text": "…" } ] }, "ParentErrorMessage": null Two things it settles that reasoning would not: the item collection's typed-array marker is 3, and the other side stays null because that side is still "keep". So the element is written for that behaviour only, and only on the child side. The syntax is SQL's, because Mendix's three behaviours ARE SQL's referential actions and MDL's FROM/TO already matches a foreign key's direction — measured on the same reference: ParentPointer -> Order (the FK owner, the FROM), ChildPointer -> Customer (referenced, the TO): FROM Shop.Order TO Shop.Customer ON DELETE RESTRICT ERROR_MESSAGE 'A customer with orders cannot be deleted'; That reads the way it does in CREATE TABLE, with no knowledge of which side Mendix calls the child. The old spelling gave none: MDL's names are Mendix's with the word "Me" dropped, and Me was the only word saying whose deletion was being described. DELETE_BEHAVIOR still parses and still means the same thing, and can take ERROR_MESSAGE too, so no existing script breaks; DESCRIBE emits the ON DELETE form because it says which side is governed. ERROR_MESSAGE rather than ERROR: the latter is already a token, and the compound mirrors Studio Pro's own label. SQL's RESTRICT has no custom message, so this clause is a Mendix extension rather than borrowed. Also removes three ast.DeleteBehavior values no grammar rule could produce and which named nothing Mendix has — the trap behind upstream #901, where String() was used as a storage encoding. Verified end-to-end on 11.13. A project authored by the pre-fix binary reproduces `None.get` at boot verbatim; the fixed one writes a delete behaviour byte-identical to Studio Pro's (the legacy engine matches including key order; modelsdk differs only in gen's pre-existing Parent-before-Child property order, which every association mxcli has ever written already has). mx check 0 errors, and DESCRIBE round-trips the message through the parser. The empty-message shape is INFERRED, not measured — the reference captures a filled-in message, and nobody has saved one with the box cleared. Flagged at deleteErrorText. Reported as CapTrackV2 FINDINGS §1. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + cmd/mxcli/lsp_completions_gen.go | 2 + cmd/mxcli/syntax/features_domain_model.go | 32 ++++- .../captrack-1-on-delete-restrict-message.mdl | 73 ++++++++++ mdl/ast/ast_association.go | 64 +++++---- mdl/backend/modelsdk/domainmodel.go | 38 ++++- .../domainmodel_delete_message_test.go | 110 +++++++++++++++ mdl/backend/modelsdk/domainmodel_write.go | 35 +++++ mdl/executor/cmd_associations.go | 58 +++++--- .../cmd_associations_delete_behavior_test.go | 55 ++++++++ mdl/grammar/MDLLexer.g4 | 17 +++ mdl/grammar/domains/MDLDomainModel.g4 | 32 ++++- mdl/grammar/domains/MDLSettings.g4 | 2 +- mdl/visitor/visitor_association.go | 37 ++++- mdl/visitor/visitor_helpers.go | 42 ++++++ mdl/visitor/visitor_on_delete_test.go | 130 ++++++++++++++++++ sdk/domainmodel/domainmodel.go | 9 ++ sdk/mpr/parser_domainmodel.go | 35 ++++- sdk/mpr/writer_domainmodel.go | 20 ++- 19 files changed, 731 insertions(+), 61 deletions(-) create mode 100644 mdl-examples/bug-tests/captrack-1-on-delete-restrict-message.mdl create mode 100644 mdl/backend/modelsdk/domainmodel_delete_message_test.go create mode 100644 mdl/visitor/visitor_on_delete_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index e0062d04d..e68335ff1 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -506,3 +506,4 @@ {"area": "mdl/executor", "date": "2026-09-03", "symptom": "Implementing a new document type from a corpus census alone produces a document that builds at 0 errors and still differs from Studio Pro's in five places — Path, the typed-array marker, an empty mandatory list, PrimitiveType, and a dropped authored field.", "cause": "The census was 36 marketplace-module collections. A module author and someone building an app by hand exercise different parts of a document, so a census over shipped modules misses whatever only hand-authoring sets, and averages away anything the modules happen not to use.", "file": "mdl/backend/modelsdk/messagedefinition_write.go, mdl/executor/cmd_messagedefinitions.go, testdata/TestApp.OrderMessageDefinitions.bson", "insight": "**One hand-authored reference document is worth more than a large census of marketplace modules.** ako/TestApp's OrderMessageDefinitions found five things a 36-collection / 4,686-element census had not: (1) Path is a chain of ORIGINAL names, not exposed ones, and an ASSOCIATION contributes TWO segments — `Order|OrderLine_Order|OrderLine|Amount` — confirmed afterwards at 4,707/4,707 once we knew to look; (2) typed-array marker is 2, the codec defaults to 3; (3) every element serializes Children even when empty (the bare [2], same MandatoryLists rule as a rule document's Flows); (4) PrimitiveType is MAPPED not passed through — Long->Integer, AutoNumber->Integer, Enumeration->String, 279 corpus elements a pass-through gets wrong; (5) Example is author-set — empty in 4,686/4,686 of the corpus, set in TestApp, so hardcoding it empty silently drops the one that exists. **The round-trip test is what finds these**: read a REAL stored document into the semantic model, re-encode, diff against the STORED BYTES. Do not diff against a re-encoding of the decoded original — a lazily-decoded element that was never marked dirty encodes as an empty document, so that baseline passes by comparing nothing to nothing. **A hand-authored document also tends to carry natural controls**: this one uses the same association in both directions, which is exactly the control the cardinality rule needed. **After a CREATE that resolves a folder, invalidate the hierarchy cache** — the cached hierarchy predates the new folder, so a later lookup by module fails and CREATE OR MODIFY writes a DUPLICATE (CE0122). The update branch gets this free from applyDocumentFolder; a create branch has to say it.", "refs": ["ako/mxcli#272"], "ce": ["CE0122", "CE1613"]} {"area": "mdl/executor", "cause": "Two DataTypes$ sub-documents the writer never emitted, both on Microflows$CallExternalAction. (1) edmReturnTypeToKind mapped only EDM primitives and returned \"\" for anything else -- documented in-code as \"Complex / collection / entity-typed returns aren't yet mapped\" -- so an action returning an ENTITY got no VariableDataType at all. (2) ExternalActionParameterMapping.ParameterType was never written, though generated/metamodel declares it WITHOUT omitempty. Separately, mdl/catalog/builder_external.go catalogued only entities whose Source is Rest$ODataRemoteEntitySource, skipping every Rest$ODataEntityTypeSource -- the derived, abstract, contained and action parameter/return types that have no entity set.", "ce": ["CE7252", "CE7269", "CE0117", "CE7251"], "date": "2026-09-03", "file": "mdl/executor/cmd_microflows_builder_calls.go (resolveExternalActionReturnKind, resolveExternalActionParameterKinds, edmBareTypeName); sdk/mpr/writer_microflow_actions.go + mdl/backend/modelsdk/microflow_external_action_write.go (both writers); sdk/microflows/microflows_actions.go (ResultEntity, ParameterDataType/ParameterEntity); mdl/catalog/builder_external.go (isODataEntitySource); new validator mdl/executor/validate_external_action_calls.go", "insight": "**Read the CE code out of Mendix's own assemblies before theorising about it.** `strings Mendix.Modeler.Texts.dll | grep CE7252` gives the symbol, the English text AND the LOCATION comment -- here CallExternalAction.cs for both codes, which settles in one command that the entity import can never fix them and that the reporter was pulling the wrong lever. Same technique found CE7253 and the CE7251 constraint (Mendix's `call external action` takes OData ACTIONS only, not Functions, and an unbound action needs an in the EntityContainer or it is not callable at all). **A missing mandatory sub-document is the recurring shape here**: generated/metamodel omitting `omitempty` on a pointer property is the tell, and the same fix pattern applied twice in one bug. **The reporter's evidence was an artifact of OUR tool**: contract_entities.UsedByExternalEntity is an mxcli catalog column filled by joining external_entities on RemoteName, so while that table skipped type-sourced entities the column was structurally always empty for exactly the entities in question -- it read as 'not linked' whether or not the import had worked. When a report cites one of our own derived columns as evidence, verify the column can be non-empty for that case before believing it. **Verification without a fixture**: no $metadata in the repo declared an action, so the contract was served from `python3 -m http.server` on 127.0.0.1 and MetadataUrl pointed at it -- a local HTTP contract makes the whole consumed-OData path testable end to end. Controls: reverting the return resolver reproduces CE7269 verbatim; before the ParameterType fix, ANY parameter of ANY type produced CE7252 + one CE0117 per argument; after, 0 errors on all three shapes. Repro mdl-examples/bug-tests/odata-1020-external-action-types.mdl", "refs": ["mendixlabs/mxcli#1020"], "symptom": "CE7252 \"The parameters for remote action '' have changed\" and CE7269 \"The return type for remote action '' has changed\" persist after CREATE OR MODIFY EXTERNAL ENTITIES, which reports success and changes nothing. A SQL query over CATALOG.contract_entities shows UsedByExternalEntity empty for the action's parameter/response entities while entity-set entities populate it, which reads as a broken link between the imported entity and the contract."} {"area": "mdl/executor", "date": "2026-09-02", "symptom": "`retrieve $L from Mod.Sub sort by asc` passes `mxcli check` AND `exec`, then mxbuild fails **CE1613** \"The selected attribute 'Mod.Sub.Attr' no longer exists.\" \u2014 and the correct spelling, `sort by Base.Entity.Attr`, is REFUSED by mxcli with \"sort by attribute '\u2026' does not belong to entity '\u2026'\". Reading the same attribute works fine. The list operation `SORT($L, Attr ASC)` writes the same broken reference.", "cause": "The sort path qualified a bare attribute name with the entity being RETRIEVED (`entityQN + \".\" + attr`), but Mendix resolves a sort reference against the entity that DECLARES the attribute. For the qualified form it treated any entity other than the retrieved one as foreign, tried to infer association traversal steps, found none, and refused \u2014 an ancestor is not a traversal, the attribute is on the object already. Both halves now consult the generalization chain: `resolveAttributeInEntityHierarchy` for the bare name, `entityIsSubtypeOf` for the qualified one.", "file": "`mdl/executor/cmd_microflows_builder_actions.go` (the SORT BY block in `addRetrieveAction`)", "insight": "**Both helpers already existed on `flowBuilder` and this path simply did not call them** \u2014 the fix is two call sites, not new machinery. The tell was in the report: reading the attribute worked while sorting on it did not, and the page builder has walked the chain for years (`declaringEntityFor`). When one path resolves a name and a sibling path does not, look for the existing resolver before writing one. Controls: an attribute the entity declares itself must still be qualified with that entity (a fix that always walked to the base breaks every ordinary sort), and an attribute on an unrelated entity must still be refused rather than passed through to become a CE1613 at the far end of a build. Verified end-to-end on a real 11.13 project with `extends System.User`: pre-fix writes `App.AppUser.Name` and mx check reports CE1613; fixed writes `System.User.Name` and mx check is 0 errors, and the qualified spelling that was refused now executes. Reported as CapTrackV2 FINDINGS \u00a713."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "An app whose domain model uses `DELETE_BEHAVIOR PREVENT` will not START. `mx check` reports 0 errors; the failure is at runtime init: `ERROR - M2EE: An error occurred while initializing the Runtime: None.get` / `java.util.NoSuchElementException: None.get` at `SchemeFactory$.\u2026$setDeleteBehavior(SchemeFactory.scala:515)`. Rewriting the associations to the default behaviour takes the app from crash to HTTP 200.", "cause": "mxcli wrote `ChildDeleteBehavior: DeleteMeIfNoReferences` with `ChildErrorMessage: null` \u2014 the codec registers `DomainModels$DeleteBehavior` with both error-message slots in `NullFields`, unconditionally. Studio Pro writes a `Texts$Text` there for that behaviour only. MDL had no syntax for the message at all. Fixed by adding SQL's referential actions (`ON DELETE CASCADE|RESTRICT|SET NULL`, plus `ERROR_MESSAGE '\u2026'`) and writing the text element when the child behaviour is DeleteMeIfNoReferences \u2014 only then, and only on the child side.", "file": "`mdl/grammar/MDLLexer.g4` + `domains/MDLDomainModel.g4` + `domains/MDLSettings.g4` (keyword rule), `mdl/visitor/visitor_association.go` + `visitor_helpers.go` (`buildReferentialAction`), `mdl/ast/ast_association.go`, `sdk/domainmodel/domainmodel.go` (`DeleteBehavior.ErrorMessage`), `mdl/backend/modelsdk/domainmodel_write.go` (`deleteErrorText`) + `domainmodel.go` (read back), `mdl/executor/cmd_associations.go` (`describeDeleteClause`)", "insight": "**The field is CONDITIONAL in Studio Pro's own dialog** \u2014 it only appears once the third radio button is selected, which is why a screenshot of the association properties showed no message field and nearly falsified the reporter's (correct) diagnosis. Ask for the reference in the state that exhibits the behaviour, not the default state. A census of 47,789 units across 122 projects found 423 delete behaviours and **zero** using this one, so no reference existed anywhere until one was authored \u2014 that absence is why the null shipped and why nobody hit it sooner. Syntax note: Mendix's three behaviours ARE SQL's referential actions, and MDL's FROM/TO already matches a foreign key's direction (ParentPointer = FROM = FK owner, ChildPointer = TO = referenced, measured), so `ON DELETE RESTRICT` needed no invention and fixed a readability complaint that `DELETE_IF_NO_REFERENCES` had earned \u2014 MDL's names are Mendix's with the word **Me** dropped, and Me was the only word saying whose deletion it described. Controls, end-to-end on 11.13: a project authored by the pre-fix binary reproduces `None.get` at boot verbatim; the fixed one boots. DESCRIBE emits the ON DELETE form with the message, or a describe->exec round trip rebuilds the crash. The empty-message shape is INFERRED, not measured \u2014 flagged in `deleteErrorText`. Reported as CapTrackV2 FINDINGS \u00a71."} diff --git a/cmd/mxcli/lsp_completions_gen.go b/cmd/mxcli/lsp_completions_gen.go index 92a1de628..b72900691 100644 --- a/cmd/mxcli/lsp_completions_gen.go +++ b/cmd/mxcli/lsp_completions_gen.go @@ -59,6 +59,8 @@ var mdlGeneratedKeywords = []protocol.CompletionItem{ {Label: "STORAGE", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, {Label: "TABLE", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, {Label: "DELETE_BEHAVIOR", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, + {Label: "RESTRICT", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, + {Label: "ERROR MESSAGE", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, {Label: "CASCADE", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, {Label: "PREVENT", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, {Label: "ALLOW_CREATE_CHANGE_LOCALLY", Kind: protocol.CompletionItemKindKeyword, Detail: "DDL keyword"}, diff --git a/cmd/mxcli/syntax/features_domain_model.go b/cmd/mxcli/syntax/features_domain_model.go index f438c3601..367513f8c 100644 --- a/cmd/mxcli/syntax/features_domain_model.go +++ b/cmd/mxcli/syntax/features_domain_model.go @@ -184,12 +184,38 @@ func init() { Path: "domain-model.association.delete-behavior", Summary: "Delete behavior options for associations", Keywords: []string{ - "delete behavior", "cascade", "prevent", + "delete behavior", "cascade", "prevent", "restrict", + "on delete", "set null", "error message", "referential action", "delete and references", "delete but keep references", "delete if no references", "referential integrity", }, - Syntax: "DELETE_BEHAVIOR options:\n DELETE_BUT_KEEP_REFERENCES Delete object, nullify FK (default)\n DELETE_AND_REFERENCES Delete object and cascade to children\n DELETE_IF_NO_REFERENCES Prevent deletion if referenced\n CASCADE Alias for DELETE_AND_REFERENCES\n PREVENT Alias for DELETE_IF_NO_REFERENCES", - Example: "CREATE ASSOCIATION Shop.Order_Customer\n FROM Shop.Order TO Shop.Customer\n TYPE Reference\n DELETE_BEHAVIOR PREVENT;\n\nCREATE ASSOCIATION Shop.Order_Lines\n FROM Shop.OrderLine TO Shop.Order\n TYPE Reference\n DELETE_BEHAVIOR CASCADE;", + Syntax: "ON DELETE [ERROR_MESSAGE '']\n" + + " ON DELETE SET NULL Keep the referencing objects, clear the reference (default)\n" + + " ON DELETE CASCADE Delete the referencing objects too\n" + + " ON DELETE RESTRICT Refuse the delete while references exist\n\n" + + "These are SQL's referential actions, and Mendix's three delete behaviours are\n" + + "exactly them. FROM/TO already matches a foreign key's direction -- FROM owns the\n" + + "key, TO is referenced -- so `FROM Order TO Customer ON DELETE RESTRICT` means\n" + + "deleting a CUSTOMER is refused while Orders reference it, the same way it would\n" + + "in CREATE TABLE.\n\n" + + "ERROR_MESSAGE is what the user sees when a RESTRICT delete is refused (Studio\n" + + "Pro's \"Error message if 'X' object cannot be deleted\"). SQL has no equivalent;\n" + + "this is a Mendix extension. Omitting it stores an empty message.\n\n" + + "The older spelling still works and means the same thing:\n" + + " DELETE_BEHAVIOR DELETE_BUT_KEEP_REFERENCES | DELETE_AND_REFERENCES\n" + + " | DELETE_IF_NO_REFERENCES | CASCADE | PREVENT\n" + + " [ERROR_MESSAGE '']\n" + + "DESCRIBE emits the ON DELETE form, because it says which side is governed.", + Example: "CREATE ASSOCIATION Shop.Order_Customer\n" + + " FROM Shop.Order TO Shop.Customer\n" + + " TYPE Reference\n" + + " ON DELETE RESTRICT\n" + + " ERROR_MESSAGE 'A customer with orders cannot be deleted';\n\n" + + "CREATE ASSOCIATION Shop.Order_Lines\n" + + " FROM Shop.OrderLine TO Shop.Order\n" + + " TYPE Reference\n" + + " ON DELETE CASCADE;\n\n" + + "ALTER ASSOCIATION Shop.Order_Customer SET ON DELETE SET NULL;", SeeAlso: []string{"domain-model.association.create"}, }) diff --git a/mdl-examples/bug-tests/captrack-1-on-delete-restrict-message.mdl b/mdl-examples/bug-tests/captrack-1-on-delete-restrict-message.mdl new file mode 100644 index 000000000..8599a5ef2 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-1-on-delete-restrict-message.mdl @@ -0,0 +1,73 @@ +-- CapTrackV2 FINDINGS §1 — `DELETE_BEHAVIOR PREVENT` wrote an app whose RUNTIME +-- WILL NOT START. Not a build error; `mx check` reports 0 errors either way. +-- +-- ERROR - M2EE: An error occurred while initializing the Runtime: None.get +-- java.util.NoSuchElementException: None.get +-- at …SchemeFactory$.…$setDeleteBehavior(SchemeFactory.scala:515) +-- +-- mxcli wrote ChildDeleteBehavior "DeleteMeIfNoReferences" with a NULL +-- ChildErrorMessage. Studio Pro writes a Texts$Text there — the field its +-- association dialog only reveals once that behaviour is selected, which is why +-- it was missed. Measured on a Studio Pro reference (ako/TestApp, +-- Mappings.Order_Customer, Mendix 11), the stored shape is: +-- +-- "ChildDeleteBehavior": "DeleteMeIfNoReferences", +-- "ChildErrorMessage": { "$Type": "Texts$Text", +-- "Items": [ 3, { "$Type": "Texts$Translation", +-- "LanguageCode": "en_US", +-- "Text": "…" } ] }, +-- "ParentErrorMessage": null +-- +-- A census of 47,789 units across 122 projects found 423 delete behaviours and +-- NOT ONE using this one — so no reference existed anywhere until one was made, +-- which is exactly how the null shipped. +-- +-- The syntax is SQL's. Mendix's three behaviours ARE SQL's referential actions, +-- and MDL's FROM/TO already matches a foreign key's direction (measured on the +-- same reference: ParentPointer -> Order, the FK owner and the FROM; +-- ChildPointer -> Customer, referenced and the TO). So this reads the way it +-- does in CREATE TABLE, with no Mendix-specific knowledge. +-- +-- Verify: +-- mxcli exec captrack-1-on-delete-restrict-message.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe association ODR.Order_Customer" +-- -- must emit: on delete restrict error_message '…' +-- mxcli run --local -p app.mpr -- boots; before the fix, None.get at startup + +create module ODR; +/ +create persistent entity ODR.Customer (Name: string(200)); +/ +create persistent entity ODR.Order (OrderNo: string(50)); +/ + +-- The reported case. Deleting a CUSTOMER is refused while Orders reference it. +create association ODR.Order_Customer + from ODR.Order to ODR.Customer + type Reference + on delete restrict + error_message 'A customer with orders cannot be deleted'; +/ + +-- CONTROL: the other two actions must NOT get a message element — Studio Pro +-- leaves both null, as all 423 behaviours in the wild are. +create association ODR.Order_Cascade + from ODR.Order to ODR.Customer + type Reference + on delete cascade; +/ +create association ODR.Order_SetNull + from ODR.Order to ODR.Customer + type Reference + on delete set null; +/ + +-- CONTROL: the older spelling still parses and still means the same thing, and +-- can carry the message too, so an existing script gains the fix without being +-- rewritten. +create association ODR.Order_Legacy + from ODR.Order to ODR.Customer + type Reference + delete_behavior prevent + error_message 'still referenced'; +/ diff --git a/mdl/ast/ast_association.go b/mdl/ast/ast_association.go index 9cd6468f2..0dfa7e8c9 100644 --- a/mdl/ast/ast_association.go +++ b/mdl/ast/ast_association.go @@ -49,29 +49,27 @@ func (o OwnerType) String() string { // DeleteBehavior represents the delete behavior of an association. type DeleteBehavior int +// Mendix has exactly three, and so does SQL: SET NULL, CASCADE, RESTRICT. +// +// Three further values used to sit here — DeleteBoth, DeleteKeepParentDeleteChild +// and DeleteKeepChildDeleteParent — which no grammar rule could produce and which +// named nothing Mendix has. They were the trap behind upstream #901: String() is a +// DISPLAY helper, and using it as a storage encoding put an out-of-domain enum on +// disk, which mxbuild tolerates and Studio Pro does not. const ( - DeleteKeepReferences DeleteBehavior = iota - DeleteCascade - DeleteBoth - DeleteKeepParentDeleteChild - DeleteKeepChildDeleteParent - DeleteIfNoReferences + DeleteKeepReferences DeleteBehavior = iota // ON DELETE SET NULL + DeleteCascade // ON DELETE CASCADE + DeleteIfNoReferences // ON DELETE RESTRICT ) +// String is for display. The storage encoding is storageDeleteBehavior in the +// executor — do not reintroduce this as one. func (d DeleteBehavior) String() string { switch d { - case DeleteKeepReferences: - return "DeleteMeButKeepReferences" case DeleteCascade: return "DeleteMeAndReferences" - case DeleteBoth: - return "DeleteBoth" - case DeleteKeepParentDeleteChild: - return "KeepParentDeleteChild" - case DeleteKeepChildDeleteParent: - return "KeepChildDeleteParent" case DeleteIfNoReferences: - return "DeleteIfNoReferences" + return "DeleteMeIfNoReferences" default: return "DeleteMeButKeepReferences" } @@ -99,17 +97,22 @@ func (s StorageType) String() string { // CreateAssociationStmt represents: CREATE ASSOCIATION Module.Name FROM ... TO ... TYPE ... type CreateAssociationStmt struct { - Name QualifiedName - Parent QualifiedName - Child QualifiedName - Type AssociationType - Owner OwnerType - Storage StorageType - DeleteBehavior DeleteBehavior - Documentation string - DocumentationSet bool // see mendixlabs/mxcli#1018: absent preserves, empty clears - Comment string - CreateOrModify bool // true for CREATE OR MODIFY / CREATE OR REPLACE + Name QualifiedName + Parent QualifiedName + Child QualifiedName + Type AssociationType + Owner OwnerType + Storage StorageType + DeleteBehavior DeleteBehavior + // DeleteErrorMessage is the text a user sees when a RESTRICT/PREVENT delete + // is refused — Studio Pro's "Error message if 'X' object cannot be deleted". + // Stored as a Texts$Text; without one the runtime fails to START, not to + // build (CapTrackV2 §1). + DeleteErrorMessage string + Documentation string + DocumentationSet bool // see mendixlabs/mxcli#1018: absent preserves, empty clears + Comment string + CreateOrModify bool // true for CREATE OR MODIFY / CREATE OR REPLACE // IfNotExists is CREATE ASSOCIATION IF NOT EXISTS: skip when it already // exists, leaving the stored definition untouched. IfNotExists bool @@ -149,9 +152,12 @@ type AlterAssociationStmt struct { Name QualifiedName Operation AlterAssociationOperation DeleteBehavior DeleteBehavior - Owner OwnerType - Storage StorageType - Comment string + // DeleteErrorMessage: see CreateAssociationStmt. An ALTER that sets + // RESTRICT/PREVENT needs it for the same reason a CREATE does. + DeleteErrorMessage string + Owner OwnerType + Storage StorageType + Comment string // SET ANCHOR FROM (x, y) TO (x, y) — both ends are always given together, // because the pair is one visual decision. diff --git a/mdl/backend/modelsdk/domainmodel.go b/mdl/backend/modelsdk/domainmodel.go index 5ac64a6d6..cedef3750 100644 --- a/mdl/backend/modelsdk/domainmodel.go +++ b/mdl/backend/modelsdk/domainmodel.go @@ -576,7 +576,13 @@ func assocFromGen(a *genDm.Association) *domainmodel.Association { out.ID = model.ID(a.ID()) if db, ok := a.DeleteBehavior().(*genDm.AssociationDeleteBehavior); ok && db != nil { out.ParentDeleteBehavior = &domainmodel.DeleteBehavior{Type: domainmodel.DeleteBehaviorType(db.ParentDeleteBehavior())} - out.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: domainmodel.DeleteBehaviorType(db.ChildDeleteBehavior())} + out.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorType(db.ChildDeleteBehavior()), + // Read the refusal message back too, or DESCRIBE cannot emit it and a + // describe -> exec round trip writes an association whose runtime will + // not start (CapTrackV2 §1). + ErrorMessage: deleteErrorMessageFromGen(db.ChildErrorMessage()), + } } // Read the external (OData) source back. RemoteParentNavigationProperty in @@ -614,7 +620,35 @@ func crossAssocFromGen(ca *genDm.CrossAssociation) *domainmodel.CrossModuleAssoc out.ID = model.ID(ca.ID()) if db, ok := ca.DeleteBehavior().(*genDm.AssociationDeleteBehavior); ok && db != nil { out.ParentDeleteBehavior = &domainmodel.DeleteBehavior{Type: domainmodel.DeleteBehaviorType(db.ParentDeleteBehavior())} - out.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: domainmodel.DeleteBehaviorType(db.ChildDeleteBehavior())} + out.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorType(db.ChildDeleteBehavior()), + // Read the refusal message back too, or DESCRIBE cannot emit it and a + // describe -> exec round trip writes an association whose runtime will + // not start (CapTrackV2 §1). + ErrorMessage: deleteErrorMessageFromGen(db.ChildErrorMessage()), + } } return out } + +// deleteErrorMessageFromGen reads the en_US text out of a delete behaviour's +// error message, or "" when there is none. The message is a Texts$Text like any +// caption; MDL carries one string, and CarryTranslations puts the other +// languages back on a rewrite. +func deleteErrorMessageFromGen(el element.Element) string { + txt, ok := el.(*genTexts.Text) + if !ok || txt == nil { + return "" + } + t := textFromGen(txt) + if t == nil { + return "" + } + if v, ok := t.Translations["en_US"]; ok { + return v + } + for _, v := range t.Translations { + return v + } + return "" +} diff --git a/mdl/backend/modelsdk/domainmodel_delete_message_test.go b/mdl/backend/modelsdk/domainmodel_delete_message_test.go new file mode 100644 index 000000000..367990a5f --- /dev/null +++ b/mdl/backend/modelsdk/domainmodel_delete_message_test.go @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// CapTrackV2 FINDINGS §1 — an association with PREVENT stopped the runtime +// STARTING, which is worse than a build error and names nothing about the model: +// +// ERROR - M2EE: An error occurred while initializing the Runtime: None.get +// at …SchemeFactory$.…$setDeleteBehavior(SchemeFactory.scala:515) +// +// mxcli wrote ChildDeleteBehavior "DeleteMeIfNoReferences" with a null +// ChildErrorMessage. Studio Pro writes a Texts$Text there — the field the +// association dialog only reveals once that behaviour is selected, which is why +// it went unnoticed. Measured on a Studio Pro reference (ako/TestApp, +// Mappings.Order_Customer, Mendix 11): +// +// "ChildDeleteBehavior": "DeleteMeIfNoReferences", +// "ChildErrorMessage": { "$Type": "Texts$Text", +// "Items": [ 3, { "$Type": "Texts$Translation", +// "LanguageCode": "en_US", +// "Text": "Aha here it is" } ] }, +// "ParentErrorMessage": null +// +// Two things that reference pins down and no amount of reasoning would have: +// the item collection's typed-array marker is **3**, and the OTHER side stays +// null because that side is still "keep". A census of 47,789 units across 122 +// projects found 423 delete behaviours and NOT ONE using this behaviour — so +// there was no reference anywhere until one was made, which is exactly why the +// null shipped. + +func assocWithBehaviour(t *testing.T, behaviour domainmodel.DeleteBehaviorType, msg string) map[string]any { + t.Helper() + a := &domainmodel.Association{ + Name: "Order_Customer", + ChildDeleteBehavior: &domainmodel.DeleteBehavior{Type: behaviour, ErrorMessage: msg}, + } + gen := assocToGen(a) + if gen == nil { + t.Fatal("assocToGen returned nil") + } + dbEl := gen.DeleteBehavior() + if dbEl == nil { + t.Fatal("no DeleteBehavior on the generated association") + } + db, ok := dbEl.(*genDm.AssociationDeleteBehavior) + if !ok { + t.Fatalf("DeleteBehavior is %T", dbEl) + } + return map[string]any{ + "child": db.ChildDeleteBehavior(), + "childMsg": db.ChildErrorMessage(), + "parent": db.ParentDeleteBehavior(), + "parentMsg": db.ParentErrorMessage(), + } +} + +// The reported case: RESTRICT must carry a message element, not null. +func TestDeleteBehaviour_RestrictWritesTheErrorMessage(t *testing.T) { + got := assocWithBehaviour(t, domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + "A customer with orders cannot be deleted") + + if got["child"] != "DeleteMeIfNoReferences" { + t.Fatalf("child behaviour = %v", got["child"]) + } + if got["childMsg"] == nil { + t.Fatal("ChildErrorMessage is nil — this is the document that stops the runtime starting") + } +} + +// CONTROL 1: the other two behaviours keep a null message, which is what the +// reference's two untouched associations show. Writing a text element for every +// association would differ from Studio Pro on the 423 behaviours that exist in +// the wild and none of which have one. +func TestDeleteBehaviour_OtherBehavioursStayNull(t *testing.T) { + for _, b := range []domainmodel.DeleteBehaviorType{ + domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences, + domainmodel.DeleteBehaviorTypeDeleteMeAndReferences, + } { + got := assocWithBehaviour(t, b, "") + if got["childMsg"] != nil { + t.Errorf("%s wrote a ChildErrorMessage; Studio Pro leaves it null", b) + } + } +} + +// CONTROL 2: the PARENT side is untouched. MDL only ever sets the child side, +// and the reference confirms ParentErrorMessage stays null even when the child +// carries a message. +func TestDeleteBehaviour_ParentSideUntouched(t *testing.T) { + got := assocWithBehaviour(t, domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, "blocked") + if got["parent"] != "DeleteMeButKeepReferences" { + t.Errorf("parent behaviour = %v, want the default", got["parent"]) + } +} + +// A message on a behaviour that has no use for one is not written. Mendix only +// reads it for the restrict case, and Studio Pro cannot even author one there. +func TestDeleteBehaviour_MessageIgnoredOnOtherBehaviours(t *testing.T) { + got := assocWithBehaviour(t, domainmodel.DeleteBehaviorTypeDeleteMeAndReferences, "not applicable") + if got["childMsg"] != nil { + t.Error("a message was written for CASCADE, which Studio Pro cannot author") + } +} diff --git a/mdl/backend/modelsdk/domainmodel_write.go b/mdl/backend/modelsdk/domainmodel_write.go index 83fcf656f..0f2867123 100644 --- a/mdl/backend/modelsdk/domainmodel_write.go +++ b/mdl/backend/modelsdk/domainmodel_write.go @@ -116,6 +116,22 @@ func assocToGen(a *domainmodel.Association) *genDm.Association { } db.SetParentDeleteBehavior(parentDB) db.SetChildDeleteBehavior(childDB) + // A "delete me if no references" child side carries the message the user sees + // when the delete is refused. Studio Pro writes a Texts$Text there and mxcli + // wrote null, which does not fail the BUILD — it stops the runtime STARTING, + // with a stack trace naming SchemeFactory and nothing about the model + // (CapTrackV2 §1). + // + // Only that behaviour, and only the child side. A census of 47,789 units + // across 122 projects found 423 delete behaviours, every one of them keep or + // cascade with both messages null — so writing a text element unconditionally + // would differ from Studio Pro on every association that exists in the wild. + // The Studio Pro reference (ako/TestApp, Mappings.Order_Customer) shows the + // same: the message appears on the restrict side, and ParentErrorMessage + // stays null. + if childDB == string(domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences) { + db.SetChildErrorMessage(textToGen(deleteErrorText(a.ChildDeleteBehavior))) + } out.SetDeleteBehavior(db) // An association between external entities carries a Rest$OData* source; a @@ -848,3 +864,22 @@ func assignID(elem element.Element) { } ider.SetID(element.ID(mmpr.GenerateID())) } + +// deleteErrorText is the message element a restrict behaviour must carry. +// +// An omitted message becomes an empty en_US translation rather than a null: null +// is the shape that stops the runtime starting, and Studio Pro's dialog lets the +// box be left empty, so an empty text is the closest thing to "no message" that +// still boots. +// +// NOTE: the empty case is INFERRED, not measured. The reference document +// (ako/TestApp) captures a message that was filled in; nobody has yet saved a +// restrict association with the box cleared, so what Studio Pro writes then is +// unverified. If this ever misbehaves, that is the line to check first. +func deleteErrorText(db *domainmodel.DeleteBehavior) *model.Text { + msg := "" + if db != nil { + msg = db.ErrorMessage + } + return &model.Text{Translations: map[string]string{"en_US": msg}} +} diff --git a/mdl/executor/cmd_associations.go b/mdl/executor/cmd_associations.go index c9189c445..4dce1c3f0 100644 --- a/mdl/executor/cmd_associations.go +++ b/mdl/executor/cmd_associations.go @@ -72,6 +72,7 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error } deleteBehavior := storageDeleteBehavior(s.DeleteBehavior) + deleteMessage := s.DeleteErrorMessage // Convert storage type (default: Column = foreign key in parent table) storageFormat := domainmodel.StorageFormatColumn @@ -96,7 +97,7 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error assoc.Type = assocType assoc.Owner = owner assoc.StorageFormat = storageFormat - assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior} + assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior, ErrorMessage: deleteMessage} assoc.Documentation = carriedDocumentation( associationDocumentationStated(s), associationDocumentation(s), assoc.Documentation) // Anchors are applied only when the statement names them — @@ -120,7 +121,7 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error ca.Type = assocType ca.Owner = owner ca.StorageFormat = storageFormat - ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior} + ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior, ErrorMessage: deleteMessage} ca.ChildRef = childRef ca.Documentation = carriedDocumentation( associationDocumentationStated(s), associationDocumentation(s), ca.Documentation) @@ -172,7 +173,8 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error ParentID: parentID, ChildRef: childRef, ChildDeleteBehavior: &domainmodel.DeleteBehavior{ - Type: deleteBehavior, + Type: deleteBehavior, + ErrorMessage: deleteMessage, }, } if err := ctx.Backend.CreateCrossAssociation(dm.ID, ca); err != nil { @@ -203,7 +205,8 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error ParentID: parentID, ChildID: childID, ChildDeleteBehavior: &domainmodel.DeleteBehavior{ - Type: deleteBehavior, + Type: deleteBehavior, + ErrorMessage: deleteMessage, }, } applyAnchors(assoc, s.FromAnchor, s.ToAnchor) @@ -251,7 +254,8 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { switch s.Operation { case ast.AlterAssociationSetDeleteBehavior: assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: storageDeleteBehavior(s.DeleteBehavior), + Type: storageDeleteBehavior(s.DeleteBehavior), + ErrorMessage: s.DeleteErrorMessage, } case ast.AlterAssociationSetOwner: assoc.Owner = domainmodel.AssociationOwner(s.Owner.String()) @@ -276,7 +280,8 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { switch s.Operation { case ast.AlterAssociationSetDeleteBehavior: ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: storageDeleteBehavior(s.DeleteBehavior), + Type: storageDeleteBehavior(s.DeleteBehavior), + ErrorMessage: s.DeleteErrorMessage, } case ast.AlterAssociationSetOwner: ca.Owner = domainmodel.AssociationOwner(s.Owner.String()) @@ -538,18 +543,7 @@ func describeAssociation(ctx *ExecContext, name ast.QualifiedName) error { // canonical name, so cascade was the odd one out and a describe → edit → // exec loop died on it (upstream #901). The round-trip test in // cmd_associations_delete_behavior_test.go feeds this back through the parser. - deleteBehavior := "DELETE_BUT_KEEP_REFERENCES" - if childDeleteBehavior != nil { - switch childDeleteBehavior.Type { - case domainmodel.DeleteBehaviorTypeDeleteMeAndReferences: - deleteBehavior = "DELETE_AND_REFERENCES" - case domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences: - deleteBehavior = "DELETE_IF_NO_REFERENCES" - case domainmodel.DeleteBehaviorTypeDeleteMeButKeepReferences: - deleteBehavior = "DELETE_BUT_KEEP_REFERENCES" - } - } - fmt.Fprintf(ctx.Output, "delete_behavior %s;\n", deleteBehavior) + fmt.Fprintf(ctx.Output, "%s;\n", describeDeleteClause(childDeleteBehavior)) } for _, assoc := range dm.Associations { @@ -717,3 +711,31 @@ func associationDocumentation(s *ast.CreateAssociationStmt) string { } return s.Comment } + +// describeDeleteClause renders a child delete behaviour as MDL. +// +// It emits the SQL spelling, because that is the one that says what the +// behaviour DOES: `ON DELETE RESTRICT` on a FROM/TO pair reads the way a foreign +// key does, while `DELETE_IF_NO_REFERENCES` leaves a reader to work out which +// side is governed. The old spelling still parses, so nothing that already +// exists breaks — this only changes what DESCRIBE chooses to write. +// +// The message is emitted whenever there is one. Dropping it would make a +// describe -> exec round trip produce an association whose runtime does not +// start, which is the failure this whole clause exists to prevent (CapTrackV2 +// §1) — and the round trip is exactly how these scripts get regenerated. +func describeDeleteClause(db *domainmodel.DeleteBehavior) string { + action := "on delete set null" + if db != nil { + switch db.Type { + case domainmodel.DeleteBehaviorTypeDeleteMeAndReferences: + action = "on delete cascade" + case domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences: + action = "on delete restrict" + } + } + if db != nil && db.ErrorMessage != "" { + return action + " error_message " + mdlQuote(db.ErrorMessage) + } + return action +} diff --git a/mdl/executor/cmd_associations_delete_behavior_test.go b/mdl/executor/cmd_associations_delete_behavior_test.go index 7d4380f02..8e11a7e52 100644 --- a/mdl/executor/cmd_associations_delete_behavior_test.go +++ b/mdl/executor/cmd_associations_delete_behavior_test.go @@ -4,6 +4,7 @@ package executor import ( "bytes" + "strings" "testing" "github.com/mendixlabs/mxcli/mdl/ast" @@ -170,3 +171,57 @@ func assertStoredBehavior(t *testing.T, assoc *domainmodel.Association, want dom t.Errorf("stored %q, want %q", got, want) } } + +// The message must survive DESCRIBE, or a describe -> exec round trip writes an +// association whose RUNTIME does not start — and regenerating scripts that way +// is exactly how these models are maintained (CapTrackV2 §1). +func TestDescribeAssociation_RoundTripsTheDeleteErrorMessage(t *testing.T) { + const msg = "A customer with orders cannot be deleted" + ctx, _ := assocFixture(t) + assertNoError(t, execCreateAssociation(ctx, &ast.CreateAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Parent: ast.QualifiedName{Module: "M", Name: "Child"}, + Child: ast.QualifiedName{Module: "M", Name: "Parent"}, + Type: ast.AssocReference, + DeleteBehavior: ast.DeleteIfNoReferences, + DeleteErrorMessage: msg, + CreateOrModify: true, + })) + + var buf bytes.Buffer + ctx.Output = &buf + assertNoError(t, describeAssociation(ctx, ast.QualifiedName{Module: "M", Name: "Child_Parent"})) + + prog, errs := visitor.Build(buf.String()) + if len(errs) > 0 { + t.Fatalf("DESCRIBE emitted MDL the parser rejects: %v\n--- output ---\n%s", errs, buf.String()) + } + stmt := prog.Statements[0].(*ast.CreateAssociationStmt) + if stmt.DeleteBehavior != ast.DeleteIfNoReferences { + t.Errorf("behaviour = %v, want DeleteIfNoReferences", stmt.DeleteBehavior) + } + if stmt.DeleteErrorMessage != msg { + t.Errorf("message = %q, want %q\n--- output ---\n%s", stmt.DeleteErrorMessage, msg, buf.String()) + } +} + +// CONTROL: a behaviour with no message emits no ERROR_MESSAGE clause. An empty +// one would re-execute into a message that is not what the author wrote. +func TestDescribeAssociation_NoMessageEmitsNoClause(t *testing.T) { + ctx, _ := assocFixture(t) + assertNoError(t, execCreateAssociation(ctx, &ast.CreateAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Parent: ast.QualifiedName{Module: "M", Name: "Child"}, + Child: ast.QualifiedName{Module: "M", Name: "Parent"}, + Type: ast.AssocReference, + DeleteBehavior: ast.DeleteCascade, + CreateOrModify: true, + })) + + var buf bytes.Buffer + ctx.Output = &buf + assertNoError(t, describeAssociation(ctx, ast.QualifiedName{Module: "M", Name: "Child_Parent"})) + if strings.Contains(strings.ToLower(buf.String()), "error_message") { + t.Errorf("emitted an ERROR_MESSAGE clause for a cascade:\n%s", buf.String()) + } +} diff --git a/mdl/grammar/MDLLexer.g4 b/mdl/grammar/MDLLexer.g4 index 25beda41c..f40d3b7f4 100644 --- a/mdl/grammar/MDLLexer.g4 +++ b/mdl/grammar/MDLLexer.g4 @@ -91,6 +91,23 @@ TABLE: T A B L E; // Delete behavior keywords DELETE_BEHAVIOR: D E L E T E '_'? B E H A V I O R; + +// SQL referential actions. Mendix's three delete behaviours are exactly SQL's +// CASCADE / RESTRICT / SET NULL, and MDL's FROM/TO already matches a foreign +// key's direction — FROM owns the key, TO is referenced — so `ON DELETE +// RESTRICT` reads the way it does in `CREATE TABLE` with no Mendix-specific +// knowledge. Verified against a Studio Pro reference (ako/TestApp +// Mappings.Order_Customer): ParentPointer -> Order (FK owner, the FROM), +// ChildPointer -> Customer (referenced, the TO). +// +// ERROR_MESSAGE, not ERROR: `ERROR` is already a token, and the compound name +// mirrors Studio Pro's own label ("Error message if 'X' object cannot be +// deleted"). SQL's RESTRICT has no custom message, so this clause is a Mendix +// extension rather than something borrowed. +RESTRICT: R E S T R I C T; +ERROR_MESSAGE: E R R O R WS+ M E S S A G E + | E R R O R '_' M E S S A G E + | E R R O R M E S S A G E; CASCADE: C A S C A D E; PREVENT: P R E V E N T; diff --git a/mdl/grammar/domains/MDLDomainModel.g4 b/mdl/grammar/domains/MDLDomainModel.g4 index cf9f4516f..efc354c8d 100644 --- a/mdl/grammar/domains/MDLDomainModel.g4 +++ b/mdl/grammar/domains/MDLDomainModel.g4 @@ -185,10 +185,37 @@ associationOption : TYPE COLON? (REFERENCE | REFERENCE_SET) | OWNER COLON? (DEFAULT | BOTH) | STORAGE COLON? (COLUMN | TABLE) - | DELETE_BEHAVIOR deleteBehavior + | DELETE_BEHAVIOR deleteBehavior errorMessageClause? + | onDeleteClause | COMMENT STRING_LITERAL ; +// The SQL spelling. Mendix's three delete behaviours ARE SQL's referential +// actions, and MDL's FROM/TO already matches a foreign key's direction, so this +// reads exactly as it does in CREATE TABLE: +// +// FROM Shop.Order TO Shop.Customer ON DELETE RESTRICT +// -> deleting a Customer is refused while Orders reference it +// +// The older DELETE_BEHAVIOR clause above keeps working unchanged; it is the one +// that gives no clue which side it governs, which is why this exists. +onDeleteClause + : ON DELETE referentialAction errorMessageClause? + ; + +referentialAction + : CASCADE // DeleteMeAndReferences + | RESTRICT // DeleteMeIfNoReferences + | SET NULL // DeleteMeButKeepReferences (default) + ; + +// SQL's RESTRICT raises a system error; Mendix lets the modeller write the text +// the user sees. Only meaningful with RESTRICT/PREVENT — the executor reports it +// on the others rather than dropping it silently. +errorMessageClause + : ERROR_MESSAGE STRING_LITERAL + ; + deleteBehavior : DELETE_AND_REFERENCES | DELETE_BUT_KEEP_REFERENCES @@ -242,7 +269,8 @@ ifExists ; alterAssociationAction - : SET DELETE_BEHAVIOR deleteBehavior + : SET DELETE_BEHAVIOR deleteBehavior errorMessageClause? + | SET onDeleteClause | SET OWNER (DEFAULT | BOTH) | SET STORAGE (COLUMN | TABLE) | SET COMMENT STRING_LITERAL diff --git a/mdl/grammar/domains/MDLSettings.g4 b/mdl/grammar/domains/MDLSettings.g4 index 62c67beab..c4479d697 100644 --- a/mdl/grammar/domains/MDLSettings.g4 +++ b/mdl/grammar/domains/MDLSettings.g4 @@ -546,7 +546,7 @@ keyword | ASSOCIATION | ASSOCIATIONS | CALCULATED | CONSTANT | CONSTANTS | ENTITY | ENTITIES | ENUMERATION | ENUMERATIONS | GENERALIZATION | EXTENDS | INDEX | PERSISTENT | NON_PERSISTENT | REFERENCE | REFERENCE_SET | STORAGE | TABLE | UNIQUE - | CASCADE | PREVENT | DELETE_BEHAVIOR | DELETE_AND_REFERENCES + | CASCADE | PREVENT | RESTRICT | ERROR_MESSAGE | DELETE_BEHAVIOR | DELETE_AND_REFERENCES | DELETE_BUT_KEEP_REFERENCES | DELETE_IF_NO_REFERENCES | ALLOW_CREATE_CHANGE_LOCALLY | CHANGED | CREATED diff --git a/mdl/visitor/visitor_association.go b/mdl/visitor/visitor_association.go index 8fb3a1fd4..2a121cbc2 100644 --- a/mdl/visitor/visitor_association.go +++ b/mdl/visitor/visitor_association.go @@ -62,11 +62,24 @@ func (b *Builder) ExitCreateAssociationStatement(ctx *parser.CreateAssociationSt } } - // DELETE_BEHAVIOR + // DELETE_BEHAVIOR , the original spelling. if delBehavior := optCtx.DeleteBehavior(); delBehavior != nil { stmt.DeleteBehavior = buildDeleteBehavior(delBehavior) } + // ON DELETE , the SQL spelling. Both set the same field — + // they are two ways of naming one Mendix behaviour, not two features. + if onDel := optCtx.OnDeleteClause(); onDel != nil { + stmt.DeleteBehavior = buildReferentialAction(onDel.ReferentialAction()) + if msg := onDel.ErrorMessageClause(); msg != nil { + stmt.DeleteErrorMessage = buildErrorMessage(msg) + } + } + // ERROR_MESSAGE hangs off either clause. + if msg := optCtx.ErrorMessageClause(); msg != nil { + stmt.DeleteErrorMessage = buildErrorMessage(msg) + } + // COMMENT if optCtx.COMMENT() != nil && optCtx.STRING_LITERAL() != nil { stmt.Comment = unquoteString(optCtx.STRING_LITERAL().GetText()) @@ -191,15 +204,31 @@ func (b *Builder) ExitAlterAssociationAction(ctx *parser.AlterAssociationActionC } name := buildQualifiedName(qn) - // SET DELETE_BEHAVIOR + // SET DELETE_BEHAVIOR , and SET ON DELETE . if ctx.DELETE_BEHAVIOR() != nil { if delBehavior := ctx.DeleteBehavior(); delBehavior != nil { - b.statements = append(b.statements, &ast.AlterAssociationStmt{ + alter := &ast.AlterAssociationStmt{ Name: name, Operation: ast.AlterAssociationSetDeleteBehavior, DeleteBehavior: buildDeleteBehavior(delBehavior), - }) + } + if msg := ctx.ErrorMessageClause(); msg != nil { + alter.DeleteErrorMessage = buildErrorMessage(msg) + } + b.statements = append(b.statements, alter) + } + return + } + if onDel := ctx.OnDeleteClause(); onDel != nil { + alter := &ast.AlterAssociationStmt{ + Name: name, + Operation: ast.AlterAssociationSetDeleteBehavior, + DeleteBehavior: buildReferentialAction(onDel.ReferentialAction()), + } + if msg := onDel.ErrorMessageClause(); msg != nil { + alter.DeleteErrorMessage = buildErrorMessage(msg) } + b.statements = append(b.statements, alter) return } diff --git a/mdl/visitor/visitor_helpers.go b/mdl/visitor/visitor_helpers.go index 5fdc42703..e4c557eb0 100644 --- a/mdl/visitor/visitor_helpers.go +++ b/mdl/visitor/visitor_helpers.go @@ -713,3 +713,45 @@ func stripMDLComments(s string) string { // ---------------------------------------------------------------------------- // ExitCreateMicroflowStatement is called when exiting the createMicroflowStatement production. + +// buildReferentialAction maps SQL's referential actions onto Mendix's three +// delete behaviours. They correspond exactly: +// +// ON DELETE CASCADE -> DeleteMeAndReferences +// ON DELETE RESTRICT -> DeleteMeIfNoReferences +// ON DELETE SET NULL -> DeleteMeButKeepReferences (Mendix's default) +// +// Every alternative the referentialAction rule admits is read explicitly, for +// the reason buildDeleteBehavior spells out above: a token that falls through to +// a zero value is a LEGAL behaviour, so the substitution is silent all the way +// to disk and overwrites whatever the association had (upstream #901). Adding an +// alternative to the rule means adding it here. The default below is unreachable +// while the rule has three alternatives, and is not a fallback to rely on. +func buildReferentialAction(ctx parser.IReferentialActionContext) ast.DeleteBehavior { + if ctx == nil { + return ast.DeleteKeepReferences + } + ra := ctx.(*parser.ReferentialActionContext) + switch { + case ra.CASCADE() != nil: + return ast.DeleteCascade + case ra.RESTRICT() != nil: + return ast.DeleteIfNoReferences + case ra.SET() != nil && ra.NULL() != nil: + return ast.DeleteKeepReferences + default: + return ast.DeleteKeepReferences + } +} + +// buildErrorMessage reads the text of an ERROR_MESSAGE clause. +func buildErrorMessage(ctx parser.IErrorMessageClauseContext) string { + if ctx == nil { + return "" + } + emc, ok := ctx.(*parser.ErrorMessageClauseContext) + if !ok || emc.STRING_LITERAL() == nil { + return "" + } + return unquoteString(emc.STRING_LITERAL().GetText()) +} diff --git a/mdl/visitor/visitor_on_delete_test.go b/mdl/visitor/visitor_on_delete_test.go new file mode 100644 index 000000000..7d6118670 --- /dev/null +++ b/mdl/visitor/visitor_on_delete_test.go @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// CapTrackV2 FINDINGS §1 — `DELETE_BEHAVIOR PREVENT` wrote an app that will not +// start: +// +// ERROR - M2EE: An error occurred while initializing the Runtime: None.get +// java.util.NoSuchElementException: None.get +// at …SchemeFactory$.…$setDeleteBehavior(SchemeFactory.scala:515) +// +// mxcli wrote ChildDeleteBehavior "DeleteMeIfNoReferences" with a null +// ChildErrorMessage, and MDL had no way to say the message. A Studio Pro +// reference (ako/TestApp, Mappings.Order_Customer) shows what the modeller +// writes instead — an ordinary Texts$Text: +// +// "ChildDeleteBehavior": "DeleteMeIfNoReferences", +// "ChildErrorMessage": { "$Type": "Texts$Text", +// "Items": [ 3, { "$Type": "Texts$Translation", +// "LanguageCode": "en_US", +// "Text": "Aha here it is" } ] } +// +// The syntax is SQL's, because Mendix's three behaviours ARE SQL's referential +// actions and MDL's FROM/TO already matches a foreign key's direction (measured +// on that same reference: ParentPointer -> Order, the FK owner and the FROM; +// ChildPointer -> Customer, referenced and the TO). So `ON DELETE RESTRICT` +// reads the way it does in CREATE TABLE, and needs no knowledge of which side +// Mendix calls the child — which is the complaint the old spelling earned. + +func parseAssoc(t *testing.T, src string) *ast.CreateAssociationStmt { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors for %q: %v", src, errs) + } + if len(prog.Statements) != 1 { + t.Fatalf("got %d statements, want 1", len(prog.Statements)) + } + stmt, ok := prog.Statements[0].(*ast.CreateAssociationStmt) + if !ok { + t.Fatalf("got %T, want *ast.CreateAssociationStmt", prog.Statements[0]) + } + return stmt +} + +// Each SQL action maps onto exactly one Mendix behaviour. +func TestOnDelete_MapsTheSQLReferentialActions(t *testing.T) { + cases := []struct { + clause string + want ast.DeleteBehavior + }{ + {"ON DELETE CASCADE", ast.DeleteCascade}, + {"ON DELETE RESTRICT", ast.DeleteIfNoReferences}, + {"ON DELETE SET NULL", ast.DeleteKeepReferences}, + } + for _, c := range cases { + stmt := parseAssoc(t, "CREATE ASSOCIATION Shop.Order_Customer FROM Shop.Order TO Shop.Customer "+c.clause+";") + if stmt.DeleteBehavior != c.want { + t.Errorf("%q gave %v, want %v", c.clause, stmt.DeleteBehavior, c.want) + } + } +} + +// The message the runtime needs, and the thing MDL could not say at all. +func TestOnDelete_CarriesTheErrorMessage(t *testing.T) { + stmt := parseAssoc(t, "CREATE ASSOCIATION Shop.Order_Customer FROM Shop.Order TO Shop.Customer "+ + "ON DELETE RESTRICT ERROR_MESSAGE 'A customer with orders cannot be deleted';") + + if stmt.DeleteBehavior != ast.DeleteIfNoReferences { + t.Fatalf("behaviour = %v, want DeleteIfNoReferences", stmt.DeleteBehavior) + } + if stmt.DeleteErrorMessage != "A customer with orders cannot be deleted" { + t.Errorf("message = %q, want the authored text", stmt.DeleteErrorMessage) + } +} + +// CONTROL: the older spelling still parses and still means the same thing. It is +// in every script written before this, and breaking it to improve readability +// would be a poor trade. +func TestOnDelete_LegacyDeleteBehaviorStillWorks(t *testing.T) { + for _, src := range []string{ + "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B DELETE_BEHAVIOR PREVENT;", + "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B DELETE_BEHAVIOR CASCADE;", + "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B DELETE_BEHAVIOR DELETE_IF_NO_REFERENCES;", + } { + parseAssoc(t, src) // a parse error fails the helper + } + stmt := parseAssoc(t, "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B DELETE_BEHAVIOR PREVENT;") + if stmt.DeleteBehavior != ast.DeleteIfNoReferences { + t.Errorf("PREVENT = %v, want DeleteIfNoReferences", stmt.DeleteBehavior) + } +} + +// …and it can carry the message too, so an existing script gains the fix without +// being rewritten to the new spelling. +func TestOnDelete_LegacySpellingTakesTheMessage(t *testing.T) { + stmt := parseAssoc(t, "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B "+ + "DELETE_BEHAVIOR PREVENT ERROR_MESSAGE 'still referenced';") + if stmt.DeleteErrorMessage != "still referenced" { + t.Errorf("message = %q", stmt.DeleteErrorMessage) + } +} + +// CONTROL: no clause at all is still the default, with no message. +func TestOnDelete_OmittedIsTheDefault(t *testing.T) { + stmt := parseAssoc(t, "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B;") + if stmt.DeleteBehavior != ast.DeleteKeepReferences { + t.Errorf("behaviour = %v, want the keep-references default", stmt.DeleteBehavior) + } + if stmt.DeleteErrorMessage != "" { + t.Errorf("message = %q, want empty", stmt.DeleteErrorMessage) + } +} + +// A quoted message containing a quote must survive, doubled the way MDL spells +// escapes everywhere else. +func TestOnDelete_MessageWithAQuote(t *testing.T) { + stmt := parseAssoc(t, "CREATE ASSOCIATION Shop.A_B FROM Shop.A TO Shop.B "+ + "ON DELETE RESTRICT ERROR_MESSAGE 'the customer''s orders block this';") + if !strings.Contains(stmt.DeleteErrorMessage, "customer's orders") { + t.Errorf("message = %q, want the apostrophe unescaped", stmt.DeleteErrorMessage) + } +} diff --git a/sdk/domainmodel/domainmodel.go b/sdk/domainmodel/domainmodel.go index 6b5cf0f89..5f5f4ae58 100644 --- a/sdk/domainmodel/domainmodel.go +++ b/sdk/domainmodel/domainmodel.go @@ -402,6 +402,15 @@ const ( type DeleteBehavior struct { model.BaseElement Type DeleteBehaviorType `json:"type"` + // ErrorMessage is the text the user sees when a DeleteMeIfNoReferences + // delete is refused — Studio Pro's "Error message if 'X' object cannot be + // deleted", which the dialog only shows once that behaviour is selected. + // + // It is stored as a Texts$Text, so it is translatable like any caption. Its + // absence is not cosmetic: an association with DeleteMeIfNoReferences and a + // null message stops the RUNTIME STARTING, which is a worse failure than a + // build error and names nothing about the model (CapTrackV2 §1). + ErrorMessage string `json:"errorMessage,omitempty"` } // DeleteBehaviorType represents the type of delete behavior. diff --git a/sdk/mpr/parser_domainmodel.go b/sdk/mpr/parser_domainmodel.go index d3843360a..1a6593705 100644 --- a/sdk/mpr/parser_domainmodel.go +++ b/sdk/mpr/parser_domainmodel.go @@ -453,6 +453,10 @@ func parseAssociation(raw map[string]any) *domainmodel.Association { if childType := extractString(deleteBehaviorRaw["ChildDeleteBehavior"]); childType != "" { assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ Type: domainmodel.DeleteBehaviorType(childType), + // Read the refusal message back, or DESCRIBE cannot emit it and a + // describe -> exec round trip rebuilds an association whose runtime + // will not start (CapTrackV2 §1). + ErrorMessage: deleteBehaviorErrorMessage(deleteBehaviorRaw["ChildErrorMessage"]), } } } @@ -503,7 +507,8 @@ func parseCrossAssociation(raw map[string]any) *domainmodel.CrossModuleAssociati } if childType := extractString(deleteBehaviorRaw["ChildDeleteBehavior"]); childType != "" { ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: domainmodel.DeleteBehaviorType(childType), + Type: domainmodel.DeleteBehaviorType(childType), + ErrorMessage: deleteBehaviorErrorMessage(deleteBehaviorRaw["ChildErrorMessage"]), } } } @@ -780,3 +785,31 @@ func parseEventHandler(raw map[string]any) *domainmodel.EventHandler { } // parseMicroflow parses microflow contents from BSON. + +// deleteBehaviorErrorMessage reads the en_US text out of a delete behaviour's +// error message. The message is an ordinary Texts$Text; MDL carries one string. +func deleteBehaviorErrorMessage(raw any) string { + m, ok := raw.(map[string]any) + if !ok { + return "" + } + items, ok := m["Items"].([]any) + if !ok { + return "" + } + first := "" + for _, it := range items { + tr, ok := it.(map[string]any) + if !ok { + continue + } + text := extractString(tr["Text"]) + if extractString(tr["LanguageCode"]) == "en_US" { + return text + } + if first == "" { + first = text + } + } + return first +} diff --git a/sdk/mpr/writer_domainmodel.go b/sdk/mpr/writer_domainmodel.go index 4186b035a..5874a8adb 100644 --- a/sdk/mpr/writer_domainmodel.go +++ b/sdk/mpr/writer_domainmodel.go @@ -1266,11 +1266,29 @@ func serializeDeleteBehavior(parentBehavior, childBehavior *domainmodel.DeleteBe childType = string(childBehavior.Type) } + // A "delete me if no references" child side carries the message the user sees + // when the delete is refused; every other behaviour leaves it null. Both were + // hardcoded null here, and for that one behaviour that produces a model whose + // RUNTIME will not start — `None.get` in SchemeFactory, with `mx check` + // reporting 0 errors either way (CapTrackV2 §1). + // + // Shape measured on a Studio Pro reference (ako/TestApp, + // Mappings.Order_Customer): an ordinary Texts$Text, which serializeText + // already produces with the typed-array marker 3 this needs. + var childMessage any + if childType == string(domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences) { + msg := "" + if childBehavior != nil { + msg = childBehavior.ErrorMessage + } + childMessage = serializeText(&model.Text{Translations: map[string]string{"en_US": msg}}) + } + return bson.D{ {Key: "$ID", Value: idToBsonBinary(generateUUID())}, {Key: "$Type", Value: "DomainModels$DeleteBehavior"}, {Key: "ChildDeleteBehavior", Value: childType}, - {Key: "ChildErrorMessage", Value: nil}, + {Key: "ChildErrorMessage", Value: childMessage}, {Key: "ParentDeleteBehavior", Value: parentType}, {Key: "ParentErrorMessage", Value: nil}, } From a7edf670278884e7527a8690280eabdd183a6070 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 07:22:09 +0000 Subject: [PATCH 04/17] docs(bootstrap): prefer the environment's cached Mendix version, else the newest MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The bootstrap-app skill pinned `11.13.0` as the Mendix version default, which is wrong in both directions: it ages, and it ignores what the session already has. A Claude Code session image can bake in an MxBuild — this one carries 11.13.0 — and asking for a different version turns a no-op into a multi-hundred-MB download of both the MxBuild and the runtime tarball. The rule is now: use whatever is already in ~/.mxcli/mxbuild/, and otherwise the newest version on the CDN. There is no environment variable to read (mxcli defines none, and `mxcli new --version` has no default of its own), so the cache directory is the signal: ls ~/.mxcli/mxbuild/ 2>/dev/null | sort -V | tail -1 A version the user names still wins over both. The CDN fallback names 11.14.0 — verified, along with the fact that it is current: mxbuild-11.14.0 and mendix-11.14.0 both answer 200, 11.15.0 answers 404. It is written as perishable and the check is parameterised on $V rather than repeating a literal that will rot the same way 11.13.0 did. Only the skill source changes; cmd/mxcli/skills/ is gitignored and regenerated by `make sync-skills`, which was run. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017s476QkXr9CFMvKspVzcvu --- .claude/skills/mendix/bootstrap-app/SKILL.md | 35 +++++++++++++++----- docs-site/src/tools/bootstrap-prompt.md | 31 ++++++++++++----- 2 files changed, 49 insertions(+), 17 deletions(-) diff --git a/.claude/skills/mendix/bootstrap-app/SKILL.md b/.claude/skills/mendix/bootstrap-app/SKILL.md index b1b85e07b..97935b51e 100644 --- a/.claude/skills/mendix/bootstrap-app/SKILL.md +++ b/.claude/skills/mendix/bootstrap-app/SKILL.md @@ -55,7 +55,8 @@ it is building. 6. **Look and feel.** One of the bundled themes: `signal` (light, high contrast), `ledger` (light, dense, data-heavy), `console` (dark), or `none` for stock Atlas. Default `signal`. -7. **Mendix version.** Default `11.13.0`. +7. **Mendix version.** Default to whatever the session environment already provides — + see below — and otherwise the newest version on the CDN (today `11.14.0`). 8. **Do you have requirements to work from?** A specification document, a prototype, a wireframe, a long description — anything that is the source of truth but is not in this repo. **Default: yes, record them.** If they say yes, @@ -71,20 +72,38 @@ it is building. If the user says "defaults" or ignores a question, choose something sensible for it, say what you chose in one line, and keep going — **do not block on them twice**. -### Checking the Mendix version default +### Choosing the Mendix version -Everything mxcli does starts with downloading MxBuild, so "supported" means "on the -CDN". If asked for a version newer than the default, verify both tarballs answer -`200` before using it — `run --local` needs the runtime as well as MxBuild: +**Prefer a version the environment already has.** A Claude Code session image may bake +in an MxBuild, and using it turns a multi-hundred-MB download into no download at all. +There is no environment variable for this — the cache directory is the signal: ```bash -curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mxbuild-11.13.0.tar.gz -curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mendix-11.13.0.tar.gz +ls ~/.mxcli/mxbuild/ 2>/dev/null | sort -V | tail -1 # e.g. 11.13.0, or empty ``` +If that names a version, use it and say so in one line ("using 11.13.0, already cached +in this environment"). If the user asked for a specific version, they win — check it on +the CDN as below and accept the download. + +**Otherwise take the newest version on the CDN.** Everything mxcli does starts with +downloading MxBuild, so "supported" means "on the CDN", and `run --local` needs the +runtime tarball as well. At the time of writing the newest is **11.14.0** (11.15.0 is +not published). Do not trust that number — it ages. Confirm the one you land on, and +walk backwards a minor if it 404s: + +```bash +V=11.14.0 +curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mxbuild-$V.tar.gz +curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mendix-$V.tar.gz +``` + +Both have to answer `200`. Run the same check for any version the user names. + In a solution, give every app the **same** version: they share the `~/.mxcli/mxbuild` cache, and a mismatch means a second multi-hundred-MB download and two runtimes to -keep straight. +keep straight — which also means the cached-version rule applies to the solution as a +whole, not per app. --- diff --git a/docs-site/src/tools/bootstrap-prompt.md b/docs-site/src/tools/bootstrap-prompt.md index ff52d6c0c..d53788974 100644 --- a/docs-site/src/tools/bootstrap-prompt.md +++ b/docs-site/src/tools/bootstrap-prompt.md @@ -110,20 +110,33 @@ tag (latest is v0.16.0) **and** as a rolling `nightly` pre-release, with assets ## Which Mendix version to ask for -The skill defaults to the newest version that has a published MxBuild — everything -mxcli does starts with downloading it, so "supported" means "on the CDN". It runs this -check itself when asked for a newer version, and it is the check to run before bumping -the default: +The skill prefers **whatever the session environment already provides**, and otherwise +the newest version with a published MxBuild. + +The environment's contribution is a *cached* MxBuild, not a variable — a Claude Code +session image may bake one in, and reusing it turns a multi-hundred-MB download into +no download at all. The cache directory is the signal: + +```bash +ls ~/.mxcli/mxbuild/ 2>/dev/null | sort -V | tail -1 # e.g. 11.13.0, or empty +``` + +With no cached version, the skill takes the newest on the CDN — everything mxcli does +starts with downloading MxBuild, so "supported" means "on the CDN". At the time of +writing that is **11.14.0**; treat the number as perishable and run the check rather +than quoting it: ```bash -curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mxbuild-11.13.0.tar.gz # 200 -curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mendix-11.13.0.tar.gz # 200 (runtime) +V=11.14.0 +curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mxbuild-$V.tar.gz # 200 +curl -sI -o /dev/null -w '%{http_code}\n' https://cdn.mendix.com/runtime/mendix-$V.tar.gz # 200 (runtime) ``` Both have to answer `200` — `run --local` needs the runtime tarball as well as -MxBuild. In a solution, give every app the **same** version: they share the -`~/.mxcli/mxbuild` cache, and a mismatch means a second multi-hundred-MB download and -two runtimes to keep straight. +MxBuild. This is also the check to run before bumping the version named in the skill. +In a solution, give every app the **same** version: they share the `~/.mxcli/mxbuild` +cache, and a mismatch means a second multi-hundred-MB download and two runtimes to +keep straight. ## Two rules that make this robust From e62ec428220f0df6f842d4f452a1c4383d83b74e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 07:27:14 +0000 Subject: [PATCH 05/17] fix(microflows): un-exempt the AS clause from MDL062, which hid a real CE0068 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A `return` inside a `loop` passed `mxcli check` and `exec`, and mxbuild then rejected it: [CE0068] "End events cannot be placed inside a loop." MDL062 exists to catch exactly that and has since #893. It did not fire, because it stood down for the WHOLE microflow whenever the header carried a `returns T as $Var` clause. Two claims justified that: 1. buildFlowGraph synthesizes the End event from the variable, so none lands inside the loop. 2. Measured: the shape builds CE0109 ("Undefined variable") rather than CE0068, so firing here would mislabel a different defect. The first was never true — `describe microflow` shows the in-loop `return` written either way. The second was an artefact of the reproduction. mxbuild reports ONE error per microflow, and the microflow that was measured never assigned its AS variable, so CE0109 won the race and hid the CE0068 underneath. Re-measured on mxbuild 11.13.0, adding a `declare` for that variable and changing nothing else: as-clause, $Done unassigned -> CE0109 "Undefined variable 'Done'." as-clause, $Done assigned -> CE0068 "End events cannot be placed inside a loop." Same body, same loop, same return. So the exemption is deleted rather than narrowed, and its test is inverted with the masking recorded in the comment — a test asserting "clean" against a source that was never clean is how this shipped. The control is the half a careless fix would break: an as-clause microflow whose loop does NOT return must stay silent, since the clause's real effect (the terminal End event taking its value from the variable) is unchanged. Both halves are in mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause{,.fail}.mdl. Reported as CapTrackV2 FINDINGS §19. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + ...track-19-return-in-loop-as-clause.fail.mdl | 61 +++++++++++++++++++ .../captrack-19-return-in-loop-as-clause.mdl | 51 ++++++++++++++++ mdl/executor/validate_microflow_ce_gaps.go | 36 ++++++----- .../validate_microflow_ce_gaps_test.go | 39 ++++++++++-- 5 files changed, 168 insertions(+), 20 deletions(-) create mode 100644 mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.fail.mdl create mode 100644 mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.mdl diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index e68335ff1..778742a0f 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -507,3 +507,4 @@ {"area": "mdl/executor", "cause": "Two DataTypes$ sub-documents the writer never emitted, both on Microflows$CallExternalAction. (1) edmReturnTypeToKind mapped only EDM primitives and returned \"\" for anything else -- documented in-code as \"Complex / collection / entity-typed returns aren't yet mapped\" -- so an action returning an ENTITY got no VariableDataType at all. (2) ExternalActionParameterMapping.ParameterType was never written, though generated/metamodel declares it WITHOUT omitempty. Separately, mdl/catalog/builder_external.go catalogued only entities whose Source is Rest$ODataRemoteEntitySource, skipping every Rest$ODataEntityTypeSource -- the derived, abstract, contained and action parameter/return types that have no entity set.", "ce": ["CE7252", "CE7269", "CE0117", "CE7251"], "date": "2026-09-03", "file": "mdl/executor/cmd_microflows_builder_calls.go (resolveExternalActionReturnKind, resolveExternalActionParameterKinds, edmBareTypeName); sdk/mpr/writer_microflow_actions.go + mdl/backend/modelsdk/microflow_external_action_write.go (both writers); sdk/microflows/microflows_actions.go (ResultEntity, ParameterDataType/ParameterEntity); mdl/catalog/builder_external.go (isODataEntitySource); new validator mdl/executor/validate_external_action_calls.go", "insight": "**Read the CE code out of Mendix's own assemblies before theorising about it.** `strings Mendix.Modeler.Texts.dll | grep CE7252` gives the symbol, the English text AND the LOCATION comment -- here CallExternalAction.cs for both codes, which settles in one command that the entity import can never fix them and that the reporter was pulling the wrong lever. Same technique found CE7253 and the CE7251 constraint (Mendix's `call external action` takes OData ACTIONS only, not Functions, and an unbound action needs an in the EntityContainer or it is not callable at all). **A missing mandatory sub-document is the recurring shape here**: generated/metamodel omitting `omitempty` on a pointer property is the tell, and the same fix pattern applied twice in one bug. **The reporter's evidence was an artifact of OUR tool**: contract_entities.UsedByExternalEntity is an mxcli catalog column filled by joining external_entities on RemoteName, so while that table skipped type-sourced entities the column was structurally always empty for exactly the entities in question -- it read as 'not linked' whether or not the import had worked. When a report cites one of our own derived columns as evidence, verify the column can be non-empty for that case before believing it. **Verification without a fixture**: no $metadata in the repo declared an action, so the contract was served from `python3 -m http.server` on 127.0.0.1 and MetadataUrl pointed at it -- a local HTTP contract makes the whole consumed-OData path testable end to end. Controls: reverting the return resolver reproduces CE7269 verbatim; before the ParameterType fix, ANY parameter of ANY type produced CE7252 + one CE0117 per argument; after, 0 errors on all three shapes. Repro mdl-examples/bug-tests/odata-1020-external-action-types.mdl", "refs": ["mendixlabs/mxcli#1020"], "symptom": "CE7252 \"The parameters for remote action '' have changed\" and CE7269 \"The return type for remote action '' has changed\" persist after CREATE OR MODIFY EXTERNAL ENTITIES, which reports success and changes nothing. A SQL query over CATALOG.contract_entities shows UsedByExternalEntity empty for the action's parameter/response entities while entity-set entities populate it, which reads as a broken link between the imported entity and the contract."} {"area": "mdl/executor", "date": "2026-09-02", "symptom": "`retrieve $L from Mod.Sub sort by asc` passes `mxcli check` AND `exec`, then mxbuild fails **CE1613** \"The selected attribute 'Mod.Sub.Attr' no longer exists.\" \u2014 and the correct spelling, `sort by Base.Entity.Attr`, is REFUSED by mxcli with \"sort by attribute '\u2026' does not belong to entity '\u2026'\". Reading the same attribute works fine. The list operation `SORT($L, Attr ASC)` writes the same broken reference.", "cause": "The sort path qualified a bare attribute name with the entity being RETRIEVED (`entityQN + \".\" + attr`), but Mendix resolves a sort reference against the entity that DECLARES the attribute. For the qualified form it treated any entity other than the retrieved one as foreign, tried to infer association traversal steps, found none, and refused \u2014 an ancestor is not a traversal, the attribute is on the object already. Both halves now consult the generalization chain: `resolveAttributeInEntityHierarchy` for the bare name, `entityIsSubtypeOf` for the qualified one.", "file": "`mdl/executor/cmd_microflows_builder_actions.go` (the SORT BY block in `addRetrieveAction`)", "insight": "**Both helpers already existed on `flowBuilder` and this path simply did not call them** \u2014 the fix is two call sites, not new machinery. The tell was in the report: reading the attribute worked while sorting on it did not, and the page builder has walked the chain for years (`declaringEntityFor`). When one path resolves a name and a sibling path does not, look for the existing resolver before writing one. Controls: an attribute the entity declares itself must still be qualified with that entity (a fix that always walked to the base breaks every ordinary sort), and an attribute on an unrelated entity must still be refused rather than passed through to become a CE1613 at the far end of a build. Verified end-to-end on a real 11.13 project with `extends System.User`: pre-fix writes `App.AppUser.Name` and mx check reports CE1613; fixed writes `System.User.Name` and mx check is 0 errors, and the qualified spelling that was refused now executes. Reported as CapTrackV2 FINDINGS \u00a713."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "An app whose domain model uses `DELETE_BEHAVIOR PREVENT` will not START. `mx check` reports 0 errors; the failure is at runtime init: `ERROR - M2EE: An error occurred while initializing the Runtime: None.get` / `java.util.NoSuchElementException: None.get` at `SchemeFactory$.\u2026$setDeleteBehavior(SchemeFactory.scala:515)`. Rewriting the associations to the default behaviour takes the app from crash to HTTP 200.", "cause": "mxcli wrote `ChildDeleteBehavior: DeleteMeIfNoReferences` with `ChildErrorMessage: null` \u2014 the codec registers `DomainModels$DeleteBehavior` with both error-message slots in `NullFields`, unconditionally. Studio Pro writes a `Texts$Text` there for that behaviour only. MDL had no syntax for the message at all. Fixed by adding SQL's referential actions (`ON DELETE CASCADE|RESTRICT|SET NULL`, plus `ERROR_MESSAGE '\u2026'`) and writing the text element when the child behaviour is DeleteMeIfNoReferences \u2014 only then, and only on the child side.", "file": "`mdl/grammar/MDLLexer.g4` + `domains/MDLDomainModel.g4` + `domains/MDLSettings.g4` (keyword rule), `mdl/visitor/visitor_association.go` + `visitor_helpers.go` (`buildReferentialAction`), `mdl/ast/ast_association.go`, `sdk/domainmodel/domainmodel.go` (`DeleteBehavior.ErrorMessage`), `mdl/backend/modelsdk/domainmodel_write.go` (`deleteErrorText`) + `domainmodel.go` (read back), `mdl/executor/cmd_associations.go` (`describeDeleteClause`)", "insight": "**The field is CONDITIONAL in Studio Pro's own dialog** \u2014 it only appears once the third radio button is selected, which is why a screenshot of the association properties showed no message field and nearly falsified the reporter's (correct) diagnosis. Ask for the reference in the state that exhibits the behaviour, not the default state. A census of 47,789 units across 122 projects found 423 delete behaviours and **zero** using this one, so no reference existed anywhere until one was authored \u2014 that absence is why the null shipped and why nobody hit it sooner. Syntax note: Mendix's three behaviours ARE SQL's referential actions, and MDL's FROM/TO already matches a foreign key's direction (ParentPointer = FROM = FK owner, ChildPointer = TO = referenced, measured), so `ON DELETE RESTRICT` needed no invention and fixed a readability complaint that `DELETE_IF_NO_REFERENCES` had earned \u2014 MDL's names are Mendix's with the word **Me** dropped, and Me was the only word saying whose deletion it described. Controls, end-to-end on 11.13: a project authored by the pre-fix binary reproduces `None.get` at boot verbatim; the fixed one boots. DESCRIBE emits the ON DELETE form with the message, or a describe->exec round trip rebuilds the crash. The empty-message shape is INFERRED, not measured \u2014 flagged in `deleteErrorText`. Reported as CapTrackV2 FINDINGS \u00a71."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "`return` inside a `loop` passes `mxcli check` AND `exec`, then mxbuild fails **CE0068** \"End events cannot be placed inside a loop.\" \u2014 even though MDL062 exists precisely to catch that. It fires only when the microflow header has NO `returns T as $Var` clause.", "cause": "MDL062 stood down for the whole microflow whenever the AS clause was present, on two claims: that buildFlowGraph synthesizes the End event from the variable so none lands in the loop, and that the shape builds CE0109 instead. `describe microflow` shows the in-loop `return` written either way, so the first was never true. The second was a measurement artefact: mxbuild reports ONE error per microflow, and in the shape that was measured the AS variable was never assigned, so CE0109 \"Undefined variable\" won the race and hid CE0068 underneath. Adding `declare $Done Boolean = false` and changing nothing else turns the same microflow from CE0109 into CE0068. The exemption was deleted, not narrowed.", "file": "`mdl/executor/validate_microflow_ce_gaps.go` (checkReturnInLoop \u2014 the `v.returnType.Variable != \"\"` early return); test inverted in `validate_microflow_ce_gaps_test.go` (TestMDL062_ExemptsReturnsAsClause -> TestMDL062_FiresWithReturnsAsClause); examples `mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause{,.fail}.mdl`", "insight": "A second error in the same document can HIDE the one you are measuring, because mxbuild reports one error per microflow. An exemption justified by \"measured: builds X instead\" is only sound if the reproduction was otherwise valid \u2014 here the repro was broken in a second way, and the error that surfaced was the one nobody was asking about. When a measurement says a construct is clean, add the minimum that removes every OTHER error from that document and measure again; the differential (CE0109 -> CE0068 on one added `declare`) is what settles it. Reported as CapTrackV2 FINDINGS \u00a719."} diff --git a/mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.fail.mdl b/mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.fail.mdl new file mode 100644 index 000000000..9bda6f977 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.fail.mdl @@ -0,0 +1,61 @@ +-- NEGATIVE TEST — `mxcli check` must REFUSE this file (MDL062). +-- +-- CapTrackV2 FINDINGS §19 — a `return` inside a loop passed `mxcli check` and +-- `exec`, and mxbuild then rejected it: +-- +-- [CE0068] "End events cannot be placed inside a loop." +-- +-- MDL062 has existed since #893 and did not fire, because it stood down for the +-- whole microflow whenever the header carried a `returns T as $Var` clause. The +-- stated reason was that the builder synthesizes the End event from the variable +-- so none lands in the loop, and that the shape builds CE0109 instead. +-- +-- Neither held. `describe microflow` shows the in-loop `return` written either +-- way, and mxbuild reports ONE error per microflow: in the shape that was +-- measured, $Var was never assigned, so CE0109 "Undefined variable" won the race +-- and hid the CE0068 underneath. Measured on mxbuild 11.13.0, changing NOTHING +-- but adding a `declare` for the variable: +-- +-- as-clause, $Done unassigned -> CE0109 "Undefined variable 'Done'." +-- as-clause, $Done assigned -> CE0068 "End events cannot be placed inside a loop." +-- +-- Verify: +-- mxcli check captrack-19-return-in-loop-as-clause.fail.mdl +-- -- must report MDL062 for BOTH microflows below and exit non-zero +-- +-- The positive control lives beside this file as +-- captrack-19-return-in-loop-as-clause.mdl, whose shapes must stay clean. + +create module RetLoop; +/ +create persistent entity RetLoop.Item ( + Code: string(50) +); +/ + +-- The exact source whose measurement went wrong: $Done is never assigned, so +-- CE0109 masked CE0068 on the build that established the exemption. +create microflow RetLoop.MF_Masked () +returns boolean as $Done +begin + retrieve $L from RetLoop.Item; + loop $I in $L + begin + return true; + end loop; +end; +/ + +-- The same microflow with the mask removed. This one builds CE0068 on 11.13.0, +-- which is what proves the two are the same defect. +create microflow RetLoop.MF_Unmasked () +returns boolean as $Done +begin + declare $Done boolean = false; + retrieve $L from RetLoop.Item; + loop $I in $L + begin + return true; + end loop; +end; +/ diff --git a/mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.mdl b/mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.mdl new file mode 100644 index 000000000..ae058aec9 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause.mdl @@ -0,0 +1,51 @@ +-- CONTROL for CapTrackV2 FINDINGS §19 (see the .fail.mdl beside this file). +-- +-- Un-exempting `returns T as $Var` from MDL062 must not turn the clause itself +-- into an error. Its real effect — buildFlowGraph taking the terminal End +-- event's value from the variable — is unchanged, and a fix that reported every +-- as-clause microflow would satisfy the negative test while breaking the idiom +-- the clause exists for. +-- +-- Verify: +-- mxcli check captrack-19-return-in-loop-as-clause.mdl -- 0 errors +-- mxcli exec captrack-19-return-in-loop-as-clause.mdl -p app.mpr +-- mx check -p app.mpr -- 0 errors + +create module RetLoopOk; +/ +create persistent entity RetLoopOk.Item ( + Code: string(50) +); +/ + +-- The as-clause idiom: the loop assigns the variable, the header returns it. +-- This is the shape MDL062's message points authors at. +create microflow RetLoopOk.MF_AsClause () +returns boolean as $Done +begin + declare $Done boolean = false; + retrieve $L from RetLoopOk.Item; + loop $I in $L + begin + set $Done = true; + end loop; +end; +/ + +-- `break` is the construct that replaces an in-loop return, with a single +-- return after the loop. +create microflow RetLoopOk.MF_BreakThenReturn () +returns boolean +begin + declare $Found boolean = false; + retrieve $L from RetLoopOk.Item; + loop $I in $L + begin + if $I/Code = 'x' then + set $Found = true; + break; + end if; + end loop; + return $Found; +end; +/ diff --git a/mdl/executor/validate_microflow_ce_gaps.go b/mdl/executor/validate_microflow_ce_gaps.go index 7b9062834..8218a9662 100644 --- a/mdl/executor/validate_microflow_ce_gaps.go +++ b/mdl/executor/validate_microflow_ce_gaps.go @@ -116,26 +116,32 @@ func zeroValueHint(t ast.DataType) string { // continues after it. Returning a value from inside a loop needs the value // stashed in a variable and a single return after the loop. // -// The rule predicts what the BUILDER emits, not what the MDL looks like, and -// two forms measured clean on mxbuild 11.6.6 are therefore exempt. Both were -// found by running the rule over the shipped examples before wiring it up: +// The rule predicts what the BUILDER emits, not what the MDL looks like, so +// `while true` is exempt: it is built as an ExclusiveMerge back-edge, not a +// LoopedActivity (#350). With no loop object there is no "inside a loop", and +// the return is an ordinary End event. A plain `while ` IS a +// LoopedActivity and is not exempt — measured separately. // -// - `while true` is built as an ExclusiveMerge back-edge, not a -// LoopedActivity (#350). With no loop object there is no "inside a loop", -// and the return is an ordinary End event. A plain `while ` IS a -// LoopedActivity and is not exempt — measured separately. -// - `returns T as $Var` makes buildFlowGraph synthesize the End event from -// the variable, and no End event lands inside the loop. Firing here would -// name the wrong defect: that shape builds CE0109 ("Undefined variable") -// instead, which is a separate gap and not what this rule is about. +// `returns T as $Var` used to be exempt too, on the stated grounds that +// buildFlowGraph synthesizes the End event from the variable so none lands in +// the loop, and that the shape builds CE0109 ("Undefined variable") rather than +// CE0068. The first half was never true and the second was an artefact of the +// measurement: mxbuild reports ONE error per microflow, and in the shape that +// was measured — `returns Boolean as $Done` with the loop's `return true` and +// nothing assigning $Done — CE0109 simply won the race and hid CE0068 +// underneath. Re-measured on 11.13.0, with `declare $Done Boolean = false` +// added and NOTHING else changed: +// +// as-clause, $Done unassigned -> CE0109 "Undefined variable 'Done'." +// as-clause, $Done assigned -> CE0068 "End events cannot be placed inside a loop." +// +// `describe microflow` shows the in-loop `return true` written in both, so the +// End event was always there. The exemption is gone; a masked error is not an +// absent one, and CapTrackV2 FINDINGS §19 is the report it cost. func (v *microflowValidator) checkReturnInLoop(body []ast.MicroflowStatement) { if v.skipCEGapRules() { return } - // See the AS-clause note above: the builder routes the return elsewhere. - if v.returnType != nil && v.returnType.Variable != "" { - return - } var walk func(stmts []ast.MicroflowStatement, depth int) walk = func(stmts []ast.MicroflowStatement, depth int) { diff --git a/mdl/executor/validate_microflow_ce_gaps_test.go b/mdl/executor/validate_microflow_ce_gaps_test.go index 3fe9e52ac..5de9fc422 100644 --- a/mdl/executor/validate_microflow_ce_gaps_test.go +++ b/mdl/executor/validate_microflow_ce_gaps_test.go @@ -374,10 +374,18 @@ end;`) } } -// With `returns T as $Var` the builder takes the End event's value from the -// variable and none lands inside the loop. Measured: no CE0068 (that shape -// builds CE0109 instead, a different defect this rule must not mislabel). -func TestMDL062_ExemptsReturnsAsClause(t *testing.T) { +// `returns T as $Var` was exempt on the grounds that the builder synthesizes the +// End event from the variable so none lands in the loop, and that the shape +// builds CE0109 rather than CE0068. Both were wrong, and this is the exact +// source that measured wrong: mxbuild reports ONE error per microflow, $Done is +// never assigned here, and CE0109 masked the CE0068 underneath. +// +// Re-measured on mxbuild 11.13.0 with `declare $Done Boolean = false` inserted +// and nothing else changed, the same microflow reports +// CE0068 "End events cannot be placed inside a loop." — and `describe +// microflow` shows the in-loop `return true` written either way, so the End +// event was always in the loop. Reported as CapTrackV2 FINDINGS §19. +func TestMDL062_FiresWithReturnsAsClause(t *testing.T) { vs := checkMicroflowSource(t, `create microflow Synthetic.MF_AsClause () returns Boolean as $Done begin retrieve $L from Synthetic.Item; @@ -385,9 +393,30 @@ begin begin return true; end loop; +end;`) + if _, ok := violationsByRule(vs)["MDL062"]; !ok { + t.Errorf("MDL062 stayed silent on a `returns … as $Var` microflow whose loop "+ + "contains a return — measured CE0068 on mxbuild 11.13.0 once CE0109 stops "+ + "masking it: %#v", vs) + } +} + +// CONTROL for the un-exemption: the AS clause on its own must stay clean. The +// clause's real effect — buildFlowGraph taking the terminal End event's value +// from the variable — is unchanged, and a fix that simply reported every +// as-clause microflow would satisfy the test above while breaking the idiom. +func TestMDL062_ReturnsAsClauseWithoutALoopReturnIsClean(t *testing.T) { + vs := checkMicroflowSource(t, `create microflow Synthetic.MF_AsClean () returns Boolean as $Done +begin + declare $Done Boolean = false; + retrieve $L from Synthetic.Item; + loop $I in $L + begin + set $Done = true; + end loop; end;`) if _, bad := violationsByRule(vs)["MDL062"]; bad { - t.Errorf("MDL062 fired on a `returns … as $Var` microflow, where no End event lands in the loop: %#v", vs) + t.Errorf("MDL062 fired on an as-clause microflow with no return inside the loop: %#v", vs) } } From 7dc75cd91da9eee2e48aa08d46835575db4efcd9 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 07:36:50 +0000 Subject: [PATCH 06/17] feat(pages): report an on-click action mxcli does not write (MDL-WIDGET23) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A data view's own `OnClick:` parses, passes `mxcli check`, is written by `exec` without a word, and the rendered element has no handler and no role="button". `OnClick:` is an alias for `Action:` — the visitor stores both under Properties["Action"] (#603) — and mxcli writes that property for three widget kinds only: container/customcontainer, the buttons, and a navigationlist item. Measured on Mendix 11.13 by writing ONE page with four widgets and reading it back with `describe page`: container OnClick kept Pages$DivContainer.OnClickAction listview OnClick GONE Mendix HAS ListView.ClickAction; no mxcli writer dataview OnClick GONE Mendix models no click action on a data view dynamictext OnClick GONE same Nothing fails the build — the document is valid, the widget simply does not do what was asked — which is why this needs saying out loud and why the severity is warning, matching MDL-WIDGET20/21. The property allow-lists behind MDL-WIDGET01/07 could never have caught it: they are widget-type AGNOSTIC, the same blind spot #928 documented for `editable:`. Two messages, because the remedy differs. Where Mendix models no click action at all, a container inside the widget is the correct modelling — it renders with tabindex/role="button". Where Mendix models one mxcli cannot write (listview, staticimage, dynamicimage, checked against generated/metamodel), the container is a workaround and the message says so. The rule NAMES the types it reports rather than reporting everything outside an allow-list of the three writers. The allow-list version looked tighter and was wrong: running it over mdl-examples/ flagged three shipped examples, because `mxcli check` without -p has no widget registry, so lookupWidgetDef returns nil for a pluggable widget too and the caller's "static widgets only" branch does not hold — `datagrid` is DataGrid 2, whose onClick the widget engine does write. A missed warning costs nothing; a false one tells an author their working page is broken. That case is now a control test. Reported as CapTrackV2 FINDINGS §21. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../mendix/create-page/reference/widgets.md | 36 +++ .../captrack-21-dataview-onclick-dropped.mdl | 70 ++++++ mdl/executor/validate_widget_onclick.go | 122 +++++++++++ mdl/executor/validate_widget_onclick_test.go | 207 ++++++++++++++++++ mdl/executor/validate_widgets.go | 3 + 6 files changed, 439 insertions(+) create mode 100644 mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl create mode 100644 mdl/executor/validate_widget_onclick.go create mode 100644 mdl/executor/validate_widget_onclick_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 778742a0f..43c4a09a6 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -508,3 +508,4 @@ {"area": "mdl/executor", "date": "2026-09-02", "symptom": "`retrieve $L from Mod.Sub sort by asc` passes `mxcli check` AND `exec`, then mxbuild fails **CE1613** \"The selected attribute 'Mod.Sub.Attr' no longer exists.\" \u2014 and the correct spelling, `sort by Base.Entity.Attr`, is REFUSED by mxcli with \"sort by attribute '\u2026' does not belong to entity '\u2026'\". Reading the same attribute works fine. The list operation `SORT($L, Attr ASC)` writes the same broken reference.", "cause": "The sort path qualified a bare attribute name with the entity being RETRIEVED (`entityQN + \".\" + attr`), but Mendix resolves a sort reference against the entity that DECLARES the attribute. For the qualified form it treated any entity other than the retrieved one as foreign, tried to infer association traversal steps, found none, and refused \u2014 an ancestor is not a traversal, the attribute is on the object already. Both halves now consult the generalization chain: `resolveAttributeInEntityHierarchy` for the bare name, `entityIsSubtypeOf` for the qualified one.", "file": "`mdl/executor/cmd_microflows_builder_actions.go` (the SORT BY block in `addRetrieveAction`)", "insight": "**Both helpers already existed on `flowBuilder` and this path simply did not call them** \u2014 the fix is two call sites, not new machinery. The tell was in the report: reading the attribute worked while sorting on it did not, and the page builder has walked the chain for years (`declaringEntityFor`). When one path resolves a name and a sibling path does not, look for the existing resolver before writing one. Controls: an attribute the entity declares itself must still be qualified with that entity (a fix that always walked to the base breaks every ordinary sort), and an attribute on an unrelated entity must still be refused rather than passed through to become a CE1613 at the far end of a build. Verified end-to-end on a real 11.13 project with `extends System.User`: pre-fix writes `App.AppUser.Name` and mx check reports CE1613; fixed writes `System.User.Name` and mx check is 0 errors, and the qualified spelling that was refused now executes. Reported as CapTrackV2 FINDINGS \u00a713."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "An app whose domain model uses `DELETE_BEHAVIOR PREVENT` will not START. `mx check` reports 0 errors; the failure is at runtime init: `ERROR - M2EE: An error occurred while initializing the Runtime: None.get` / `java.util.NoSuchElementException: None.get` at `SchemeFactory$.\u2026$setDeleteBehavior(SchemeFactory.scala:515)`. Rewriting the associations to the default behaviour takes the app from crash to HTTP 200.", "cause": "mxcli wrote `ChildDeleteBehavior: DeleteMeIfNoReferences` with `ChildErrorMessage: null` \u2014 the codec registers `DomainModels$DeleteBehavior` with both error-message slots in `NullFields`, unconditionally. Studio Pro writes a `Texts$Text` there for that behaviour only. MDL had no syntax for the message at all. Fixed by adding SQL's referential actions (`ON DELETE CASCADE|RESTRICT|SET NULL`, plus `ERROR_MESSAGE '\u2026'`) and writing the text element when the child behaviour is DeleteMeIfNoReferences \u2014 only then, and only on the child side.", "file": "`mdl/grammar/MDLLexer.g4` + `domains/MDLDomainModel.g4` + `domains/MDLSettings.g4` (keyword rule), `mdl/visitor/visitor_association.go` + `visitor_helpers.go` (`buildReferentialAction`), `mdl/ast/ast_association.go`, `sdk/domainmodel/domainmodel.go` (`DeleteBehavior.ErrorMessage`), `mdl/backend/modelsdk/domainmodel_write.go` (`deleteErrorText`) + `domainmodel.go` (read back), `mdl/executor/cmd_associations.go` (`describeDeleteClause`)", "insight": "**The field is CONDITIONAL in Studio Pro's own dialog** \u2014 it only appears once the third radio button is selected, which is why a screenshot of the association properties showed no message field and nearly falsified the reporter's (correct) diagnosis. Ask for the reference in the state that exhibits the behaviour, not the default state. A census of 47,789 units across 122 projects found 423 delete behaviours and **zero** using this one, so no reference existed anywhere until one was authored \u2014 that absence is why the null shipped and why nobody hit it sooner. Syntax note: Mendix's three behaviours ARE SQL's referential actions, and MDL's FROM/TO already matches a foreign key's direction (ParentPointer = FROM = FK owner, ChildPointer = TO = referenced, measured), so `ON DELETE RESTRICT` needed no invention and fixed a readability complaint that `DELETE_IF_NO_REFERENCES` had earned \u2014 MDL's names are Mendix's with the word **Me** dropped, and Me was the only word saying whose deletion it described. Controls, end-to-end on 11.13: a project authored by the pre-fix binary reproduces `None.get` at boot verbatim; the fixed one boots. DESCRIBE emits the ON DELETE form with the message, or a describe->exec round trip rebuilds the crash. The empty-message shape is INFERRED, not measured \u2014 flagged in `deleteErrorText`. Reported as CapTrackV2 FINDINGS \u00a71."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`return` inside a `loop` passes `mxcli check` AND `exec`, then mxbuild fails **CE0068** \"End events cannot be placed inside a loop.\" \u2014 even though MDL062 exists precisely to catch that. It fires only when the microflow header has NO `returns T as $Var` clause.", "cause": "MDL062 stood down for the whole microflow whenever the AS clause was present, on two claims: that buildFlowGraph synthesizes the End event from the variable so none lands in the loop, and that the shape builds CE0109 instead. `describe microflow` shows the in-loop `return` written either way, so the first was never true. The second was a measurement artefact: mxbuild reports ONE error per microflow, and in the shape that was measured the AS variable was never assigned, so CE0109 \"Undefined variable\" won the race and hid CE0068 underneath. Adding `declare $Done Boolean = false` and changing nothing else turns the same microflow from CE0109 into CE0068. The exemption was deleted, not narrowed.", "file": "`mdl/executor/validate_microflow_ce_gaps.go` (checkReturnInLoop \u2014 the `v.returnType.Variable != \"\"` early return); test inverted in `validate_microflow_ce_gaps_test.go` (TestMDL062_ExemptsReturnsAsClause -> TestMDL062_FiresWithReturnsAsClause); examples `mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause{,.fail}.mdl`", "insight": "A second error in the same document can HIDE the one you are measuring, because mxbuild reports one error per microflow. An exemption justified by \"measured: builds X instead\" is only sound if the reproduction was otherwise valid \u2014 here the repro was broken in a second way, and the error that surfaced was the one nobody was asking about. When a measurement says a construct is clean, add the minimum that removes every OTHER error from that document and measure again; the differential (CE0109 -> CE0068 on one added `declare`) is what settles it. Reported as CapTrackV2 FINDINGS \u00a719."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "`dataview dv (\u2026, OnClick: SHOW_PAGE \u2026)` parses, `mxcli check` is clean, `exec` writes the page without a word \u2014 and the rendered element has no handler and no role=\"button\". The same silence on `dynamictext`, `listview` and every other widget except a container or a button.", "cause": "`OnClick:` is an ALIAS for `Action:` (both stored as Properties[\"Action\"], #603), and mxcli writes that property for three widget kinds only: container/customcontainer, the buttons, and a navigationlist item. Every other widget drops it, and the property allow-lists behind MDL-WIDGET01/07 could not see that because they are widget-type AGNOSTIC \u2014 the same blind spot as #928's `editable:`. Added MDL-WIDGET23 (warning), with two messages: Mendix models no click action at all (dataview, dynamictext, inputs, groupbox\u2026) vs Mendix models one that mxcli cannot write (listview, staticimage, dynamicimage \u2014 measured against generated/metamodel).", "file": "`mdl/executor/validate_widget_onclick.go` (new), wired in `validate_widgets.go` beside validateWidgetEditability; `.claude/skills/mendix/create-page/reference/widgets.md`; example `mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl`", "insight": "The first draft reported everything OUTSIDE an allow-list of the three writers, and running it over the shipped examples flagged three of them. Cause: `mxcli check` without `-p` has no widget registry, so `lookupWidgetDef` returns nil for a PLUGGABLE widget too and the caller's \"static widgets only\" branch silently does not hold \u2014 `datagrid` is DataGrid 2, a pluggable widget whose onClick the engine does write. For any rule keyed on widget type, an allow-list makes the unknown case an ERROR and a deny-list makes it silence; pick the deny-list, because a missed warning costs nothing and a false one tells an author their working page is broken. Running a new rule across mdl-examples/ before wiring it up is what caught it \u2014 the same exercise #893 describes. Reported as CapTrackV2 FINDINGS \u00a721."} diff --git a/.claude/skills/mendix/create-page/reference/widgets.md b/.claude/skills/mendix/create-page/reference/widgets.md index 681eebda4..33f48bd73 100644 --- a/.claude/skills/mendix/create-page/reference/widgets.md +++ b/.claude/skills/mendix/create-page/reference/widgets.md @@ -723,6 +723,42 @@ actionbutton btnSubmit ( ) ``` +### Only a CONTAINER (or a Button) Can Be Clicked + +`onclick:` is an alias for `action:`, and mxcli writes it for three widget kinds +only: `container`/`customcontainer` (`Forms$DivContainer.onClickAction`), the +buttons, and a `navigationlist` `item`. Anywhere else it parses, passes check, +is written, and reaches nothing — the rendered element has no handler and no +`role="button"`. MDL-WIDGET23 reports it rather than letting it go quiet. + +Two shapes, two remedies: + +- **Mendix models no click there at all** (`dataview`, `dynamictext`, `title`, + the input widgets, `groupbox`, `tabcontainer`, `layoutgrid`, `snippetcall`) — + put the action on a `container` inside the widget. A container renders with + `tabindex="0" role="button"`, so it is the correct modelling, not a workaround. +- **Mendix models one but mxcli cannot write it yet** (`listview`, + `staticimage`, `dynamicimage`) — the container is a workaround here; the model + could hold the action. + +```sql +-- WRONG: silently does nothing +dataview dvOrder (datasource: microflow Mod.DS_Order, onclick: show_page Mod.Detail) { + dynamictext t (content: 'Open') +} + +-- RIGHT: the container carries the click +dataview dvOrder (datasource: microflow Mod.DS_Order) { + container clickable (onclick: show_page Mod.Detail) { + dynamictext t (content: 'Open') + } +} +``` + +Pluggable widgets are exempt: their action slots come from the widget's own +definition and are routed by the widget engine, so `datagrid` (DataGrid 2) and +`pluggablewidget '…' (onclick: …)` are written normally. + ### CONTAINER / CUSTOMCONTAINER Widgets Generic container for grouping widgets. `customcontainer` is an alias for `container` (both map to `Forms$DivContainer`): diff --git a/mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl b/mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl new file mode 100644 index 000000000..c25c3efa2 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl @@ -0,0 +1,70 @@ +-- CapTrackV2 FINDINGS §21 — a data view's own `OnClick:` is dropped. +-- +-- dataview dv (…, OnClick: SHOW_PAGE …) +-- -> parses, `mxcli check` clean, `exec` writes the page without a word, +-- and the rendered element has no handler and no role="button". +-- +-- `OnClick:` is an alias for `Action:` (the visitor stores both under +-- Properties["Action"], issue #603), and mxcli writes that property for three +-- widget kinds only: container/customcontainer, the buttons, and a +-- navigationlist item. Measured on Mendix 11.13 by writing ONE page with four +-- widgets and reading it back with `describe page`: +-- +-- container OnClick kept (Pages$DivContainer.OnClickAction) +-- listview OnClick GONE (Mendix HAS ListView.ClickAction — no mxcli writer) +-- dataview OnClick GONE (Mendix models no click action on a data view) +-- dynamictext OnClick GONE (same) +-- +-- MDL-WIDGET23 now reports the drop, with a different remedy for each of the two +-- shapes. The container is the correct modelling for the first, a workaround for +-- the second. +-- +-- Verify: +-- mxcli check captrack-21-dataview-onclick-dropped.mdl +-- -- 3 MDL-WIDGET23 warnings: dvDropped, lvDropped, txtDropped +-- -- and NOTHING on cClickable or btnGo +-- mxcli exec captrack-21-dataview-onclick-dropped.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe page Clicks.P_Clicks" +-- -- only the container's action comes back +-- mx check -p app.mpr -- 0 errors either way; this never was a build error + +create module Clicks; +/ +create persistent entity Clicks.Thing ( + Name: string(100) +); +/ +create microflow Clicks.DS_OneThing () +returns Clicks.Thing as $T +begin + retrieve $L from Clicks.Thing; + set $T = head($L); +end; +/ + +create or replace page Clicks.P_Clicks + (Title: 'Clicks', Layout: Atlas_Core.Atlas_Default) +{ + -- CONTROL: the one shape that works, and the remedy the rule points at. + container cClickable (OnClick: show_page Clicks.P_Clicks) { + dynamictext tIn (Content: 'this one really is clickable') + } + + -- CONTROL: a button's action is written too. + actionbutton btnGo (Caption: 'Go', Action: show_page Clicks.P_Clicks) + + -- Reported (no slot in Mendix at all). + dataview dvDropped (DataSource: microflow Clicks.DS_OneThing, OnClick: show_page Clicks.P_Clicks) { + dynamictext tDv (Content: 'inside the data view') + } + + -- Reported (no slot in Mendix at all). + dynamictext txtDropped (Content: 'bare text', OnClick: show_page Clicks.P_Clicks) + + -- Reported, with the OTHER message: Mendix models ListView.ClickAction, + -- mxcli has no writer for it. + listview lvDropped (DataSource: database from Clicks.Thing, OnClick: show_page Clicks.P_Clicks) { + dynamictext tLv (Content: 'a row') + } +} +/ diff --git a/mdl/executor/validate_widget_onclick.go b/mdl/executor/validate_widget_onclick.go new file mode 100644 index 000000000..5ba04477e --- /dev/null +++ b/mdl/executor/validate_widget_onclick.go @@ -0,0 +1,122 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Check-time validation for the `onclick:` / `action:` widget property. +// +// `OnClick:` is an ALIAS for `Action:` — the visitor stores both under +// Properties["Action"] (issue #603, the clickable container) — so by the time a +// validator sees it the two are indistinguishable, and this rule covers both +// spellings on purpose. +// +// mxcli writes that property for exactly three static widget kinds. Everywhere +// else it parses, passes check, passes exec, reaches no stored property, and +// the rendered element has no handler: measured on Mendix 11.13, a data view, +// a listview and a dynamictext each keep their OnClick through `exec` and lose +// it by `describe`, while the container beside them keeps it. The same +// type-agnostic property allow-list that hid #928's `editable:` hides this one — +// isBuiltinPropName answers "is this a real MDL property anywhere", not "is it +// valid on THIS widget". (CapTrackV2 FINDINGS §21) +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// The rule names the widget types it reports rather than reporting everything +// outside an allow-list of the three that DO write the action (container, +// button, navigation-list item). An allow-list looked tighter and was wrong: +// running it over the shipped examples turned up two false positives, both +// because `mxcli check` without `-p` has no widget registry, so `lookupWidgetDef` +// returns nil for a PLUGGABLE widget too and the caller's "static only" branch +// does not hold. `datagrid` is DataGrid 2 — a pluggable widget whose `onClick` +// the widget engine writes — and a bare `pluggablewidget` is the same story. +// +// So the default is silence, and a type earns a report only by being named +// below. A missed report costs a warning; a false one tells an author their +// working page is broken. + +// clickDroppedNoSlot are the STATIC widget types whose Mendix counterpart has no +// click action at all, so the property is meaningless rather than unimplemented. +// Checked against generated/metamodel — the arbiter per CLAUDE.md — by +// TestClickCapableTypesCarryClickActionInMetamodel. +var clickDroppedNoSlot = map[string]bool{ + "dataview": true, // Pages$DataView (the reported case) + "dynamictext": true, // Pages$DynamicText + "title": true, // Pages$Title + "textbox": true, // Pages$TextBox (has OnChange/OnEnter/OnLeave, no click) + "textarea": true, // Pages$TextArea (same) + "datepicker": true, // Pages$DatePicker (same) + "dropdown": true, // Pages$DropDown (same) + "checkbox": true, // Pages$CheckBox (same) + "radiobuttons": true, // Pages$RadioButtonGroup (same) + "radiobuttongroup": true, // Pages$RadioButtonGroup (alternate spelling) + "groupbox": true, // Pages$GroupBox + "tabcontainer": true, // Pages$TabContainer (has ActivePageOnChangeAction, no click) + "layoutgrid": true, // Pages$LayoutGrid + "snippetcall": true, // Pages$SnippetCall +} + +// clickCapableInMendix are the STATIC widget types Mendix DOES give a click +// action but mxcli does not write. Measured against generated/metamodel: +// +// Pages$ListView.ClickAction +// Pages$StaticImageViewer.ClickAction +// Pages$DynamicImageViewer.ClickAction +// +// They earn a different sentence, because the remedy is different: the model +// can hold the action, mxcli simply has no writer for it, so moving the action +// to a container is a workaround rather than the correct modelling. +// +// Bare `image` is deliberately absent — it is not always the legacy viewer. +var clickCapableInMendix = map[string]bool{ + "listview": true, + "staticimage": true, + "dynamicimage": true, +} + +// validateWidgetOnClick reports (MDL-WIDGET23) an `onclick:`/`action:` property +// that mxcli does not write for this widget type. +// +// A warning rather than an error, matching MDL-WIDGET20/21 — the same "silently +// dropped on write" family. Nothing fails the build here: the document is +// perfectly valid, the widget just does not do what the author asked, which is +// precisely why it needs saying out loud. +// +// Pluggable widgets are excluded by the caller: their action slots come from +// their own definition and are routed by the widget engine. +func validateWidgetOnClick(w *ast.WidgetV3, locationPrefix string) []linter.Violation { + if w == nil || w.GetAction() == nil { + return nil + } + typ := strings.ToLower(w.Type) + if !clickDroppedNoSlot[typ] && !clickCapableInMendix[typ] { + return nil + } + + var message, suggestion string + if clickCapableInMendix[typ] { + message = fmt.Sprintf( + "%s: widget `%s` (%s) has an on-click action, and Mendix does model one on %s — "+ + "but mxcli has no writer for it, so the value is dropped and the widget does nothing", + locationPrefix, w.Name, w.Type, w.Type) + suggestion = "Wrap the clickable part in a `container` and put the action there — a container's " + + "on-click IS written (Pages$DivContainer.OnClickAction)" + } else { + message = fmt.Sprintf( + "%s: widget `%s` (%s) has an on-click action, but Mendix models no click action on %s at all — "+ + "the value is dropped on write and the rendered element has no handler", + locationPrefix, w.Name, w.Type, w.Type) + suggestion = "Put the action on a `container` inside this widget (a container renders with " + + "tabindex/role=\"button\" and its on-click is written), or use an `actionbutton`/`linkbutton`" + } + + return []linter.Violation{{ + RuleID: "MDL-WIDGET23", + Severity: linter.SeverityWarning, + Message: message, + Suggestion: suggestion, + }} +} diff --git a/mdl/executor/validate_widget_onclick_test.go b/mdl/executor/validate_widget_onclick_test.go new file mode 100644 index 000000000..b24a72e49 --- /dev/null +++ b/mdl/executor/validate_widget_onclick_test.go @@ -0,0 +1,207 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "go/ast" + "go/parser" + "go/token" + "path/filepath" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// CapTrackV2 FINDINGS §21 — `DATAVIEW dv (…, OnClick: SHOW_PAGE …)` parses, +// passes `mxcli check`, is written by `exec` without a word, and the rendered +// element has no handler and no role="button". +// +// Measured on Mendix 11.13 by writing one page with four widgets and reading it +// back with `describe page`: +// +// container OnClick kept (Pages$DivContainer.OnClickAction) +// listview OnClick GONE (Mendix HAS ListView.ClickAction; mxcli has no writer) +// dataview OnClick GONE (Mendix has no click action on a data view at all) +// dynamictext OnClick GONE (same) +// +// `OnClick:` is an alias for `Action:` — the visitor stores both under +// Properties["Action"] — so the rule keys on the stored property and covers +// both spellings. + +func clickPage(widget string) string { + return `create page W.P ( Title: 'P' ) +{ + ` + widget + ` +}` +} + +// The reported case, plus the two other silent drops measured beside it. +func TestMDLWIDGET23_ReportsTheDroppedAction(t *testing.T) { + cases := []struct { + name string + widget string + want string // a phrase the message must carry + }{ + { + name: "data view (the reported case)", + widget: `DATAVIEW dv (DataSource: MICROFLOW W.DS, OnClick: SHOW_PAGE W.P) { DYNAMICTEXT t (Content: 'x') }`, + want: "no click action on dataview at all", + }, + { + name: "dynamictext", + widget: `DYNAMICTEXT t (Content: 'x', OnClick: SHOW_PAGE W.P)`, + want: "no click action on dynamictext at all", + }, + { + // Mendix models this one, so it earns the other sentence: the model + // could hold it, mxcli just does not write it. + name: "listview, which Mendix does model", + widget: `LISTVIEW lv (DataSource: DATABASE W.Product, OnClick: SHOW_PAGE W.P) { DYNAMICTEXT t (Content: 'x') }`, + want: "Mendix does model one on listview", + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := widgetViolations(t, clickPage(tc.widget), "MDL-WIDGET23") + if len(got) != 1 { + t.Fatalf("got %d MDL-WIDGET23 violations, want 1: %#v", len(got), got) + } + if got[0].Severity != linter.SeverityWarning { + t.Errorf("severity = %v, want Warning (the MDL-WIDGET20/21 family — nothing fails the build)", + got[0].Severity) + } + if !strings.Contains(got[0].Message, tc.want) { + t.Errorf("message should say %q, got %q", tc.want, got[0].Message) + } + if !strings.Contains(got[0].Suggestion, "container") { + t.Errorf("suggestion should point at a container, the widget whose on-click IS written: %q", + got[0].Suggestion) + } + }) + } +} + +// CONTROL 1: the widgets whose action mxcli DOES write must stay silent. A rule +// that fired on these would report the one shape that works, and `container +// (OnClick: …)` is the documented spelling for a clickable container (#603). +func TestMDLWIDGET23_WidgetsThatWriteTheActionAreClean(t *testing.T) { + for _, w := range []string{ + `CONTAINER c (OnClick: SHOW_PAGE W.P) { DYNAMICTEXT t (Content: 'x') }`, + `CONTAINER c (Action: SHOW_PAGE W.P) { DYNAMICTEXT t (Content: 'x') }`, + `ACTIONBUTTON b (Caption: 'Go', Action: SHOW_PAGE W.P)`, + `LINKBUTTON b (Caption: 'Go', Action: SHOW_PAGE W.P)`, + } { + t.Run(strings.Fields(w)[0]+"/"+strings.Fields(w)[3], func(t *testing.T) { + if got := widgetViolations(t, clickPage(w), "MDL-WIDGET23"); len(got) != 0 { + t.Errorf("MDL-WIDGET23 fired on a widget whose action is written: %#v", got) + } + }) + } +} + +// CONTROL 2: PLUGGABLE widgets are written by the widget engine from their own +// definition, so they must stay silent — including when no project is given. +// +// This is the regression an earlier draft actually shipped: the rule reported +// everything outside an allow-list, and `mxcli check` without `-p` has no widget +// registry, so `lookupWidgetDef` returns nil for a pluggable widget too and the +// caller's "static widgets only" branch does not hold. Three shipped examples +// were flagged — `datagrid` is DataGrid 2, a pluggable widget whose `onClick` +// the engine writes. The rule now names the types it reports instead. +func TestMDLWIDGET23_PluggableWidgetsAreClean(t *testing.T) { + for _, w := range []string{ + `DATAGRID dg (DataSource: DATABASE W.Product, onClick: microflow W.ACT) { COLUMN c1 (Attribute: Name) }`, + `PLUGGABLEWIDGET 'com.mendix.widget.custom.badgebutton.BadgeButton' bb (onClick: microflow W.ACT)`, + } { + t.Run(strings.Fields(w)[0], func(t *testing.T) { + if got := widgetViolations(t, clickPage(w), "MDL-WIDGET23"); len(got) != 0 { + t.Errorf("MDL-WIDGET23 fired on a pluggable widget, whose action slot IS written: %#v", got) + } + }) + } +} + +// CONTROL 3: the rule keys on the property being present, not on the widget +// type — a data view without one is an ordinary data view. +func TestMDLWIDGET23_SilentWithoutTheProperty(t *testing.T) { + src := clickPage(`DATAVIEW dv (DataSource: MICROFLOW W.DS) { DYNAMICTEXT t (Content: 'x') }`) + if got := widgetViolations(t, src, "MDL-WIDGET23"); len(got) != 0 { + t.Errorf("MDL-WIDGET23 fired on a data view with no action: %#v", got) + } +} + +// clickCapableInMendix claims three Pages types carry a ClickAction. That claim +// decides which of the two messages an author sees, and it is hand-written, so +// read it back off generated/metamodel — the arbiter per CLAUDE.md. +func TestClickCapableTypesCarryClickActionInMetamodel(t *testing.T) { + fset := token.NewFileSet() + path := filepath.Join("..", "..", "generated", "metamodel", "types.go") + f, err := parser.ParseFile(fset, path, nil, 0) + if err != nil { + t.Fatalf("parse metamodel: %v", err) + } + + withClickAction := map[string]bool{} + ast.Inspect(f, func(n ast.Node) bool { + ts, ok := n.(*ast.TypeSpec) + if !ok || !strings.HasPrefix(ts.Name.Name, "Pages") { + return true + } + st, ok := ts.Type.(*ast.StructType) + if !ok { + return true + } + for _, fld := range st.Fields.List { + for _, nm := range fld.Names { + if nm.Name == "ClickAction" || nm.Name == "OnClickAction" { + withClickAction[ts.Name.Name] = true + } + } + } + return true + }) + + // Positive control: a parse that found nothing would let every assertion + // below pass vacuously. + if len(withClickAction) == 0 { + t.Fatal("found no Pages type with a click action — the parse is wrong, and a passing run would prove nothing") + } + if !withClickAction["PagesDivContainer"] { + t.Error("PagesDivContainer has no click action in the metamodel, yet mxcli writes one for a container") + } + + // Every type the rule claims Mendix models must really carry one. Getting + // this wrong sends the author the wrong remedy — "mxcli has no writer" when + // in fact the model has no slot. + for mdlType, metamodelType := range map[string]string{ + "listview": "PagesListView", + "staticimage": "PagesStaticImageViewer", + "dynamicimage": "PagesDynamicImageViewer", + } { + if !clickCapableInMendix[mdlType] { + t.Errorf("%s dropped out of clickCapableInMendix", mdlType) + } + if !withClickAction[metamodelType] { + t.Errorf("clickCapableInMendix says Mendix models a click action on %s, but %s carries none", + mdlType, metamodelType) + } + } + + // And the other direction, for the types the rule says have no slot at all. + for mdlType, metamodelType := range map[string]string{ + "dataview": "PagesDataView", + "dynamictext": "PagesDynamicText", + "textbox": "PagesTextBox", + "groupbox": "PagesGroupBox", + } { + if clickCapableInMendix[mdlType] { + t.Errorf("%s is listed as click-capable; the message would name the wrong remedy", mdlType) + } + if withClickAction[metamodelType] { + t.Errorf("%s DOES carry a click action in the metamodel — %s belongs in clickCapableInMendix", + metamodelType, mdlType) + } + } +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index e3628d8d8..7ee78a8ae 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -141,6 +141,9 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc // "silently dropped on write" family, but the flat property // allow-list cannot see it because it is type-agnostic. out = append(out, validateWidgetEditability(w, locationPrefix)...) + // FINDINGS §21: the same blind spot for `onclick:`/`action:`, which + // only three static widget kinds actually store. + out = append(out, validateWidgetOnClick(w, locationPrefix)...) } else if def != nil { out = append(out, validatePluggableEditability(w, locationPrefix)...) } From 020bb8530e9733b2f53c2d6a123eb2072b20032b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 07:49:57 +0000 Subject: [PATCH 07/17] feat(settings): check the after-startup microflow returns Boolean (MDL073) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `alter settings model AfterStartupMicroflow = 'Mod.MF_Seed'` accepted a microflow with no return type, `mxcli check` passed, and the build failed: [CE0142] "After startup microflow should return a boolean" #274 made ALTER SETTINGS resolve the qualified names it writes, so a MISSPELLED microflow is caught. This one is not misspelled: the name resolves perfectly, and the constraint is on the thing the setting names rather than on the reference. Nothing looked at the return type. The shape that trips it in practice is a seed/demo-data microflow wired to after-startup — it does its work, returns nothing, and the build refuses it, well away from the statement that caused it. `alter microflow … returns …` does not parse either, so the remedy is DROP + CREATE (the setting survives that, being stored by name). Two halves, one function, so they cannot drift: - ValidateAfterStartupReturnType runs with NO project, over a microflow the script itself creates. That is the usual shape — create the seed microflow, then wire it — and the answer is in the script. - validateSettingsReferences covers a microflow that was already stored, reading its return type from the backend. flowSignature gained ReturnKind: its existing Returns field holds the entity name for object/list returns and so cannot tell a Boolean from a void. Deliberately narrow, with a control test for each limit. BeforeShutdown and HealthCheck are NOT type-checked — their rules have not been measured here, and asserting one on a guess is the same defect facing the other way. A microflow whose return type cannot be established says nothing rather than refusing a script that builds. And a Boolean after-startup microflow stays clean, which a check that simply rejected the setting would also have to pass. Reported as CapTrackV2 FINDINGS §6. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .claude/skills/fix-issue/findings/sdk.jsonl | 1 + CLAUDE.md | 4 +- ...after-startup-must-return-boolean.fail.mdl | 41 ++++++ ...ck-6-after-startup-must-return-boolean.mdl | 30 ++++ mdl/executor/cmd_settings.go | 4 +- mdl/executor/helpers.go | 29 ++++ mdl/executor/validate.go | 4 +- mdl/executor/validate_datasource_args.go | 8 +- mdl/executor/validate_program.go | 7 + mdl/executor/validate_settings_refs.go | 135 +++++++++++++++++- mdl/executor/validate_settings_refs_test.go | 112 ++++++++++++++- 12 files changed, 360 insertions(+), 16 deletions(-) create mode 100644 mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.fail.mdl create mode 100644 mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.mdl diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 43c4a09a6..34eb9618f 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -509,3 +509,4 @@ {"area": "mdl/executor", "date": "2026-09-04", "symptom": "An app whose domain model uses `DELETE_BEHAVIOR PREVENT` will not START. `mx check` reports 0 errors; the failure is at runtime init: `ERROR - M2EE: An error occurred while initializing the Runtime: None.get` / `java.util.NoSuchElementException: None.get` at `SchemeFactory$.\u2026$setDeleteBehavior(SchemeFactory.scala:515)`. Rewriting the associations to the default behaviour takes the app from crash to HTTP 200.", "cause": "mxcli wrote `ChildDeleteBehavior: DeleteMeIfNoReferences` with `ChildErrorMessage: null` \u2014 the codec registers `DomainModels$DeleteBehavior` with both error-message slots in `NullFields`, unconditionally. Studio Pro writes a `Texts$Text` there for that behaviour only. MDL had no syntax for the message at all. Fixed by adding SQL's referential actions (`ON DELETE CASCADE|RESTRICT|SET NULL`, plus `ERROR_MESSAGE '\u2026'`) and writing the text element when the child behaviour is DeleteMeIfNoReferences \u2014 only then, and only on the child side.", "file": "`mdl/grammar/MDLLexer.g4` + `domains/MDLDomainModel.g4` + `domains/MDLSettings.g4` (keyword rule), `mdl/visitor/visitor_association.go` + `visitor_helpers.go` (`buildReferentialAction`), `mdl/ast/ast_association.go`, `sdk/domainmodel/domainmodel.go` (`DeleteBehavior.ErrorMessage`), `mdl/backend/modelsdk/domainmodel_write.go` (`deleteErrorText`) + `domainmodel.go` (read back), `mdl/executor/cmd_associations.go` (`describeDeleteClause`)", "insight": "**The field is CONDITIONAL in Studio Pro's own dialog** \u2014 it only appears once the third radio button is selected, which is why a screenshot of the association properties showed no message field and nearly falsified the reporter's (correct) diagnosis. Ask for the reference in the state that exhibits the behaviour, not the default state. A census of 47,789 units across 122 projects found 423 delete behaviours and **zero** using this one, so no reference existed anywhere until one was authored \u2014 that absence is why the null shipped and why nobody hit it sooner. Syntax note: Mendix's three behaviours ARE SQL's referential actions, and MDL's FROM/TO already matches a foreign key's direction (ParentPointer = FROM = FK owner, ChildPointer = TO = referenced, measured), so `ON DELETE RESTRICT` needed no invention and fixed a readability complaint that `DELETE_IF_NO_REFERENCES` had earned \u2014 MDL's names are Mendix's with the word **Me** dropped, and Me was the only word saying whose deletion it described. Controls, end-to-end on 11.13: a project authored by the pre-fix binary reproduces `None.get` at boot verbatim; the fixed one boots. DESCRIBE emits the ON DELETE form with the message, or a describe->exec round trip rebuilds the crash. The empty-message shape is INFERRED, not measured \u2014 flagged in `deleteErrorText`. Reported as CapTrackV2 FINDINGS \u00a71."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`return` inside a `loop` passes `mxcli check` AND `exec`, then mxbuild fails **CE0068** \"End events cannot be placed inside a loop.\" \u2014 even though MDL062 exists precisely to catch that. It fires only when the microflow header has NO `returns T as $Var` clause.", "cause": "MDL062 stood down for the whole microflow whenever the AS clause was present, on two claims: that buildFlowGraph synthesizes the End event from the variable so none lands in the loop, and that the shape builds CE0109 instead. `describe microflow` shows the in-loop `return` written either way, so the first was never true. The second was a measurement artefact: mxbuild reports ONE error per microflow, and in the shape that was measured the AS variable was never assigned, so CE0109 \"Undefined variable\" won the race and hid CE0068 underneath. Adding `declare $Done Boolean = false` and changing nothing else turns the same microflow from CE0109 into CE0068. The exemption was deleted, not narrowed.", "file": "`mdl/executor/validate_microflow_ce_gaps.go` (checkReturnInLoop \u2014 the `v.returnType.Variable != \"\"` early return); test inverted in `validate_microflow_ce_gaps_test.go` (TestMDL062_ExemptsReturnsAsClause -> TestMDL062_FiresWithReturnsAsClause); examples `mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause{,.fail}.mdl`", "insight": "A second error in the same document can HIDE the one you are measuring, because mxbuild reports one error per microflow. An exemption justified by \"measured: builds X instead\" is only sound if the reproduction was otherwise valid \u2014 here the repro was broken in a second way, and the error that surfaced was the one nobody was asking about. When a measurement says a construct is clean, add the minimum that removes every OTHER error from that document and measure again; the differential (CE0109 -> CE0068 on one added `declare`) is what settles it. Reported as CapTrackV2 FINDINGS \u00a719."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`dataview dv (\u2026, OnClick: SHOW_PAGE \u2026)` parses, `mxcli check` is clean, `exec` writes the page without a word \u2014 and the rendered element has no handler and no role=\"button\". The same silence on `dynamictext`, `listview` and every other widget except a container or a button.", "cause": "`OnClick:` is an ALIAS for `Action:` (both stored as Properties[\"Action\"], #603), and mxcli writes that property for three widget kinds only: container/customcontainer, the buttons, and a navigationlist item. Every other widget drops it, and the property allow-lists behind MDL-WIDGET01/07 could not see that because they are widget-type AGNOSTIC \u2014 the same blind spot as #928's `editable:`. Added MDL-WIDGET23 (warning), with two messages: Mendix models no click action at all (dataview, dynamictext, inputs, groupbox\u2026) vs Mendix models one that mxcli cannot write (listview, staticimage, dynamicimage \u2014 measured against generated/metamodel).", "file": "`mdl/executor/validate_widget_onclick.go` (new), wired in `validate_widgets.go` beside validateWidgetEditability; `.claude/skills/mendix/create-page/reference/widgets.md`; example `mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl`", "insight": "The first draft reported everything OUTSIDE an allow-list of the three writers, and running it over the shipped examples flagged three of them. Cause: `mxcli check` without `-p` has no widget registry, so `lookupWidgetDef` returns nil for a PLUGGABLE widget too and the caller's \"static widgets only\" branch silently does not hold \u2014 `datagrid` is DataGrid 2, a pluggable widget whose onClick the engine does write. For any rule keyed on widget type, an allow-list makes the unknown case an ERROR and a deny-list makes it silence; pick the deny-list, because a missed warning costs nothing and a false one tells an author their working page is broken. Running a new rule across mdl-examples/ before wiring it up is what caught it \u2014 the same exercise #893 describes. Reported as CapTrackV2 FINDINGS \u00a721."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ALTER SETTINGS MODEL AfterStartupMicroflow = 'Mod.MF_Seed'` accepts a microflow with no return type, `mxcli check` passes, and the build fails **CE0142** \"After startup microflow should return a boolean\". `ALTER MICROFLOW \u2026 RETURNS \u2026` does not parse either, so the remedy is DROP + CREATE.", "cause": "#274 made ALTER SETTINGS resolve the qualified names it writes, which catches a MISSPELLED microflow. Here the name resolves perfectly \u2014 the constraint is on the thing the setting names, not on the reference, and nothing looked at the return type. Added MDL073: a project-less pass (ValidateAfterStartupReturnType) for a microflow the script itself creates, which is the usual shape, plus the stored return type on the project path. Both call one function so they cannot drift. flowSignature gained ReturnKind because its existing Returns field is the entity name and cannot tell Boolean from void.", "file": "`mdl/executor/validate_settings_refs.go` (checkAfterStartupReturnsBoolean + ValidateAfterStartupReturnType), `validate_program.go`, `helpers.go` (buildMicroflowReturnTypes), `validate_datasource_args.go` (flowSignature.ReturnKind); examples `mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean{,.fail}.mdl`", "insight": "\"The reference resolves\" and \"the reference is usable\" are different questions, and a resolver answers only the first. Whenever a setting stores a NAME, ask what the platform requires of the named thing \u2014 an existence check will pass and the build will still fail, and the error arrives with no connection to the statement that caused it. The scope discipline that goes with it: only AfterStartup is type-checked, because only its rule was measured; BeforeShutdown and HealthCheck are left alone rather than constrained on a guess, and there is a control test asserting that. Reported as CapTrackV2 FINDINGS \u00a76."} diff --git a/.claude/skills/fix-issue/findings/sdk.jsonl b/.claude/skills/fix-issue/findings/sdk.jsonl index 128289889..9c12a0f34 100644 --- a/.claude/skills/fix-issue/findings/sdk.jsonl +++ b/.claude/skills/fix-issue/findings/sdk.jsonl @@ -38,3 +38,4 @@ {"area": "sdk/pages", "date": "2026-08-28", "symptom": "`editable: Never` on a widget in `CREATE PAGE` has no effect — the field is editable in the browser, while `mxcli check`, `mxcli exec` and `mx check` are all clean, and `describe page` shows nothing either way", "cause": "The property was parsed and validated but nothing carried it to the writers, which hardcoded `SetEditable(\"Always\")` for TextBox, TextArea, CheckBox, DatePicker and RadioButtons. The semantic model had no field for it at all", "file": "`sdk/pages/pages_widgets.go` (`BaseWidget.Editable`, `CanonicalEditability`, `WidgetEditability`), `mdl/executor/cmd_pages_builder_v3.go` (`applyConditionalSettings`), `mdl/backend/modelsdk/widget_write.go`, `sdk/mpr/writer_widgets.go`, `mdl/executor/cmd_pages_describe_{parse,output}.go`", "insight": "**The isolating control is ALTER**: `ALTER PAGE … SET Editable = Never ON w` always wrote it correctly, which proves the value is representable and only the CREATE path loses it — run that before touching the writers. Put the field on `BaseWidget`, not on each input widget: one field covers all twelve editable types and MDL-WIDGET20 already refuses the property on the rest. `EDITABLE IF` must WIN over a plain `editable:` — the conditional settings element is what makes the enum `Conditional`, so honouring both writes an enum contradicting the element beside it. **`describe` must emit it too**, or a describe/diff compares equal while the document is wrong; it lived in the CheckBox case alone, so hoist it to `appendAppearanceProps`. **Pluggable widgets are NOT fixed by this** — no `.def.json` maps `editable` and the stored `CustomWidgets$CustomWidget` has no `Editable` key, so a combobox is reported (MDL-WIDGET21) rather than dropped; do not \"fix\" it by calling `SetEditable` on the gen CustomWidget, which is an inert sink. ako/mxcli-maintenance-2"} {"area": "sdk/mpr", "date": "2026-08-31", "raw": "| Every page menu item in an **mxcli-authored navigation profile** loses the page's own title, and `mx check` reports one **CW0263 \"Empty template\"** warning per item while errors stay at 0 (16 of them on a real 11.12.3 app) | `Forms$FormSettings.TitleOverride` written as an **empty** `Microflows$TextTemplate` instead of `null` — the #812 defect above, in the three writers #812 did not touch. An empty template is an override to `\"\"`, not the absence of one. Unlike the ShowPage/button paths there is nothing to preserve: `types.NavMenuItemSpec` has no title field, so MDL cannot author a navigation title override and the value is unconditionally null | `sdk/mpr/writer_navigation.go` (`buildFormSettingsBson`), `mdl/backend/modelsdk/navigation_write.go` (`navFormSettingsBson`), `modelsdk/mpr/nav_patch.go` (`navpBuildFormSettingsBson`) | Emit `{Key: \"TitleOverride\", Value: nil}` in all three. **Grep the builder's callers before concluding a navigation fix is menu-only** — each of the three is also the profile's **login page** builder (`writer_navigation.go:116`, `navigation_write.go:127`, `nav_patch.go:122`, plus `navigation_profile_add.go:108`), so one edit per engine covers both `Forms$FormAction` and `LoginPageSettings`, and a fix aimed only at menu items would have missed half the emitters. Verify against a **Studio Pro document already in the project**, not against the warning count: a blank app's own navigation unit stores `TitleOverride = null` on both the login settings and the home menu item. Read it with `f=$(grep -ral NavigationDocument app/mprcontents \\| head -1)` — `grep -a` is required, a `.mxunit` is raw BSON and plain `grep -rl` skips it as binary — then `strings -a \"$f\" \\| grep -c TextTemplate`: **4 before (3 page items + login page), 0 after**, with `grep -c TitleOverride` staying at 4 so the key is still written. Measured on 11.12.0 on both engines. These three paths are raw `bson.D` → `bson.Marshal` → `UpdateRawUnit`, so #812's second trap (a `codec.RegisterTypeDefaults` `NullFields` entry clobbered by another registration for the same `$Type`) cannot apply — which also means a shape assertion on the builder is the only unit-level guard there is. Repro `mdl-examples/bug-tests/989-navigation-title-override.mdl`. PR #989 |", "refs": ["#812", "#989"]} {"area": "sdk/mpr", "date": "2026-08-31", "symptom": "A document mxcli writes carries a different **typed-array marker** (the leading `int32` of a Mendix array) than the equivalent Studio Pro document — e.g. every list in a `CREATE OR REPLACE NAVIGATION` profile was `1` where Studio Pro writes `2` or `3`. No error, no warning, no build failure: it renders and opens", "cause": "The writers hand-build `bson.A{int32(1)}` per list. The marker is a **per-field constant**, not a function of the list's contents (`Forms$FormSettings.ParameterMappings` is `2` in 816 empty and 306 non-empty documents alike), so it cannot be derived — it has to be read off real documents", "file": "`sdk/mpr/writer_navigation.go` + `mdl/backend/modelsdk/navigation_write.go` + `modelsdk/mpr/nav_patch.go` (`navMarker*` / `navpMarker*` constants), `mdl/backend/modelsdk/navigation_profile_add.go`, `modelsdk/codec/defaults.go` (`RegisterListMarker`) for the codec paths", "insight": "**Census, don't reason.** Walk every `.mxunit` on the machine, tabulate `(parent $Type, field, marker, empty?)`, and take the value the Studio Pro documents carry — 19,078 files across 54 projects settled five of six navigation fields outright. **`int32(1)` is NOT invalid**, whatever `debug-bson.md` used to say: a Marketplace `.mpk` mxcli has never touched uses it for `CustomWidgets$WidgetValueType.AllowedTypes` (212k occurrences) and `Forms$Page.AllowedModuleRoles`. Believing otherwise turns a per-field mismatch into a phantom corruption bug and sends the fix in the wrong direction. Where the census has no observation, **find a document that has one** rather than picking: `HomeItems` was `2` in all 51 stored profiles but every one was empty, and `navigation_profile_add.go` wrote `3` from a PED session that could not be re-run. ako/TestApp settled it — a Studio Pro-authored profile whose `HomeItems` holds two `Navigation$RoleBasedHomePage` elements at marker **2**, the non-empty case the census could not reach. One project with the feature actually configured beats any amount of reasoning about empty lists. Verify by dumping the written document and the project's own pristine reference and diffing the marker column, not by `mx check`, which is silent on all of it"} +{"area": "sdk", "date": "2026-09-04", "symptom": "REPORTED AS A BUG, MEASURED AS A NON-BUG. `create association \u2026 type ReferenceSet owner Both` without `STORAGE TABLE` writes `StorageFormat: \"Column\"`, which was reported as \"not a legal many-to-many\" and worked around by respelling every such association.", "cause": "Nothing is broken. Measured on Mendix 11.13 against a live PostgreSQL, with the two spellings side by side in one app: `App.PA_PB` (ReferenceSet, StorageFormat Column) and `App.PC_PD` (ReferenceSet, StorageFormat Table) produce IDENTICAL DDL \u2014 `app$pa_pb(app$paid, app$pbid)` and `app$pc_pd(app$pcid, app$pdid)`, two FK constraints each. The app boots and serves HTTP 200, and `mx check` reports 0 errors. Mendix ignores StorageFormat for a reference set and always uses a junction table.", "file": "no code change \u2014 `mdl/executor/cmd_associations.go` defaults storageFormat to Column for every association type, and that is harmless", "insight": "A reported bug is a symptom plus an EXPLANATION, and the explanation is the part to re-measure. \"mxcli writes Column\" was true; \"which is not a legal many-to-many\" was the inference, and it cost the reporter a rewrite of ten associations. The cheap discriminator was to author BOTH spellings in one app and compare the DDL the runtime actually creates \u2014 a side-by-side control in the same boot, rather than reasoning about what a column could hold. Note the pkill trap from the same FINDINGS (\u00a711) applies when tidying up afterwards: `pkill -f \"mxcli run\"` matches the calling shell and kills it (exit 144)."} diff --git a/CLAUDE.md b/CLAUDE.md index f523e4965..9384aa2e8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -255,7 +255,9 @@ The reserved-word lists live in `mdl/executor/cmd_enumerations.go` (`mendixReser ### AfterStartupMicroflow Must Return Boolean -A microflow wired as the project's **after-startup** microflow must return `Boolean` — Mendix build fails with **CE0142** on a void (no-return) microflow. A common trip-up: a seed/demo-data microflow wired to after-startup will not build until it ends with a `return true` (Boolean). This is a Mendix platform rule, not an mxcli check. +A microflow wired as the project's **after-startup** microflow must return `Boolean` — Mendix build fails with **CE0142** on a void (no-return) microflow. A common trip-up: a seed/demo-data microflow wired to after-startup will not build until it ends with a `return true` (Boolean). + +`mxcli check` now reports it (**MDL073**), which it could not before: #274 made `ALTER SETTINGS` resolve the qualified names it writes, but the name here *resolves* — the constraint is on the thing it names, not on the reference. The check runs with **no project** when the script creates the microflow itself (the usual shape), and against the stored return type when it does not. A microflow whose return type cannot be established is left alone rather than guessed at. `BeforeShutdownMicroflow` and `HealthCheckMicroflow` are deliberately **not** type-checked — their rules have not been measured here. ### Overlay Writes: Never Invent a Key, Branch on `$Type` diff --git a/mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.fail.mdl b/mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.fail.mdl new file mode 100644 index 000000000..495c440ab --- /dev/null +++ b/mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.fail.mdl @@ -0,0 +1,41 @@ +-- NEGATIVE TEST — `mxcli check` must REFUSE this file. +-- +-- CapTrackV2 FINDINGS §6 — `ALTER SETTINGS MODEL AfterStartupMicroflow = '…'` +-- accepted a microflow with no return type and `mxcli check` passed. The build +-- then failed: +-- +-- [CE0142] "After startup microflow should return a boolean" +-- +-- #274 made ALTER SETTINGS resolve the qualified names it writes, so a +-- MISSPELLED microflow is caught. But the name here resolves perfectly — the +-- constraint is on the thing it names, not on the reference. The shape that +-- trips it in practice is a seed/demo-data microflow wired to after-startup: +-- it does its work, returns nothing, and the build refuses it. +-- +-- `ALTER MICROFLOW … RETURNS …` does not parse either, so the remedy is DROP + +-- CREATE. The setting survives that, since it is stored by qualified name. +-- +-- Verify: +-- mxcli check captrack-6-after-startup-must-return-boolean.fail.mdl +-- -- must report the return type and name CE0142, and exit non-zero +-- +-- The control lives beside this file as +-- captrack-6-after-startup-must-return-boolean.mdl. + +create module Startup; +/ +create persistent entity Startup.Seed ( + Code: string(50) +); +/ + +-- A seed microflow: does its work, returns nothing. +create microflow Startup.MF_SeedData () +begin + $S = create Startup.Seed (Code = 'demo'); + commit $S; +end; +/ + +alter settings model AfterStartupMicroflow = 'Startup.MF_SeedData'; +/ diff --git a/mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.mdl b/mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.mdl new file mode 100644 index 000000000..70d3f83d2 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean.mdl @@ -0,0 +1,30 @@ +-- CONTROL for CapTrackV2 FINDINGS §6 (see the .fail.mdl beside this file). +-- +-- A Boolean after-startup microflow is the whole point of the setting and must +-- stay clean — a check that refused every after-startup microflow would satisfy +-- the negative test and break the feature. +-- +-- Verify: +-- mxcli check captrack-6-after-startup-must-return-boolean.mdl -- 0 errors +-- mxcli exec captrack-6-after-startup-must-return-boolean.mdl -p app.mpr +-- mx check -p app.mpr -- 0 errors + +create module StartupOk; +/ +create persistent entity StartupOk.Seed ( + Code: string(50) +); +/ + +-- The same seed microflow, ending with the Boolean the platform requires. +create microflow StartupOk.MF_SeedData () +returns boolean +begin + $S = create StartupOk.Seed (Code = 'demo'); + commit $S; + return true; +end; +/ + +alter settings model AfterStartupMicroflow = 'StartupOk.MF_SeedData'; +/ diff --git a/mdl/executor/cmd_settings.go b/mdl/executor/cmd_settings.go index 0245a41dd..4d8ea5f8f 100644 --- a/mdl/executor/cmd_settings.go +++ b/mdl/executor/cmd_settings.go @@ -384,13 +384,15 @@ func alterSettings(ctx *ExecContext, stmt *ast.AlterSettingsStmt) error { } } else { var mfs, ents map[string]bool + var rets map[string]string if section == "model" { mfs = buildMicroflowQualifiedNames(ctx) + rets = buildMicroflowReturnTypes(ctx) } if section == "workflows" { ents = buildEntityQualifiedNames(ctx) } - if errs := validateSettingsReferences(stmt, mfs, ents, nil); len(errs) > 0 { + if errs := validateSettingsReferences(stmt, mfs, ents, rets, nil); len(errs) > 0 { return errs[0] } } diff --git a/mdl/executor/helpers.go b/mdl/executor/helpers.go index abba155a0..7da52990e 100644 --- a/mdl/executor/helpers.go +++ b/mdl/executor/helpers.go @@ -414,6 +414,35 @@ func buildMicroflowQualifiedNames(ctx *ExecContext) map[string]bool { return result } +// buildMicroflowReturnTypes maps each stored microflow's qualified name to the +// name of its return type ("Boolean", "String", …), or "" for a microflow that +// returns nothing. +// +// Separate from buildMicroflowQualifiedNames rather than folded into it: that +// set answers "does this name resolve", which several callers want and which +// must keep working when the return type is unavailable. A microflow missing +// here is a microflow whose type nothing may conclude anything from. +func buildMicroflowReturnTypes(ctx *ExecContext) map[string]string { + result := make(map[string]string) + h, err := getHierarchy(ctx) + if err != nil { + return result + } + mfs, err := ctx.Backend.ListMicroflows() + if err != nil { + return result + } + for _, mf := range mfs { + qn := h.GetQualifiedName(mf.ContainerID, mf.Name) + if mf.ReturnType == nil { + result[qn] = "" + continue + } + result[qn] = mf.ReturnType.GetTypeName() + } + return result +} + // buildQueueQualifiedNames returns the set of task queue qualified names in the // project, lower-cased — Mendix name resolution is case-insensitive and the // caller compares an author-written name against it. diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index 112c8ce68..85e03184d 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -658,13 +658,15 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext } } else { var mfs, ents map[string]bool + var rets map[string]string if strings.EqualFold(s.Section, "model") { mfs = buildMicroflowQualifiedNames(ctx) + rets = buildMicroflowReturnTypes(ctx) } if strings.EqualFold(s.Section, "workflows") { ents = buildEntityQualifiedNames(ctx) } - errs = validateSettingsReferences(s, mfs, ents, sc) + errs = validateSettingsReferences(s, mfs, ents, rets, sc) } if len(errs) > 0 { return errs[0] diff --git a/mdl/executor/validate_datasource_args.go b/mdl/executor/validate_datasource_args.go index 67c014cd4..2c8b04394 100644 --- a/mdl/executor/validate_datasource_args.go +++ b/mdl/executor/validate_datasource_args.go @@ -75,6 +75,11 @@ type flowSignature struct { // Returns is the entity qualified name of an object or list return type, // "" for anything else (including a flow that returns nothing). Returns string + // ReturnKind is the return type as declared, which Returns cannot express: + // a Boolean and a void microflow both leave Returns empty, and the + // after-startup check has to tell them apart (CE0142). TypeVoid where the + // flow declares no return at all. + ReturnKind ast.DataTypeKind } // paramNames returns the parameter names in declaration order, which is what the @@ -437,9 +442,10 @@ func sdkDataTypeEntity(dt microflows.DataType) string { // astFlowSignature builds a signature from a CREATE MICROFLOW / CREATE NANOFLOW // statement, for a flow this script has not written yet. func astFlowSignature(params []ast.MicroflowParam, ret *ast.MicroflowReturnType) *flowSignature { - sig := &flowSignature{} + sig := &flowSignature{ReturnKind: ast.TypeVoid} if ret != nil { sig.Returns = astDataTypeEntity(ret.Type) + sig.ReturnKind = ret.Type.Kind } for _, p := range params { entity := astDataTypeEntity(p.Type) diff --git a/mdl/executor/validate_program.go b/mdl/executor/validate_program.go index 792dd753e..263500cd2 100644 --- a/mdl/executor/validate_program.go +++ b/mdl/executor/validate_program.go @@ -186,6 +186,13 @@ func ValidateProgram(prog *ast.Program, projectPath string) []linter.Violation { // under --references, where it would only fire with -p (#836). violations = append(violations, ValidateGrantRoles(prog)...) + // Flag an after-startup microflow that does not return Boolean (CE0142). + // The reference RESOLVES — the name exists — so #274's existence check has + // nothing to say; the constraint is on the thing the setting names. When the + // script creates the microflow itself, which is the usual shape, the answer + // is in the script and needs no project (CapTrackV2 FINDINGS §6). + violations = append(violations, ValidateAfterStartupReturnType(prog)...) + // Flag an export mapping value whose member is a nested path — an export has // to produce the intermediate node, so Mendix rejects it with CE5015. The // answer is in the statement, so it runs here rather than under --references diff --git a/mdl/executor/validate_settings_refs.go b/mdl/executor/validate_settings_refs.go index 58e5597e8..7b8598579 100644 --- a/mdl/executor/validate_settings_refs.go +++ b/mdl/executor/validate_settings_refs.go @@ -9,6 +9,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/linter" ) // settingsMicroflowKeys are the model settings whose value is a microflow's @@ -30,7 +31,7 @@ var settingsMicroflowKeys = []string{ // Empty is not a reference — it clears the setting, which is Studio Pro's // "(none)" — and a name created earlier in the same script is not missing, the // same escape hatch every other family has. -func validateSettingsReferences(stmt *ast.AlterSettingsStmt, knownMicroflows, knownEntities map[string]bool, sc *scriptContext) []error { +func validateSettingsReferences(stmt *ast.AlterSettingsStmt, knownMicroflows, knownEntities map[string]bool, returnTypes map[string]string, sc *scriptContext) []error { if stmt == nil { return nil } @@ -47,13 +48,16 @@ func validateSettingsReferences(stmt *ast.AlterSettingsStmt, knownMicroflows, kn if ref == "" { continue // clearing the setting } - if knownMicroflows[ref] || (sc != nil && sc.microflows[ref]) { + if !knownMicroflows[ref] && !(sc != nil && sc.microflows[ref]) { + errs = append(errs, mdlerrors.NewValidationf( + "microflow not found: %s (referenced by %s) — the model stores the name as written, "+ + "so the build reports CE1613 \"The selected microflow no longer exists\"", + ref, key)) continue } - errs = append(errs, mdlerrors.NewValidationf( - "microflow not found: %s (referenced by %s) — the model stores the name as written, "+ - "so the build reports CE1613 \"The selected microflow no longer exists\"", - ref, key)) + if err := checkAfterStartupReturnsBoolean(key, ref, returnTypes, sc); err != nil { + errs = append(errs, err) + } } } @@ -69,6 +73,63 @@ func validateSettingsReferences(stmt *ast.AlterSettingsStmt, knownMicroflows, kn return errs } +// checkAfterStartupReturnsBoolean reports an after-startup microflow that does +// not return Boolean. +// +// Mendix requires it, and the build says so late and out of context: +// CE0142 "After startup microflow should return a boolean". The setting itself +// resolves fine — the name exists — so #274's existence check passes and the +// script writes a project that cannot build. The trip-up in the wild is a +// seed/demo-data microflow wired to after-startup: it does its work, returns +// nothing, and the build refuses it (CapTrackV2 FINDINGS §6). +// +// Only AfterStartupMicroflow is checked. BeforeShutdown and HealthCheck have +// their own rules, and neither has been measured here — asserting one on a +// guess would be the same defect in the other direction. +// +// A microflow whose return type is unknown is left alone. That covers a +// script-created flow the context did not record and a backend that could not +// list return types: in both cases nothing is KNOWN to be wrong, and a refusal +// would block a script that builds. +func checkAfterStartupReturnsBoolean(key, ref string, returnTypes map[string]string, sc *scriptContext) error { + if !strings.EqualFold(key, "AfterStartupMicroflow") { + return nil + } + var actual string + switch { + case sc != nil && sc.microflows[ref]: + sig, ok := sc.flowParams[strings.ToLower(ref)] + if !ok || sig == nil { + return nil // created in this script, signature not recorded + } + if sig.ReturnKind == ast.TypeBoolean { + return nil + } + actual = sig.ReturnKind.String() + if sig.ReturnKind == ast.TypeVoid { + actual = "nothing" + } + default: + stored, ok := returnTypes[ref] + if !ok { + return nil // return type unavailable — say nothing rather than guess + } + if stored == "Boolean" { + return nil + } + actual = stored + if stored == "" { + actual = "nothing" + } + } + + return mdlerrors.NewValidationf( + "after-startup microflow %s returns %s, but Mendix requires Boolean — the build reports "+ + "CE0142 \"After startup microflow should return a boolean\". The setting stores only the "+ + "name, so this is not caught by resolving the reference.", + ref, actual) +} + // validateSettingsConstantRef resolves the constant an override names. // // This one is not merely a dangling pointer: `alter settings constant 'Typo' @@ -121,3 +182,65 @@ func nearestConstant(ref string, known map[string]bool) string { sort.Strings(candidates) return candidates[0] } + +// ValidateAfterStartupReturnType (MDL073) is the project-less half of the +// after-startup check. +// +// The rule needs the microflow's return type, and the overwhelmingly common +// shape is one script that CREATES the seed microflow and wires it in the same +// breath — so the answer is in the script itself and `mxcli check` with no +// project can give it. The project path (validateSettingsReferences) covers the +// other half, where the microflow was already stored. +// +// Both call checkAfterStartupReturnsBoolean, so the two cannot drift. +func ValidateAfterStartupReturnType(prog *ast.Program) []linter.Violation { + if prog == nil { + return nil + } + + returnTypes := map[string]string{} + for _, stmt := range prog.Statements { + mf, ok := stmt.(*ast.CreateMicroflowStmt) + if !ok { + continue + } + if mf.ReturnType == nil { + returnTypes[mf.Name.String()] = "" + continue + } + returnTypes[mf.Name.String()] = mf.ReturnType.Type.Kind.String() + } + if len(returnTypes) == 0 { + return nil + } + + var out []linter.Violation + for _, stmt := range prog.Statements { + s, ok := stmt.(*ast.AlterSettingsStmt) + if !ok || !strings.EqualFold(s.Section, "model") { + continue + } + raw, ok := s.Properties["AfterStartupMicroflow"] + if !ok { + continue + } + ref := settingsValueToString(raw) + if ref == "" { + continue + } + // Only a microflow this script defines: anything else needs the project, + // and reporting it from here would guess. + if _, defined := returnTypes[ref]; !defined { + continue + } + if err := checkAfterStartupReturnsBoolean("AfterStartupMicroflow", ref, returnTypes, nil); err != nil { + out = append(out, linter.Violation{ + RuleID: "MDL073", + Severity: linter.SeverityError, + Message: err.Error(), + Suggestion: "End the microflow with `return true;` and declare `returns boolean` on it", + }) + } + } + return out +} diff --git a/mdl/executor/validate_settings_refs_test.go b/mdl/executor/validate_settings_refs_test.go index e0265350b..5a884225d 100644 --- a/mdl/executor/validate_settings_refs_test.go +++ b/mdl/executor/validate_settings_refs_test.go @@ -26,11 +26,11 @@ func TestValidateSettingsRefs_MicroflowKeys(t *testing.T) { for _, key := range []string{"AfterStartupMicroflow", "BeforeShutdownMicroflow", "HealthCheckMicroflow"} { if errs := validateSettingsReferences( - settingsRefStmt("model", map[string]any{key: "SM.MF_Startup"}), known, nil, sc); len(errs) != 0 { + settingsRefStmt("model", map[string]any{key: "SM.MF_Startup"}), known, nil, nil, sc); len(errs) != 0 { t.Errorf("%s: a microflow that exists was rejected: %v", key, errs) } errs := validateSettingsReferences( - settingsRefStmt("model", map[string]any{key: "Nope.MF_Missing"}), known, nil, sc) + settingsRefStmt("model", map[string]any{key: "Nope.MF_Missing"}), known, nil, nil, sc) if len(errs) != 1 { t.Fatalf("%s: errors = %v, want exactly one", key, errs) } @@ -45,7 +45,7 @@ func TestValidateSettingsRefs_MicroflowKeys(t *testing.T) { func TestValidateSettingsRefs_EmptyValueClearsAndIsNotAReference(t *testing.T) { if errs := validateSettingsReferences( settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": ""}), - map[string]bool{}, nil, newScriptContext()); len(errs) != 0 { + map[string]bool{}, nil, nil, newScriptContext()); len(errs) != 0 { t.Errorf("clearing the setting was reported as a bad reference: %v", errs) } } @@ -58,7 +58,7 @@ func TestValidateSettingsRefs_ScriptCreatedMicroflowCounts(t *testing.T) { if errs := validateSettingsReferences( settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Startup"}), - map[string]bool{}, nil, sc); len(errs) != 0 { + map[string]bool{}, nil, nil, sc); len(errs) != 0 { t.Errorf("a microflow created in this script was reported missing: %v", errs) } } @@ -68,11 +68,11 @@ func TestValidateSettingsRefs_WorkflowUserEntity(t *testing.T) { sc := newScriptContext() if errs := validateSettingsReferences( - settingsRefStmt("workflows", map[string]any{"UserEntity": "System.User"}), nil, entities, sc); len(errs) != 0 { + settingsRefStmt("workflows", map[string]any{"UserEntity": "System.User"}), nil, entities, nil, sc); len(errs) != 0 { t.Errorf("an entity that exists was rejected: %v", errs) } errs := validateSettingsReferences( - settingsRefStmt("workflows", map[string]any{"UserEntity": "Nope.User"}), nil, entities, sc) + settingsRefStmt("workflows", map[string]any{"UserEntity": "Nope.User"}), nil, entities, nil, sc) if len(errs) != 1 || !strings.Contains(errs[0].Error(), "Nope.User") { t.Errorf("errors = %v, want one naming Nope.User", errs) } @@ -130,3 +130,103 @@ func TestNearestConstant_OnlySuggestsARealNearMiss(t *testing.T) { t.Errorf("unrelated name got a suggestion: %q", got) } } + +// CapTrackV2 FINDINGS §6 — `ALTER SETTINGS MODEL AfterStartupMicroflow = '…'` +// accepted a microflow with no return type and `mxcli check` passed. The build +// then failed with CE0142 "After startup microflow should return a boolean". +// +// The reference resolved fine — the name exists — so #274's existence check had +// nothing to say. The setting stores only a name; the constraint is on the thing +// it names. The trip-up in the wild is a seed/demo-data microflow wired to +// after-startup: it does its work, returns nothing, and the build refuses it. +func TestValidateSettingsRefs_AfterStartupMustReturnBoolean(t *testing.T) { + known := map[string]bool{"SM.MF_Seed": true, "SM.MF_Ok": true} + rets := map[string]string{"SM.MF_Seed": "", "SM.MF_Ok": "Boolean"} + + errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Seed"}), + known, nil, rets, newScriptContext()) + if len(errs) != 1 { + t.Fatalf("errors = %v, want one about the return type", errs) + } + if !strings.Contains(errs[0].Error(), "CE0142") { + t.Errorf("the message should name the build error it predicts: %q", errs[0]) + } + if !strings.Contains(errs[0].Error(), "returns nothing") { + t.Errorf("the message should say what it returns instead: %q", errs[0]) + } + + // CONTROL: a Boolean microflow is the whole point of the setting and must + // stay clean. A check that rejected every after-startup microflow would + // satisfy the assertion above and break the feature. + if errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Ok"}), + known, nil, rets, newScriptContext()); len(errs) != 0 { + t.Errorf("a Boolean after-startup microflow was rejected: %v", errs) + } +} + +// A non-Boolean return type is reported by name, not lumped in with void. +func TestValidateSettingsRefs_AfterStartupWrongTypeNamesIt(t *testing.T) { + errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Str"}), + map[string]bool{"SM.MF_Str": true}, nil, map[string]string{"SM.MF_Str": "String"}, + newScriptContext()) + if len(errs) != 1 || !strings.Contains(errs[0].Error(), "returns String") { + t.Errorf("errors = %v, want one naming String", errs) + } +} + +// CONTROL: the OTHER two microflow settings are left alone. Their rules have not +// been measured here, and asserting one on a guess is the same defect in the +// other direction. +func TestValidateSettingsRefs_OtherMicroflowSettingsAreNotTypeChecked(t *testing.T) { + known := map[string]bool{"SM.MF_Void": true} + rets := map[string]string{"SM.MF_Void": ""} + for _, key := range []string{"BeforeShutdownMicroflow", "HealthCheckMicroflow"} { + if errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{key: "SM.MF_Void"}), + known, nil, rets, newScriptContext()); len(errs) != 0 { + t.Errorf("%s was type-checked on a guess: %v", key, errs) + } + } +} + +// CONTROL: an unknown return type says nothing. That covers a backend that could +// not list return types and a script-created flow whose signature was not +// recorded — in neither case is anything KNOWN to be wrong, and a refusal would +// block a script that builds. +func TestValidateSettingsRefs_UnknownReturnTypeIsSilent(t *testing.T) { + if errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Startup"}), + map[string]bool{"SM.MF_Startup": true}, nil, nil, newScriptContext()); len(errs) != 0 { + t.Errorf("errors = %v, want none when the return type is unavailable", errs) + } +} + +// A microflow created EARLIER IN THE SAME SCRIPT is the common shape — create +// the seed microflow, then wire it — so the check has to reach it too, from the +// signature the script context already records. +func TestValidateSettingsRefs_AfterStartupCreatedInTheSameScript(t *testing.T) { + mk := func(ret *ast.MicroflowReturnType) *scriptContext { + sc := newScriptContext() + sc.microflows["SM.MF_Seed"] = true + sc.recordFlowParams("SM.MF_Seed", nil, ret) + return sc + } + + errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Seed"}), + nil, nil, nil, mk(nil)) + if len(errs) != 1 || !strings.Contains(errs[0].Error(), "returns nothing") { + t.Errorf("errors = %v, want one about a void script-created microflow", errs) + } + + // CONTROL: the same flow declared Boolean is clean. + boolean := &ast.MicroflowReturnType{Type: ast.DataType{Kind: ast.TypeBoolean}} + if errs := validateSettingsReferences( + settingsRefStmt("model", map[string]any{"AfterStartupMicroflow": "SM.MF_Seed"}), + nil, nil, nil, mk(boolean)); len(errs) != 0 { + t.Errorf("a Boolean script-created microflow was rejected: %v", errs) + } +} From a54e8b40f0812d06e4646c2534e5b5af215d158d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 08:20:17 +0000 Subject: [PATCH 08/17] fix(pages): write SIGN_OUT instead of dropping it (both engines) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ACTIONBUTTON … (Action: SIGN_OUT)` was refused by the default engine: client action *pages.SignOutClientAction not yet supported by the modelsdk engine — rerun with MXCLI_ENGINE=legacy The refusal was honest. The advice was not. The legacy writer had no case for the action either, and its default branch is QUIET — it returns Forms$NoAction for anything unmatched — so the recommended escape hatch produced a button that rendered, said "Sign out", and did nothing, with `mxcli check`, `exec` and `mx check` all clean. Measured on Mendix 11.13, `describe page` came back `actionbutton btnOut (Caption: 'Sign out')` with no action at all, and the stored BSON held Forms$NoAction. Both engines now write the same document, and DESCRIBE renders `sign_out` so it round-trips: { "$Type": "Forms$SignOutClientAction", "DisabledDuringExecution": true } Two keys and no more, pinned against a Studio Pro-authored sign-out button in ako/TestApp. That reference is provably Studio Pro's rather than mxcli's, because until this change NEITHER engine could emit the type — which is also why the shape could not have been guessed from the writers. Verified with the fix reverted, one engine at a time: modelsdk fails with the refusal verbatim, legacy fails by writing Forms$NoAction. mx check is 0 errors on both engines' output, before and after — this was never a build error, which is exactly what made it dangerous. The control pins the fallback separately. A test asserting "SIGN_OUT is no longer NoAction" would also pass if someone had merely softened the default, so OPEN_LINK — still unimplemented — is asserted to STILL hit Forms$NoAction on legacy and still be refused on modelsdk. OPEN_LINK is left unimplemented deliberately: gen calls it OpenLinkClientAction and its Address is an element rather than a string, so it is a separate job. `mxcli syntax page.action` listed it as available with no caveat; it now says it is written by neither engine and points at a nanoflow instead. Reported as CapTrackV2 FINDINGS §10. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + cmd/mxcli/syntax/features_page.go | 2 +- .../bug-tests/captrack-10-sign-out-action.mdl | 43 ++++++++++ mdl/backend/modelsdk/widget_write.go | 17 ++++ .../modelsdk/widget_write_signout_test.go | 78 +++++++++++++++++++ mdl/executor/cmd_pages_describe_output.go | 2 + sdk/mpr/writer_widgets_action.go | 15 ++++ sdk/mpr/writer_widgets_action_test.go | 65 ++++++++++++++++ 8 files changed, 222 insertions(+), 1 deletion(-) create mode 100644 mdl-examples/bug-tests/captrack-10-sign-out-action.mdl create mode 100644 mdl/backend/modelsdk/widget_write_signout_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 34eb9618f..47f8b44fc 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -510,3 +510,4 @@ {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`return` inside a `loop` passes `mxcli check` AND `exec`, then mxbuild fails **CE0068** \"End events cannot be placed inside a loop.\" \u2014 even though MDL062 exists precisely to catch that. It fires only when the microflow header has NO `returns T as $Var` clause.", "cause": "MDL062 stood down for the whole microflow whenever the AS clause was present, on two claims: that buildFlowGraph synthesizes the End event from the variable so none lands in the loop, and that the shape builds CE0109 instead. `describe microflow` shows the in-loop `return` written either way, so the first was never true. The second was a measurement artefact: mxbuild reports ONE error per microflow, and in the shape that was measured the AS variable was never assigned, so CE0109 \"Undefined variable\" won the race and hid CE0068 underneath. Adding `declare $Done Boolean = false` and changing nothing else turns the same microflow from CE0109 into CE0068. The exemption was deleted, not narrowed.", "file": "`mdl/executor/validate_microflow_ce_gaps.go` (checkReturnInLoop \u2014 the `v.returnType.Variable != \"\"` early return); test inverted in `validate_microflow_ce_gaps_test.go` (TestMDL062_ExemptsReturnsAsClause -> TestMDL062_FiresWithReturnsAsClause); examples `mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause{,.fail}.mdl`", "insight": "A second error in the same document can HIDE the one you are measuring, because mxbuild reports one error per microflow. An exemption justified by \"measured: builds X instead\" is only sound if the reproduction was otherwise valid \u2014 here the repro was broken in a second way, and the error that surfaced was the one nobody was asking about. When a measurement says a construct is clean, add the minimum that removes every OTHER error from that document and measure again; the differential (CE0109 -> CE0068 on one added `declare`) is what settles it. Reported as CapTrackV2 FINDINGS \u00a719."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`dataview dv (\u2026, OnClick: SHOW_PAGE \u2026)` parses, `mxcli check` is clean, `exec` writes the page without a word \u2014 and the rendered element has no handler and no role=\"button\". The same silence on `dynamictext`, `listview` and every other widget except a container or a button.", "cause": "`OnClick:` is an ALIAS for `Action:` (both stored as Properties[\"Action\"], #603), and mxcli writes that property for three widget kinds only: container/customcontainer, the buttons, and a navigationlist item. Every other widget drops it, and the property allow-lists behind MDL-WIDGET01/07 could not see that because they are widget-type AGNOSTIC \u2014 the same blind spot as #928's `editable:`. Added MDL-WIDGET23 (warning), with two messages: Mendix models no click action at all (dataview, dynamictext, inputs, groupbox\u2026) vs Mendix models one that mxcli cannot write (listview, staticimage, dynamicimage \u2014 measured against generated/metamodel).", "file": "`mdl/executor/validate_widget_onclick.go` (new), wired in `validate_widgets.go` beside validateWidgetEditability; `.claude/skills/mendix/create-page/reference/widgets.md`; example `mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl`", "insight": "The first draft reported everything OUTSIDE an allow-list of the three writers, and running it over the shipped examples flagged three of them. Cause: `mxcli check` without `-p` has no widget registry, so `lookupWidgetDef` returns nil for a PLUGGABLE widget too and the caller's \"static widgets only\" branch silently does not hold \u2014 `datagrid` is DataGrid 2, a pluggable widget whose onClick the engine does write. For any rule keyed on widget type, an allow-list makes the unknown case an ERROR and a deny-list makes it silence; pick the deny-list, because a missed warning costs nothing and a false one tells an author their working page is broken. Running a new rule across mdl-examples/ before wiring it up is what caught it \u2014 the same exercise #893 describes. Reported as CapTrackV2 FINDINGS \u00a721."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ALTER SETTINGS MODEL AfterStartupMicroflow = 'Mod.MF_Seed'` accepts a microflow with no return type, `mxcli check` passes, and the build fails **CE0142** \"After startup microflow should return a boolean\". `ALTER MICROFLOW \u2026 RETURNS \u2026` does not parse either, so the remedy is DROP + CREATE.", "cause": "#274 made ALTER SETTINGS resolve the qualified names it writes, which catches a MISSPELLED microflow. Here the name resolves perfectly \u2014 the constraint is on the thing the setting names, not on the reference, and nothing looked at the return type. Added MDL073: a project-less pass (ValidateAfterStartupReturnType) for a microflow the script itself creates, which is the usual shape, plus the stored return type on the project path. Both call one function so they cannot drift. flowSignature gained ReturnKind because its existing Returns field is the entity name and cannot tell Boolean from void.", "file": "`mdl/executor/validate_settings_refs.go` (checkAfterStartupReturnsBoolean + ValidateAfterStartupReturnType), `validate_program.go`, `helpers.go` (buildMicroflowReturnTypes), `validate_datasource_args.go` (flowSignature.ReturnKind); examples `mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean{,.fail}.mdl`", "insight": "\"The reference resolves\" and \"the reference is usable\" are different questions, and a resolver answers only the first. Whenever a setting stores a NAME, ask what the platform requires of the named thing \u2014 an existence check will pass and the build will still fail, and the error arrives with no connection to the statement that caused it. The scope discipline that goes with it: only AfterStartup is type-checked, because only its rule was measured; BeforeShutdown and HealthCheck are left alone rather than constrained on a guess, and there is a control test asserting that. Reported as CapTrackV2 FINDINGS \u00a76."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ACTIONBUTTON \u2026 (Action: SIGN_OUT)` is refused by the default engine \u2014 \"client action *pages.SignOutClientAction not yet supported by the modelsdk engine \u2014 rerun with MXCLI_ENGINE=legacy\" \u2014 and the suggested workaround SILENTLY produces a dead button: on legacy the action is written as `Forms$NoAction`, so it renders, says \"Sign out\", and does nothing, with `mxcli check`, `exec` and `mx check` all clean.", "cause": "Neither engine had a case for the action. modelsdk's clientActionToGen ended in a loud default; sdk/mpr's serializeClientAction ended in a QUIET one that returns Forms$NoAction for anything unmatched. Added the case to both. The document is two keys \u2014 `Forms$SignOutClientAction` + `DisabledDuringExecution: true` \u2014 pinned against a Studio Pro-authored button in ako/TestApp, plus `sign_out` in the DESCRIBE renderer so it round-trips. Both engines now emit byte-identical documents; mx check 0 errors on each. OPEN_LINK is still unwritten by both (gen calls it OpenLinkClientAction and its Address is an element, not a string) \u2014 the syntax topic now says so instead of listing it as available.", "file": "`mdl/backend/modelsdk/widget_write.go` (clientActionToGen), `sdk/mpr/writer_widgets_action.go` (serializeClientAction), `mdl/executor/cmd_pages_describe_output.go` (renderClientActionMDL), `cmd/mxcli/syntax/features_page.go`; example `mdl-examples/bug-tests/captrack-10-sign-out-action.mdl`", "insight": "When one engine refuses something and points at the other, CHECK THE OTHER before repeating the advice \u2014 the refusal is visible and the fallback is not, so the recommended escape hatch can be the strictly worse path. The structural tell is the shape of the default branch: modelsdk's raises, legacy's returns Forms$NoAction, and a silent default in a serializer converts every unimplemented type into data loss rather than an error. Grep for the fallthrough before trusting a switch. Note the control this needs: a test that SIGN_OUT is no longer NoAction can pass because someone softened the default, so pin the fallback separately with a type that is still unimplemented (OPEN_LINK). Reported as CapTrackV2 FINDINGS \u00a710."} diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 35cdd470d..0eab96e28 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -140,7 +140,7 @@ func init() { "button style", "primary", "danger", "success", "icon", "linkbutton", "link button", }, - Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", + Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nOPEN_LINK parses but is written by NEITHER engine: the modelsdk engine\nrefuses it, and the legacy engine writes Forms$NoAction, so the button\nrenders and does nothing (measured on 11.13). Call a nanoflow that opens\nthe URL instead. SIGN_OUT is written by both.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", Example: "ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\nACTIONBUTTON btnEdit (Caption: 'Edit',\n Action: SHOW_PAGE Module.EditPage(Item: $currentObject))\nLINKBUTTON btnDelete (Caption: 'Delete', Action: DELETE,\n Icon: 'Atlas_Core.Atlas_Filled.pencil')", SeeAlso: []string{"page.widgets"}, }) diff --git a/mdl-examples/bug-tests/captrack-10-sign-out-action.mdl b/mdl-examples/bug-tests/captrack-10-sign-out-action.mdl new file mode 100644 index 000000000..6432a9052 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-10-sign-out-action.mdl @@ -0,0 +1,43 @@ +-- CapTrackV2 FINDINGS §10 — `ACTIONBUTTON … (Action: SIGN_OUT)` was refused by +-- the default engine: +-- +-- client action *pages.SignOutClientAction not yet supported by the +-- modelsdk engine — rerun with MXCLI_ENGINE=legacy +-- +-- The refusal was honest. The advice was not: the legacy writer had no case for +-- the action either, so it fell through to the default and wrote +-- `Forms$NoAction`. The button rendered, said "Sign out", and did nothing — +-- with `mxcli check`, `exec` and `mx check` all clean. Measured on Mendix 11.13: +-- +-- before, either engine: describe -> actionbutton btnOut (Caption: 'Sign out') +-- stored -> Forms$NoAction +-- after, either engine: describe -> ... Action: sign_out +-- stored -> Forms$SignOutClientAction, +-- DisabledDuringExecution: true +-- +-- That two-key shape is pinned against a Studio Pro-authored sign-out button +-- (ako/TestApp). The reference is provably Studio Pro's rather than mxcli's, +-- because until this fix NEITHER engine could emit the type. +-- +-- Verify: +-- mxcli exec captrack-10-sign-out-action.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe page SignOut.P_Account" +-- -- must emit: Action: sign_out +-- MXCLI_ENGINE=legacy mxcli exec … -p other.mpr -- same document +-- mx check -p app.mpr -- 0 errors, before and after (this never was a +-- build error, which is what made it dangerous) + +create module SignOut; +/ + +create or replace page SignOut.P_Account + (Title: 'Account', Layout: Atlas_Core.Atlas_Default) +{ + container cActions { + actionbutton btnSignOut (Caption: 'Sign out', Action: sign_out, ButtonStyle: Default) + + -- CONTROL: the actions that already worked are untouched by the new case. + actionbutton btnClose (Caption: 'Close', Action: close_page) + } +} +/ diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index 4ee902beb..c7814838b 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -1483,6 +1483,23 @@ func clientActionToGen(a pages.ClientAction) (element.Element, error) { g.SetNumberOfPagesToClose2("") g.SetPageSettings(formSettingsToGen(x.PageName)) return g, nil + case *pages.SignOutClientAction: + // sign_out → Forms$SignOutClientAction. One property, and the reference + // pins its value: a Studio Pro-authored sign-out button (ako/TestApp, + // Mendix 11) stores exactly + // + // { "$Type": "Forms$SignOutClientAction", "DisabledDuringExecution": true } + // + // That document is provably Studio Pro's rather than mxcli's, because + // until now NEITHER engine could emit the type — modelsdk refused it and + // legacy wrote Forms$NoAction (CapTrackV2 FINDINGS §10). + g := genPg.NewSignOutClientAction() + if x.ID != "" { + g.SetID(element.ID(x.ID)) + } + assignID(g) + g.SetDisabledDuringExecution(true) + return g, nil case *pages.SetTaskOutcomeClientAction: g := genPg.NewSetTaskOutcomeClientAction() if x.ID != "" { diff --git a/mdl/backend/modelsdk/widget_write_signout_test.go b/mdl/backend/modelsdk/widget_write_signout_test.go new file mode 100644 index 000000000..62dcc8cf5 --- /dev/null +++ b/mdl/backend/modelsdk/widget_write_signout_test.go @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "strings" + "testing" + + genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// CapTrackV2 FINDINGS §10 — `ACTIONBUTTON … (Action: SIGN_OUT)` was refused +// outright by the default engine: +// +// client action *pages.SignOutClientAction not yet supported by the +// modelsdk engine — rerun with MXCLI_ENGINE=legacy +// +// The refusal was honest; the advice was not. The legacy writer had no case for +// the action either and fell through to Forms$NoAction, so the recommended +// escape hatch produced a button that rendered, said "Sign out", and did +// nothing — with check, exec and mx check all clean. +// +// The document is one property, pinned against a Studio Pro-authored button +// (ako/TestApp, Mendix 11): +// +// { "$Type": "Forms$SignOutClientAction", "DisabledDuringExecution": true } +// +// That reference is provably Studio Pro's rather than mxcli's, because until +// this change NEITHER engine could emit the type. +func TestClientActionToGen_SignOut(t *testing.T) { + el, err := clientActionToGen(&pages.SignOutClientAction{ + BaseElement: model.BaseElement{ID: "action-id"}, + }) + if err != nil { + t.Fatalf("SIGN_OUT is still refused by the modelsdk engine: %v", err) + } + g, ok := el.(*genPg.SignOutClientAction) + if !ok { + t.Fatalf("got %T, want *pages.SignOutClientAction", el) + } + if g.TypeName() != "Forms$SignOutClientAction" { + t.Errorf("$Type = %q, want Forms$SignOutClientAction", g.TypeName()) + } + if !g.DisabledDuringExecution() { + t.Error("DisabledDuringExecution is false; the Studio Pro reference stores true") + } +} + +// CONTROL 1: an action that is still unimplemented must still be REFUSED, not +// quietly written. Without this the test above could pass because the default +// branch had been softened, which is the exact failure the legacy engine had. +func TestClientActionToGen_StillRefusesWhatItCannotWrite(t *testing.T) { + _, err := clientActionToGen(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + Address: "https://example.com", + }) + if err == nil { + t.Fatal("OPEN_LINK was accepted; it has no writer, so accepting it means dropping it") + } + if !strings.Contains(err.Error(), "not yet supported") { + t.Errorf("unexpected message: %v", err) + } +} + +// CONTROL 2: the actions that already worked are untouched. +func TestClientActionToGen_ExistingActionsUnchanged(t *testing.T) { + for _, a := range []pages.ClientAction{ + &pages.SaveChangesClientAction{BaseElement: model.BaseElement{ID: "a"}}, + &pages.ClosePageClientAction{BaseElement: model.BaseElement{ID: "b"}}, + &pages.DeleteClientAction{BaseElement: model.BaseElement{ID: "c"}}, + } { + if _, err := clientActionToGen(a); err != nil { + t.Errorf("%T was refused: %v", a, err) + } + } +} diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index f96cab77b..d758689c5 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -1168,6 +1168,8 @@ func renderClientActionMDL(ctx *ExecContext, action map[string]any) string { case "Forms$SetTaskOutcomeClientAction", "Pages$SetTaskOutcomeClientAction": outcomeValue, _ := action["OutcomeValue"].(string) return "complete_task '" + strings.ReplaceAll(outcomeValue, "'", "''") + "'" + case "Forms$SignOutClientAction", "Pages$SignOutClientAction": + return "sign_out" case "Forms$NoClientAction", "Pages$NoClientAction": return "" default: diff --git a/sdk/mpr/writer_widgets_action.go b/sdk/mpr/writer_widgets_action.go index c7ec8942d..cdb79b6dd 100644 --- a/sdk/mpr/writer_widgets_action.go +++ b/sdk/mpr/writer_widgets_action.go @@ -53,6 +53,21 @@ func serializeClientAction(action pages.ClientAction) bson.D { {Key: "$Type", Value: "Forms$DeleteClientAction"}, {Key: "ClosePage", Value: a.ClosePage}, } + case *pages.SignOutClientAction: + // Until this case existed, SIGN_OUT fell through to the default below + // and was written as Forms$NoAction — so the button rendered, said + // "Sign out", and did nothing, with `mxcli check`, `exec` and `mx check` + // all clean. That made the documented workaround for the modelsdk + // engine's refusal ("rerun with MXCLI_ENGINE=legacy") the more dangerous + // of the two paths (CapTrackV2 FINDINGS §10). + // + // One property, pinned against a Studio Pro-authored button (ako/TestApp, + // Mendix 11): DisabledDuringExecution, true. + return bson.D{ + {Key: "$ID", Value: idToBsonBinary(string(a.ID))}, + {Key: "$Type", Value: "Forms$SignOutClientAction"}, + {Key: "DisabledDuringExecution", Value: true}, + } case *pages.CreateObjectClientAction: // Build EntityRef if entity is specified var entityRef any diff --git a/sdk/mpr/writer_widgets_action_test.go b/sdk/mpr/writer_widgets_action_test.go index 6b6b5de46..d534e2c5f 100644 --- a/sdk/mpr/writer_widgets_action_test.go +++ b/sdk/mpr/writer_widgets_action_test.go @@ -174,3 +174,68 @@ func TestPageClientAction_RequiredFields(t *testing.T) { t.Error("TitleOverride key missing entirely; Studio Pro writes it as an explicit null") } } + +// CapTrackV2 FINDINGS §10 — `ACTIONBUTTON … (Action: SIGN_OUT)` was refused by +// the default modelsdk engine with "client action *pages.SignOutClientAction +// not yet supported … rerun with MXCLI_ENGINE=legacy". +// +// That advice was the more dangerous of the two paths. The legacy writer had no +// case for the action either, so it fell through to the default below and wrote +// Forms$NoAction: the button rendered, said "Sign out", and did nothing, with +// `mxcli check`, `exec` and `mx check` all clean. Measured on Mendix 11.13 — +// `describe page` came back `actionbutton btnOut (Caption: 'Sign out')`, no +// action at all, and the stored BSON held Forms$NoAction. +// +// The shape is pinned against a Studio Pro-authored sign-out button +// (ako/TestApp), which is provably Studio Pro's rather than mxcli's: until this +// change NEITHER engine could emit the type. +func TestSignOutClientAction_IsNotSilentlyDroppedToNoAction(t *testing.T) { + doc := serializeClientAction(&pages.SignOutClientAction{ + BaseElement: model.BaseElement{ID: "action-id"}, + }) + if doc == nil { + t.Fatal("serializeClientAction returned nil") + } + + got := map[string]any{} + for _, e := range doc { + got[e.Key] = e.Value + } + + if got["$Type"] == "Forms$NoAction" { + t.Fatal("SIGN_OUT was written as Forms$NoAction — the button renders and does nothing, " + + "which check, exec and mx check all report as fine") + } + if got["$Type"] != "Forms$SignOutClientAction" { + t.Errorf("$Type = %v, want Forms$SignOutClientAction", got["$Type"]) + } + if got["DisabledDuringExecution"] != true { + t.Errorf("DisabledDuringExecution = %v, want true (the Studio Pro reference's only property)", + got["DisabledDuringExecution"]) + } + // The reference carries exactly these three keys and no more. An extra + // property is what Studio Pro refuses to open even when mxbuild accepts it. + if len(doc) != 3 { + t.Errorf("the action has %d keys, want 3 ($ID, $Type, DisabledDuringExecution): %v", len(doc), doc) + } +} + +// CONTROL: the default branch still exists and still yields Forms$NoAction, so +// this test proves something about SIGN_OUT rather than about the fallback being +// removed. OPEN_LINK is the action that still lands there — see FINDINGS §10. +func TestUnhandledClientActionStillFallsBackToNoAction(t *testing.T) { + doc := serializeClientAction(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + Address: "https://example.com", + }) + var typeName string + for _, e := range doc { + if e.Key == "$Type" { + typeName, _ = e.Value.(string) + } + } + if typeName != "Forms$NoAction" { + t.Errorf("$Type = %q; this control pins the fallback SIGN_OUT used to hit, "+ + "so that the test above cannot pass for the wrong reason", typeName) + } +} From 3bd242c410c520b16d1bd96f1ac114713a536f86 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 08:49:26 +0000 Subject: [PATCH 09/17] feat(pages): write OPEN_LINK instead of dropping it to Forms$NoAction MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ACTIONBUTTON … (Action: OPEN_LINK 'https://…')` reached storage on neither engine. modelsdk refused it; legacy fell through to its QUIET default and wrote Forms$NoAction, so the button rendered, said "Docs", and did nothing — with `mxcli check`, `exec` and `mx check` all clean. Same defect as the SIGN_OUT case in the previous commit, and the syntax help listed the action as available either way. Two traps here that a Studio Pro reference settled and reasoning would not. The STORAGE NAME is Forms$OpenLinkClientAction. The semantic type is LinkClientAction and the executor stamped "Forms$LinkClientAction", which is not what Mendix stores — a wrong $Type that never reached disk only because nothing could write the action at all. And the address is not a string property: it is a nested Forms$StaticOrDynamicString. Pinned against 31 Studio Pro-authored link buttons (ako/TestApp, FeedbackModule) — exactly five keys, LinkType "Web" in all 31: { "$Type": "Forms$OpenLinkClientAction", "Address": { "$Type": "Forms$StaticOrDynamicString", "AttributeRef": null, "IsDynamic": false, "Value": "https://www.mendix.com/" }, "DisabledDuringExecution": true, "LinkType": "Web" } 6 of those 31 are DYNAMIC — the address is read from an attribute at runtime. MDL cannot author that, so DESCRIBE flags such a button rather than printing its address as a literal, which would round-trip into a different link. gen declares a fourth property on Forms$StaticOrDynamicString, `Attribute`, that not one of the 31 documents carries. It is deliberately left unset: writing a key Mendix does not store is what produces a document mxbuild accepts and Studio Pro cannot open. The previous commit's "still unimplemented" controls named LinkClientAction, which stops being a valid control the moment this lands — they now name ShowHomePageClientAction, which has no gen type, no metamodel counterpart and no MDL statement that builds one, so it is structurally unwritable rather than merely not yet written. Both engines emit the same document; mx check 0 errors on each. Reported as CapTrackV2 FINDINGS §10. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- cmd/mxcli/syntax/features_page.go | 7 +- .../captrack-10-open-link-action.mdl | 53 ++++++++++++++ mdl/backend/modelsdk/widget_write.go | 44 ++++++++++++ .../modelsdk/widget_write_signout_test.go | 72 +++++++++++++++++-- mdl/executor/cmd_pages_builder_v3.go | 8 ++- mdl/executor/cmd_pages_describe_output.go | 19 +++++ sdk/mpr/writer_widgets_action.go | 27 +++++++ sdk/mpr/writer_widgets_action_test.go | 61 +++++++++++++--- 8 files changed, 273 insertions(+), 18 deletions(-) create mode 100644 mdl-examples/bug-tests/captrack-10-open-link-action.mdl diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 0eab96e28..9522aea0c 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -140,7 +140,7 @@ func init() { "button style", "primary", "danger", "success", "icon", "linkbutton", "link button", }, - Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nOPEN_LINK parses but is written by NEITHER engine: the modelsdk engine\nrefuses it, and the legacy engine writes Forms$NoAction, so the button\nrenders and does nothing (measured on 11.13). Call a nanoflow that opens\nthe URL instead. SIGN_OUT is written by both.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", + Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nOPEN_LINK takes a static web address and stores it as a\nForms$StaticOrDynamicString. Mendix also supports a DYNAMIC address, read\nfrom an attribute at runtime; MDL cannot author that one, and DESCRIBE\nflags such a button rather than printing its address as a literal.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", Example: "ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\nACTIONBUTTON btnEdit (Caption: 'Edit',\n Action: SHOW_PAGE Module.EditPage(Item: $currentObject))\nLINKBUTTON btnDelete (Caption: 'Delete', Action: DELETE,\n Icon: 'Atlas_Core.Atlas_Filled.pencil')", SeeAlso: []string{"page.widgets"}, }) @@ -375,7 +375,7 @@ func init() { "menu", "menus", "menu document", "menu item", }, Syntax: "CREATE [OR MODIFY] MENU Module.Name [FOLDER 'path'] (\n" + - " MENU ITEM '' [PAGE Module.Page | MICROFLOW Module.Flow] [ICON Module.Collection.name];\n" + + " MENU ITEM '' [PAGE Module.Page | MICROFLOW Module.Flow | SIGN_OUT] [ICON Module.Collection.name];\n" + " MENU '' [ICON Module.Collection.name] ( );\n" + ");\n" + "DESCRIBE MENU Module.Name;\n" + @@ -394,6 +394,9 @@ func init() { "-- SHOW NAVIGATION MENU and ALTER NAVIGATION. Both use these same items.\n" + "-- * OR MODIFY replaces the item list wholesale; an omitted item is removed.\n" + "-- The document's identity and export level are preserved.\n" + + "-- * SIGN_OUT is the log-out menu item. It needs no target and stores the\n" + + "-- same Forms$SignOutClientAction a sign-out BUTTON carries. Works both\n" + + "-- here and in a navigation profile's menu.\n" + "-- * ICON names an icon collection entry. A glyph or image icon cannot be\n" + "-- expressed in MDL; DESCRIBE flags those rather than dropping them silently.\n" + "-- * A page with required parameters cannot be opened from a menu item\n" + diff --git a/mdl-examples/bug-tests/captrack-10-open-link-action.mdl b/mdl-examples/bug-tests/captrack-10-open-link-action.mdl new file mode 100644 index 000000000..3f736ec46 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-10-open-link-action.mdl @@ -0,0 +1,53 @@ +-- The other half of the same defect as SIGN_OUT (CapTrackV2 FINDINGS §10): +-- `ACTIONBUTTON … (Action: OPEN_LINK '…')` was written by NEITHER engine. +-- +-- modelsdk: refused — "client action *pages.LinkClientAction not yet +-- supported by the modelsdk engine" +-- legacy: fell through to the quiet default and wrote Forms$NoAction, so +-- the button rendered and did nothing, with `mxcli check`, `exec` +-- and `mx check` all clean. +-- +-- Two things a reference settles that reasoning would not. The STORAGE NAME is +-- Forms$OpenLinkClientAction, not the "Forms$LinkClientAction" the semantic type +-- carried — a wrong $Type that never reached disk only because nothing could +-- write the action. And the address is not a string field: it is a nested +-- Forms$StaticOrDynamicString. +-- +-- Pinned against 31 Studio Pro-authored link buttons (ako/TestApp, +-- FeedbackModule): exactly five keys, LinkType "Web" in all 31. +-- +-- { "$Type": "Forms$OpenLinkClientAction", +-- "Address": { "$Type": "Forms$StaticOrDynamicString", +-- "AttributeRef": null, "IsDynamic": false, +-- "Value": "https://www.mendix.com/" }, +-- "DisabledDuringExecution": true, +-- "LinkType": "Web" } +-- +-- 6 of those 31 are DYNAMIC (IsDynamic true, an AttributeRef, an empty Value) — +-- the address is read from an attribute at runtime. MDL cannot author that, so +-- DESCRIBE flags such a button instead of printing its address as a literal, +-- which would round-trip into a different link. +-- +-- Verify: +-- mxcli exec captrack-10-open-link-action.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe page OpenLink.P_Links" +-- -- must emit: Action: open_link 'https://example.com' +-- mx check -p app.mpr -- 0 errors + +create module OpenLink; +/ + +create or replace page OpenLink.P_Links + (Title: 'Links', Layout: Atlas_Core.Atlas_Default) +{ + container cLinks { + actionbutton btnDocs (Caption: 'Docs', Action: open_link 'https://example.com') + + -- A link button is the render mode a URL usually wants. + linkbutton lnkHome (Caption: 'Mendix', Action: open_link 'https://www.mendix.com/') + + -- CONTROL: the actions that already worked are untouched. + actionbutton btnClose (Caption: 'Close', Action: close_page) + } +} +/ diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index c7814838b..773e157a6 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -1445,6 +1445,27 @@ func formSettingsToGen(pageName string) element.Element { return ps } +// staticAddressToGen builds the Forms$StaticOrDynamicString an open-link action +// nests as its Address. +// +// Only the STATIC form is authored: MDL spells `OPEN_LINK 'https://…'` and has +// no syntax for the dynamic one. 6 of the 31 Studio Pro references are dynamic +// (IsDynamic true, an AttributeRef, an empty Value) — DESCRIBE flags those +// rather than rendering them as a literal, because a dynamic address printed as +// a static one round-trips into a different link. +// +// AttributeRef is deliberately left unset. gen declares a fourth property here, +// `Attribute`, that not one of the 31 documents carries; writing a key Mendix +// does not store is what makes a document mxbuild accepts and Studio Pro cannot +// open (CLAUDE.md, "Overlay Writes: Never Invent a Key"). +func staticAddressToGen(address string) element.Element { + s := genPg.NewStaticOrDynamicString() + assignID(s) + s.SetIsDynamic(false) + s.SetValue(address) + return s +} + // clientActionToGen converts a widget client action. Simple actions are supported; // the page/microflow/nanoflow/create-object actions (which carry settings sub- // objects) are refused loudly for now. @@ -1483,6 +1504,29 @@ func clientActionToGen(a pages.ClientAction) (element.Element, error) { g.SetNumberOfPagesToClose2("") g.SetPageSettings(formSettingsToGen(x.PageName)) return g, nil + case *pages.LinkClientAction: + // open_link → Forms$OpenLinkClientAction. Note the storage name: the + // semantic type is LinkClientAction and the executor stamped it + // "Forms$LinkClientAction", which is not what Mendix stores — a wrong + // $Type that never reached disk only because neither engine could write + // the action at all. + // + // Pinned against 31 Studio Pro-authored link buttons (ako/TestApp, + // FeedbackModule): exactly five keys, LinkType "Web" in all 31, and the + // address nested as a Forms$StaticOrDynamicString. + g := genPg.NewOpenLinkClientAction() + if x.ID != "" { + g.SetID(element.ID(x.ID)) + } + assignID(g) + g.SetDisabledDuringExecution(true) + linkType := string(x.LinkType) + if linkType == "" { + linkType = "Web" + } + g.SetLinkType(linkType) + g.SetAddress(staticAddressToGen(x.Address)) + return g, nil case *pages.SignOutClientAction: // sign_out → Forms$SignOutClientAction. One property, and the reference // pins its value: a Studio Pro-authored sign-out button (ako/TestApp, diff --git a/mdl/backend/modelsdk/widget_write_signout_test.go b/mdl/backend/modelsdk/widget_write_signout_test.go index 62dcc8cf5..a454018d1 100644 --- a/mdl/backend/modelsdk/widget_write_signout_test.go +++ b/mdl/backend/modelsdk/widget_write_signout_test.go @@ -6,8 +6,8 @@ import ( "strings" "testing" - genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" "github.com/mendixlabs/mxcli/model" + genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" "github.com/mendixlabs/mxcli/sdk/pages" ) @@ -49,21 +49,81 @@ func TestClientActionToGen_SignOut(t *testing.T) { } // CONTROL 1: an action that is still unimplemented must still be REFUSED, not -// quietly written. Without this the test above could pass because the default +// quietly written. Without this the tests above could pass because the default // branch had been softened, which is the exact failure the legacy engine had. +// +// ShowHomePage is the stand-in: gen has no type for it, generated/metamodel has +// no Pages counterpart, and no MDL statement builds one — so it is a semantic +// type nothing can write, which is precisely what this control needs. (The +// earlier draft used LinkClientAction, which stopped being a valid control the +// moment OPEN_LINK was implemented.) func TestClientActionToGen_StillRefusesWhatItCannotWrite(t *testing.T) { - _, err := clientActionToGen(&pages.LinkClientAction{ - BaseElement: model.BaseElement{ID: "link-id"}, - Address: "https://example.com", + _, err := clientActionToGen(&pages.ShowHomePageClientAction{ + BaseElement: model.BaseElement{ID: "home-id"}, }) if err == nil { - t.Fatal("OPEN_LINK was accepted; it has no writer, so accepting it means dropping it") + t.Fatal("an action with no writer was accepted, which means dropping it") } if !strings.Contains(err.Error(), "not yet supported") { t.Errorf("unexpected message: %v", err) } } +// OPEN_LINK, the other action that used to fall through. gen calls it +// OpenLinkClientAction — the storage name differs from the semantic type's +// "Forms$LinkClientAction", a wrong $Type that never reached disk only because +// nothing could write the action at all. +// +// Pinned against 31 Studio Pro-authored link buttons (ako/TestApp, +// FeedbackModule): five keys, LinkType "Web" in all 31, address nested as a +// Forms$StaticOrDynamicString. +func TestClientActionToGen_OpenLink(t *testing.T) { + el, err := clientActionToGen(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + LinkType: pages.LinkTypeWeb, + Address: "https://example.com", + }) + if err != nil { + t.Fatalf("OPEN_LINK is still refused: %v", err) + } + g, ok := el.(*genPg.OpenLinkClientAction) + if !ok { + t.Fatalf("got %T, want *pages.OpenLinkClientAction", el) + } + if g.TypeName() != "Forms$OpenLinkClientAction" { + t.Errorf("$Type = %q — Mendix stores OpenLink, not Link", g.TypeName()) + } + if g.LinkType() != "Web" { + t.Errorf("LinkType = %q, want Web", g.LinkType()) + } + addr, ok := g.Address().(*genPg.StaticOrDynamicString) + if !ok { + t.Fatalf("Address is %T, want *pages.StaticOrDynamicString", g.Address()) + } + if addr.IsDynamic() { + t.Error("IsDynamic is true; MDL authors the static form only") + } + if addr.Value() != "https://example.com" { + t.Errorf("Value = %q, want the authored URL", addr.Value()) + } +} + +// An empty LinkType must not reach storage: Mendix's enum is Call/Email/Text/Web +// and every one of the 31 references is Web, so that is the default rather than +// writing a blank a build would reject. +func TestClientActionToGen_OpenLinkDefaultsLinkType(t *testing.T) { + el, err := clientActionToGen(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + Address: "https://example.com", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if lt := el.(*genPg.OpenLinkClientAction).LinkType(); lt != "Web" { + t.Errorf("LinkType = %q, want Web", lt) + } +} + // CONTROL 2: the actions that already worked are untouched. func TestClientActionToGen_ExistingActionsUnchanged(t *testing.T) { for _, a := range []pages.ClientAction{ diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 555fdee1c..d448c0587 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -1449,8 +1449,12 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc case "openLink": return &pages.LinkClientAction{ BaseElement: model.BaseElement{ - ID: model.ID(types.GenerateID()), - TypeName: "Forms$LinkClientAction", + ID: model.ID(types.GenerateID()), + // Mendix stores this as Forms$OpenLinkClientAction — the + // storage name differs from the SDK type name, the split + // CLAUDE.md documents. The wrong value here never reached disk + // only because neither engine could write the action at all. + TypeName: "Forms$OpenLinkClientAction", }, LinkType: pages.LinkTypeWeb, Address: action.LinkURL, diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index d758689c5..64d1d55d9 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -1170,6 +1170,25 @@ func renderClientActionMDL(ctx *ExecContext, action map[string]any) string { return "complete_task '" + strings.ReplaceAll(outcomeValue, "'", "''") + "'" case "Forms$SignOutClientAction", "Pages$SignOutClientAction": return "sign_out" + case "Forms$OpenLinkClientAction", "Pages$OpenLinkClientAction": + // The address is a nested Forms$StaticOrDynamicString. MDL can spell the + // static form only; a DYNAMIC address (6 of the 31 Studio Pro references + // use one) reads its value from an attribute at runtime, so rendering it + // as a literal would round-trip into a different link. Say so instead. + addr := actionMapForKey(action, "Address") + if addr == nil { + return "open_link ''" + } + if isDynamic, _ := addr["IsDynamic"].(bool); isDynamic { + attr := "" + if ref := actionMapForKey(addr, "AttributeRef"); ref != nil { + attr, _ = ref["Attribute"].(string) + } + return "-- open_link with a dynamic address (" + attr + ") — MDL cannot author this; " + + "the button is left as-is" + } + value, _ := addr["Value"].(string) + return "open_link '" + strings.ReplaceAll(value, "'", "''") + "'" case "Forms$NoClientAction", "Pages$NoClientAction": return "" default: diff --git a/sdk/mpr/writer_widgets_action.go b/sdk/mpr/writer_widgets_action.go index cdb79b6dd..1274b331b 100644 --- a/sdk/mpr/writer_widgets_action.go +++ b/sdk/mpr/writer_widgets_action.go @@ -53,6 +53,33 @@ func serializeClientAction(action pages.ClientAction) bson.D { {Key: "$Type", Value: "Forms$DeleteClientAction"}, {Key: "ClosePage", Value: a.ClosePage}, } + case *pages.LinkClientAction: + // OPEN_LINK fell through to the default below and was written as + // Forms$NoAction, exactly as SIGN_OUT was — the button rendered and did + // nothing (CapTrackV2 FINDINGS §10). + // + // The storage name is Forms$OpenLinkClientAction, NOT the + // "Forms$LinkClientAction" the semantic type carries. Pinned against 31 + // Studio Pro-authored link buttons: five keys, LinkType "Web" in all 31, + // address nested as a Forms$StaticOrDynamicString whose AttributeRef is + // null for the static form MDL authors. + linkType := string(a.LinkType) + if linkType == "" { + linkType = "Web" + } + return bson.D{ + {Key: "$ID", Value: idToBsonBinary(string(a.ID))}, + {Key: "$Type", Value: "Forms$OpenLinkClientAction"}, + {Key: "Address", Value: bson.D{ + {Key: "$ID", Value: idToBsonBinary(generateUUID())}, + {Key: "$Type", Value: "Forms$StaticOrDynamicString"}, + {Key: "AttributeRef", Value: nil}, + {Key: "IsDynamic", Value: false}, + {Key: "Value", Value: a.Address}, + }}, + {Key: "DisabledDuringExecution", Value: true}, + {Key: "LinkType", Value: linkType}, + } case *pages.SignOutClientAction: // Until this case existed, SIGN_OUT fell through to the default below // and was written as Forms$NoAction — so the button rendered, said diff --git a/sdk/mpr/writer_widgets_action_test.go b/sdk/mpr/writer_widgets_action_test.go index d534e2c5f..aaafcc660 100644 --- a/sdk/mpr/writer_widgets_action_test.go +++ b/sdk/mpr/writer_widgets_action_test.go @@ -220,13 +220,17 @@ func TestSignOutClientAction_IsNotSilentlyDroppedToNoAction(t *testing.T) { } } -// CONTROL: the default branch still exists and still yields Forms$NoAction, so -// this test proves something about SIGN_OUT rather than about the fallback being -// removed. OPEN_LINK is the action that still lands there — see FINDINGS §10. +// CONTROL: the quiet default still exists and still yields Forms$NoAction, so +// the tests here prove something about the actions they name rather than about +// the fallback having been removed. +// +// ShowHomePage is the stand-in: no MDL statement builds one, so it is a +// semantic type nothing writes. (An earlier draft used LinkClientAction, which +// stopped being a valid control the moment OPEN_LINK was implemented — a +// control has to name something still genuinely unhandled.) func TestUnhandledClientActionStillFallsBackToNoAction(t *testing.T) { - doc := serializeClientAction(&pages.LinkClientAction{ - BaseElement: model.BaseElement{ID: "link-id"}, - Address: "https://example.com", + doc := serializeClientAction(&pages.ShowHomePageClientAction{ + BaseElement: model.BaseElement{ID: "home-id"}, }) var typeName string for _, e := range doc { @@ -235,7 +239,48 @@ func TestUnhandledClientActionStillFallsBackToNoAction(t *testing.T) { } } if typeName != "Forms$NoAction" { - t.Errorf("$Type = %q; this control pins the fallback SIGN_OUT used to hit, "+ - "so that the test above cannot pass for the wrong reason", typeName) + t.Errorf("$Type = %q; this control pins the fallback SIGN_OUT and OPEN_LINK used to hit, "+ + "so the tests above cannot pass for the wrong reason", typeName) + } +} + +// OPEN_LINK on the legacy engine, which fell to that same NoAction default. +// Pinned against the 31 Studio Pro references: five keys, and the address a +// nested Forms$StaticOrDynamicString whose AttributeRef is null for the static +// form MDL authors. +func TestOpenLinkClientAction_IsNotSilentlyDroppedToNoAction(t *testing.T) { + doc := serializeClientAction(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + LinkType: pages.LinkTypeWeb, + Address: "https://example.com", + }) + got := map[string]any{} + for _, e := range doc { + got[e.Key] = e.Value + } + if got["$Type"] != "Forms$OpenLinkClientAction" { + t.Fatalf("$Type = %v, want Forms$OpenLinkClientAction (NOT Forms$LinkClientAction, "+ + "which is the SDK name and not what Mendix stores)", got["$Type"]) + } + if got["LinkType"] != "Web" { + t.Errorf("LinkType = %v, want Web", got["LinkType"]) + } + if len(doc) != 5 { + t.Errorf("the action has %d keys, want 5: %v", len(doc), doc) + } + addr, ok := got["Address"].(bson.D) + if !ok { + t.Fatalf("Address is %T, want a nested document", got["Address"]) + } + a := map[string]any{} + for _, e := range addr { + a[e.Key] = e.Value + } + if a["$Type"] != "Forms$StaticOrDynamicString" || a["IsDynamic"] != false || + a["Value"] != "https://example.com" { + t.Errorf("Address = %v", addr) + } + if _, present := a["AttributeRef"]; !present { + t.Error("AttributeRef is absent; all 31 references carry it as null") } } From d4d562712637c47afdebb0e0fc68551d000d262e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 08:49:39 +0000 Subject: [PATCH 10/17] feat(navigation): author and read a sign-out MENU ITEM MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A navigation menu can carry a log-out item. mxcli could neither author one nor read one back: authoring MDL's `menu item` took PAGE or MICROFLOW only, so there was no spelling for it at all. reading ako/TestApp's sign-out menu item came back as a plain `menu item 'Item 5';`, so DESCRIBE -> exec turned a working log-out entry into a dead one — silently, with mx check clean. Fixing the sign-out BUTTON proved nothing about this, because a menu item's action reaches storage through four switches that share no code with the button path — two writers (a standalone menu document and the menu inside a navigation profile) and two readers. All four had to be wired: menuActionToGen NoAction default modelsdk, menu document navMenuAction NoAction default raw BSON, navigation profile resolveMenuAction raw type name modelsdk read parseNavMenuItem raw type name legacy read The readers are the subtler half. Both had a fallback that stored the unmapped $Type, which LOOKS like it preserves information — ActionType became "Forms$SignOutClientAction" — while breaking the round trip, because DESCRIBE and both writers key on "SignOutAction". A round trip closes only when the reader produces the exact string the writer consumes. Studio Pro stores the same element a button carries — Forms$SignOutClientAction, DisabledDuringExecution true, nothing else — which is why SIGN_OUT sits beside PAGE and MICROFLOW rather than getting a syntax of its own. It names no target, so the visitor reads it separately from the PAGE/MICROFLOW switch; folding it in would consume a qualifiedName and mis-assign a trailing ICON. That case is in the example. Measured on ako/TestApp (Mendix 11.14), and controlled by neutralising both readers and re-reading it: before show navigation menu Responsive -> Item 5 after show navigation menu Responsive -> Item 5 -> sign out describe -> exec now puts an identical Forms$SignOutClientAction back on disk, and mx check reports 0 errors on the result. Reported as CapTrackV2 FINDINGS §10. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 2 + .../captrack-10-sign-out-menu-item.mdl | 54 ++++++++ mdl/ast/ast_navigation.go | 1 + mdl/backend/modelsdk/menu_signout_test.go | 94 +++++++++++++ mdl/backend/modelsdk/menu_write.go | 10 ++ mdl/backend/modelsdk/navigation_read.go | 6 + mdl/backend/modelsdk/navigation_write.go | 9 ++ mdl/executor/cmd_menus.go | 5 + mdl/executor/cmd_navigation.go | 6 + mdl/executor/menu_signout_test.go | 127 ++++++++++++++++++ mdl/grammar/MDLParser.g4 | 6 +- mdl/types/navigation.go | 4 + mdl/visitor/visitor_navigation.go | 5 + sdk/mpr/parser_menu_signout_test.go | 56 ++++++++ sdk/mpr/parser_misc.go | 5 + 15 files changed, 389 insertions(+), 1 deletion(-) create mode 100644 mdl-examples/bug-tests/captrack-10-sign-out-menu-item.mdl create mode 100644 mdl/backend/modelsdk/menu_signout_test.go create mode 100644 mdl/executor/menu_signout_test.go create mode 100644 sdk/mpr/parser_menu_signout_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 47f8b44fc..6c5d2ad1f 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -511,3 +511,5 @@ {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`dataview dv (\u2026, OnClick: SHOW_PAGE \u2026)` parses, `mxcli check` is clean, `exec` writes the page without a word \u2014 and the rendered element has no handler and no role=\"button\". The same silence on `dynamictext`, `listview` and every other widget except a container or a button.", "cause": "`OnClick:` is an ALIAS for `Action:` (both stored as Properties[\"Action\"], #603), and mxcli writes that property for three widget kinds only: container/customcontainer, the buttons, and a navigationlist item. Every other widget drops it, and the property allow-lists behind MDL-WIDGET01/07 could not see that because they are widget-type AGNOSTIC \u2014 the same blind spot as #928's `editable:`. Added MDL-WIDGET23 (warning), with two messages: Mendix models no click action at all (dataview, dynamictext, inputs, groupbox\u2026) vs Mendix models one that mxcli cannot write (listview, staticimage, dynamicimage \u2014 measured against generated/metamodel).", "file": "`mdl/executor/validate_widget_onclick.go` (new), wired in `validate_widgets.go` beside validateWidgetEditability; `.claude/skills/mendix/create-page/reference/widgets.md`; example `mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl`", "insight": "The first draft reported everything OUTSIDE an allow-list of the three writers, and running it over the shipped examples flagged three of them. Cause: `mxcli check` without `-p` has no widget registry, so `lookupWidgetDef` returns nil for a PLUGGABLE widget too and the caller's \"static widgets only\" branch silently does not hold \u2014 `datagrid` is DataGrid 2, a pluggable widget whose onClick the engine does write. For any rule keyed on widget type, an allow-list makes the unknown case an ERROR and a deny-list makes it silence; pick the deny-list, because a missed warning costs nothing and a false one tells an author their working page is broken. Running a new rule across mdl-examples/ before wiring it up is what caught it \u2014 the same exercise #893 describes. Reported as CapTrackV2 FINDINGS \u00a721."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ALTER SETTINGS MODEL AfterStartupMicroflow = 'Mod.MF_Seed'` accepts a microflow with no return type, `mxcli check` passes, and the build fails **CE0142** \"After startup microflow should return a boolean\". `ALTER MICROFLOW \u2026 RETURNS \u2026` does not parse either, so the remedy is DROP + CREATE.", "cause": "#274 made ALTER SETTINGS resolve the qualified names it writes, which catches a MISSPELLED microflow. Here the name resolves perfectly \u2014 the constraint is on the thing the setting names, not on the reference, and nothing looked at the return type. Added MDL073: a project-less pass (ValidateAfterStartupReturnType) for a microflow the script itself creates, which is the usual shape, plus the stored return type on the project path. Both call one function so they cannot drift. flowSignature gained ReturnKind because its existing Returns field is the entity name and cannot tell Boolean from void.", "file": "`mdl/executor/validate_settings_refs.go` (checkAfterStartupReturnsBoolean + ValidateAfterStartupReturnType), `validate_program.go`, `helpers.go` (buildMicroflowReturnTypes), `validate_datasource_args.go` (flowSignature.ReturnKind); examples `mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean{,.fail}.mdl`", "insight": "\"The reference resolves\" and \"the reference is usable\" are different questions, and a resolver answers only the first. Whenever a setting stores a NAME, ask what the platform requires of the named thing \u2014 an existence check will pass and the build will still fail, and the error arrives with no connection to the statement that caused it. The scope discipline that goes with it: only AfterStartup is type-checked, because only its rule was measured; BeforeShutdown and HealthCheck are left alone rather than constrained on a guess, and there is a control test asserting that. Reported as CapTrackV2 FINDINGS \u00a76."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ACTIONBUTTON \u2026 (Action: SIGN_OUT)` is refused by the default engine \u2014 \"client action *pages.SignOutClientAction not yet supported by the modelsdk engine \u2014 rerun with MXCLI_ENGINE=legacy\" \u2014 and the suggested workaround SILENTLY produces a dead button: on legacy the action is written as `Forms$NoAction`, so it renders, says \"Sign out\", and does nothing, with `mxcli check`, `exec` and `mx check` all clean.", "cause": "Neither engine had a case for the action. modelsdk's clientActionToGen ended in a loud default; sdk/mpr's serializeClientAction ended in a QUIET one that returns Forms$NoAction for anything unmatched. Added the case to both. The document is two keys \u2014 `Forms$SignOutClientAction` + `DisabledDuringExecution: true` \u2014 pinned against a Studio Pro-authored button in ako/TestApp, plus `sign_out` in the DESCRIBE renderer so it round-trips. Both engines now emit byte-identical documents; mx check 0 errors on each. OPEN_LINK is still unwritten by both (gen calls it OpenLinkClientAction and its Address is an element, not a string) \u2014 the syntax topic now says so instead of listing it as available.", "file": "`mdl/backend/modelsdk/widget_write.go` (clientActionToGen), `sdk/mpr/writer_widgets_action.go` (serializeClientAction), `mdl/executor/cmd_pages_describe_output.go` (renderClientActionMDL), `cmd/mxcli/syntax/features_page.go`; example `mdl-examples/bug-tests/captrack-10-sign-out-action.mdl`", "insight": "When one engine refuses something and points at the other, CHECK THE OTHER before repeating the advice \u2014 the refusal is visible and the fallback is not, so the recommended escape hatch can be the strictly worse path. The structural tell is the shape of the default branch: modelsdk's raises, legacy's returns Forms$NoAction, and a silent default in a serializer converts every unimplemented type into data loss rather than an error. Grep for the fallthrough before trusting a switch. Note the control this needs: a test that SIGN_OUT is no longer NoAction can pass because someone softened the default, so pin the fallback separately with a type that is still unimplemented (OPEN_LINK). Reported as CapTrackV2 FINDINGS \u00a710."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "A navigation menu's LOG-OUT item could not be authored and did not survive a round trip. MDL's `menu item` took PAGE or MICROFLOW only, so there was no spelling for it; and ako/TestApp's sign-out menu item read back as a plain `menu item 'Item 5';`, so DESCRIBE -> exec turned a working log-out entry into a dead one \u2014 silently, with `mx check` clean.", "cause": "A menu item's action goes through FOUR places that share no code with the button path: menuActionToGen (menu document, modelsdk), navMenuAction (navigation profile, raw BSON), resolveMenuAction (modelsdk read) and parseNavMenuItem (legacy read). Both writers ended in a NoAction default and both readers left the type name unmapped. Added SIGN_OUT to navMenuItemDef in the grammar (it consumes no qualifiedName, so it is read separately from the PAGE/MICROFLOW switch or an ICON after it is mis-assigned), carried it as ActionType \"SignOutAction\" / NavMenuItemSpec.SignOut, and wired all four. Studio Pro stores the same Forms$SignOutClientAction a button carries: DisabledDuringExecution true, nothing else.", "file": "`mdl/grammar/MDLParser.g4` (navMenuItemDef), `mdl/ast/ast_navigation.go`, `mdl/visitor/visitor_navigation.go`, `mdl/executor/cmd_menus.go` + `cmd_navigation.go` (conversion + printMenuMDL + the show summary), `mdl/types/navigation.go`, `mdl/backend/modelsdk/menu_write.go` + `navigation_write.go` + `navigation_read.go`, `sdk/mpr/parser_misc.go`; example `mdl-examples/bug-tests/captrack-10-sign-out-menu-item.mdl`", "insight": "A round trip closes only if the READER produces the exact string the WRITER consumes \u2014 here both readers had a raw-type-name fallback that looked like it preserved information (ActionType became \"Forms$SignOutClientAction\") while breaking the round trip, because DESCRIBE and the writers key on \"SignOutAction\". A fallback that stores the raw name is not the same as handling the case, and it hides the gap better than a NoAction default would. Also: the same logical action reaches storage through four unrelated switches (two writers x two constructs, two readers), so fixing the button path proved nothing about the menu path \u2014 grep for every switch on the action before calling such a fix complete. Controlled by neutralising both readers and re-reading TestApp: `Item 5 -> sign out` goes back to `Item 5`."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ACTIONBUTTON \u2026 (Action: OPEN_LINK 'https://\u2026')` was written by neither engine: modelsdk refused it, legacy fell through to its quiet default and wrote Forms$NoAction, so the button rendered and did nothing with check, exec and mx check all clean.", "cause": "Same missing-case defect as SIGN_OUT, but with two traps a reference settled and reasoning would not. (1) The STORAGE NAME is Forms$OpenLinkClientAction, while the semantic type is LinkClientAction and the executor stamped `Forms$LinkClientAction` \u2014 a wrong $Type that never reached disk only because nothing could write the action. (2) The address is not a string field but a nested Forms$StaticOrDynamicString. Pinned against 31 Studio Pro link buttons (ako/TestApp, FeedbackModule): exactly five keys, LinkType \"Web\" in all 31, and 6 of 31 DYNAMIC (IsDynamic true + AttributeRef + empty Value). MDL authors the static form only, so DESCRIBE flags a dynamic one instead of printing its address as a literal.", "file": "`mdl/backend/modelsdk/widget_write.go` (clientActionToGen + staticAddressToGen), `sdk/mpr/writer_widgets_action.go`, `mdl/executor/cmd_pages_builder_v3.go` ($Type), `mdl/executor/cmd_pages_describe_output.go`, `cmd/mxcli/syntax/features_page.go`; example `mdl-examples/bug-tests/captrack-10-open-link-action.mdl`", "insight": "gen declares a fourth property on Forms$StaticOrDynamicString \u2014 `Attribute` \u2014 that not one of the 31 stored documents carries. Writing it would be the 'never invent a key' failure: a document mxbuild accepts and Studio Pro cannot open. When gen offers more properties than the references show, the references win. Second lesson, about controls: the SIGN_OUT commit used LinkClientAction as its 'still unimplemented' control, and implementing OPEN_LINK silently invalidated it \u2014 the test then failed for a good reason, but a control naming a specific unimplemented feature has a shelf life. Point it at something structurally unwritable instead (ShowHomePageClientAction: no gen type, no metamodel counterpart, no MDL statement that builds one)."} diff --git a/mdl-examples/bug-tests/captrack-10-sign-out-menu-item.mdl b/mdl-examples/bug-tests/captrack-10-sign-out-menu-item.mdl new file mode 100644 index 000000000..5a12bbb90 --- /dev/null +++ b/mdl-examples/bug-tests/captrack-10-sign-out-menu-item.mdl @@ -0,0 +1,54 @@ +-- A navigation menu can carry a LOG-OUT item, and mxcli could neither author +-- one nor read one back. This is the menu half of CapTrackV2 FINDINGS §10; the +-- button half is captrack-10-sign-out-action.mdl. +-- +-- MDL's menu item took PAGE or MICROFLOW only, so: +-- +-- authoring: there was no spelling for it at all. +-- reading: ako/TestApp's sign-out menu item came back as a plain +-- `menu item 'Item 5';` — so DESCRIBE -> exec turned a working +-- log-out entry into a dead one, silently, `mx check` clean. +-- +-- Measured on ako/TestApp (Mendix 11.14), before and after: +-- +-- before: show navigation menu Responsive -> Item 5 +-- after: show navigation menu Responsive -> Item 5 -> sign out +-- +-- Studio Pro stores it as the same element a sign-out BUTTON carries — +-- Forms$SignOutClientAction, DisabledDuringExecution true, nothing else — which +-- is why SIGN_OUT sits beside PAGE and MICROFLOW rather than having a syntax of +-- its own. It needs no target. +-- +-- Both menu constructs are covered: a standalone menu document and the menu +-- inside a navigation profile. They have separate writers and separate readers, +-- and BOTH ended in a NoAction default. +-- +-- Verify: +-- mxcli exec captrack-10-sign-out-menu-item.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe menu SignOutMenu.Account_Menu" +-- -- must emit: menu item 'Sign out' sign_out; +-- mx check -p app.mpr -- 0 errors + +create module SignOutMenu; +/ +create or replace page SignOutMenu.Home (Title: 'Home', Layout: Atlas_Core.Atlas_Default) +{ + dynamictext t (Content: 'home') +} +/ + +create or modify menu SignOutMenu.Account_Menu ( + menu item 'Home' page SignOutMenu.Home; + + -- The item that could not be spelled. + menu item 'Sign out' sign_out; + + -- SIGN_OUT names no target, so it must not disturb the qualified-name list it + -- shares with PAGE/MICROFLOW and ICON — an icon after it still lands as one. + menu item 'Log out' sign_out icon Atlas_Core.Atlas.home; + + -- CONTROL: an item with no action must stay actionless. A conversion that + -- stamped every actionless item as sign-out would look correct above. + menu item 'Plain'; +); +/ diff --git a/mdl/ast/ast_navigation.go b/mdl/ast/ast_navigation.go index 8dd26a300..af81e8f31 100644 --- a/mdl/ast/ast_navigation.go +++ b/mdl/ast/ast_navigation.go @@ -28,6 +28,7 @@ type NavMenuItemDef struct { Caption string // from STRING_LITERAL Page *QualifiedName // PAGE target Microflow *QualifiedName // MICROFLOW target + SignOut bool // SIGN_OUT — the third action a menu item can carry Icon string // ICON 'Module.Collection.name', empty for none Items []NavMenuItemDef // Sub-items (for MENU 'caption' (...)) } diff --git a/mdl/backend/modelsdk/menu_signout_test.go b/mdl/backend/modelsdk/menu_signout_test.go new file mode 100644 index 000000000..74b2eba42 --- /dev/null +++ b/mdl/backend/modelsdk/menu_signout_test.go @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/types" + genPages "github.com/mendixlabs/mxcli/modelsdk/gen/pages" +) + +// A sign-out MENU ITEM reaches storage through two writers that do not share +// code with the button path: menuActionToGen for a standalone menu document, +// and navMenuAction for the menu inside a navigation profile. Both ended in a +// NoAction default, so both silently produced a dead entry. +// +// Studio Pro stores the same element a button carries — measured on +// ako/TestApp's sign-out menu item: Forms$SignOutClientAction with +// DisabledDuringExecution true and nothing else. + +func TestMenuActionToGen_SignOut(t *testing.T) { + el := menuActionToGen(&types.NavMenuItem{Caption: "Sign out", ActionType: "SignOutAction"}) + g, ok := el.(*genPages.SignOutClientAction) + if !ok { + t.Fatalf("got %T, want *pages.SignOutClientAction — a menu item's sign-out fell to NoAction", el) + } + if !g.DisabledDuringExecution() { + t.Error("DisabledDuringExecution is false; the Studio Pro reference stores true") + } +} + +// CONTROL: the two actions that already worked, plus the actionless item, are +// unchanged. A writer that answered SignOut for everything would pass the test +// above. +func TestMenuActionToGen_OtherItemsUnchanged(t *testing.T) { + cases := []struct { + name string + item *types.NavMenuItem + want string + }{ + {"page", &types.NavMenuItem{Page: "M.P"}, "Forms$FormAction"}, + {"microflow", &types.NavMenuItem{Microflow: "M.MF"}, "Forms$MicroflowAction"}, + {"plain", &types.NavMenuItem{Caption: "Plain"}, "Forms$NoAction"}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := menuActionToGen(c.item).TypeName(); got != c.want { + t.Errorf("$Type = %q, want %q", got, c.want) + } + }) + } +} + +// The navigation-profile writer is the other half, and it builds raw BSON. +func TestNavMenuAction_SignOut(t *testing.T) { + doc := navMenuAction(types.NavMenuItemSpec{Caption: "Sign out", SignOut: true}) + got := map[string]any{} + for _, e := range doc { + got[e.Key] = e.Value + } + if got["$Type"] != "Forms$SignOutClientAction" { + t.Fatalf("$Type = %v, want Forms$SignOutClientAction", got["$Type"]) + } + if got["DisabledDuringExecution"] != true { + t.Errorf("DisabledDuringExecution = %v, want true", got["DisabledDuringExecution"]) + } + if len(doc) != 3 { + t.Errorf("the action has %d keys, want 3 ($ID, $Type, DisabledDuringExecution): %v", len(doc), doc) + } +} + +// CONTROL: an item with no action still writes Forms$NoAction, so the test +// above proves something about SIGN_OUT rather than about the default going +// away. +func TestNavMenuAction_PlainItemStillNoAction(t *testing.T) { + doc := navMenuAction(types.NavMenuItemSpec{Caption: "Plain"}) + for _, e := range doc { + if e.Key == "$Type" && e.Value != "Forms$NoAction" { + t.Errorf("$Type = %v, want Forms$NoAction", e.Value) + } + } +} + +// The reader has to produce the name the writers consume, or the round trip +// does not close. That pairing is the actual fix: reading TestApp's item as a +// raw type name would have described it as a plain item. +func TestResolveMenuAction_SignOutUsesTheWriterSName(t *testing.T) { + item := &types.NavMenuItem{} + resolveMenuAction(item, genPages.NewSignOutClientAction()) + if item.ActionType != "SignOutAction" { + t.Errorf("ActionType = %q, want SignOutAction — the writers key on that exact string", + item.ActionType) + } +} diff --git a/mdl/backend/modelsdk/menu_write.go b/mdl/backend/modelsdk/menu_write.go index 80b5a9d5e..0f6e4869e 100644 --- a/mdl/backend/modelsdk/menu_write.go +++ b/mdl/backend/modelsdk/menu_write.go @@ -156,6 +156,16 @@ func menuActionToGen(item *types.NavMenuItem) element.Element { ms.SetMicroflowQualifiedName(item.Microflow) a.SetMicroflowSettings(ms) return a + case item.ActionType == "SignOutAction": + // A menu item's sign-out is the same Forms$SignOutClientAction a button + // carries — measured on ako/TestApp's own sign-out menu item, which + // stores exactly DisabledDuringExecution: true and nothing else. Before + // this case it fell to NoAction below, so describe -> exec turned a + // working sign-out item into a dead one. + a := genPages.NewSignOutClientAction() + a.SetID(element.ID(mmpr.GenerateID())) + a.SetDisabledDuringExecution(true) + return a default: a := genPages.NewNoClientAction() a.SetID(element.ID(mmpr.GenerateID())) diff --git a/mdl/backend/modelsdk/navigation_read.go b/mdl/backend/modelsdk/navigation_read.go index 916978a6e..7b98df53d 100644 --- a/mdl/backend/modelsdk/navigation_read.go +++ b/mdl/backend/modelsdk/navigation_read.go @@ -296,6 +296,12 @@ func resolveMenuAction(item *types.NavMenuItem, action element.Element) { if ms, ok := a.MicroflowSettings().(*genPages.MicroflowSettings); ok && ms != nil { item.Microflow = ms.MicroflowQualifiedName() } + case *genPages.SignOutClientAction: + // Named explicitly rather than falling to the raw-type-name default + // below: DESCRIBE and the writer both key on "SignOutAction", and a + // round trip only closes if the reader produces the name the writer + // consumes. + item.ActionType = "SignOutAction" default: t := action.TypeName() switch { diff --git a/mdl/backend/modelsdk/navigation_write.go b/mdl/backend/modelsdk/navigation_write.go index c200d9754..1fd184d02 100644 --- a/mdl/backend/modelsdk/navigation_write.go +++ b/mdl/backend/modelsdk/navigation_write.go @@ -341,6 +341,15 @@ func navMenuAction(mi types.NavMenuItemSpec) bson.D { }}, } } + if mi.SignOut { + // Same element a sign-out BUTTON carries, pinned against the sign-out + // menu item in ako/TestApp: two properties and nothing else. + return bson.D{ + {Key: "$ID", Value: navID()}, + {Key: "$Type", Value: "Forms$SignOutClientAction"}, + {Key: "DisabledDuringExecution", Value: true}, + } + } return bson.D{ {Key: "$ID", Value: navID()}, {Key: "$Type", Value: "Forms$NoAction"}, diff --git a/mdl/executor/cmd_menus.go b/mdl/executor/cmd_menus.go index 21fc9dcb0..d985ab06c 100644 --- a/mdl/executor/cmd_menus.go +++ b/mdl/executor/cmd_menus.go @@ -148,6 +148,11 @@ func menuItemsFromAST(defs []ast.NavMenuItemDef) []*types.NavMenuItem { } else if d.Microflow != nil { item.Microflow = d.Microflow.String() item.ActionType = "MicroflowAction" + } else if d.SignOut { + // Studio Pro stores a sign-out menu item as the same + // Forms$SignOutClientAction a button carries (measured on + // ako/TestApp), so it is an ActionType rather than a target. + item.ActionType = "SignOutAction" } else { item.ActionType = "NoAction" } diff --git a/mdl/executor/cmd_navigation.go b/mdl/executor/cmd_navigation.go index 9d9a32f35..5c6a36c20 100644 --- a/mdl/executor/cmd_navigation.go +++ b/mdl/executor/cmd_navigation.go @@ -126,6 +126,7 @@ func convertMenuItemDef(def ast.NavMenuItemDef) types.NavMenuItemSpec { if def.Microflow != nil { spec.Microflow = def.Microflow.String() } + spec.SignOut = def.SignOut for _, sub := range def.Items { spec.Items = append(spec.Items, convertMenuItemDef(sub)) } @@ -383,6 +384,9 @@ func menuItemTarget(item *types.NavMenuItem) string { if item.Microflow != "" { return " -> MF:" + item.Microflow } + if item.ActionType == "SignOutAction" { + return " -> sign out" + } return "" } @@ -402,6 +406,8 @@ func printMenuMDL(w io.Writer, items []*types.NavMenuItem, depth int, reproducer fmt.Fprintf(w, "%smenu item '%s' page %s%s;\n", indent, item.Caption, item.Page, icon) } else if item.Microflow != "" { fmt.Fprintf(w, "%smenu item '%s' microflow %s%s;\n", indent, item.Caption, item.Microflow, icon) + } else if item.ActionType == "SignOutAction" { + fmt.Fprintf(w, "%smenu item '%s' sign_out%s;\n", indent, item.Caption, icon) } else { fmt.Fprintf(w, "%smenu item '%s'%s;\n", indent, item.Caption, icon) } diff --git a/mdl/executor/menu_signout_test.go b/mdl/executor/menu_signout_test.go new file mode 100644 index 000000000..06fc80c94 --- /dev/null +++ b/mdl/executor/menu_signout_test.go @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// A navigation menu can carry a log-out item, and mxcli could neither author one +// nor read one back. MDL's menu item took PAGE or MICROFLOW only, so: +// +// - authoring: there was no spelling for it at all; +// - reading: ako/TestApp's sign-out menu item ("Item 5") came back as a +// plain `menu item 'Item 5';`, so DESCRIBE -> exec turned a working +// sign-out into a dead menu entry — silently, with `mx check` clean. +// +// Studio Pro stores it as the same Forms$SignOutClientAction a BUTTON carries +// (measured on that item): DisabledDuringExecution true, and nothing else. + +func signOutMenuStmt(t *testing.T, src string) *ast.CreateMenuStmt { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors for %q: %v", src, errs) + } + stmt, ok := prog.Statements[0].(*ast.CreateMenuStmt) + if !ok { + t.Fatalf("got %T, want *ast.CreateMenuStmt", prog.Statements[0]) + } + return stmt +} + +// The spelling that did not exist. +func TestMenuItem_SignOutParses(t *testing.T) { + stmt := signOutMenuStmt(t, `create or modify menu M.Main ( + menu item 'Sign out' sign_out; +);`) + if len(stmt.Items) != 1 { + t.Fatalf("got %d items, want 1", len(stmt.Items)) + } + if !stmt.Items[0].SignOut { + t.Error("SIGN_OUT did not reach the AST") + } +} + +// SIGN_OUT names no target, so it must not disturb the qualifiedName list that +// PAGE/MICROFLOW and ICON share — an icon after it still has to land as an icon. +func TestMenuItem_SignOutWithAnIcon(t *testing.T) { + stmt := signOutMenuStmt(t, `create or modify menu M.Main ( + menu item 'Sign out' sign_out icon Atlas_Core.Atlas.home; +);`) + item := stmt.Items[0] + if !item.SignOut { + t.Error("SignOut lost when an icon follows") + } + if item.Icon != "Atlas_Core.Atlas.home" { + t.Errorf("Icon = %q, want the collection entry — SIGN_OUT must consume no qualifiedName", item.Icon) + } +} + +// The menu-document path: AST -> semantic model. +func TestMenuItemsFromAST_SignOutBecomesAnActionType(t *testing.T) { + items := menuItemsFromAST([]ast.NavMenuItemDef{ + {Caption: "Sign out", SignOut: true}, + {Caption: "Plain"}, + }) + if items[0].ActionType != "SignOutAction" { + t.Errorf("ActionType = %q, want SignOutAction", items[0].ActionType) + } + // CONTROL: an item with no action is still NoAction. A conversion that + // stamped every actionless item as sign-out would satisfy the line above. + if items[1].ActionType != "NoAction" { + t.Errorf("a plain item became %q", items[1].ActionType) + } +} + +// The navigation-profile path uses a different spec type and its own converter. +func TestConvertMenuItemDef_CarriesSignOut(t *testing.T) { + spec := convertMenuItemDef(ast.NavMenuItemDef{Caption: "Sign out", SignOut: true}) + if !spec.SignOut { + t.Error("SignOut did not reach NavMenuItemSpec") + } + // CONTROL: the two existing targets are untouched. + page := ast.QualifiedName{Module: "M", Name: "P"} + if got := convertMenuItemDef(ast.NavMenuItemDef{Caption: "Home", Page: &page}); got.Page != "M.P" || got.SignOut { + t.Errorf("a page item came back as %+v", got) + } +} + +// DESCRIBE must emit the spelling exec accepts, or the round trip does not +// close — which is the half that lost TestApp's item. +func TestPrintMenuMDL_RendersSignOut(t *testing.T) { + var b bytes.Buffer + printMenuMDL(&b, []*types.NavMenuItem{ + {Caption: "Sign out", ActionType: "SignOutAction"}, + {Caption: "Plain", ActionType: "NoAction"}, + }, 0, "CREATE NAVIGATION") + + out := b.String() + if !strings.Contains(out, "menu item 'Sign out' sign_out;") { + t.Errorf("describe output does not round-trip the sign-out item:\n%s", out) + } + // CONTROL: a plain item must not gain an action. + if strings.Contains(out, "'Plain' sign_out") { + t.Errorf("a plain item was rendered as sign-out:\n%s", out) + } +} + +// The whole point, end to end in one test: describe output must parse back to +// the same thing. A renderer and a parser can each be individually right and +// still not agree. +func TestMenuItem_SignOutRoundTripsThroughDescribe(t *testing.T) { + var b bytes.Buffer + printMenuMDL(&b, []*types.NavMenuItem{{Caption: "Sign out", ActionType: "SignOutAction"}}, + 0, "CREATE NAVIGATION") + + stmt := signOutMenuStmt(t, "create or modify menu M.Main (\n"+b.String()+");") + if !stmt.Items[0].SignOut { + t.Errorf("describe emitted %q, which does not parse back as a sign-out item", b.String()) + } +} diff --git a/mdl/grammar/MDLParser.g4 b/mdl/grammar/MDLParser.g4 index b6d1bb06c..272508707 100644 --- a/mdl/grammar/MDLParser.g4 +++ b/mdl/grammar/MDLParser.g4 @@ -337,8 +337,12 @@ navigationClause // those segments are double-quoted the same way a keyword-colliding name is: // ICON Atlas_Core.Atlas.home // ICON Atlas_Core.Atlas."align-center" +// SIGN_OUT is the third action a menu item can carry. Studio Pro writes it as +// the same Forms$SignOutClientAction a button uses (measured on ako/TestApp), +// which is why it sits beside PAGE and MICROFLOW rather than in a syntax of its +// own. navMenuItemDef - : MENU_KW ITEM STRING_LITERAL ((PAGE qualifiedName) | (MICROFLOW qualifiedName))? (ICON qualifiedName)? SEMICOLON? + : MENU_KW ITEM STRING_LITERAL ((PAGE qualifiedName) | (MICROFLOW qualifiedName) | SIGN_OUT)? (ICON qualifiedName)? SEMICOLON? | MENU_KW STRING_LITERAL (ICON qualifiedName)? LPAREN navMenuItemDef* RPAREN SEMICOLON? ; diff --git a/mdl/types/navigation.go b/mdl/types/navigation.go index 8fd704117..4b55c2d27 100644 --- a/mdl/types/navigation.go +++ b/mdl/types/navigation.go @@ -107,6 +107,10 @@ type NavMenuItemSpec struct { Caption string Page string Microflow string + // SignOut is the third action a menu item can carry. Studio Pro stores it + // as the same Forms$SignOutClientAction a button uses, so it needs no + // target — which is why it is a flag rather than another name field. + SignOut bool // Icon is a qualified icon-collection name (Atlas_Core.Atlas.home). Empty // means no icon, which serializes as a null Icon. Icon string diff --git a/mdl/visitor/visitor_navigation.go b/mdl/visitor/visitor_navigation.go index 10de013c7..460a529ed 100644 --- a/mdl/visitor/visitor_navigation.go +++ b/mdl/visitor/visitor_navigation.go @@ -108,6 +108,11 @@ func buildNavMenuItemDef(ctx parser.INavMenuItemDefContext) ast.NavMenuItemDef { item.Microflow = &built next++ } + // SIGN_OUT names no target, so it consumes none of the qualifiedName list — + // which is why it is read separately rather than as a third switch arm. + if c.SIGN_OUT() != nil { + item.SignOut = true + } if c.ICON() != nil && len(names) > next { item.Icon = buildQualifiedName(names[next]).String() } diff --git a/sdk/mpr/parser_menu_signout_test.go b/sdk/mpr/parser_menu_signout_test.go new file mode 100644 index 000000000..ed85a207d --- /dev/null +++ b/sdk/mpr/parser_menu_signout_test.go @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import "testing" + +// menuItemRaw builds the minimum a Menus$MenuItem needs to parse: a caption with +// a real translation (parseNavMenuItem deliberately returns nil for an item with +// no caption, no page and no children) plus the action under test. +func menuItemRaw(actionType string) map[string]any { + return map[string]any{ + "Caption": map[string]any{ + "$Type": "Texts$Text", + "Items": []any{ + int32(3), + map[string]any{"$Type": "Texts$Translation", "LanguageCode": "en_US", "Text": "Sign out"}, + }, + }, + "Action": map[string]any{"$Type": actionType}, + } +} + +// The legacy reader is the other half of reading a sign-out MENU ITEM back. +// Before this case it fell to the raw-type-name default, so the item was +// described as a plain `menu item 'x';` and DESCRIBE -> exec turned ako/TestApp's +// working sign-out entry into a dead one — silently, with mx check clean. +func TestParseNavMenuItem_SignOut(t *testing.T) { + mi := parseNavMenuItem(menuItemRaw("Forms$SignOutClientAction")) + if mi == nil { + t.Fatal("parseNavMenuItem returned nil") + } + if mi.ActionType != "SignOutAction" { + t.Errorf("ActionType = %q, want SignOutAction — the writers and DESCRIBE key on that string", + mi.ActionType) + } +} + +// CONTROL: the action types already read must be unchanged, and an unknown one +// must still fall through to its raw name rather than being absorbed. +func TestParseNavMenuItem_OtherActionsUnchanged(t *testing.T) { + cases := []struct { + typeName string + want string + }{ + {"Forms$FormAction", "PageAction"}, + {"Forms$MicroflowAction", "MicroflowAction"}, + {"Forms$NoAction", "NoAction"}, + {"Forms$SomethingElseAction", "Forms$SomethingElseAction"}, + } + for _, c := range cases { + mi := parseNavMenuItem(menuItemRaw(c.typeName)) + if mi.ActionType != c.want { + t.Errorf("%s -> %q, want %q", c.typeName, mi.ActionType, c.want) + } + } +} diff --git a/sdk/mpr/parser_misc.go b/sdk/mpr/parser_misc.go index bf687a621..6eeb70b12 100644 --- a/sdk/mpr/parser_misc.go +++ b/sdk/mpr/parser_misc.go @@ -594,6 +594,11 @@ func parseNavMenuItem(raw map[string]any) *NavMenuItem { if ms, ok := action["MicroflowSettings"].(map[string]any); ok { mi.Microflow = extractString(ms["Microflow"]) } + case strings.HasSuffix(actionType, "SignOutClientAction"): + // Named rather than left to the raw-type-name default: DESCRIBE and + // both writers key on "SignOutAction", so a round trip only closes + // if the reader produces the name the writer consumes. + mi.ActionType = "SignOutAction" case strings.HasSuffix(actionType, "OpenLinkAction") || strings.HasSuffix(actionType, "OpenLinkClientAction"): mi.ActionType = "OpenLinkAction" case strings.HasSuffix(actionType, "NoAction") || strings.HasSuffix(actionType, "NoClientAction"): From a17d8fa5b316bca297864e4867bec561e38f74a1 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 4 Sep 2026 08:58:13 +0000 Subject: [PATCH 11/17] fix(brain): give project.md the session-start load its cap assumes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The store describes project.md as loaded every session, and justifies giving it the tightest cap in the store on exactly that basis. Nothing made it true. The only route to the brain in a generated project was a row in CLAUDE.md's skills table, and the skill's own description is symptom-triggered ("use before designing something that looks like it was decided before") — so a session that never hits the symptom never learns the project's decisions, and the cap was resting on a load that did not happen. The generated CLAUDE.md now names docs/brain/project.md as the first thing to read, with the module shards on demand and `brain plan` for picking work up. That is the mechanism; the skill remains the detail. Asserted by a test rather than left to review, including that the section sits in the first third of the file — "read this first" is otherwise a claim the document's own ordering contradicts. Control: removing the block fails the test with the two paths named. Noticed by comparing the design against Anthropic's AI-native SDLC playbook, which is explicit that CLAUDE.md is what gets read at the start of a session. The gap was ours: we had the policy without the mechanism. Co-Authored-By: Claude Opus 5 --- cmd/mxcli/cmd_brain_test.go | 25 +++++++++++++++++++++++++ cmd/mxcli/init_claudemd.go | 19 +++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/cmd/mxcli/cmd_brain_test.go b/cmd/mxcli/cmd_brain_test.go index a11292996..9331df42e 100644 --- a/cmd/mxcli/cmd_brain_test.go +++ b/cmd/mxcli/cmd_brain_test.go @@ -6,6 +6,7 @@ import ( "os" "os/exec" "path/filepath" + "strings" "testing" "time" @@ -207,3 +208,27 @@ func TestChangedShardsMapsPathsToShards(t *testing.T) { } func day() time.Time { return time.Date(2026, 9, 3, 0, 0, 0, 0, time.UTC) } + +// The store's project.md is documented as loaded every session, and its cap — +// the tightest in the store — is justified by exactly that. Routing to it +// through the skill alone does not make it true: a skill is triggered by +// symptom, so a session that never hits the symptom never reads the project's +// decisions. The generated CLAUDE.md is the only thing that makes the claim +// mechanical, which is why it is asserted here rather than left to review. +func TestGeneratedClaudeMDRoutesToTheBrainAtSessionStart(t *testing.T) { + md := generateClaudeMD("Demo", "Demo.mpr") + for _, want := range []string{ + "docs/brain/project.md", // the unconditional read + "docs/brain/modules/", // the on-demand shards + "brain plan", // how to pick work up + } { + if !strings.Contains(md, want) { + t.Errorf("generated CLAUDE.md does not mention %q — project.md is then loaded only when a symptom happens to trigger the skill, and the cap that assumes otherwise is unfounded", want) + } + } + // It has to come before the bulk of the file, or "read this first" is a + // claim the document's own ordering contradicts. + if i := strings.Index(md, "docs/brain/project.md"); i < 0 || i > len(md)/3 { + t.Errorf("the brain section is at byte %d of %d; it is meant to be read first", i, len(md)) + } +} diff --git a/cmd/mxcli/init_claudemd.go b/cmd/mxcli/init_claudemd.go index 5cce90c5f..5acd0fe10 100644 --- a/cmd/mxcli/init_claudemd.go +++ b/cmd/mxcli/init_claudemd.go @@ -60,6 +60,25 @@ func generateClaudeMD(projectName, mprFile string) string { w("This is a Mendix project configured for AI-assisted development using mxcli and MDL (Mendix Definition Language).\n\n") // ── Communication Style ──────────────────────────────────────── + // The brain's project.md is described as "loaded every session", and this + // is the only thing that makes that true. Routing to it through the skill + // alone does not: a skill is triggered by symptom, so a session that never + // hits the symptom never learns the project's own decisions — and the + // tightest cap in the store was justified by an unconditional load that + // nothing actually performed. + w("## Project Brain — read this first\n\n") + w("If " + bt + "docs/brain/" + bt + " exists, read " + bt + "docs/brain/project.md" + bt + " before doing anything\n") + w("else. It holds the decisions this project has already made — things no command can\n") + w("tell you, and that are cheap to contradict by accident.\n\n") + w("Then, depending on what you are doing:\n\n") + w("- **Building in a module** — also read " + bt + "docs/brain/modules/.md" + bt + " for the\n") + w(" modules you are about to touch. Not the whole directory; only those.\n") + w("- **Planning, or picking work up** — run " + bt + "./mxcli brain plan -p " + mprPath + bt + ".\n") + w(" It reports what is built from the model itself, so it cannot be out of date.\n\n") + w("Record what you learn with " + bt + "./mxcli brain capture" + bt + ". Read\n") + w(bt + ".ai-context/skills/project-brain/SKILL.md" + bt + " for what belongs there and what does not —\n") + w("the short version is that anything mxcli can answer must never be written down.\n\n") + w("## Communication Style\n\n") w("When discussing changes with the user:\n\n") w("- **Never show raw MDL scripts in chat.** Instead, describe changes in plain language as a numbered list.\n") From a78f36ed86c843bb3c65b527977c7332f057f30b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 07:36:48 +0000 Subject: [PATCH 12/17] fix(test): attribute a failed build to the test that caused it An @expect that is syntactically valid but only rejected by MxBuild took down an entire `mxcli test --local` run: no test results at all, valid tests in the same file never executed, and the cause arrived as ~200 lines of mxbuild JSON with the real error among dozens of unrelated Atlas warnings. BuildResult parsed only status and message and left the rest of the response unread, though mxbuild returns every problem with a severity, an error code and a location. Measured on 11.13, a failing build returns 18 problems of which one is the error, so printing the body meant 11,580 bytes in which nothing marked the line that mattered. Filtering to errors renders it as: [CE0117] Error(s) in expression. -- at MxTest / Microflow 'Test_test_3' / Decision '$result = 3' The location's document names the generated test microflow, so it maps back to the test exactly: that test is reported ERROR with the consistency message and every other test as SKIP -- never PASS, because nothing ran. An error in the project rather than the suite is reported as such instead of being blamed on a test. The finding's other suggested remedy -- refusing an unbound variable at injection time rather than letting the build find it -- was implemented and then removed. It passed check-mdl's 465 scripts and the whole unit suite and was still wrong: `mxcli test` execs its microflows, and the microflow validator's scope model tracks variables where they are ASSIGNED. Reusing it to check READS refuses valid work. Two independent holes, both surfaced only by `make test-integration`: - $latestHttpResponse is a Mendix system variable that no MDL statement declares; it is populated after SEND REST REQUEST - a loop iterator is registered only when the list's type is known Both refuse a microflow `mx check` accepts at 0 errors. A variable model built for checking writes only has to know the names being bound; the read side has to know every name that can legally be in scope, including ones the platform supplies. That set cannot be enumerated confidently here, and each miss refuses a working microflow -- so the build stays the authority and this change makes its verdict legible instead. Nothing is lost by dropping it: CE0109 reaches the build and is attributed to its test by the same path as CE0117. Controls: an error in a generated test microflow produces per-test rows, an error in the user's own model produces none, and no test may be reported PASS after a failed build. The response shape is captured from a real failing build rather than inferred -- no fixture here had recorded one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../skills/fix-issue/findings/cmd-mxcli.jsonl | 1 + CHANGELOG.md | 8 + cmd/mxcli/docker/localapp.go | 2 +- cmd/mxcli/docker/mxserve.go | 133 +++++++++++++++ cmd/mxcli/docker/mxserve_problems_test.go | 130 +++++++++++++++ cmd/mxcli/docker/runlocal.go | 2 +- cmd/mxcli/testrunner/build_attribution.go | 154 ++++++++++++++++++ .../testrunner/build_attribution_test.go | 140 ++++++++++++++++ cmd/mxcli/testrunner/runner_endpoint.go | 15 ++ 9 files changed, 583 insertions(+), 2 deletions(-) create mode 100644 cmd/mxcli/docker/mxserve_problems_test.go create mode 100644 cmd/mxcli/testrunner/build_attribution.go create mode 100644 cmd/mxcli/testrunner/build_attribution_test.go diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl index 28f9dc541..24e2612f7 100644 --- a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl +++ b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl @@ -97,3 +97,4 @@ {"area": "cmd/mxcli", "date": "2026-09-01", "symptom": "After `mxcli test --local`, a live `mxcli run --local` serving the SAME project starts answering **HTTP 200 with a zero-byte body** on every microflow-backed resource \u2014 not a 500, not an error page \u2014 while source-backed ones keep working, so half the app looks fine. The runtime log shows `java.lang.NoClassDefFoundError` on a project class. In a two-app solution it surfaces as tests failing in the OTHER app.", "cause": "The test run recompiles the project's Java into `deployment/run/bin`, which is the classpath the running JVM is holding open. Measured on a real 11.13 project: after one test run all 134 class files have **new inodes and byte-identical content** \u2014 every one deleted and rewritten. A JVM loads classes lazily, so one it has not reached yet can fail permanently. mxcli cannot prevent this: mxbuild's Gradle pass owns the compile and the deployment directory cannot be moved (ledger \u00a7150). So it warns instead, which is what was missing.", "file": "`cmd/mxcli/devloop_recompile_warning.go` (new \u2014 `warnIfDevLoopServing`, `recompileWarning`), `cmd/mxcli/cmd_test_run.go`; reads the existing `cmd/mxcli/devloop_handshake.go`", "insight": "**The mechanism already existed and a duplicate would have broken it.** `mxcli run --local` publishes `devLoopHandshake` at `.mxcli/run-local.json` for `mxcli constant set --apply` \u2014 same path, same pid-liveness staleness check, plus the `adminPass` and `bootConfig` that `--apply` and `--attach` depend on. A second state file was written at that path before this was noticed; it parsed fine (JSON ignores unknown fields) but its WRITER would have silently dropped those two keys. Grep the path before inventing a file. **The liveness check is the feature**: a `run --local` killed or ended by its development licence (\u00a760, measured lifetimes under six hours) leaves the file behind, and a warning driven by the file alone fires forever \u2014 one that is always wrong teaches the reader to skip it. It **warns rather than refuses**, since the warm loop exists so an app can stay up while you work and the reporting project runs two apps that way; neither `--attach` nor `--skip-build` builds, so neither warns. The finding's cost was diagnosis, not breakage \u2014 108 log lines and a wrong hypothesis about a different app, for something whose remedy is one restart \u2014 so the warning names the symptom (HTTP 200, empty body), the part nobody guesses. Controls, end-to-end against a real `run --local`: the warning carries that app's actual pid and port and its handshake still has adminPass and 9 bootConfig keys afterwards; with the loop stopped, and with a stale dead-pid handshake, the same command is silent. Reported as mxcli-formula1 FINDINGS \u00a781."} {"area": "cmd/mxcli", "date": "2026-09-03", "symptom": "`mxcli brain check` reports an entry as MISFILED even though the entry is correct and its anchor points at a real document — the anchor's target is simply of a document type the catalog's `objects` view does not index", "cause": "Misfiling was decided by comparing the shard against the modules of *resolved* anchors. An entry whose only anchor came back NotIndexable had an empty resolved-module list, so the comparison found no match and reported it misfiled — reintroducing, through the misfiling axis, exactly the false staleness that the NotIndexable state exists on the anchor axis to prevent", "file": "`cmd/mxcli/brain/entry.go` (`MisfiledIn`)", "insight": "When a check has two axes, an 'unknown' outcome on one of them must not be read as a negative on the other. The fix is to make misfiling *undecidable* rather than false when nothing resolved: with no resolved anchor there is no evidence about where the entry belongs, and an anchor that truly names nothing is already a failure on its own axis. Caught in development by a table test whose control stubbed the guard to `if false` — a control that deletes the block instead fails to compile on unused variables, which is not a control", "refs": ["ako/mxcli#385", "PROPOSAL_project_brain.md A1"]} {"area": "cmd/mxcli", "date": "2026-09-03", "symptom": "`mxcli brain check` exits 1 on a requirement that is simply not built yet — the entry is correct and current, and the check reports its anchor as NOT FOUND", "cause": "Requirements were recorded as ordinary brain entries, but an entry's anchor was assumed to point BACKWARD at something that exists. A decision's unresolved anchor means the decision is stale; a requirement's unresolved anchor means the work is not done. Same syntax, opposite meaning, and the store had no way to tell them apart", "file": "`cmd/mxcli/brain/entry.go` (`Kind`), `cmd/mxcli/brain/check.go` (`checkSlice`)", "insight": "Before adding a record type to an existing store, ask what a FAILED validation means for it — not just what it looks like. Requirements and decisions share the anchor syntax exactly, which is what made them look like the same thing; they differ only in the direction the anchor points, and that difference is the whole lifecycle. Measured before designing: one unbuilt requirement filed as a decision took `brain check` to exit 1, which settled it in one command. The inversion then pays for itself — a requirement is 'built' when its anchors resolve, so `brain plan` reports progress derived from the model (measured 0/1 -> 1/0 after creating the microflow, with the plan file untouched) instead of a status column that goes stale silently", "refs": ["ako/mxcli#385"]} +{"area": "cmd/mxcli", "cause": "BuildResult parsed only status/restartRequired/message and left everything else in Raw, so a failed build was reported by dumping the whole response body; nothing looked at the per-problem severity/errorCode/locations mxbuild actually returns. The consistency error that stopped the build therefore arrived unmarked among the warnings, and no test was named.", "ce": ["CE0109", "CE0117"], "date": "2026-09-03", "file": "cmd/mxcli/docker/mxserve.go (BuildProblem/BuildLocation/Errors/ErrorSummary/BuildFailedError), cmd/mxcli/testrunner/build_attribution.go (new), wired at cmd/mxcli/testrunner/runner_endpoint.go", "insight": "**The serve /build response already carries everything needed to attribute a build failure, and nothing was reading it.** Measured on 11.13: `problems` is an OBJECT whose inner `problems` list holds each consistency message with severity, errorCode and locations[] {module, document, element} \u2014 the document being `Microflow 'Test_test_3'` WITHOUT its module. So an error in a generated test microflow maps back exactly. The ratio is the point: a failing blank app returns 18 problems of which 1 is the error, and printing the body meant 11,580 bytes in which nothing marked the line that mattered; filtering severity==Error renders it as one line naming the test AND the decision. **Do not guess a response shape \u2014 POST to the serve API and look.** `mxbuild --serve --host=127.0.0.1 --port=N` plus a curl to /build is the whole harness, and no fixture in the repo had ever recorded a FAILING build.\n\n**The abandoned half is the more useful lesson.** Catching these earlier \u2014 refusing an unbound variable in an IF condition at injection time \u2014 was built, passed 465 check-mdl scripts and the whole unit suite, and was WRONG. `mxcli test` execs its microflows, and the microflow validator's scope model tracks variables where they are ASSIGNED; reusing it to check READS refuses valid work. Two independent holes, both found only by `make test-integration`: `$latestHttpResponse` is a Mendix system variable that no MDL statement declares, and a loop iterator is registered only when the list's type is known (`if listType, ok := fb.varTypes[...]`). Both refuse a microflow `mx check` accepts at 0 errors. **Generalisable: a variable model built for checking writes is not a variable model for checking reads** \u2014 the write side only has to know the names being bound, the read side has to know every name that can legally be in scope, including ones the platform supplies. Before reusing any scope model in the opposite direction, enumerate what populates it and assume the list is incomplete.\n\n**Process: `make check-mdl` is NOT the over-reach guard for an exec-path change.** It runs `mxcli check` with no project, so it exercises syntax only; a new refusal on the exec path sails through all 465 scripts. `make test-integration` (what CI runs, and runnable locally with mxbuild cached) execs every doctype script against a real project and runs `mx check` on the result \u2014 that is the guard, and skipping it cost a red CI. Also note the exec/check validator split (#833): a validator fix wired only into ValidateMicroflowBody looks correct under `check -p` and does nothing under `exec`.", "refs": ["ako/mxcli-sudoku FINDINGS #46 follow-up"], "symptom": "`mxcli test --local`: an @expect that is syntactically valid but only fails inside mxbuild takes down the ENTIRE run \u2014 `Error: local runtime: build failed: The project cannot be deployed, because it contains errors.` No test results at all, valid tests in the same file never run, and the cause arrives as ~200 lines of mxbuild JSON in which the real error sits among dozens of unrelated Atlas warnings."} diff --git a/CHANGELOG.md b/CHANGELOG.md index 087a6c3db..0e7c97413 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +- **A failed build now says which test caused it** (ako/mxcli-sudoku FINDINGS #46 follow-up) — an `@expect` that is syntactically valid but only rejected by MxBuild took down an entire `mxcli test --local` run: no test results at all, valid tests in the same file never executed, and the cause arrived as ~200 lines of mxbuild JSON with the real error among dozens of unrelated Atlas warnings. + + `BuildResult` parsed only the status and message and left the rest of the response unread, though mxbuild returns every problem with a severity, an error code and a location. Measured on 11.13, a failing build returns **18 problems of which one is the error**, so printing the body meant 11,580 bytes in which nothing marked the line that mattered. Filtering to errors renders it as `[CE0117] Error(s) in expression. — at MxTest / Microflow 'Test_test_3' / Decision '$result = 3'`. + + The location's document names the generated test microflow, so it maps back to the test exactly: that test is reported `ERROR` with the consistency message, every other test as `SKIP` — never `PASS`, because nothing ran. An error in the project rather than the suite is reported as such instead of being blamed on a test. + + Catching these earlier — refusing an unbound variable at injection time rather than letting the build find it — was built and then **removed**. The microflow validator tracks variables where they are *assigned*, and reusing that model to check *reads* refuses valid microflows: `$latestHttpResponse` is a Mendix system variable no MDL statement declares, and a loop iterator is only registered when the list's type is known. Both are accepted by `mx check` at 0 errors. The scope model is right for the bar it was built for and wrong for this one, so the build stays the authority. + - **`CATALOG.strings` indexes every translatable string, not five hand-picked kinds** — `SHOW LANGUAGES` listed 8 of a project's 9 languages and `search` could not find a widget caption that `DESCRIBE TRANSLATIONS` had just listed. The index was filled by per-type extractors reaching five sites (page title, enum caption, three microflow message templates), so a text anywhere else was never indexed: measured on a stock 11.13 app, **69 of 3265 texts and 8 of 9 languages**. A language present only on an unindexed site is *invisible* rather than undercounted, which also blinded lint rule QUAL005 — it discovers its language set from the same table. The rows now come from the type-agnostic `Texts$Text` walk that `DESCRIBE TRANSLATIONS` already uses, so the two subsystems cannot disagree about what the project contains; the typed path keeps only the strings that are *not* translatable (URLs, log node names, REST paths, documentation, and the `Microflows$StringTemplate` a workflow name is stored in). `StringContext` now names the site — `Forms$ActionButton.Caption` rather than `page_title` — and `ObjectType` is derived from the unit `$Type` mechanically, so a document type Mendix adds later is named correctly with nobody maintaining a list. Same project after: 1496 rows, 9 languages, counts identical to an independent BSON walk. Atlas design templates are ~70% of the corpus and are indexed rather than dropped, because `CREATE TRANSLATIONS` writes them and a `SHOW LANGUAGES` that excluded them would reopen the same split; `ObjectType` is how a consumer filters them. diff --git a/cmd/mxcli/docker/localapp.go b/cmd/mxcli/docker/localapp.go index a44b60072..f951f2611 100644 --- a/cmd/mxcli/docker/localapp.go +++ b/cmd/mxcli/docker/localapp.go @@ -212,7 +212,7 @@ func StartLocalApp(opts LocalAppOptions) (*LocalApp, error) { } if !build.OK() { app.Stop() - return nil, fmt.Errorf("build failed: %s\n%s", build.Message, string(build.Raw)) + return nil, &BuildFailedError{Result: build} } } diff --git a/cmd/mxcli/docker/mxserve.go b/cmd/mxcli/docker/mxserve.go index 951b544c8..a48d69a2c 100644 --- a/cmd/mxcli/docker/mxserve.go +++ b/cmd/mxcli/docker/mxserve.go @@ -11,6 +11,7 @@ import ( "os" "os/exec" "path/filepath" + "strings" "sync" "syscall" "time" @@ -69,12 +70,98 @@ type BuildResult struct { Status string `json:"status"` RestartRequired bool `json:"restartRequired"` Message string `json:"message"` + Problems BuildProblems `json:"problems"` Raw json.RawMessage `json:"-"` } +// BuildProblems is the serve response's problems object. The consistency errors +// are in the inner list; the outer one carries only the summary. +type BuildProblems struct { + Problems []BuildProblem `json:"problems"` +} + +// BuildProblem is one consistency message from a build. +// +// Only the fields anything reads are declared. Severity is the load-bearing one: +// a failing build of a blank 11.13 app returns 18 problems of which 16 are +// warnings and one a deprecation, so printing the response wholesale buries the +// single error that actually stopped the build. +type BuildProblem struct { + Severity string `json:"severity"` // "Error", "Warning", "Deprecation" + Message string `json:"message"` + ErrorCode string `json:"errorCode"` // e.g. "CE0109" + Locations []BuildLocation `json:"locations"` +} + +// BuildLocation is where a problem was found. Document is the human name Studio +// Pro would show — `Microflow 'Test_test_3'` — which is what lets a caller +// attribute an error to the document it generated. +type BuildLocation struct { + Module string `json:"module"` + Document string `json:"document"` + Element string `json:"element"` +} + // OK reports whether the build succeeded. func (r *BuildResult) OK() bool { return r.Status == "Success" } +// Errors returns only the problems that failed the build. +func (r *BuildResult) Errors() []BuildProblem { + var out []BuildProblem + for _, p := range r.Problems.Problems { + if strings.EqualFold(p.Severity, "Error") { + out = append(out, p) + } + } + return out +} + +// ErrorSummary renders the build errors one per line, with the code and the +// document each was found in. +// +// This is what a caller should print instead of the raw response body: the body +// is ~200 lines of JSON in which the real error sits among unrelated Atlas +// warnings, and a reader has no way to tell which line failed the build. +// Returns "" when the response carried no structured errors, so a caller can +// fall back rather than print nothing. +func (r *BuildResult) ErrorSummary() string { + errs := r.Errors() + if len(errs) == 0 { + return "" + } + var b strings.Builder + for i, p := range errs { + if i > 0 { + b.WriteString("\n") + } + b.WriteString(" ") + if p.ErrorCode != "" { + b.WriteString("[" + p.ErrorCode + "] ") + } + b.WriteString(p.Message) + if loc := p.Where(); loc != "" { + b.WriteString(" — at " + loc) + } + } + return b.String() +} + +// Where renders a problem's first location as `Module / Document / Element`, +// skipping the parts the response left empty. +func (p BuildProblem) Where() string { + if len(p.Locations) == 0 { + return "" + } + l := p.Locations[0] + parts := make([]string, 0, 3) + for _, s := range []string{l.Module, l.Document, l.Element} { + if s != "" { + parts = append(parts, s) + } + } + return strings.Join(parts, " / ") +} + // ServeServer wraps a long-lived `mxbuild --serve` process and its build API. type ServeServer struct { Host string @@ -284,3 +371,49 @@ func (w *syncBuffer) String() string { defer w.mu.Unlock() return w.b.String() } + +// buildFailureDetail renders what to show a user after a failed build. +// +// The structured error list when the response carried one, and the raw body only +// as a fallback. Printing the body was the previous behaviour everywhere, and it +// is close to useless: on a blank 11.13 app a single consistency error arrives +// alongside 16 Atlas warnings in ~200 lines of JSON, with nothing marking which +// one stopped the build. +func buildFailureDetail(r *BuildResult) string { + if r == nil { + return "" + } + if summary := r.ErrorSummary(); summary != "" { + return summary + } + return string(r.Raw) +} + +// BuildFailedError is returned when mxbuild rejected the model. +// +// It carries the parsed result so a caller can do something better than print +// the message: `mxcli test` maps each error back to the generated test microflow +// it was found in, which is the difference between "the build failed" and +// "test 3's assertion does not compile". +type BuildFailedError struct { + Result *BuildResult +} + +func (e *BuildFailedError) Error() string { + msg := "build failed" + if e.Result != nil && e.Result.Message != "" { + msg += ": " + e.Result.Message + } + if detail := buildFailureDetail(e.Result); detail != "" { + msg += "\n" + detail + } + return msg +} + +// BuildErrors returns the consistency errors that failed the build. +func (e *BuildFailedError) BuildErrors() []BuildProblem { + if e == nil || e.Result == nil { + return nil + } + return e.Result.Errors() +} diff --git a/cmd/mxcli/docker/mxserve_problems_test.go b/cmd/mxcli/docker/mxserve_problems_test.go new file mode 100644 index 000000000..824786a91 --- /dev/null +++ b/cmd/mxcli/docker/mxserve_problems_test.go @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "encoding/json" + "strings" + "testing" +) + +// serveFailureBody is the shape mxbuild 11.13's serve /build returns when the +// model does not deploy, trimmed to the fields that matter. +// +// Captured from a real response rather than written from the docs: nothing in +// this repo had ever parsed a failing build, and the nesting is easy to get +// wrong — `problems` is an OBJECT with its own `problems` list inside, and the +// outer `errors` list carries only the summary sentence, not the consistency +// errors. The ratio is real too: a blank app returns 16 warnings and a +// deprecation alongside the single error that stopped the build. +const serveFailureBody = `{ + "status": "Failure", + "message": "The project cannot be deployed, because it contains errors.", + "problems": { + "errors": [{"message": "The project cannot be deployed, because it contains errors.", "details": ""}], + "problems": [ + { + "severity": "Warning", + "message": "No 'On click' action specified.", + "errorCode": "CW0055", + "locations": [{"element": "Menu item", "document": "Menu 'Tablet_Menu'", "module": "Atlas_Core"}] + }, + { + "severity": "Deprecation", + "message": "Something is deprecated.", + "errorCode": "CD0001", + "locations": [] + }, + { + "severity": "Error", + "message": "Undefined variable 'nosuchvar'.", + "errorCode": "CE0109", + "locations": [{"element": "End event", "document": "Microflow 'Test_test_3'", "module": "MxTest"}] + } + ] + } +}` + +func TestBuildResultErrors(t *testing.T) { + var r BuildResult + if err := json.Unmarshal([]byte(serveFailureBody), &r); err != nil { + t.Fatalf("decode: %v", err) + } + if r.OK() { + t.Fatal("a Failure status must not read as OK") + } + + // The filter is the whole point: 3 problems in, 1 error out. + errs := r.Errors() + if len(errs) != 1 { + t.Fatalf("Errors() = %d, want 1 (warnings and deprecations must not be reported as errors)", len(errs)) + } + if errs[0].ErrorCode != "CE0109" { + t.Errorf("errorCode = %q, want CE0109", errs[0].ErrorCode) + } + if got := errs[0].Where(); got != "MxTest / Microflow 'Test_test_3' / End event" { + t.Errorf("Where() = %q", got) + } +} + +func TestBuildResultErrorSummary(t *testing.T) { + var r BuildResult + if err := json.Unmarshal([]byte(serveFailureBody), &r); err != nil { + t.Fatalf("decode: %v", err) + } + + summary := r.ErrorSummary() + for _, want := range []string{"CE0109", "Undefined variable 'nosuchvar'", "Microflow 'Test_test_3'"} { + if !strings.Contains(summary, want) { + t.Errorf("summary %q does not carry %q", summary, want) + } + } + // The noise the summary exists to drop. + for _, unwanted := range []string{"CW0055", "Tablet_Menu", "Deprecation"} { + if strings.Contains(summary, unwanted) { + t.Errorf("summary should not carry the warning %q", unwanted) + } + } +} + +// TestBuildFailureDetailFallsBackToRaw guards the case that keeps this safe on a +// future mxbuild: if the response carries no structured errors, the caller must +// still see the body rather than an empty message. +func TestBuildFailureDetailFallsBackToRaw(t *testing.T) { + r := &BuildResult{Status: "Failure", Message: "nope", Raw: json.RawMessage(`{"status":"Failure"}`)} + if got := buildFailureDetail(r); !strings.Contains(got, `"status":"Failure"`) { + t.Errorf("detail = %q, want the raw body as a fallback", got) + } + + // And with structured errors it must prefer them. + var parsed BuildResult + if err := json.Unmarshal([]byte(serveFailureBody), &parsed); err != nil { + t.Fatalf("decode: %v", err) + } + parsed.Raw = json.RawMessage(serveFailureBody) + got := buildFailureDetail(&parsed) + if strings.Contains(got, "CW0055") { + t.Errorf("detail should be the filtered summary, not the raw body: %q", got) + } + if !strings.Contains(got, "CE0109") { + t.Errorf("detail %q lost the error", got) + } +} + +// TestBuildFailedErrorMessage covers what a user sees when a build fails. +func TestBuildFailedErrorMessage(t *testing.T) { + var r BuildResult + if err := json.Unmarshal([]byte(serveFailureBody), &r); err != nil { + t.Fatalf("decode: %v", err) + } + err := &BuildFailedError{Result: &r} + msg := err.Error() + for _, want := range []string{"build failed", "cannot be deployed", "CE0109", "Test_test_3"} { + if !strings.Contains(msg, want) { + t.Errorf("message %q does not carry %q", msg, want) + } + } + if len(err.BuildErrors()) != 1 { + t.Errorf("BuildErrors() = %d, want 1", len(err.BuildErrors())) + } +} diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index 81741c5ae..73defaed7 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -645,7 +645,7 @@ func RunLocal(opts LocalRunOptions) error { return fmt.Errorf("initial build: %w", err) } if !build.OK() { - return fmt.Errorf("initial build failed: %s\n%s", build.Message, string(build.Raw)) + return fmt.Errorf("initial build failed: %s\n%s", build.Message, buildFailureDetail(build)) } // 5b. Bundle the browser client (web/dist). The serve Deploy target writes the diff --git a/cmd/mxcli/testrunner/build_attribution.go b/cmd/mxcli/testrunner/build_attribution.go new file mode 100644 index 000000000..b67523e37 --- /dev/null +++ b/cmd/mxcli/testrunner/build_attribution.go @@ -0,0 +1,154 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Attributing a failed build back to the test that caused it. +// +// An @expect the runner cannot compile is already an ERROR at parse time. What +// is left is everything only MxBuild decides — an undefined variable (CE0109), a +// String compared to a number (CE0117) — where nothing upstream of the build has +// the information to object. +// +// Checking those earlier was tried and abandoned. The microflow validator tracks +// variables where they are ASSIGNED, and reusing that model to check READS +// produces false refusals: `$latestHttpResponse` is a Mendix system variable that +// no MDL statement declares, and a loop iterator is only registered when the +// list's type is known. Both refuse a microflow `mx check` accepts at 0 errors. +// The scope model is right for the bar it was built for and wrong for this one, +// so the build stays the authority and this file makes its verdict legible. +// +// When that happens the build fails, the runtime never boots, and the run +// produces no test results at all. The failure is real; what was missing is +// saying which test caused it. MxBuild locates every problem +// (`module` + `document`), and the test microflows are generated with names this +// package chose, so the mapping back is exact rather than a guess. +// +// ako/mxcli-sudoku FINDINGS #46, follow-up. +package testrunner + +import ( + "fmt" + "regexp" + "strings" + + "github.com/mendixlabs/mxcli/cmd/mxcli/docker" +) + +// testFlowDocumentPattern matches the `document` MxBuild reports for a generated +// test microflow. +// +// MxBuild names the document WITHOUT its module — `Microflow 'Test_test_3'`, +// with the module carried separately in the same location — so this matches the +// bare name and the module is checked alongside it. Anchored on the generated +// prefix, so an error in the user's own microflow is never blamed on a test. +var testFlowDocumentPattern = regexp.MustCompile(`^Microflow '(Test_[^']*)'$`) + +// attributeBuildProblems splits a failed build's errors into those belonging to +// a generated test microflow and those that do not. +// +// The second group matters as much as the first: an error in the user's own +// model also fails the build, and reporting it as a test failure would send them +// looking in the wrong place. +func attributeBuildProblems(problems []docker.BuildProblem, suite *TestSuite) (map[string][]docker.BuildProblem, []docker.BuildProblem) { + byTest := map[string][]docker.BuildProblem{} + var other []docker.BuildProblem + + // Keyed on the BARE document name MxBuild reports, not the qualified one. + known := map[string]string{} // "Test_test_3" -> test ID + if suite != nil { + for _, tc := range suite.Tests { + known[strings.TrimPrefix(testFlowName(tc), mxTestModule+".")] = tc.ID + } + } + + for _, p := range problems { + id := "" + for _, loc := range p.Locations { + if !strings.EqualFold(loc.Module, mxTestModule) { + continue + } + m := testFlowDocumentPattern.FindStringSubmatch(loc.Document) + if m == nil { + continue + } + if tid, ok := known[m[1]]; ok { + id = tid + break + } + } + if id == "" { + other = append(other, p) + continue + } + byTest[id] = append(byTest[id], p) + } + return byTest, other +} + +// buildProblemMessage renders one test's build errors for its result row. +func buildProblemMessage(problems []docker.BuildProblem) string { + parts := make([]string, 0, len(problems)) + for _, p := range problems { + msg := p.Message + if p.ErrorCode != "" { + msg = p.ErrorCode + ": " + msg + } + if p.Locations != nil && p.Where() != "" { + // Only the element is worth repeating here — the module and document + // are the test itself, which the row already names. + if el := p.Locations[0].Element; el != "" { + msg += " (at " + el + ")" + } + } + parts = append(parts, msg) + } + return "the assertion could not be built: " + strings.Join(parts, "; ") +} + +// resultsFromFailedBuild turns a failed build into one result per test, so a run +// that cannot boot still reports something per test instead of nothing at all. +// +// A test MxBuild blamed becomes an ERROR carrying the consistency message. Every +// other test becomes a SKIP: they were not run and must not be counted as +// passing — the whole point of this area is that a test framework never reports +// green for something it did not evaluate. +// +// Returns nil when no error could be attributed to a test, so the caller reports +// the build failure as it always did rather than inventing per-test rows for a +// problem in the user's own model. +func resultsFromFailedBuild(problems []docker.BuildProblem, suite *TestSuite) []TestResult { + byTest, _ := attributeBuildProblems(problems, suite) + if len(byTest) == 0 || suite == nil { + return nil + } + + results := make([]TestResult, 0, len(suite.Tests)) + for _, tc := range suite.Tests { + r := newResult(tc) + if ps, ok := byTest[tc.ID]; ok { + r.Status = StatusError + r.Message = buildProblemMessage(ps) + } else { + r.Status = StatusSkip + r.Message = "not run: another test in this run failed to build" + } + results = append(results, r) + } + return results +} + +// buildFailureHint is appended to the error when a build failure could not be +// attributed to any test, which means it is in the project rather than in the +// suite. +func buildFailureHint(other []docker.BuildProblem) string { + if len(other) == 0 { + return "" + } + var b strings.Builder + b.WriteString("\n The build errors are in the project, not in the tests:") + for _, p := range other { + b.WriteString(fmt.Sprintf("\n %s %s", p.ErrorCode, p.Message)) + if w := p.Where(); w != "" { + b.WriteString(" — at " + w) + } + } + return b.String() +} diff --git a/cmd/mxcli/testrunner/build_attribution_test.go b/cmd/mxcli/testrunner/build_attribution_test.go new file mode 100644 index 000000000..ed626abfb --- /dev/null +++ b/cmd/mxcli/testrunner/build_attribution_test.go @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testrunner + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/cmd/mxcli/docker" +) + +// problem builds one MxBuild problem located in a document. +func problem(code, msg, module, document, element string) docker.BuildProblem { + return docker.BuildProblem{ + Severity: "Error", + ErrorCode: code, + Message: msg, + Locations: []docker.BuildLocation{{Module: module, Document: document, Element: element}}, + } +} + +func suiteOf(ids ...string) *TestSuite { + s := &TestSuite{Name: "suite"} + for _, id := range ids { + s.Tests = append(s.Tests, TestCase{ID: id, Name: "name of " + id}) + } + return s +} + +// TestAttributeBuildProblems covers the mapping from an MxBuild location back to +// the test whose generated microflow it names. +// +// The shape is measured, not assumed: on Mendix 11.13 a serve build reports the +// document WITHOUT its module — `Microflow 'Test_test_3'` — with the module in +// the same location object. +func TestAttributeBuildProblems(t *testing.T) { + suite := suiteOf("test_1", "test_3") + + t.Run("an error in a generated test microflow is attributed to it", func(t *testing.T) { + p := problem("CE0117", "Error(s) in expression.", "MxTest", "Microflow 'Test_test_3'", "End event") + byTest, other := attributeBuildProblems([]docker.BuildProblem{p}, suite) + if len(other) != 0 { + t.Fatalf("unattributed: %v", other) + } + if got := byTest["test_3"]; len(got) != 1 { + t.Fatalf("byTest[test_3] = %v, want 1 problem", got) + } + }) + + t.Run("an error in the user's own model is not blamed on a test", func(t *testing.T) { + // The whole point of the split: reporting this as a test failure sends + // the reader to the wrong file. + p := problem("CE0109", "Undefined variable 'x'.", "Sudoku", "Microflow 'SUB_Deal'", "End event") + byTest, other := attributeBuildProblems([]docker.BuildProblem{p}, suite) + if len(byTest) != 0 { + t.Errorf("must not attribute a project error to a test: %v", byTest) + } + if len(other) != 1 { + t.Errorf("other = %v, want the problem", other) + } + }) + + t.Run("a microflow in MxTest that is not a generated test is not attributed", func(t *testing.T) { + // MxTest also holds the endpoint registration flow. + p := problem("CE0109", "Undefined variable 'x'.", "MxTest", "Microflow 'RegisterEndpoint'", "End event") + byTest, other := attributeBuildProblems([]docker.BuildProblem{p}, suite) + if len(byTest) != 0 || len(other) != 1 { + t.Errorf("byTest=%v other=%v — a non-test MxTest document must stay unattributed", byTest, other) + } + }) + + t.Run("a generated name from a different run is not attributed", func(t *testing.T) { + // test_9 is not in this suite; claiming it would invent a result. + p := problem("CE0117", "Error(s) in expression.", "MxTest", "Microflow 'Test_test_9'", "End event") + byTest, other := attributeBuildProblems([]docker.BuildProblem{p}, suite) + if len(byTest) != 0 || len(other) != 1 { + t.Errorf("byTest=%v other=%v — an unknown test id must stay unattributed", byTest, other) + } + }) +} + +// TestResultsFromFailedBuild covers what a run reports when the build fails: the +// blamed test is an ERROR and every other test is a SKIP. +// +// No test may be reported as passing. A build that never produced a runtime +// evaluated nothing, and a suite that reports green for something it did not +// evaluate is the defect this whole area exists to prevent. +func TestResultsFromFailedBuild(t *testing.T) { + suite := suiteOf("test_1", "test_2", "test_3") + p := problem("CE0117", "Error(s) in expression.", "MxTest", "Microflow 'Test_test_2'", "End event") + + results := resultsFromFailedBuild([]docker.BuildProblem{p}, suite) + if len(results) != 3 { + t.Fatalf("got %d results, want one per test", len(results)) + } + + byID := map[string]TestResult{} + for _, r := range results { + byID[r.ID] = r + if r.Status == StatusPass { + t.Errorf("%s reported PASS after a failed build — nothing ran", r.ID) + } + } + + if got := byID["test_2"]; got.Status != StatusError { + t.Errorf("test_2 status = %v, want ERROR", got.Status) + } else { + for _, want := range []string{"CE0117", "Error(s) in expression"} { + if !strings.Contains(got.Message, want) { + t.Errorf("test_2 message %q does not carry %q", got.Message, want) + } + } + } + for _, id := range []string{"test_1", "test_3"} { + if got := byID[id]; got.Status != StatusSkip { + t.Errorf("%s status = %v, want SKIP", id, got.Status) + } + } +} + +// TestResultsFromFailedBuildDeclinesUnattributableErrors is the control on the +// other side: when the failure is in the user's model, the runner must NOT +// manufacture per-test rows. Without this, every build error in the project +// would be reported as a suite of skipped tests and the real cause would vanish. +func TestResultsFromFailedBuildDeclinesUnattributableErrors(t *testing.T) { + suite := suiteOf("test_1") + p := problem("CE0109", "Undefined variable 'x'.", "Sudoku", "Microflow 'SUB_Deal'", "End event") + + if results := resultsFromFailedBuild([]docker.BuildProblem{p}, suite); results != nil { + t.Fatalf("expected no results for a project-level failure, got %v", results) + } + + _, other := attributeBuildProblems([]docker.BuildProblem{p}, suite) + hint := buildFailureHint(other) + for _, want := range []string{"in the project, not in the tests", "CE0109", "SUB_Deal"} { + if !strings.Contains(hint, want) { + t.Errorf("hint %q does not mention %q", hint, want) + } + } +} diff --git a/cmd/mxcli/testrunner/runner_endpoint.go b/cmd/mxcli/testrunner/runner_endpoint.go index d94e90d75..f0d03069a 100644 --- a/cmd/mxcli/testrunner/runner_endpoint.go +++ b/cmd/mxcli/testrunner/runner_endpoint.go @@ -3,6 +3,7 @@ package testrunner import ( + "errors" "fmt" "io" "os" @@ -99,6 +100,20 @@ func (s *testAppSession) applyModelChange(projectPath string) (string, error) { func runViaEndpoint(opts RunOptions, suite *TestSuite, token string, timeout time.Duration, w io.Writer) (*SuiteResult, error) { sess, err := bootForTests(opts, token, timeout, w) if err != nil { + // A build MxBuild rejected because of a generated test microflow is that + // test's problem, not the run's. Reporting it as an ERROR row — and the + // rest as SKIP — says which assertion broke, where the bare failure said + // only that the project would not deploy (FINDINGS #46 follow-up). + var bf *docker.BuildFailedError + if errors.As(err, &bf) { + if results := resultsFromFailedBuild(bf.BuildErrors(), suite); results != nil { + return &SuiteResult{Name: suite.Name, Tests: results, Started: time.Now()}, nil + } + // Not the tests' doing: the model itself does not build. Say so + // rather than letting the reader assume a test is at fault. + _, other := attributeBuildProblems(bf.BuildErrors(), suite) + return nil, fmt.Errorf("%w%s", err, buildFailureHint(other)) + } return nil, err } defer sess.stop() From 1ae6f4eca4bd7cbe83d17d5006d026974eaf8ba7 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 4 Sep 2026 09:10:30 +0000 Subject: [PATCH 13/17] feat(brain): open questions, and a trigger for capturing decisions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two gaps, both surfaced by comparing the design against Anthropic's AI-native SDLC playbook. OPEN QUESTIONS. The store recorded what had been decided and what was going to be built, but not what was still undecided. A known unknown is brain-shaped by the store's own test — not derivable from the model, lost when the conversation ends, and expensive to rediscover — so it had nowhere to go and was simply forgotten. A question is a decision that has not been made yet, and it needs its own treatment for one reason: its anchors must NOT be checked. It routinely names something that does not exist, because the question is often precisely whether it should. Measured, with the identical anchor: exit 1 as a decision, exit 0 as a question. That is the same property that separates requirements from decisions — what a failed anchor MEANS — so this is a third point on an axis the store already had, not a new mechanism. Questions live beside the decisions they will join rather than in a file of their own, because the moment you need to see one is while reading what that module already decided. The OPEN marker therefore travels in the entry rather than being implied by the file, as a requirement's kind is. That is still one copy of the fact: what the store forbids is two. `brain resolve` converts a question into a decision IN PLACE, keeping its id and its position — an answered question is the same piece of knowledge as the question — with the question retained as the answer's context. From that moment its anchors are checked like any other decision, which is the transition the kind exists for and is asserted rather than assumed. A question filed against a slice is counted apart from its requirements: an unanswered question is not outstanding scope, and counting it as such would overstate what is left to do. A CAPTURE TRIGGER. Capture had no doctrine for WHEN, which left the two halves lopsided: the plan fills at bootstrap, while decisions fill only if someone remembers. The skill now names the trigger the playbook uses for CLAUDE.md — a correction you have had to make twice, because the second one predicts a third for someone else — plus a choice between real alternatives where the losing one would look reasonable later. `mxcli lint` now reports unanswered questions alongside unpromoted entries. Same reasoning as before: a question nobody answers is the one kind of entry that gets more expensive the longer it sits, and a report only `brain check` prints is a report nothing demands. Controls in both directions throughout, including that `--open` does not quietly disable the check for everything, that an entry written before the marker existed still reads as a settled decision, and that a resolved question stays where it was rather than moving to the end of the file. Co-Authored-By: Claude Opus 5 --- .claude/skills/mendix/project-brain/SKILL.md | 56 +++++- CLAUDE.md | 4 +- cmd/mxcli/brain/brain_test.go | 193 +++++++++++++++++++ cmd/mxcli/brain/check.go | 31 ++- cmd/mxcli/brain/entry.go | 58 +++++- cmd/mxcli/brain/shard.go | 9 +- cmd/mxcli/brain/store.go | 64 ++++++ cmd/mxcli/cmd_brain.go | 62 +++++- cmd/mxcli/cmd_lint.go | 33 +++- docs-site/src/tools/project-brain.md | 49 +++++ 10 files changed, 541 insertions(+), 18 deletions(-) diff --git a/.claude/skills/mendix/project-brain/SKILL.md b/.claude/skills/mendix/project-brain/SKILL.md index 132bbeda9..a41a99774 100644 --- a/.claude/skills/mendix/project-brain/SKILL.md +++ b/.claude/skills/mendix/project-brain/SKILL.md @@ -1,6 +1,6 @@ --- name: project-brain -description: "Project-specific knowledge mxcli cannot compute — the requirements and slices being built from (a spec, a prototype, a conversation), why a pattern was chosen here, which marketplace version broke what. Use when starting from requirements that live outside git, before designing something that looks like it was decided before, and when an mxbuild error is resolved by something non-obvious." +description: "Project-specific knowledge mxcli cannot compute — the requirements and slices being built from (a spec, a prototype, a conversation), why a pattern was chosen here, what is still undecided, which marketplace version broke what. Use when starting from requirements that live outside git, before designing something that looks like it was decided before, when you have had to correct the same thing twice, and when an mxbuild error is resolved by something non-obvious." --- # Project brain @@ -9,6 +9,8 @@ The brain holds what mxcli **cannot** compute about this project. Two halves: - **Decisions** — why a pattern was chosen here, which marketplace version broke what, what a recurring mxbuild error means in *this* app. +- **Open questions** — what is *not* decided yet, so it is not silently + forgotten and rediscovered expensively later. - **The plan** — the requirements being built from and the slices they are grouped into, when the source is a specification document, a prototype or a conversation rather than GitHub issues. @@ -143,6 +145,56 @@ A slice holds source material, so its budget is much larger than a decision shard's — and it is not loaded every session. But it is still a budget: **a slice too long to read is a slice that should be split.** +## When to capture a decision + +Capture is easy to postpone forever, so it needs a trigger rather than good +intentions. Two, and the first is the reliable one: + +1. **You have had to correct the same thing twice.** The second correction is + the signal: it will happen a third time to someone else. Capture what the + right answer is and why, anchored at whatever you were working on. +2. **You chose between real alternatives** and the losing one would look + reasonable to the next person. Record the choice *and* what ruled the other + out — a decision without its reason gets re-litigated. + +If you are unsure whether something qualifies, capture it. Staging costs +nothing and is reversible; a person decides what is worth committing. + +## Recording what is NOT decided + +An open question is a decision that has not been made yet. Record it rather +than carrying it in your head — the conversation ends, and the question is +expensive to rediscover. + +```bash +mxcli brain capture "Do approvers see rejected orders? +The spec is silent. Affects the overview page and the access rules." \ + --open -a @Sales.Order -p app.mpr +``` + +A question's **anchors are not checked**. It may name something that does not +exist — often the question is precisely whether it should — so the staleness +rule that keeps decisions honest does not apply to it. + +`--open` combines with `--slice`: a question about a slice's scope is filed with +that slice, and is counted apart from its requirements. An unanswered question +is not outstanding scope, so it never inflates the slice. + +Answering it turns it into a decision, in place: + +```bash +mxcli brain resolve "Yes, for 30 days +Agreed with the product owner; drives the overview filter and the access rule." +``` + +The entry keeps its id and its position, and the question survives as the +answer's context. From that moment its anchors **are** checked, like any other +decision. + +`mxcli brain check` and `mxcli lint` both report unanswered questions until +someone resolves one. That is deliberate: a question nobody answers is the one +kind of entry that gets more expensive the longer it sits. + ## Write the anchor, not the name `@Sales.Order.Status` is what makes an entry **routable** (its module decides @@ -211,6 +263,8 @@ cap: the cap is what stops the store becoming a file nobody reads. | `mxcli brain promote [--to ]` | Writes it into its shard. The human step | | `mxcli brain drop ` | Removes it from the queue or from its shard | | `mxcli brain capture "" --slice [-a @Anchor]…` | Queues a **requirement** of that slice | +| `mxcli brain capture "" --open [-a @Anchor]…` | Queues an **open question**; its anchors are not checked | +| `mxcli brain resolve ""` | Answers a question, turning it into a decision in place | | `mxcli brain plan` | The roadmap: each slice's requirements counted against the model | | `mxcli brain check [--changed]` | Anchors still resolve, entries in the right shard, plus slice progress | | `mxcli brain show []` | Entries, lines and headroom per shard | diff --git a/CLAUDE.md b/CLAUDE.md index 9384aa2e8..af2151e10 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -717,7 +717,7 @@ go build -o bin/mxcli ./cmd/mxcli | **Marketplace drift** | `mxcli marketplace diff -p app.mpr [--to V] [--json]` | Which elements of an installed marketplace module have been edited locally, and what an upgrade would overwrite | | **Model repair** | `mxcli fix widgets`, `mxcli fix design-properties` | Runs `mx update-widgets` / `mx rename-design-properties` and **persists** the result without their MPR v2 → v1 collapse (harvest: let the tool convert, read the units back, restore v2, write the changed ones through mxcli's writer). Clears CE0463 / CE6087 after a headless install — measured 203 → 0 errors on a vanilla 11.12.1 app | | **Diagnostics** | `mxcli diag [--bundle]` | Session logs, version info, bug report bundles | -| **Project brain** | `mxcli brain init\|capture\|staged\|promote\|drop\|check\|show\|plan` | Opt-in store in `docs/brain/` for what mxcli **cannot** compute (why a pattern was chosen here, which marketplace version broke what). Sharded by module — an entry's first anchor names its file — so a session loads `project.md` plus the modules it is touching, not the whole store. Also holds the **plan**: requirements grouped into slices, whose anchors point *forward*, so `brain plan` reports progress **derived from the model** rather than from a status column. An agent captures to a git-ignored queue; a person promotes | +| **Project brain** | `mxcli brain init\|capture\|staged\|promote\|drop\|check\|show\|plan\|resolve` | Opt-in store in `docs/brain/` for what mxcli **cannot** compute (why a pattern was chosen here, which marketplace version broke what). Sharded by module — an entry's first anchor names its file — so a session loads `project.md` plus the modules it is touching, not the whole store. Also holds the **plan**: requirements grouped into slices, whose anchors point *forward*, so `brain plan` reports progress **derived from the model** rather than from a status column. An agent captures to a git-ignored queue; a person promotes | | **New project** | `mxcli new --version X.Y.Z [--output-dir dir] [--theme none] [--layout none]` | Downloads mxbuild, creates blank project, applies default styling, scaffolds a project-owned layout, runs init, installs Linux mxcli for devcontainer | | **Default styling** | `mxcli theme list\|show\|apply\|remove` | Applies a theme (signal/ledger/console) — files under `theme/` only, the model is never touched | | **Project themes** | `mxcli theme create [--from ]` | Scaffolds a theme the project owns into `theme/mxcli-themes/`; `--from ` seeds the palette from `--mxt-*` declarations | @@ -814,7 +814,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati ## Current Implementation Status **Implemented:** -- Project brain (`mxcli brain init/capture/staged/promote/drop/check/show`): an **opt-in** store in `docs/brain/` for the project knowledge mxcli cannot compute. The governing rule is that anything derivable from the model is answered by a command and never written down — a note that transcribes the model disagrees with it silently. Records shard by **anchor scope**: an entry's first anchor names its file (`@Sales.Order` → `modules/Sales.md`), an anchorless entry is cross-cutting (`project.md`), and there is no index to maintain because the module prefix *is* the file name. That is what makes the cap per-shard rather than a project-wide budget, and lets a session load `project.md` plus the modules it is touching. `check` answers two independent questions: each anchor is **resolved / not found / not indexable** — only the middle one fails, and the third exists because the catalog's `objects` view covers the describable types only, so a scheduled event would otherwise read as *missing* (separated with `FindDocumentUnit`, which cannot miss a kind because it never asks what kind anything is). Misfiling is a **second axis, not a fourth state**: every anchor can resolve and the entry still be in the wrong file, and it is only decided when something resolved — judging it on an all-not-indexable entry reintroduced the same false staleness through the other axis (caught by a test, with the guard stubbed as the control). An agent `capture`s to a git-ignored queue and a person `promote`s; the queue is deliberately **not** sharded, because routing it would force the file decision before a human has looked at the entry. `mxcli lint` prints the unpromoted-queue count, because a report only `brain check` prints is a report nothing demands. Sizes are computed by `brain show` and never written into a committed file. A second record kind, **requirement**, lives in `plan/.md` and inverts the anchor's meaning: a decision's anchor points backward (not resolving = stale, fails), a requirement's points forward (not resolving = not built yet, passes). Measured: filed as an ordinary entry, one unbuilt requirement takes `brain check` to exit 1 — which is why it is a separate kind rather than more entries in the same files. That inversion is also what makes `brain plan` a real progress report: a requirement is *built* when its anchors resolve, so creating the microflow it names moves the count with the plan file untouched (measured 0/1 → 1/0). A status written beside a requirement is therefore refused by the skill, not just discouraged. Slices are ordered by name (`01-accounts`), span modules by design (so misfiling does not apply), and carry a generous cap that enforces the slicing discipline — a slice too long to read should be split. `bootstrap-app` asks for requirements at the interview and records them by default. Package: `cmd/mxcli/brain/`. See `docs-site/src/tools/project-brain.md` and `docs/11-proposals/PROPOSAL_project_brain.md` +- Project brain (`mxcli brain init/capture/staged/promote/drop/check/show`): an **opt-in** store in `docs/brain/` for the project knowledge mxcli cannot compute. The governing rule is that anything derivable from the model is answered by a command and never written down — a note that transcribes the model disagrees with it silently. Records shard by **anchor scope**: an entry's first anchor names its file (`@Sales.Order` → `modules/Sales.md`), an anchorless entry is cross-cutting (`project.md`), and there is no index to maintain because the module prefix *is* the file name. That is what makes the cap per-shard rather than a project-wide budget, and lets a session load `project.md` plus the modules it is touching. `check` answers two independent questions: each anchor is **resolved / not found / not indexable** — only the middle one fails, and the third exists because the catalog's `objects` view covers the describable types only, so a scheduled event would otherwise read as *missing* (separated with `FindDocumentUnit`, which cannot miss a kind because it never asks what kind anything is). Misfiling is a **second axis, not a fourth state**: every anchor can resolve and the entry still be in the wrong file, and it is only decided when something resolved — judging it on an all-not-indexable entry reintroduced the same false staleness through the other axis (caught by a test, with the guard stubbed as the control). An agent `capture`s to a git-ignored queue and a person `promote`s; the queue is deliberately **not** sharded, because routing it would force the file decision before a human has looked at the entry. `mxcli lint` prints the unpromoted-queue count, because a report only `brain check` prints is a report nothing demands. Sizes are computed by `brain show` and never written into a committed file. A second record kind, **requirement**, lives in `plan/.md` and inverts the anchor's meaning: a decision's anchor points backward (not resolving = stale, fails), a requirement's points forward (not resolving = not built yet, passes). Measured: filed as an ordinary entry, one unbuilt requirement takes `brain check` to exit 1 — which is why it is a separate kind rather than more entries in the same files. That inversion is also what makes `brain plan` a real progress report: a requirement is *built* when its anchors resolve, so creating the microflow it names moves the count with the plan file untouched (measured 0/1 → 1/0). A status written beside a requirement is therefore refused by the skill, not just discouraged. Slices are ordered by name (`01-accounts`), span modules by design (so misfiling does not apply), and carry a generous cap that enforces the slicing discipline — a slice too long to read should be split. A third kind, **open question** (`--open`), records what is *not* decided; its anchors are deliberately **not** checked, since the question is often whether the thing should exist at all — measured, the identical anchor exits 1 as a decision and 0 as a question. `brain resolve` converts one into a decision in place, keeping its id and position and starting to check its anchors, which is the transition the kind exists for. Unanswered questions are reported by `brain check` and by `mxcli lint`. The skill also gives capture a **trigger** rather than good intentions — a correction you have had to make twice — because the decisions half otherwise under-fills while the plan half fills at bootstrap. `bootstrap-app` asks for requirements at the interview and records them by default. Package: `cmd/mxcli/brain/`. See `docs-site/src/tools/project-brain.md` and `docs/11-proposals/PROPOSAL_project_brain.md` - Default styling + runtime theme switching (`mxcli theme list/show/create/apply/remove/switcher`, `mxcli new --theme`): three embedded themes (**signal** light-first, **ledger** light-first, **console** dark-first), each a palette in `theme/web/custom-variables.scss` + a shared Atlas wiring partial + a theme partial imported from `theme/web/main.scss` (which compiles last), plus vendored fonts. **No model changes**, so it hot-applies under `run --local --watch` and cannot affect a build. Generated regions are digest-fenced: a block carrying local edits is refused rather than overwritten. Applying a theme removes the previous one. `--variant auto` (default) ships both palettes — the app follows `prefers-color-scheme` before first paint and honours a `theme-light`/`theme-dark` class on ``; `light`/`dark` bakes one. `theme switcher install` is the only part that writes to the model (JS actions + a nanoflow for a toggle button). A project can add its own themes under `theme/mxcli-themes//` (committed, not compiled); `theme create [--from ]` scaffolds one from an existing theme, renaming the identifiers built from the name and optionally seeding the palette from `--mxt-*` declarations in a design artifact. A local theme shadows a built-in of the same name. Package: `cmd/mxcli/theme/`. See `docs/11-proposals/PROPOSAL_default_styling.md` - MPR v1/v2 reading and writing - Idempotent writes (ADR-0008): a unit whose new content is semantically equal to what is stored is **not written**, so re-running an MDL script against an in-sync project leaves the `.mpr` and `mprcontents/` byte-identical and Studio Pro shows no version-control changes. Comparison is on a canonical form (element `$ID`s normalised away — a rebuild mints them randomly, so byte comparison would skip nothing); `Microflows$Microflow.StableId` is carried from the stored document rather than re-minted, because the build derives every client-callable microflow's operation id from it. When a write **does** land, `canon.TransplantIDs` matches the rebuild against the stored document and reuses its element `$ID`s (rewriting every pointer in the same pass), so a changed document's diff is the change rather than a wholesale replacement — measured on #910's nanoflow: 1 of 37 identities survived an argument edit before, 37 of 37 after, and a change plus its revert returns to the original bytes. Inserting or deleting an activity mints IDs only for the genuinely new elements. One policy in `modelsdk/canon`, called from both engines' write choke points. `MXCLI_ALWAYS_WRITE=1` disables elision (not preservation) for bisecting — which means it no longer changes the resulting bytes, only the mtimes. The executor's output distinguishes the two: `Unchanged nanoflow: …` where the write was skipped. See `docs-site/src/internals/idempotent-writes.md` diff --git a/cmd/mxcli/brain/brain_test.go b/cmd/mxcli/brain/brain_test.go index 02c54bda6..bb0b7d51f 100644 --- a/cmd/mxcli/brain/brain_test.go +++ b/cmd/mxcli/brain/brain_test.go @@ -634,3 +634,196 @@ func TestPlanSlicesGetMoreRoomThanDecisions(t *testing.T) { t.Fatal("a slice holds source material and is not loaded every session; it needs more room than a decision shard") } } + +func mustQuestion(t *testing.T, text, slice string, anchors ...string) Entry { + t.Helper() + e, err := NewQuestion(text, anchors, slice, day) + if err != nil { + t.Fatalf("NewQuestion(%q): %v", text, err) + } + return e +} + +// A question's anchors are not checked, and the control is the identical +// anchor recorded as a decision. Often the question IS whether the thing should +// exist, so the staleness rule that keeps decisions honest would report every +// question as a defect. +func TestOpenQuestionIsNotCheckedButTheSameDecisionIs(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + r := stubResolver{} // resolves nothing + + q := mustQuestion(t, "Should approvers see rejected orders?", "", "@Sales.ACT_Nope") + if err := s.Promote(q, "Sales"); err != nil { + t.Fatal(err) + } + rep, err := Check(s, r, []string{"Sales"}) + if err != nil { + t.Fatal(err) + } + if rep.Failed() { + t.Errorf("an open question must not fail the check: %+v", rep) + } + if len(rep.Open) != 1 || rep.Open[0].EntryID != q.ID { + t.Fatalf("the question must still be reported: %+v", rep.Open) + } + if len(rep.Findings) != 0 { + t.Errorf("a question's anchors must not be resolved at all: %+v", rep.Findings) + } + + // Control: the same anchor as a settled decision still fails. + d := mustEntry(t, "Approvers see rejected orders", "@Sales.ACT_Nope") + if err := s.Promote(d, "Sales"); err != nil { + t.Fatal(err) + } + rep, err = Check(s, r, []string{"Sales"}) + if err != nil { + t.Fatal(err) + } + if !rep.Failed() { + t.Error("a decision anchored at nothing must still fail — otherwise --open disables the check for everything") + } +} + +// Resolution is the transition the whole kind exists for: the entry becomes a +// decision in place, keeps its identity, and its anchors start being checked. +func TestResolvingAQuestionMakesItACheckedDecision(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + q := mustQuestion(t, "Should approvers see rejected orders?", "", "@Sales.Order") + if err := s.Promote(q, "Sales"); err != nil { + t.Fatal(err) + } + resolved, err := q.Resolve("Yes, for 30 days", day) + if err != nil { + t.Fatal(err) + } + if resolved.ID != q.ID { + t.Errorf("id changed on resolution (%s -> %s); an answered question is the same knowledge", q.ID, resolved.ID) + } + if resolved.Open { + t.Error("a resolved question is no longer open") + } + if !strings.Contains(resolved.Body, "Should approvers see rejected orders?") { + t.Errorf("the question must survive as the answer's context: %q", resolved.Body) + } + if err := s.Replace("Sales", resolved); err != nil { + t.Fatal(err) + } + + // Now it IS checked: a dead anchor on the answer fails. + rep, err := Check(s, stubResolver{}, []string{"Sales"}) + if err != nil { + t.Fatal(err) + } + if !rep.Failed() { + t.Error("once answered, the entry's anchors must be checked like any other decision") + } + if len(rep.Open) != 0 { + t.Errorf("it is no longer an open question: %+v", rep.Open) + } +} + +func TestResolveRefusesSomethingThatIsNotAQuestion(t *testing.T) { + if _, err := mustEntry(t, "A settled decision").Resolve("an answer", day); err == nil { + t.Fatal("resolving a decision must be refused") + } + if _, err := mustQuestion(t, "A question", "").Resolve("", day); err == nil { + t.Fatal("an empty answer must be refused") + } +} + +func TestReplaceKeepsPositionInTheShard(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + first := mustEntry(t, "First decision", "@Sales.A") + q := mustQuestion(t, "A question?", "", "@Sales.B") + last := mustEntry(t, "Last decision", "@Sales.C") + for _, e := range []Entry{first, q, last} { + if err := s.Promote(e, "Sales"); err != nil { + t.Fatal(err) + } + } + resolved, err := q.Resolve("An answer", day) + if err != nil { + t.Fatal(err) + } + if err := s.Replace("Sales", resolved); err != nil { + t.Fatal(err) + } + entries, _, err := s.LoadShard("Sales") + if err != nil { + t.Fatal(err) + } + if len(entries) != 3 || entries[1].ID != q.ID || entries[1].Title != "An answer" { + t.Fatalf("a resolved question must stay where it was, not move to the end: %+v", entries) + } +} + +// A question filed against a slice is not scope until it is answered, so it +// must not inflate the slice's outstanding work. +func TestSliceQuestionsAreCountedApartFromRequirements(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + req := mustRequirement(t, "Orders can be approved", "02-approvals", "@Sales.ACT_Approve") + q := mustQuestion(t, "Do approvers see rejected orders?", "02-approvals", "@Sales.Order") + for _, e := range []Entry{req, q} { + if err := s.Promote(e, e.Shard()); err != nil { + t.Fatal(err) + } + } + rep, err := Check(s, stubResolver{}, []string{PlanShard("02-approvals")}) + if err != nil { + t.Fatal(err) + } + got := rep.Slices[0] + if got.Questions != 1 { + t.Errorf("questions = %d, want 1: %+v", got.Questions, got) + } + if got.Planned != 1 || got.Total() != 1 { + t.Errorf("an unanswered question is not outstanding scope: %+v", got) + } + if len(rep.Open) != 1 { + t.Errorf("a slice's questions must still be reported: %+v", rep.Open) + } +} + +func TestOpenMarkerRoundTrips(t *testing.T) { + in := []Entry{ + mustQuestion(t, "Still open?", "", "@Sales.Order"), + mustEntry(t, "Settled", "@Sales.Order"), + } + out, malformed, err := ParseShard("Sales", RenderShard("Sales", in)) + if err != nil || len(malformed) != 0 { + t.Fatalf("err=%v malformed=%v", err, malformed) + } + if !out[0].Open || out[1].Open { + t.Fatalf("the OPEN marker did not round-trip: %+v", out) + } +} + +// Entries written before questions existed carry no marker and must read back +// as settled decisions, not as questions. +func TestEntryWithoutTheMarkerIsNotOpen(t *testing.T) { + content := "# Sales\n\n" + shardMarker + "\n\n## An older entry\n\n" + + "Anchors: `@Sales.Order` · id `abc123` · 2026-08-01\n" + out, malformed, err := ParseShard("Sales", content) + if err != nil || len(malformed) != 0 { + t.Fatalf("err=%v malformed=%v", err, malformed) + } + if len(out) != 1 || out[0].Open { + t.Fatalf("an entry with no marker is a settled decision: %+v", out) + } +} diff --git a/cmd/mxcli/brain/check.go b/cmd/mxcli/brain/check.go index 17ca24410..0b06f969e 100644 --- a/cmd/mxcli/brain/check.go +++ b/cmd/mxcli/brain/check.go @@ -67,6 +67,13 @@ type AnchorFinding struct { Kind string } +// OpenQuestion is something the project has not decided yet. +type OpenQuestion struct { + Shard string + EntryID string + Title string +} + // MisfiledFinding is an entry sitting in a shard none of its anchors belong to. type MisfiledFinding struct { Shard string @@ -85,12 +92,18 @@ type SliceProgress struct { // Planned is requirements with at least one anchor that does not resolve // yet. Not a failure: that is what a requirement is until it is built. Planned int + // Questions is open questions filed against this slice — scope that is not + // settled. They are not requirements and are not counted as either built + // or planned; counting an unanswered question as outstanding work would + // overstate the slice. + Questions int // Unanchored is requirements with no anchor at all. They cannot be // measured, and are counted apart rather than silently called planned. Unanchored int } -// Total is every requirement in the slice. +// Total is every requirement in the slice. Open questions are excluded: they +// are not scope until they are answered. func (p SliceProgress) Total() int { return p.Built + p.Planned + p.Unanchored } // Report is what `brain check` prints and exits on. @@ -103,6 +116,7 @@ type Report struct { Misfiled []MisfiledFinding Malformed []string // entry blocks whose metadata line could not be read Slices []SliceProgress + Open []OpenQuestion } // Failed reports whether the check should exit non-zero. @@ -143,10 +157,19 @@ func Check(s *Store, r Resolver, shards []string) (Report, error) { rep.Anchors += progress.anchors rep.ResolvedN += progress.resolved rep.Slices = append(rep.Slices, progress.SliceProgress) + rep.Open = append(rep.Open, progress.open...) continue } for _, e := range entries { rep.Entries++ + if e.Open { + // A question's anchors are not checked. It may name something + // that does not exist — often the question IS whether it + // should — so the staleness rule that keeps decisions honest + // would report every question as a defect. + rep.Open = append(rep.Open, OpenQuestion{Shard: shard, EntryID: e.ID, Title: e.Title}) + continue + } var resolvedModules []string for _, a := range e.ParsedAnchors() { rep.Anchors++ @@ -178,6 +201,7 @@ func Check(s *Store, r Resolver, shards []string) (Report, error) { type sliceCounts struct { SliceProgress anchors, resolved int + open []OpenQuestion } // checkSlice counts a slice's requirements against the model. It records no @@ -192,6 +216,11 @@ type sliceCounts struct { func checkSlice(r Resolver, shard string, entries []Entry) (sliceCounts, error) { out := sliceCounts{SliceProgress: SliceProgress{Slice: SliceOf(shard)}} for _, e := range entries { + if e.Open { + out.Questions++ + out.open = append(out.open, OpenQuestion{Shard: shard, EntryID: e.ID, Title: e.Title}) + continue + } anchors := e.ParsedAnchors() if len(anchors) == 0 { out.Unanchored++ diff --git a/cmd/mxcli/brain/entry.go b/cmd/mxcli/brain/entry.go index 320b89714..feff2de6e 100644 --- a/cmd/mxcli/brain/entry.go +++ b/cmd/mxcli/brain/entry.go @@ -39,7 +39,17 @@ type Entry struct { // existed still reads correctly. Use EntryKind rather than this field. Kind Kind `json:"kind,omitempty"` // Slice is the deliverable a requirement belongs to. Empty for a decision. - Slice string `json:"slice,omitempty"` + Slice string `json:"slice,omitempty"` + // Open marks an unresolved question: a decision that has not been made + // yet. It lives beside the decisions it will join rather than in a file of + // its own, because the moment you need to see it is when you are reading + // what that module already decided. + // + // Unlike Kind for a requirement — which is implied by the file — this has + // to travel with the entry, since a question shares its shard with settled + // decisions. There is still only ONE copy of the fact, so nothing can + // drift; what the store forbids is two copies, not one in an odd place. + Open bool `json:"open,omitempty"` Title string `json:"title"` Body string `json:"body,omitempty"` Anchors []string `json:"anchors,omitempty"` @@ -129,6 +139,52 @@ func NewRequirement(text string, anchors []string, slice string, now time.Time) // user's choice rather than a field mxcli maintains. var sliceName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]*$`) +// NewQuestion builds an open question: something undecided, recorded so it is +// not silently forgotten and rediscovered expensively later. +// +// A question's anchors are not checked for staleness. It may name something +// that does not exist — the question is often precisely whether it should — so +// the rule that keeps decisions honest would report every question as a defect. +func NewQuestion(text string, anchors []string, slice string, now time.Time) (Entry, error) { + var ( + e Entry + err error + ) + if slice != "" { + e, err = NewRequirement(text, anchors, slice, now) + } else { + e, err = NewEntry(text, anchors, now) + } + if err != nil { + return Entry{}, err + } + e.Open = true + return e, nil +} + +// Resolve turns a question into the decision it was always going to become, +// keeping the entry's id so that anything referring to it still resolves, and +// keeping the question itself as the answer's context. +// +// The id therefore stops matching the title it was derived from. That is +// deliberate: an answered question is the same piece of knowledge as the +// question, and re-minting the id would make it a different one. +func (e Entry) Resolve(answer string, now time.Time) (Entry, error) { + if !e.Open { + return Entry{}, fmt.Errorf("entry %s is not an open question", e.ID) + } + title, body := splitTitle(answer) + if title == "" { + return Entry{}, fmt.Errorf("an answer needs at least one line") + } + out := e + out.Open = false + out.Title = title + out.Date = now.Format("2006-01-02") + out.Body = strings.TrimSpace("Resolves: " + e.Title + "\n\n" + strings.TrimSpace(body+"\n\n"+e.Body)) + return out, nil +} + // Shard is where this entry belongs. A requirement goes to its slice; a // decision to the module of its first anchor. func (e Entry) Shard() string { diff --git a/cmd/mxcli/brain/shard.go b/cmd/mxcli/brain/shard.go index ca89fd2b1..6c870776d 100644 --- a/cmd/mxcli/brain/shard.go +++ b/cmd/mxcli/brain/shard.go @@ -22,7 +22,7 @@ const shardMarker = "" // metaLine matches an entry's one metadata line. The separator is a middle dot // so that a title or body containing a hyphen cannot be mistaken for it. -var metaLine = regexp.MustCompile("^Anchors: (.*?) · id `([0-9a-f]{6})` · (\\d{4}-\\d{2}-\\d{2})\\s*$") +var metaLine = regexp.MustCompile("^Anchors: (.*?) · id `([0-9a-f]{6})` · (\\d{4}-\\d{2}-\\d{2})( · OPEN)?\\s*$") // anchorRef matches one backticked anchor inside the metadata line. var anchorRef = regexp.MustCompile("`(@[A-Za-z_][A-Za-z0-9_.]*)`") @@ -45,7 +45,11 @@ func RenderShard(shard string, entries []Entry) string { func renderEntry(e Entry) string { var b strings.Builder fmt.Fprintf(&b, "## %s\n\n", e.Title) - fmt.Fprintf(&b, "Anchors: %s · id `%s` · %s\n", renderAnchors(e.Anchors), e.ID, e.Date) + open := "" + if e.Open { + open = " · OPEN" + } + fmt.Fprintf(&b, "Anchors: %s · id `%s` · %s%s\n", renderAnchors(e.Anchors), e.ID, e.Date, open) if e.Body != "" { fmt.Fprintf(&b, "\n%s\n", e.Body) } @@ -125,6 +129,7 @@ func parseEntry(block string) (Entry, bool) { e.Anchors = append(e.Anchors, a[1]) } e.ID, e.Date = m[2], m[3] + e.Open = m[4] != "" metaAt = i break } diff --git a/cmd/mxcli/brain/store.go b/cmd/mxcli/brain/store.go index 1637c2eff..cd584ed50 100644 --- a/cmd/mxcli/brain/store.go +++ b/cmd/mxcli/brain/store.go @@ -189,6 +189,47 @@ func (s *Store) Promote(e Entry, shard string) error { return s.SaveShard(shard, next) } +// Replace swaps an entry for a new one with the same id, in place. Used by +// resolution, where a question becomes the decision it was heading towards and +// should not move, jump to the end of the file, or change identity. +func (s *Store) Replace(shard string, e Entry) error { + entries, _, err := s.LoadShard(shard) + if err != nil { + return err + } + found := false + for i := range entries { + if entries[i].ID == e.ID { + entries[i], found = e, true + break + } + } + if !found { + return fmt.Errorf("%s does not carry entry %s", shard, e.ID) + } + return s.SaveShard(shard, entries) +} + +// Find returns the entry with the given id and the shard holding it. +func (s *Store) Find(id string) (Entry, string, error) { + shards, err := s.ListShards() + if err != nil { + return Entry{}, "", err + } + for _, sh := range shards { + entries, _, err := s.LoadShard(sh) + if err != nil { + return Entry{}, "", err + } + for _, e := range entries { + if e.ID == id { + return e, sh, nil + } + } + } + return Entry{}, "", nil +} + // Drop removes an entry by id, reporting which shard it came from and whether // that emptied the shard. func (s *Store) Drop(id string) (shard string, deletedFile bool, err error) { @@ -221,6 +262,29 @@ func (s *Store) Drop(id string) (shard string, deletedFile bool, err error) { return "", false, nil } +// OpenQuestions lists the unanswered questions across the store. It needs no +// resolver and no catalog — just the files — so a caller that only wants to +// nag about them pays a few small reads rather than a model load. +func (s *Store) OpenQuestions() ([]OpenQuestion, error) { + shards, err := s.ListShards() + if err != nil { + return nil, err + } + var out []OpenQuestion + for _, sh := range shards { + entries, _, err := s.LoadShard(sh) + if err != nil { + return nil, err + } + for _, e := range entries { + if e.Open { + out = append(out, OpenQuestion{Shard: sh, EntryID: e.ID, Title: e.Title}) + } + } + } + return out, nil +} + // Usage computes size and headroom for every shard. Nothing here is cached or // written down (A6). func (s *Store) Usage() ([]Usage, error) { diff --git a/cmd/mxcli/cmd_brain.go b/cmd/mxcli/cmd_brain.go index d710aea52..7decb0a69 100644 --- a/cmd/mxcli/cmd_brain.go +++ b/cmd/mxcli/cmd_brain.go @@ -93,9 +93,13 @@ var brainCaptureCmd = &cobra.Command{ e brain.Entry err error ) - if slice != "" { + open, _ := cmd.Flags().GetBool("open") + switch { + case open: + e, err = brain.NewQuestion(args[0], anchors, slice, time.Now()) + case slice != "": e, err = brain.NewRequirement(args[0], anchors, slice, time.Now()) - } else { + default: e, err = brain.NewEntry(args[0], anchors, time.Now()) } if err != nil { @@ -109,7 +113,11 @@ var brainCaptureCmd = &cobra.Command{ fmt.Printf("Already queued as %s — not added again.\n", e.ID) return } - fmt.Printf("Queued %s -> would promote into %s\n", e.ID, shardLabel(e.Shard())) + what := "Queued" + if e.Open { + what = "Queued open question" + } + fmt.Printf("%s %s -> would promote into %s\n", what, e.ID, shardLabel(e.Shard())) fmt.Println("Review with 'mxcli brain staged'; commit it with 'mxcli brain promote " + e.ID + "'.") }, } @@ -229,6 +237,39 @@ var brainShowCmd = &cobra.Command{ }, } +var brainResolveCmd = &cobra.Command{ + Use: "resolve ", + Short: "Answer an open question, turning it into a decision", + Long: `Answer an open question. + +The entry becomes an ordinary decision in place: same id, same position in the +file, with the question kept as the answer's context. It does not move and does +not get a new identity, because an answered question is the same piece of +knowledge as the question — anything referring to it still resolves. + +This is the step that stops questions accumulating. A question nobody answers +is reported by 'brain check' and by 'mxcli lint' until someone does.`, + Args: cobra.ExactArgs(2), + Run: func(cmd *cobra.Command, args []string) { + store := brain.NewStore(brainProjectDir(cmd)) + e, shard, err := store.Find(args[0]) + if err != nil { + brainFatal(err) + } + if shard == "" { + brainFatal(fmt.Errorf("no committed entry with id %s", args[0])) + } + resolved, err := e.Resolve(args[1], time.Now()) + if err != nil { + brainFatal(err) + } + if err := store.Replace(shard, resolved); err != nil { + brainFatal(err) + } + fmt.Printf("Resolved %s in %s\n", resolved.ID, shardLabel(shard)) + }, +} + var brainPlanCmd = &cobra.Command{ Use: "plan", Short: "The roadmap: each slice's requirements counted against the model", @@ -388,8 +429,18 @@ func printBrainReport(rep brain.Report) { fmt.Println() printBrainPlan(rep.Slices) } + for _, q := range rep.Open { + fmt.Printf("OPEN %s: %q (%s)\n", q.EntryID, q.Title, shardLabel(q.Shard)) + } fmt.Printf("\n%d entries, %d anchors, %d resolved, across %d shard(s).\n", rep.Entries, rep.Anchors, rep.ResolvedN, len(rep.Shards)) + if n := len(rep.Open); n > 0 { + noun := "questions" + if n == 1 { + noun = "question" + } + fmt.Printf("%d open %s — answer one with 'mxcli brain resolve \"\"'.\n", n, noun) + } if !rep.Failed() { fmt.Println("OK") } @@ -582,6 +633,8 @@ func init() { } brainCaptureCmd.Flags().StringSliceP("anchor", "a", nil, "Anchor into the model (@Module, @Module.Element, @Module.Entity.Attribute); repeatable") + brainCaptureCmd.Flags().Bool("open", false, + "Record this as an OPEN QUESTION — something not decided yet, so its anchors are not checked") brainCaptureCmd.Flags().StringP("slice", "s", "", "Record this as a requirement of the named slice (plan/.md) instead of a decision") brainPromoteCmd.Flags().String("to", "", @@ -590,7 +643,8 @@ func init() { brainCheckCmd.Flags().Bool("ci", false, "Machine-friendly output for CI") brainPlanCmd.Flags().StringP("project", "p", "", "Path to the .mpr file") + brainResolveCmd.Flags().StringP("project", "p", "", "Path to the .mpr file") brainCmd.AddCommand(brainInitCmd, brainCaptureCmd, brainStagedCmd, - brainPromoteCmd, brainDropCmd, brainShowCmd, brainCheckCmd, brainPlanCmd) + brainPromoteCmd, brainDropCmd, brainShowCmd, brainCheckCmd, brainPlanCmd, brainResolveCmd) rootCmd.AddCommand(brainCmd) } diff --git a/cmd/mxcli/cmd_lint.go b/cmd/mxcli/cmd_lint.go index c39241791..404e4de13 100644 --- a/cmd/mxcli/cmd_lint.go +++ b/cmd/mxcli/cmd_lint.go @@ -379,14 +379,33 @@ func reportBrainGap(projectDir string) { return } staged, err := brain.NewQueue(projectDir).Load() - if err != nil || len(staged) == 0 { + if err != nil { return } - noun := "entries" - if len(staged) == 1 { - noun = "entry" + // An unanswered question is the same shape of gap as an unpromoted entry: + // something a person has to act on, that no other command would mention. + open, err := brain.NewStore(projectDir).OpenQuestions() + if err != nil { + return + } + if len(staged) == 0 && len(open) == 0 { + return + } + var parts []string + if len(staged) > 0 { + parts = append(parts, fmt.Sprintf("%d staged %s not yet promoted ('mxcli brain staged')", + len(staged), plural(len(staged), "entry", "entries"))) + } + if len(open) > 0 { + parts = append(parts, fmt.Sprintf("%d open %s ('mxcli brain check')", + len(open), plural(len(open), "question", "questions"))) + } + fmt.Fprintf(os.Stderr, "\nProject brain: %s.\n", strings.Join(parts, "; ")) +} + +func plural(n int, one, many string) string { + if n == 1 { + return one } - fmt.Fprintf(os.Stderr, - "\nProject brain: %d staged %s not yet promoted — 'mxcli brain staged' to review.\n", - len(staged), noun) + return many } diff --git a/docs-site/src/tools/project-brain.md b/docs-site/src/tools/project-brain.md index 546fc6785..b3c3f603d 100644 --- a/docs-site/src/tools/project-brain.md +++ b/docs-site/src/tools/project-brain.md @@ -4,6 +4,8 @@ - **Decisions** — why a pattern was chosen here, which marketplace version broke what, what a recurring mxbuild error means in *this* app. +- **Open questions** — what is not decided yet, recorded so it is not silently + forgotten and rediscovered expensively later. - **The plan** — the requirements being built from, grouped into slices, when the source is a specification document, a prototype or a conversation. @@ -171,6 +173,51 @@ implement it. Misfiling is not checked for slices: a slice spans modules by design. +## Open questions + +An open question is a decision that has not been made yet. + +```bash +mxcli brain capture "Do approvers see rejected orders?" --open -a @Sales.Order -p app.mpr +``` + +Its **anchors are not checked**. A question may name something that does not +exist — often the question is precisely whether it should — so the staleness +rule that keeps decisions honest would report every question as a defect. +Measured: the identical anchor takes `brain check` to exit 1 as a decision and +exit 0 as a question. + +`--open` combines with `--slice`. A question about a slice's scope is filed with +that slice and counted apart from its requirements: an unanswered question is +not outstanding scope, so it never inflates the slice's numbers. + +Answering it converts it in place: + +```bash +mxcli brain resolve "Yes, for 30 days +Agreed with the product owner; drives the overview filter and the access rule." +``` + +The entry keeps its **id** and its position in the file, and the question +survives as the answer's context. From that moment its anchors are checked like +any other decision — which is the whole point of the transition, and is asserted +by a test. + +Both `brain check` and `mxcli lint` report unanswered questions until one is +resolved. A question nobody answers is the one kind of entry that gets more +expensive the longer it sits. + +## When to capture a decision + +Capture needs a trigger, not good intentions. The reliable one: + +> **You have had to correct the same thing twice.** + +The second correction is the signal — it will happen a third time, to someone +else. The other trigger is choosing between real alternatives where the losing +one would look reasonable to the next person: record the choice *and* what ruled +the other out, or it gets re-litigated. + ## Size ```bash @@ -209,6 +256,8 @@ stale the next time anyone promotes. | `brain promote [--to ]` | Writes it into its shard | | `brain drop ` | Removes it from the queue or from its shard | | `brain capture "" --slice [-a @Anchor]…` | Queues a **requirement** of that slice | +| `brain capture "" --open [-a @Anchor]…` | Queues an **open question**; anchors not checked | +| `brain resolve ""` | Answers it, turning it into a decision in place | | `brain plan` | Each slice's requirements counted against the model | | `brain check [--changed]` | Anchors resolve, entries filed correctly, plus slice progress | | `brain show []` | Entries, lines and headroom per shard | From 6b74924044307c509ac5b42b42bf9cf0a94ff359 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 4 Sep 2026 09:10:50 +0000 Subject: [PATCH 14/17] =?UTF-8?q?docs(proposal):=20=C2=A74.5=20=E2=80=94?= =?UTF-8?q?=20the=20two=20further=20record=20kinds,=20and=20what=20separat?= =?UTF-8?q?es=20them?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The proposal described one kind of record, a decision, while the store has three. Adds §4.5 with amendments A10 and A11. The separating property is not three separate arguments but one: what a FAILED anchor means. A decision's anchor points backward at what exists, so one that stops resolving means the decision is stale. A requirement's points forward, so one that does not resolve means not built yet. A question's points at what is under discussion, so it means nothing at all — the question is often whether the thing should exist. Identical syntax; only the direction differs, and that is the whole lifecycle. Records that this was measured before it was designed: as an ordinary entry, a single not-yet-built requirement takes `brain check` to exit 1, and so does a question. Neither could be more decisions without making the check useless. Also records where this departs from the AI-native SDLC playbook and where it was behind it. Departs: the playbook keeps a plan.md and recommends a hook enforcing that the diff still matches it — a self-reported artifact being policed, which is only necessary when there is no queryable model. Behind: the playbook's trigger for CLAUDE.md ("when Claude makes a mistake twice, the correction goes into CLAUDE.md") is the one thing this design was missing outright, and is now A11. And notes the defect the comparison exposed, because the lesson generalises: the proposal asserted project.md was loaded every session and justified the store's tightest cap on that basis, with nothing behind it — a generated project mentioned the brain once, in a skills-table row, behind a symptom-triggered description. A policy about what is loaded is worth nothing without the thing that loads it, and the same question should be asked of any future claim of that shape here. Co-Authored-By: Claude Opus 5 --- docs/11-proposals/PROPOSAL_project_brain.md | 81 ++++++++++++++++++++- 1 file changed, 78 insertions(+), 3 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_project_brain.md b/docs/11-proposals/PROPOSAL_project_brain.md index 14399f51b..16db0207a 100644 --- a/docs/11-proposals/PROPOSAL_project_brain.md +++ b/docs/11-proposals/PROPOSAL_project_brain.md @@ -412,13 +412,88 @@ first is the one that matters: 4. **Write the anchor, not the name.** `@Sales.Order.Status` is what makes an entry checkable and routable; the same fact written as prose is neither. +### 4.5 Two more record kinds, and the property that separates them + +The brief and §4.1–4.4 describe one kind of record: a **decision**. Two more are +needed, and the reason is a single property rather than three separate +arguments — **what a failed anchor means.** + +| Kind | Its anchor points | An anchor that does not resolve means | Checked? | +|---|---|---|---| +| decision | backward, at what exists | the decision is **stale** | yes — fails | +| requirement | forward, at what is intended | **not built yet** | counted, never fails | +| open question | at what is *under discussion* | nothing — the question is often whether it should exist | not at all | + +The syntax is identical in all three. Only the direction differs, and that is +the whole lifecycle. + +**This was measured before it was designed**, which is what settled it in one +command: recorded as an ordinary entry, a single not-yet-built requirement takes +`brain check` to exit 1. The same is true of a question. Requirements and +questions could not be more decisions without making the check useless. + +**Requirements** (`plan/.md`, `capture --slice`) exist because the source +of truth is frequently outside git — a specification document, a prototype, a +conversation. None of that is an issue or a commit message, so hours of work can +end with nothing recording what they were for, and a resumed session has no idea +what it was building towards. This is the gap the analogous store never had, +because mxcli's own work is driven by GitHub issues. + +The inversion then pays for the feature rather than merely accommodating it: a +requirement is **built** when its anchors resolve, so `brain plan` reports +progress *derived from the model*. Measured end to end — a slice at 0 built / +1 planned became 1 / 0 after creating the microflow its requirement named, with +the plan file untouched. There is no status column to maintain and none that can +be silently wrong, which is A6 applied to scope rather than to size. It is also +where this design departs from the AI-native SDLC playbook, which keeps a +`plan.md` and recommends a hook to enforce that the diff still matches it: that +is a self-reported artifact being policed, and it is only necessary when there is +no queryable model. There is one here. + +**Open questions** (`--open`, `brain resolve`) are decisions not yet made. They +live beside the decisions they will join rather than in a file of their own, +because the moment you need to see one is while reading what that module already +decided. Resolution converts the entry **in place**, keeping its id and position +— an answered question is the same piece of knowledge as the question — and from +then on its anchors are checked like any other decision. + +Two consequences worth stating, because both are the kind of thing that looks +like an oversight: + +- **A question filed against a slice is not scope.** It is counted apart from + the slice's requirements; counting an unanswered question as outstanding work + would overstate what is left to do. +- **Requirements and questions are never misfiled.** A slice spans modules by + design, and a question has no resolved anchor to compare a shard against. + +Two amendments follow, in the table's numbering: + +| # | Amendment | Because | +|---|---|---| +| A10 | Three record kinds, distinguished by anchor direction. Requirements live in `plan/.md`; questions live beside decisions and carry an `OPEN` marker | §4.5: measured — a requirement or question recorded as a decision takes `check` to exit 1 | +| A11 | Capture gets a **trigger**: a correction made twice, or a choice between real alternatives. `mxcli lint` reports unpromoted entries *and* unanswered questions | §4.5: the plan half fills at bootstrap while the decisions half fills only if someone remembers — A7 applied to the imbalance | + +A11's trigger is taken from the AI-native SDLC playbook's rule for `CLAUDE.md` +("when Claude makes a mistake twice, the correction goes into `CLAUDE.md`"), +which is the one piece of that document this design was missing outright. + +Its other contribution was to expose a defect: the playbook is explicit that +`CLAUDE.md` is read at the **start of a session**, and this proposal asserted the +same of `project.md` — using it to justify the tightest cap in the store — with +no mechanism behind it. A generated project mentioned the brain exactly once, in +a skills-table row, behind a symptom-triggered description. The generated +`CLAUDE.md` now names `docs/brain/project.md` directly. **A policy about what is +loaded is worth nothing without the thing that loads it**, and the same question +should be asked of any future claim of that shape here. + ## 5. Phasing Unchanged from the brief, with A4 inserted: -1. Storage, anchors, and the seven verbs of §4.3 — `init` / `capture` / - `staged` / `promote` / `drop` / `check` / `show` — plus the skill of §4.4. - Markdown destinations only. +1. Storage, anchors, and the verbs of §4.3 — `init` / `capture` / `staged` / + `promote` / `drop` / `check` / `show`, plus `plan` and `resolve` for the two + further record kinds of §4.5 — and the skill of §4.4. Markdown destinations + only. 2. The mxbuild error → resolution trigger. 3. **Documentation audit**, then promotion into model documentation and lint-rule generation. No longer gated on A4. From 3c94e096fd1b54da6a146d4b061e27c8ecae7fa3 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 4 Sep 2026 11:36:55 +0000 Subject: [PATCH 15/17] fix(executor): version-gate the four doctypes that need Mendix 11.9 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The nightly is red on 10.24 and 11.6.8 while 11.12, 11.13 and 11.14 are green. Four of the 29 #1018 documentation fixtures cover doctypes that do not exist below 11.9 — ai model, knowledge base, consumed mcp service, agent — and had no version gate, so on an older project they failed at CREATE: --- FAIL: TestDocumentation_SurvivesRewrite/ai_model create: create model requires Mendix 11.9.0+ (project is 11.6.8) That failure says nothing about documentation carry. The doctype is simply absent, which is what a gate is for. Reproduced locally rather than reasoned about: mxbuild 11.6.0 is already cached, and MX_BINARY pins it, giving the identical message and line in four seconds. Both controls run: 11.6.0 4 SKIP, 25 run and pass, both control tests pass 11.13.0 no skips — all four actually run and pass The second one is the one that matters. A gate that always skipped would turn the whole matrix green while testing nothing, and nothing else would notice. The minimum is carried as a version on the case rather than a boolean, so the reason is legible where the case is written and it mirrors the registry entries in sdk/versions/mendix-11.yaml (agent_model, agent_knowledge_base, agent_consumed_mcp_service, agent — all min_version 11.9.0). Every other doctype passes on 10.24 and 11.6.8 untouched, which is a useful incidental result: the documentation carry itself holds across all three supported majors. Co-Authored-By: Claude Opus 5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + mdl/executor/documentation_preserved_test.go | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 34eb9618f..3ab7bf78b 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -510,3 +510,4 @@ {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`return` inside a `loop` passes `mxcli check` AND `exec`, then mxbuild fails **CE0068** \"End events cannot be placed inside a loop.\" \u2014 even though MDL062 exists precisely to catch that. It fires only when the microflow header has NO `returns T as $Var` clause.", "cause": "MDL062 stood down for the whole microflow whenever the AS clause was present, on two claims: that buildFlowGraph synthesizes the End event from the variable so none lands in the loop, and that the shape builds CE0109 instead. `describe microflow` shows the in-loop `return` written either way, so the first was never true. The second was a measurement artefact: mxbuild reports ONE error per microflow, and in the shape that was measured the AS variable was never assigned, so CE0109 \"Undefined variable\" won the race and hid CE0068 underneath. Adding `declare $Done Boolean = false` and changing nothing else turns the same microflow from CE0109 into CE0068. The exemption was deleted, not narrowed.", "file": "`mdl/executor/validate_microflow_ce_gaps.go` (checkReturnInLoop \u2014 the `v.returnType.Variable != \"\"` early return); test inverted in `validate_microflow_ce_gaps_test.go` (TestMDL062_ExemptsReturnsAsClause -> TestMDL062_FiresWithReturnsAsClause); examples `mdl-examples/bug-tests/captrack-19-return-in-loop-as-clause{,.fail}.mdl`", "insight": "A second error in the same document can HIDE the one you are measuring, because mxbuild reports one error per microflow. An exemption justified by \"measured: builds X instead\" is only sound if the reproduction was otherwise valid \u2014 here the repro was broken in a second way, and the error that surfaced was the one nobody was asking about. When a measurement says a construct is clean, add the minimum that removes every OTHER error from that document and measure again; the differential (CE0109 -> CE0068 on one added `declare`) is what settles it. Reported as CapTrackV2 FINDINGS \u00a719."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`dataview dv (\u2026, OnClick: SHOW_PAGE \u2026)` parses, `mxcli check` is clean, `exec` writes the page without a word \u2014 and the rendered element has no handler and no role=\"button\". The same silence on `dynamictext`, `listview` and every other widget except a container or a button.", "cause": "`OnClick:` is an ALIAS for `Action:` (both stored as Properties[\"Action\"], #603), and mxcli writes that property for three widget kinds only: container/customcontainer, the buttons, and a navigationlist item. Every other widget drops it, and the property allow-lists behind MDL-WIDGET01/07 could not see that because they are widget-type AGNOSTIC \u2014 the same blind spot as #928's `editable:`. Added MDL-WIDGET23 (warning), with two messages: Mendix models no click action at all (dataview, dynamictext, inputs, groupbox\u2026) vs Mendix models one that mxcli cannot write (listview, staticimage, dynamicimage \u2014 measured against generated/metamodel).", "file": "`mdl/executor/validate_widget_onclick.go` (new), wired in `validate_widgets.go` beside validateWidgetEditability; `.claude/skills/mendix/create-page/reference/widgets.md`; example `mdl-examples/bug-tests/captrack-21-dataview-onclick-dropped.mdl`", "insight": "The first draft reported everything OUTSIDE an allow-list of the three writers, and running it over the shipped examples flagged three of them. Cause: `mxcli check` without `-p` has no widget registry, so `lookupWidgetDef` returns nil for a PLUGGABLE widget too and the caller's \"static widgets only\" branch silently does not hold \u2014 `datagrid` is DataGrid 2, a pluggable widget whose onClick the engine does write. For any rule keyed on widget type, an allow-list makes the unknown case an ERROR and a deny-list makes it silence; pick the deny-list, because a missed warning costs nothing and a false one tells an author their working page is broken. Running a new rule across mdl-examples/ before wiring it up is what caught it \u2014 the same exercise #893 describes. Reported as CapTrackV2 FINDINGS \u00a721."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ALTER SETTINGS MODEL AfterStartupMicroflow = 'Mod.MF_Seed'` accepts a microflow with no return type, `mxcli check` passes, and the build fails **CE0142** \"After startup microflow should return a boolean\". `ALTER MICROFLOW \u2026 RETURNS \u2026` does not parse either, so the remedy is DROP + CREATE.", "cause": "#274 made ALTER SETTINGS resolve the qualified names it writes, which catches a MISSPELLED microflow. Here the name resolves perfectly \u2014 the constraint is on the thing the setting names, not on the reference, and nothing looked at the return type. Added MDL073: a project-less pass (ValidateAfterStartupReturnType) for a microflow the script itself creates, which is the usual shape, plus the stored return type on the project path. Both call one function so they cannot drift. flowSignature gained ReturnKind because its existing Returns field is the entity name and cannot tell Boolean from void.", "file": "`mdl/executor/validate_settings_refs.go` (checkAfterStartupReturnsBoolean + ValidateAfterStartupReturnType), `validate_program.go`, `helpers.go` (buildMicroflowReturnTypes), `validate_datasource_args.go` (flowSignature.ReturnKind); examples `mdl-examples/bug-tests/captrack-6-after-startup-must-return-boolean{,.fail}.mdl`", "insight": "\"The reference resolves\" and \"the reference is usable\" are different questions, and a resolver answers only the first. Whenever a setting stores a NAME, ask what the platform requires of the named thing \u2014 an existence check will pass and the build will still fail, and the error arrives with no connection to the statement that caused it. The scope discipline that goes with it: only AfterStartup is type-checked, because only its rule was measured; BeforeShutdown and HealthCheck are left alone rather than constrained on a guess, and there is a control test asserting that. Reported as CapTrackV2 FINDINGS \u00a76."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "Nightly integration tests fail on older Mendix versions only (10.24 and 11.6.8 red, 11.12/11.13/11.14 green): `--- FAIL: TestDocumentation_SurvivesRewrite/ai_model … create: create model requires Mendix 11.9.0+`", "cause": "The #1018 documentation-carry fixture table covers all 29 rewrite-capable doctypes, four of which (ai model, knowledge base, consumed mcp service, agent) do not exist below Mendix 11.9. The cases had no version gate, so on an older project they failed at CREATE — which says nothing about documentation carry, because the doctype is simply absent", "file": "`mdl/executor/documentation_preserved_test.go` (docPreserveCase.minMajor/minMinor + requireMinVersion in the runner)", "insight": "A doctype fixture table is a version-compatibility surface, not just coverage: adding a row for a gated doctype silently commits you to every version in the nightly matrix. Develop-version bias is what hides it — written against 11.13, all 29 pass, and only the matrix disagrees. Reproduce locally instead of reasoning: `MX_BINARY=~/.mxcli/mxbuild/11.6.0/modeler/mx go test -tags integration` gives the identical failure in seconds, and the cached mxbuild versions under ~/.mxcli/mxbuild/ are usually enough to cover the matrix. Two controls are needed, not one — that the gate SKIPS below the minimum, and that it is INERT above it. A gate that always skips turns the matrix green while testing nothing, which is the worse failure and the one nobody notices", "refs": ["ako/mxcli#1018", "mendixlabs/mxcli actions run 33846778362"], "ce": []} diff --git a/mdl/executor/documentation_preserved_test.go b/mdl/executor/documentation_preserved_test.go index da5fab8db..d519ed5c6 100644 --- a/mdl/executor/documentation_preserved_test.go +++ b/mdl/executor/documentation_preserved_test.go @@ -39,6 +39,12 @@ type docPreserveCase struct { // anything else modelsdk-only). The harness defaults to legacy, so without // this the case fails at its own precondition and says nothing about #1018. modelsdk bool + // minMajor/minMinor gate a doctype that does not exist in every supported + // Mendix version. Without this the case fails at CREATE on an older + // project, which says nothing about #1018 — the doctype is simply absent. + // Kept as a version rather than a boolean so the reason is legible at the + // case, and mirrors sdk/versions/mendix-11.yaml. + minMajor, minMinor int // create carries a doc comment; rewrite deliberately does not. create string rewrite string @@ -231,24 +237,32 @@ func docPreserveCases() []docPreserveCase { }, { name: "ai model", + minMajor: 11, + minMinor: 9, storedOnly: true, create: doc + "create model TestModule.DocModel ( Provider: MxCloudGenAI );", rewrite: "create or modify model TestModule.DocModel ( Provider: MxCloudGenAI );", }, { name: "knowledge base", + minMajor: 11, + minMinor: 9, storedOnly: true, create: doc + "create knowledge base TestModule.DocKb ( Provider: MxCloudGenAI );", rewrite: "create or modify knowledge base TestModule.DocKb ( Provider: MxCloudGenAI );", }, { name: "consumed mcp service", + minMajor: 11, + minMinor: 9, storedOnly: true, create: doc + "create consumed mcp service TestModule.DocMcp ( ProtocolVersion: 'v2025_03_26' );", rewrite: "create or modify consumed mcp service TestModule.DocMcp ( ProtocolVersion: 'v2025_03_26' );", }, { name: "agent", + minMajor: 11, + minMinor: 9, storedOnly: true, create: "create model TestModule.DocAgentModel ( Provider: MxCloudGenAI );\n" + doc + "create agent TestModule.DocAgent ( UsageType: Task, Model: TestModule.DocAgentModel, SystemPrompt: 'p' );", @@ -273,6 +287,10 @@ func TestDocumentation_SurvivesRewrite(t *testing.T) { } defer env.teardown() + if tc.minMajor > 0 { + env.requireMinVersion(t, tc.minMajor, tc.minMinor) + } + if err := env.executeMDL(tc.create); err != nil { t.Fatalf("create: %v", err) } From 54c4a314be716d587e067b52bea3c68de66783a5 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 12:33:11 +0000 Subject: [PATCH 16/17] feat(domain model): lay entities out instead of stacking them in one row MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A generated domain model opened in Studio Pro as a single horizontal line of entities with the boxes touching — unreadable at any zoom. Reported against a 40-entity model (ako/CapTrackV2, Mendix 11.13). The default position for a CREATE ENTITY with no @Position was: location = model.Point{X: 100 + len(dm.Entities)*150, Y: 100} Same y for every entity ever created. 40 entities is a 6,950px row, and 150px is narrower than an entity box, so they overlapped as well. Two changes, because there are two different questions. **The default** (mdl/dmlayout.GridSlot) is now a wrapping grid. It cannot be better than that: the first entity of a script is placed before the last one exists, so no create-time rule can see the association graph. Slot n stays a function of n alone, so adding an entity never moves one already written. Those same 40 entities now span ~1,400px. **mxcli layout** does the real thing, from the whole model once it exists. Entities are layered on the association graph — an entity referencing nothing else in the module is a lookup and goes left, everything else one column past the furthest thing it references — so association lines mostly run one way instead of crossing the diagram. Entities with no association at all go in a band underneath rather than among the lookups, which is where the non-persistent helpers belong. Run against CapTrack's own 01-domain.mdl, the layering falls out of the model: six lookups in one column, then Team/GoalBucket, then PlanScope's band, out to EmployeeMonth/Movement. mx check 0 errors, and the positions round-trip into MDL as @Position. It is opt-in and it overwrites hand-arranged positions in the modules it touches, so --dry-run lists the moves first. Marketplace modules and System are skipped, and NAMING one is an error rather than a silent skip — a silent skip reports success having done nothing. Three properties the tests pin, each measured rather than assumed: - idempotent: a second run detects nothing to move and never calls the writer (16 of 16 moved, then "already laid out"). - local: adding one entity with one association moved 3 of 17, not the model. A layout that reshuffled everything would make each domain-model commit an unreadable diff. - deterministic: an unsorted walk gives a different diagram every run, which rewrites the unit every time — the churn ADR-0008 exists to prevent. The control fails on run 0. Two constraints that shaped the geometry: an entity stores only Location and NO Size (Studio Pro derives the box when it draws), so spacing is estimated from name length and attribute count; and a Mendix position is the box's CENTRE, so placement adds half a box rather than none. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + CLAUDE.md | 1 + cmd/mxcli/cmd_layout.go | 211 ++++++++++++ cmd/mxcli/cmd_layout_test.go | 134 ++++++++ mdl/dmlayout/dmlayout.go | 322 ++++++++++++++++++ mdl/dmlayout/dmlayout_test.go | 294 ++++++++++++++++ mdl/executor/cmd_entities.go | 11 +- 7 files changed, 972 insertions(+), 2 deletions(-) create mode 100644 cmd/mxcli/cmd_layout.go create mode 100644 cmd/mxcli/cmd_layout_test.go create mode 100644 mdl/dmlayout/dmlayout.go create mode 100644 mdl/dmlayout/dmlayout_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 6c5d2ad1f..c4df56080 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -513,3 +513,4 @@ {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ACTIONBUTTON \u2026 (Action: SIGN_OUT)` is refused by the default engine \u2014 \"client action *pages.SignOutClientAction not yet supported by the modelsdk engine \u2014 rerun with MXCLI_ENGINE=legacy\" \u2014 and the suggested workaround SILENTLY produces a dead button: on legacy the action is written as `Forms$NoAction`, so it renders, says \"Sign out\", and does nothing, with `mxcli check`, `exec` and `mx check` all clean.", "cause": "Neither engine had a case for the action. modelsdk's clientActionToGen ended in a loud default; sdk/mpr's serializeClientAction ended in a QUIET one that returns Forms$NoAction for anything unmatched. Added the case to both. The document is two keys \u2014 `Forms$SignOutClientAction` + `DisabledDuringExecution: true` \u2014 pinned against a Studio Pro-authored button in ako/TestApp, plus `sign_out` in the DESCRIBE renderer so it round-trips. Both engines now emit byte-identical documents; mx check 0 errors on each. OPEN_LINK is still unwritten by both (gen calls it OpenLinkClientAction and its Address is an element, not a string) \u2014 the syntax topic now says so instead of listing it as available.", "file": "`mdl/backend/modelsdk/widget_write.go` (clientActionToGen), `sdk/mpr/writer_widgets_action.go` (serializeClientAction), `mdl/executor/cmd_pages_describe_output.go` (renderClientActionMDL), `cmd/mxcli/syntax/features_page.go`; example `mdl-examples/bug-tests/captrack-10-sign-out-action.mdl`", "insight": "When one engine refuses something and points at the other, CHECK THE OTHER before repeating the advice \u2014 the refusal is visible and the fallback is not, so the recommended escape hatch can be the strictly worse path. The structural tell is the shape of the default branch: modelsdk's raises, legacy's returns Forms$NoAction, and a silent default in a serializer converts every unimplemented type into data loss rather than an error. Grep for the fallthrough before trusting a switch. Note the control this needs: a test that SIGN_OUT is no longer NoAction can pass because someone softened the default, so pin the fallback separately with a type that is still unimplemented (OPEN_LINK). Reported as CapTrackV2 FINDINGS \u00a710."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "A navigation menu's LOG-OUT item could not be authored and did not survive a round trip. MDL's `menu item` took PAGE or MICROFLOW only, so there was no spelling for it; and ako/TestApp's sign-out menu item read back as a plain `menu item 'Item 5';`, so DESCRIBE -> exec turned a working log-out entry into a dead one \u2014 silently, with `mx check` clean.", "cause": "A menu item's action goes through FOUR places that share no code with the button path: menuActionToGen (menu document, modelsdk), navMenuAction (navigation profile, raw BSON), resolveMenuAction (modelsdk read) and parseNavMenuItem (legacy read). Both writers ended in a NoAction default and both readers left the type name unmapped. Added SIGN_OUT to navMenuItemDef in the grammar (it consumes no qualifiedName, so it is read separately from the PAGE/MICROFLOW switch or an ICON after it is mis-assigned), carried it as ActionType \"SignOutAction\" / NavMenuItemSpec.SignOut, and wired all four. Studio Pro stores the same Forms$SignOutClientAction a button carries: DisabledDuringExecution true, nothing else.", "file": "`mdl/grammar/MDLParser.g4` (navMenuItemDef), `mdl/ast/ast_navigation.go`, `mdl/visitor/visitor_navigation.go`, `mdl/executor/cmd_menus.go` + `cmd_navigation.go` (conversion + printMenuMDL + the show summary), `mdl/types/navigation.go`, `mdl/backend/modelsdk/menu_write.go` + `navigation_write.go` + `navigation_read.go`, `sdk/mpr/parser_misc.go`; example `mdl-examples/bug-tests/captrack-10-sign-out-menu-item.mdl`", "insight": "A round trip closes only if the READER produces the exact string the WRITER consumes \u2014 here both readers had a raw-type-name fallback that looked like it preserved information (ActionType became \"Forms$SignOutClientAction\") while breaking the round trip, because DESCRIBE and the writers key on \"SignOutAction\". A fallback that stores the raw name is not the same as handling the case, and it hides the gap better than a NoAction default would. Also: the same logical action reaches storage through four unrelated switches (two writers x two constructs, two readers), so fixing the button path proved nothing about the menu path \u2014 grep for every switch on the action before calling such a fix complete. Controlled by neutralising both readers and re-reading TestApp: `Item 5 -> sign out` goes back to `Item 5`."} {"area": "mdl/executor", "date": "2026-09-04", "symptom": "`ACTIONBUTTON \u2026 (Action: OPEN_LINK 'https://\u2026')` was written by neither engine: modelsdk refused it, legacy fell through to its quiet default and wrote Forms$NoAction, so the button rendered and did nothing with check, exec and mx check all clean.", "cause": "Same missing-case defect as SIGN_OUT, but with two traps a reference settled and reasoning would not. (1) The STORAGE NAME is Forms$OpenLinkClientAction, while the semantic type is LinkClientAction and the executor stamped `Forms$LinkClientAction` \u2014 a wrong $Type that never reached disk only because nothing could write the action. (2) The address is not a string field but a nested Forms$StaticOrDynamicString. Pinned against 31 Studio Pro link buttons (ako/TestApp, FeedbackModule): exactly five keys, LinkType \"Web\" in all 31, and 6 of 31 DYNAMIC (IsDynamic true + AttributeRef + empty Value). MDL authors the static form only, so DESCRIBE flags a dynamic one instead of printing its address as a literal.", "file": "`mdl/backend/modelsdk/widget_write.go` (clientActionToGen + staticAddressToGen), `sdk/mpr/writer_widgets_action.go`, `mdl/executor/cmd_pages_builder_v3.go` ($Type), `mdl/executor/cmd_pages_describe_output.go`, `cmd/mxcli/syntax/features_page.go`; example `mdl-examples/bug-tests/captrack-10-open-link-action.mdl`", "insight": "gen declares a fourth property on Forms$StaticOrDynamicString \u2014 `Attribute` \u2014 that not one of the 31 stored documents carries. Writing it would be the 'never invent a key' failure: a document mxbuild accepts and Studio Pro cannot open. When gen offers more properties than the references show, the references win. Second lesson, about controls: the SIGN_OUT commit used LinkClientAction as its 'still unimplemented' control, and implementing OPEN_LINK silently invalidated it \u2014 the test then failed for a good reason, but a control naming a specific unimplemented feature has a shelf life. Point it at something structurally unwritable instead (ShowHomePageClientAction: no gen type, no metamodel counterpart, no MDL statement that builds one)."} +{"area": "mdl/executor", "date": "2026-09-04", "symptom": "A generated domain model opens in Studio Pro as ONE horizontal line of entities, boxes touching, unreadable at any zoom. Reported on a 40-entity model (ako/CapTrackV2, Mendix 11.13).", "cause": "The default position for a CREATE ENTITY with no `@Position` was `model.Point{X: 100 + len(dm.Entities)*150, Y: 100}` \u2014 same y for every entity ever created, x stepping by 150. 40 entities = a 6,950px row; and 150px is narrower than an entity box, so they also overlapped. Replaced with a wrapping grid in the new `mdl/dmlayout` package, and added `mxcli layout` for a real layered layout off the association graph.", "file": "`mdl/executor/cmd_entities.go` (the default), `mdl/dmlayout/dmlayout.go` (new: GridSlot + Plan), `cmd/mxcli/cmd_layout.go` (new command)", "insight": "The default could not have been much better than a grid, and that is the design point: the first entity of a script is placed before the last one exists, so no create-time rule can see the graph. Layout needs the whole model, so it belongs in a separate pass, not as a side effect of authoring \u2014 and because it necessarily overwrites hand-arranged positions it has to be opt-in with a dry run. Two constraints that are easy to miss: an entity stores only Location and NO Size (Studio Pro derives the box when it draws), so spacing must be estimated from name length and attribute count; and a Mendix position is the box's CENTRE, not its top-left, so placement adds half a box. Determinism is load-bearing rather than cosmetic \u2014 an unsorted walk gives a different diagram every run, which rewrites the unit every time and is exactly the churn ADR-0008 exists to prevent (the test catches it on run 0)."} diff --git a/CLAUDE.md b/CLAUDE.md index af2151e10..c4e4c14b3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -716,6 +716,7 @@ go build -o bin/mxcli ./cmd/mxcli | **Connector gen** | `sql generate connector into [tables (...)] [views (...)] [exec]` | Auto-generate Database Connector MDL from discovered schema | | **Marketplace drift** | `mxcli marketplace diff -p app.mpr [--to V] [--json]` | Which elements of an installed marketplace module have been edited locally, and what an upgrade would overwrite | | **Model repair** | `mxcli fix widgets`, `mxcli fix design-properties` | Runs `mx update-widgets` / `mx rename-design-properties` and **persists** the result without their MPR v2 → v1 collapse (harvest: let the tool convert, read the units back, restore v2, write the changed ones through mxcli's writer). Clears CE0463 / CE6087 after a headless install — measured 203 → 0 errors on a vanilla 11.12.1 app | +| **Domain-model layout** | `mxcli layout -p app.mpr [--module M] [--dry-run]` | Arranges entities from the **association graph**: an entity referencing nothing is a lookup and goes left, everything else one column past the furthest thing it references, so lines run one way instead of crossing. Unconnected entities (non-persistent helpers) go in a band below rather than among the lookups. Positions are a function of the model, so a second run moves nothing. Replaces hand-arranged positions in the modules it touches — hence opt-in, with `--dry-run`; Marketplace modules and System are skipped. The **default** for an entity with no `@Position` is a wrapping grid (`mdl/dmlayout`), not the single 6,000px row it used to be | | **Diagnostics** | `mxcli diag [--bundle]` | Session logs, version info, bug report bundles | | **Project brain** | `mxcli brain init\|capture\|staged\|promote\|drop\|check\|show\|plan\|resolve` | Opt-in store in `docs/brain/` for what mxcli **cannot** compute (why a pattern was chosen here, which marketplace version broke what). Sharded by module — an entry's first anchor names its file — so a session loads `project.md` plus the modules it is touching, not the whole store. Also holds the **plan**: requirements grouped into slices, whose anchors point *forward*, so `brain plan` reports progress **derived from the model** rather than from a status column. An agent captures to a git-ignored queue; a person promotes | | **New project** | `mxcli new --version X.Y.Z [--output-dir dir] [--theme none] [--layout none]` | Downloads mxbuild, creates blank project, applies default styling, scaffolds a project-owned layout, runs init, installs Linux mxcli for devcontainer | diff --git a/cmd/mxcli/cmd_layout.go b/cmd/mxcli/cmd_layout.go new file mode 100644 index 000000000..979532681 --- /dev/null +++ b/cmd/mxcli/cmd_layout.go @@ -0,0 +1,211 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "fmt" + "os" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/dmlayout" + "github.com/mendixlabs/mxcli/model" + "github.com/spf13/cobra" +) + +// cmd_layout.go arranges a module's domain model from its association graph. +// +// Positions are presentation, and MDL treats them that way: `@Position` exists +// but almost no script writes one, so entities take the default slot. The +// default is deliberately dumb — it cannot be anything else, because the first +// entity of a script is placed before the last one is known. Laying the model +// out properly needs the whole graph, which only exists once the script has run, +// so it is a separate operation rather than a side effect of CREATE. +// +// It is opt-in for a second reason: a domain model somebody arranged by hand in +// Studio Pro is not improved by being rearranged. Nothing here runs unless +// asked, --dry-run shows the moves first, and Marketplace modules are skipped +// outright — mxcli does not rearrange a module the next update replaces. + +var ( + layoutModules []string + layoutDryRun bool + layoutIncludeMarketplace bool +) + +var layoutCmd = &cobra.Command{ + Use: "layout", + Short: "Arrange a module's domain model from its association graph", + Long: `Arrange the entities of a domain model so related ones sit together. + +Entities are layered on the association graph: an entity that references nothing +else in the module is a lookup and goes on the left, and everything else sits +one column past the furthest thing it references. Association lines then mostly +run one way instead of crossing the diagram. Entities with no association at all +— non-persistent helpers, mostly — go in a band underneath rather than being +mixed in with the lookups. + +Positions are a function of the model alone, so running this twice in a row +changes nothing the second time and re-running after adding an entity moves only +what the new relationships require. + +This REPLACES the positions of every entity in the modules it touches, including +any you arranged by hand. Use --dry-run to see the moves first. Marketplace +modules and System are never touched.`, + Example: ` mxcli layout -p app.mpr + mxcli layout -p app.mpr --module CapTrack + mxcli layout -p app.mpr --dry-run`, + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + return runLayout(cmd) + }, +} + +func init() { + layoutCmd.Flags().StringSliceVar(&layoutModules, "module", nil, + "module to lay out (repeatable; default: every module the project owns)") + layoutCmd.Flags().BoolVar(&layoutDryRun, "dry-run", false, + "report the moves without writing") + layoutCmd.Flags().BoolVar(&layoutIncludeMarketplace, "include-marketplace", false, + "also lay out Marketplace modules (a module update replaces them, so this is normally pointless)") + rootCmd.AddCommand(layoutCmd) +} + +func runLayout(cmd *cobra.Command) error { + projectPath, _ := cmd.Flags().GetString("project") + if projectPath == "" { + return fmt.Errorf("no project given: pass -p ") + } + if _, err := os.Stat(projectPath); err != nil { + return fmt.Errorf("project not found: %s", projectPath) + } + + b := newBackendFactory()() + if err := b.Connect(projectPath); err != nil { + return err + } + defer func() { _ = b.Disconnect() }() + + modules, err := b.ListModules() + if err != nil { + return err + } + + // Keyed lower-cased because Mendix resolves module names case-insensitively, + // but the value keeps the spelling the author used so a "not found" quotes + // their typo back rather than a normalised version of it. + wanted := map[string]string{} + for _, m := range layoutModules { + if t := strings.TrimSpace(m); t != "" { + wanted[strings.ToLower(t)] = t + } + } + + targets, err := layoutTargets(modules, wanted) + if err != nil { + return err + } + + movedTotal, unchangedTotal := 0, 0 + for _, m := range targets { + dm, err := b.GetDomainModel(m.ID) + if err != nil { + return fmt.Errorf("read domain model of %s: %w", m.Name, err) + } + if dm == nil || len(dm.Entities) == 0 { + continue + } + + plan := dmlayout.Plan(dm) + moved := 0 + var moves []string + for _, e := range dm.Entities { + p, ok := plan[e.ID] + if !ok || p == e.Location { + continue + } + moves = append(moves, fmt.Sprintf(" %s.%s (%d, %d) -> (%d, %d)", + m.Name, e.Name, e.Location.X, e.Location.Y, p.X, p.Y)) + if !layoutDryRun { + e.Location = p + } + moved++ + } + sort.Strings(moves) + + if moved == 0 { + unchangedTotal += len(dm.Entities) + fmt.Fprintf(cmd.OutOrStdout(), "%s: already laid out (%d entities)\n", m.Name, len(dm.Entities)) + continue + } + for _, line := range moves { + fmt.Fprintln(cmd.OutOrStdout(), line) + } + if layoutDryRun { + fmt.Fprintf(cmd.OutOrStdout(), "%s: %d of %d entities would move\n", m.Name, moved, len(dm.Entities)) + } else { + if err := b.UpdateDomainModel(dm); err != nil { + return fmt.Errorf("write domain model of %s: %w", m.Name, err) + } + fmt.Fprintf(cmd.OutOrStdout(), "%s: moved %d of %d entities\n", m.Name, moved, len(dm.Entities)) + } + movedTotal += moved + } + + switch { + case movedTotal == 0 && unchangedTotal == 0: + fmt.Fprintln(cmd.OutOrStdout(), "Nothing to lay out.") + case layoutDryRun: + fmt.Fprintf(cmd.OutOrStdout(), "Dry run: %d entities would move. Re-run without --dry-run to apply.\n", movedTotal) + } + return nil +} + +// layoutTargets picks the modules to touch. +// +// A Marketplace module is excluded by default for the same reason CREATE LAYOUT +// refuses to write into one: an update replaces the module wholesale, so any +// arrangement is thrown away. System is excluded because it is not the user's to +// arrange. A --module the project does not have is an error rather than a silent +// no-op — a typo there would otherwise report success having done nothing. +func layoutTargets(modules []*model.Module, wanted map[string]string) ([]*model.Module, error) { + var out []*model.Module + matched := map[string]bool{} + for _, m := range modules { + if m == nil { + continue + } + if _, ok := wanted[strings.ToLower(m.Name)]; len(wanted) > 0 && !ok { + continue + } + matched[strings.ToLower(m.Name)] = true + if m.Name == "System" { + if len(wanted) > 0 { + return nil, fmt.Errorf("the System module cannot be laid out") + } + continue + } + fromMarketplace := m.FromAppStore || strings.TrimSpace(m.AppStoreGuid) != "" + if fromMarketplace && !layoutIncludeMarketplace { + if len(wanted) > 0 { + return nil, fmt.Errorf("%s comes from the Marketplace, and a module update replaces it — "+ + "pass --include-marketplace to lay it out anyway", m.Name) + } + continue + } + out = append(out, m) + } + + var missing []string + for key, spelled := range wanted { + if !matched[key] { + missing = append(missing, spelled) + } + } + if len(missing) > 0 { + sort.Strings(missing) + return nil, fmt.Errorf("module not found: %s", strings.Join(missing, ", ")) + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return out, nil +} diff --git a/cmd/mxcli/cmd_layout_test.go b/cmd/mxcli/cmd_layout_test.go new file mode 100644 index 000000000..c05c36abe --- /dev/null +++ b/cmd/mxcli/cmd_layout_test.go @@ -0,0 +1,134 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/model" +) + +func mod(name string, marketplace bool) *model.Module { + m := &model.Module{Name: name, FromAppStore: marketplace} + m.ID = model.ID("id-" + name) + if marketplace { + m.AppStoreGuid = "guid-" + name + } + return m +} + +func layoutFixture() []*model.Module { + return []*model.Module{ + mod("System", false), + mod("Administration", true), + mod("Atlas_Core", true), + mod("CapTrack", false), + mod("MyFirstModule", false), + } +} + +// With no --module, only the modules the project owns are touched. Rearranging +// a Marketplace module is work the next update throws away, and System is not +// the user's to arrange. +func TestLayoutTargets_SkipsMarketplaceAndSystemByDefault(t *testing.T) { + defer func(prev bool) { layoutIncludeMarketplace = prev }(layoutIncludeMarketplace) + layoutIncludeMarketplace = false + + got, err := layoutTargets(layoutFixture(), nil) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + var names []string + for _, m := range got { + names = append(names, m.Name) + } + want := "CapTrack,MyFirstModule" + if strings.Join(names, ",") != want { + t.Errorf("targets = %v, want %s", names, want) + } +} + +// Naming a Marketplace module explicitly is a mistake worth reporting, not +// something to silently drop — a silent skip reports success having done +// nothing, which is the failure mode this whole session kept finding. +func TestLayoutTargets_NamedMarketplaceModuleIsRefused(t *testing.T) { + defer func(prev bool) { layoutIncludeMarketplace = prev }(layoutIncludeMarketplace) + layoutIncludeMarketplace = false + + _, err := layoutTargets(layoutFixture(), map[string]string{"administration": "Administration"}) + if err == nil { + t.Fatal("naming a Marketplace module was accepted") + } + if !strings.Contains(err.Error(), "--include-marketplace") { + t.Errorf("the error should name the escape hatch: %v", err) + } +} + +// ...and the escape hatch works. +func TestLayoutTargets_IncludeMarketplaceOptsIn(t *testing.T) { + defer func(prev bool) { layoutIncludeMarketplace = prev }(layoutIncludeMarketplace) + layoutIncludeMarketplace = true + + got, err := layoutTargets(layoutFixture(), map[string]string{"administration": "Administration"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(got) != 1 || got[0].Name != "Administration" { + t.Errorf("targets = %v, want [Administration]", got) + } +} + +// A typo must fail loudly, quoting what was typed rather than a normalised form. +func TestLayoutTargets_UnknownModuleIsAnError(t *testing.T) { + _, err := layoutTargets(layoutFixture(), map[string]string{"nope": "Nope"}) + if err == nil { + t.Fatal("an unknown module was accepted") + } + if !strings.Contains(err.Error(), "Nope") { + t.Errorf("the error should quote the name as typed, got: %v", err) + } +} + +// Module names resolve case-insensitively in Mendix, so --module captrack has to +// find CapTrack. +func TestLayoutTargets_MatchesCaseInsensitively(t *testing.T) { + got, err := layoutTargets(layoutFixture(), map[string]string{"captrack": "captrack"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(got) != 1 || got[0].Name != "CapTrack" { + t.Errorf("targets = %v, want [CapTrack]", got) + } +} + +// System is refused by name for the same reason the Marketplace one is: a silent +// skip would look like success. +func TestLayoutTargets_SystemIsRefusedByName(t *testing.T) { + _, err := layoutTargets(layoutFixture(), map[string]string{"system": "System"}) + if err == nil || !strings.Contains(err.Error(), "System") { + t.Errorf("naming System should be refused, got: %v", err) + } +} + +// The order the modules are processed in must not depend on the order the +// backend happened to list them — the output is a report someone reads. +func TestLayoutTargets_IsSorted(t *testing.T) { + shuffled := []*model.Module{ + mod("MyFirstModule", false), + mod("CapTrack", false), + mod("Zeta", false), + mod("Alpha", false), + } + got, err := layoutTargets(shuffled, nil) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + var names []string + for _, m := range got { + names = append(names, m.Name) + } + if strings.Join(names, ",") != "Alpha,CapTrack,MyFirstModule,Zeta" { + t.Errorf("targets = %v, want them sorted", names) + } +} diff --git a/mdl/dmlayout/dmlayout.go b/mdl/dmlayout/dmlayout.go new file mode 100644 index 000000000..c3077b67e --- /dev/null +++ b/mdl/dmlayout/dmlayout.go @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package dmlayout decides where entities sit in the domain-model editor. +// +// It has two entry points, for the two moments a position is needed. +// +// GridSlot answers "an entity is being created and the script said nothing +// about where" — a wrapping grid, no knowledge of the model. Plan answers "lay +// this module out properly", using the association graph so related entities end +// up near each other. +// +// # Why a package rather than a formula at the call site +// +// The default used to be one line in the CREATE ENTITY handler: +// +// location = model.Point{X: 100 + len(dm.Entities)*150, Y: 100} +// +// Every entity on one row at y=100. A 40-entity model is a 6,000px line, and +// 150px is narrower than an entity box, so the boxes touch. Both halves of the +// fix want the same notion of "how big is a box and how far apart do they go", +// so it lives here once. +// +// # What is NOT known +// +// An entity stores only its Location. There is no Size in the model — Studio Pro +// derives the box from the entity's name and member list when it draws, and +// mxcli never sees the result. So every dimension below is an ESTIMATE from the +// model, deliberately generous: too much space is a diagram that scrolls, too +// little is the overlap this package exists to remove. +// +// A Mendix position is the box's CENTRE (RelativeMiddlePoint), not its top-left +// corner, which is why the placement code adds half a box rather than none. +package dmlayout + +import ( + "sort" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// Grid geometry for auto-placed entities. +// +// GridColumns is fixed rather than derived from the entity count: the count is +// not known when the first entity of a script is created, and a column count +// that grew mid-script would move entities that were already placed. +const ( + OriginX = 100 + OriginY = 100 + + GridColumns = 6 + GridColumnPitch = 260 + GridRowPitch = 280 +) + +// GridSlot returns the position for the n-th entity placed without an explicit +// @Position, counting from 0. +// +// Stable by construction: slot n does not depend on anything but n, so adding an +// entity never moves one already written. +func GridSlot(n int) model.Point { + if n < 0 { + n = 0 + } + return model.Point{ + X: OriginX + (n%GridColumns)*GridColumnPitch, + Y: OriginY + (n/GridColumns)*GridRowPitch, + } +} + +// Box size estimates. Studio Pro's real dimensions are not available to mxcli +// (see the package comment), so these are chosen to clear a typical box with +// room to spare rather than to match it. +const ( + minBoxWidth = 160 + pixelsPerChar = 8 // rough advance width of the editor's font + boxPadding = 24 // name inset plus the type column's share + headerHeight = 34 // the entity's name bar + memberHeight = 16 // one attribute row + minBoxHeight = 60 + + columnGutter = 90 // horizontal space between layers + rowGutter = 40 // vertical space between entities within a layer + bandGutter = 140 +) + +// Plan computes a position for every entity in one domain model. +// +// The layout is layered on the association graph. An entity that references +// nothing else in the module is layer 0; anything else sits one layer past the +// furthest thing it references. Layers become columns, so the things everything +// points at — the lookup tables — end up on the left and the leaves on the +// right, with most association lines running the same way. +// +// Entities with no local edge at all (a non-persistent helper, a freshly created +// entity nothing points at yet) are not lookups and do not belong in layer 0 +// beside them; they go in their own band underneath. +// +// The result is deterministic for a given model: every iteration order here is +// sorted, because a map range would produce a different diagram on every run and +// re-writing the unit each time is exactly the churn ADR-0008 exists to prevent. +func Plan(dm *domainmodel.DomainModel) map[model.ID]model.Point { + if dm == nil { + return nil + } + g := newGraph(dm) + if len(g.order) == 0 { + return map[model.ID]model.Point{} + } + + layers, isolated := g.layers() + out := make(map[model.ID]model.Point, len(g.order)) + + x := OriginX + maxY := OriginY + for _, layer := range layers { + widest := 0 + y := OriginY + for _, id := range layer { + e := g.byID[id] + w, h := boxSize(e) + if w > widest { + widest = w + } + out[id] = model.Point{X: x + w/2, Y: y + h/2} + y += h + rowGutter + } + if y > maxY { + maxY = y + } + x += widest + columnGutter + } + + // The isolated band: a plain grid under everything else, so it reads as a + // separate group rather than as another layer of the graph. + if len(isolated) > 0 { + bandY := maxY + bandGutter + for i, id := range isolated { + e := g.byID[id] + w, h := boxSize(e) + slot := GridSlot(i) + out[id] = model.Point{ + X: slot.X + w/2, + Y: bandY + (slot.Y - OriginY) + h/2, + } + } + } + return out +} + +// boxSize estimates the drawn size of an entity from its members. +func boxSize(e *domainmodel.Entity) (w, h int) { + if e == nil { + return minBoxWidth, minBoxHeight + } + longest := len(e.Name) + for _, a := range e.Attributes { + if n := len(a.Name); n > longest { + longest = n + } + } + w = longest*pixelsPerChar + boxPadding + if w < minBoxWidth { + w = minBoxWidth + } + h = headerHeight + len(e.Attributes)*memberHeight + if h < minBoxHeight { + h = minBoxHeight + } + return w, h +} + +// graph is the module's entities plus the local edges between them. +type graph struct { + byID map[model.ID]*domainmodel.Entity + order []model.ID // entity ids, sorted by name — the determinism anchor + out map[model.ID][]model.ID + deg map[model.ID]int // total edges, either direction +} + +// newGraph indexes the domain model. An edge runs FROM the entity that holds the +// reference TO the entity referenced — the same direction MDL's `from`/`to` +// spells, which for an association is ParentID -> ChildID (see CLAUDE.md on the +// inverted parent/child naming). +// +// A generalization is an edge too: a specialisation belongs beside its base, and +// the direction matches (the specialisation depends on the base). +// +// Cross-module associations are skipped: their target is not in this domain +// model, so it cannot be placed relative to anything here. +func newGraph(dm *domainmodel.DomainModel) *graph { + g := &graph{ + byID: make(map[model.ID]*domainmodel.Entity, len(dm.Entities)), + out: map[model.ID][]model.ID{}, + deg: map[model.ID]int{}, + } + for _, e := range dm.Entities { + if e == nil { + continue + } + g.byID[e.ID] = e + } + for id := range g.byID { + g.order = append(g.order, id) + } + sort.Slice(g.order, func(i, j int) bool { + return g.byID[g.order[i]].Name < g.byID[g.order[j]].Name + }) + + add := func(from, to model.ID) { + if from == to || g.byID[from] == nil || g.byID[to] == nil { + return + } + g.out[from] = append(g.out[from], to) + g.deg[from]++ + g.deg[to]++ + } + for _, a := range dm.Associations { + if a != nil { + add(a.ParentID, a.ChildID) + } + } + for _, e := range dm.Entities { + if e != nil && e.GeneralizationID != "" { + add(e.ID, e.GeneralizationID) + } + } + for from := range g.out { + sort.Slice(g.out[from], func(i, j int) bool { + return g.byID[g.out[from][i]].Name < g.byID[g.out[from][j]].Name + }) + } + return g +} + +// layers assigns each connected entity a layer and returns the layers in order, +// plus the entities that have no local edge at all. +func (g *graph) layers() (layers [][]model.ID, isolated []model.ID) { + depth := make(map[model.ID]int, len(g.order)) + const ( + unvisited = 0 + active = 1 + done = 2 + ) + state := make(map[model.ID]int, len(g.order)) + + // Longest path to a sink. A cycle would make that undefined, so an edge back + // into the current path contributes nothing — the entities in the cycle land + // in the same layer, which is where they belong anyway. + var visit func(id model.ID) int + visit = func(id model.ID) int { + switch state[id] { + case done: + return depth[id] + case active: + return 0 + } + state[id] = active + best := 0 + for _, t := range g.out[id] { + if d := visit(t) + 1; d > best { + best = d + } + } + state[id] = done + depth[id] = best + return best + } + + maxDepth := 0 + for _, id := range g.order { + if g.deg[id] == 0 { + isolated = append(isolated, id) + continue + } + if d := visit(id); d > maxDepth { + maxDepth = d + } + } + + layers = make([][]model.ID, maxDepth+1) + for _, id := range g.order { + if g.deg[id] == 0 { + continue + } + layers[depth[id]] = append(layers[depth[id]], id) + } + + // Within a layer, order by where an entity's targets already sit, so lines + // run roughly straight instead of crossing the diagram. Ties — and layer 0, + // which has no targets — fall back to the name order g.order is already in, + // which is what keeps the result deterministic. + pos := map[model.ID]int{} + for li, layer := range layers { + if li > 0 { + sort.SliceStable(layer, func(i, j int) bool { + return g.barycentre(layer[i], pos) < g.barycentre(layer[j], pos) + }) + } + for i, id := range layer { + pos[id] = i + } + } + return layers, isolated +} + +// barycentre is the mean row of an entity's already-placed targets, or a large +// sentinel when it has none placed yet so those sink to the bottom of the layer +// rather than jostling the ones that do. +func (g *graph) barycentre(id model.ID, pos map[model.ID]int) float64 { + sum, n := 0, 0 + for _, t := range g.out[id] { + if p, ok := pos[t]; ok { + sum += p + n++ + } + } + if n == 0 { + return 1 << 20 + } + return float64(sum) / float64(n) +} diff --git a/mdl/dmlayout/dmlayout_test.go b/mdl/dmlayout/dmlayout_test.go new file mode 100644 index 000000000..51b310bdb --- /dev/null +++ b/mdl/dmlayout/dmlayout_test.go @@ -0,0 +1,294 @@ +// SPDX-License-Identifier: Apache-2.0 + +package dmlayout + +import ( + "fmt" + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// The reported symptom: a generated domain model came out as one horizontal +// line. mxcli placed every entity at y=100, stepping x by 150 — so a 40-entity +// model was a 6,000px row, and since 150px is narrower than an entity box the +// boxes touched as well (ako/CapTrackV2, Mendix 11.13). + +func ent(name string, attrs ...string) *domainmodel.Entity { + e := &domainmodel.Entity{Name: name} + e.ID = model.ID("id-" + name) + for _, a := range attrs { + e.Attributes = append(e.Attributes, &domainmodel.Attribute{Name: a}) + } + return e +} + +func assoc(name, from, to string) *domainmodel.Association { + // ParentID is the FROM entity and ChildID the TO entity — Mendix's inverted + // naming, per CLAUDE.md. + return &domainmodel.Association{ + Name: name, + ParentID: model.ID("id-" + from), + ChildID: model.ID("id-" + to), + } +} + +// GridSlot must wrap. A default that only ever increments x is the bug. +func TestGridSlot_Wraps(t *testing.T) { + first := GridSlot(0) + if first.X != OriginX || first.Y != OriginY { + t.Errorf("slot 0 = %+v, want the origin", first) + } + // The old default put entity 39 at x=100+39*150=6950, y=100. + last := GridSlot(39) + if last.Y == OriginY { + t.Error("slot 39 is still on the first row — the grid does not wrap") + } + if last.X > OriginX+GridColumns*GridColumnPitch { + t.Errorf("slot 39 x=%d is past the last column; the row never wrapped", last.X) + } +} + +// The width of a 40-entity model is the number the report was about. +func TestGridSlot_KeepsAFortyEntityModelOnScreen(t *testing.T) { + widest := 0 + for n := 0; n < 40; n++ { + if x := GridSlot(n).X; x > widest { + widest = x + } + } + if widest > 2000 { + t.Errorf("40 entities span %dpx horizontally; the single-row default spanned 6950", widest) + } +} + +// Slot n must depend on nothing but n: an entity written by an earlier statement +// must not move when a later one is added, or every re-run rewrites the unit. +func TestGridSlot_IsStableAsEntitiesAreAdded(t *testing.T) { + for n := 0; n < 50; n++ { + a, b := GridSlot(n), GridSlot(n) + if a != b { + t.Fatalf("slot %d is not a function of n: %+v vs %+v", n, a, b) + } + } +} + +// capTrack is the reported model's shape: six lookups nothing points out of, a +// chain four deep, and two unconnected helpers. +func capTrack() *domainmodel.DomainModel { + dm := &domainmodel.DomainModel{ + Entities: []*domainmodel.Entity{ + ent("Department", "Name", "Code"), + ent("Region", "Code", "Name"), + ent("PlanType", "Code", "Name"), + ent("PlanningYear", "YearNr"), + ent("EmploymentType", "Name"), + ent("MovementReason", "Name"), + ent("Team", "Name"), + ent("GoalBucket", "Name", "Scope"), + ent("PlanScope", "ESPreviousYear"), + ent("ScopeMonth", "MonthNr", "Value"), + ent("Employee", "Name", "BaseFTE"), + ent("EmployeeMonth", "MonthNr", "FTE"), + ent("Movement", "MovementType", "Status", "MonthNr"), + ent("NP_NewEmployee", "Name"), + ent("NP_UserDeptToggle", "Active"), + }, + Associations: []*domainmodel.Association{ + assoc("Team_Department", "Team", "Department"), + assoc("GoalBucket_Department", "GoalBucket", "Department"), + assoc("GoalBucket_PlanningYear", "GoalBucket", "PlanningYear"), + assoc("PlanScope_Department", "PlanScope", "Department"), + assoc("PlanScope_Team", "PlanScope", "Team"), + assoc("PlanScope_Region", "PlanScope", "Region"), + assoc("PlanScope_PlanType", "PlanScope", "PlanType"), + assoc("PlanScope_PlanningYear", "PlanScope", "PlanningYear"), + assoc("ScopeMonth_PlanScope", "ScopeMonth", "PlanScope"), + assoc("Employee_PlanScope", "Employee", "PlanScope"), + assoc("Employee_EmploymentType", "Employee", "EmploymentType"), + assoc("Employee_GoalBucket", "Employee", "GoalBucket"), + assoc("EmployeeMonth_Employee", "EmployeeMonth", "Employee"), + assoc("Movement_PlanScope", "Movement", "PlanScope"), + assoc("Movement_Employee", "Movement", "Employee"), + assoc("Movement_MovementReason", "Movement", "MovementReason"), + }, + } + return dm +} + +// The point of the graph layout: an entity sits to the right of everything it +// references, so association lines run one way instead of across the diagram. +func TestPlan_ReferencedEntitiesSitLeftOfTheirReferrers(t *testing.T) { + dm := capTrack() + pos := Plan(dm) + + byName := map[string]model.Point{} + for _, e := range dm.Entities { + p, ok := pos[e.ID] + if !ok { + t.Fatalf("%s got no position", e.Name) + } + byName[e.Name] = p + } + + for _, c := range []struct{ from, to string }{ + {"Team", "Department"}, + {"PlanScope", "Region"}, + {"ScopeMonth", "PlanScope"}, + {"Employee", "PlanScope"}, + {"EmployeeMonth", "Employee"}, + {"Movement", "Employee"}, + } { + if byName[c.from].X <= byName[c.to].X { + t.Errorf("%s (x=%d) should sit right of %s (x=%d) — it references it", + c.from, byName[c.from].X, c.to, byName[c.to].X) + } + } +} + +// The lookups nothing points out of share the leftmost column, and the deepest +// entity is well clear of it. +func TestPlan_LayersTheModel(t *testing.T) { + dm := capTrack() + pos := Plan(dm) + byName := map[string]model.Point{} + for _, e := range dm.Entities { + byName[e.Name] = pos[e.ID] + } + + lookups := []string{"Department", "Region", "PlanType", "PlanningYear", "EmploymentType", "MovementReason"} + first := byName[lookups[0]].X + for _, n := range lookups[1:] { + if byName[n].X != first { + t.Errorf("%s is at x=%d, not in the lookup column x=%d", n, byName[n].X, first) + } + } + if byName["EmployeeMonth"].X <= byName["PlanScope"].X { + t.Error("the deepest entity did not end up past the middle of the graph") + } +} + +// No two entities may overlap. This is the half the old default got wrong even +// ignoring the single row: a 150px step is narrower than a box. +func TestPlan_NoOverlap(t *testing.T) { + dm := capTrack() + pos := Plan(dm) + + type rect struct { + name string + x1, y1, x2, y2 int + } + var boxes []rect + for _, e := range dm.Entities { + p := pos[e.ID] + w, h := boxSize(e) + boxes = append(boxes, rect{e.Name, p.X - w/2, p.Y - h/2, p.X + w/2, p.Y + h/2}) + } + for i := range boxes { + for j := i + 1; j < len(boxes); j++ { + a, b := boxes[i], boxes[j] + if a.x1 < b.x2 && b.x1 < a.x2 && a.y1 < b.y2 && b.y1 < a.y2 { + t.Errorf("%s and %s overlap: %+v vs %+v", a.name, b.name, a, b) + } + } + } +} + +// Unconnected entities are not lookups. Putting them in layer 0 would mix the +// non-persistent helpers in with the real reference data. +func TestPlan_IsolatedEntitiesGoInTheirOwnBand(t *testing.T) { + dm := capTrack() + pos := Plan(dm) + byName := map[string]model.Point{} + for _, e := range dm.Entities { + byName[e.Name] = pos[e.ID] + } + for _, n := range []string{"NP_NewEmployee", "NP_UserDeptToggle"} { + if byName[n].Y <= byName["Department"].Y { + t.Errorf("%s (y=%d) is level with the graph, not in the band below (Department y=%d)", + n, byName[n].Y, byName["Department"].Y) + } + } +} + +// Determinism is load-bearing, not cosmetic: a layout that shuffles rewrites the +// domain-model unit on every run, which is the churn ADR-0008 exists to prevent. +// Go randomises map iteration, so this would fail on an unsorted walk. +func TestPlan_IsDeterministic(t *testing.T) { + want := fmt.Sprint(Plan(capTrack())) + for i := 0; i < 50; i++ { + if got := fmt.Sprint(Plan(capTrack())); got != want { + t.Fatalf("run %d produced a different layout", i) + } + } +} + +// A cycle has no longest path to a sink. The layout must still terminate and +// place everything — a self-reference or a mutual pair is legal in Mendix. +func TestPlan_SurvivesCycles(t *testing.T) { + dm := &domainmodel.DomainModel{ + Entities: []*domainmodel.Entity{ent("A", "x"), ent("B", "y"), ent("C", "z")}, + Associations: []*domainmodel.Association{ + assoc("A_B", "A", "B"), + assoc("B_A", "B", "A"), // mutual + assoc("C_C", "C", "C"), // self + assoc("C_A", "C", "A"), + }, + } + pos := Plan(dm) + if len(pos) != 3 { + t.Fatalf("got %d positions, want 3: %v", len(pos), pos) + } + seen := map[model.Point]string{} + for _, e := range dm.Entities { + if prev, dup := seen[pos[e.ID]]; dup { + t.Errorf("%s and %s were placed at the same point %+v", prev, e.Name, pos[e.ID]) + } + seen[pos[e.ID]] = e.Name + } +} + +// CONTROL: an empty or nil model must not panic and must not invent positions. +func TestPlan_EmptyModel(t *testing.T) { + if got := Plan(nil); got != nil { + t.Errorf("Plan(nil) = %v, want nil", got) + } + if got := Plan(&domainmodel.DomainModel{}); len(got) != 0 { + t.Errorf("Plan(empty) = %v, want no positions", got) + } +} + +// Adding an entity must perturb the diagram locally, not reshuffle it. A layout +// that moved everything whenever the model changed would make every domain-model +// commit an unreadable diff, which is the practical reason to care about this +// beyond aesthetics. +// +// Measured on the real thing: adding one entity with one association to +// CapTrack's 16 moved 3 of 17 — the new entity and the two below it in its +// column. +func TestPlan_AddingAnEntityMovesFewOthers(t *testing.T) { + before := capTrack() + posBefore := Plan(before) + + after := capTrack() + extra := ent("AuditEntry", "Note") + after.Entities = append(after.Entities, extra) + after.Associations = append(after.Associations, assoc("AuditEntry_Department", "AuditEntry", "Department")) + posAfter := Plan(after) + + movedNames := []string{} + for _, e := range before.Entities { + if posBefore[e.ID] != posAfter[e.ID] { + movedNames = append(movedNames, e.Name) + } + } + // A quarter of the model is a generous ceiling; the real figure here is 2. + if len(movedNames) > len(before.Entities)/4 { + t.Errorf("adding one entity moved %d of %d existing entities (%v) — the layout is not local", + len(movedNames), len(before.Entities), movedNames) + } + if _, ok := posAfter[extra.ID]; !ok { + t.Error("the added entity got no position") + } +} diff --git a/mdl/executor/cmd_entities.go b/mdl/executor/cmd_entities.go index 50dbe5c38..2836adf38 100644 --- a/mdl/executor/cmd_entities.go +++ b/mdl/executor/cmd_entities.go @@ -8,6 +8,7 @@ import ( "strings" "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/dmlayout" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" @@ -154,8 +155,14 @@ func execCreateEntity(ctx *ExecContext, s *ast.CreateEntityStmt) error { } else if existingEntity != nil { location = existingEntity.Location } else { - // Auto-position based on existing entities - location = model.Point{X: 100 + len(dm.Entities)*150, Y: 100} + // No @Position and no stored entity: take the next grid slot. + // + // This used to be `X: 100 + len(dm.Entities)*150, Y: 100` — one row, for + // every entity ever created. A 40-entity domain model came out as a + // 6,000px line, and 150px is narrower than an entity box, so the boxes + // touched as well. The grid is not a good layout, only a defensible + // default; `mxcli layout` computes one from the association graph. + location = dmlayout.GridSlot(len(dm.Entities)) } // Determine persistable based on entity kind From fc77d74517007d499e0158f2a3704d613ef9decb Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 12:46:49 +0000 Subject: [PATCH 17/17] docs(layout): document the command where users actually look MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The layout change landed with CLAUDE.md and the Cobra help updated and nothing else. The three that were missed are the user-facing ones. `mxcli syntax domain-model.entity.alter` did not document SET POSITION AT ALL — a gap that predates this work — so the topic now lists it, says the coordinate is the box's CENTRE rather than its top-left, and points at `mxcli layout` for arranging a whole module. The generate-domain-model skill was actively wrong: IMPORTANT: All entities MUST have @Position annotation Without it, entities appear at (0,0) or random locations. Neither half held. An entity without a position took the next slot in a deterministic row — now a grid — and never (0,0). The "MUST" was advice nothing enforced, and the model that prompted this work ignored it, which is part of how it came out as a 6,000px line. The section now says positions are optional, that a grid is a default rather than a layout, and that a generated domain model is better served by writing none and running `mxcli layout` afterwards. New docs-site page tools/domain-model-layout.md, linked from SUMMARY.md: what the layering does, the measured column breakdown, the flags, the fact that it replaces hand-placed positions, and the idempotence/locality properties that make it safe to leave in a build script. make check-skill-mdl passes (205 blocks). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018hifgRSawfaRWXS44YKtSJ --- .../generate-domain-model/reference/syntax.md | 33 ++++++-- cmd/mxcli/syntax/features_domain_model.go | 2 +- docs-site/src/SUMMARY.md | 1 + docs-site/src/tools/domain-model-layout.md | 84 +++++++++++++++++++ 4 files changed, 112 insertions(+), 8 deletions(-) create mode 100644 docs-site/src/tools/domain-model-layout.md diff --git a/.claude/skills/mendix/generate-domain-model/reference/syntax.md b/.claude/skills/mendix/generate-domain-model/reference/syntax.md index 238db627a..1fcea2abe 100644 --- a/.claude/skills/mendix/generate-domain-model/reference/syntax.md +++ b/.claude/skills/mendix/generate-domain-model/reference/syntax.md @@ -96,15 +96,34 @@ identifiers; a value whose name is a reserved word can't be targeted by `alter`. ### Entities -**IMPORTANT: All entities MUST have @Position annotation** +**Positions are optional. Prefer `mxcli layout` over hand-placing them.** -The `@position(x, y)` annotation specifies where the entity appears in the domain model diagram. Without it, entities appear at (0,0) or random locations. +`@position(x, y)` sets where the entity sits in the domain-model diagram. An +entity without one is NOT lost — it takes the next slot in a wrapping grid — but +a grid is a default, not a layout: it knows nothing about which entities are +related, so association lines still cross the diagram. -**Position Guidelines:** -- Use increments of 50 or 100 for spacing (e.g., 100, 200, 300) -- Leave space between entities (at least 200 pixels) -- Organize related entities in logical groups -- Example layout: Categories at y=100, Transactions at y=300, Reports at y=500 +The better answer for a generated domain model is to write no positions at all +and arrange the module once the script has run: + +```bash +mxcli layout -p app.mpr --module MyModule --dry-run # see the moves +mxcli layout -p app.mpr --module MyModule # apply +``` + +That lays entities out from the association graph — lookups on the left, each +entity one column past the furthest thing it references — so the lines mostly +run one way. It is idempotent (a second run moves nothing) and local (adding an +entity later moves a handful, not the model). It REPLACES positions you set by +hand in the module it touches, which is the reason it is a separate command +rather than something `exec` does on its own. + +Write `@position` when you want explicit control of a particular entity, and +remember the coordinate is the box's **CENTRE**, not its top-left corner: + +- 250+ apart horizontally, 250+ vertically, to clear a typical box +- group related entities, and keep the lookups together +- `alter entity Mod.Name set position (x, y)` moves one without restating it **Association line anchors** — where the connector attaches to each entity box — are set with `@anchor`, as a **percentage of the box** (0..100, whole numbers): diff --git a/cmd/mxcli/syntax/features_domain_model.go b/cmd/mxcli/syntax/features_domain_model.go index 367513f8c..f9df6bf95 100644 --- a/cmd/mxcli/syntax/features_domain_model.go +++ b/cmd/mxcli/syntax/features_domain_model.go @@ -95,7 +95,7 @@ func init() { "event handler", "documentation", "if not exists", "if exists", "idempotent", }, - Syntax: "ALTER ENTITY Module.Name ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [constraints];\nALTER ENTITY Module.Name DROP ATTRIBUTE [IF EXISTS] AttrName;\nALTER ENTITY Module.Name RENAME ATTRIBUTE OldName TO NewName;\nALTER ENTITY Module.Name MODIFY ATTRIBUTE AttrName Type [DEFAULT val];\nALTER ENTITY Module.Name DROP DEFAULT ON ATTRIBUTE AttrName;\nALTER ENTITY Module.Name ADD INDEX [name] [ON] (attr1, attr2);\nALTER ENTITY Module.Name SET DOCUMENTATION 'text';\nALTER ENTITY Module.Name ADD EVENT HANDLER ON BEFORE COMMIT CALL Module.MF RAISE ERROR;\n\nMODIFY ATTRIBUTE always takes a type — restate it even when you are only\nchanging the default. There is no 'MODIFY ATTRIBUTE X SET DEFAULT v' form:\nSET would be read as the type name. Use DROP DEFAULT to clear one.\n\nIF NOT EXISTS / IF EXISTS make the add/drop a no-op (skipped, not an error)\nwhen the attribute is already present / already gone — so a domain script\nre-runs cleanly. For a whole script, 'mxcli exec --continue-on-error' reports\neach failed statement and keeps going instead of halting at the first.\n\nRENAME ATTRIBUTE also rewrites every reference to the attribute: the stored\nqualified names (microflow create/change members, page widgets, the entity's own\nvalidation and access rules) AND the bare steps inside XPath constraints, which\nare resolved to their owning entity first so another entity's identically-named\nattribute is left alone. A constraint that cannot be resolved is reported and\nleft unchanged, never guessed at. Uses inside microflow expressions ($obj/Attr)\nare free text and are NOT rewritten; mxbuild reports those as CE0117.", + Syntax: "ALTER ENTITY Module.Name ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [constraints];\nALTER ENTITY Module.Name DROP ATTRIBUTE [IF EXISTS] AttrName;\nALTER ENTITY Module.Name RENAME ATTRIBUTE OldName TO NewName;\nALTER ENTITY Module.Name MODIFY ATTRIBUTE AttrName Type [DEFAULT val];\nALTER ENTITY Module.Name DROP DEFAULT ON ATTRIBUTE AttrName;\nALTER ENTITY Module.Name ADD INDEX [name] [ON] (attr1, attr2);\nALTER ENTITY Module.Name SET DOCUMENTATION 'text';\nALTER ENTITY Module.Name SET POSITION (x, y);\nALTER ENTITY Module.Name ADD EVENT HANDLER ON BEFORE COMMIT CALL Module.MF RAISE ERROR;\n\nSET POSITION places the entity in the domain-model editor, and CREATE ENTITY\ntakes the same thing as an @Position(x, y) annotation. Both are the box's\nCENTRE, not its top-left corner. An entity created without one takes the next\nslot in a wrapping grid, which is a default rather than a layout: to arrange a\nwhole module from its association graph, run 'mxcli layout -p app.mpr'\n(--dry-run first; it replaces positions you set by hand).\n\nMODIFY ATTRIBUTE always takes a type — restate it even when you are only\nchanging the default. There is no 'MODIFY ATTRIBUTE X SET DEFAULT v' form:\nSET would be read as the type name. Use DROP DEFAULT to clear one.\n\nIF NOT EXISTS / IF EXISTS make the add/drop a no-op (skipped, not an error)\nwhen the attribute is already present / already gone — so a domain script\nre-runs cleanly. For a whole script, 'mxcli exec --continue-on-error' reports\neach failed statement and keeps going instead of halting at the first.\n\nRENAME ATTRIBUTE also rewrites every reference to the attribute: the stored\nqualified names (microflow create/change members, page widgets, the entity's own\nvalidation and access rules) AND the bare steps inside XPath constraints, which\nare resolved to their owning entity first so another entity's identically-named\nattribute is left alone. A constraint that cannot be resolved is reported and\nleft unchanged, never guessed at. Uses inside microflow expressions ($obj/Attr)\nare free text and are NOT rewritten; mxbuild reports those as CE0117.", Example: "ALTER ENTITY Shop.Customer ADD ATTRIBUTE Phone: String(20);\nALTER ENTITY Shop.Customer ADD ATTRIBUTE IF NOT EXISTS Phone: String(20); -- re-runnable\nALTER ENTITY Shop.Customer DROP ATTRIBUTE IF EXISTS OldField; -- re-runnable\nALTER ENTITY Shop.Customer RENAME ATTRIBUTE Email TO EmailAddress;\nALTER ENTITY Shop.Customer MODIFY ATTRIBUTE Phone String(30) DEFAULT ''; -- type restated\nALTER ENTITY Shop.Customer DROP DEFAULT ON ATTRIBUTE Phone; -- clear a default\nALTER ENTITY Shop.Customer ADD INDEX ON (EmailAddress);\nALTER ENTITY Shop.Customer\n ADD EVENT HANDLER ON BEFORE COMMIT CALL Shop.Validate($currentObject) RAISE ERROR;", SeeAlso: []string{"domain-model.entity.create", "domain-model.entity.attributes"}, }) diff --git a/docs-site/src/SUMMARY.md b/docs-site/src/SUMMARY.md index 9b4342d07..992b04634 100644 --- a/docs-site/src/SUMMARY.md +++ b/docs-site/src/SUMMARY.md @@ -120,6 +120,7 @@ # Part V: Project Tools - [Default Styling](tools/theme.md) +- [Domain Model Layout](tools/domain-model-layout.md) - [Code Navigation](tools/code-navigation.md) - [SHOW CALLERS / CALLEES](tools/callers-callees.md) - [SHOW REFERENCES / IMPACT](tools/references-impact.md) diff --git a/docs-site/src/tools/domain-model-layout.md b/docs-site/src/tools/domain-model-layout.md new file mode 100644 index 000000000..0b38d18c6 --- /dev/null +++ b/docs-site/src/tools/domain-model-layout.md @@ -0,0 +1,84 @@ +# Domain Model Layout + +A domain model generated from an MDL script has to be *placed* somewhere, and +MDL says nothing about placement unless you write `@position` on every entity. +`mxcli layout` arranges a module from its association graph, so you do not have +to. + +```bash +mxcli layout -p app.mpr --module Sales --dry-run # list the moves +mxcli layout -p app.mpr --module Sales # apply them +mxcli layout -p app.mpr # every module the project owns +``` + +## What it does + +Entities are layered on the associations between them: + +- an entity that references nothing else in the module is a **lookup**, and goes + in the leftmost column; +- everything else sits one column past the furthest thing it references; +- entities with **no association at all** — non-persistent helpers, mostly — go + in a band underneath, rather than being mixed in with the lookups. + +Association lines then mostly run one way instead of crossing the diagram. + +On a real 16-entity model the layering falls out of the associations with no +hints: + +| column | entities | +|---|---| +| 1 | `Department`, `EmploymentType`, `MovementReason`, `PlanType`, `PlanningYear`, `Region` | +| 2 | `Team`, `GoalBucket` | +| 3 | `GoalChange`, `GoalRegionValue`, `PlanScope`, `CapTrackUser` | +| 4 | `Employee`, `ScopeMonth` | +| 5 | `EmployeeMonth`, `Movement` | + +## It replaces positions you set by hand + +This is the reason it is a command you run rather than something `exec` does on +its own. Inside the modules it touches, every entity is repositioned — including +any you arranged yourself in Studio Pro. Use `--dry-run` first; it prints every +move as `Module.Entity (x, y) -> (x, y)` and writes nothing. + +Marketplace modules and `System` are never touched. Naming one explicitly is an +error rather than a silent skip: + +``` +$ mxcli layout -p app.mpr --module Administration +Administration comes from the Marketplace, and a module update replaces it — +pass --include-marketplace to lay it out anyway +``` + +## Running it more than once + +Positions are a function of the model alone, so the command is safe to leave in +a build script: + +- **Idempotent.** A second run reports `already laid out` and does not write, so + it produces no version-control noise. +- **Local.** Adding an entity and re-running moves only what the new + relationships require — measured at 3 of 17 for one added entity with one + association, not a reshuffle of the diagram. + +## Flags + +| Flag | Meaning | +|---|---| +| `--module ` | Lay out this module (repeatable). Default: every module the project owns. | +| `--dry-run` | Print the moves, write nothing. | +| `--include-marketplace` | Also lay out Marketplace modules. A module update replaces them, so this is normally pointless. | + +## When you do want explicit positions + +`@position(x, y)` on `CREATE ENTITY`, and `ALTER ENTITY … SET POSITION (x, y)` +to move one afterwards, both still work — and `describe entity` emits the stored +position, so arranging a model and describing it back is a way to capture a +layout into MDL. + +Two things to know if you place entities yourself: + +- The coordinate is the box's **centre**, not its top-left corner. +- An entity created with no position takes the next slot in a wrapping grid. + That is a default, not a layout: it keeps a large model on screen and stops + boxes overlapping, but it knows nothing about which entities are related.