diff --git a/.gitignore b/.gitignore index 82e7a97..4142471 100644 --- a/.gitignore +++ b/.gitignore @@ -42,3 +42,8 @@ wip/ # OS noise .DS_Store .session-notes.md + +# Claude Code session state (park notes, checkpoints, pending-write ledgers) — +# machine-local by nature, and park notes routinely name real clients, which +# the leak guard (bin/check-no-client-data.sh) must never see published. +.claude/ diff --git a/CLAUDE.md b/CLAUDE.md index 03f423c..eaf31e9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -153,6 +153,13 @@ changed `bin/gate-check.sh` → also run `test-bug03-gates.sh`, `test-source-suf whatever else targets it). Grep `tests/wave2/` for the filename you changed if you're unsure which fixtures cover it. +There are two separate suites. `tests/run-tests.sh [-v]` is self-contained (guard-script +fixtures with no external argument). Each `tests/wave2/test-*.sh` fixture instead takes its +subject-under-test as `$1` — read its own `usage:` comment rather than assuming a fixed target; +several take something other than `gate-check.sh` (`exec.sh`, `sync-project.sh`, +`check-docs-numbering.sh`, a skill file). Once you have permission for a single fixture (see +below), run it as e.g. `bash tests/wave2/test-bug02-register.sh bin/gate-check.sh`. + **Why:** this repo is routinely edited by more than one agent session at once (real incident, 2026-08-19 — a peer session mid-edit reported known, unrelated failures already present in the shared working tree). A full-suite run over a moving target mixes someone else's in-flight diff --git a/README.md b/README.md index 7692359..d9e29c9 100644 --- a/README.md +++ b/README.md @@ -554,7 +554,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini | Task | Skill to load | |---|---| -| Writing or debugging a Mendix native Workflow (CREATE WORKFLOW/USER TASK/OUTCOMES) — syntax, the 11 workflow microflow statements, and the binary-version $Type corruption class | `skills/learned-workflow-patterns.md` | +| Writing or debugging a Mendix native Workflow (CREATE WORKFLOW/USER TASK/OUTCOMES) — syntax, the 11 workflow microflow statements, DECISION vs CALL MICROFLOW, and the two corruption classes (binary-version $Type, and create-before-reference) | `skills/learned-workflow-patterns.md` | **Build · Integration** diff --git a/ROUTING.md b/ROUTING.md index 0cf7299..6f90ff1 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -146,7 +146,7 @@ picks the row up. That is the whole procedure — there is no second list to rem | Always relevant for | Load this | Agent(s) | Stage(s) | Tier | |---|---|---|---|---| -| Writing or debugging a Mendix native Workflow (CREATE WORKFLOW/USER TASK/OUTCOMES) — syntax, the 11 workflow microflow statements, and the binary-version $Type corruption class | `skills/learned-workflow-patterns.md` | mdl | 5 | ondemand | +| Writing or debugging a Mendix native Workflow (CREATE WORKFLOW/USER TASK/OUTCOMES) — syntax, the 11 workflow microflow statements, DECISION vs CALL MICROFLOW, and the two corruption classes (binary-version $Type, and create-before-reference) | `skills/learned-workflow-patterns.md` | mdl | 5 | ondemand | #### Build · Integration @@ -173,6 +173,7 @@ picks the row up. That is the whole procedure — there is no second list to rem | Tracking scope delta between the BRD and the built state | `skills/learned-skill-scope-delta.md` | review | 6 | ondemand | | Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer | `skills/report-schema.md` | test,review,gate | 5,6 | ondemand | | Installing, extending, debugging or porting the verification harness — which part owns what, which parts run standalone, and what a missing part must report | `skills/harness-architecture.md` | test,review | 5,6 | ondemand | +| A user asks for "a full end-to-end test," "does everything work," or a click-through proof — before running anything, or if a prior pass turns out to have covered one module and gotten called e2e | `skills/full-harness-audit.md` | test,review | 5,6 | ondemand | | Checking whether the whole journey hangs together rather than each piece — finds correctly-built components nothing reaches, which per-element conformance and UI tests both miss | `skills/process-coherence-pass.md` | review | 5,6 | ondemand | | After every module's CONFIRM stage — counts proven modules since the last cluster/full coherence pass and exits DUE once the threshold is reached, so the cadence isn't left to memory | `project-bin/coherence-cadence.sh` | mdl,gate,review | 5,6 | ondemand | | Turning an already-rigorous run into a narrated proof a stakeholder can trust without running anything | `skills/e2e-evidence-report.md` | test,review | 6 | ondemand | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index fd40820..668b71f 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -64,7 +64,7 @@ a rule below names an asset (e.g. "the wireframe", "the brief"), it means the pa | `project-bin/lint-gate.sh` | Running lint as a gate rather than a report — per-rule ratchet against a committed baseline, plus the crash and collapse guards that stop a blind rule passing | | `skills/improvement-register.md` | Any review pass that runs more than once — module-review, coherence, monkey, wiring-sweep: findings accumulate across runs, a per-run report cannot show a trend | | `skills/wiring-sweep.md` | Every module before it is called done — does every clickable thing actually do something; run AFTER the happy-path journey is green, never before | -| `skills/learned-workflow-patterns.md` | Writing or debugging a Mendix native Workflow (CREATE WORKFLOW/USER TASK/OUTCOMES) — syntax, the 11 workflow microflow statements, and the binary-version $Type corruption class | +| `skills/learned-workflow-patterns.md` | Writing or debugging a Mendix native Workflow (CREATE WORKFLOW/USER TASK/OUTCOMES) — syntax, the 11 workflow microflow statements, DECISION vs CALL MICROFLOW, and the two corruption classes (binary-version $Type, and create-before-reference) | | `skills/rest-integration-first-time-right.md` | Building a REST integration (consumed or published) for the first time on a project — the checks that avoid a rebuild after the first live call | | `skills/bug-submission-checklist.md` | Preparing an mxcli/Studio Pro bug for submission — scope pinning, read-back-vs-write-path verification, gate-sensitivity negative controls, severity scoping, before it's called filable | | `skills/empty-widget-triage.md` | A page/grid/combobox renders empty (blank cells, zero rows, zero options) during UI review or an e2e run — before assuming a single cause | diff --git a/bin/gate-check.sh b/bin/gate-check.sh index e67488e..ba0c333 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -6,7 +6,7 @@ # actual project state. A stage-specific run answers one question and writes nothing: that is the # invocation used in agent loops and hooks, and a read-only query must leave no trace. # -# Usage: bin/gate-check.sh [--html|--no-html] [--ack-protocol|--force-stale] +# Usage: bin/gate-check.sh [--html|--no-html] [--ack-protocol [--approved-in-chat]|--force-stale] [--verbose] # [--adopt --reason "..."] # [--waive --reason "..."] # [stage] @@ -46,6 +46,8 @@ set -uo pipefail # The obvious `set -- $POSITIONAL` form word-splits the path and was rejected for that reason. HTML_MODE="auto" ACK_PROTOCOL=0 +APPROVED_IN_CHAT=0 +PROTOCOL_VERBOSE=0 FORCE_STALE="${MXTK_ACK_STALE:-0}" STRICT_PROTOCOL="${MXTK_STRICT_PROTOCOL:-0}" ADOPT_STAGE="" @@ -58,6 +60,8 @@ while [ $# -gt 0 ]; do --html) HTML_MODE="always" ;; --no-html) HTML_MODE="never" ;; --ack-protocol) ACK_PROTOCOL=1 ;; + --approved-in-chat) APPROVED_IN_CHAT=1 ;; + --verbose) PROTOCOL_VERBOSE=1 ;; --adopt) shift; ADOPT_STAGE="${1:-}" ;; --waive) shift; WAIVE_STAGE="${1:-}" ;; --reason) shift; WAIVER_REASON="${1:-}" ;; @@ -76,6 +80,32 @@ if [ $# -lt 1 ]; then exit 1 fi +# --- Escape hatches, for when the gate itself is the thing blocking the room ------------- +# +# MXTK_SKIP_GATES=1 — return "no verdict" immediately and do no work at all. Added during a +# training round (2026-08-25) where gate-check took 5-15 minutes per invocation on participant +# machines that could not be reproduced here: on this Mac a full run is 1.9s empty / 7.8s on a +# real project, and the artifact tree walk — the previously known slow path, see the prune +# comment further down — costs 0.30s worst case over a 107k-file project. +# +# Exits 0 DELIBERATELY, including for a stage-specific run that would normally exit non-zero +# on failure. A skipped gate must not read as a failed one, and it must not wedge a caller +# that branches on the exit code. It is loud on stderr for the same reason the WAIVED verdict +# exists: a check nobody performed has to say so, every time, or it becomes green-by-absence. +# +# MXTK_NO_FETCH=1 — keep every gate, drop only the network. The protocol-freshness check runs +# `git fetch` against the toolkit remote on EVERY invocation (see TOOLKIT_REF below) over an +# HTTPS remote with no timeout, so a proxy that swallows packets or a credential prompt nobody +# is there to answer stalls the whole run. The fallback that this forces is already designed +# and already non-blocking: local HEAD, labelled UNVERIFIED. Prefer this over SKIP_GATES — +# it keeps the verdicts and only gives up the "is your toolkit current?" answer. +if [ "${MXTK_SKIP_GATES:-0}" = "1" ]; then + echo "gate-check: SKIPPED — MXTK_SKIP_GATES=1 is set in this environment." >&2 + echo " No stage was evaluated and no verdict was produced. This is NOT a pass." >&2 + echo " Re-enable with: unset MXTK_SKIP_GATES" >&2 + exit 0 +fi + PROJECT_DIR="$1" REQUESTED_STAGE="${2:-}" @@ -1409,8 +1439,19 @@ TOOLKIT_HEAD="$(git -C "$TOOLKIT_DIR" rev-parse --short HEAD 2>/dev/null || echo # fall back to local HEAD so offline work isn't stranded, but label the verdict UNVERIFIED so a # pass that proved nothing cannot read as a pass that did. SYNC_REF_LABEL="origin" -if git -C "$TOOLKIT_DIR" rev-parse --abbrev-ref '@{u}' >/dev/null 2>&1 \ - && git -C "$TOOLKIT_DIR" fetch --quiet 2>/dev/null; then +# The fetch is the only network call in this script and it had no timeout and no opt-out, so a +# slow or captive network turned a local file check into an unbounded wait. Two guards, both of +# which fall through to the existing UNVERIFIED path rather than failing: +# - MXTK_NO_FETCH=1 skips it outright. +# - Otherwise it is capped by git's own low-speed abort (~8s). A `timeout`/`gtimeout` wrapper +# was rejected: macOS ships NEITHER, so it would have silently broken every trainer's Mac. +# GIT_TERMINAL_PROMPT=0 stops it blocking forever on credentials nobody is there to type. +if [ "${MXTK_NO_FETCH:-0}" = "1" ]; then + TOOLKIT_REF="$TOOLKIT_HEAD" + SYNC_REF_LABEL="local HEAD — UNVERIFIED, fetch skipped (MXTK_NO_FETCH=1)" +elif git -C "$TOOLKIT_DIR" rev-parse --abbrev-ref '@{u}' >/dev/null 2>&1 \ + && GIT_TERMINAL_PROMPT=0 GIT_SSH_COMMAND="${GIT_SSH_COMMAND:-ssh -o BatchMode=yes -o ConnectTimeout=5}" \ + git -C "$TOOLKIT_DIR" -c http.lowSpeedLimit=1000 -c http.lowSpeedTime=8 fetch --quiet 2>/dev/null; then TOOLKIT_REF="$(git -C "$TOOLKIT_DIR" rev-parse --short '@{u}' 2>/dev/null || echo "unknown")" else TOOLKIT_REF="$TOOLKIT_HEAD" @@ -1519,35 +1560,69 @@ fi SYNC_BLOCKING=0 [ "$STRICT_PROTOCOL" = "1" ] && [ "$SYNC_STATUS" = "NOTICE" ] && SYNC_BLOCKING=1 -printf "Sync (Protocol freshness): %s — %s\n" "$SYNC_STATUS" "$SYNC_NOTE" +# The one-line verdict is jargon by construction — it names commits and paths so a maintainer +# can act on it without a second command. Non-verbose readers get the same verdict in words. +if [ "$PROTOCOL_VERBOSE" = "1" ]; then + printf "Sync (Protocol freshness): %s — %s\n" "$SYNC_STATUS" "$SYNC_NOTE" +else + case "$SYNC_STATUS" in + PASS) printf "Toolkit updates: up to date.\n" ;; + WARN) printf "Toolkit updates: the shared toolkit moved, but nothing this stage uses changed. No action needed.\n" ;; + NOTICE) printf "Toolkit updates: available, not yet reviewed. Nothing is blocked.\n" ;; + *) printf "Toolkit updates: %s\n" "$SYNC_STATUS" ;; + esac +fi if [ "$SYNC_STATUS" = "NOTICE" ]; then # Lead with the CHOICE, not with the diagnosis. The old message named files, showed no diff, # offered no command, and did not say where the value it wanted was written — so the only # available response was to go and find the four-step ritual in another file. echo "" - echo " ⚠ $SYNC_HEADLINE" - [ -n "$SYNC_DETAIL" ] && printf ' %s\n' "$SYNC_DETAIL" - # An UPGRADE IS AN OFFER, NOT A DEMAND (2026-08-20). When the toolkit grows a new artifact or - # a new stage requirement, the projects that see it first are the ones already mid-build — and - # for them "the protocol changed" often means nothing needs to happen: they did that analysis - # their own way, or they are past the stage entirely. Saying only "the toolkit moved" leaves - # them to guess whether they are now behind, and the safe-looking guess (regenerate it) is the - # expensive one. So name the stages it touches and put the "not needed here" answer on the - # same screen as the "yes please" one, with equal billing. + # LAYERED, and plain-language by default (2026-08-26). The old notice was written for someone + # who already knew what a "protocol commit" was: it led with two abbreviated SHAs, said "ack", + # and named skill file paths. A TAM running this in an enablement session reads that and can + # only conclude something is broken. Nothing about the SHAs helps them decide; the decision is + # "has the shared toolkit changed in a way that affects what I am about to build?" — so ask + # exactly that, in those words, and put the identifiers behind --verbose for the people who + # actually diff things. + if [ "$PROTOCOL_VERBOSE" = "1" ]; then + echo " ⚠ $SYNC_HEADLINE" + [ -n "$SYNC_DETAIL" ] && printf ' %s\n' "$SYNC_DETAIL" + else + echo " ⚠ The shared toolkit has been updated since this project last checked." + if [ -n "${SYNC_RELEVANT:-}" ]; then + SYNC_N="$(printf '%s' "$SYNC_RELEVANT" | wc -w | tr -d ' ')" + echo " $SYNC_N guidance file(s) that this stage relies on changed. Nothing is broken and" + echo " nothing is blocked — this is asking whether you want to take the update." + fi + fi if [ -n "${SYNC_RELEVANT:-}" ]; then # shellcheck disable=SC2086 — SYNC_RELEVANT is a deliberate list. SYNC_STAGES="$(stages_for_protocol_files $SYNC_RELEVANT)" [ -n "$SYNC_STAGES" ] && echo " Stages this touches: $SYNC_STAGES" + if [ "$PROTOCOL_VERBOSE" != "1" ]; then + echo " For the file list and commit ids, re-run with --verbose." + fi + fi + echo "" + if [ "$PROTOCOL_VERBOSE" = "1" ]; then + echo " A) $SYNC_OPT_A" + else + echo " If you are an agent: run \"$0 $PROJECT_DIR --ack-protocol --verbose\" to see what" + echo " changed, summarise it for the user in plain language, ask whether to take the" + echo " update, and only if they say yes re-run with --ack-protocol --approved-in-chat." + echo " Do not decide this on the user's behalf and do not ask them to type anything." + echo "" + echo " A) Take the update — the agent records it for you once you say yes." fi - echo " A) $SYNC_OPT_A" if [ "$SYNC_BLOCKING" = "1" ]; then echo " B) Proceed anyway: $0 --force-stale $PROJECT_DIR ${REQUESTED_STAGE:-}" echo " (--strict-protocol is on, so this run WILL stop; the bypass is logged.)" else - echo " B) Ignore for now — nothing is blocked. This notice repeats until acked." - echo " C) Not needed here — if the change asks for artifacts this project already covered" - echo " its own way, or is past: $0 --waive --reason \"...\" $PROJECT_DIR" - echo " Acking (A) never obliges you to produce anything; it records that you read it." + echo " B) Decide later — nothing is blocked. This notice repeats until you answer." + echo " C) Not needed here — if the update asks for work this project already did its own" + echo " way, or is past: $0 --waive --reason \"...\" $PROJECT_DIR" + echo " Taking the update (A) never obliges you to produce anything; it records that" + echo " you were told." fi echo "" fi @@ -1681,24 +1756,56 @@ if [ "$ACK_PROTOCOL" = "1" ]; then echo "Cannot acknowledge: the toolkit clone at $TOOLKIT_DIR has no resolvable commit." >&2 exit 1 fi - # Non-interactive: REFUSE, and name the path forward at the moment of refusal. + # Non-interactive: the ack still needs a HUMAN, but it no longer needs a TERMINAL. # - # An ack asserts that a HUMAN READ THE DIFF. Prompting for that with no human present is - # theatre, and an unattended auto-yes (the proposal's MXTK_ACK_YES=1) is worse than theatre: - # an agent would write PROTOCOL-ACK — a record asserting a read that never happened — into the - # one log this design relies on to tell reads from rubber-stamps, poisoning that signal - # permanently and silently. So an agent session takes the explicit, logged bypass instead: - # PROTOCOL-BYPASS is honest about what actually occurred and is countable. There is no - # MXTK_ACK_YES, and the refusal below always names --force-stale and says it is logged. - if [ ! -t 0 ]; then - echo "Refusing to acknowledge protocol non-interactively (no TTY on stdin)." >&2 - echo "An ack asserts a human read the diff; nothing here can make that true." >&2 - echo " A) A human runs: $0 $PROJECT_DIR --ack-protocol" >&2 - echo " B) Proceed anyway: $0 --force-stale $PROJECT_DIR ${REQUESTED_STAGE:-}" >&2 + # The original rule refused any ack without a TTY, because an ack asserts that a human read + # the diff and an unattended auto-yes would write a record of a read that never happened — + # poisoning the one signal that tells reads from rubber-stamps. That reasoning is intact and + # this code still enforces it. What it got wrong was equating "a human" with "a human at a + # terminal". The people this now runs in front of — TAMs in an enablement session, consultants + # in a workshop — are humans who will never open a shell, and refusing them left the notice + # repeating forever with no reachable answer, which is its own kind of rubber-stamp. + # + # So there are two honest ways to ack, and the log records WHICH: + # at a terminal — the human ran this, saw the diffstat, optionally paged the full diff + # approved in chat — the agent ran --ack-protocol --verbose, summarised what changed in + # plain language, ASKED, and the human said yes + # Both are real reads. Neither is an agent deciding alone: --approved-in-chat is a claim the + # agent makes on the record, in a countable log line, and an agent that sets it without having + # asked has falsified an audit trail rather than skipped a step. That is the same trust model + # every other write in this toolkit already runs on. + # + # There is still no env-var auto-yes, and --approved-in-chat is deliberately not implied by + # anything: it must be typed, once, per ack, after the question was actually put. + if [ ! -t 0 ] && [ "$APPROVED_IN_CHAT" != "1" ]; then + if [ "$PROTOCOL_VERBOSE" = "1" ] && [ -n "${RECORDED:-}" ] \ + && git -C "$TOOLKIT_DIR" cat-file -e "${RECORDED}^{commit}" 2>/dev/null; then + # The agent asked to SEE it. Show it, record nothing, and say what to do next. + echo "What changed in the shared toolkit since this project last checked:" + echo "" + # shellcheck disable=SC2086 — PROTOCOL_PATHS is a deliberate multi-pathspec list. + git -C "$TOOLKIT_DIR" --no-pager diff --stat "$RECORDED" "$TOOLKIT_REF" -- $PROTOCOL_PATHS + echo "" + # shellcheck disable=SC2086 + git -C "$TOOLKIT_DIR" --no-pager log --oneline "$RECORDED..$TOOLKIT_REF" -- $PROTOCOL_PATHS + echo "" + echo "NOTHING HAS BEEN RECORDED. This was a read-only preview." + echo "Agent: summarise the above for the user in plain language — what changed and what it" + echo "means for what they are building — then ask whether to take the update. Only if they" + echo "say yes, re-run: $0 $PROJECT_DIR --ack-protocol --approved-in-chat" + exit 0 + fi + echo "Refusing to record this without a human having been asked." >&2 + echo "An ack asserts a person was told what changed; nothing here can make that true." >&2 + echo " A) Agent: preview it, ask the user, then record their answer:" >&2 + echo " $0 $PROJECT_DIR --ack-protocol --verbose (shows it, records nothing)" >&2 + echo " $0 $PROJECT_DIR --ack-protocol --approved-in-chat (after they say yes)" >&2 + echo " B) Human at a terminal: $0 $PROJECT_DIR --ack-protocol" >&2 + echo " C) Proceed anyway: $0 --force-stale $PROJECT_DIR ${REQUESTED_STAGE:-}" >&2 echo " — that works, and records a PROTOCOL-BYPASS line in docs/BUILD-LOG.md." >&2 if [ "$STRICT_PROTOCOL" != "1" ]; then echo " Note: without --strict-protocol nothing is blocked anyway — this run can" >&2 - echo " simply proceed, and the notice repeats until a human acks it." >&2 + echo " simply proceed, and the notice repeats until someone answers it." >&2 fi exit 1 fi @@ -1712,8 +1819,10 @@ if [ "$ACK_PROTOCOL" = "1" ]; then # shellcheck disable=SC2086 — PROTOCOL_PATHS is a deliberate multi-pathspec list. git -C "$TOOLKIT_DIR" --no-pager diff --stat "$RECORDED" "$TOOLKIT_REF" -- $PROTOCOL_PATHS echo "" + if [ "$APPROVED_IN_CHAT" = "1" ]; then reply=y; else printf "Print the full diff before acknowledging? [d=diff / y=ack / n=abort] " read -r reply + fi case "$reply" in d|D) # shellcheck disable=SC2086 git -C "$TOOLKIT_DIR" diff "$RECORDED" "$TOOLKIT_REF" -- $PROTOCOL_PATHS @@ -1729,16 +1838,20 @@ if [ "$ACK_PROTOCOL" = "1" ]; then # No usable prior ack (a fresh scaffold, or a sha this clone has never seen). There is no # diff to show, so there is nothing to pretend was read — stamp it and say so in the log. echo "$REGISTER records no toolkit commit this clone can diff from." + if [ "$APPROVED_IN_CHAT" = "1" ]; then reply=y; else printf "Record the current protocol commit %s as this project's baseline? [y/N] " "$TOOLKIT_REF" read -r reply + fi case "$reply" in y|Y) ;; *) echo "Aborted — Toolkit commit line unchanged."; exit 1 ;; esac ACK_STAT="no prior ack to diff from — recorded as a baseline" ACK_FILES="(baseline)" ACK_FROM="${RECORDED:-none}" fi if register_stamp_commit "$TOOLKIT_REF"; then - build_log_append "PROTOCOL-ACK $ACK_FROM -> $TOOLKIT_REF ($ACK_STAT) files: $ACK_FILES" - echo "Acknowledged $TOOLKIT_REF — $REGISTER updated, recorded in $BUILD_LOG." + ACK_HOW="read at a terminal" + [ "$APPROVED_IN_CHAT" = "1" ] && ACK_HOW="approved in chat after the agent summarised it" + build_log_append "PROTOCOL-ACK $ACK_FROM -> $TOOLKIT_REF [$ACK_HOW] ($ACK_STAT) files: $ACK_FILES" + echo "Update taken — $REGISTER now records $TOOLKIT_REF, logged in $BUILD_LOG ($ACK_HOW)." exit 0 fi echo "Could not find a 'Toolkit commit:' line in $REGISTER — add one reading:" >&2 diff --git a/bin/init-project.sh b/bin/init-project.sh index 3277ec0..e42946d 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -285,15 +285,23 @@ session that will touch the pipeline: 1. \`git -C $TOOLKIT_ROOT pull --ff-only\` 2. \`$TOOLKIT_ROOT/bin/gate-check.sh \` — it reports protocol freshness and tells you whether anything you depend on moved. -3. If it says so: re-read the named files, then - \`$TOOLKIT_ROOT/bin/gate-check.sh --ack-protocol\`. That one command shows the - diffstat, offers the full diff, rewrites the \`Toolkit commit:\` line in \`PROJECT.md\`, and - records which files and how many lines you accepted in \`docs/BUILD-LOG.md\`. +3. If it says an update is available, **you (the agent) handle it — never ask the user to type + a command.** Run \`$TOOLKIT_ROOT/bin/gate-check.sh --ack-protocol --verbose\`, + which shows what changed and **records nothing**. Re-read the named files. Then tell the user + in plain language what changed and what it means for what they are building, and ask whether + to take the update. Only if they say yes: + \`$TOOLKIT_ROOT/bin/gate-check.sh --ack-protocol --approved-in-chat\` — that + rewrites the \`Toolkit commit:\` line in \`PROJECT.md\` and records which files, how many + lines, and that it was approved in chat, in \`docs/BUILD-LOG.md\`. 4. State in chat which commit you're working from. -Steps 1-3 used to be a four-step manual ritual; \`--ack-protocol\` replaces the middle of it. -It is interactive on purpose — an ack asserts a human read the diff, so it refuses when there -is no TTY and points an unattended caller at \`--force-stale\`, which works and is logged. +An ack asserts a human was told what changed, so it cannot happen unattended: without a TTY and +without \`--approved-in-chat\` it refuses and names the routes out. \`--approved-in-chat\` is a +claim you are making on the record — set it only after you actually asked and they actually +answered. There is no env-var auto-yes. + +By default the output is written for whoever is in the room, not for a maintainer: plain +language, no commit ids, no file paths. Add \`--verbose\` for the ids, paths and diffstat. **Protocol staleness NEVER blocks a gate.** It prints a notice with lettered options and the stage verdict is reported on its own merits either way — a project is not broken because diff --git a/bin/lib/obligation-check.sh b/bin/lib/obligation-check.sh index 35a032f..4f1bb1e 100755 --- a/bin/lib/obligation-check.sh +++ b/bin/lib/obligation-check.sh @@ -73,36 +73,91 @@ _ob_register_lines() { # Spelling, matching the per-stage waivers gate-check.sh already reads: # Waived obligation look/Orders: integration module, no pages # Waived obligation sweep: this project tests in the client's own suite (all units) +# --- Register parsing: ONCE per file, not once per lookup --------------------------------- +# +# WHY (2026-08-25). _ob_waiver() is called for every (obligation x unit) pair, and each call +# re-read the whole register, forking `tr` AND `sed` for EVERY LINE. Measured on a one-module +# project: 603 `sed` + 345 `tr` out of 1,127 total forks in a single gate-check run — and it +# scaled at roughly +800 forks per module, so it got worse exactly as a project grew. +# +# That is free on Linux/macOS (~4ms a fork) and ruinous under Git Bash on Windows, where MSYS +# emulates fork() with CreateProcess and a spawn measured 152ms on a training laptop. 1,127 +# forks x 152ms = 171s, which is why gate-check "hung" for 5-15 minutes there and got switched +# off. See the same root cause in project-bin/_common.sh's platform note. +# +# The parse is now one awk pass, memoized per register path. Trim + lowercase happen inside +# that pass rather than per line per lookup. +# +# NOTE ON \t: the old `sed 's/^[ \t]*//'` was not portable. BSD sed (macOS) does not read \t as +# tab inside a bracket expression — it read it as the literal characters backslash and 't', so +# a key beginning with 't' had it stripped. GNU sed (Git Bash, Linux) read a real tab. The +# awk below trims real whitespace on every platform, which is what the code always meant. +_OB_REG_NORM="" # cached "keyvalue" lines, key lowercased, both trimmed +_OB_REG_NORM_FILE="" # which register path _OB_REG_NORM was built from + +_ob_register_norm() { # sets _OB_REG_NORM + local reg="$1" + [ "$_OB_REG_NORM_FILE" = "$reg" ] && return 0 + _OB_REG_NORM_FILE="$reg" + _OB_REG_NORM="$(_ob_register_lines "$reg" | awk ' + { i = index($0, ":"); if (i == 0) next + k = substr($0, 1, i-1); v = substr($0, i+1) + gsub(/^[ \t]+/, "", k); gsub(/[ \t]+$/, "", k) + gsub(/^[ \t]+/, "", v); gsub(/[ \t]+$/, "", v) + if (v == "") next + print tolower(k) "\t" v }')" + return 0 +} + +# _ob_lc — sets _OB_LC to the lowercased string. Sets a global instead of echoing +# because "$(_ob_lc x)" would fork a subshell, which is the cost this whole change removes. +# Hand-rolled because bash 3.2 (macOS's /bin/bash) has no ${var,,}. +_OB_LC="" +_ob_lc() { + local s="$1" out="" c i=0 n=${#1} + while [ $i -lt $n ]; do + c=${s:$i:1} + case $c in + A) c=a ;; B) c=b ;; C) c=c ;; D) c=d ;; E) c=e ;; F) c=f ;; G) c=g ;; H) c=h ;; + I) c=i ;; J) c=j ;; K) c=k ;; L) c=l ;; M) c=m ;; N) c=n ;; O) c=o ;; P) c=p ;; + Q) c=q ;; R) c=r ;; S) c=s ;; T) c=t ;; U) c=u ;; V) c=v ;; W) c=w ;; X) c=x ;; + Y) c=y ;; Z) c=z ;; + esac + out="$out$c"; i=$((i+1)) + done + _OB_LC="$out" +} + _ob_waiver() { local reg="$1" ob="$2" unit="$3" line key val want_u want_a # Compared case-insensitively: reg_field() lowercases the key it reads, and a waiver that # silently missed because someone typed "orders" for module "Orders" would be a waiver the # author believes is in force and the gate does not — the worst of both. - want_u="$(printf 'waived obligation %s/%s' "$ob" "$unit" | tr '[:upper:]' '[:lower:]')" - want_a="$(printf 'waived obligation %s' "$ob" | tr '[:upper:]' '[:lower:]')" - while IFS= read -r line; do - key="${line%%:*}"; val="${line#*:}" - [ "$key" = "$line" ] && continue - key="$(printf '%s' "$key" | tr '[:upper:]' '[:lower:]' | sed 's/^[ \t]*//;s/[ \t]*$//')" - val="$(printf '%s' "$val" | sed 's/^[ \t]*//;s/[ \t]*$//')" - [ -n "$val" ] || continue + _ob_lc "waived obligation $ob/$unit"; want_u="$_OB_LC" + _ob_lc "waived obligation $ob"; want_a="$_OB_LC" + _ob_register_norm "$reg" + local IFS=$'\n' + for line in $_OB_REG_NORM; do + key="${line%%$'\t'*}"; val="${line#*$'\t'}" if [ "$key" = "$want_u" ] || [ "$key" = "$want_a" ]; then printf '%s\n' "$val"; return 0 fi - done < <(_ob_register_lines "$reg") + done return 1 } _ob_adopted_stage() { local reg="$1" line key val - while IFS= read -r line; do - key="${line%%:*}"; val="${line#*:}" - [ "$key" = "$line" ] && continue - key="$(printf '%s' "$key" | tr '[:upper:]' '[:lower:]' | sed 's/^[ \t]*//;s/[ \t]*$//')" + _ob_register_norm "$reg" + local IFS=$'\n' + for line in $_OB_REG_NORM; do + key="${line%%$'\t'*}"; val="${line#*$'\t'}" [ "$key" = "adopted at stage" ] || continue - printf '%s\n' "$(printf '%s' "$val" | sed 's/^[ \t]*//;s/[ \t]*$//' | awk '{print $1}')" + # First whitespace-delimited word, as `awk '{print $1}'` gave. Value is already trimmed, + # so the leading-blank case awk tolerated cannot arise here. + printf '%s\n' "${val%%[ ]*}" return 0 - done < <(_ob_register_lines "$reg") + done return 1 } diff --git a/bin/sync-project.sh b/bin/sync-project.sh index 3036d3d..099858f 100755 --- a/bin/sync-project.sh +++ b/bin/sync-project.sh @@ -125,7 +125,13 @@ while [ $# -gt 0 ]; do --upgrade-bin) shift [ $# -gt 0 ] || { echo "--upgrade-bin needs a script name or 'all'" >&2; exit 1; } - UPGRADE_BIN="$1"; shift ;; + # ACCUMULATE, do not assign. This was `UPGRADE_BIN="$1"`, so `--upgrade-bin _common.sh + # --upgrade-bin exec.sh` silently upgraded only exec.sh and left _common.sh drifted -- + # with no warning, because the drift report is suppressed as soon as UPGRADE_BIN is + # non-empty. That is the worst possible shape: the one file carrying the Windows mxbuild + # fix stayed stale while the run looked like it had succeeded. Found 2026-08-26 by + # actually running the command a workshop prompt was about to hand out. + UPGRADE_BIN="$UPGRADE_BIN $1"; shift ;; --upgrade-lint-rules) shift [ $# -gt 0 ] || { echo "--upgrade-lint-rules needs a rule file name or 'all'" >&2; exit 1; } @@ -643,6 +649,10 @@ known_fix_note() { case "$1" in verify-module.sh) echo "bin/verify-module.sh is missing the design-audit wiring fix (toolkit, 2026-08-21) — wires tests/e2e/design-audit.js (rungs 6-7, UI/a11y) into the composed pass as an informational, non-gating rung. Recommended upgrade." ;; + _common.sh) + echo "bin/_common.sh predates the WINDOWS MXBUILD GATE fix (toolkit, 2026-08-25). This is where find_sp_app/find_mxbuild, JAVA_HOME resolution and mxtk_platform actually live — exec.sh only calls them. So upgrading exec.sh ALONE does not deliver the fix, and grepping exec.sh for mxtk_platform will report 0 even on a fully patched project: grep _common.sh instead. Without this file, the mxbuild gate is skipped on every Windows exec and nothing checks your builds. Upgrade BOTH: --upgrade-bin _common.sh --upgrade-bin exec.sh." ;; + exec.sh) + echo "bin/exec.sh predates two fixes worth naming. (1) The WINDOWS MXBUILD GATE (toolkit, 2026-08-25): find_sp_app/find_mxbuild and JAVA_HOME resolution were macOS-only, so under Git Bash the gate block was skipped entirely and every exec on a Windows machine went UNVERIFIED — it reported 'skipped', not a false pass, but a skip nobody acts on is the same outcome. (2) The MODULE-BRIEF GUARD (toolkit, 2026-08-25): refuses a write to a module with no module-brief.md (or no '## Module brief — ' section in the build plan), overridable with FORCE_EXEC=1. Strongly recommended upgrade on Windows — without (1) nothing checks your builds. NOTE: (1) mostly lives in _common.sh, so upgrade that too or the fix is incomplete." ;; esac } @@ -699,7 +709,7 @@ elif [ -d "$CRASHNET_SRC" ]; then DRIFTED="$DRIFTED $s" add=$(diff "$src" "$dst" | grep -c '^>' || true) del=$(diff "$src" "$dst" | grep -c '^<' || true) - if [ "$UPGRADE_BIN" = "all" ] || [ "$UPGRADE_BIN" = "$s" ]; then + if case " $UPGRADE_BIN " in *" all "*|*" $s "*) true ;; *) false ;; esac; then if [ "$DRY_RUN" -eq 1 ]; then echo "Would upgrade: bin/$s (+$add/-$del local) — local copy would be backed up first" else diff --git a/bug-logs/mxcli-bugs.md b/bug-logs/mxcli-bugs.md index ea0c268..72d0a1f 100644 --- a/bug-logs/mxcli-bugs.md +++ b/bug-logs/mxcli-bugs.md @@ -4624,3 +4624,215 @@ None found via MDL. Either: action button to open a related detail page with the current run passed as a parameter, outside any dataview), confirmed via `DESCRIBE PAGE` round-trip and two independent rewrite attempts before reverting the widget to restore a green build. + +## BUG-96: full page regeneration never wires a parameterized microflow datasource on a top-level `dataview` — only implicit binding through nesting works + +### Summary + +A `dataview` whose `DataSource: microflow X` requires **any** parameter cannot have that +parameter wired by a full `create or modify page` regeneration, when the dataview is a direct +child of the page body (not nested inside another dataview/snippet). Confirmed even when the +page is reproduced **byte-for-byte identical to its own pre-existing, working original** — no +edits at all — native `mx check` still throws `CE1571 "No argument has been selected for +parameter '...' and no default is available."` for every such parameter. `mxcli check +--references` and the exec's own "Created page ..." success message both stay silent; this is +caught only by a real native `mx check` (or Studio Pro's own error pane). + +### What does NOT fix it + +- Adding an explicit `Params: { Param: $value }` clause on the dataview: `mxcli check` accepts + this syntactically (no error), but the writer still drops the mapping — native check still + fails identically. +- Using an enum literal as a `Params:` value (e.g. `Params: { StationKey: + Approval.StationKey.WFST010 }`): rejected outright by `mxcli check` itself + (`mismatched input 'Approval' expecting VARIABLE`) — `Params:` accepts only `$variable` + references, never literals, on any binding. +- Reducing the microflow to a single parameter, while keeping the dataview a direct child of + the page body: still fails. Parameter *count* is not the variable — nesting is. + +### What does fix it — implicit binding through nesting + +The writer *can* wire a microflow-datasource dataview's parameter, but only when the dataview +is nested one level inside another dataview/snippet whose own current-object type exactly +matches the microflow's sole parameter type, and the inner dataview has **no** `Params:` clause +at all: + +``` +dataview dvOuter (DataSource: $PageParam) { + dataview dvInner (DataSource: microflow Module.SingleParamMicroflow) { + -- $currentObject here is whatever SingleParamMicroflow returns + } +} +``` + +`SingleParamMicroflow` must take exactly one parameter, of the same type as `$PageParam` +(or whatever the outer dataview's current-object type is). If the real business logic needs +more inputs than that one type provides (e.g. an enum literal that varies per page instance), +write a thin single-parameter wrapper microflow that hardcodes the rest internally via a normal +`call microflow` expression — enum literals *are* valid inside a microflow body, just never +inside a page's `Params:` clause. + +Verify the mechanism on a disposable throwaway page first (create it, native `mx check`, then +`DROP PAGE` it) before rolling out to real pages — a clean throwaway page next to N still-broken +real ones in the same `mx check` run isolates the fix from everything else in flight. + +### Relationship to other bugs + +Same failure signature and root category as BUG-95 (`show_page` action ignoring the named +variable, defaulting to `$currentObject`) and the already-documented "snippetcall doesn't +auto-infer Params on full regen" / "`ALTER PAGE REPLACE` silently unbinds `Attribute:` +shorthand" findings in `skills/datagrid-customcontent-text-binding.md` — all are instances of +mxcli's writer silently failing to wire a parameter/argument mapping that Studio Pro's own GUI +always forces the user to complete, while `mxcli check` has no way to see the gap. Distinct from +BUG-56 (DataGrid2 datasource parameterized-microflow binding, resolved v0.17.0) — this is a +plain `dataview`, not a DataGrid2 grid, and remains open as of v0.18.0. + +### Workaround + +Use the nested-nesting pattern above. Do not attempt a third variation of the `Params:` clause +on a non-nested dataview — the defect is structural (missing implicit-binding pass for +top-level dataviews), not a syntax problem. + +**Discovered:** 2026-08-21, a client project (15 native-Workflow "station task" pages, each +needing a dataview scoped to a lookup microflow keyed by the page's `WorkflowUserTask` +parameter plus a per-page enum literal), confirmed via a disposable throwaway test page that +isolated the nesting mechanism as the actual fix, after a full 15-page rebuild attempt without +nesting produced `CE1571` on all 15 despite `mxcli check --references` passing clean throughout. + +--- + +## BUG-97: `ALTER PAGE REPLACE` cannot reliably bind a multi-entry `ContentParams` array on a gallery/listview-template `dynamictext`, even with a distinct widget name and bare attribute names + +**Severity:** Medium — silent failure (`mxcli check --references` passes, `mxcli exec` reports success, only native `mx check` catches it via CE0402) +**Reproducible:** Yes +**Mendix version:** 11.13.0 +**mxcli version when found:** v0.18.0 + +### Steps to reproduce + +1. On a page with a `gallery`/`listview` template containing a working `dynamictext` with a + single-entry `ContentParams` (e.g. the pre-existing `txtChainLabel (Content: '{1}', ContentParams: + [{1} = StationKey])` sibling widget) — confirming the bare-attribute-name convention already + works in this exact template context. +2. Add a *new* `dynamictext` in the same template via + `ALTER PAGE Module.Page { REPLACE "oldWidget" WITH { DYNAMICTEXT "newDistinctName" (Content: '{1} · {2}', ContentParams: [{1} = Attr1, {2} = Attr2], Class: 'hint') } }`, + using a placeholder/unbound `oldWidget` as the anchor, a genuinely distinct replacement name + (ruling out BUG-08's duplicate-name case), and bare (unqualified) attribute names for both + `ContentParams` entries (ruling out BUG-23's `$currentObject/` prefix case). +3. Run `mxcli check