Remaining calibration work from the 2026-07-08 audit (High#3 + Medium#11) — the quality score no longer discriminates (catalog median 89 vs the rubric's own 72–78 target; 87% of 3,261 scores ≥85).
Design (worked out 2026-07-10, blocked on #9623 merging first — same prompt regions):
- AR score-0 collision (Medium#11): AR-06/08/09 legitimately set score 0, but
impl-review.yml treats score 0 as "invalid output" → retry → ai-review-failed, so legit rejections never reach impl-repair. Fix: the review prompt additionally writes an auto_reject.txt sentinel (AR id + one-line reason); the extract-score step maps score 0 → 1 when the sentinel exists, so AR rejections flow into the normal rejected path (quality label, ai-rejected, repair cascade). Known tradeoff: AR-06 (NOT_FEASIBLE) wastes repair runs — still better than the current dead end.
- Store
first_review_score (High#3a): in impl-review.yml's update_metadata.py, pass ATTEMPT as argv[4]; write first_review_score on attempt 1 (overwrites on regeneration — correct). Lets us measure repair-loop lift and reviewer drift. Postgres sync can pick the field up later (automation/scripts/sync_to_postgres.py whitelist).
- Calibration anchors (High#3b): extend the Anti-Inflation section in
prompts/quality-criteria.md with 2–3 concrete anchors (~60 / ~75 / ~90) — textual rationale primarily, preview URL as optional illustration (no pipeline dependency on fetching images).
- Worst-first daily-regen (High#3c):
daily-regen.yml picks oldest by updated and only scans metadata/python/ (pre-15-lib relic). Change: scan metadata/*/*.yaml, sort candidates by mean quality score ascending (tie-break oldest-first), keep the min_age_hours guard. Accept rotation among the lowest bucket.
Audit: agentic/audits/2026-07-08-product-ux.md (High#3, Medium#11). Prior art: #9623 fixed the rubric's 15-library drift.
Remaining calibration work from the 2026-07-08 audit (High#3 + Medium#11) — the quality score no longer discriminates (catalog median 89 vs the rubric's own 72–78 target; 87% of 3,261 scores ≥85).
Design (worked out 2026-07-10, blocked on #9623 merging first — same prompt regions):
impl-review.ymltreats score 0 as "invalid output" → retry →ai-review-failed, so legit rejections never reach impl-repair. Fix: the review prompt additionally writes anauto_reject.txtsentinel (AR id + one-line reason); the extract-score step maps score 0 → 1 when the sentinel exists, so AR rejections flow into the normal rejected path (quality label,ai-rejected, repair cascade). Known tradeoff: AR-06 (NOT_FEASIBLE) wastes repair runs — still better than the current dead end.first_review_score(High#3a): inimpl-review.yml'supdate_metadata.py, passATTEMPTas argv[4]; writefirst_review_scoreon attempt 1 (overwrites on regeneration — correct). Lets us measure repair-loop lift and reviewer drift. Postgres sync can pick the field up later (automation/scripts/sync_to_postgres.pywhitelist).prompts/quality-criteria.mdwith 2–3 concrete anchors (~60 / ~75 / ~90) — textual rationale primarily, preview URL as optional illustration (no pipeline dependency on fetching images).daily-regen.ymlpicks oldest byupdatedand only scansmetadata/python/(pre-15-lib relic). Change: scanmetadata/*/*.yaml, sort candidates by mean quality score ascending (tie-break oldest-first), keep themin_age_hoursguard. Accept rotation among the lowest bucket.Audit:
agentic/audits/2026-07-08-product-ux.md(High#3, Medium#11). Prior art: #9623 fixed the rubric's 15-library drift.