Skip to content

Commit cd01580

Browse files
dmealingclaude
andcommitted
fix(java): a test read files another test class writes, with nothing declaring the order
`conformance`'s full-reactor job went red a second time — a different module, a different test, the same shape as the last one: WorkingSchemaValidationTest.demonstrateConstraintSystemSuccess:124 Should have substantial constraint evidence in generated schemas It had been hidden behind the failure fixed in `1c24b8f9d`: Maven fails fast, so the earlier run died in `metaobjects-metadata` and never built `codegen-base`. The three tests in this class read `target/working-{metadata-schema,ai-documentation} .json`, which `WorkingSchemaGeneratorTest` writes. Nothing declared that order, so it was surefire's `runOrder` — default `filesystem`, i.e. whatever order the class files happen to sit in on THAT machine. Reproduced locally with `-Dsurefire.runOrder=reversealphabetical` (Validation sorts after Generator, so reversing puts the consumer first): green by default, red reversed, identical to the runner. **Both failure modes here are bad, and they are opposite.** The test that failed logs a warning for the missing file, `continue`s, and then asserts on an evidence count of zero — so an absent PRECONDITION is reported as "Should have substantial constraint evidence in generated schemas", which reads like a codegen regression and sends you looking in the wrong place. The other two `return` early and PASS, having checked nothing at all: on the machine where the files are missing, this class silently validates no schema. Fixed by generating the artifacts in `@BeforeClass` — the pattern the two sibling consumers in this module (`SimpleSchemaValidationTest`, `SchemaValidationTest`) already use, so this closes the last door rather than adding a new rule. The early `return`s become unreachable instead of load-bearing, and those two tests now assert for real. Verified from a clean `target/` under all four run orders — default, alphabetical, reversealphabetical (the one that reproduced CI), and random: 70 tests, 0 failures each. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NTcEKXTQMYt84fAjuw5A2M
1 parent 1c24b8f commit cd01580

1 file changed

Lines changed: 36 additions & 0 deletions

File tree

server/java/codegen-base/src/test/java/com/metaobjects/generator/WorkingSchemaValidationTest.java

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package com.metaobjects.generator;
22

33
import org.junit.Before;
4+
import org.junit.BeforeClass;
45
import org.junit.Test;
56
import org.slf4j.Logger;
67
import org.slf4j.LoggerFactory;
@@ -18,6 +19,41 @@ public class WorkingSchemaValidationTest {
1819

1920
private static final Logger log = LoggerFactory.getLogger(WorkingSchemaValidationTest.class);
2021

22+
/**
23+
* Generate the artifacts this class validates, rather than hoping another test class
24+
* already did.
25+
*
26+
* These tests read `target/working-{metadata-schema,ai-documentation}.json`, which are
27+
* written by {@link WorkingSchemaGeneratorTest}. Nothing declared that order, so it was
28+
* surefire's `runOrder` — which defaults to `filesystem`, i.e. whatever order the class
29+
* files happen to sit in on THAT machine. Reproduced 2026-09-08: green on a developer
30+
* box from a clean `target/`, RED in the hosted full-reactor job, and red locally under
31+
* `-Dsurefire.runOrder=reversealphabetical` (Validation sorts after Generator, so
32+
* reversing puts the consumer first).
33+
*
34+
* The failure was also mute about its cause. `demonstrateConstraintSystemSuccess`
35+
* logs a warning for a missing file, `continue`s, and then asserts on the evidence
36+
* count — so an absent PRECONDITION was reported as "Should have substantial constraint
37+
* evidence in generated schemas", which reads like a codegen regression. The other two
38+
* tests in this class have the opposite failure mode: they `return` early and PASS,
39+
* having checked nothing.
40+
*
41+
* Generating here fixes both. It is the pattern {@code SimpleSchemaValidationTest}
42+
* already uses, and it makes the early `return`s below unreachable rather than
43+
* load-bearing.
44+
*/
45+
@BeforeClass
46+
public static void generateSchemas() throws Exception {
47+
File jsonSchemaFile = new File("target/working-metadata-schema.json");
48+
File aiDocFile = new File("target/working-ai-documentation.json");
49+
if (jsonSchemaFile.exists() && aiDocFile.exists()) return;
50+
51+
log.info("Schema artifacts absent — generating them before validating.");
52+
WorkingSchemaGeneratorTest generator = new WorkingSchemaGeneratorTest();
53+
generator.setUp();
54+
generator.generateWorkingSchemas();
55+
}
56+
2157
@Before
2258
public void setUp() {
2359
log.info("=== SCHEMA VALIDATION TEST SETUP ===");

0 commit comments

Comments
 (0)