Skip to content

feat(referee): list the api-json matches a run lost and gained, and record why naming-feature keys cost manual-examples 171 - #1323

Merged
HuiJun merged 8 commits into
developfrom
fix/api-json-naming-feature-keys
Oct 21, 2026
Merged

HuiJun merged 8 commits into
developfrom
fix/api-json-naming-feature-keys

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

What and why

Naming a member by its naming feature (#1304) raised the api-json differential's matched elements on every corpus root but docs/manual/examples, which fell 6834 → 6663. This PR adds the referee diagnostic that names the moved matches, uses it to account for every one of them, and records the finding.

No derivation of ours disagrees with the spec. The 171 is the net of 357 matches lost and 186 gained:

movement elements shape verdict
lost 357 = 87 members + the 270 elements they own 84 AttributeUsage + 3 ReferenceUsage, each attribute redefines caption|title|text|term|description|kind|direction|style|groupBy = … or ref redefines source|target = … of a feature of DocumentQueries (internal/workspace/libs/stdlib/OpenSysML Libraries/), with their Redefinition, FeatureValue and literal / FeatureReferenceExpression value referee limitation. Ours keys them …::coverage::title per KerML 8.3.3.3.4 Feature::effectiveName (namingFeature().effectiveName() when nothing is declared) with the default namingFeature() = ownedRedefinition->at(1).redefinedFeature. The pilot's export shows name/qualifiedName null and the Redefinition's redefinedFeature pointing at an id outside its key table — the bridge loads only the pinned sysml.library, so it cannot resolve this repository's own libraries and keys the member structurally (…::coverage/AttributeUsage[0]). The earlier match was that structural key lining up with our then-positional one. All 87 verified in the pilot's JSON.
gained 186 = 46 members + 138 owned + 2 46 attribute :>> x = … redefining features the model declares (with Redefinition, FeatureValue, 27 LiteralString / 19 LiteralRational), plus render asInterconnectionDiagram; (SysML 8.3.26.6 RenderingUsage::namingFeature) and its ReferenceSubsetting the intended effect of #1304; both sides now derive the same name

Specification basis

Clauses read for the adjudication: KerML 8.3.2.1.2 Element (effectiveName, name = effectiveName()), 8.3.3.3.4 Feature (effectiveName, namingFeature); SysML 8.3.6.4 Usage (variant), 8.3.6.3 ReferenceUsage (transition payload), 8.3.17.14 PerformActionUsage, 8.3.20.4 ConstraintUsage, 8.3.26.6 RenderingUsage. Nothing was made stricter than the spec; no derivation code changed.

The same shape on the other seven roots

Before → after #1304, fresh cache per tree, every root (the -lost-against listing of the after run against the before run's inventory):

root matched before → after lost gained lost members and their shape
training 6097 → 7287 0 1190 —
pilot-examples 19299 → 25778 0 6479 —
pilot-validation 6997 → 9246 0 2249 —
kerml-examples 5399 → 6387 0 988 —
spec-examples 6654 → 6768 57 171 13 members whose naming feature the pinned excerpt leaves undeclared (perform VehicleActions::move :> … 7.17.6/2, exhibit VehicleStates::monitoring 7.18.4/2, attribute :>> massActual under an undefined MaximumMass 7.21.2/3, render asMyTreeDiagram 7.26.2/2–3, frame 'modularity' 7.26.3/2, individual … redefines aircraft|preflight|… 7.9.4/2, step redefines focus KerML 7.4.7.3/3) + what they own; the pilot resolves nothing and names nothing, we name by the reference as written (the same reference we export as @ref)
testdata 1124 → 1165 0 41 —
examples 21488 → 23379 914 2805 205 DocumentQueries / AnalysisRecords redefinitions (OpenSysMLDocument, Records, Reporting, RoverChecks, RelationshipMatrixReport) + 2 render asMixedDiagram|asCaseDiagram of OpenSysMLRenderings (LanderViews), + what they own
manual-examples 6834 → 6663 357 186 the 87 above

Every lost member on every root is the one shape — a naming feature the pilot does not resolve (an OpenSysML-only library it does not load, or a feature the excerpt never declares) — verified element by element in the pilot's JSON (name/qualifiedName null, redefinedFeature/referencedFeature id outside its key table). No lost member has a naming feature both sides resolve.

Diagnostic

Every run now writes pilot-apijson-diff-matches.json (per root: every key under matched / ours-only / pilot-only with its metaclass). -lost-against <earlier run>/pilot-apijson-diff-matches.json writes pilot-apijson-diff-movements.txt|json: the matches lost and gained since that run, grouped by root and metaclass, each with the key's fate in the run that does not match it. Tested without Java (TestMatchMovementsNameEveryMovedKey, TestCompareRootRecordsTheInventory) and end to end: a run of this branch against the pre-#1304 inventory lists exactly the 357 / 186 above.

Record

docs/project/api-json-differential.md gains "Where the pilot cannot name a member" (the limitation, the clauses, the counts) and documents the new files and flag. The baseline is unchanged (already 6663 on the base branch). Changelog fragment changes/unreleased/api-json-match-movements.added.md.

Behaviour changes

None in the writer or the semantics; the referee's report and baseline shape are unchanged (the inventory is an extra file).

Overlapping work checked

Follow-up (not in this PR)

Loading OpenSysML Libraries/ into the bridge alongside the pinned sysml.library would let the pilot resolve these features and turn the 87 into real comparisons; it changes the bridge source (and so its recorded digest) and the baseline of two roots, so it is a change of its own.

How it was verified

  • go build ./..., go vet ./..., gofmt -l . (empty), go test ./..., go test -C tools ./..., make lint, make docs-counts, python3 scripts/changelog.py check — all pass.
  • go run -C tools ./cmd/pilot-apijson-diff -check reproduces the committed baseline on every root.
  • OPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 go test -count=1 ./tests/corpus -run TestCorpusRoundTrip passes (no export change, nothing moved).
  • The inventory and -lost-against are pinned without Java by TestMatchMovementsNameEveryMovedKey and TestCompareRootRecordsTheInventory, and exercised end to end as described above (a run of this branch over docs/manual/examples with -lost-against the pre-fix(export): name a view's render tree; and other naming-feature members by their effective name #1304 inventory lists exactly 357 lost / 186 gained, every lost member re-read in the pilot's own JSON).

Checklist

  • make test and make lint pass locally
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved (no count moved; the baseline is unchanged)
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/556a00af681842bf9d52a532275fc761
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/556a00af681842bf9d52a532275fc761?variant=devin
Requested by: @HuiJun

devin-ai-integration Bot and others added 4 commits October 20, 2026 18:53
…ost against an earlier run

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…1 api-json matches

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…-json root

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 20, 2026 19:17
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 3 commits October 20, 2026 19:20
… movement groups once, drop stale movement files

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…gosec

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Base automatically changed from feat/spec-examples-oracle to develop October 20, 2026 21:03
devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant