Skip to content

test: pin the enduser-landscape classifier's role and scope rules - #1771

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-enduser-landscape-mutants
Oct 22, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-enduser-landscape-mutants

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

scripts/lib/enduser-landscape.mjs — the classifier that turns cncf/landscape
entries into End User directory records — scored 83.54% under StrykerJS,
with 27 of 164 mutants surviving. The repo's break threshold is 83, so the
module was sitting half a point above the gate while its central rules went
unasserted.

This adds six tests to tests/enduser-landscape.test.mjs (no production code
changes) covering the behavioural survivors:

  • ROLE_SUFFIX is anchored to the end of the name, so Acme (member) Holdings
    is not classified as a member and its display name is not rewritten from the
    middle.
  • A whitespace-only item name is rejected (sourceName.trim()), not emitted as
    a blank directory card.
  • The unnamed-item error names its subcategory, falling back to <unknown>
    rather than interpolating undefined.
  • The emitted enduser flag mirrors the landscape's own flag and is asserted
    apart from included (a contributor is included without it).
  • A (supporter) suffix outside End User Supporter and Contributor is
    ignored — the authoritative-subcategory half of the relevance rule.
  • A stray null landscape entry, subcategory or item is skipped or reported as
    a data error, not a TypeError naming no entry.

Verified locally at b490a7f:

TZ=UTC npx stryker run --mutate 'scripts/lib/enduser-landscape.mjs' --concurrency 4
before: 83.54%  (137 killed, 27 survived)
after:  93.90%  (154 killed, 10 survived)

The 10 remaining mutants are equivalent: item is provably non-null past the
name guard, and the || [] fallbacks on document.landscape /
category.subcategories reach the same thrown error either way. They are not
worth contorting the tests to kill.

TZ=UTC node --test tests/enduser-landscape.test.mjs, npx prettier --check
and npx eslint --max-warnings 0 are green on the changed file.

This touches only tests/enduser-landscape.test.mjs and no production source,
so it is disjoint from the open hold-gated PRs (#1761 tests/validate-utils,
#1765 tests/svg-active-content, #1767 scripts/lib/duplicate-check and four
validators, #1763 scripts/audit-gate, #1769 scripts/pr-queue-hygiene).

Related Issue

Closes #1770


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

scripts/lib/enduser-landscape.mjs scored 83.54% under StrykerJS with 27 of
164 mutants surviving -- half a point above the repo's break threshold of
83. The survivors were behavioural: the ROLE_SUFFIX end anchor, the
sourceName.trim() emptiness check, the '<unknown>' subcategory fallback in
the unnamed-item error, the record's mirrored enduser flag, the
authoritative-subcategory half of the supporter relevance test, and the
optional chaining that keeps a stray null YAML list entry from crashing the
collector.

Add tests that assert each of those decisions directly. The module now
scores 93.90%; the 10 mutants that remain are equivalent -- item is
provably non-null past the name guard, and the '|| []' fallbacks reach the
same thrown error either way.

Closes #1770

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Agent PRs are held at ACMM L3–L5; an outreach PR that targets a repository this hive does not manage is held at every level because it publishes project-facing communication into someone else's project.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@netlify

netlify Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for endusers ready!

Name Link
🔨 Latest commit b10cacf
🔍 Latest deploy log https://app.netlify.com/projects/endusers/deploys/6ad9cae49ee67d07303721f5
😎 Deploy Preview https://deploy-preview-1771--endusers.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the classifier at scripts/lib/enduser-landscape.mjs and traced each new test against it; all six pin behavior the source actually implements. Looks correct to me.

Verified against the tree at b10cacf:

  • ROLE_SUFFIX (enduser-landscape.mjs:5) is $-anchored, so Acme (member) Holdings with enduser: true matches no role and hits the enduser record without a member role suffix throw at line 80 — exactly what the first test asserts, and the trailing-suffix variant strips only the final (member) via displayName() (line 8).
  • The unnamed-item message interpolates subcategoryName || '<unknown>' (line 45), so both the named and fallback assertions match the real template.
  • !sourceName.trim() (line 43) rejects the whitespace-only name; a null item makes item?.name undefined and takes the same path, and entry?.name / subcategory?.items || [] (lines 25, 41) skip null categories/subcategories — matching the null-tolerance test.
  • A (supporter) suffix in Gold is neither included nor relevant (lines 84–93), so the empty-records assertion holds; enduser: item?.enduser === true (line 112) is independent of included, so asserting a contributor record with enduser: false is the right separation.

Ran TZ=UTC node --test tests/enduser-landscape.test.mjs on the PR head: 15 pass, 0 fail. The diff touches only the test file — the body's claim of no production changes and disjointness from #1761/#1763/#1765/#1767/#1769 checks out. All required checks (including Mutation testing) are green.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

@mrbobbytables

Copy link
Copy Markdown
Member

Maintainer-delegated P2 disposition: reviewed exact head b10cacf against classifyLandscapeDocument. The added assertions distinguish trailing-role anchoring, nonblank names, null data diagnostics, supporter subcategory scope and the landscape enduser flag from inclusion. They exercise production classifier behavior rather than test-tree guards; no production semantics or policy changes. Required CI and all fresh-head checks are green. The maintainer explicitly approved this designated P2 hold release and protected merge. Agent inspection is not independent human approval. Releasing only this hold and submitting the exact head through the queue, with no bypass.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 26ed06a Oct 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] enduser-landscape classifier: 27 surviving mutants leave role-suffix, supporter-scope and enduser-flag rules unasserted

1 participant