Skip to content

Add 2.0 CBOM fixtures for family-only observations - #1176

Open
Lennonhaha wants to merge 3 commits into
CycloneDX:2.0-devfrom
Lennonhaha:cbom-family-only-fixtures-2.0
Open

Lennonhaha wants to merge 3 commits into
CycloneDX:2.0-devfrom
Lennonhaha:cbom-family-only-fixtures-2.0

Conversation

@Lennonhaha

@Lennonhaha Lennonhaha commented Oct 26, 2026 •

Copy link
Copy Markdown

Hi @Mehrn0ush,

As discussed in #1175, here are two 2.0 fixtures for family-only observations, where the NIST category is omitted rather than asserted.

  • valid-cbom-family-only-mlkem-2.0.json — algorithmFamily: "ML-KEM", primitive: "kem", no parameter set
  • valid-cbom-family-only-sm2-2.0.json — algorithmFamily: "SM2", primitive: "signature"

Per your guidance, neither fixture carries a nistQuantumSecurityLevel value, and neither uses a custom namespaced property, so they validate against the 2.0 schema alone. That keeps the distinction you drew: omission makes no claim, while 0 means none of the categories are met.

Both live under tools/src/test/resources/2.0 and use the valid- prefix, because AbstractJsonSchemaVerificationTest only loads resources whose names start with valid or invalid; a free-form name would be silently skipped.

The commit carries a DCO sign-off.

Local verification controls

Per your review request, here is exactly what was run, against which revision, and what it produced. The controls are local checks only — they are not committed as fixtures.

Schema revision: 2.0-dev at 76f185c45deb28c98b2555da9c752b9d43f50ecd, file schema/2.0/cyclonedx-2.0-bundled.schema.json ($id https://cyclonedx.org/schema/2.0/cyclonedx-2.0-bundled.schema.json, $schema draft 2020-12).

Validator: ajv 8.20.0 with ajv-formats 3.0.1, new Ajv2020({ strict: false, validateSchema: false, allErrors: true }). The companion schemas (cryptography-defs, behavior-taxonomy, spdx, jsf-0.82) are registered from schema/ under both https://cyclonedx.org/schema/<name> and https://cyclonedx.org/schema/2.0/<name>; validateSchema is disabled because the two draft-07 companions cannot be meta-validated by a 2020-12 instance.

Result (exit code 0, five cases):

case expect result
valid-cbom-family-only-mlkem-2.0.json as committed valid valid
valid-cbom-family-only-sm2-2.0.json as committed valid valid
control: algorithmFamily: "ML-KEM-NOT-A-FAMILY" invalid invalid
control: nistQuantumSecurityLevel: 7 invalid invalid
control: nistQuantumSecurityLevel: 0 valid valid

One caveat so the claim is not overstated: ajv-formats 3.0.1 does not provide iri-reference or idn-email, and Ajv reports those as ignored. No field in these two fixtures uses either format, and the structural assertions above are unaffected, but those two formats are not enforced by this local run.

Change since the first revision

The initial descriptions said the omission existed "because the assessment was not performed", which asserts an assessment status that omission cannot establish, and implies SM2 cannot be assessed. Both now use your suggested wording verbatim:

Algorithm family observed without a recorded parameter set or asserted NIST security strength category. Omission of nistQuantumSecurityLevel does not establish whether an assessment was performed.

Happy to adjust naming, placement or scope if you would prefer a different layout.

@Lennonhaha
Lennonhaha requested a review from a team as a code owner October 26, 2026 01:52
"bom-ref": "ml-kem-family-only",
"type": "cryptographic-asset",
"name": "ML-KEM",
"description": "Algorithm family observed without a parameter set and without a NIST category. nistQuantumSecurityLevel is omitted because the assessment was not performed; the omission is not a statement that no category is met.",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description explicitly asserts that an assessment was not performed. That conflicts with the PR description’s statement that these fixtures “say nothing about assessment status.”

An unperformed assessment is a legitimate example, but it is a narrower scenario than the neutral omission case discussed in #1175. To demonstrate that omission itself establishes neither a category nor assessment status, please remove the asserted reason for omission. Also, “without a NIST category” would be clearer as “without an asserted NIST category,” since the fixture does not establish that the algorithm lacks a category.

Suggested change
"description": "Algorithm family observed without a parameter set and without a NIST category. nistQuantumSecurityLevel is omitted because the assessment was not performed; the omission is not a statement that no category is met.",
"description": "Algorithm family observed without a recorded parameter set or asserted NIST security strength category. Omission of nistQuantumSecurityLevel does not establish whether an assessment was performed.",

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, and this is the same error the fixture pair exists to prevent: I wrote the omission as evidence of "not assessed", which is an inference the schema does not support. Omission records that no assertion was made — nothing about whether anyone looked.

I adopted your suggested wording verbatim in both fixtures:

Algorithm family observed without a recorded parameter set or asserted NIST security strength category. Omission of nistQuantumSecurityLevel does not establish whether an assessment was performed.

Pushed as 055d96d. I also added the local control commands, the schema revision (2.0-dev at 76f185c4) and their results to the PR description, including the caveat that ajv-formats does not provide iri-reference/idn-email.

"bom-ref": "sm2-family-only",
"type": "cryptographic-asset",
"name": "SM2",
"description": "Algorithm family observed without a parameter set and without a NIST category. nistQuantumSecurityLevel is omitted because the assessment was not performed; the omission is not a statement that no category is met.",

@Mehrn0ush Mehrn0ush Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same correction as the ML-KEM fixture. This example only needs to show that no category assertion was recorded. It shouldn’t state a reason for omission or imply SM2 can’t be assessed.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same correction applied here: the SM2 fixture now uses the neutral wording rather than implying SM2 cannot be assessed or that no assessment happened.

Algorithm family observed without a recorded parameter set or asserted NIST security strength category. Omission of nistQuantumSecurityLevel does not establish whether an assessment was performed.

Pushed as 055d96d, together with the local control details in the PR description.

@Mehrn0ush Mehrn0ush left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lennonhaha.
Thanks.
The two fixtures cover the omitted-category case for a family-only ML-KEM and SM2 observation, with no nistQuantumSecurityLevel and no namespaced property.

The controls from the PR description (bogus family, level 7, explicit 0) aren’t in the commit, so they’re local checks. Could you add the command or script and the schema revision to the PR description? That keeps them reproducible and separate from the committed fixtures.

I left inline comments on both description values: they state the assessment was not performed, but omission establishes nothing about that. The suggested wording describes only the recorded observation.

@Mehrn0ush Mehrn0ush left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I reviewed 055d96d. Both fixture descriptions now correctly distinguish an omitted category from assessment status. This addresses my requested changes, and I approve the fixture changes.

@Lennonhaha

Copy link
Copy Markdown
Author

The workflows on this PR are stopped at action_required — GitHub is waiting for a maintainer to approve the run, so none of the CT / Lint / Build jobs have ever executed (only DCO and GitGuardian report status). mergeStateStatus has read UNSTABLE since 26 Oct for that reason, not because a check failed.

@Mehrn0ush @jkowalleck when one of you has a moment, could you approve the pending workflow run so the fixtures get a real CI result?

The branch is now some way behind 2.0-dev (the 2.0 strictness and description changes landed since 12 Nov). I left it un-rebased so the diff stays reviewable as-is; say the word and I will rebase and re-run the local ajv matrix against the current 2.0-dev schema.

@jkowalleck

Copy link
Copy Markdown
Member

say the word and I will rebase and re-run the local ajv matrix against the current 2.0-dev schema.

if in doubt, just rebase :-)

@Lennonhaha

Copy link
Copy Markdown
Author

Hi Jan,

Thank you so much for the proactive help! Yes, please go ahead and rebase and re-run the ajv matrix. Also, could you please approve the pending workflows (it says 1 workflow awaiting approval)?

A huge thanks to @Mehrn0ush for the detailed review. The distinction between "omitted" and "not assessed" is spot on, and I appreciate the careful reading.

Let me know if anything else is needed from my side before merging. Thanks again!

@jkowalleck
jkowalleck force-pushed the cbom-family-only-fixtures-2.0 branch from 055d96d to 1c538db Compare November 16, 2026 15:13
@jkowalleck jkowalleck added this to the 2.0 milestone Nov 16, 2026
@jkowalleck

This comment was marked as outdated.

@jkowalleck jkowalleck left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI/QA tests are failing https://github.com/CycloneDX/specification/actions/runs/43894718646/job/133396007088?pr=1176#step:5:369 ff

test /home/runner/work/specification/specification/tools/src/test/resources/2.0/valid-cbom-family-only-mlkem-2.0.json ...
  !!! ERROR: Unexpected validation error 
      schema: /home/runner/work/specification/specification/schema/2.0/cyclonedx-2.0.schema.json 
    for file: /home/runner/work/specification/specification/tools/src/test/resources/2.0/valid-cbom-family-only-mlkem-2.0.json 
       error: [
    {
      instancePath: '/inventories/0/subject',
      schemaPath: '#/additionalProperties',
      keyword: 'additionalProperties',
      params: { additionalProperty: 'bom-ref' },
      message: 'must NOT have additional properties',
      schema: false,
      parentSchema: {
        type: 'object',
        title: 'Component',
        required: [Array],
        additionalProperties: false,
        properties: [Object],
        allOf: [Array]
      },
      data: {
        'bom-ref': 'subject-mlkem-family-only',
        type: 'application',
        name: 'family-only observation: ML-KEM',
        version: '1.0'
      }
    },
    {
      instancePath: '/inventories/0/subject',
      schemaPath: '#/required',
      keyword: 'required',
      params: { missingProperty: 'ref' },
      parentSchema: {
        type: 'object',
        title: 'Component',
        required: [Array],
        additionalProperties: false,
        properties: [Object],
        allOf: [Array]
      },
      data: {
        'bom-ref': 'subject-sm2-family-only',
        type: 'application',
        name: 'family-only observation: SM2',
        version: '1.0'
      }
    },
    {
      instancePath: '/inventories/0/subject',
      schemaPath: '#/required',
      keyword: 'required',
      params: { missingProperty: 'ref' },
      message: "must have required property 'ref'",
      schema: [ 'ref' ],
      parentSchema: {
        type: 'object',
        title: 'Component Reference',
        description: 'A reference to a component declared elsewhere, such as in the definitions registry, in another position of the same document, or in another document via BOM-Link. A reference includes the referenced component exactly as declared: every position that references the same component includes an identical component. Where the inclusion context differs, such as a different scope or quantity, declare the component inline instead.',
        required: [Array],
        additionalProperties: false,
        properties: [Object]
      },
      data: {
        'bom-ref': 'subject-sm2-family-only',
        type: 'application',
        name: 'family-only observation: SM2',
        version: '1.0'
      }
    },
    {
      instancePath: '/inventories/0/subject',
      schemaPath: '#/properties/subject/oneOf',
      keyword: 'oneOf',
      params: { passingSchemas: null },
      message: 'must match exactly one schema in oneOf',
      schema: [ [Object], [Object] ],
      parentSchema: {
        title: 'Subject',
        description: 'The subject whose inventory this statement declares. Either a concrete component or a reference using bom-link or `refId` to a component declared elsewhere, such as in the definitions.',
        oneOf: [Array]
      },
      data: {
        'bom-ref': 'subject-sm2-family-only',
        type: 'application',
        name: 'family-only observation: SM2',
        version: '1.0'
      }
    }
  ]

@jkowalleck

Copy link
Copy Markdown
Member

Two fixtures for the case discussed in CycloneDX#1175: an
algorithm family is observed, the parameter set is unknown, and no NIST
category was assessed.

- nistQuantumSecurityLevel is omitted, not set to 0: 0 means "none of the
  categories are met", which is a different and stronger claim
- no custom namespaced property encodes assessment status, so the fixtures
  stay valid against the 2.0 schema on their own
- the names carry the valid- prefix because AbstractJsonSchemaVerificationTest
  only picks up resources whose names start with "valid" or "invalid"

Validated with ajv 8.20 against both the bundled and the modular 2.0 schema.
Controls confirm the validation is not vacuous: a bogus algorithmFamily is
rejected, nistQuantumSecurityLevel = 7 is rejected, = 0 is accepted.

Signed-off-by: 刘天赫 <research@fibemate.net>
Both family-only fixtures said the omission existed "because the assessment was
not performed". That is the same inference this fixture pair is meant to
disprove: omission records that no assertion was made, and says nothing about
whether an assessment happened. It also implied SM2 cannot be assessed.

Adopt the reviewer's wording, which describes only the recorded observation:

  Algorithm family observed without a recorded parameter set or asserted NIST
  security strength category. Omission of nistQuantumSecurityLevel does not
  establish whether an assessment was performed.

No schema or structural change; the two description strings are the only edit.

Validated against schema/2.0/cyclonedx-2.0-bundled.schema.json from 2.0-dev
76f185c with ajv 8.20.0 (draft 2020-12):
both fixtures valid, bogus algorithmFamily rejected, level 7 rejected,
explicit level 0 accepted.

Signed-off-by: 刘天赫 <research@fibemate.net>
… examples

CycloneDX#1113 renamed bom-ref to refId across the 2.0 schema and its test data. These two
fixtures still used bom-ref, so once 2.0-dev moved on, the component definition
rejected them as an additional property and the subject failed its oneOf against
both component and componentReference.

The subject here is an inline component, not a reference to one, so refId is the
right field; a reference would need ref plus a definition elsewhere, which would
change what the examples demonstrate.

Verified against this commit's schema with the repository's own functional harness
(tools/src/test/js/schema-v2, v2.0): both fixtures fail before the change with the
reported additionalProperties/missing ref/oneOf errors, and both pass after it.

Signed-off-by: 刘天赫 <research@fibemate.net>
@Lennonhaha
Lennonhaha force-pushed the cbom-family-only-fixtures-2.0 branch from 1c538db to a548a83 Compare November 16, 2026 16:17
@Lennonhaha

Copy link
Copy Markdown
Author

Updated — thanks for the pointer to #1113.

Both fixtures now use refId (lowercase d, as renamed in #1113). The subject stays an inline component rather than a ref: it is the thing being inventoried, not a reference to a component declared elsewhere, so ref would change what the examples demonstrate.

Also rebased onto current 2.0-dev so the fixtures and the schema they are validated against agree.

Verified locally against this commit's schema with the repo's own harness (tools/src/test/js/schema-v2, npm run test:v2.0):

  • before the rename: both fixtures fail with exactly the reported errors — additionalProperties: bom-ref, required: ref, and the resulting oneOf mismatch;
  • after: 0 errors.

One unrelated thing I hit while running it: on Windows the harness silently tests nothing. glob(join(testDataDir, 'valid-*.json')) builds a backslash pattern, and glob treats \ as an escape, so t1 and t3 load zero files and report found 0 errors. CI is Linux, so it only affects local runs — but a Windows contributor would see green for the wrong reason.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cap: cryptography Capability: Cryptography (CBOM)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants