From 5b6b5b42445043cb53e6ecaa8309a14d04a7fa20 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dan=20Gr=C3=B8ndahl?= Date: Thu, 3 Sep 2026 13:15:36 +0200 Subject: [PATCH 1/2] docs: document summary definitions on custom attestation types Custom attestation types can define a summary: an ordered list of named jq expressions that Kosli renders as labeled rows on the attestation detail page instead of only raw JSON. - getting_started/attestations.md: new "Summaries" subsection in the Custom accordion, covering the CLI flags and the rendering behavior (versioning, render-time evaluation, N/A, URL links, array payloads). - tutorials/attest_custom.md: the tutorial type is now created with --summary entries so the walkthrough ends on a readable summary. - administration/managing_custom_attestation_types/overview.md: summary added to the "Each type can include" list, plus a Terraform example. Refs #364 --- .../overview.md | 43 +++++++++++++++- getting_started/attestations.md | 51 +++++++++++++++++++ tutorials/attest_custom.md | 13 ++++- 3 files changed, 104 insertions(+), 3 deletions(-) diff --git a/administration/managing_custom_attestation_types/overview.md b/administration/managing_custom_attestation_types/overview.md index 08c7c08..b0c4d08 100644 --- a/administration/managing_custom_attestation_types/overview.md +++ b/administration/managing_custom_attestation_types/overview.md @@ -1,6 +1,6 @@ --- title: Managing Custom Attestation Types -description: Learn how to manage Kosli custom attestation types via Terraform, including creating and importing types with JSON Schema and jq evaluation rules. +description: Learn how to manage Kosli custom attestation types via Terraform, including creating and importing types with JSON Schema, jq evaluation rules, and summaries. --- The preferred way to manage custom attestation types is via the Kosli Terraform provider, so your Kosli configuration is version-controlled alongside your infrastructure. You can also manage custom attestation types through the Kosli CLI. @@ -13,8 +13,11 @@ Custom attestation types define how Kosli validates evidence from tools that don - A **JSON Schema** (optional) that defines the expected structure of attestation data - **jq rules** (optional) that evaluate the data to determine compliance +- A **summary** (optional) — ordered, labeled jq expressions that Kosli renders as rows on the attestation detail page -At least one of the two must be provided. +At least one of the schema and the jq rules must be provided. The summary is independent of both: +it only affects how attestations of the type are displayed, never whether they are compliant. See +[Summaries](/getting_started/attestations#summaries) for how summaries render. ## Create a custom attestation type @@ -72,6 +75,42 @@ resource "kosli_custom_attestation_type" "deployment_record" { } ``` +### With a summary + +`summary` takes a JSON array of `{name, expression}` objects. Each expression is a jq expression +evaluated against the attestation data when the attestation is displayed, and the entries render in +the order given. A value that is a valid URL renders as a clickable link. + +```hcl +resource "kosli_custom_attestation_type" "vulnerability_scan" { + name = "vulnerability-scan" + description = "Validates vulnerability scan results" + + jq_rules = [".critical_vulnerabilities == 0"] + + summary = jsonencode([ + { name = "Critical", expression = ".critical_vulnerabilities" }, + { name = "High", expression = ".high_vulnerabilities" }, + { name = "Scanner", expression = ".scanner_version" }, + { name = "Report", expression = ".report_url" }, + ]) +} +``` + +Use `file()` instead of `jsonencode()` to keep the summary in a standalone JSON file, so the same +definition can be shared with other tooling: + +```hcl +summary = file("${path.module}/summaries/vulnerability-scan.json") +``` + + +The summary is part of the versioned type definition, so changing it creates a new version of the +attestation type — exactly like changing the schema or the jq rules. Removing `summary` from a type +that had one clears the summary, and its attestations fall back to showing the jq evaluation results +as a pass/fail checklist. + + ## Import an existing custom attestation type If you have custom attestation types created via the CLI, you can bring them under Terraform management by importing them into your Terraform state. diff --git a/getting_started/attestations.md b/getting_started/attestations.md index 34ef73a..7786863 100644 --- a/getting_started/attestations.md +++ b/getting_started/attestations.md @@ -349,6 +349,57 @@ Currently, we support the following types of evidence: - So `32 / 1209 * 100 <= 5` evaluates to `2.64 <= 5` which is `true` + #### Summaries + + By default, the attestation detail page in Kosli shows the type's evaluation rules as a + pass/fail checklist, plus the raw attestation data. Add a **summary** to the type to pull named + values out of the attestation data and display them as labeled rows instead — the way the + built-in Sonar, Snyk and JUnit attestations do. + + A summary is an ordered list of `name`/`expression` pairs, where each expression is a + [jq expression](https://jqlang.org/manual/) evaluated against the attestation data. Add one + entry per `--summary` flag: + + ```bash + kosli create attestation-type coverage-metrics + --jq=".code.lines.missed / .code.lines.total * 100 <= 5" + --summary="Lines missed=.code.lines.missed" + --summary="Lines total=.code.lines.total" + ``` + + Attestations of the `coverage-metrics` type above then show `Lines missed` and `Lines total`, + in that order, with the values taken from each attestation's own data. + + Each `--summary` value is split on its first `=` only, so `==` inside a jq expression is safe. + Use [`--summary-json`](/client_reference/kosli_create_attestation-type) instead if the list is + easier to express as JSON, or the + [`summary` attribute](/terraform-reference/resources/custom_attestation_type) if you manage + your types with Terraform: + + ```json + [ + { "name": "Lines missed", "expression": ".code.lines.missed" }, + { "name": "Lines total", "expression": ".code.lines.total" } + ] + ``` + + Worth knowing: + + - The summary is part of the **versioned** type definition. Changing it creates a new version of + the type, just like changing the schema or the evaluation rules. Each attestation keeps the + version of the type it was reported against. + - Expressions are evaluated when the attestation is **displayed**, against the stored + attestation data. A summary never affects the compliance status of an attestation. + - Invalid jq is rejected when the type is created, so authoring mistakes surface immediately + rather than as a broken detail page later. + - An expression that returns `null`, or that fails against a particular attestation's data, + renders as `N/A`. The rest of the summary still renders. + - A value that is a valid URL renders as a clickable link, the same as + [annotation values](#annotating-attestations). + - If the attestation data is a top-level array, Kosli renders one summary group per element + (`Summary 1:`, `Summary 2:`, and so on). Write the expressions against a single element + (`.code.lines.missed`) — each element is evaluated separately. + See: * [create custom attestation type](/client_reference/kosli_create_attestation-type) and * [report custom attestation to an artifact or a trail](/client_reference/kosli_attest_custom/) for usage details and examples. diff --git a/tutorials/attest_custom.md b/tutorials/attest_custom.md index 550b291..7cddcf6 100644 --- a/tutorials/attest_custom.md +++ b/tutorials/attest_custom.md @@ -27,9 +27,20 @@ For this tutorial we'll create a minimal `coverage-report` type that requires a ```shell kosli create attestation-type coverage-report \ --description "Code coverage report" \ - --jq '.coverage >= 80' + --jq '.coverage >= 80' \ + --summary "Coverage=.coverage" \ + --summary "Tool=.tool" ``` +The two `--summary` entries are optional, but they're worth adding: each one is a +`'NAME=EXPRESSION'` pair whose [jq expression](https://jqlang.org/manual/) is evaluated against the +attestation data, and Kosli renders the results as labeled rows on the attestation detail page. So +the attestation you report below opens on a readable `Coverage: 92` / `Tool: pytest-cov` summary +instead of only the raw JSON. See +[Summaries](/getting_started/attestations#summaries) for the details, and +[`kosli create attestation-type`](/client_reference/kosli_create_attestation-type) for the +`--summary-json` alternative. + Prepare a JSON file with the data you want to attest. Save it as `coverage.json`: ```json From 09e392f1fb3f0a5fbcbebffebdb7c7646b95e70e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dan=20Gr=C3=B8ndahl?= Date: Thu, 3 Sep 2026 14:27:04 +0200 Subject: [PATCH 2/2] docs: address review on custom attestation type summaries - Move the summaries content out of the collapsed "Custom" accordion into a top-level "Summarizing custom attestations" section. Verified with headless Chrome that a cold load of the in-accordion anchor left every accordion at aria-expanded="false", so the deep link both cross-links rely on landed on hidden content. Also fixes the ####-after-## skip. - Fix the versioning contradiction: attestation.py resolves get_versioned_type(self.type_version) before evaluating, so a rendered attestation uses the type version it was reported against. Both pages now say that, and the Terraform note scopes the fallback to the new version. - Narrow the URL claim: link rendering requires a string value with an http/https scheme (kosli-dev/server#6428 hardening), not "any valid URL". - Trim the tutorial paragraph to one sentence plus one link, and drop "minimal" now that the type carries summary entries. - "At least one of the schema or the jq rules" (was "and"). - Add \ line continuations to the two command blocks in the Custom accordion, and note that re-running create updates an existing type. Refs #364 --- .../overview.md | 12 +- getting_started/attestations.md | 114 +++++++++--------- tutorials/attest_custom.md | 13 +- 3 files changed, 71 insertions(+), 68 deletions(-) diff --git a/administration/managing_custom_attestation_types/overview.md b/administration/managing_custom_attestation_types/overview.md index b0c4d08..32c1bed 100644 --- a/administration/managing_custom_attestation_types/overview.md +++ b/administration/managing_custom_attestation_types/overview.md @@ -15,9 +15,10 @@ Custom attestation types define how Kosli validates evidence from tools that don - **jq rules** (optional) that evaluate the data to determine compliance - A **summary** (optional) — ordered, labeled jq expressions that Kosli renders as rows on the attestation detail page -At least one of the schema and the jq rules must be provided. The summary is independent of both: +At least one of the schema or the jq rules must be provided. The summary is independent of both: it only affects how attestations of the type are displayed, never whether they are compliant. See -[Summaries](/getting_started/attestations#summaries) for how summaries render. +[Summarizing custom attestations](/getting_started/attestations#summarizing-custom-attestations) for +how summaries render. ## Create a custom attestation type @@ -79,7 +80,7 @@ resource "kosli_custom_attestation_type" "deployment_record" { `summary` takes a JSON array of `{name, expression}` objects. Each expression is a jq expression evaluated against the attestation data when the attestation is displayed, and the entries render in -the order given. A value that is a valid URL renders as a clickable link. +the order given. A string value beginning with `http://` or `https://` renders as a clickable link. ```hcl resource "kosli_custom_attestation_type" "vulnerability_scan" { @@ -107,8 +108,9 @@ summary = file("${path.module}/summaries/vulnerability-scan.json") The summary is part of the versioned type definition, so changing it creates a new version of the attestation type — exactly like changing the schema or the jq rules. Removing `summary` from a type -that had one clears the summary, and its attestations fall back to showing the jq evaluation results -as a pass/fail checklist. +that had one clears the summary on the new version, so attestations reported against it fall back to +showing the jq evaluation results as a pass/fail checklist. Attestations reported against an earlier +version keep the summary that version defined. ## Import an existing custom attestation type diff --git a/getting_started/attestations.md b/getting_started/attestations.md index 7786863..d067049 100644 --- a/getting_started/attestations.md +++ b/getting_started/attestations.md @@ -329,15 +329,15 @@ Currently, we support the following types of evidence: You could create a custom attestation type called `coverage-metrics` using a [jq expression](https://jqlang.org/manual/) rule defining a minimum line coverage of 95%: ```bash - kosli create attestation-type coverage-metrics + kosli create attestation-type coverage-metrics \ --jq=".code.lines.missed / .code.lines.total * 100 <= 5" ``` You could then make your custom attestation with the json file: ```bash - kosli attest custom - --type=coverage-metrics - --attestation-data=unit-test-coverage.json + kosli attest custom \ + --type=coverage-metrics \ + --attestation-data=unit-test-coverage.json \ ... ``` @@ -349,56 +349,9 @@ Currently, we support the following types of evidence: - So `32 / 1209 * 100 <= 5` evaluates to `2.64 <= 5` which is `true` - #### Summaries - - By default, the attestation detail page in Kosli shows the type's evaluation rules as a - pass/fail checklist, plus the raw attestation data. Add a **summary** to the type to pull named - values out of the attestation data and display them as labeled rows instead — the way the - built-in Sonar, Snyk and JUnit attestations do. - - A summary is an ordered list of `name`/`expression` pairs, where each expression is a - [jq expression](https://jqlang.org/manual/) evaluated against the attestation data. Add one - entry per `--summary` flag: - - ```bash - kosli create attestation-type coverage-metrics - --jq=".code.lines.missed / .code.lines.total * 100 <= 5" - --summary="Lines missed=.code.lines.missed" - --summary="Lines total=.code.lines.total" - ``` - - Attestations of the `coverage-metrics` type above then show `Lines missed` and `Lines total`, - in that order, with the values taken from each attestation's own data. - - Each `--summary` value is split on its first `=` only, so `==` inside a jq expression is safe. - Use [`--summary-json`](/client_reference/kosli_create_attestation-type) instead if the list is - easier to express as JSON, or the - [`summary` attribute](/terraform-reference/resources/custom_attestation_type) if you manage - your types with Terraform: - - ```json - [ - { "name": "Lines missed", "expression": ".code.lines.missed" }, - { "name": "Lines total", "expression": ".code.lines.total" } - ] - ``` - - Worth knowing: - - - The summary is part of the **versioned** type definition. Changing it creates a new version of - the type, just like changing the schema or the evaluation rules. Each attestation keeps the - version of the type it was reported against. - - Expressions are evaluated when the attestation is **displayed**, against the stored - attestation data. A summary never affects the compliance status of an attestation. - - Invalid jq is rejected when the type is created, so authoring mistakes surface immediately - rather than as a broken detail page later. - - An expression that returns `null`, or that fails against a particular attestation's data, - renders as `N/A`. The rest of the summary still renders. - - A value that is a valid URL renders as a clickable link, the same as - [annotation values](#annotating-attestations). - - If the attestation data is a top-level array, Kosli renders one summary group per element - (`Summary 1:`, `Summary 2:`, and so on). Write the expressions against a single element - (`.code.lines.missed`) — each element is evaluated separately. + A custom attestation type can also define a **summary**, so that attestations of the type show + named values from their data instead of only raw JSON. See + [Summarizing custom attestations](#summarizing-custom-attestations) below. See: * [create custom attestation type](/client_reference/kosli_create_attestation-type) and @@ -415,3 +368,56 @@ Currently, we support the following types of evidence: +## Summarizing custom attestations + +By default, the attestation detail page in Kosli shows a custom attestation type's evaluation rules +as a pass/fail checklist, plus the raw attestation data. Add a **summary** to the type to pull named +values out of the attestation data and display them as labeled rows instead — the way the built-in +Sonar, Snyk and JUnit attestations do. + +A summary is an ordered list of `name`/`expression` pairs, where each expression is a +[jq expression](https://jqlang.org/manual/) evaluated against the attestation data. Add one entry +per `--summary` flag: + +```bash +kosli create attestation-type coverage-metrics \ + --jq=".code.lines.missed / .code.lines.total * 100 <= 5" \ + --summary="Lines missed=.code.lines.missed" \ + --summary="Lines total=.code.lines.total" +``` + +Attestations of the `coverage-metrics` type then show `Lines missed` and `Lines total`, in that +order, with the values taken from each attestation's own data. Re-running `kosli create +attestation-type` for a name that already exists updates that type rather than creating a second +one — see the note on versioning below. + +Each `--summary` value is split on its first `=` only, so `==` inside a jq expression is safe. Use +[`--summary-json`](/client_reference/kosli_create_attestation-type) instead if the list is easier to +express as JSON, or the +[`summary` attribute](/terraform-reference/resources/custom_attestation_type) if you manage your +types with Terraform: + +```json +[ + { "name": "Lines missed", "expression": ".code.lines.missed" }, + { "name": "Lines total", "expression": ".code.lines.total" } +] +``` + +Worth knowing: + +- The summary is part of the **versioned** type definition. Changing it creates a new version of the + type, just like changing the schema or the evaluation rules. Each attestation is displayed using + the version of the type it was reported against, so editing a summary does not change how existing + attestations render. +- Expressions are evaluated when the attestation is **displayed**, against the stored attestation + data. A summary never affects the compliance status of an attestation. +- Invalid jq is rejected when the type is created, so authoring mistakes surface immediately rather + than as a broken detail page later. +- An expression that returns `null`, or that fails against a particular attestation's data, renders + as `N/A`. The rest of the summary still renders. +- A value that is a string beginning with `http://` or `https://` renders as a clickable link, as + [annotation values](#annotating-attestations) do. Other values render as text. +- If the attestation data is a top-level array, Kosli renders one summary group per element + (`Summary 1:`, `Summary 2:`, and so on). Write the expressions against a single element + (`.code.lines.missed`) — each element is evaluated separately. diff --git a/tutorials/attest_custom.md b/tutorials/attest_custom.md index 7cddcf6..9c67865 100644 --- a/tutorials/attest_custom.md +++ b/tutorials/attest_custom.md @@ -22,7 +22,7 @@ Before you can report a custom attestation, the type referenced by `--type` must * **CLI** — [`kosli create attestation-type`](/client_reference/kosli_create_attestation-type) (good for quick experiments). * **Terraform** — the [`kosli_custom_attestation_type` resource](/terraform-reference/resources/custom_attestation_type) (recommended so the type is version-controlled). -For this tutorial we'll create a minimal `coverage-report` type that requires a `coverage` field of at least 80: +For this tutorial we'll create a small `coverage-report` type that requires a `coverage` field of at least 80: ```shell kosli create attestation-type coverage-report \ @@ -32,14 +32,9 @@ kosli create attestation-type coverage-report \ --summary "Tool=.tool" ``` -The two `--summary` entries are optional, but they're worth adding: each one is a -`'NAME=EXPRESSION'` pair whose [jq expression](https://jqlang.org/manual/) is evaluated against the -attestation data, and Kosli renders the results as labeled rows on the attestation detail page. So -the attestation you report below opens on a readable `Coverage: 92` / `Tool: pytest-cov` summary -instead of only the raw JSON. See -[Summaries](/getting_started/attestations#summaries) for the details, and -[`kosli create attestation-type`](/client_reference/kosli_create_attestation-type) for the -`--summary-json` alternative. +The `--summary` entries are optional. Each is a `'NAME=EXPRESSION'` pair, so the attestation you +report below opens on a readable `Coverage: 92` / `Tool: pytest-cov` summary instead of raw JSON — +see [Summarizing custom attestations](/getting_started/attestations#summarizing-custom-attestations). Prepare a JSON file with the data you want to attest. Save it as `coverage.json`: