Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,48 @@ follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- **`piace explain`** — an optional, advisory **change assessment** of a stored
result document. It is a second, independent step: it reads a JSON report
`compare` already wrote, asks a configured **inference service** to judge the
aggregate groups in it, and writes a separately versioned assessment artifact
(`ai_schema_version: 1`) plus a re-rendered HTML report whose assessment
section sits *below* the deterministic outcome.
- **`services.yaml` gains an `inference:` section** — endpoint (https only),
model, `token_env` or `token_file` (never an inline token), `timeout`,
`max_tokens`, `max_groups`, `pseudonymize`, `structured_output`, and
`policy_notes_file`. It loads independently: a services file containing
nothing but this section is valid for `explain`, so an assessment needs no
Puppet infrastructure named at all.
- **`--change CHANGE.yaml`** — a caller-supplied **change context** describing
the repository change under test: refs, commit subjects, changed paths, and a
capped title and description. PIACE reads the file and never invokes git;
`scripts/change-context.sh` generates one for the common CI case. Its free
text is transmitted inside an explicit fence labelled as untrusted data.
- **Pseudonymized identities** — certnames in an outbound inference request are
replaced by stable per-run substitutes, and the compiler and PuppetDB
authorities are absent from it entirely. Resource identities pass through
untouched: `File[/etc/sudoers]` is the signal. A pseudonym never appears in an
assessment or any report. `pseudonymize: false` sends real certnames and is
documented as the deliberate loosening it is.
- **`--fail-on-inference-error`** — exit 30 when the assessment could not be
produced. Without it a failed assessment is recorded in the artifact with
every risk indication `unknown`, and the command still exits 0.
- **`report.DecodeJSON`** — a result document can now be read back into the
model it was rendered from, strictly: unknown fields and trailing content are
refused, and numbers keep their exact decimal digits.

### Unchanged

- **`piace compare` is untouched by this feature.** Its result document stays
`schema_version: 1`, its reports are byte-identical for identical inputs, its
exit codes are the same, and it contacts no inference service. A report
rendered without an assessment is byte-for-byte the artifact v0.1.0 wrote,
asserted against a golden captured before the feature existed. `explain`
contacts no compiler and no PuppetDB, asserted by failing the test if either
configured endpoint is reached.

## [0.1.0] - 2026-08-28

First release: the whole tool, so this entry describes what it does rather
Expand Down
39 changes: 39 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ PuppetDB with catalogs compiled for an already deployed candidate environment.

## Language

### Comparison

**Baseline catalog**:
The catalog selected from PuppetDB or a local catalog snapshot for a target,
used as the state against which a candidate catalog is compared. A PuppetDB
Expand Down Expand Up @@ -78,3 +80,40 @@ whose latest stored catalog contains a changed exact resource type and title.
It is bounded by configured limits and is a potential-impact estimate, not
proof that those nodes will change.
_Avoid_: affected nodes, blast radius

### Change assessment

**Change assessment**:
The advisory, model-generated document `piace explain` derives from one stored
result document and an optional change context. It is not deterministic, is not
part of the result document, and never affects a comparison outcome or exit
status.
_Avoid_: AI report, analysis, blast radius

**Risk indication**:
A change assessment's closed-enum judgement (`low`, `medium`, `high`,
`unknown`) for one aggregate group or for the run. It is a model's opinion
about a change, not a measurement of it.
_Avoid_: risk score, severity, danger level, safety rating

**Review focus**:
The ordered list of resource identities or targets a change assessment suggests
a reviewer look at first. It is a reading order, not a work list.
_Avoid_: recommendations, action items, findings

**Inference service**:
The configured external OpenAI-compatible endpoint a change assessment is
requested from. It is the only service PIACE contacts that is not the compiler
or PuppetDB, and `piace compare` never contacts it.
_Avoid_: AI provider, LLM, model backend

**Change context**:
The caller-supplied file describing the repository change under test: refs,
commit subjects, changed paths, and optional capped title and description.
PIACE reads it, never invokes git, and treats its free text as untrusted data.
_Avoid_: git diff, commit info, PR metadata

**Pseudonymized identity**:
A stable per-run substitute for a certname or service authority used only in an
inference request body. It never appears in a change assessment or any report.
_Avoid_: anonymized, masked, redacted
Loading
Loading