Skip to content

Enforce optimization constraints and show their verdicts in the study view - #9604

Draft
kube wants to merge 4 commits into
claude/opt-proto-dedicated-viewfrom
claude/opt-proto-constraints
Draft

Enforce optimization constraints and show their verdicts in the study view#9604
kube wants to merge 4 commits into
claude/opt-proto-dedicated-viewfrom
claude/opt-proto-constraints

Conversation

@kube

@kube kube commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Important

Experimental
Behind the In-browser optimization feature flag.

Summary

Before this PR, a study recorded its constraints and nothing evaluated them. Every draw the optimizer proposed was simulated, every run counted the same, and the study view had no word on feasibility.

This PR evaluates constraints for a study run in the browser. A parameter constraint is checked before a step simulates: a negative margin prunes the step as infeasible with the constraint named, and the step draws grey in the steps table, on the surface and on the objective chart. A state constraint compiles to a 0/1 indicator metric whose minimum over time says whether a run held it throughout, and the trial event carries runs passed over runs total per constraint. The study view gains a Constraints card, a Steps clear stat and a Runs passed column, every rate printed as a raw fraction beside its percentage, and the create drawer gains a Pass threshold. The objective stays the plain mean over every run; verdicts sit beside it and exclude nothing.

Links

Changes

Core

  • constraintMargin walks a constraint's HIR and gives its signed slack at a point

    Comparison slack, minimum over &&, maximum over ||, a flipped sign under !, the chosen branch of a conditional, scoped let, one unit for a bare boolean. margin >= 0 means satisfied.
    A strict comparison at exact equality returns -Number.MIN_VALUE, so the sign is the verdict for integer parameters too.
    evaluateParameterConstraints gives one margin per constraint and marginForOptuna is the one negation.

  • compileStateConstraintIndicator compiles a state constraint as a body ? 1 : 0 metric

    Emitted through emitBufferMetricJs with no TypeScript compiler, so it runs on the main thread. interpretHirExpr is exported for the margin walk.

  • getRunValues honours aggregateTime for scalar output

    min over a 0/1 indicator is the "held throughout" quantifier.

  • Manifest constraintPolicy and trial event constraints

    constraintPolicy.alpha strictly between 0 and 1, default 0.05. The trial event carries parameter margins, per-constraint runs passed over runs total, and an optional infeasible name. Both trial outcome variants carry the block.

  • Browser capability merges the channel's results into the trial event

    Kept per trial on the run record and cleared when the segment ends. The worker and the Python core see nothing of them.

Channel

  • Parameter constraints checked first, before any simulation

    A negative margin returns pruned with Infeasible: <name> before runDetachedObjective and before trialStarted, so no batch appears in the activity list.

  • State constraints run as auxiliary metrics of the objective batch

    auxiliaryMetrics on the detached objective request are appended as precompiled scalar expression specs with their aggregateTime. Indicators are compiled once per run id and dropped on dispose.

  • trialOutcome attaches { parameters, state } beside the plain mean

    A missing run value counts as failed. A constraint the emitter declines throws with its name and the trial is pruned with that reason.

  • Followed step's selection stream carries the objective's frames only

    The batch streams the indicator frames too.

Study view

  • Constraints card, fourth in a connected study's card row

    Headline "N / M · P%" over steps clear across the study, subtitle with the pass threshold, alpha and the infeasible draw count, one reserved line for the latest step's verdict, one bar per state constraint with the threshold marked, a note counting parameter constraints.

  • Steps clear stat in the summary strip and a Runs passed column in the steps table

    The column shows the binding constraint's fraction, the lowest pass rate in the step. An infeasible draw's mark is grey, titled "Infeasible: ", with greyed row text.

  • Infeasible draws are grey on the objective chart and muted rings on the surface

    trialFeasibility reads constraints.infeasible and is the default feasibilityOf, so an infeasible draw is never the best so far.

  • Pass threshold in the create drawer

    A percent input under the constraint rows, default 95, shown once a constraint exists. constraintPolicyFor writes { alpha } only when the threshold differs from the default.

  • constraint-rates.ts derives every rate from the trials in render

    stepVerdict reads clear, limited, infeasible or unconstrained. formatRate prints "52 / 60 · 87%". Nothing is stored.

  • Fixtures and stories for a constrained study

    fakeStudyConstraints, fakeConstrainedStudyInput, makeConstrainedTrials, and sirConstrainedOptimizationInput lowered with the real lowerConstraint. Stories Connected study with constraints and Connected study with constraints, following steps.

Known issues

  • Studies on the optimization service report no verdicts

    The card is mounted for connected studies only.

  • Real Pyodide run with a constraint not exercised in this PR

    The channel and capability tests run constraints lowered from the SIR model and the story carries results through the UI.

Next steps

  • Per-constraint pass rate over steps as a line chart with the threshold marked
  • Margins to Optuna through marginForOptuna
  • AutoStudy harness constraints list for the real Pyodide story

Test coverage

  • margin.test.ts, indicator-metric.test.ts:

    Slack per operator, the sign at exact equality, let scopes, the interpreter's errors, and a compiled indicator that reads 1 then 0 on a two-place net.

  • user-defined.test.ts, optimization.test.ts, browser-optimization.test.ts:

    Per-run minimum over frames, the policy's alpha bounds, the trial event's constraints block, and the capability carrying the channel's results.

  • trial-constraints.test.ts, create-optimization-channel.test.ts, detached-objective.test.ts:

    A pruned infeasible draw before any simulation, state constraints as auxiliary metrics with per-run verdicts, nothing attached without constraints, auxiliary specs precompiled with their aggregation.

  • constraint-rates.test.ts, objective-history-data.test.ts, surface-plot.test.tsx:

    The threshold rule at equality, the binding constraint, study rates with infeasible draws apart, feasibility read off the constraints block, a muted ring for an infeasible draw.

  • create-optimization-drawer.test.tsx, view-optimization-drawer.test.tsx:

    A changed threshold written as alpha, the Constraints card, the Steps clear stat, the Runs passed column, greyed marks, and none of it without constraints.

How to test

  • Open Petrinaut preview on Vercel
  • Viewport controls > Settings > Simulation > In-browser optimization
  • Load example > Supply Chain Profit
  • Simulate > Optimizations > Create
  • Select metric Profit, direction Maximize, tick Optimize production_rate
  • Constraints > Add parameter constraint, type scenario.production_rate <= 320
  • Constraints > Add state constraint, type return state.places.FinishedGoods.count <= 500;

    Expect a Pass threshold field under the rows reading 95

  • Run, then open the study row

    Expect a Constraints card with the steps clear headline and one bar for the state constraint
    Expect a Steps clear stat in the summary and a Runs passed column in the steps table

  • Wait for a draw above 320

    Expect its row greyed with a mark titled Infeasible, a grey dot on the Objective by step chart and a muted ring on the surface

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/apps > hash.design Affects the `hash.design` design site (app) labels Sep 9, 2026
@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 9, 2026 6:50pm UTC
petrinaut Ready Ready Preview Sep 9, 2026 6:50pm UTC
petrinaut-docs Ready Ready Preview Sep 9, 2026 6:50pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 9, 2026 6:50pm UTC

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash.design Affects the `hash.design` design site (app) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

1 participant