Skip to content

fix(cli): record the command in plain simulate's run manifest - #36

Merged
isarasua merged 3 commits into
mainfrom
isarasua/fix/simulate-manifest-invocation
Oct 13, 2026
Merged

isarasua merged 3 commits into
mainfrom
isarasua/fix/simulate-manifest-invocation

Conversation

@isarasua

Copy link
Copy Markdown
Contributor

What changed and why

A run manifest records how the run was launched (source.kind, source.invocation). Plain
usersim simulate passed source_kind="cli" to write_simulator_manifest. The manifest fills in
the invocation only when it detects the kind itself, so plain runs always recorded an empty
source.invocation.

For example, usersim simulate --locale en_US --num-rows 50 --max-turns 6 recorded:

"source": {"kind": "cli", "invocation": ""}

So a month later, the manifest couldn't say which flags made the run.

The override is gone. The manifest now detects the source itself, as it already does for
simulate --inputs, eval and the other writers. A CLI run records its command line (sys.argv).

As agreed in the review of #30.

How it was verified

Note

Under SLURM, or when _simulate_to_parquet is called inside a notebook, the manifest now says
slurm (with the job id) or notebook, as the other writers' manifests already do. A command-line
run outside SLURM records cli and its command.

馃 Generated with Claude Code

Plain `usersim simulate` named its source kind (`cli`) when writing the
run manifest, and the manifest records an invocation only when it
detects the kind itself, so `source.invocation` was always empty. The
override is gone: the manifest detects how the run was launched, as it
already does for `simulate --inputs`, `eval` and the other writers,
and records the command line (`sys.argv`) for a CLI run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: isarasua <isarasua@nvidia.com>

@3mei 3mei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to merge!

@isarasua
isarasua merged commit 7997972 into main Oct 13, 2026
10 checks passed
isarasua added a commit that referenced this pull request Oct 13, 2026
Brings in #34, #36 and #30.

Resolves two conflicts with #34:
- the notebook's manifest cell, which #34 rewrote: #34's version, with
  this branch's `replay_assistant_reasoning` line added to its config
- the fake `Designer.create` in this branch's manifest test, which now
  takes the `artifact_path` that #34 passes, as #34's own fakes do

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: isarasua <isarasua@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants