Repository navigation
fix(cli): record the command in plain simulate's run manifest - #36
Merged
Merged
Conversation
Plain `usersim simulate` named its source kind (`cli`) when writing the run manifest, and the manifest records an invocation only when it detects the kind itself, so `source.invocation` was always empty. The override is gone: the manifest detects how the run was launched, as it already does for `simulate --inputs`, `eval` and the other writers, and records the command line (`sys.argv`) for a CLI run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: isarasua <isarasua@nvidia.com>
isarasua
added a commit
that referenced
this pull request
Oct 13, 2026
Brings in #34, #36 and #30. Resolves two conflicts with #34: - the notebook's manifest cell, which #34 rewrote: #34's version, with this branch's `replay_assistant_reasoning` line added to its config - the fake `Designer.create` in this branch's manifest test, which now takes the `artifact_path` that #34 passes, as #34's own fakes do Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: isarasua <isarasua@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
A run manifest records how the run was launched (
source.kind,source.invocation). Plainusersim simulatepassedsource_kind="cli"towrite_simulator_manifest. The manifest fills inthe invocation only when it detects the kind itself, so plain runs always recorded an empty
source.invocation.For example,
usersim simulate --locale en_US --num-rows 50 --max-turns 6recorded:So a month later, the manifest couldn't say which flags made the run.
The override is gone. The manifest now detects the source itself, as it already does for
simulate --inputs,evaland the other writers. A CLI run records its command line (sys.argv).As agreed in the review of #30.
How it was verified
test_simulate_helper_mints_new_run_when_skip_existing_falsenow fixessys.argvand turns off SLURM and notebook detection. It asserts
source.kind == "cli"and the exactinvocation. It failed before the change (empty invocation) and passes after.
fixed.
make check-allpasses, and the full suite passes: 3,835 tests.src, the docs or the notebooks readssource.kindorsource.invocation. Only tests do.Note
Under SLURM, or when
_simulate_to_parquetis called inside a notebook, the manifest now saysslurm(with the job id) ornotebook, as the other writers' manifests already do. A command-linerun outside SLURM records
cliand its command.馃 Generated with Claude Code