Repository navigation
Clean up temporary benchmark output on errors - #2350
Open
jabrailkhalil wants to merge 1 commit into
Open
jabrailkhalil wants to merge 1 commit into
jabrailkhalil wants to merge 1 commit into
Conversation
Remove only tool-owned output when a benchmark fails to launch, exits with an error, or returns unusable results. Preserve explicit output files and tolerate a child removing its temporary output. AI assistance: analysis, implementation, tests, validation, review, and drafting.
Author
|
The Google CLA check has passed. The workflows currently show AI assistance: checking CI status and drafting this update. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Failed comparisons can leave automatically created result files in the temporary directory. Cleanup now runs after both benchmark execution and result loading, while explicitly requested output files remain untouched. An already-removed temporary file is tolerated; other cleanup errors still surface.
The regression tests use real temporary files and child Python processes. They cover success, nonzero exit, launch failure, invalid JSON, unsupported schema, explicit output on success/failure, and a child removing its temporary output. Four regressions fail on the old implementation; all eight pass with the fix.
Validation:
compare.pyCLI checks reproduce the leak before the fix and verify cleanup afterward, with original exit codes and explicit output preserved.git diff --checkpass for the changed Python files.This changes Python tooling only. The C++ library and compiled bindings were not rebuilt. AUTHORS/CONTRIBUTORS are updated as requested by CONTRIBUTING; the Google CLA already passes on my existing Google contributions.
Fixes #2349.
I used AI assistance for analysis, implementation, tests, validation, review and drafting this contribution.