Introduce conformance test CLI - #346
Open
henrybear327 wants to merge 2 commits into
Open
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
henrybear327
force-pushed
the
conf/harness
branch
3 times, most recently
from
August 31, 2026 20:28
d07c9e3 to
9482ae7
Compare
henrybear327
marked this pull request as draft
August 31, 2026 20:38
henrybear327
marked this pull request as ready for review
August 31, 2026 20:38
jserv
reviewed
Aug 31, 2026
jserv
requested changes
Aug 31, 2026
jserv
left a comment
jserv
left a comment
Contributor
There was a problem hiding this comment.
Introduce re-usable shell scripts as common library.
There was a problem hiding this comment.
All reported issues were addressed across 48 files
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Re-trigger cubic
henrybear327
force-pushed
the
conf/harness
branch
from
August 31, 2026 21:46
9482ae7 to
40f41ac
Compare
Collaborator
Author
Extracted the SSH part out. |
henrybear327
force-pushed
the
conf/harness
branch
2 times, most recently
from
August 31, 2026 22:36
51d5d32 to
1523520
Compare
The parser owns one capability-first command tree used by Make and CI. Suite registration cannot add commands. qemu-runner.sh stores VM state across start and stop commands. A command that starts a VM owns its cleanup trap, preserving the trap installed by test-matrix.sh. Start rejects an unusable guest /tmp, and stop verifies the recorded process still names its pidfile before sending a signal.
docs/testing.md lists every public command and its defaults. The conformance reference defines the result and extension contracts.
henrybear327
force-pushed
the
conf/harness
branch
from
August 31, 2026 23:02
1523520 to
417be2a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build extensible CLI to host gvisor and LTP (and potentially much more) conformance test suites
Summary by cubic
Introduces a conformance test harness that runs registered Linux suites (gvisor, LTP, and more) against elfuse and a QEMU reference, judging results against JSONC expectations. The
scripts/conformanceCLI has a fixed capability-first command set that suite registration cannot extend.New Features
scripts/conformancecommands for listing suites, enumerating cases, running lanes, seeding expectations, and writing reports.Other Changes
/tmpsetup fails, arm its cleanup per CLI command, and verify a recorded pid still belongs to the VM before stopping.make checkvia selftests and addsdocs/conformance.mdplus the command reference indocs/testing.md.Written for commit 417be2a. Summary will update on new commits.