Skip to content

fix(ci): preserve manual Android suite timing - #3554

Open
thymikee wants to merge 2 commits into
mainfrom
t3/android-manual-shell-3077
Open

thymikee wants to merge 2 commits into
mainfrom
t3/android-manual-shell-3077

Conversation

@thymikee

@thymikee thymikee commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

The pinned emulator action starts a separate shell for each script line, losing Android setup/full-suite timestamps and reporting negative setup and zero suite durations.

Run the existing pipeline through one shell using the action's emulator serial. Record both timings even if full E2E fails, then preserve its exit code and stop before ordinary replay. Four files changed; prerequisite for #3555 and #3077 validation.

Validation

Tested head: a85a0f09b7916c7627a50aa94cc34ce82da4a0d0.

Regression Before After
Lost timestamp state 8255c735: FAIL (-90s/0s) PASS (10s/25s)
Failed full suite loses timing f9260b1ee: 1 FAIL / 2 PASS 3 PASS; exit 13, no replay

Three focused cases cover two independent serials; hardcoding the first serial fails both other-serial cases. Mutation restored. Repository format, shell syntax and fresh review pass. Rebased onto main a95d2d539 without changing either implementation commit. Exact-head pnpm check:affected --base a95d2d539 --run passes all runnable gates; current head a85a0f09b has 21 successful checks and one expected cache skip, including Coverage/Integration and all four platform smoke suites. No unresolved review threads; no merge conflicts.

Earlier hosted Android on runtime head a3aba45a reports setup 92s/full 398s. Its success evidence is retained; the failure-path extension is covered independently. Pure workflow repair; no new device behavior.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread scripts/replays/android-manual.test.ts Outdated
Comment thread scripts/replays/android-manual.sh Outdated
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.18 MB 5.18 MB 0 B
Package (unpacked) 5.18 MB 5.18 MB 0 B
Package (download) 1.56 MB 1.56 MB +1 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 25.2 ms 26.6 ms +1.4 ms
CLI --help 65.9 ms 63.7 ms -2.2 ms

@thymikee
thymikee force-pushed the t3/android-manual-shell-3077 branch from f9260b1 to 22fb210 Compare October 13, 2026 00:32
@thymikee thymikee changed the title fix(ci): preserve manual Android timing in one shell fix(ci): preserve manual Android suite timing Oct 13, 2026
@thymikee

Copy link
Copy Markdown
Member Author

I reviewed 22fb210 and found no problems in the code. The manual Android replay job now keeps the suite timing, and the script keeps the real exit status after it prints both timings. The test uses distinct serials, so a hardcoded serial would fail. All 22 checks pass, and there are no conflicts. Nothing else blocks merge. The earlier hosted manual Android run reproduced the timings, but I did not run the vitest case or the failure path on the hosted emulator, so exit-code preservation rests on the stubbed test and reading the code.

Both inline threads from cubic-dev-ai are fixed at this commit, so you can resolve them: #3554 (comment) (the test now uses emulator-7000 and emulator-8100, and the adb stub rejects a mismatch) and #3554 (comment) (the script saves the status with || FULL_STATUS=$?, prints both timings, then exits with it).

Two small, optional notes. The stubbed adb devices branch in android-manual.test.ts is never reached now, because the script no longer calls it. A short comment in the script saying the action exports ANDROID_SERIAL would help the next reader. android.yml still guesses the serial, which could be a separate change.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 13, 2026
@thymikee
thymikee force-pushed the t3/android-manual-shell-3077 branch from 22fb210 to a85a0f0 Compare October 13, 2026 11:42
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. The Android serial and exit-code handling from the earlier pass (22fb210) are now fixed at a85a0f0, and I found nothing new in the code. The two inline threads on the distinct emulator serials and on keeping the full-suite exit status both look fixed at this head, so you can resolve them: #3554 (comment) and #3554 (comment).

I judged the stubbed test from the diff and the script. I did not run the vitest case or the hosted emulator path. I also did not open hosted run 38324090019, so I have not confirmed that the emulator action exports ANDROID_SERIAL to the script. The exit-code behavior on a real emulator rests on the stubbed test and on reading the code. I did not review the 122-file rebase delta, since it is upstream content.

No conflicts. The two Smoke Tests runs and Mutants (snapshot-occlusion) are still pending, not failed. This patch only touches the manual workflow, the Android manual script and its test, and a vitest exclusion, so those jobs can only be affected by the rebased-in base changes. Merge once those jobs finish green.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant