Skip to content

fix(e2e): compile every timed route first in the HTTP suites and stop sessions completely - #8701

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/http-e2e-warmups
Oct 7, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/http-e2e-warmups

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Version compare E2E: a new first check compiles the deployment, version, compare and MCP routes under a 300s budget, using anonymous requests that each get a 401. Later requests keep their 60s bound. A timeout now names the method and route instead of "The operation timed out." Before this, a cold next dev compile of /api/workflows/[id]/deployments/[version] or /api/mcp could use up the whole 60s request budget, and the suite failed that way in CI.
  • Desktop inbox E2E:
    • The warm-up now also reads /api/desktop/inbox, so the executor's 10s window covers no route compile.
    • executor.stop() waits for the pull in flight, and the loop exits between items. Teardown therefore never deletes the device and user under a claim, and a timed-out check reports the timeout instead of a later 401.
  • CI: the embedded CLI step from perf(cli): load a manual run's draft once and keep embedded CLI results lean #8625 now follows the same lifecycle as the other HTTP E2E apps. It starts from an empty .next/dev, runs under setsid and is stopped with stop-session.sh. Before, it restored the SCIM app's dev cache, which is the restore that caused the fix(ci): stop each HTTP end-to-end app fully and start the next from an empty dev cache #8686 flake. The shared-cache note now sits on the first app step, so new steps will see it.
  • stop-session.sh:
    • It signals every PID in the session, including a member that moved to its own process group.
    • SIGKILL comes 10s after SIGTERM, matching the warning, instead of up to about 20s.
    • "Outlived its leader" is printed only once the leader has exited.
  • Stop-after E2E: when the CLI's output overflows maxBuffer, the failure is reported as an overflow, not as "did not exit within 60s".

Type of Change

  • Bug fix

Testing

  • Ran the CI sequence 6 times on Linux (8 cores, cold .next for each app): SCIM and version compare, then stop-after, then desktop inbox. Every iteration passed, and all three warm-up checks passed each time.
  • Checked stop-session.sh against three cases, and it stopped every process in each:
    • a session member in its own process group, which stopped within 1s;
    • a member that ignores SIGTERM, which was SIGKILLed at 10s;
    • a leader that ignores SIGTERM, which was SIGKILLed at 10s with no "outlived" line.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@vercel

vercel Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 6, 2026 11:35pm UTC

Request Review

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@greptile-apps

greptile-apps Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Refactors test infrastructure and session cleanup for end-to-end tests.

The PR appears safe to merge; no blocking issue was found.

What we checked:

  • Cleanup waits for active work: stop() removes future triggers and awaits inFlight. The caller awaits stop() before leaving the checks that use the executor.
  • Warm-up does not fake presence: The inbox read marks the device present, but the suite deletes that key before opening the stream and checking presence.

Summary

This PR separates cold route compilation from timed HTTP checks and waits for active work before cleanup.

  • Warms the version, comparison, MCP, and desktop inbox routes before timed checks.
  • Waits for the desktop executor’s active pull before deleting fixtures.
  • Stops every live process in the app’s session and shares one 10-second SIGTERM deadline.
  • Gives the CLI suite the same clean-cache startup and session cleanup as the other suites.
  • Reports CLI output overflow separately from a timeout.

No actionable issue was found. The previous-review comparison did not pass the ancestry and merge-base checks, so this review covered the full PR diff. No previous findings were supplied.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Clear dev cache] --> B[Start app in its own session]
  B --> C[Warm routes]
  C --> D[Run timed checks]
  D --> E[Await active executor pull]
  E --> F[Send SIGTERM to session members]
  F --> G{Any live members after 10 seconds?}
  G -- No --> H[Finish cleanup]
  G -- Yes --> I[Send SIGKILL]
  I --> J[Wait for session to empty]
  J --> H
Loading

Reviews (2) · Last reviewed commit: "fix(ci): run the embedded CLI app with t..."

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

… sessions completely

- version compare: a first check compiles the deployment, version, compare and MCP
  routes under a 300s budget; later requests stay at 60s and a timeout names the route
- desktop inbox: the warm-up also reads the inbox, so the 10s executor window covers
  no compile; stopping the executor waits for its in-flight pull, so teardown never
  runs under a claim and a timeout is reported as itself instead of a later 401
- stop-session.sh signals every process in the session, including one in its own
  process group, escalates to SIGKILL 10s after SIGTERM, and reports processes that
  outlived the leader only once the leader has exited
- stop-after: a CLI that overflows its output buffer is reported as an overflow, not
  as a timeout
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit f98df1b into staging Oct 7, 2026
33 of 34 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/http-e2e-warmups branch October 7, 2026 00:51

This branch was successfully deployed

1 active deployment
Preview — 6802ccdb Deployed Oct 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant