Repository navigation
ci: retry flaky E2E tests once; hosting_market_e2e gets CI headroom - #380
Merged
Merged
Conversation
preferred_hosts_hold_pass_challenges_and_get_paid failed on most master runs on 2026-11-15 (and on #371/#379): the first three copies land within seconds, but one host's 5 MiB fetch fails and retries ("chunks still missing"), the second preferred copy arrives ~60 s in, and the extra copies still need their trim grace, which ran past the 90 s wait on loaded runners. The test still requires both preferred hosts and exactly three stable copies; only the time allowed grows (agreement window 60 s -> 120 s too). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ThreatCrush Security Scan14 finding(s) HIGH/CRITICAL: 2 | MEDIUM: 8 | LOW: 4
Snippets are redacted; ThreatCrush never prints matched credential material. |
Multi-node loopback E2Es time out now and then on loaded runners; on 2026-11-15 hosting_market_e2e, hosting_e2e, git_e2e and a private-objects replication test failed across most PRs and on master. The Test step now runs with --no-fail-fast, and only if it fails reruns each failed test once by exact name; a test that fails twice still fails the job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anthony: "the github action … PR checks failed, fix it".
Multi-node loopback E2E tests time out now and then on loaded GitHub runners. Today
hosting_market_e2e,hosting_e2e,git_e2eand the private-objects replication test failed across most PRs and on master, one or another each time.ci(rust): failed tests get one retry by exact name. The Test step runscargo test --workspace --locked --no-fail-fast. Only if that fails does it rerun each failed test once (cargo test --workspace --locked -- --exact <name>). A test that fails twice still fails the job, and a failure with no test name (a build or harness error) fails immediately. Retries show as a::warning::annotation, so flaky tests stay visible.hosting_market_e2egets convergence headroom: 90 s → 240 s, and the agreement window 60 s → 120 s. In the failing logs, one host's 5 MiB fetch fails and retries (chunks still missing), the second preferred copy lands about 60 s in, and the extra copies still need their trim grace. The assertions are unchanged.🤖 Generated with Claude Code