fix: bot-serving monitor must target the Cloud Run origin (Cloudflare 403s runner IPs)#9619
Merged
Merged
Conversation
The first dispatched run failed with 403 on all four checks including the human-UA control: Cloudflare's bot management blocks GitHub-runner (datacenter) IPs, and UA-spoofed Googlebot only passes verified-bot checks from real Google IPs. The origin is exactly the layer that broke in the 2026-06/07 incident, so checking it directly is also the more precise probe. Adds an llms.txt check while here (must be served directly, never proxied). Dry-run against the live origin: 5/5 OK. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes the structural limitation of the GitHub Actions “bot-serving” synthetic monitor by switching its probes from the Cloudflare-proxied domain (anyplot.ai) to the Cloud Run origin, avoiding Cloudflare bot-management 403s from GitHub-hosted runner IPs while still validating the exact serving layer implicated in the recent incident.
Changes:
- Updated
bot-serving-check.ymlto target the Cloud Run origin for all UA checks and added an explicit/llms.txtprobe. - Documented the rationale for origin-targeting (Cloudflare runner-IP blocking) directly in the workflow header comment.
- Updated the changelog entry to reflect the new origin-based monitoring behavior and the added
/llms.txtcheck.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| CHANGELOG.md | Updates the [Unreleased] entry to reflect origin-targeted monitoring and the new llms.txt check. |
| .github/workflows/bot-serving-check.yml | Switches curl targets to the Cloud Run origin and adds an /llms.txt validation check; documents why Cloudflare edge is out of scope. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bot-serving-check.ymlrun failed with 403 on all four checks — including the human-UA control: Cloudflare's bot management blocks GitHub-runner (datacenter) IPs, and a UA-spoofed "Googlebot" only passes verified-bot checks when coming from real Google IPs. The monitor was structurally unable to run from Actions against the proxied domain.anyplot-app-…-ez.a.run.app) — exactly the layer that broke in the 2026-06/07 incident, so this is also the more precise probe. Cloudflare-edge issues are explicitly out of this monitor's reach (documented in the header comment).llms.txtcheck (Googlebot UA must get the file, never a seo-proxy 404) — guards the exact-match nginx location from feat: spec-conformant llms.txt for AI agents #9618.Verification
Test plan
workflow_dispatchrun is green