fix(deploy): survive Hugging Face 429 rate limits - #74
Merged
Merged
Conversation
… sharded model, optional HF_TOKEN Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The retry, authentication, caching, and test changes are coherent with no unresolved correctness issues found.
Review effort: Balanced
Findings: None
What changed in this PR
Adds resilient Hugging Face model downloads to reduce deployment failures from shared-runner rate limits.
Changes:
- Adds retry/backoff handling with
Retry-Aftersupport and tests. - Adds optional authenticated downloads and request throttling.
- Caches model artifacts in deployment CI.
| File | Description |
|---|---|
scripts/fetch-retry.mjs |
Implements retry and backoff behavior. |
scripts/fetch-retry.d.mts |
Defines retry helper types. |
scripts/prepare-llm.mjs |
Uses retries, authentication, and throttling. |
src/lib/fetchRetry.test.ts |
Tests retry behavior and delays. |
.github/workflows/ci.yml |
Caches the sharded deployment model. |
.github/workflows/llm-bench.yml |
Supplies optional Hugging Face authentication. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The deploy failed with HTTP 429 while fetching the model from Hugging Face (shared runner IPs are rate-limited).
scripts/fetch-retry.mjs: retries 408/429/5xx and network errors with exponential backoff, honouringRetry-After(capped at 60 s). 404 and other 4xx fail immediately.prepare-llm.mjs: uses the retry helper, sendsHF_TOKENwhen present, and pauses 0.5 s between files.ci.yml: cachesdist/llmkeyed on the script's hash, so a normal deploy no longer touches Hugging Face.llm-bench.yml: passesHF_TOKENas well.src/lib/fetchRetry.test.ts(6 cases).HF_TOKENis optional: add it as a repo secret (read-only HF token) to lift the anonymous limit.