fix(extraction): retry transient model failures once and give a failed terminal flush one retry - #997
Draft
gorkem2020 wants to merge 1 commit into
Draft
Conversation
…d terminal flush one retry completeJson folds every failure into null, so the extraction path could not tell a silent upstream (timeout, 5xx, connection reset, host-transport failure) from an unusable answer; the grounding rejudge failed closed on both and demoted every real-tagged durable in the batch, and the batch counted as settled. A terminal flush whose extraction failed handed its turns back to the deferred bucket, but the session had already ended and nothing consumed that bucket again. Both model calls in the extraction path now go through completeJsonWithTransientRetry: the client's last error classifies a null with the repository's existing transient classifier, a transient upstream failure is retried once after the shared backoff, and an upstream failure of either class reports the run as unavailable (ExtractionStats.llmUnavailable, paired with extractionFailed) so the caller defers the batch instead of judging it. A terminal flush whose extraction fails schedules exactly one unref()ed retry per session key; the retry re-runs the terminal flush, is cancelled when a later run consumes the bucket, and gives up after its single attempt. Regressions: upstream-failure detection, retry-once outcomes for the transient, non-retryable, unusable-answer and no-diagnostics clients, a full extractAndPersist run whose judge never answers (deferred, nothing persisted, nothing demoted), a failed terminal flush retried from its own timer, a second failure that gives up, and a later flush cancelling the pending retry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two deferred items from earlier review rounds (#927 and #934), both about the extraction path's behavior when the model or the gateway does not answer.
completeJsonfolds every failure intonull, so the extraction path could not tell a silent upstream (timeout, 5xx, connection reset, host-transport failure) from an unusable answer. The grounding rejudge failed closed on both, demoting every real-tagged durable in the batch, and the batch counted as settled. Failing closed on a verdict is right; failing closed on silence should not look the same.session_endby hand.Changes
src/extraction-transient-retry.ts(new):completeJsonWithTransientRetrywraps a model call, reads the client's last error after anull, and classifies it with the classifier the repository already ships insrc/reflection-retry.ts. An upstream request failure of the transient class is retried once after the shared backoff; an upstream failure of either class reports the run asunavailable; a parse failure or empty answer is still treated as the model's answer (no retry, not unavailable).src/smart-extractor.ts: both model calls in the extraction path (extract-candidates,grounding-rejudge) go through that wrapper. When the judge is unavailable the batch is handed back (status: "llm_unavailable") instead of failing closed; nothing is persisted and nothing is demoted.SmartExtractorConfig.transientRetrySleepis a test seam for the backoff.src/memory-categories.ts:ExtractionStats.llmUnavailable, always paired withextractionFailed, so the caller can tell "the model never answered" from "the model answered nothing usable".index.ts: the auto-capture hook logs the unavailable case explicitly (the rate limiter was already not charged onextractionFailed). A terminal flush whose extraction fails now schedules exactly oneunref()ed retry per session key (15 s); the retry re-runs the terminal flush for that key, is cancelled when any later run consumes the bucket, and gives up after its single attempt._setAutoCaptureTerminalFlushRetryDelayMsForTestshortens the delay for tests only.test/extraction-transient-retry.test.mjs(upstream-failure detection, retry-once outcomes for transient, non-retryable, unusable-answer and no-diagnostics clients, and a fullextractAndPersistrun whose judge never answers: deferred, nothing persisted, nothing demoted) andtest/autocapture-terminal-flush-retry.test.mjs(a failed terminal flush retries from its own timer and persists on the retry; a second failure gives up without a third attempt; a later flush that consumes the bucket cancels the pending retry). Both registered in thenpm testchain and the CI manifest (core-regression).Notes
Verification
npm run build,npm test, the two new regression files (red against the previous code by construction), and the existinggrounding-rejudge,extraction-grounding-register,autocapture-watermark-reset,autocapture-fallback-gating,smart-extractor-noise-gating,reflection-embed-transient-retryandcontext-support-e2esuites.