media-model-proxy: fail closed when every NSFW replica drops scores - #167
Open
Pitchfork-and-Torch wants to merge 1 commit into
Open
Conversation
collectToTry + filterException swallowed every NonFatal xx-nsfw error, including greyscale decode failures that were already mapped to GreyScaleImageException. The controller then returned a successful empty annotation map, which Media Analysis persists as unlabeled / SFW. Keep partial-replica failover; fail the request when every attempted replica errors. Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Media Model Proxy is the production path that scores tweet images, videos, and GIFs for
XX_NSFW(and the other media-understanding models) before Media Analysis persists the result. When every Deepbird replica errors,getPredictioncurrently succeeds with an empty score list. Callers treat that as unlabeled / SFW, so adult media ranks and ships as safe whenever xx-nsfw is down, times out, or rejects the bytes (including greyscale).This is not a Scala-parity leftover and is not in the claimed VF / Grok / Thunder / DNA / ads / Phoenix lanes.
Five-line proof
ModelsService.getPredictionusesFuture.collectToTryandfilterExceptionto drop everyNonFatalreplica error, includingPredictionServiceExceptionandGreyScaleImageException.assert(result.isEmpty).ProxyControllerthen buildsAnnotationResponse(..., annotations = Some(predictions.toMap))and increments success. An emptySeq[DataRecord]is a successfulXxNsfwkey.nsfw.default.scoreis "not NSFW."GreyScaleImageException(BadRequest, MediaNotSupported), thenfilterExceptionswallows it so the client never sees the BadRequest.Partial replica failure still returns the surviving scores. Kill-switched models (no replica attempted) still return empty.
Fix
If any replica was attempted and zero valid scores remain, fail the Future with the first error (
fail_closedcounter). Greyscale decode errors now surface asGreyScaleImageExceptioninstead of a successful empty map.Tests
PredictionServiceExceptionand otherNonFatalerrors nowinterceptinstead of accepting empty.GreyScaleImageException.Scala/Bazel is not available in this environment, so
ModelsServiceSpecwas not executed here.