Repository navigation
test(ffe): cover the SPLIT reason for time-bounded sharded allocations - #73
Merged
Merged
Conversation
Add a holdout-shaped flag: a time-bounded allocation with sharded splits returns SPLIT, and subjects outside the holdout traffic fall through to an unbounded rollout that returns STATIC.
danyal002
marked this pull request as ready for review
November 10, 2026 21:21
4 tasks done
leoromanovsky
approved these changes
Nov 11, 2026
leoromanovsky
left a comment
Collaborator
There was a problem hiding this comment.
The combined fixture preserves the original holdout coverage, adds targeting precedence and selected-metadata assertions, and includes regenerated artifacts. Local Go and Java validation passes; pre-fix Java fails exactly the four expected SPLIT cases. Fixture CI passes; remaining downstream advisory jobs are still running.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Time-bounded holdouts must report
SPLITfor both sharded groups. ReportingDEFAULThides the assignment reason, while incorrect allocation selection can return the wrong serial ID or logging decision. The existing bounded, single-unsharded fixture coversDEFAULT; this adds the holdout shape and consolidates the complementary coverage from #74.Changes
Add eight cases for
time-bounded-holdout-flag: four inside the holdout, two outside it, and two matching a targeting override. Every case asserts the value, reason, selected serial ID, anddoLogdecision. The original six subjects, salts, holdout activation metadata, and unbounded rollout are preserved.flowchart LR Subject[Subject] --> Target{tier is internal?} Target -->|yes| Override[TARGETING_MATCH / targeted / no log] Target -->|no| Member{Inside holdout traffic?} Member -->|yes| Group[SPLIT / status-quo or shipped / log] Member -->|no| Rollout[STATIC / shipped / no log]Document the subject buckets and regenerate the committed protobuf, offline wire, and manifest from the updated UFC with the existing converter lock.
Decisions
For eligible allocations, matching targeting rules take precedence when classifying the reason. Without targeting rules, a bounded allocation with one unsharded split reports
DEFAULT, a sharded selection reportsSPLIT, and the unbounded unsharded fallback reportsSTATIC. The existing date fixture supplies the boundedDEFAULTcontrol, while this fixture keeps an unboundedSTATICfallback.The active 2022–2050 window avoids depending on an imminent clock boundary. These cases verify evaluator output, including the logging decision; they do not prove exposure delivery or exact start/end-boundary behavior. Generated protobuf reasons remain unspecified where absent in authored UFC; generation does not imply protobuf evaluator parity.
Validation of the combined fixture:
dd-trace-go@69789cc762d04a203b80b54aebef7d839f5fdeca: 447 fixture evaluations pass, including all eight holdout cases and existing opt-out cases.dd-trace-java@8061195730434bc7b58a8a5b38f8dad80c658168(#13352): 448 canonical-runner tests pass, including all eight holdout cases and existing opt-out cases.98c28c6c01208f18053a026b471b951f2282d16b): exactly the four holdoutSPLITcases fail with actualDEFAULT; targeting, fallback, and all other tests pass.UNKNOWNreason.Checklist
config/ufc-config.jsonevaluation-cases/The template's config path refers to the canonical
ufc-config.jsonat the repository root.