Skip to content

FFL-3568: Define shared mobile precomputed response conformance - #65

Merged
leoromanovsky merged 14 commits into
mainfrom
leo.romanovsky/precompute-fixture-fanout
Nov 4, 2026
Merged

leoromanovsky merged 14 commits into
mainfrom
leo.romanovsky/precompute-fixture-fanout

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

The same precomputed flag response can return different values across client SDKs. A JSON decoder can round a fractional value into an integer before a getter checks it, and a test runner can hide that defect by parsing and re-encoding the response. Fixture PRs also need to show their effect on native client tests before the fixtures reach downstream branches.

This PR defines the shared precomputed-response contract for numeric getters, flag-key obfuscation, and emitted events, and adds native compatibility previews. Ref: FFL-3568.

Changes

  • Expand the corpus to 63 fixtures, including plain and generated obfuscated numeric matrices, native accessor boundaries, fixed Unicode digest vectors, malformed encodings, and unsupported consumers. CI rejects missing, stale, or orphaned companions.
  • Preserve original numeric tokens through obfuscation and snapshot import. Snapshots include source SHA, file digests, an index, and checksum-covered raw response sidecars so runners feed the SDK the actual fixture input.
  • Protect snapshot refreshes from deleting unrelated files: reject repository roots, .git paths, symlink destinations, nonempty directories without valid provenance, and files outside the previous snapshot.
  • Enforce the rejection contract in the schema: INVALID_RESPONSE cases perform no evaluations, require noUnmatchedEvents: true, and cannot expect exposure or evaluation events. Correct the documentation for unsupported variation types, numeric compatibility, and duplicate-evaluation aggregation.
  • Add advisory iOS XCTest on macOS and Android JVM previews to fixture PRs. Each job resolves one SDK commit, runs both base and proposed fixture snapshots against it, and uploads native reports, logs, and a comparison. Completed mismatches are advisory; setup/build failures or missing test results are reported as incomplete previews.
flowchart LR
    P[Fixture PR: base and proposed] --> I[Import snapshots preserving response bytes]
    I --> S[One recorded SDK commit per platform]
    S --> X[iOS XCTest on macOS]
    S --> A[Android JVM tests]
    X --> R[PR comparison and native artifacts]
    A --> R
Loading

The preview temporarily follows leo.romanovsky/precompute-fixture-fanout for iOS and leo.romanovsky/ffl-3568/precompute-fixture-fanout for Android. Switch these to develop after the canonical runners land. Flutter is validated in its downstream PR and locally; it is not a job in this preview.

Consumer implementation and validation

All three consumer snapshots pin fixture commit f2e3dc4. Validation below is local unless linked to a hosted run; it does not establish native-bridge or deployed-backend compatibility.

Consumer Revision Evidence and remaining limits
Android #4101 c84ae6142e 925 feature tests and 257 integration tests pass; canonical subset: 104 executed, 3 platform exclusions. Formatting and detekt pass.
iOS #3397 fb0d73f71c 246 getter/cache checks pass against production model source on Swift/Linux. Full revised XCTest integration awaits the macOS preview. Per-fixture error isolation remains open.
Flutter #1281 6b669f4df9 Package suite: 330 passed, 1 existing skip; after an additional cache regression, the focused 32-test suite passes on VM and Node/dart2js. Analyzer is clean. Browser/WASM corpus portability, strict result comparisons/UTF-8 transport, and additional public-getter/boundary tests remain open in that PR.

Shared validation: 63 fixtures validate; generated companions match; 30 fixture-tooling and 10 preview-tooling tests pass. Hosted fixture validation passes at f2e3dc4; the mobile preview for that revision is the source for native results. Earlier preview results cover the older corpus.

Decisions

Numeric reads depend on the original JSON value, regardless of the integer, float, or number label, and retain the same behavior after cache restoration:

Getter Rule Example
Integer Accept mathematically whole values within the accessor range; reject fractions even if decoding would round them to integers. 4503599627370497.5 → TYPE_MISMATCH
Double, fractional input Round to the nearest finite Double. 1.20000000000000001 → 1.2
Double, whole input Accept only when the original integer is exactly representable. 9007199254740993 → TYPE_MISMATCH
Double, overflow or nonzero underflow Return the caller default with TYPE_MISMATCH. Signed zero is numerically zero. 1e400 and 1e-400 → TYPE_MISMATCH

There is no new global safe-integer limit for native Integer getters. Platform filters describe accessor capabilities, not implementation bugs. The precompute service retains number; no numeric capability is added.

Unsupported variation types are omitted independently and read as FLAG_NOT_FOUND. Rejected conversions suppress exposures; evaluation events record the error. Duplicate evaluations aggregate into one event with evaluation_count: 2. Obfuscated lookup retains plaintext event identity, does not fall back to plaintext lookup, and rejects encoded responses on unsupported consumers.

Consumers import generated snapshots with immutable provenance and preserve shared expectations. Merge this contract before the consumer PRs; after merge, refresh their pins to the merged revision. Unity and React Native runtime changes are outside this implementation. The downstream work and open review items above remain separate from approval of this contract.

Checklist

  • JSON files are valid (no syntax errors)
  • New test cases reference existing flags in config/ufc-config.json — not applicable; these fixtures contain precomputed responses.
  • New flags have corresponding test cases in evaluation-cases/ — not applicable; no rules-based flags changed.
  • Expected results match the flag configuration logic — precomputed expectations follow the contract above.

@leoromanovsky
leoromanovsky marked this pull request as ready for review November 4, 2026 00:07
@leoromanovsky
leoromanovsky requested a review from a team as a code owner November 4, 2026 00:07
Comment thread precomputed-assignments/README.md Resolved
Comment thread precomputed-assignments/README.md Resolved
Comment thread ci/sync-precomputed-fixtures.py Resolved
Comment thread AGENTS.md Resolved
Comment thread precomputed-assignments/README.md Resolved

@aarsilv aarsilv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Approving with some p2s and p3s love for you to consider.

@leoromanovsky
leoromanovsky merged commit 9afc3f7 into main Nov 4, 2026
5 checks passed
@leoromanovsky
leoromanovsky deleted the leo.romanovsky/precompute-fixture-fanout branch November 4, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants