Skip to content

Refresh the shared FFE fixture snapshot - #8529

Draft
typotter wants to merge 2 commits into
mainfrom
typo/system-tests-corpus-pin
Draft

typotter wants to merge 2 commits into
mainfrom
typo/system-tests-corpus-pin

Conversation

@typotter

Copy link
Copy Markdown
Contributor

Motivation

Refresh the shared FFE fixture snapshot, including explicit UFC evaluation reasons from DataDog/ffe-system-test-data#76.

Changes

Import the upstream evaluator and selection fixtures with the repository fixture-update script and update SOURCE.md. This includes the latest upstream dependent-flag cases and time-bounded holdout case. The snapshot contains 453 evaluator cases and 38 selection SDK cases. Runner code and SDK versions are unchanged.

Validation:

  • Fixture importer and mock flag-selection regression tests: 119 passed.
  • All 65 imported fixture files match the upstream snapshot byte for byte.
  • Repository formatter and git diff --check passed.
  • Node.js parametric execution is blocked before tests run: Docker lists a pre-existing container that its inspect API cannot find. SDK conformance remains unverified.

Two imported holdout cases still expect STATIC while the input UFC supplies DEFAULT; that upstream discrepancy is preserved.

This draft depends on fixture PR #76. It targets main directly and contains no changes from RUM FIT #221.

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Tests, manifest, weblog are modified -> you'll need a review from system-tests-reviewers: ask to one of youre co-worker familiar with the tested feature.
    • Framework is modified, or non obvious usage of it -> get a review from system-tests-core (slack)

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/parametric/test_ffe/ffe-system-test-data/evaluation-cases/test-case-time-bounded-holdout-flag.json  @DataDog/feature-flagging-and-experimentation-sdk
tests/parametric/test_ffe/ffe-system-test-data/SOURCE.md                @DataDog/feature-flagging-and-experimentation-sdk
tests/parametric/test_ffe/ffe-system-test-data/evaluation-suites/dependent-flags/cases/dependent-flags.json  @DataDog/feature-flagging-and-experimentation-sdk
tests/parametric/test_ffe/ffe-system-test-data/evaluation-suites/dependent-flags/ufc-config.json  @DataDog/feature-flagging-and-experimentation-sdk
tests/parametric/test_ffe/ffe-system-test-data/evaluation-suites/flag-selection/ufc-config.json  @DataDog/feature-flagging-and-experimentation-sdk
tests/parametric/test_ffe/ffe-system-test-data/ufc-config.json          @DataDog/feature-flagging-and-experimentation-sdk

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant