Skip to content

Evaluate feature flag regex conditions in linear time - #13487

Draft
leoromanovsky wants to merge 3 commits into
leo.romanovsky/ffe-scale-suite-benchmarkfrom
leo.romanovsky/ffe-regex-linear-time
Draft

leoromanovsky wants to merge 3 commits into
leo.romanovsky/ffe-scale-suite-benchmarkfrom
leo.romanovsky/ffe-regex-linear-time

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

This PR is stacked on #13485, which adds the scale-suite benchmarks and the master baseline. Review only the commits after it.

The FFE provider now evaluates MATCHES and NOT_MATCHES in linear time, with the dialect of the Rust regex crate that libdatadog uses. A long attribute value can no longer make one evaluation backtrack for milliseconds.

Motivation

java.util.regex backtracks. On a 2 KB attribute value that does not match, the customer pattern shape .*@(site-a|site-b)\.example$ takes about 15 ms for each evaluation. A chain such as .*.*.*.*.*= does not finish within 5 s. Attribute values come from the application (emails, user agents), so this is a latency risk and a ReDoS risk. ReDoS is a denial of service that uses slow regular expressions.

Java also gives different results from the Rust-based SDKs (libdatadog) for some patterns. For example, $ before a final line feed matches in Java but not in Rust, and lookahead patterns match in Java but are a parse error in Rust.

Changes

  • Add RegexMatcher. At parse time, it gives each pattern the cheapest matcher with the same result, in the same order of tiers as the Rust evaluator (dd-source rkyv/regex.rs):
    1. Literal checks (equals, startsWith, endsWith, contains) for patterns that are only text, with anchors, a leading or trailing .*, and groups of literal alternatives.
    2. Checks that need no engine: .* and the empty pattern always match; .+; and \bliteral\b with Unicode word boundaries.
    3. RE2/J (com.google.re2j, already in the version catalog), a linear-time engine. A required literal prefix, or a set of required literal suffixes, is checked first.
  • Translate patterns to the Rust dialect before RE2/J compiles them:
    • \d and \s become Unicode classes.
    • The u flag is removed.
    • \Q, class set operations and nested classes are rejected.
  • The parser compiles each distinct pattern once for each configuration. The customer-scale fixture has 404 regex conditions but only 70 distinct patterns.
  • Add RegexMatcherBenchmark (JMH) to compare java.util.regex with RegexMatcher for each tier.

Scale suite results

Same machine and settings as the #13485 baseline: AWS workspace, Intel Xeon Platinum 8175M, 16 vCPUs, JDK 21.0.10, JMH AverageTime, 1 fork, 3 warm-up and 5 measurement iterations of 2 s, -prof gc. Each row is the mean over the cases of the file. The runs were back to back.

Case file master (#13485) This PR Change
scale/regex (2 KB inputs, 300 duplicate conditions) 1.82 ms, 12,201 B 32.2 µs, 444 B 57× faster
scale/regex-many (1,000 distinct patterns, 500 alternatives, counted repetition) 21.0 µs, 42,164 B 14.3 µs, 506 B 1.5× faster
scale/rules (one MATCHES in each rule) 3.29 µs, 1,904 B 2.74 µs, 368 B 1.2× faster
regex/backtracking does not finish 77.4 µs, 305 B
regex/literals not run 274 ns, 304 B
Suite load master This PR
scale parse time 95.8 ms 103 ms (±17 ms)
scale allocated by one parse 90.6 MB 96.6 MB
scale retained heap about 24.2 MB about 25.6 MB

This PR adds the literals and backtracking case files of the regex suite to ScaleSuiteBenchmark, because the JDK engine on master does not finish the backtracking cases.

Decisions

  • A pattern that the Rust crate rejects is a parse error. This covers lookaround, backreferences, atomic groups and \Q. The flag is dropped with PARSE_ERROR, as libdatadog does and as the shared case test-case-invalid-regex-isolation.json expects. Before this change, Java evaluated these patterns.
  • RE2/J, not a new engine. RE2/J is already a dependency of the repository (dd-trace-core, IAST). A DFA tier, as in Rust, could make complex patterns on short inputs faster, but it needs a memory limit and more code.
  • Behavior changes, because the Rust dialect is now used:
    • $ does not match before a final line feed.
    • . matches \r.
    • \d and \s are Unicode.
    • The bare POSIX translation ([:alnum:] to \p{Alnum}) is removed. Bracketed POSIX classes, such as [[:alnum:]], still work.
  • Known differences from the Rust crate, listed in the RegexMatcher Javadoc:
    • In the RE2/J tier, \w and \b use ASCII word characters.
    • \D and \S inside a bracketed class are ASCII.
    • Some syntax is a parse error here but valid in Rust: the x and R flags, \<, \>, \b{...}, class set operations, and nested repetition such as a*+.

Additional Notes

Benchmark. I used the flag-eval-bench harness, with the Datadog provider called through the OpenFeature client and evalPrepared. The runs used Linux x86_64 with JDK 21, pinned to 4 CPUs, on a shared host with a load average of about 4.8. I ran 1.69.0 and this branch back to back. The branch numbers are from before the review-fix commit. The in-module results are in "Scale suite results" above.

Scenario 1.69.0 mean This branch mean Ratio
regex-wildcard-long (2 KB miss) 15.6 ms 2.2 µs about 7,000× faster
regex-charclass-long 46 µs 8.4 µs 5.5× faster
regex-wildcard 6.8 µs 2.8 µs 2.5× faster
regex-charclass 3.6 µs 2.5 µs 1.5× faster
mixed (customer-scale profile) 6.1 µs 5.3 µs 1.2× faster
regex-dup 3.5 µs 3.5 µs same
regex-complex 3.9 µs 4.6 µs 0.8× (slower)
  • Allocation per evaluation is lower in every scenario.
  • regex-complex (^u[0-9]{1,N}@(?:a|b|…)\.(?:io|net)$) is slower when the required suffix matches. In that case RE2/J runs its NFA, which is slower than the JDK engine on short inputs.

Cross-SDK check. DataDog/ffe-system-test-data#79 adds an opt-in regex suite with 63 cases. Its expected results come from libdatadog, and this branch gives the same result for all 63. The submodule pin of #13485 includes that suite, and this PR benchmarks its literals and backtracking case files.

Tests. On this stack, all 1,621 dd-openfeature tests pass, including the canonical fixtures. /sdk-review ran on the first commit, and the second commit fixes its findings.

Size. This PR adds more than 1,000 counted lines. The author will add tag: override pr size check.

Jira ticket: [PROJ-IDENT]

@leoromanovsky leoromanovsky added type: feature Enhancements and improvements tag: performance Performance related changes tag: ai generated Largely based on code generated by an AI or LLM comp: openfeature OpenFeature labels Nov 17, 2026
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
• Patch Coverage: 83.30%
• Overall Coverage: 82.46%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a571f3a | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.83 s 14.82 s [-0.8%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent 13.58 s 13.72 s [-1.8%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent 17.22 s 17.11 s [-0.2%; +1.5%] (no difference)
startup:petclinic:iast:Agent 17.12 s 17.07 s [-0.6%; +1.1%] (no difference)
startup:petclinic:profiling:Agent 16.68 s 16.71 s [-1.2%; +0.7%] (no difference)
startup:petclinic:sca:Agent 17.17 s 17.10 s [-0.4%; +1.3%] (no difference)
startup:petclinic:tracing:Agent 16.33 s 16.22 s [-0.2%; +1.6%] (no difference)

Commit: a571f3a8 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

leoromanovsky and others added 3 commits November 17, 2026 00:55
Replace java.util.regex for MATCHES and NOT_MATCHES with tiers like the
Rust evaluator: literal checks for patterns that are only text, checks
that need no engine for .*, .+ and \b-literal patterns, and RE2/J with
required prefix and suffix checks for all other patterns. A long
attribute value can no longer make one evaluation backtrack for
milliseconds.

Use the dialect of the Rust regex crate, which libdatadog uses: $ matches
only at the end of the input, . does not match a line feed, the u flag
is accepted, and lookaround, backreferences and atomic groups are a
parse error. The parser compiles each distinct pattern once per
configuration.
- Read each multi-character escape (\p{..}, \pX, \x{..}, \xHH, octal) as
  one token, so that its characters are not a required literal suffix.
- Translate \d, \D, \s and \S to the Unicode classes of the Rust regex
  crate. Reject \Q, class set operations and nested classes, which the
  Rust crate rejects or reads differently from RE2/J.
- Remove a - that is left without a flag after the u flag is removed.
- Keep a surrogate pair together when a quantifier follows it.
- Return an unmodifiable list of required suffixes.
- Share the group and alternation helpers, remove the unused pattern
  field, and use matches() in place of an expectMatch parameter.
- Move the parser tests to UniversalFlagConfigParserTest.
- Add a JMH benchmark that compares java.util.regex with RegexMatcher.
The base scale benchmark cannot run the regex suite, because the JDK
engine does not finish its backtracking cases. With the linear-time
matcher it does, so add its literal and backtracking case files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leoromanovsky
leoromanovsky force-pushed the leo.romanovsky/ffe-regex-linear-time branch from ad69415 to a571f3a Compare November 17, 2026 01:00
@leoromanovsky
leoromanovsky changed the base branch from master to leo.romanovsky/ffe-scale-suite-benchmark November 17, 2026 01:00
@leoromanovsky leoromanovsky added the crucible: reviewing A Crucible review has started; this PR has reviewer attention. label Nov 17, 2026 — with ddtool CLI
@leoromanovsky leoromanovsky removed the crucible: reviewing A Crucible review has started; this PR has reviewer attention. label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: openfeature OpenFeature tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant