DOC-6827 Add Lettuce probabilistic data type examples and rundown page - #3610
Open
andy-stark-redis wants to merge 4 commits into
Open
DOC-6827 Add Lettuce probabilistic data type examples and rundown page#3610andy-stark-redis wants to merge 4 commits into
andy-stark-redis wants to merge 4 commits into
Conversation
Stage Lettuce async + reactive doctests for the bloom, cuckoo, CMS, top-k, t-digest and HLL tutorial sets plus the combined home_prob_dts set, and add the client-specific rundown page at content/develop/clients/lettuce/prob.md. This is preemptive work: probabilistic command support is not in a released lettuce-core yet, so the examples are written against the Lettuce feature PRs (BF/CF/top-k merged to main; CMS and t-digest still open on the Dgramada fork, so their signatures may still shift). A Codex review flagged every non-HLL call as an unknown method - that is expected against the repo's pinned released jar, not a defect. Multi-step sets (t-digest, home_prob_dts) are authored as self-contained per-STEP blocks rather than one method chain threaded through the STEP markers, because the clients-example shortcode extracts each step by line range and a threaded chain renders as broken mid-chain fragments. Learned: multi-step doctests must be self-contained per STEP block or the shortcode extracts broken mid-chain fragments Constraint: each STEP_START/STEP_END block must stand alone (its own async chain, or per-statement reactive blocks), never span a single threaded method chain Rejected: one async/reactive chain threaded through all STEP markers | shortcode extracts per-step line ranges, so a threaded chain yields uncompilable fragments Recheck: when Lettuce CMS (redis/lettuce#3821) and t-digest (#3823) merge and release - re-pin lettuce-core, re-run the doctest harness so the REMOVE-block asserts become real oracles, and re-diff the CMS/t-digest signatures Ticket: DOC-6827 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Contributor
andy-stark-redis
added
the
parked
Lettuce 7.7.0.RELEASE (2026-08-18) ships all six probabilistic families,
so the park trigger fired and the examples could finally be compiled and
run rather than reasoned about. Three defects surfaced that five rounds
of signature analysis had all passed as clean.
t-digest took `double`, not `V`. 32 compile errors across the four
t-digest-touching files: `tdigestAdd`, `tdigestRank` and `tdigestCDF`
declare `(K, double...)`, and the examples passed quoted strings
("45.88", "50"). Every park-time note checked method name, arity and
return type — never the parameter type — and `(K, V...)` vs
`(K, double...)` are identical on all three. The 08-10 note's own
shorthand `tdigestAdd(k, v)` encoded the wrong assumption and it was
carried forward through four re-verifications. Fixed by unquoting 68
numeric literals, scoped to the t-digest call spans so the BF/CF/CMS
sibling calls in the same files kept their string values.
Reactor forbids a null from map(). The two reactive examples did
`.map(v -> v.getValueOrElse(null))` on Top-K replies, which throws
NullPointerException at runtime — so those files had never once been
executed. Lettuce returns `Flux<Value<String>>` reactively but
`RedisFuture<List<String>>` async, precisely because a Flux cannot carry
null while a List can. The examples had copied the async expected output
and forced nulls through a mapper to match it. Dropping the mapper lets
the real `Value.empty` / `Value[handlebars]` shape surface, which is the
honest reactive idiom rather than a fiction that matches the async tab.
One expected output was simply wrong, in the async file too. Top-K
eviction for home_prob_dts printed
[null, null, null, Rock me, Handel, Only one more time, null] — two
evictions at positions 4 and 5 — where the comment claimed five nulls
then a single "Rock me, Handel". It survived because that println had no
assertion at all; the reactive Top-K one had only a .size() check. Added
content assertions at all three sites so the >>> values are now guarded,
and verified the eviction order is stable across three fresh runs before
baking it in.
Verified: all 14 compile against 7.7.0 and all 14 pass against Redis
8.8.0, with 132 of 132 expected-output comments matching observed stdout
(mechanically diffed, not spot-checked). Negative control run: perturbing
one new assertion does fail the build.
Learned: a tripwire guarding a false conclusion preserves the falsehood
perfectly — all four pinned blob SHAs matched at the release tag, which
correctly proved nothing had moved and told us nothing about whether the
claim was ever true.
Recheck: prob.md needed no edit — no bannerText, no version-gating prose.
Recheck: 7.7.0 has neither the reactive() deprecation nor the
commands(factory) replacement, though #3731's diff read "@deprecated
since 7.7"; #3781 landed and was reworked away before the tag. Do not
document commands(factory) as a migration path yet.
Gaps: tdigestByRevRank, tdigestRevRank, tdigestTrimmedMean, tdigestInfo,
cmsInitByDim and cmsMerge remain uncovered by these examples.
The harness pinned lettuce-core 6.5.5.RELEASE, which cannot compile the repo's own Lettuce examples: the same 43 non-DJL example files give 200 compile errors at 6.5.5 and 0 at 7.7.0. Only 104 of those errors are this PR's probabilistic files — the other 96 are pre-existing arrays, JSON and query examples that had already outgrown the pin. So this is a fix to a stale pin, not a bump taken for this PR's convenience. Verified through the real harness rather than a throwaway project: all 7 probabilistic sets pass in both async and reactive, and so do cmds_hash (incl. sync), set_and_get, cmds_string, cmds_sorted_set, cmds_generic and arrays_tutorial. Nothing regressed. Learned: run.sh:407 seeds a toolchain pom with `[ -f "$d/pom.xml" ] || cp ...`, so editing a tracked pom-*.xml has no effect while a cached work/<client>/pom.xml exists. The first run after this bump reported FAIL with every tdigest method "cannot find symbol" — which reads exactly like a missing-API problem rather than a stale toolchain, and my throwaway maven project had already passed on the same sources, which is what exposed the contradiction. Delete work/<client>/pom.xml after changing a client version. Recheck: the four DJL-dependent vec examples (HomeQueryVecExample, HomeVecSetsExample) cannot compile under any harness pom — none carries ai.djl.* — so they are outside this verification either way, not a regression from the bump.
Contributor
🧠 Redis MemoryFound 5 related items from repository history (5 new this commit):
Memory updated at 512ba42 |
andy-stark-redis
removed
do not merge yet
parked
andy-stark-redis
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Lettuce (Java) coverage for the probabilistic data types, in two parts.
Examples
Staged Lettuce async + reactive doctests under
local_examples/<set>/lettuce-async|lettuce-reactive/:bf_tutorialcuckoo_tutorialcms_tutorialtopk_tutorialtdigest_tutorialhll_tutorialhome_prob_dtsThe per-type data-type pages embed
clients-examplewithout alang_filter, so thenew Java-Async and Java-Reactive tabs appear automatically once the example
pipeline regenerates
examples.json— no edits to those pages are needed, and noneare in this PR. Confirmed in a local build: all six probabilistic data-type pages
pick up both tabs.
Rundown page
New
content/develop/clients/lettuce/prob.md, adapted from the Jedis page, withlang_filter="Java-Async,Java-Reactive".Test harness
Re-pins
build/example-test-harness/pom-lettuce-{async,reactive,sync}.xmlfromlettuce-core6.5.5.RELEASE to 7.7.0.RELEASE.Flagging this as the one change here that isn't docs, in case you'd rather it landed
separately. It is a fix to a stale pin rather than something this PR needs for
convenience: the same 43 non-DJL Lettuce example files give 200 compile errors at
6.5.5 and 0 at 7.7.0, and only 104 of those errors are this PR's files — the other 96
are pre-existing arrays, JSON and query examples that had already outgrown the pin.
Every other Lettuce set still passes at the new pin (
cmds_hashincl. sync,set_and_get,cmds_string,cmds_sorted_set,cmds_generic,arrays_tutorial).Requirements
The probabilistic commands need Lettuce 7.7.0 or later (released 2026-08-18), which
is the first release carrying Bloom, Cuckoo, CMS, t-digest, Top-K and HLL support.
In line with the Jedis, redis-py, Go and Node.js probabilistic pages — none of which
state a client version requirement —
prob.mdcarries no version banner.Verification
Not inspection-only; the examples were compiled and run:
lettuce-core7.7.0.RELEASE.(
./run.sh <set> lettuce-async lettuce-reactive) as well as directly.>>>expected-output comments were mechanically diffed againstobserved stdout, rather than spot-checked. All match.
fresh runs.
Notes for review
Two things worth a look, both of which only surfaced once the examples could actually
be executed:
double, notString.tdigestAdd,tdigestRankandtdigestCDFtake adoublevalue parameter, so the numeric arguments are unquoted.Valuewrapper, and the async ones shownull.This asymmetry is deliberate in Lettuce, not an inconsistency in the docs: reactive
returns
Flux<Value<String>>while async returnsRedisFuture<List<String>>,because a
Fluxcannot emitnullwhereas aListcan hold it. So the reactive tabprints
Value.empty/Value[handlebars]where the async tab printsnull/handlebars.Not covered
tdigestByRevRank,tdigestRevRank,tdigestTrimmedMean,tdigestInfo,cmsInitByDimandcmsMergeexist in 7.7.0 but are not exercised by these examples —possible future coverage, not a gap in what is documented here.