MOD-17269 Document built-in and custom detectors for sensitive-data exclusions - #3917
Draft
plamilieva wants to merge 5 commits into
Draft
MOD-17269 Document built-in and custom detectors for sensitive-data exclusions#3917plamilieva wants to merge 5 commits into
plamilieva wants to merge 5 commits into
Conversation
Replace both console screenshots with captures of the shipped card, which now offers three mechanisms: built-in detectors, custom detectors, and semantic exclusions. NOTE: the prose still documents only semantic exclusions, so the pages are now inconsistent with their own screenshots. Documenting the two detector mechanisms is follow-up work, not done here. Gaps: Built-in detectors, custom detectors, per-detector on-match actions (redact match / drop memory), and the drop-wins precedence rule are visible in both screenshots but undocumented. Ticket: MOD-17269 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console card now offers three exclusion mechanisms; the pages described one. Adds the two detector mechanisms as sibling subsections in the card's own order, so the screenshots and the prose agree again. Detector facts come from the shipped catalog rather than the console tooltips, which truncate: the five entries' Matches text is the catalog's own Description for each id (email, credit-card, us-ssn, phone, ip-address), and each one states its precision trade-off deliberately, since a false positive silently costs a memory. Custom detector limits are the domain's: 32 detectors, 512-character patterns, 64-character identifier names. Patterns compile through Go's regexp, so the syntax is RE2 and lookaround and backreferences are unavailable. On match is documented once, shared by both detector mechanisms, because the two groups resolve into one flat list and a disagreement is settled by drop-wins (StrongerOf) rather than by which group a detector came from. Redaction's placeholder is a single fixed [REDACTED] token, not a per-detector label. Moved the scope sentence up to the section: screening runs in the extraction worker for all three mechanisms, so "not applied to direct writes" was never semantic-specific. The advisory warning stays with semantic exclusions and loses its direct-writes clause, which now sits above and covers all three. Detector matches are deterministic, so the advisory caveat must not appear to cover them. Early-stage/selected-accounts note left as is, per product. Constraint: Detector matches are deterministic. Never fold them under the advisory caveat, which is true only of semantic exclusions. Ticket: MOD-17269 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Contributor
🧠 Redis MemoryFound 8 related items from repository history (1 new this commit):
Memory updated at 52e2ddc |
An exclusion prompt is prose, not a selection from a taxonomy. Calling it "plain-language categories" was already loose; it became actively misleading once built-in detectors shipped, because selecting from a fixed set of named patterns is exactly what that other mechanism is. A reader meeting both would reasonably take the prompt for a category picker. The developer guide keeps its examples and now says the prompt describes what must not be kept. The feature list drops the word: "information that should not be kept" is true of every mechanism, so the summary no longer implies one. Ticket: MOD-17269 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The section was titled for the feature but described only the semantic prompt, so a developer reading it would not learn that a store can exclude by pattern at all. Now names the three mechanisms and what each is for, and states the on-match choice and the drop-wins rule once. Kept at guide altitude: no settings tables, no authoring limits, no field-level rules. Those live on the console page this section already links to. Deliberately omits detector ids and the detector catalog endpoint. The API model warns that ids copied from documentation drift, and the public data-plane spec exposes no exclusions surface at all, so this would be pointing developers at an API they cannot yet read. Detectors are named the way the console labels them. The advisory caveat now opens by distinguishing deterministic detector matches from the advisory prompt, so the warning cannot be read as covering all three. Constraint: Do not publish built-in detector ids in prose. The catalog endpoint is the source of truth; ids in docs drift. Ticket: MOD-17269 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The self-managed section had no mention of exclusions, though the on-prem control plane accepts a full policy on store create and update and serves the detector catalog at GET /v1/detectors. Adds both to the Control Plane API examples: a catalog request, and a store created with all three mechanisms, plus a field table covering the constraints a caller hits on write. The example payload is validated JSON, its detector ids are real catalog entries, and its custom pattern compiles and consumes text. Ids appear here only as a response body and an echo of one, never as a list to copy: the endpoint is the contract, and the model warns that ids taken from docs drift. Concepts are not restated — the mechanisms, the on-match choice, and the paths exclusions cover are linked to the developer guide. Scoped to Control Plane managed stores on purpose. Static stores are the other documented mode, but iris removed that mode (MOD-17629) and the on-prem data plane now rejects metadata.stores outright, telling the operator to create stores through the control plane. Documenting exclusions there would have described a removed feature. Gaps: The self-managed pages still document the removed static-stores mode (plan-deployment, deploy-static, data-plane-configuration, prerequisites), and the bundled openapi-control-plane.json predates exclusions, so the fields used here are absent from the rendered API reference until it is regenerated. Constraint: Do not list built-in detector ids as copyable reference. GET /v1/detectors is the source of truth. Ticket: MOD-17269 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
plamilieva
marked this pull request as draft
September 3, 2026 15:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: MOD-17269 (epic: MOD-17267)
Follow-up to #3760, which documented sensitive-data exclusions when only the semantic mechanism had shipped. The console card now offers three mechanisms, so the pages described one third of what a reader sees.
What this adds
Updated both screenshots to the shipped card, and documented the two detector mechanisms as sibling subsections in the card's own order:
The mechanism-neutral intro from #3760 meant this slotted in without rewriting the intro, the early-access note, or the page's section list.
Where the facts come from
Not the console tooltips — two of them truncate mid-sentence. The Matches text is each catalog entry's own
Descriptioninmemory/common/domain/detectors/catalog.go(email,credit-card,us-ssn,phone,ip-address). Every entry deliberately states its precision trade-off, because a false positive silently costs a memory, so those caveats are carried through verbatim.Custom detector limits are the domain's: 32 detectors, 512-character patterns, 64-character identifier names. Patterns compile through Go's
regexp, so the syntax is RE2 — lookaround and backreferences are unavailable, which is worth saying in a field where someone types a regex.On matchis shared rather than duplicated per mechanism because the two groups resolve into one flat list, and a disagreement is settled by drop-wins (StrongerOf), not by which group a detector came from. Redaction's placeholder is a single fixed[REDACTED]token, not a per-detector label.One correctness fix
The scope sentence moved up to the section, and the advisory warning lost its direct-writes clause.
Screening runs in the extraction worker for all three mechanisms, so "not applied to long-term memories created directly through the API or an SDK" was never semantic-specific — it belongs above, covering all three. Conversely, detector matches are deterministic, so leaving the advisory caveat where it could be read as covering them would have been wrong rather than merely imprecise.
Availability
The early-stage / selected-accounts note is unchanged — still accurate per product.
Verification
.claude/hooks/check_shortcode_paths.py --scanreports 0 issues, 0 broken file refs, 0 broken relrefs on both pages. All four new in-page anchors (#built-in-detectors,#custom-detectors,#on-match,#semantic-exclusions) resolve to real headings, and#custom-detectorsdoes not collide with the existing#custom-memory-types. Still no code blocks increate-service.md.🤖 Generated with Claude Code
Note
Low Risk
Documentation-only updates with no runtime or security behavior changes in this repository.
Overview
Expands Redis Agent Memory sensitive-data exclusions docs so they match the console: three independently enabled mechanisms—built-in detectors, custom regex detectors, and semantic exclusions—instead of describing semantic exclusions alone.
Create / view service pages add catalog details for the five built-in detectors (what each matches and precision caveats), custom detector settings (RE2 limits, up to 32), shared On match behavior (
redactvsdrop, drop-wins on conflict), and section-level scope (automatic extraction only; session memory and direct API writes unchanged). Warnings now separate deterministic detector behavior from advisory semantic steering.The developer guide and landing bullet are aligned with that model. Self-managed API examples add
GET /v1/detectors, a samplelongTermMemoryExclusionsstore payload, and a field reference for store create/update.Reviewed by Cursor Bugbot for commit 52e2ddc. Bugbot is set up for automated code reviews on this repo. Configure here.