You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9c3aab4
Browse filesBrowse the repository at this point in the historyBrowse files
fix(pii): mask texts over spaCy's length limit and drop encoded files from logs (#9979)
* fix(pii): keep encoded files away from Presidio and split texts too long for one NER pass
* fix(pii): mask long texts inside Presidio and limit encoded-file replacement to the log
- Presidio runs pattern and custom recognizers over the whole text and spaCy NER in overlapping windows when a text exceeds spaCy's max_length, so no match can straddle a cut; the app no longer splits texts
- Encoded file bodies are replaced only in the execution log (explicit replaceEncodedBinary option set by the log entry points); live data keeps every string on the masking path
- Data URL headers are parsed through the first comma, and log-mode detection no longer requires mixed character classes
- e2e: setup failures are recorded in the report
* fix(pii): keep context scoring for thresholded long texts and match data: case-insensitively
- Long-text NER windows also run the pattern entities when a score_threshold is set, so context-boosted matches clear it as they do on short texts
- The data: URI scheme is matched case-insensitively
- e2e asserts each name is masked as one <PERSON> span and is registered as test:pii-redaction:e2e
* fix(pii): keep every NER window's spans instead of assuming a neighbour found them
NER reads context, so a window can find a name the previous one missed or cut short; spans are now only deduplicated by exact position, and overlapping spans are left to the anonymizer to merge
Copy file name to clipboardExpand all lines: apps/docs/content/docs/platform/enterprise/data-retention.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -118,6 +118,8 @@ Redaction applies to text strings at the configured workflow stages. It does not
118
118
119
119
If log redaction fails, affected strings are replaced with `[REDACTION_FAILED]` rather than stored unmasked. Failure at Workflow input or Block outputs stops the run instead of passing unredacted values downstream.
120
120
121
+
Base64-encoded file content, such as a `data:` URL or a long base64 string, can't be checked for PII as text, because any PII is inside the encoded file. In stored logs it is replaced with `[BINARY_CONTENT_REDACTED]`. At Workflow input and Block outputs it is masked like any other string. **Run from block** and re-running with a previous run's input read that run's log, so they see the marker in place of that content, the same way they see masked PII.
0 commit comments