fix(health): stop reporting time series as supersede candidates, and hot topics as sediment gaps - #36
Merged
Conversation
…hot topics as sediment gaps
Both signals in memory-health were firing on the wrong thing. On a 9301-row
library the supersede band listed 125 pairs and the repeat-query list 4; a hand
review found ~1 real item across both. A report that is 99% false positive does
not get acted on — it gets skimmed, and the one real pair in it goes with the rest.
## (a2) — time series read as rewrites
The band exists to catch "an already-replaced version stays active and can be
recalled as if current". A time series has no replaced version: every entry is
still true about its own moment, and superseding one destroys the sequence.
The old guard only excluded pairs where BOTH rows were concrete_trace. Series
written at semi_abstract sailed through — nightly metric snapshots, timestamped
log lines from one evening, a request and its reply, a decision and its
refinement three minutes later.
isLikelySeries now takes three signals, any one sufficient:
1. both concrete_trace (the old rule, kept)
2. written inside one working session (default 8h)
3. both summaries carry a temporal marker AND the markers differ
Signal 3 requires the markers to DIFFER on purpose: two rows citing the same
date are one event described twice, which is exactly the rewrite we want.
Calibrated against 10 hand-classified pairs pulled from the live library, not
from intuition — the widest same-session false positive sat 6.8h apart, the
closest true rewrite 5 days. Those 10 shapes are now series-detection.test.mjs
so the next change to the predicate has to argue with the data.
Effect: 125 -> 22 candidates. The one pair the band exists to find
("千夏不爱吃香菜" vs "千夏讨厌香菜的味道", 5d apart) moved from buried to first.
## (d) — frequency mistaken for evidence of a gap
Repeat queries were labelled "sediment-worthy" on frequency alone, while
SUM(hit_count) was already selected and then ignored. Frequency measures how hot
a topic is, not whether it is answered — and hot-and-answered is the normal case,
since hooks fire recall on every prompt containing the word.
Measured before fixing: every freq>=3 query in the 7d window averaged 5-20 hits.
The signal was 100% false positive. "watchdog" x12 was the loudest, and recall
returns three good cards for it.
Now filtered to avg hits < 1, with the answered ones reported as a count so the
information is not lost. Live: 4 -> 0 flagged, 4 hidden as hot topics.
[prediction] 修复: (a2) 和 repeat-queries 从"扫一眼就跳过"变成可逐条判的短名单,
真正的 stale rewrite 不再被同一晚的日志淹掉
[prediction] 风险: 8h 同会话窗口是 10 个样本标定的,会漏掉"当天内真的改写了同一条事实"
这种;代价是漏一条候选,比 125 条噪音便宜。若将来出现实例,调 seriesSameSessionHours
而不是拆掉规则
[prediction] 验证: 下次跑 memory-health,(a2) 应在 20-30 量级且首条是真候选;
repeat queries 段若非空,每条 avg_hits 都应 <1
Co-Authored-By: 千夏 <qianxia@clawgamers.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both signals in
memory-healthwere firing on the wrong thing. On a 9301-row library the supersede band listed 125 pairs and the repeat-query list 4; a hand review found ~1 real item across both.A report that is 99% false positive doesn't get acted on — it gets skimmed, and the one real pair goes with the rest.
(a2) — time series read as rewrites
The band exists to catch "an already-replaced version stays active and can be recalled as if current." A time series has no replaced version: every entry is still true about its own moment, and superseding one destroys the sequence.
The old guard only excluded pairs where both rows were
concrete_trace. Series written atsemi_abstractsailed through — nightly metric snapshots, timestamped log lines from one evening, a request and its reply, a decision and its refinement three minutes later.isLikelySeriesnow takes three signals, any one sufficient:concrete_trace(the old rule, kept)Signal 3 requires the markers to differ on purpose: two rows citing the same date are one event described twice, which is exactly the rewrite we want to surface.
Calibrated against 10 hand-classified pairs pulled from the live library rather than from intuition — the widest same-session false positive sat 6.8h apart, the closest true rewrite 5 days. Those 10 shapes are now
series-detection.test.mjs, so the next change to the predicate has to argue with the data.Effect: 125 → 22 candidates. The one pair the band exists to find (
千夏不爱吃香菜vs千夏讨厌香菜的味道, 5d apart) moved from buried to first.(d) — frequency mistaken for evidence of a gap
Repeat queries were labelled
sediment-worthyon frequency alone, whileSUM(hit_count)was already selected and then ignored.Frequency measures how hot a topic is, not whether it's answered — and hot-and-answered is the normal case, since hooks fire recall on every prompt containing the word.
Measured before fixing: every
freq>=3query in the 7d window averaged 5–20 hits. The signal was 100% false positive.watchdog×12 was the loudest, and recall returns three good cards for it.Now filtered to
avg hits < 1, with the answered ones reported as a count so the information isn't lost. Live: 4 → 0 flagged, 4 hidden as hot topics.Verification
series-detection.test.mjs— 11 new cases, red before the change (no such export), green aftermemory-health.test.mjs— 68/68, unchangedRisk
The 8h same-session window is calibrated on 10 samples. It will miss "genuinely rewrote the same fact within one day" — that costs one missed candidate, against 103 removed false ones. If a real instance shows up, tune
seriesSameSessionHoursrather than removing the rule.