Skip to content

fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374) - #5445

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5374-notcontains-inert-predicate
Aug 5, 2026
Merged

fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374)#5445
os-zhuang merged 1 commit into
mainfrom
claude/issue-5374-notcontains-inert-predicate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5374

MemoryAnalyticsService 把每个 cube 算子映射成一个 mingo 算子的名字,调用点再把这个名字填成 matchStage[field] = {[name]: comparand}。这个形状只能表达「拿这个字段和这个值比」,别的都表达不了 —— 于是那两个需要包装比较数的算子被硬塞了进去。

notContains'$not' 编译出 {name: {$not: 'et'}}。mingo 的 $not 期望正则或算子表达式,给它一个裸标量时它不构成约束,所以谓词生成了、在管线里看得见、然后放过了全表:3 行,而 find() 是 2 行。对作者而言,一个「生成了但不起作用」的谓词和一个正确的谓词完全不可区分 —— 与 #3948 同一个放大方向,只是第三次从另一个地方到达。

这也是同一张面上第三个独立的层:#5345 裁的是没有映射的算子,#5373 裁的是比较数编码,这一条是有映射但指向了错的目标#5431 没有让它变小:那个调用点现在收到的是真实值而不是字符串化的值,但那与算子层做了什么正交。

采用的路线

issue 给了两条:调用点包装,或者让映射表返回一个结构而不是算子名。选了后者 —— issue 本身倾向它,代码也确实支撑得起。

CUBE_OPERATOR_TO_MONGO_PREDICATE 每个算子持有一个 builder,返回整个 {$op: …} 对象。于是 notContains 可以说 {$not: {$regex: …}},而「这个算子需要一个结构,可表里只放得下一个名字」这一问题消失了,而不只是这一个实例。调用点原本为 $in / $nin / $lte / $exists 长出来的那串 if,现在都是这张表里的普通行。

实测(issue 的 3 行固定数据,analytics vs find())

where 修复前 修复后 find()
{name: {$notContains: 'et'}} 3 2 2
{name: {$notContains: 'a'}} 3 0 0
{name: {$contains: 'a.p'}} 1 0 0
{name: {$contains: 'ALPHA'}} 0 1 1
{name: {$notContains: 'ALPHA'}} 3 2 2
{made_at: {$contains: '(完整 ISO)'}} 1 0 0
{code: {$in: []}} 3 0 0
{code: []} 3 0 0

同一调用点上另外三处,一并关掉了

不是扩范围 —— 要写出一个正确的 notContains,这三条每一条都必须先裁掉:

issue 里标为「未实测」的两条,已经裁了

  • 'inDateRange': '$gte':今天什么都编译不出来。没有任何 MONGO_TO_CUBE_OPERATOR 条目降级到这个名字,timeDimensions 走的是 Stage 2、根本不经过这个函数,而两个出口都只消费 normalizeFilters 的输出。所以它是死的 —— 并且万一被走到还是错的(用单边 >= 回答一个双端区间,这一点它自己的注释也承认了 "Will need special handling")。删掉了,同时删掉旁边同样是死的、并且方向反了的 'notSet': '$exists'
  • opMap[operator] || '$eq':出于同样的理由不可达 —— 但只是在有人拓宽词表之前,而 driver-memory 的 analytics 面静默丢弃大半个 filter:$or/$not 整条丢,$between/$startsWith/$null/$regex 因无 cube 映射而丢 —— 聚合结果被放大 #5345 恰恰刻意把拓宽做成了对 MONGO_TO_CUBE_OPERATOR 的一行编辑。所以不只是删掉:那张表现在是 as const,谓词表的键类型是从它派生出来的算子 union,于是那一行拓宽编辑编译不过,直到对应的谓词写出来为止。剩下的那个 throw 是全域性的兜底(totality floor),不是 fallback。

这一点是本 PR 长期价值的所在:兜底不再是「不可达」,而是「不必要」—— 全域性被证明了,而不是被防守。对 AI 写的元数据尤其重要:宽容的消费端正是 AI 生成的错误藏身并繁殖的地方。

测试

放在共享的 conformance 文件里,紧挨着 #5345 的 shape 表和 #5373 的比较数类型表,作为第三条轴,持有同一个不变量:与 find() 一致,或者拒收,绝不给出第三个更安静的答案。

外加「declared = enforced」那一半:ANALYTICS_FILTER_CAPABILITIES 声明的每一个算子都被驱动着走两条路并必须一致,且探针必须至少排除一行(否则「一致」什么也证明不了 —— 那正是本 bug 的形状)。往词表里加算子却没有可用的降级,现在会在这里失败,而不是发布一个安静的错数字。

只回退源码改动(保留测试),新断言失败 14 条:

 × $notContains excludes the rows that contain the comparand
 × a $notContains that matches every row selects none of them
 × $contains treats a metacharacter as a literal
 × $contains is case-insensitive, as the live path is
 × $notContains is case-insensitive, as the live path is
 × $contains matches a mixed-case comparand
 × $contains does not rewrite its pattern into a datetime storage form
 × $notContains does not rewrite its pattern either
 × $notContains over a column holding nulls
 × an empty $in selects nothing
 × an empty implicit-equality list selects nothing
 × $contains and $notContains partition the table for every comparand
 × every operator this face DECLARES compiles to a predicate that agrees with find()
 × the emitted $match wraps the negation around a pattern instead of a bare scalar
 Tests  14 failed | 74 passed (88)

各门实际输出:

pnpm --filter @objectstack/driver-memory typecheck   -> tsc --noEmit, 无输出
npx eslint packages/plugins/driver-memory/src        -> LINT OK
pnpm --filter @objectstack/driver-memory test        -> Test Files 17 passed (17) / Tests 512 passed (512)
pnpm --filter @objectstack/service-analytics test    -> Test Files 47 passed (47) / Tests 745 passed (745)

packages/runtime 的 suite 在本 worktree 里因为工作区依赖未构建而整文件 import 失败(Failed to resolve entry for package "@objectstack/objectql" 等)。把本改动 stash 掉后失败完全相同,与本 diff 无关。

范围之外,已单独立项

operatorToSqlgenerateSql 一个字节都没动 —— #5433 是同一类缺陷在另一个出口上,刻意不打包在一起。packages/specservice-analytics 同样未触碰。

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7


Generated by Claude Code

…dicate that excludes rows (#5374)

`MemoryAnalyticsService` mapped each cube operator to the NAME of a mingo
operator, and the call site filled that name in as
`matchStage[field] = {[name]: comparand}`. That shape can express "compare this
field to this value" and nothing else, so the two operators that need to WRAP
their comparand were pushed through it anyway.

`notContains` -> `'$not'` became `{name: {$not: 'et'}}`. mingo's `$not` takes a
regex or an operator expression; handed a bare scalar it constrains nothing, so
the predicate was emitted, appeared in the pipeline, and passed the whole table:
3 rows where `find()` returns 2. A predicate that is emitted and inert is
indistinguishable from a working one at the author's end — the same amplifying
direction as #3948, arrived at a third way. #5345 ruled on operators with NO
mapping, #5373 on the comparand ENCODING; this is a mapping pointing at the
wrong target, and #5431 did not shrink it (that call site now receives a real
value, which is orthogonal to what the operator layer does with it).

Route: let the map return a STRUCTURE rather than an operator name, which the
issue prefers and the code supports cleanly. `CUBE_OPERATOR_TO_MONGO_PREDICATE`
holds a builder per operator that returns the whole `{$op: …}` object, so
`notContains` can say `{$not: {$regex: …}}` and the CLASS of "this operator
needs a structure and the table can only hold a name" is gone rather than this
one instance. `$in`/`$nin`/`$lte`/`$exists`, which the call site had grown an
`if` chain for, are ordinary rows in that table now.

Measured on the issue's 3-row fixture, analytics vs `find()`:

| where                            | before | after | find() |
|----------------------------------|--------|-------|--------|
| {name:{$notContains:'et'}}       |      3 |     2 |      2 |
| {name:{$notContains:'a'}}        |      3 |     0 |      0 |
| {name:{$contains:'a.p'}}         |      1 |     0 |      0 |
| {name:{$contains:'ALPHA'}}       |      0 |     1 |      1 |
| {name:{$notContains:'ALPHA'}}    |      3 |     2 |      2 |
| {made_at:{$contains:'<full ISO>'}}|     1 |     0 |      0 |
| {code:{$in:[]}}                  |      3 |     0 |      0 |
| {code:[]}                        |      3 |     0 |      0 |

Three more defects at the same call site fall inside this fix and are closed
with it, because writing a correct `notContains` requires settling each:

- `contains` was the right operator with the comparand handed in RAW, so it was
  neither escaped (`.` matched any character) nor case-folded, while the live
  path escapes and matches `/…/i`. Leaving that would have made the two
  non-complementary in a new way — `alpha` would be in BOTH answers. The rule is
  now borrowed from the driver (new narrow `filterSubstringPattern`, alongside
  `filterComparandStorageForm`) rather than re-derived, per #5240.
- An operand that is NOT a comparand went through the storage-form conversion
  anyway, so on a declared `datetime` column a `$contains` PATTERN was rewritten
  into canonical form and then matched rows `find()` does not match. The builder
  input carries both lists, the same split `normalizeFieldOperators` makes
  (#4047).
- The call site's `values.length > 0` guard meant an empty `$in` emitted no
  predicate at all and widened to the whole table. A list operator taking the
  whole list has nothing to guard.

The two items the issue flagged as unmeasured, settled:

- `'inDateRange': '$gte'` compiles to NOTHING today — no `MONGO_TO_CUBE_OPERATOR`
  entry lowers to that name, `timeDimensions` never reaches this function, and
  both exits consume only `normalizeFilters` output. Dead, and wrong if it ever
  had been reached (a one-ended `>=` for a two-ended range, which its own
  comment conceded). Deleted, with the dead-and-inverted `'notSet': '$exists'`
  beside it.
- `opMap[operator] || '$eq'` is unreachable for the same reason — but only until
  someone widens the vocabulary, which #5345 deliberately made a one-line edit
  to `MONGO_TO_CUBE_OPERATOR`. So it is not merely deleted: that table is `as
  const`, the predicate table is keyed by the operator union derived from it,
  and the widening edit now FAILS TO COMPILE until the predicate exists. The
  remaining throw is a totality floor, not a fallback.

Tests go in the shared conformance file beside the #5345 shape table and the
#5373 comparand-type table, as a third axis with the same invariant: agree with
`find()`, or refuse. Plus the "declared = enforced" half — every operator
`ANALYTICS_FILTER_CAPABILITIES` declares is driven through both faces and must
agree, with a probe that must exclude at least one row, so an operator added to
the vocabulary without a working lowering fails here instead of shipping a
quietly wrong number. Reverting only the source change fails 14 of the new
assertions.

Out of scope, filed not fixed: #5440 (two operators on one field clobber each
other — the `$match` assembly layer, still broken after this), #5442
(`flattenFilterCondition` spreads an array comparand for every operator), #5444
(the `generateSql` exit emits `LIKE 'et'` with no `%` wildcards — filed as a
sub-issue of #5433, whose completion scope it falls inside). `operatorToSql` and
`generateSql` are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 5, 2026 11:55am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/l labels Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-memory.

9 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-memory)
  • content/docs/deployment/vercel.mdx (via @objectstack/driver-memory)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-memory)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-memory)
  • content/docs/permissions/authentication.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/index.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-memory)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-memory)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-memory)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review August 5, 2026 12:02
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit 01c0bae Aug 5, 2026
24 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5374-notcontains-inert-predicate branch August 5, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

driver-memory analytics 面的 $notContains 编译成裸 mingo {$not: 'x'},该谓词不约束任何行 —— 结果被放大到全表

2 participants