Skip to content

[ISSUE-830] Fix inverted embedding value filter that dropped digit-free text - #832

Open
E2ern1ty wants to merge 1 commit into
apache:masterfrom
E2ern1ty:fix/embedding-verbalize-filter-inverted
Open

[ISSUE-830] Fix inverted embedding value filter that dropped digit-free text#832
E2ern1ty wants to merge 1 commit into
apache:masterfrom
E2ern1ty:fix/embedding-verbalize-filter-inverted

Conversation

@E2ern1ty

@E2ern1ty E2ern1ty commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Fixes #830.

What changes were proposed in this pull request?

The value filter feeding EmbeddingIndexStore kept values containing a digit or one of . _ - @ + ! $ % & = ~ and dropped everything else, so ordinary prose was never embedded while a bare id, a date or a run of punctuation was. SearchUtils.isAllAllowedChars returned on the first character inside the ignorable set rather than the first one outside it, which is the negation of both its name and its Javadoc, and SubgraphSemanticPromptFunction.verbalize(GraphEntity) keeps a value when that predicate is false.

Two changes, matching the split agreed in the issue.

Correct the predicate, and rename it to isAllIgnorableChars. It now returns true only when every character is ignorable, and treats null and empty as ignorable since callers use it to decide what to skip and there is nothing in them to index. The rename is deliberate rather than a fix in place: with the name unchanged, a caller depending on the previous meaning would silently flip behaviour, whereas a rename makes it fail to compile. Both call sites are in SubgraphSemanticPromptFunction and read the same as before, !isAllIgnorableChars(value). IGNORE_CHARS becomes IGNORABLE_CHARS, and the Javadoc now describes the set as characters that carry no meaning on their own.

Stop reporting entities with no embeddable text as indexed. This is the half that held regardless of the predicate's intended semantics, and it was requested on the issue. A digit-free entity logged Successfully added 1 new index items. Total indexed: 1 while producing nothing: the entity was registered in indexStoreMap with an empty vector list, so it could never be recalled, and nothing distinguished that from a successful build. indexBatch now warns with a count and an example key when entities yield no text, and the summary line reports how many entities actually hold vectors rather than how many were queued.

How was this PR tested?

  • Tests have Added for the changes
  • Production environment verified

Unit tests. IgnorableTextFilterTest, 7 cases: the predicate over values that do and do not carry meaning, the property that whether a value is kept must not depend on it containing a digit, verbalization of digit-free vertex and edge text, verbalization dropping a value that is only a date, and the store either attempting a request or correctly declining to. All 7 fail against the previous behaviour. The two store cases run offline: an unusable ModelConfig makes "a request was attempted" observable without a service, since reaching the request at all fails on the null url. Full geaflow-ai suite passes, 13 tests, 0 checkstyle violations.

End-to-end against a real embedding service. Both master (3f73eb55) and this branch were run through the full production path against SiliconFlow BAAI/bge-m3: real HTTPS requests, real 1024-dimension vectors, real on-disk index file, real recall through GraphMemoryServer. Corpus is the module's own text/Confucius, 532 chunks of Chinese prose, of which 532 contain no digit. Same harness source compiled against both trees, so the numbers are comparable.

master this branch
entities holding vectors 0 of 532 532 of 532
index file lines written 0 548
embedding requests issued 0 23, for 549 texts
summary log Successfully added 532 new index items. Total indexed: 532 Successfully added 532 new index items. Entities holding vectors: 532 of 532
recall of a known chunk, embedding vector only not possible, no vectors returned, cosine 0.9999
recall for an unseen question, 学习和思考的关系是什么? not possible top chunk is 子曰:“学而时习之,不亦说乎?…”, cosine 0.5835
rebuild from the index file just written n/a 0 new requests, 532 of 532 restored, recall still returns the chunk

So on master the whole corpus was reported as indexed while nothing was embedded and no request was ever issued. Recall was exercised with an embedding vector only, no keyword vector and no other index store registered, so anything returned had to come from the embedding index. The written index file holds 548 records of 1024 real dimensions each, 11.35 MB, keyed by entity, with no zero vectors.

The counterpart case on the same endpoint, 20 date-only values, which is what the old predicate selected for:

master this branch
entities holding vectors 20 of 20 0 of 20
embedding requests issued 1, for 20 inputs 0
log Successfully added 20 new index items. Total indexed: 20 WARN 20 of 20 entities have no embeddable text and will hold no vectors, for example Vd1chunk, then Entities holding vectors: 0 of 20

The two tables together are the inversion the issue describes, measured against a live model rather than argued: master spends requests on dates and none on prose, this branch does the opposite and says so when there is nothing to embed.

Blast radius, measured before changing anything. On the LDBC test dataset, of 168 entities the set wanting vectors is identical before and after the correction, and none of them are absent from the committed LDBCEmbeddingIndexStore file. The two predicates differ only on values that are entirely digits or punctuation, and on values that are entirely letters; LDBC entities have neither. The committed embedding index therefore needs no regeneration and GraphMemoryTest passes unchanged, which is also why the defect was invisible until now. It does mean the module's own end-to-end scenario was affected: MemoryServerTest imports those same 532 prose chunks, none of which would previously have been embedded.

The value filter feeding the embedding store dropped ordinary prose and kept
digit-only noise, so an embedding store silently produced nothing for any text
that happened to contain no digit. No request was issued, no error was raised,
and the entity was still registered as indexed with an empty vector list, which
makes it unrecallable.

SearchUtils.isAllAllowedChars returned on the first character *inside* the
ignorable set instead of the first one outside it, which is the negation of both
its name and its Javadoc. Reproduced on master with an unusable ModelConfig:
"no digits here at all" completes without contacting a model and writes zero
index lines while logging "Successfully added 1 new index items", whereas the
same text with a digit appended does attempt the request.

- Correct the predicate and rename it to isAllIgnorableChars. Renaming rather
  than fixing in place is deliberate: a caller depending on the old meaning now
  fails to compile instead of silently flipping behaviour. Null and empty become
  ignorable, since callers use this to decide what to skip and there is nothing
  in them to index. Both call sites are in SubgraphSemanticPromptFunction and
  read the same as before, `!isAllIgnorableChars(value)`.
- Stop reporting entities with no embeddable text as indexed. indexBatch now
  warns with a count and an example key, and the summary line reports how many
  entities actually hold vectors rather than how many were queued. Silence was
  half of this defect.

Measured on the LDBC test dataset before changing anything: of 168 entities,
the set wanting vectors is the same before and after the fix, and none of them
are absent from the committed index file. LDBC values mix letters and digits, so
both the old and the new predicate keep them; the two differ only on values that
are entirely digits or punctuation, and on values that are entirely letters. The
committed embedding index therefore needs no regeneration and GraphMemoryTest
passes unchanged.

Tests in IgnorableTextFilterTest, 7 cases: the predicate over values that do and
do not carry meaning, the property that keeping a value must not depend on it
containing a digit, verbalization of digit free vertex and edge text, and the
store either attempting a request or correctly declining to. All 7 fail against
the previous behaviour.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

geaflow-ai: is the embedding value filter meant to drop prose and keep digits?

1 participant