Bug: vector_chunks rows whose embed failed are stored text-only and never repaired, so they stay invisible to vector recall forever

[YellowSnnowmann]

Version or commit

tinymemory v1.14.1 (52836fd), running as the prebuilt module in openhuman main f5c5ba892.

Rust toolchain

rustc 1.96.1 (aarch64-apple-darwin)

Reproduction

1. Ingest documents while the embedder cannot answer: signed out, budget exhausted, provider down, or a local Ollama that is not running (Memory sync ignores the configured memory embedding provider and 401s against the cloud embedder on local installs openhuman#5730 is one common way in).
2. Later restore the embedder and keep using the app: sync again, restart, sign in, switch providers.
3. Inspect memory.db:

SELECT namespace, COUNT(*) AS rows, SUM(embedding IS NULL) AS no_vector
  FROM vector_chunks GROUP BY 1 ORDER BY 2 DESC;

Live desktop store, 2026-09-04:

namespace	rows	no vector
skill-gmail	39,328	30,178
global	148	124
conversation_raw	120	18
source_gmail_ca_RiwezSJ_5CiT	8,835	36

30,358 rows in total, the oldest from 2026-05-27, the newest from 2026-09-04 10:28. None was ever retried.

Expected behavior

A chunk whose embed failed is a durable to-do, not a permanent hole: some pass (startup, sync completion, sign-in, provider change, or a scheduler tick) re-derives the set of vector-less rows, embeds them in provider-sized batches, and writes a vector back only when its width matches the active embedder. Transient failures back off; unrecoverable ones (no session) wait for the next trigger.

Actual behavior

• crates/tinymemory-core/src/store/namespace_store/documents.rs (upsert_document_presanitized): when the batch embed fails, every chunk of the document is written with embedding = NULL and model_signature = NULL, by design ("stores all chunks WITHOUT a vector").
• Nothing revisits those rows. The only writers of vector_chunks are the per-document DELETE … WHERE document_id on rewrite and DELETE … WHERE namespace on forget. The reembed_backfill job (engine/queue_driver.rs::reembed_batch) covers mem_tree_chunk_embeddings and mem_tree_summary_embeddings only.
• The reader (namespace_store/query.rs:630-637) silently skips rows with no usable vector, so the documents stay findable by keyword only and invisible to vector recall, and no health surface reports the count.

Additional context

This is item 1 of tinyhumansai/openhuman#5300, whose fix PR (openhuman#5258) implemented exactly this sweep against the host's legacy memory subsystem and became un-rebasable when that subsystem was removed. The store side now lives here.

Shape that fits the current crate:

1. A vector_chunks counterpart of has_uncovered_reembed_work: SELECT … WHERE embedding IS NULL (plus rows whose model_signature is an alias of the active one, see Bug: namespace store and tree sign the same embedder with different provider labels (managed vs cloud), so every vector written under the other alias is skipped by recall #139) bounded by LIMIT.
2. A queue job or maintenance member that embeds one bounded batch (the tree backfill already sizes to 1000 texts per request), writes width-matching vectors in one transaction, marks unrecoverable rows (empty text, oversize) in a skip ledger the way mem_tree_chunk_reembed_skipped does, and defers while more remain.
3. Triggers: the same ones the tree backfill uses (ensure_reembed_backfill call sites) plus sync completion and sign-in.
4. Surface the pending count in doctor / pipeline status so the Sources row can say "N chunks waiting for vectors" instead of nothing (Sources row warns "Stored without vectors" for a transient embedding backlog after a large sync openhuman#6025 adds the row-side wiring).

Related: #139 (label alias; the re-sign migration there should run before this sweep so aliased rows are not re-embedded), #138, tinyhumansai/openhuman#5300, tinyhumansai/openhuman#5730.