Context
bm reindex --embeddings already has a Rich progress bar in a real terminal, and #579 fixed the non-TTY rendering issue there back in v0.19.0 — that part is solved for someone actively watching the CLI run.
The gap is different: there's no supported way for something running outside that terminal session — a background/headless process — to poll current embedding progress. On a local FastEmbed/CPU setup, an initial sync or reindex over a large vault can run for a long time, and anything watching from outside (a status widget, a monitoring script, a CI step) has no stable signal to read.
I ended up writing a small script that polls ~/.basic-memory/memory.db directly, counting rows in search_vector_chunks vs search_vector_embeddings_rowids to compute pending chunks, and uses that to drive a simple progress display. It works, but it depends on internal table names that could change without notice — not something I'd want to rely on long-term.
Feature request
Expose embedding/chunk progress through an existing, documented surface instead of requiring internal DB reads:
- Add total/embedded/pending chunk counts to
bm status --json (alongside the existing file-sync info), so any external tool or script can poll a stable, documented field instead of reading internal tables.
This would let background tooling (dashboards, monitoring scripts, CI health checks) observe embedding progress without depending on Rich/TTY output or undocumented schema.
Alternatives considered
- Parsing
bm reindex --embeddings terminal output — fragile, and not available to a process that isn't the one running the reindex.
- Reading
~/.basic-memory/memory.db directly (what I'm doing now) — works today but is an implementation detail, not a supported interface.
Happy to test against a real large vault if useful.
Context
bm reindex --embeddingsalready has a Rich progress bar in a real terminal, and #579 fixed the non-TTY rendering issue there back in v0.19.0 — that part is solved for someone actively watching the CLI run.The gap is different: there's no supported way for something running outside that terminal session — a background/headless process — to poll current embedding progress. On a local FastEmbed/CPU setup, an initial sync or reindex over a large vault can run for a long time, and anything watching from outside (a status widget, a monitoring script, a CI step) has no stable signal to read.
I ended up writing a small script that polls
~/.basic-memory/memory.dbdirectly, counting rows insearch_vector_chunksvssearch_vector_embeddings_rowidsto compute pending chunks, and uses that to drive a simple progress display. It works, but it depends on internal table names that could change without notice — not something I'd want to rely on long-term.Feature request
Expose embedding/chunk progress through an existing, documented surface instead of requiring internal DB reads:
bm status --json(alongside the existing file-sync info), so any external tool or script can poll a stable, documented field instead of reading internal tables.This would let background tooling (dashboards, monitoring scripts, CI health checks) observe embedding progress without depending on Rich/TTY output or undocumented schema.
Alternatives considered
bm reindex --embeddingsterminal output — fragile, and not available to a process that isn't the one running the reindex.~/.basic-memory/memory.dbdirectly (what I'm doing now) — works today but is an implementation detail, not a supported interface.Happy to test against a real large vault if useful.