Skip to content

Let the structure reading vectors say how they are read - #16682

Open
jimczi wants to merge 1 commit into
apache:mainfrom
jimczi:agent/knn-read-advice-main
Open

jimczi wants to merge 1 commit into
apache:mainfrom
jimczi:agent/knn-read-advice-main

Conversation

@jimczi

@jimczi jimczi commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

A format opening a file can attach hints to say what it holds and how it will be read, and a
directory can act on them. MMapDirectory turns them into madvise calls if an application asks it
to. Today the vectors formats get two things wrong there.

The first is who is talking. How a file is read depends on what is reading it, not on what it holds.
The same flat vectors are walked all over the place under an HNSW graph and scanned front to back by
an exact search, yet it was the flat format claiming random access for both. And during a merge it
could say nothing at all, because IOContext.merge(...).withHints(...) returns the same context and
drops the hints, as does flush(...).

So merge and flush contexts now carry hints, and the HNSW formats carry the random hint rather than
the flat ones, at search time and while a merge builds the graph. Formats that can be wrapped follow
their caller; a format reading its own files, like the HNSW graph or the Faiss index, still sets its
own.

The second is that RANDOM is too coarse to act on. It is really two claims in one: don't read
ahead, and don't bother remembering these pages. On mmap they come together, since random advice
takes a mapping out of the page reclaim's recency tracking. That is what you want for stored fields,
term vectors and the raw vectors kept only to rescore, which land somewhere else on every query. It
is the opposite of what you want for the vectors a graph walks, which are the ones you most want to
keep in memory. So NoReuseHint splits them: those three set it, the vectors under a graph don't.

Nothing behaves differently here. The point is that an application can now tell the two apart and
pick, which it could not do before.

@jimczi
jimczi force-pushed the agent/knn-read-advice-main branch from 870c58c to 935462d Compare September 18, 2026 08:29
@jimczi
jimczi force-pushed the agent/knn-read-advice-main branch from 935462d to 51e7467 Compare September 18, 2026 08:52
@jimczi
jimczi force-pushed the agent/knn-read-advice-main branch 3 times, most recently from 3702692 to de1246c Compare September 18, 2026 13:39
@jimczi jimczi changed the title Carry file open hints on merge and flush contexts Let the format that walks vectors say how they are read Sep 18, 2026
@jimczi
jimczi force-pushed the agent/knn-read-advice-main branch 2 times, most recently from fd8af2a to 00fe77d Compare September 18, 2026 14:26
@jimczi jimczi changed the title Let the format that walks vectors say how they are read Let the structure reading vectors say how they are read Sep 18, 2026
How a file is read depends on what is reading it, not on what it holds.
The same flat vectors are walked all over under an HNSW graph and
scanned front to back by an exact search, yet the flat format claimed
random access for both, and during a merge it could say nothing at all:
IOContext.merge(...).withHints(...) returned the same context and
dropped the hints, as did flush(...).

Merge and flush contexts now carry hints, and the HNSW formats carry the
random hint rather than the flat ones. Formats that can be wrapped
follow their caller, and one reading its own files still sets its own.

Random access is really two claims in one: don't read ahead, and don't
bother remembering these pages. On mmap they come together. Add
NoReuseHint for the files where both are wanted, stored fields, term
vectors and the raw vectors kept only to rescore, so an application can
tell them apart from the vectors a graph walks. SegmentReadState and
SegmentWriteState gain shallow copy constructors taking an IOContext.
@jimczi
jimczi force-pushed the agent/knn-read-advice-main branch from 00fe77d to a2d28af Compare September 18, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant