Repository navigation
Python: fix FAISS delete leaving stale vectors after removing several records - #14702
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Conversation
… records remove_ids shifts every later vector down by one position, but the key map kept the old positions, so later deletes removed the wrong vectors. Update the remaining positions after each removal.
alexcastgonz (pcbeingused333)
deployed
to
github-app-auth
November 1, 2026 00:07 — with 
GitHub Actions
Active
GitHub Actions
Active
semantic-kernel-automation
Bot
added
the
python
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Deletion can corrupt key mappings when index removal fails and incorrectly renumber stable-ID indexes, leaving orphaned vectors.
2 open issues
What changed in this PR
This PR addresses #14695 by updating the Python FAISS connector’s key mappings after flat-index deletions, preventing subsequent deletes from targeting stale positions.
Changes:
- Decrement later mapped positions after each vector removal.
- Add regression tests for deleting all records and maintaining correct positions after partial deletes.
| File | Description |
|---|---|
python/tests/unit/connectors/memory/test_faiss.py |
Adds full and partial deletion regression tests. |
python/semantic_kernel/connectors/faiss.py |
Updates remaining key positions after vector deletion. |
Review effort: Balanced
Comment on lines
+193
to
194
| vector_index = key_map.pop(key) | ||
| self.indexes[vector_field].remove_ids(np.array([vector_index])) |
Comment on lines
+195
to
+198
| # removing a vector shifts every later vector down by one position | ||
| for other_key, position in key_map.items(): | ||
| if position > vector_index: | ||
| key_map[other_key] = position - 1 |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #14695
Motivation and Context
FaissCollection.deletewith several keys leaves stale vectors in the FAISS index: the repro in #14695 ends withntotal == 1and an empty key map, so later searches can return vectors for records that no longer exist.Root cause:
_inner_deleteremoves a vector withremove_ids, which shifts every later vector in a flat index down by one position, but the key map kept the old positions. Every delete after the first then targeted the wrong position.Description
After removing a key's vector, decrement the stored position of every key that sat after it in the same vector field's map. The change is limited to
_inner_deleteinpython/semantic_kernel/connectors/faiss.py.Tests added in
tests/unit/connectors/memory/test_faiss.py:test_delete_multiple_keeps_index_in_sync: the issue's scenario; index and key map are empty after deleting all records.test_delete_updates_remaining_positions: positions stay correct after partial deletes.Both fail on main and pass with the change; the rest of
test_faiss.pypasses, andruff format --check/ruff checkare clean on the touched files.Contribution Checklist
This change was prepared with the help of an AI coding assistant.