Repository navigation
Require full unique indexes for lookup tables - #915
workstonedai-collab wants to merge 1 commit into
Conversation
Muguett-DBY
left a comment
There was a problem hiding this comment.
The ordinary-index duplicate case and partial-index case behave as described in my checks. I also tried a full expression index, which still makes lookup() fail:
db = sqlite_utils.Database(memory=True)
db.execute("CREATE TABLE species(id INTEGER PRIMARY KEY, name TEXT)")
db.execute("CREATE UNIQUE INDEX species_lower ON species(lower(name))")
db["species"].lookup({"name": "Palm"})
# AttributeError: 'NoneType' object has no attribute 'translate'SQLite's PRAGMA index_info reports None for the expression's column name, and the new comprehension passes that to fold_identifier_case(). This also happens on the base, so it isn't a regression from this PR. A non-unique expression index and a partial unique expression index now work because they're filtered out before the comprehension.
Would it be reasonable to skip indexes containing expressions when looking for an exact column constraint, and add a test? I checked the exact base and PR lookup() methods with in-memory SQLite and the existing sqlite-utils support modules; I didn't run the full branch suite.
Fixes #914.
Problem
Table.lookup()checked only whether an index covered the lookup columns. A non-unique index could therefore be mistaken for the required uniqueness constraint, and a partial unique index could be treated as equivalent to a full unique index. This could allow duplicate lookup rows.Behavior
lookup()from creating the required full unique index.lookup()reuses an existing full unique index on exactly the lookup columns, or creates one when needed.sqlite3.IntegrityErrorbefore any rows are changed.This change is limited to
Table.lookup()unique-index detection and creation. It makes no broad performance or indexing claims and is independent of #913; both are based on the same upstream commit.Validation
Validation was run by Codex:
Implementation and tests: MiniMax. Review and validation: Codex.
📚 Documentation preview 📚: https://sqlite-utils--915.org.readthedocs.build/en/915/