refactor(index): remove connect, set_client, connection_args, redis_kwargs - #727
Draft
vishal-bala wants to merge 4 commits into
Conversation
SearchIndex.connect, SearchIndex.set_client and their async counterparts have warned since v0.4.0, and AsyncSearchIndex.connect emitted three DeprecationWarnings per call because it delegated to set_client. None had working ownership semantics: set_client attached a finalizer based on whatever ownership the index already had, so it closed a caller's client on an index built from redis_url and never took ownership on one built with a client. Pass connection parameters to the constructor instead, with owns_client when the index should close a client you supplied. AsyncSearchIndex._validate_client goes with them, since set_client was its only caller. It carried the sync-to-async client coercion, so handing an async index a sync client is now rejected rather than silently converted. Both constructors gained a guard for the wrong client flavour, which also restores the rejection set_client used to provide for the sync index. connection_args and redis_kwargs are removed too. They had a second, undecorated entry point through from_existing that warned about nothing, so the deprecated spelling would otherwise have outlived the removal. Both now raise a TypeError naming connection_kwargs, because **kwargs would otherwise swallow them in silence; the message names keywords only, never values, since connection kwargs carry passwords. With those gone every caller of _split_from_existing_kwargs passes the same tuple, so nested_connection_keys is inlined. One side effect worth recording: @deprecated_argument wraps the function it decorates in an untyped wrapper, so mypy had never checked a single call site of either constructor. Removing it surfaced four pre-existing type errors in the semantic cache, fixed here with the cast idiom the cache base module already uses.
Six review perspectives ran against the removal. Their substantive findings, in one commit because they overlap: The removed-keyword table was unreachable on one path that mattered. AsyncSearchIndex.from_existing checked for a client before splitting kwargs, so a caller passing redis_kwargs — the async alias, and so the likeliest caller — got a generic "must provide redis_url or redis_client" instead of being told the current spelling. Split first. The wrong-flavour guard was duplicated in both constructors and unreachable from from_existing, where validate_sync_redis fired first with a message that says what is wrong but not what to do. It now lives on BaseSearchIndex, driven by two class attributes, and runs ahead of the broader validation on both paths. Its message leads with what the index needs, names the class it got — qualified, since redis.Redis and redis.asyncio.Redis share a bare __name__ — and ends with the advice, which for the async index now includes sync_to_async_redis for callers who relied on the coercion that went away with _validate_client. The guard is deliberately one-sided: it rejects the wrong flavour but does not assert the object is a client at all, because a positive check would reject the unspecced mocks much of the suite injects. That is now recorded in the docstring rather than left for a reader to infer, and the isinstance choice is pinned by a test instead of only asserted in a comment. _owns_redis_client folds into the same table, replacing two inline blocks and leaving one mechanism for "that keyword is gone" rather than two. The table drops its claim that these keywords were "removed rather than renamed", which its own values contradicted, and no longer says "no longer supported" — the callers sharing it never all accepted every name, so the router was telling users a keyword it never took had been withdrawn. Both guard tests move to the unit suite: neither touches Redis, and the sync one requested the async fixture, which pytest-asyncio 0.24 stopped allowing. Also fixes a pre-existing duplicate in the async ownership tests, where the disconnect_sync case awaited disconnect() instead and so left disconnect_sync on an owned client untested.
set_client()'s docstring was the only place in the rendered API reference that explained using a client you configured yourself, and docs/api/searchindex.rst renders both classes with a bare autoclass, so deleting the method took the explanation with it. redis_client= now appears in no example anywhere. Both class docstrings gain one, along with when the index will and will not close such a client. Neither constructor documented the three TypeErrors it can now raise, so the rename of connection_args and redis_kwargs existed only inside an exception string and a private table. Both from_existing docstrings also omitted connection_kwargs, the keyword those exceptions name as the replacement, even though they accept it through **kwargs. _split_from_existing_kwargs gains a docstring: its name promises splitting, it now also rejects, and it consumes the dict it is handed. Replaces the four casts the previous commit added, plus four that predated it, with a TypedDict on BaseCache.redis_kwargs. The casts were all working around one missing annotation on a heterogeneous dict literal, and cast() asserted a type the checker could not verify, so nothing stopped a later edit from making the assertion false. A TypedDict is a plain dict at runtime, so this is annotation-only.
The async ownership tests had two copies of the same assertion: the disconnect_sync case awaited disconnect() instead of calling disconnect_sync(), so the method it was named for was never exercised. Calling it revealed why that mattered — inside a running loop it is a silent no-op, because sync_wrapper reaches loop.run_until_complete on an already-running loop and swallows the RuntimeError. That is the intended design: the method exists for callers outside a loop, such as __del__ and shutdown hooks, and the closing path is covered there by the finalizer unit tests. But an undocumented silent no-op reads as a bug to whoever finds it next, so the test now asserts it and the docstring says it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
SearchIndex.connect()andset_client()and theirAsyncSearchIndexcounterparts have warned since v0.4.0, and the project has shipped 23 further minor versions without removing them. #660 reported thatset_client()closes a client the caller still owns; two pull requests tried to repair the ownership logic inside it and both were closed, because patching a method nobody should be on is the wrong fix.None of these methods had working ownership semantics to preserve.
set_client()attached a finalizer based on whatever ownership the index already had, so it closed a caller's client on an index built fromredis_urland never took ownership on one built with a client. It also silently abandoned the client it replaced. The method over-claimed or under-claimed depending on how the index was constructed, so this removes a broken capability rather than a working one.This is the second of three stacked pull requests. The first added
owns_client, which is what a caller migrating offset_client()needs.Changes
The four methods, and the coercion that went with them
AsyncSearchIndex.connect()emitted threeDeprecationWarnings per call: its own decorator, a redundant inlinewarnings.warn(), andset_client()'s decorator via delegation. All three go together.AsyncSearchIndex._validate_client()goes too, sinceset_client()was its only caller. It carried the sync-to-async client coercion, so handing an async index a sync client is now rejected rather than silently converted. Both constructors gained a guard for the wrong client flavour, which also restores theTypeErrorthatset_client()used to raise for the sync index — previously the only place that rejection existed.The guard uses
isinstancerather thanissubclass(type(...)), deliberately:Mock(spec=AsyncRedis)sets__class__, soisinstancecatches a mis-flavoured spec'd mock while an unspecced mock still passes, which much of the test suite relies on. It is also deliberately one-sided — it rejects the wrong flavour but does not assert the object is a Redis client at all, because a positive check would reject those mocks.connection_argsandredis_kwargsBoth are removed, and both had a second entry point the issue discussion never mentioned: they appeared in the
nested_connection_keystuples passed to_split_from_existing_kwargs, sofrom_existing(name, connection_args={...})was accepted with no warning at all. Removing only the decorated path would have left the deprecated spelling alive.**kwargswould otherwise swallow both names in silence, which is worse than the warning it replaces, so a table of old spellings raises aTypeErrornaming the current one. The message contains parameter names only, never argument values: connection keywords carry passwords, and a message shaped as "unexpected keys: {...}" would put a live credential into an exception string and from there into logs.With those gone every caller of
_split_from_existing_kwargspassed the same tuple, sonested_connection_keysis inlined.Minor
castcalls in the semantic cache were replaced by aTypedDictonBaseCache.redis_kwargs, removing eight casts in that file. They were all working around one missing annotation on a heterogeneous dict literal, andcast()asserted a type the checker could not verify.set_client()'s docstring was the only place in the rendered API reference that explained using a client you configured yourself, anddocs/api/searchindex.rstrenders both classes with a bareautoclass, so deleting the method took the explanation with it.disconnect()was documented as "Disconnect from the Redis database" on two classes and not at all on the async one, and its log line claimed the index did not own a client it had in fact created.Notes
Removing
@deprecated_argumentfrom the two constructors surfaced four pre-existing type errors in the semantic cache. The decorator returns a bareCallableand wraps the function in an untyped wrapper, which erases the signature, so mypy had never checked a single call site ofSearchIndex(...)orAsyncSearchIndex(...)anywhere in the codebase. There are roughly 45 remaining applications of that decorator, each a hole inmake check-types; retyping it withParamSpecis tracked separately, and would surface all of them at once.RedisConnectionFactory.sync_to_async_redis()survives even though_validate_client()goes, becauseBaseCache._get_async_redis_clientis a second, non-deprecated caller. Its coverage survives too, through the LLM cache integration tests that build a cache on a sync client and then await an async method.from_existingnow checks client flavour before the broadervalidate_sync_redis, which otherwise fired first with a message that says what is wrong but not what to do. The two checks cannot be merged:validate_sync_redisis an allow-list that rejects any non-client including a bare mock, while the constructor guard is a deny-list that must let mocks through.disconnect_sync()on an async index is a no-op when a loop is already running, because it cannot await the close from inside one. That is intended — it exists for__del__and shutdown hooks — but it was undocumented and one of the ownership tests had been assertingawait disconnect()under a name promisingdisconnect_sync(), so the method it was named for was never exercised. Both are fixed.Release Notes
SearchIndex.connect(),SearchIndex.set_client()and theirAsyncSearchIndexcounterparts are removed. Passredis_clientorredis_urlto the constructor instead, and addowns_client=Trueif the index should close a client you supplied.The
connection_argsandredis_kwargsconstructor keywords are removed in favour ofconnection_kwargs; the value is unchanged. Both now raiseTypeErrornaming the replacement rather than being silently ignored.AsyncSearchIndexno longer converts a sync Redis client for you. Pass an async client, or convert a non-cluster client yourself withRedisConnectionFactory.sync_to_async_redis(). Passing the wrong flavour of client to either index class now raisesTypeErrorat construction rather than failing at first use, and handing an async client toAsyncSearchIndex's counterpart raisesTypeErrorwhere it previously raisedValueError.