Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ CACHEKIT_MASTER_KEY=<64-hex-char-key; use exactly 32 bytes>
# same per-key requirements as CACHEKIT_MASTER_KEY). Entries written under a
# listed key stay readable through the rotation window; writes always use
# CACHEKIT_MASTER_KEY. More than 3 keys, or the current master key re-appearing
# in this list, is rejected at load (a master_key= argument: when its cache is built).
# in this list, is rejected at load (a master_key= on an encrypting cache: when it is built).
CACHEKIT_PREVIOUS_MASTER_KEYS=<old-key-hex>,<older-key-hex>
# Fail closed on decrypt authentication failures (default: false = fail open/recompute).
# When true, AES-GCM auth failures and key-fingerprint mismatches raise
Expand Down
19 changes: 14 additions & 5 deletions docs/features/zero-knowledge-encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ export CACHEKIT_MASTER_KEY=<new-key-hex>
export CACHEKIT_PREVIOUS_MASTER_KEYS=<old-key-hex>
```

Rules enforced at config load — rejected, never truncated or silently fixed:
Rules enforced at config load or cache build — rejected, never truncated or silently fixed:

- **Cap**: at most 3 decrypt-only keys.
- **Per-key validation**: identical to `CACHEKIT_MASTER_KEY` (hex-encoded, at least 32 bytes; use exactly 32).
Expand Down Expand Up @@ -680,10 +680,13 @@ config = EncryptionConfig(enabled=True, master_key=secret_key,
raises `KeyringConfigurationError` (a `ValueError` subclass, exported from
`cachekit.serializers`) when the decrypt-only keyring is unusable: a previous master key
passed directly that is not exactly 32 bytes, more than three previous keys, or the current
key repeated among them. Settings check all three at load, and an encrypting cache checks its
current key against the previous keys when it is built, so behind the decorators this surfaces
only when settings are assigned after that, or on a config-drift read (below). Outside config-drift reads, the fault never
evicts and is not counted on `cachekit_decrypt_failures_total`. Direct `EncryptionWrapper` users
key repeated among them. Settings check all three at load and refuse an assignment that breaks
them, leaving the settings as they were (`previous_master_keys` is a tuple, so a change has to
be an assignment), and an encrypting cache checks its current key against the previous keys when
it is built. Behind the decorators this surfaces on a config-drift read (below), or when a
previous key assigned to the settings repeats the `master_key=` of a cache already built: the
settings never see that key, so its next wrapper build refuses it. Outside config-drift reads, the
fault never evicts and is not counted on `cachekit_decrypt_failures_total`. Direct `EncryptionWrapper` users
and callers of the `CacheOperationHandler` read and write methods receive it in both fail modes. Behind
the `@cache` decorators, a read of an existing encrypted entry raises it too, from L1 or L2 and
from the re-read after a distributed-lock wait, so the function does not run and no
Expand All @@ -695,6 +698,12 @@ or one of the wrong length, raises `EncryptionError`, and an encryption-disabled
claims encryption treats the fault as corruption (miss + evict), because only the
unauthenticated header sent it down the decrypt path.

A wrapper keeps the keyring it built. A cache builds one per tenant, on that tenant's first use and
after LRU eviction, so a key change in the settings reaches only wrappers built after it: their
previous keys, and their current key too unless the cache was given `master_key=`. One multi-tenant
cache can therefore run the old keyring for some tenants and the new one for others. To apply a key
change everywhere at once, change the environment and restart the process.

> **⚠️ Key rotation under fail-closed:** with `fail_closed` enabled there is no
> silent self-heal — rotating `CACHEKIT_MASTER_KEY` **without retaining the old key
> in `CACHEKIT_PREVIOUS_MASTER_KEYS`** makes every pre-rotation entry raise
Expand Down
14 changes: 7 additions & 7 deletions src/cachekit/cache_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -698,8 +698,9 @@ def __init__(
None: no prefix, for a handler with no backend behind it.

Raises:
ConfigurationError: If encryption config is invalid (missing mode or both modes), or a
master key is present while encryption is None.
ConfigurationError: If encryption config is invalid (missing mode or both modes), a
master key is present while encryption is None, or an encrypting handler's current key
(master_key= or CACHEKIT_MASTER_KEY) is also among the previous master keys.
TypeError: If serializer_name is not a string or SerializerProtocol instance, or master_key is
bytes (pass ``key.hex()``).

Expand Down Expand Up @@ -774,11 +775,10 @@ def __init__(
encryption = False

# The keyring's forward-only rule, refused here at decoration before any backend call, whichever route the
# current key took. Settings run the same check at load, but never see master_key= and take later assignments
# unvalidated; without this, the native keyring refuses only when the first read or write builds it. An
# encryption-disabled handler never encrypts, so it spends no nonce budget: its keyring is built only for a
# config-drift read, whose fault is a miss by design. Nor does it load settings here: a plaintext handler must
# not fail on a malformed keyring setting it never uses.
# current key took. Settings run the same check at load and on assignment, but never see master_key=; without
# this, the native keyring refuses only when the first read or write builds it. An encryption-disabled handler
# never encrypts, so it spends no nonce budget: its keyring is built only for a config-drift read, whose fault
# is a miss by design. This check loads no settings for a plaintext handler.
if encryption:
settings = get_settings()
try:
Expand Down
61 changes: 57 additions & 4 deletions src/cachekit/config/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,24 +17,34 @@

from __future__ import annotations

import functools
import threading
from typing import Annotated, Any, Literal, Optional

from pydantic import (
BaseModel,
Field,
SecretStr,
field_validator,
model_validator,
)
from pydantic_settings import NoDecode, SettingsConfigDict
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict

from .validation import RedactingSettings, refuse_current_key_in_previous_keys
from .validation import RedactingSettings, _redacting, refuse_current_key_in_previous_keys

# Keyring cap from the protocol spec (spec/encryption.md → "Key Rotation (Keyring)"):
# at most 3 decrypt-only previous keys. Exceeding the cap is a configuration error,
# rejected at load — never silently truncated. Mirrors cachekit-core's
# MAX_DECRYPT_ONLY_KEYS, which re-validates behind the FFI boundary.
MAX_PREVIOUS_MASTER_KEYS = 3

# Serializes CachekitConfig assignments: each validates the whole state it would leave, so two
# assignments that are each valid alone (a new master_key, and that key added to previous_master_keys)
# cannot land together unchecked. An assignment commits by swapping in its copy's whole state, so every
# other change (a private attribute, a delete) takes the lock too, or the swap would undo it. Reentrant:
# a subclass's validator or property setter may assign a field or a private attribute while it is held.
_ASSIGNMENT_LOCK = threading.RLock()


class CachekitConfig(RedactingSettings):
"""Backend-agnostic cache configuration.
Expand Down Expand Up @@ -122,6 +132,8 @@ class CachekitConfig(RedactingSettings):
# logs. errors()/json() ignore this flag; RedactingSettings sanitizes
# those surfaces.
hide_input_in_errors=True,
# Assignment after load runs every field and model validator; __setattr__ makes a refusal write nothing.
validate_assignment=True,
)

# Generic cache configuration (backend-agnostic)
Expand Down Expand Up @@ -205,8 +217,9 @@ class CachekitConfig(RedactingSettings):
default=None,
description="Master encryption key (hex-encoded; use exactly 32 bytes, 64 hex characters)",
)
previous_master_keys: Annotated[list[SecretStr], NoDecode] = Field(
default_factory=list,
# A tuple, so the keyring cannot be edited in place, where no validator runs: a change is an assignment.
previous_master_keys: Annotated[tuple[SecretStr, ...], NoDecode] = Field(
default_factory=tuple,
description=(
"Decrypt-only previous master keys for key rotation (env: "
"CACHEKIT_PREVIOUS_MASTER_KEYS, comma-separated hex). Entries written "
Expand Down Expand Up @@ -277,6 +290,46 @@ def validate_previous_master_keys(self) -> CachekitConfig:

return self

def __setattr__(self, name: str, value: Any) -> None:
"""Validate an assignment on a copy first, and write it only if the copy passes.

validate_assignment alone writes the new value before the model validator runs, so a refused keyring
assignment would stay on the instance, readable by other threads until the error surfaces. The value
is dropped in a finally: the raised error's traceback holds this frame (CWE-532).
"""
# A private attribute: nothing to validate, so no copy. It still takes the lock, so it cannot land between
# another thread's model_copy() and swap and be lost. Any other name goes through the guard, so a mistyped
# field's value is refused inside _redacting too.
if name in type(self).__private_attributes__:
with _ASSIGNMENT_LOCK:
super().__setattr__(name, value)
return
candidate = None
try:
with _ASSIGNMENT_LOCK:
# A subclass property: run its setter once, on this instance. Each field it assigns comes back
# through this guard; replaying it on a copy would apply a non-idempotent setter twice. Redacted like
# the copy path, so a refused key leaves no pydantic frame holding it.
if isinstance(getattr(type(self), name, None), property):
_redacting(functools.partial(BaseSettings.__setattr__, self, name, value), type(self).__name__)
return
Comment thread
kodus-27b[bot] marked this conversation as resolved.
candidate = self.model_copy()
# BaseSettings.__setattr__, not this override: pydantic's validated assignment, on the copy.
_redacting(functools.partial(BaseSettings.__setattr__, candidate, name, value), type(self).__name__)
# Commit the state the copy validated, with no second validation: `value` may be a spent generator,
# and a non-idempotent validator would change it again. Every slot pydantic keeps instance state in, as
# model_copy() fills them: __dict__, the fields set, an extra="allow" subclass's undeclared names, and
# the private state a subclass's model validator may have derived on the copy.
for slot in BaseModel.__slots__:
object.__setattr__(self, slot, getattr(candidate, slot))
finally:
del value, candidate # the copy holds the refused value

def __delattr__(self, name: str) -> None:
"""Delete under the assignment lock, so an assignment's swap cannot undo the delete."""
with _ASSIGNMENT_LOCK:
super().__delattr__(name)

def __repr__(self) -> str:
"""Return string representation with sensitive information masked.

Expand Down
13 changes: 7 additions & 6 deletions src/cachekit/serializers/encryption_wrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,10 +311,11 @@ def _setup_encryption(self, master_key: SecretBytes | None, previous_master_keys
# Decrypt-only previous keys from settings if not provided (key rotation,
# spec/encryption.md → "Key Rotation (Keyring)"). Settings enforce the cap
# of 3, per-key hex/length validation, and the forward-only subset check
# at load, and CacheSerializationHandler repeats the subset check for an
# encrypting cache's master_key= when it is built; the Rust Keyring
# re-validates all three behind the FFI boundary for wrappers constructed
# with explicit parameters and for settings assigned after those checks.
# at load and on every assignment, and CacheSerializationHandler repeats the
# subset check for an encrypting cache's master_key= when it is built; the
# Rust Keyring re-validates all three behind the FFI boundary. This wrapper
# keeps the keyring it builds: a later change to the settings reaches only
# wrappers built after it.
# Keyring config errors below raise KeyringConfigurationError, NEVER
# EncryptionError: EncryptionError is a SerializationError, which the
# read-path policy (handle_decrypt_failure) classifies as corruption →
Expand All @@ -331,8 +332,8 @@ def _setup_encryption(self, master_key: SecretBytes | None, previous_master_keys
settings = get_settings()
previous_master_keys = [SecretBytes(bytes.fromhex(key.get_secret_value())) for key in settings.previous_master_keys]

# The hex rule again, for keys read from settings: settings validate them at load, but the settings
# object takes later assignments unvalidated, and the Rust Keyring below checks only 16 bytes.
# The hex rule again, for keys read from settings: settings validate them at load and on assignment, and
# this stays as defence in depth because the Rust Keyring below checks only 16 bytes.
for position, previous_key in enumerate(previous_master_keys):
if len(previous_key) < 32:
raise KeyringConfigurationError(
Expand Down
64 changes: 64 additions & 0 deletions tests/unit/config/test_redacting_settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,16 @@
from cachekit.serializers.encryption_wrapper import EncryptionError, EncryptionWrapper

_KEY_HEX = "ab" * 32
# No two 16-character windows of it are alike, so a truncated rendering of it is still found.
_DISTINCT_KEY_HEX = bytes(range(32)).hex()

# Post-load assignments to the settings that are refused: by a field, by the model validator, and to a name
# that is no field.
_SETTINGS_ASSIGNMENT_REFUSALS: dict[str, tuple[str, object]] = {
"field-level": ("previous_master_keys", {_DISTINCT_KEY_HEX: 1}),
"model-level": ("previous_master_keys", [_DISTINCT_KEY_HEX, "01" * 32, "02" * 32, "03" * 32]),
"mistyped-field": ("previous_master_key", _DISTINCT_KEY_HEX),
}

BACKEND_CONFIGS: list[type[BaseBackendConfig]] = [
RedisBackendConfig,
Expand Down Expand Up @@ -633,6 +643,51 @@ def _cachekit_locals_holding(exc: BaseException, secret: str | bytes, *, below_c
return found


@pytest.mark.unit
class TestSettingsAssignmentRedaction:
"""A refused assignment to the loaded settings carries no route to the key it was given, whole or truncated."""

@pytest.mark.parametrize(
("field", "value"), _SETTINGS_ASSIGNMENT_REFUSALS.values(), ids=_SETTINGS_ASSIGNMENT_REFUSALS.keys()
)
def test_refused_assignment_redacts_the_key(self, field: str, value: object) -> None:
with pytest.raises(ValidationError) as exc_info:
setattr(singleton.get_settings(), field, value)

_assert_no_route_to(exc_info.value, _DISTINCT_KEY_HEX)
rendered = str(exc_info.value) + repr(exc_info.value.errors()) + exc_info.value.json()
fragments = {_DISTINCT_KEY_HEX[i : i + 16] for i in range(len(_DISTINCT_KEY_HEX) - 15)}
assert [fragment for fragment in fragments if fragment in rendered] == []

@pytest.mark.parametrize(
("field", "value"), _SETTINGS_ASSIGNMENT_REFUSALS.values(), ids=_SETTINGS_ASSIGNMENT_REFUSALS.keys()
)
def test_refused_assignment_leaves_no_frame_local(self, field: str, value: object) -> None:
"""Pydantic's own assignment frames hold the raw value too, so the guard must re-raise from above them: the
cachekit-only walk of TestEntryPointFrameLocals would pass without that."""
with pytest.raises(ValidationError) as exc_info:
setattr(singleton.get_settings(), field, value)

assert _cachekit_locals_holding(exc_info.value, _DISTINCT_KEY_HEX, below_caller=True) == []

def test_refused_assignment_through_a_property_leaves_no_frame_local(self) -> None:
"""A subclass property setter that assigns a key runs outside the copy, so it needs the same boundary."""

class _Rotating(CachekitConfig):
def _rotate(self, key: str) -> None:
self.master_key = key # type: ignore[assignment]

rotate = property(fset=_rotate)

config = _Rotating(previous_master_keys=(_DISTINCT_KEY_HEX,)) # type: ignore[arg-type]
with pytest.raises(ValidationError) as exc_info:
config.rotate = _DISTINCT_KEY_HEX

assert config.master_key is None
_assert_no_route_to(exc_info.value, _DISTINCT_KEY_HEX)
assert _cachekit_locals_holding(exc_info.value, _DISTINCT_KEY_HEX, below_caller=True) == []


@pytest.mark.unit
class TestRedactingSettingsFrameLocals:
"""No cachekit frame on a raised config error's traceback keeps the raw input (CWE-532)."""
Expand Down Expand Up @@ -1073,6 +1128,15 @@ def _api_key_rows(form: Callable[[str], Any]) -> dict[str, _EntryPointRow]:
ConfigurationError,
_SHORT_KEY_HEX,
),
**{
f"settings-assignment-{name}": (
{},
lambda field=field, value=value: setattr(singleton.get_settings(), field, value),
ValidationError,
_DISTINCT_KEY_HEX,
)
for name, (field, value) in _SETTINGS_ASSIGNMENT_REFUSALS.items()
},
}


Expand Down
5 changes: 3 additions & 2 deletions tests/unit/protocol/test_encryption_master_key_vectors.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@
# One refusal each: a non-hex string, or one that decodes short. Anything else (a missing key) fails the match.
DECORATOR_REFUSAL = r"CACHEKIT_MASTER_KEY must be (hex-encoded|at least 32 bytes)"
WRAPPER_REFUSAL = r"Invalid master key format|Master key must be at least 32 bytes"
# The refusal each keyring reject row gets at decoration, from the settings validator, on both current-key routes.
# The refusal each keyring reject row gets at decoration from the settings validator; _keyring_refusal names the one
# exception, a repeat row on master_key_argument.
_SETTINGS_REFUSAL = r"(?s)^1 validation error for CachekitConfig\n.*Value error, "
_REPEAT_REFUSAL = "master_key must not appear in previous_master_keys"
KEYRING_REFUSALS = {
Expand All @@ -88,7 +89,7 @@


def _keyring_refusal(name: str, route: str) -> tuple[type[Exception], str]:
"""The settings never see a master_key=, so the decorator's handler refuses its repeat, as it does a bad key."""
"""The settings never see a master_key=, so the decorator's handler refuses its repeat when it is built."""
if name in REPEAT_ROWS and route == "master_key_argument":
return ConfigurationError, f"^{_REPEAT_REFUSAL}: "
return ValidationError, KEYRING_REFUSALS[name]
Expand Down
Loading
Loading