Skip to content

Add effective_cache_key(model, context) helper - #265

Merged
ptomecek merged 1 commit into
mainfrom
pit/public-effective-identity
Aug 25, 2026
Merged

Add effective_cache_key(model, context) helper#265
ptomecek merged 1 commit into
mainfrom
pit/public-effective-identity

Conversation

@ptomecek

Copy link
Copy Markdown
Collaborator

Summary

Adds a public helper effective_cache_key(model, context) to ccflow.evaluators.common.

cache_key(..., effective=True) already derives effective (opt-in) identity, but only for a ModelEvaluationContext. Callers that hold a bare model plus a context — for example keyed persistent caches — otherwise have to build a throwaway ModelEvaluationContext and then compare the structural and effective keys to detect opt-out models. This helper wraps the existing internal derivation directly:

def effective_cache_key(model: CallableModel, context: ContextBase) -> bytes:
    try:
        key = _effective_model_key(model, context, {}, set())
    except _EffectiveEvaluationKeyUnavailable:
        key = None
    return key if key is not None else cache_key(model)
  • Models that declare an effective identity (non-None identity payload) are keyed by it.
  • Models that do not opt in fall back byte-for-byte to cache_key(model).
  • Unlike cache_key(mec, effective=True), the result is the model-level effective key without the surrounding evaluation-context envelope (fn/options), and opt-out models get the context-independent structural key rather than a context-dependent one.

No behavior change to existing APIs — this is purely additive (new public function + __all__ entry).

Tests

TestEffectiveCacheKey covers: opt-out equals cache_key(model) and is context-independent; opt-in is keyed by the identity payload (and collapses contexts the payload treats as equivalent); payload-relevant differences produce distinct keys; and errors raised inside an identity payload propagate rather than being masked by the structural fallback.

ruff check clean; ccflow/tests/evaluators/test_common.py (45) and the effective-key characterization suite pass.

Expose a public helper that returns the effective identity key for evaluating a
model on a context, given the model and context directly rather than a
ModelEvaluationContext.

cache_key(..., effective=True) already derives effective identity, but only for a
ModelEvaluationContext. Callers that hold a bare model plus a context (for example
keyed persistent caches) otherwise have to reconstruct a ModelEvaluationContext and
compare structural versus effective keys to detect opt-out models. This helper wraps
the existing internal derivation instead: models that declare an effective identity
are keyed by it, and models that do not fall back byte-for-byte to cache_key(model).

Signed-off-by: Pascal Tomecek <40371786+ptomecek@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Test Results

    1 files  ±0      1 suites  ±0   2m 32s ⏱️ -29s
1 352 tests +5  1 350 ✅ +5  2 💤 ±0  0 ❌ ±0 
1 358 runs  +5  1 356 ✅ +5  2 💤 ±0  0 ❌ ±0 

Results for commit 6b652de. ± Comparison against base commit a143162.

♻️ This comment has been updated with latest results.

@codecov

codecov Bot commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.05882% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.53%. Comparing base (a143162) to head (6b652de).

Files with missing lines Patch % Lines
ccflow/evaluators/common.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #265   +/-   ##
=======================================
  Coverage   93.53%   93.53%           
=======================================
  Files         176      176           
  Lines       20460    20494   +34     
  Branches     1352     1352           
=======================================
+ Hits        19137    19170   +33     
- Misses       1052     1053    +1     
  Partials      271      271           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ptomecek
ptomecek marked this pull request as ready for review August 25, 2026 16:39
@ptomecek
ptomecek merged commit bb008b5 into main Aug 25, 2026
20 checks passed
@ptomecek
ptomecek deleted the pit/public-effective-identity branch August 25, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants