Skip to content

fix(google-ai): bill Gemini context-cache hits at the cached input rate - #1502

Open
pcbeingused333 wants to merge 1 commit into
SciSharp:masterfrom
pcbeingused333:fix/google-ai-cached-input-tokens
Open

pcbeingused333 wants to merge 1 commit into
SciSharp:masterfrom
pcbeingused333:fix/google-ai-cached-input-tokens

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

No linked issue (sibling of the Gemini thinking-token fix: found by checking which UsageMetadata fields the Google AI provider reads for cost stats).

Gemini's UsageMetadata.PromptTokenCount already includes the tokens served from the context cache (implicit or explicit), and CachedContentTokenCount reports that part. The provider only read PromptTokenCount, so every prompt token was reported as TextInputTokens and billed at TextInputCost; CachedTextInputCost in the model settings never applied to Gemini, which overstates the cost whenever implicit caching hits.

Fix: ProviderHelper.GetCachedInputTokens returns CachedContentTokenCount, and GetTextInputTokens returns PromptTokenCount minus the cached part (never negative). The chat, async chat, streaming and text completion providers now fill TextInputTokens and CachedTextInputTokens from them, the same split the OpenAI provider already uses.

Test: two new cases in GoogleAiUsageTests (cached tokens read from usage; text input excludes cached tokens). With the old behaviour they fail (Expected: 200, Actual: 1000 and Expected: 800, Actual: 0).

This change was prepared with the assistance of an AI coding tool (Claude Code); the author is responsible for the submission.

Gemini's PromptTokenCount includes the tokens served from the context
cache, and UsageMetadata.CachedContentTokenCount was never read. All
prompt tokens were therefore reported as TextInputTokens and billed at
the full input price, so CachedTextInputCost had no effect.

Report CachedContentTokenCount as CachedTextInputTokens and subtract it
from TextInputTokens in the chat, streaming and text providers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant