Skip to content

Fix distributed token bucket refill across gateways - #51

Merged
yashdotdev13 merged 1 commit into
mainfrom
fix/distributed-token-bucket-refill
Oct 15, 2026
Merged

yashdotdev13 merged 1 commit into
mainfrom
fix/distributed-token-bucket-refill

Conversation

@yashdotdev13

Copy link
Copy Markdown
Owner

Overview

Implements a Redis-backed distributed token bucket to share rate-limit capacity and refill state atomically across gateway instances.

Changes

  • Added atomic Redis Lua scripts for token consumption, initialization, and time-based refill.
  • Integrated distributed token-bucket enforcement into TrafficDecisionEngine.
  • Added tests for token refill, shared capacity across coordinator instances, and decision-engine behavior.
  • Updated the distributed load test to distribute requests across multiple gateways and track traffic decisions.

Testing

  • mvn test — 120 tests passed, 0 failures, 0 errors.
  • git diff --cached --check — passed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T15:08:25.880448Z f5f7878 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@sonarqubecloud

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5f7878237

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +34 to +35
this(policyProvider, quotaCoordinator, capacityAllocator,
null, null, null);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve token-bucket behavior for legacy constructors

Any caller using one of the pre-existing public constructors now gets a null bucket. When such an engine evaluates a TOKEN_BUCKET policy, evaluateDecision throws IllegalStateException instead of performing the previously supported quota evaluation. This affects existing consumers that instantiate the engine through the unchanged 3-, 4-, or 5-argument constructors; either supply a compatible bucket/fallback or make the new dependency mandatory rather than retaining broken overloads.

Useful? React with 👍 / 👎.

Comment on lines +505 to +506
Boolean tokenBucketDeleted = redisTemplate.delete(
buildTokenBucketRedisKey(capacityKey));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prevent removed buckets from being recreated

After removeCapacity deletes this hash, the next tryConsume recreates it with a full bucket because the Lua script initializes any missing state. Thus removing capacity does not actually prevent an active token-bucket policy from serving traffic: a request after removal restores and consumes the configured capacity. Preserve a removal marker or require registered capacity before lazy initialization so removal remains effective.

Useful? React with 👍 / 👎.

@yashdotdev13
yashdotdev13 merged commit dfb1e91 into main Oct 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant