Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 87 additions & 17 deletions .generator/schemas/v2/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11380,7 +11380,6 @@ components:
description: Maximum priced spend per rolling minute, in microcents of 1e-8 USD each, shared by all matching traffic, or per counter for a template rate limit. Null when this rule does not limit cost.
example: 100000000
format: int64
maximum: 10000000000000
minimum: 1
nullable: true
type: integer
Expand Down Expand Up @@ -11483,6 +11482,12 @@ components:
format: uuid
nullable: true
type: string
window_limits:
description: Limits over rolling windows longer than a minute. Each limit counts the rule's matching traffic, or each counter of a template rate limit, over its window. Per-minute limits are in the `*_per_minute` fields. This list is empty when the rule has no window limits.
items:
$ref: "#/components/schemas/AiGatewayRateLimitWindowLimit"
maxItems: 4
type: array
required:
- scope_restricted
- name
Expand All @@ -11501,10 +11506,9 @@ components:
description: Attributes used to create an AI Gateway rate limit.
properties:
cost_microcents_per_minute:
description: Maximum priced spend per rolling minute, in microcents of 1e-8 USD each. At most 10000000000000, that is $100,000. In `ENFORCE` mode, requests to a model with no pricing are rejected, because their cost can't be counted.
description: Maximum priced spend per rolling minute, in microcents of 1e-8 USD each. In `ENFORCE` mode, the gateway rejects requests to models with no pricing because their cost can't be counted.
example: 100000000
format: int64
maximum: 10000000000000
minimum: 1
nullable: true
type: integer
Expand All @@ -11529,7 +11533,7 @@ components:
nullable: true
type: integer
requests_per_minute:
description: Maximum requests per rolling minute. At least one of `requests_per_minute`, `input_tokens_per_minute`, `output_tokens_per_minute`, and `cost_microcents_per_minute` is required.
description: Maximum requests per rolling minute. At least one of `requests_per_minute`, `input_tokens_per_minute`, `output_tokens_per_minute`, `cost_microcents_per_minute`, or a non-empty `window_limits` is required.
example: 600
format: int64
maximum: 10000000
Expand All @@ -11553,6 +11557,12 @@ components:
example: 00000000-0000-4000-8000-000000000001
format: uuid
type: string
window_limits:
description: Limits over rolling windows longer than a minute, such as $20 of spend per 24 hours. This field is available only to organizations with the feature enabled. Other organizations receive a 400 Bad Request response for a non-empty list. Each limit counts the rule's matching traffic, or each counter of a template rate limit, over its window. Set per-minute limits in the `*_per_minute` fields. A rule can have at most four limits in total, including any per-minute limits that are set. Each metric and window pair can appear only once, and for the same metric, a longer window must have a larger amount. In `ENFORCE` mode, the gateway rejects a request that exceeds any limit with HTTP 429 and a `Retry-After` header. If the list includes a `cost` limit, the gateway also rejects requests to models with no pricing because their cost can't be counted.
items:
$ref: "#/components/schemas/AiGatewayRateLimitWindowLimit"
maxItems: 4
type: array
required:
- name
type: object
Expand All @@ -11579,12 +11589,12 @@ components:
description: One counter of a template rate limit and its live usage.
properties:
cost_microcents:
description: Priced spend, in microcents, counted in the rolling one-minute window. 0 when the rate limit doesn't limit cost.
description: Priced spend, in microcents, counted in the rolling one-minute window. The value is 0 when the rate limit doesn't limit cost.
example: 25000000
format: int64
type: integer
cost_microcents_per_minute:
description: The rate limit's per-counter spend limit, in microcents. Null when it doesn't limit cost.
description: The rate limit's per-counter spend limit, in microcents. Null when the rate limit doesn't limit cost.
example: 100000000
format: int64
nullable: true
Expand All @@ -11610,7 +11620,7 @@ components:
example: "v1|7:user_id=36:11111111-1111-1111-1111-111111111111"
type: string
last_seen:
description: When the gateway last saw this counter hot, that is near or at a limit or limiting; up to a few minutes behind live traffic.
description: When the gateway last recorded this counter as hot, meaning it was at 80% or more of a limit, or it limited or would have limited a request. This value can lag live traffic by a few minutes.
example: "2026-10-28T10:05:00Z"
format: date-time
type: string
Expand Down Expand Up @@ -11733,7 +11743,7 @@ components:
description: Aggregate of a template rate limit's counters.
properties:
instance_count:
description: Number of counters that were near or at a limit, or limited a request, in the last hour. Counters with room to spare aren't recorded.
description: Number of counters that were at 80% or more of a limit, or that limited or would have limited a request, in the last hour. Other counters aren't recorded.
example: 14
format: int64
type: integer
Expand Down Expand Up @@ -11915,17 +11925,16 @@ components:
description: Attributes used to update an AI Gateway rate limit. Only sent fields are changed.
properties:
cost_microcents_per_minute:
description: Spend limit in microcents per rolling minute. Set to 0 to remove the cost limit. Null is ignored. At least one limit must remain.
description: Spend limit in microcents per rolling minute. Set to 0 to remove the cost limit. Null is ignored. At least one limit, per-minute or in `window_limits`, must remain.
format: int64
maximum: 10000000000000
minimum: 0
nullable: true
type: integer
enabled:
description: Whether this rate limit is active.
type: boolean
input_tokens_per_minute:
description: Set to 0 to remove the request limit. Null is ignored. At least one limit must remain.
description: Set to 0 to remove the input token limit. Null is ignored. At least one limit, per-minute or in `window_limits`, must remain.
format: int64
maximum: 100000000000
minimum: 0
Expand All @@ -11937,14 +11946,14 @@ components:
description: Rate limit name.
type: string
output_tokens_per_minute:
description: Set to 0 to remove the request limit. Null is ignored. At least one limit must remain.
description: Set to 0 to remove the output token limit. Null is ignored. At least one limit, per-minute or in `window_limits`, must remain.
format: int64
maximum: 100000000000
minimum: 0
nullable: true
type: integer
requests_per_minute:
description: Set to 0 to remove the request limit. Null is ignored. At least one limit must remain.
description: Set to 0 to remove the request limit. Null is ignored. At least one limit, per-minute or in `window_limits`, must remain.
format: int64
maximum: 10000000
minimum: 0
Expand All @@ -11964,6 +11973,13 @@ components:
description: Replaces the user scope. An empty string clears it. Null is ignored.
nullable: true
type: string
window_limits:
description: Replaces all limits over rolling windows longer than a minute. If `window_limits` is null or omitted, the existing limits are unchanged. An empty array removes them all. This field is available only to organizations with the feature enabled. Other organizations receive a 400 Bad Request response for a non-empty list. The same rules as on create apply. A rule can have at most four limits in total, including any per-minute limits that are set. Each metric and window pair can appear only once, and for the same metric, a longer window must have a larger amount. In `ENFORCE` mode, if the list includes a `cost` limit, the gateway rejects requests to models with no pricing because their cost can't be counted.
items:
$ref: "#/components/schemas/AiGatewayRateLimitWindowLimit"
maxItems: 4
nullable: true
type: array
type: object
AiGatewayRateLimitUpdateData:
description: Data used to update an AI Gateway rate limit.
Expand Down Expand Up @@ -11994,7 +12010,7 @@ components:
description: Live consumption of one AI Gateway rate limit.
properties:
cost_microcents:
description: Priced spend, in microcents, counted in the rolling one-minute window. Only rules with a cost limit count it, so it is 0 for other rules and for the untracked and total entries.
description: Priced spend, in microcents, counted in the rolling one-minute window. Only rules with a cost limit count spend, so the value is 0 for other rules and for the untracked and total entries.
example: 25000000
format: int64
type: integer
Expand Down Expand Up @@ -12065,6 +12081,60 @@ components:
type: string
x-enum-varnames:
- RATE_LIMIT_USAGE
AiGatewayRateLimitWindowLimit:
description: A limit on one metric over a rolling window longer than a minute.
properties:
amount:
description: Maximum amount of the metric over the window. For `cost`, the amount is priced spend in microcents of 1e-8 USD each, as in `cost_microcents_per_minute`. For the same metric, a longer window must have a larger amount.
example: 2000000000
format: int64
minimum: 1
type: integer
metric:
$ref: "#/components/schemas/AiGatewayRateLimitWindowLimitMetric"
window:
$ref: "#/components/schemas/AiGatewayRateLimitWindowLimitWindow"
required:
- metric
- window
- amount
type: object
AiGatewayRateLimitWindowLimitMetric:
description: The metric that a window limit counts. `cost` counts priced spend in microcents.
enum:
- requests
- input_tokens
- output_tokens
- cost
example: cost
type: string
x-enum-varnames:
- REQUESTS
- INPUT_TOKENS
- OUTPUT_TOKENS
- COST
AiGatewayRateLimitWindowLimitWindow:
description: The rolling window that a limit applies to. For per-minute limits, use the `*_per_minute` fields instead.
enum:
- 5m
- 15m
- 30m
- 1h
- 3h
- 6h
- 12h
- 24h
example: 24h
type: string
x-enum-varnames:
- FIVE_MINUTES
- FIFTEEN_MINUTES
- THIRTY_MINUTES
- ONE_HOUR
- THREE_HOURS
- SIX_HOURS
- TWELVE_HOURS
- TWENTY_FOUR_HOURS
AiGatewayRestrictedDetailsMeta:
description: Visibility information for identifying instance details.
properties:
Expand Down Expand Up @@ -231610,7 +231680,7 @@ paths:
If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/).
/api/v2/rate-limits/instances/summary:
get:
description: Aggregate the counters of every template rate limit in the organization in one call, with one summary per template in the order the rate limit list returns them. Each summary is the one the single-template counter summary endpoint returns without split_dim, including scope_restricted and truncated. Rate limits with one shared allowance are left out. Summaries are cached for up to 15 seconds.
description: Aggregate the counters of every template rate limit in the organization in one call. The response has one summary per template, in the same order as the rate limit list. Each summary matches the response of the single-template counter summary endpoint without split_dim, including scope_restricted and truncated. Rate limits with one shared allowance aren't included. Summaries are cached for up to 15 seconds.
operationId: ListAiGatewayRateLimitInstancesSummaries
responses:
"200":
Expand Down Expand Up @@ -231867,7 +231937,7 @@ paths:
If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/).
/api/v2/rate-limits/{id}/instances:
get:
description: List the hot counters of a template rate limit, one per matched user, Team, or value, with their live usage. Only counters at 80% or more of a limit, or that limited or would have limited a request, in the last hour are listed. A counter appears within a few minutes of becoming hot. Usage is read for at most 200 counters per request; past that, rows carry usage_known false. Counters for a provider or model the caller can't see are left out, and when the caller can't see the template's selector, the list is empty. A rate limit with one shared allowance, including one whose selector has no per_value entry, returns an empty list. Returns 503 when counter usage can't be read.
description: List the hot counters of a template rate limit, one per matched user, Team, or value, with their live usage. The list includes only counters that, in the last hour, reached 80% or more of a limit or limited or would have limited a request. A counter appears within a few minutes of becoming hot. Usage is read for at most 200 counters per request; past that, rows carry usage_known false. Counters for a provider or model the caller can't see are left out, and when the caller can't see the template's selector, the list is empty. A rate limit with one shared allowance, including one whose selector has no per_value entry, returns an empty list. Returns 503 when counter usage can't be read.
operationId: ListAiGatewayRateLimitInstances
parameters:
- $ref: "#/components/parameters/AiGatewayRateLimitID"
Expand Down Expand Up @@ -231928,7 +231998,7 @@ paths:
If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/).
/api/v2/rate-limits/{id}/instances/summary:
get:
description: Aggregate a template rate limit's hot counters (those near or at a limit, or limiting, in the last hour), including how many there are, how many are near or at their limit, and the fullest counter's utilization. Use split_dim to aggregate only counters that match every given dimension value. When the caller can't see the template's selector, every count is zero and scope_restricted is true. Summaries are cached for up to 15 seconds. Returns 503 when counter usage can't be read.
description: Aggregate a template rate limit's hot counters, including how many there are, how many are near or at their limit, and the fullest counter's utilization. Hot counters are those that reached 80% or more of a limit, or that limited or would have limited a request, in the last hour. Use split_dim to aggregate only counters that match every given dimension value. When the caller can't see the template's selector, every count is zero and scope_restricted is true. Summaries are cached for up to 15 seconds. Returns 503 when counter usage can't be read.
operationId: GetAiGatewayRateLimitInstancesSummary
parameters:
- $ref: "#/components/parameters/AiGatewayRateLimitID"
Expand Down
10 changes: 10 additions & 0 deletions examples/v2/ai-gateway/CreateAiGatewayRateLimit.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
from datadog_api_client.v2.model.ai_gateway_rate_limit_mode import AiGatewayRateLimitMode
from datadog_api_client.v2.model.ai_gateway_rate_limit_selector_entry import AiGatewayRateLimitSelectorEntry
from datadog_api_client.v2.model.ai_gateway_rate_limit_type import AiGatewayRateLimitType
from datadog_api_client.v2.model.ai_gateway_rate_limit_window_limit import AiGatewayRateLimitWindowLimit
from datadog_api_client.v2.model.ai_gateway_rate_limit_window_limit_metric import AiGatewayRateLimitWindowLimitMetric
from datadog_api_client.v2.model.ai_gateway_rate_limit_window_limit_window import AiGatewayRateLimitWindowLimitWindow
from uuid import UUID

body = AiGatewayRateLimitCreateRequest(
Expand All @@ -33,6 +36,13 @@
],
team_id=UUID("00000000-0000-4000-8000-000000000002"),
user_id=UUID("00000000-0000-4000-8000-000000000001"),
window_limits=[
AiGatewayRateLimitWindowLimit(
amount=2000000000,
metric=AiGatewayRateLimitWindowLimitMetric.COST,
window=AiGatewayRateLimitWindowLimitWindow.TWENTY_FOUR_HOURS,
),
],
),
type=AiGatewayRateLimitType.RATE_LIMITS,
),
Expand Down
10 changes: 10 additions & 0 deletions examples/v2/ai-gateway/UpdateAiGatewayRateLimit.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
from datadog_api_client.v2.model.ai_gateway_rate_limit_update_attributes import AiGatewayRateLimitUpdateAttributes
from datadog_api_client.v2.model.ai_gateway_rate_limit_update_data import AiGatewayRateLimitUpdateData
from datadog_api_client.v2.model.ai_gateway_rate_limit_update_request import AiGatewayRateLimitUpdateRequest
from datadog_api_client.v2.model.ai_gateway_rate_limit_window_limit import AiGatewayRateLimitWindowLimit
from datadog_api_client.v2.model.ai_gateway_rate_limit_window_limit_metric import AiGatewayRateLimitWindowLimitMetric
from datadog_api_client.v2.model.ai_gateway_rate_limit_window_limit_window import AiGatewayRateLimitWindowLimitWindow
from uuid import UUID

body = AiGatewayRateLimitUpdateRequest(
Expand All @@ -23,6 +26,13 @@
tags=None,
team_id=None,
user_id=None,
window_limits=[
AiGatewayRateLimitWindowLimit(
amount=2000000000,
metric=AiGatewayRateLimitWindowLimitMetric.COST,
window=AiGatewayRateLimitWindowLimitWindow.TWENTY_FOUR_HOURS,
),
],
),
id=UUID("11111111-1111-1111-1111-111111111111"),
type=AiGatewayRateLimitType.RATE_LIMITS,
Expand Down
Loading
Loading