Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions .generator/schemas/v2/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77999,6 +77999,13 @@ components:
LLMObsAnnotationItem:
description: A single annotation on an interaction.
properties:
annotator:
description: |-
Caller-supplied name or identifier for the external person or automated system that produced the labels.
The value is stored and returned as provided. It is not a Datadog user ID or login, and it is not
resolved against Datadog users.
example: "Alex Contractor"
type: string
created_at:
description: Timestamp when the annotation was created.
example: "2024-01-15T10:30:00Z"
Expand Down Expand Up @@ -78044,6 +78051,13 @@ components:
LLMObsAnnotationItemResponse:
description: A single annotation on an interaction, as returned by the API.
properties:
annotator:
description: |-
Caller-supplied name or identifier for the external person or automated system that produced the labels.
The value is stored and returned as provided. It is not a Datadog user ID or login, and it is not
resolved against Datadog users.
example: "Alex Contractor"
type: string
created_at:
description: Timestamp when the annotation was created.
example: "2024-01-15T10:30:00Z"
Expand Down Expand Up @@ -84929,6 +84943,18 @@ components:
A single annotation to create or update. The annotation is matched by
`interaction_id` and the requesting user's identity.
properties:
annotator:
description: |-
Caller-supplied name or identifier for the external person or automated system that produced the labels.
The value is an opaque string with no required format. Use a stable, recognizable value for the same
producer across annotations.
The value is stored and returned as provided. It is not a Datadog user ID or login, and it is not
resolved against Datadog users.
Omit this field or set it to `null` to preserve existing attribution. Set it to an empty string to clear
attribution.
example: "Alex Contractor"
nullable: true
type: string
interaction_id:
description: ID of the interaction to annotate.
example: "00000000-0000-0000-0000-000000000001"
Expand Down Expand Up @@ -208224,6 +208250,8 @@ paths:
description: |-
Create or update annotations on interactions in a queue. Each annotation is matched
by `interaction_id` and the requesting user's identity.
Use the optional `annotator` field to name an external person or automated system that produced the labels
without changing the authenticated creator, modifier, permissions, or per-user matching behavior.
Results and errors in the response are linked to request items by `interaction_id`.
Errors for individual items are returned in the `errors` field without blocking the rest of the batch.
Requests where every item is denied by queue access rules return `403`.
Expand All @@ -208239,7 +208267,8 @@ paths:
data:
attributes:
annotations:
- interaction_id: 00000000-0000-0000-0000-000000000001
- annotator: "Alex Contractor"
interaction_id: 00000000-0000-0000-0000-000000000001
label_values:
- label_schema_id: abc-123
value: good
Expand All @@ -208260,7 +208289,8 @@ paths:
data:
attributes:
annotations:
- created_at: "2024-01-15T10:30:00Z"
- annotator: "Alex Contractor"
created_at: "2024-01-15T10:30:00Z"
created_by: 00000000-0000-0000-0000-000000000002
id: 00000000-0000-0000-0000-000000000000
interaction_id: 00000000-0000-0000-0000-000000000001
Expand All @@ -208277,7 +208307,8 @@ paths:
data:
attributes:
annotations:
- created_at: "2024-01-15T10:30:00Z"
- annotator: "Alex Contractor"
created_at: "2024-01-15T10:30:00Z"
created_by: 00000000-0000-0000-0000-000000000002
id: 00000000-0000-0000-0000-000000000000
interaction_id: 00000000-0000-0000-0000-000000000001
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
attributes: DatadogAPIClient::V2::LLMObsAnnotationsDataAttributesRequest.new({
annotations: [
DatadogAPIClient::V2::LLMObsUpsertAnnotationItem.new({
annotator: "Alex Contractor",
interaction_id: "00000000-0000-0000-0000-000000000001",
label_values: [
DatadogAPIClient::V2::LLMObsAnnotationLabelValue.new({
Expand Down
6 changes: 3 additions & 3 deletions features/v2/agent_observability.feature
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,7 @@ Feature: Agent Observability
Given operation "UpsertLLMObsAnnotations" enabled
And new "UpsertLLMObsAnnotations" request
And request contains "queue_id" parameter from "REPLACE.ME"
And body with value {"data": {"attributes": {"annotations": [{"interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [{"label_schema_id": "abc-123", "value": "good"}, {"label_schema_id": "ef56gh78", "value": "positive"}]}]}, "type": "annotations"}}
And body with value {"data": {"attributes": {"annotations": [{"annotator": "Alex Contractor", "interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [{"label_schema_id": "abc-123", "value": "good"}, {"label_schema_id": "ef56gh78", "value": "positive"}]}]}, "type": "annotations"}}
When the request is sent
Then the response status is 400 Bad Request

Expand All @@ -404,7 +404,7 @@ Feature: Agent Observability
Given operation "UpsertLLMObsAnnotations" enabled
And new "UpsertLLMObsAnnotations" request
And request contains "queue_id" parameter from "REPLACE.ME"
And body with value {"data": {"attributes": {"annotations": [{"interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [{"label_schema_id": "abc-123", "value": "good"}, {"label_schema_id": "ef56gh78", "value": "positive"}]}]}, "type": "annotations"}}
And body with value {"data": {"attributes": {"annotations": [{"annotator": "Alex Contractor", "interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [{"label_schema_id": "abc-123", "value": "good"}, {"label_schema_id": "ef56gh78", "value": "positive"}]}]}, "type": "annotations"}}
When the request is sent
Then the response status is 404 Not Found — the queue does not exist.

Expand All @@ -413,7 +413,7 @@ Feature: Agent Observability
Given operation "UpsertLLMObsAnnotations" enabled
And new "UpsertLLMObsAnnotations" request
And request contains "queue_id" parameter from "REPLACE.ME"
And body with value {"data": {"attributes": {"annotations": [{"interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [{"label_schema_id": "abc-123", "value": "good"}, {"label_schema_id": "ef56gh78", "value": "positive"}]}]}, "type": "annotations"}}
And body with value {"data": {"attributes": {"annotations": [{"annotator": "Alex Contractor", "interaction_id": "00000000-0000-0000-0000-000000000001", "label_values": [{"label_schema_id": "abc-123", "value": "good"}, {"label_schema_id": "ef56gh78", "value": "positive"}]}]}, "type": "annotations"}}
When the request is sent
Then the response status is 200 OK — annotations created or updated. For mixed batches, denied items and other per-item errors are listed in `errors`.

Expand Down
2 changes: 2 additions & 0 deletions lib/datadog_api_client/v2/api/agent_observability_api.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5619,6 +5619,8 @@ def upsert_llm_obs_annotations(queue_id, body, opts = {})
#
# Create or update annotations on interactions in a queue. Each annotation is matched
# by `interaction_id` and the requesting user's identity.
# Use the optional `annotator` field to name an external person or automated system that produced the labels
# without changing the authenticated creator, modifier, permissions, or per-user matching behavior.
# Results and errors in the response are linked to request items by `interaction_id`.
# Errors for individual items are returned in the `errors` field without blocking the rest of the batch.
# Requests where every item is denied by queue access rules return `403`.
Expand Down
14 changes: 13 additions & 1 deletion lib/datadog_api_client/v2/models/llm_obs_annotation_item.rb
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@ module DatadogAPIClient::V2
class LLMObsAnnotationItem
include BaseGenericModel

# Caller-supplied name or identifier for the external person or automated system that produced the labels.
# The value is stored and returned as provided. It is not a Datadog user ID or login, and it is not
# resolved against Datadog users.
attr_accessor :annotator

# Timestamp when the annotation was created.
attr_reader :created_at

Expand Down Expand Up @@ -48,6 +53,7 @@ class LLMObsAnnotationItem
# @!visibility private
def self.attribute_map
{
:'annotator' => :'annotator',
:'created_at' => :'created_at',
:'created_by' => :'created_by',
:'id' => :'id',
Expand All @@ -62,6 +68,7 @@ def self.attribute_map
# @!visibility private
def self.openapi_types
{
:'annotator' => :'String',
:'created_at' => :'Time',
:'created_by' => :'String',
:'id' => :'String',
Expand Down Expand Up @@ -90,6 +97,10 @@ def initialize(attributes = {})
end
}

if attributes.key?(:'annotator')
self.annotator = attributes[:'annotator']
end

if attributes.key?(:'created_at')
self.created_at = attributes[:'created_at']
end
Expand Down Expand Up @@ -229,6 +240,7 @@ def to_hash
def ==(o)
return true if self.equal?(o)
self.class == o.class &&
annotator == o.annotator &&
created_at == o.created_at &&
created_by == o.created_by &&
id == o.id &&
Expand All @@ -243,7 +255,7 @@ def ==(o)
# @return [Integer] Hash code
# @!visibility private
def hash
[created_at, created_by, id, interaction_id, label_values, modified_at, modified_by, additional_properties].hash
[annotator, created_at, created_by, id, interaction_id, label_values, modified_at, modified_by, additional_properties].hash
end
end
end
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@ module DatadogAPIClient::V2
class LLMObsAnnotationItemResponse
include BaseGenericModel

# Caller-supplied name or identifier for the external person or automated system that produced the labels.
# The value is stored and returned as provided. It is not a Datadog user ID or login, and it is not
# resolved against Datadog users.
attr_accessor :annotator

# Timestamp when the annotation was created.
attr_reader :created_at

Expand Down Expand Up @@ -49,6 +54,7 @@ class LLMObsAnnotationItemResponse
# @!visibility private
def self.attribute_map
{
:'annotator' => :'annotator',
:'created_at' => :'created_at',
:'created_by' => :'created_by',
:'id' => :'id',
Expand All @@ -63,6 +69,7 @@ def self.attribute_map
# @!visibility private
def self.openapi_types
{
:'annotator' => :'String',
:'created_at' => :'Time',
:'created_by' => :'String',
:'id' => :'String',
Expand Down Expand Up @@ -91,6 +98,10 @@ def initialize(attributes = {})
end
}

if attributes.key?(:'annotator')
self.annotator = attributes[:'annotator']
end

if attributes.key?(:'created_at')
self.created_at = attributes[:'created_at']
end
Expand Down Expand Up @@ -232,6 +243,7 @@ def to_hash
def ==(o)
return true if self.equal?(o)
self.class == o.class &&
annotator == o.annotator &&
created_at == o.created_at &&
created_by == o.created_by &&
id == o.id &&
Expand All @@ -246,7 +258,7 @@ def ==(o)
# @return [Integer] Hash code
# @!visibility private
def hash
[created_at, created_by, id, interaction_id, label_values, modified_at, modified_by, additional_properties].hash
[annotator, created_at, created_by, id, interaction_id, label_values, modified_at, modified_by, additional_properties].hash
end
end
end
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,15 @@ module DatadogAPIClient::V2
class LLMObsUpsertAnnotationItem
include BaseGenericModel

# Caller-supplied name or identifier for the external person or automated system that produced the labels.
# The value is an opaque string with no required format. Use a stable, recognizable value for the same
# producer across annotations.
# The value is stored and returned as provided. It is not a Datadog user ID or login, and it is not
# resolved against Datadog users.
# Omit this field or set it to `null` to preserve existing attribution. Set it to an empty string to clear
# attribution.
attr_accessor :annotator

# ID of the interaction to annotate.
attr_reader :interaction_id

Expand All @@ -35,6 +44,7 @@ class LLMObsUpsertAnnotationItem
# @!visibility private
def self.attribute_map
{
:'annotator' => :'annotator',
:'interaction_id' => :'interaction_id',
:'label_values' => :'label_values'
}
Expand All @@ -44,11 +54,20 @@ def self.attribute_map
# @!visibility private
def self.openapi_types
{
:'annotator' => :'String',
:'interaction_id' => :'String',
:'label_values' => :'Array<LLMObsAnnotationLabelValue>'
}
end

# List of attributes with nullable: true
# @!visibility private
def self.openapi_nullable
Set.new([
:'annotator',
])
end

# Initializes the object
# @param attributes [Hash] Model attributes in the form of hash
# @!visibility private
Expand All @@ -67,6 +86,10 @@ def initialize(attributes = {})
end
}

if attributes.key?(:'annotator')
self.annotator = attributes[:'annotator']
end

if attributes.key?(:'interaction_id')
self.interaction_id = attributes[:'interaction_id']
end
Expand Down Expand Up @@ -137,6 +160,7 @@ def to_hash
def ==(o)
return true if self.equal?(o)
self.class == o.class &&
annotator == o.annotator &&
interaction_id == o.interaction_id &&
label_values == o.label_values &&
additional_properties == o.additional_properties
Expand All @@ -146,7 +170,7 @@ def ==(o)
# @return [Integer] Hash code
# @!visibility private
def hash
[interaction_id, label_values, additional_properties].hash
[annotator, interaction_id, label_values, additional_properties].hash
end
end
end
Loading