Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
### Added

- **Inline REST LLM tools** — Global and CN LLM vendors now accept dictionary-based `tools` definitions and serialize them to `llm.tools`. Added public `LlmToolConfig`, `LlmToolExecutionConfig`, `LlmToolFunctionConfig`, `LlmToolFunctionParametersConfig`, and `LlmToolServerConfig` aliases. Tool execution must be enabled explicitly with `Agent.with_tools()`.
- **Gemini ASR** — Added `GeminiSTT` to the standard AgentKit STT vendors using the Fern-generated `GeminiAsrParams` schema. It requires `api_key` and `model`; `language`, `word_timestamp`, and `sample_rate` are optional.

### Changed

- **Generated filler words configuration** — Generated filler word settings are now optional. The service can use default generator settings when `generated_config` is omitted, and `llm_provider`, `prompt`, and `fallback_strategy` may be omitted individually.
- **Gemini ASR routing** — Gemini ASR now uses the normal regional API endpoint and generated request validation. The temporary preview routing layer has been removed.

## [v2.7.2] — 2026-08-26

Expand Down
3 changes: 1 addition & 2 deletions docs/concepts/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ top-level `asr.keywords`. Both vendors also accept `additional_params`, serializ
| `DeepgramSTT` | Deepgram | `model` for Agora-managed `nova-2`/`nova-3`; `api_key` for BYOK; `language?`, `keyterm?` |
| `MicrosoftSTT` | Microsoft Azure | `key`, `region`, `language` |
| `OpenAISTT` | OpenAI | `api_key` |
| `GeminiSTT` | Google Gemini | `api_key`, `model`; optional `language`, `word_timestamp`, `sample_rate` |
| `GoogleSTT` | Google Cloud | `project_id`, `location`, `adc_credentials_string`, `language` |
| `AmazonSTT` | Amazon Transcribe | `access_key`, `secret_key`, `region`, `language` |
| `AssemblyAISTT` | AssemblyAI | `api_key`, `language` |
Expand Down Expand Up @@ -146,8 +147,6 @@ from agora_agent import DeepgramSTT
stt = DeepgramSTT(api_key='your-deepgram-key', language='en-US', model='nova-2')
```

> **Preview providers** — `GeminiSTT` (ASR) lives in `agora_agent.agentkit.preview`. Sessions using it route to the preview gateway automatically. See [Preview Endpoint](../guides/preview-endpoint.md).

## MLLM Vendors

Used with `agent.with_mllm()` for the [MLLM flow](../guides/mllm-flow.md). These handle audio input and output end-to-end.
Expand Down
231 changes: 0 additions & 231 deletions docs/guides/preview-endpoint.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/guides/regional-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ If you omit `with_stt()`, AgentKit uses `FengmingSTT` by default for `Area.CN` c

| Client area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GeminiSTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |

Global client example:
Expand Down
1 change: 0 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@ The Agora Conversational AI Python SDK lets you build voice-powered AI agents on
| [Pagination](./guides/pagination.md) | Iterate over paginated list endpoints |
| [Advanced](./guides/advanced.md) | Raw response, retries, timeouts, custom httpx client |
| [Low-Level API](./guides/low-level-api.md) | Generated REST APIs |
| [Preview Endpoint](./guides/preview-endpoint.md) | Session-scoped preview routing and the `agora-feature` gate header |
| [Client Reference](./reference/client.md) | Full `Agora` / `AsyncAgora` API |
| [Agent Reference](./reference/agent.md) | Full `Agent` builder API |
| [Session Reference](./reference/session.md) | Full `AgentSession` / `AsyncAgentSession` API |
Expand Down
24 changes: 23 additions & 1 deletion docs/reference/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Construct vendors directly from `agora_agent`, then bind a client with `Agent(cl

| Area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GeminiSTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |

Global example:
Expand Down Expand Up @@ -541,6 +541,28 @@ For `nova-2` and `nova-3`, omit `api_key` to use Agora-managed credentials. For
| `model` | `str` | No | `None` | Recognition model |
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional parameters |

### `GeminiSTT`

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | `str` | Yes | — | Google Gemini API key |
| `model` | `str` | Yes | — | Gemini transcription model, such as `gemini-3.7-transcribe-live` |
| `language` | `str` | No | `None` | Language code for speech recognition. Takes precedence over top-level `asr.language`. |
| `word_timestamp` | `bool` | No | `None` | Include word-level timestamps in transcription results |
| `sample_rate` | `int` | No | `None` | Audio sample rate in Hz |
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Gemini ASR parameters |

```python
from agora_agent import GeminiSTT

stt = GeminiSTT(
api_key="your-google-api-key",
model="gemini-3.7-transcribe-live",
language="en-US",
word_timestamp=True,
)
```

### `AmazonSTT`

| Parameter | Type | Required | Default | Description |
Expand Down
1 change: 1 addition & 0 deletions src/agora_agent/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
ElevenLabsTTS,
FishAudioTTS,
Gemini,
GeminiSTT,
GeminiLive,
GenericAvatar,
GenericTTS,
Expand Down
2 changes: 2 additions & 0 deletions src/agora_agent/agentkit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@
Dify,
FishAudioTTS,
Gemini,
GeminiSTT,
GeminiLive,
GenericAvatar,
GoogleSTT,
Expand Down Expand Up @@ -436,6 +437,7 @@
"MicrosoftSTT",
"MicrosoftCNSTT",
"OpenAISTT",
"GeminiSTT",
"GoogleSTT",
"AmazonSTT",
"AssemblyAISTT",
Expand Down
34 changes: 1 addition & 33 deletions src/agora_agent/agentkit/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -202,34 +202,10 @@ class SessionOptions(typing_extensions.TypedDict, total=False):
warn: typing.Callable[[str], None]



def _drop_none(value: typing.Any) -> typing.Any:
"""Recursively remove None-valued mapping entries."""
if isinstance(value, dict):
return {k: _drop_none(v) for k, v in value.items() if v is not None}
if isinstance(value, list):
return [_drop_none(item) for item in value]
return value


def _start_properties_from_mapping(
properties: typing.Mapping[str, typing.Any],
) -> StartAgentsRequestProperties:
try:
return parse_obj_as(StartAgentsRequestProperties, dict(properties))
except Exception:
# Preview providers are absent from the generated unions by design — the
# schema models what production serves. A config the preview gateway
# does understand is passed through unvalidated rather than rejected.
# Imported lazily: preview.client imports the pool client, which imports
# this module.
from .preview.client import required_preview_features

if required_preview_features(properties):
# The typed path serializes with exclude_none; strip None here so the
# bypass puts the same bytes on the wire instead of explicit nulls.
return typing.cast(StartAgentsRequestProperties, _drop_none(dict(properties)))
raise
return parse_obj_as(StartAgentsRequestProperties, dict(properties))


# LLM sub-type aliases
Expand Down Expand Up @@ -1051,10 +1027,6 @@ def to_properties(
if is_mllm_mode:
if self._mllm is not None:
mllm_config = dict(self._mllm)
# These are production wire spellings. A route that spells one of them
# differently needs a rename entry in `preview/client.py`, or the value
# lands in a field the provider ignores and fails silently. See
# docs/guides/preview-endpoint.md#the-vendor-class-is-not-the-whole-wire-shape.
if self._greeting is not None:
mllm_config.setdefault("greeting_message", self._greeting)
if self._failure_message is not None:
Expand Down Expand Up @@ -1123,10 +1095,6 @@ def _resolve_asr_config(self, turn_detection_config: TurnDetectionInput) -> typi
if not asr_config:
area_scope = getattr(self._client, "area_scope", None)
asr_config["vendor"] = "fengming" if area_scope == "cn" else "ares"
# Unconditional: turn detection is the single source of truth for the
# interaction language, so a vendor-level ``language`` would be silently
# discarded here. Do not add one to a vendor class — see
# docs/guides/preview-endpoint.md#the-vendor-class-is-not-the-whole-wire-shape.
asr_config["language"] = self._field_value(turn_detection_config, "language")
return asr_config

Expand Down
23 changes: 0 additions & 23 deletions src/agora_agent/agentkit/agent_session.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@
normalize_preset_input,
resolve_session_presets,
)
from .preview.client import create_preview_session_clients, required_preview_features
from .token import _parse_numeric_uid, generate_convo_ai_token


Expand Down Expand Up @@ -182,24 +181,6 @@ def _require_agent_management(self) -> typing.Any:
)
return self._agent_management

def _bind_session_clients(self, features: typing.Sequence[str]) -> None:
"""Pin this session to production or preview without mutating its client."""
if features:
self._agents, self._agent_management = create_preview_session_clients(
self._client, features
)
from .preview.client import PREVIEW_API_BASE_URL

self._api_base_url = PREVIEW_API_BASE_URL
return
self._agents = self._client.agents
self._agent_management = getattr(self._client, "agent_management", None)
self._api_base_url = (
self._client.get_current_url()
if hasattr(self._client, "get_current_url")
else None
)

# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
Expand Down Expand Up @@ -636,8 +617,6 @@ def start(self) -> str:
properties,
)

self._bind_session_clients(required_preview_features(resolved_properties))

if self._debug:
print("[Agora Debug] Starting agent session...")
if hasattr(self._client, "get_current_url"):
Expand Down Expand Up @@ -1003,8 +982,6 @@ async def start(self) -> str:
properties,
)

self._bind_session_clients(required_preview_features(resolved_properties))

if self._debug:
print("[Agora Debug] Starting agent session...")
if hasattr(self._client, "get_current_url"):
Expand Down
29 changes: 0 additions & 29 deletions src/agora_agent/agentkit/preview/__init__.py

This file was deleted.

118 changes: 0 additions & 118 deletions src/agora_agent/agentkit/preview/client.py

This file was deleted.

Loading
Loading