feat:Implement Gemini ASR support and remove preview routing

changelog.md

@@ -9,11 +9,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
 ### Added
 
 - **Inline REST LLM tools** — Global and CN LLM vendors now accept dictionary-based definitions or the exported typed `LlmToolConfig` models and serialize them to `llm.tools`. Added public `LlmToolConfig`, `LlmToolExecutionConfig`, `LlmToolFunctionConfig`, `LlmToolFunctionParametersConfig`, and `LlmToolServerConfig` aliases. Tool execution must be enabled explicitly with `Agent.with_tools()`.
+- **Gemini ASR** — Added `GeminiSTT` to the standard AgentKit STT vendors using the Fern-generated `GeminiAsrParams` schema while retaining the preview API's `language_codes`, `custom_vocabulary`, default model, default sample rate, and validation behavior. The production API's optional `language` parameter is also available.
 
 ### Changed
 
 - **Generated filler words configuration** — Generated filler word settings are now optional. The service can use default generator settings when `generated_config` is omitted, and `llm_provider`, `prompt`, and `fallback_strategy` may be omitted individually.
 - **ASR hotwords** — `keywords` on `AresSTT` and `FengmingSTT` now serialize as top-level `asr.keywords`, matching the current OpenAPI schema. Vendor-specific `additional_params` remain under `asr.params`; nested `additional_params["keywords"]` is rejected to prevent ambiguous requests.
+- **Gemini ASR routing** — Gemini ASR now uses the normal regional API endpoint and generated request validation. Existing `GeminiSTT` calls and imports from `agora_agent.agentkit.preview` remain compatible. The provider-agnostic preview client and session routing infrastructure remain available for future preview providers.
+- **Gemini ASR language mapping** — Added `language_hints` for candidate transcription languages. Deprecated `language_codes` remains supported as an alias; `language_hints` takes precedence when both are provided.
 
 ## [v2.7.2] — 2026-08-26
 

docs/concepts/vendors.md

@@ -119,6 +119,7 @@ top-level `asr.keywords`. Both vendors also accept `additional_params`, serializ
 | `DeepgramSTT` | Deepgram | `model` for Agora-managed `nova-2`/`nova-3`; `api_key` for BYOK; `language?`, `keyterm?` |
 | `MicrosoftSTT` | Microsoft Azure | `key`, `region`, `language` |
 | `OpenAISTT` | OpenAI | `api_key` |
+| `GeminiSTT` | Google Gemini | `api_key`; `model` defaults to `gemini-3.5-transcribe-live`; optional `mode` supports SMART or VERBATIM |
 | `GoogleSTT` | Google Cloud | `project_id`, `location`, `adc_credentials_string`, `language` |
 | `AmazonSTT` | Amazon Transcribe | `access_key`, `secret_key`, `region`, `language` |
 | `AssemblyAISTT` | AssemblyAI | `api_key`, `language` |
@@ -146,7 +147,9 @@ from agora_agent import DeepgramSTT
 stt = DeepgramSTT(api_key='your-deepgram-key', language='en-US', model='nova-2')
 ```
 
-> **Preview providers** — `GeminiSTT` (ASR) lives in `agora_agent.agentkit.preview`. Sessions using it route to the preview gateway automatically. See [Preview Endpoint](../guides/preview-endpoint.md).
+Preview providers, when available, are exposed from `agora_agent.agentkit.preview` and route through the
+preview gateway automatically. Gemini STT has graduated to the production API; its old preview import remains
+available as a compatibility alias. See [Preview Endpoint](../guides/preview-endpoint.md).
 
 ## MLLM Vendors
 

docs/guides/regional-routing.md

@@ -40,7 +40,7 @@ If you omit `with_stt()`, AgentKit uses `FengmingSTT` by default for `Area.CN` c
 
 | Client area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes |
 |---|---|---|---|---|---|
-| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
+| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GeminiSTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
 | `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |
 
 Global client example:

docs/index.md

@@ -49,11 +49,11 @@ The Agora Conversational AI Python SDK lets you build voice-powered AI agents on
 | [MLLM Flow](./guides/mllm-flow.md) | Use a global or CN realtime multimodal model for end-to-end audio |
 | [Avatars](./guides/avatars.md) | Add a digital avatar with LiveAvatar, Akool, Anam, Generic Avatar, SenseTime, or Spatius |
 | [Regional Routing](./guides/regional-routing.md) | Route requests to the nearest region |
+| [Preview Endpoint](./guides/preview-endpoint.md) | Understand session-scoped routing for preview providers |
 | [Error Handling](./guides/error-handling.md) | Handle API errors with ApiError |
 | [Pagination](./guides/pagination.md) | Iterate over paginated list endpoints |
 | [Advanced](./guides/advanced.md) | Raw response, retries, timeouts, custom httpx client |
 | [Low-Level API](./guides/low-level-api.md) | Generated REST APIs |
-| [Preview Endpoint](./guides/preview-endpoint.md) | Session-scoped preview routing and the `agora-feature` gate header |
 | [Client Reference](./reference/client.md) | Full `Agora` / `AsyncAgora` API |
 | [Agent Reference](./reference/agent.md) | Full `Agent` builder API |
 | [Session Reference](./reference/session.md) | Full `AgentSession` / `AsyncAgentSession` API |

docs/reference/vendors.md

@@ -20,7 +20,7 @@ Construct vendors directly from `agora_agent`, then bind a client with `Agent(cl
 
 | Area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes |
 |---|---|---|---|---|---|
-| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
+| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GeminiSTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
 | `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |
 
 Global example:
@@ -541,6 +541,36 @@ For `nova-2` and `nova-3`, omit `api_key` to use Agora-managed credentials. For
 | `model` | `str` | No | `None` | Recognition model |
 | `additional_params` | `Dict[str, Any]` | No | `None` | Additional parameters |
 
+### `GeminiSTT`
+
+| Parameter | Type | Required | Default | Description |
+|---|---|---|---|---|
+| `api_key` | `str` | Yes | — | Google Gemini API key |
+| `model` | `str` | No | `gemini-3.5-transcribe-live` | Gemini transcription model |
+| `language` | `str` | No | `None` | Language code for speech recognition. Takes precedence over top-level `asr.language`. |
+| `language_hints` | `List[str]` | No | `None` | Candidate transcription languages; `None` is omitted and an empty list is sent explicitly |
+| `language_codes` | `List[str]` | No | `None` | Deprecated alias for `language_hints`; ignored when `language_hints` is provided |
+| `custom_vocabulary` | `List[str]` | No | `None` | Words and phrases used to bias recognition |
+| `word_timestamp` | `bool` | No | `None` | Include word-level timestamps; incompatible with non-empty `custom_vocabulary` and SMART mode |
+| `mode` | `GeminiAsrParamsMode` | No | `None` | `SMART` or `VERBATIM`; `None` and an empty value are validated as VERBATIM and omitted from the request |
+| `diarization` | `bool` | No | `None` | Include speaker labels; `None` is omitted and treated as false during validation; true is incompatible with SMART mode |
+| `sample_rate` | `int` | No | `16000` | Audio sample rate in Hz |
+| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Gemini ASR parameters |
+
+Construction fails when non-empty `custom_vocabulary` is combined with `word_timestamp=True`, or when SMART mode is combined with `word_timestamp=True` or `diarization=True`. `additional_params` remains an unchecked passthrough dictionary.
+Existing imports from `agora_agent.agentkit.preview` remain supported and
+resolve to the same production `GeminiSTT` class.
+
+```python
+from agora_agent import GeminiSTT
+
+stt = GeminiSTT(
+    api_key="your-google-api-key",
+    language_hints=["en-US", "es-ES"],
+    custom_vocabulary=["Agora"],
+)
+```
+
 ### `AmazonSTT`
 
 | Parameter | Type | Required | Default | Description |

src/agora_agent/__init__.py

@@ -45,6 +45,8 @@
         ElevenLabsTTS,
         FishAudioTTS,
         Gemini,
+        GeminiSTT,
+        GeminiSTTModels,
         GeminiLive,
         GenericAvatar,
         GenericTTS,

src/agora_agent/agentkit/__init__.py

@@ -180,6 +180,8 @@
     Dify,
     FishAudioTTS,
     Gemini,
+    GeminiSTT,
+    GeminiSTTModels,
     GeminiLive,
     GenericAvatar,
     GoogleSTT,
@@ -436,6 +438,8 @@
     "MicrosoftSTT",
     "MicrosoftCNSTT",
     "OpenAISTT",
+    "GeminiSTT",
+    "GeminiSTTModels",
     "GoogleSTT",
     "AmazonSTT",
     "AssemblyAISTT",

src/agora_agent/agentkit/agent.py

@@ -202,11 +202,10 @@ class SessionOptions(typing_extensions.TypedDict, total=False):
     warn: typing.Callable[[str], None]
 
 
-
 def _drop_none(value: typing.Any) -> typing.Any:
     """Recursively remove None-valued mapping entries."""
     if isinstance(value, dict):
-        return {k: _drop_none(v) for k, v in value.items() if v is not None}
+        return {key: _drop_none(item) for key, item in value.items() if item is not None}
     if isinstance(value, list):
         return [_drop_none(item) for item in value]
     return value
@@ -218,16 +217,10 @@ def _start_properties_from_mapping(
     try:
         return parse_obj_as(StartAgentsRequestProperties, dict(properties))
     except Exception:
-        # Preview providers are absent from the generated unions by design — the
-        # schema models what production serves. A config the preview gateway
-        # does understand is passed through unvalidated rather than rejected.
-        # Imported lazily: preview.client imports the pool client, which imports
-        # this module.
+        # Preview providers may not exist in the production-generated union yet.
         from .preview.client import required_preview_features
 
         if required_preview_features(properties):
-            # The typed path serializes with exclude_none; strip None here so the
-            # bypass puts the same bytes on the wire instead of explicit nulls.
             return typing.cast(StartAgentsRequestProperties, _drop_none(dict(properties)))
         raise
 
@@ -1051,10 +1044,6 @@ def to_properties(
         if is_mllm_mode:
             if self._mllm is not None:
                 mllm_config = dict(self._mllm)
-                # These are production wire spellings. A route that spells one of them
-                # differently needs a rename entry in `preview/client.py`, or the value
-                # lands in a field the provider ignores and fails silently. See
-                # docs/guides/preview-endpoint.md#the-vendor-class-is-not-the-whole-wire-shape.
                 if self._greeting is not None:
                     mllm_config.setdefault("greeting_message", self._greeting)
                 if self._failure_message is not None:
@@ -1123,10 +1112,6 @@ def _resolve_asr_config(self, turn_detection_config: TurnDetectionInput) -> typi
         if not asr_config:
             area_scope = getattr(self._client, "area_scope", None)
             asr_config["vendor"] = "fengming" if area_scope == "cn" else "ares"
-        # Unconditional: turn detection is the single source of truth for the
-        # interaction language, so a vendor-level ``language`` would be silently
-        # discarded here. Do not add one to a vendor class — see
-        # docs/guides/preview-endpoint.md#the-vendor-class-is-not-the-whole-wire-shape.
         asr_config["language"] = self._field_value(turn_detection_config, "language")
         return asr_config
 

src/agora_agent/agentkit/agent_session.py

@@ -185,19 +185,15 @@ def _require_agent_management(self) -> typing.Any:
     def _bind_session_clients(self, features: typing.Sequence[str]) -> None:
         """Pin this session to production or preview without mutating its client."""
         if features:
-            self._agents, self._agent_management = create_preview_session_clients(
-                self._client, features
-            )
+            self._agents, self._agent_management = create_preview_session_clients(self._client, features)
             from .preview.client import PREVIEW_API_BASE_URL
 
             self._api_base_url = PREVIEW_API_BASE_URL
             return
         self._agents = self._client.agents
         self._agent_management = getattr(self._client, "agent_management", None)
         self._api_base_url = (
-            self._client.get_current_url()
-            if hasattr(self._client, "get_current_url")
-            else None
+            self._client.get_current_url() if hasattr(self._client, "get_current_url") else None
         )
 
     # ------------------------------------------------------------------

src/agora_agent/agentkit/preview/client.py

@@ -90,7 +90,9 @@ def create_preview_session_clients(
 
 
 #: ASR vendors served only by the preview endpoint.
-_PREVIEW_ASR_VENDORS = frozenset({"gemini"})
+_PREVIEW_FEATURES_BY_CATEGORY: typing.Dict[str, typing.Dict[str, PreviewFeature]] = {
+    "asr": {},
+}
 
 
 def required_preview_features(properties: typing.Mapping[str, typing.Any]) -> typing.List[str]:
@@ -100,11 +102,16 @@ def required_preview_features(properties: typing.Mapping[str, typing.Any]) -> ty
     hand-written configs are covered too.
     """
     features: typing.List[str] = []
-
-    asr = properties.get("asr")
-    if isinstance(asr, dict) and asr.get("vendor") in _PREVIEW_ASR_VENDORS:
-        features.append(PreviewFeatures.GEMINI_LIVE)
-
+    for category, vendors in _PREVIEW_FEATURES_BY_CATEGORY.items():
+        config = properties.get(category)
+        if not isinstance(config, dict):
+            continue
+        vendor = config.get("vendor")
+        if not isinstance(vendor, str):
+            continue
+        feature = vendors.get(vendor)
+        if feature is not None and feature not in features:
+            features.append(feature)
     return features