diff --git a/changelog.md b/changelog.md index c229546..b4c848a 100644 --- a/changelog.md +++ b/changelog.md @@ -9,11 +9,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/). ### Added - **Inline REST LLM tools** — Global and CN LLM vendors now accept dictionary-based definitions or the exported typed `LlmToolConfig` models and serialize them to `llm.tools`. Added public `LlmToolConfig`, `LlmToolExecutionConfig`, `LlmToolFunctionConfig`, `LlmToolFunctionParametersConfig`, and `LlmToolServerConfig` aliases. Tool execution must be enabled explicitly with `Agent.with_tools()`. +- **Gemini ASR** — Added `GeminiSTT` to the standard AgentKit STT vendors using the Fern-generated `GeminiAsrParams` schema while retaining the preview API's `language_codes`, `custom_vocabulary`, default model, default sample rate, and validation behavior. The production API's optional `language` parameter is also available. ### Changed - **Generated filler words configuration** — Generated filler word settings are now optional. The service can use default generator settings when `generated_config` is omitted, and `llm_provider`, `prompt`, and `fallback_strategy` may be omitted individually. - **ASR hotwords** — `keywords` on `AresSTT` and `FengmingSTT` now serialize as top-level `asr.keywords`, matching the current OpenAPI schema. Vendor-specific `additional_params` remain under `asr.params`; nested `additional_params["keywords"]` is rejected to prevent ambiguous requests. +- **Gemini ASR routing** — Gemini ASR now uses the normal regional API endpoint and generated request validation. Existing `GeminiSTT` calls and imports from `agora_agent.agentkit.preview` remain compatible. The provider-agnostic preview client and session routing infrastructure remain available for future preview providers. +- **Gemini ASR language mapping** — Added `language_hints` for candidate transcription languages. Deprecated `language_codes` remains supported as an alias; `language_hints` takes precedence when both are provided. ## [v2.7.2] — 2026-08-26 diff --git a/docs/concepts/vendors.md b/docs/concepts/vendors.md index 1fc868b..bf398bd 100644 --- a/docs/concepts/vendors.md +++ b/docs/concepts/vendors.md @@ -119,6 +119,7 @@ top-level `asr.keywords`. Both vendors also accept `additional_params`, serializ | `DeepgramSTT` | Deepgram | `model` for Agora-managed `nova-2`/`nova-3`; `api_key` for BYOK; `language?`, `keyterm?` | | `MicrosoftSTT` | Microsoft Azure | `key`, `region`, `language` | | `OpenAISTT` | OpenAI | `api_key` | +| `GeminiSTT` | Google Gemini | `api_key`; `model` defaults to `gemini-3.5-transcribe-live`; optional `mode` supports SMART or VERBATIM | | `GoogleSTT` | Google Cloud | `project_id`, `location`, `adc_credentials_string`, `language` | | `AmazonSTT` | Amazon Transcribe | `access_key`, `secret_key`, `region`, `language` | | `AssemblyAISTT` | AssemblyAI | `api_key`, `language` | @@ -146,7 +147,9 @@ from agora_agent import DeepgramSTT stt = DeepgramSTT(api_key='your-deepgram-key', language='en-US', model='nova-2') ``` -> **Preview providers** — `GeminiSTT` (ASR) lives in `agora_agent.agentkit.preview`. Sessions using it route to the preview gateway automatically. See [Preview Endpoint](../guides/preview-endpoint.md). +Preview providers, when available, are exposed from `agora_agent.agentkit.preview` and route through the +preview gateway automatically. Gemini STT has graduated to the production API; its old preview import remains +available as a compatibility alias. See [Preview Endpoint](../guides/preview-endpoint.md). ## MLLM Vendors diff --git a/docs/guides/preview-endpoint.md b/docs/guides/preview-endpoint.md index 8ea8859..ba9d566 100644 --- a/docs/guides/preview-endpoint.md +++ b/docs/guides/preview-endpoint.md @@ -1,231 +1,51 @@ --- sidebar_position: 10 title: Preview Endpoint -description: How AgentSession routes preview providers and pins the gateway's agora-feature gate header. +description: How AgentSession routes preview providers and pins the agora-feature gate header. --- # Preview Endpoint -Some providers ship on a preview gateway before they reach the production Conversational AI environment. Standard `Agora` and `AsyncAgora` clients detect these providers from the resolved start body and route that session automatically. +Some providers may be released through a preview gateway before their production API is available. `AgentSession` +and `AsyncAgentSession` detect registered preview providers from the resolved start request and route the entire +session automatically. -Everything in `agentkit/preview/` is temporary by design. When these providers go GA, the package is deleted and the vendor classes move into `vendors/stt.py`. +There are currently no providers registered for preview routing. Gemini STT has graduated to production and uses +the normal regional endpoint. Existing imports of `GeminiSTT` and `GeminiSTTModels` from +`agora_agent.agentkit.preview` remain supported as compatibility aliases. -## Using a preview provider +## Session-scoped routing -```python -import os - -from agora_agent import Agora, Area -from agora_agent.agentkit import Agent -from agora_agent.agentkit import Gemini, GoogleTTS -from agora_agent.agentkit.preview import GeminiSTT - -client = Agora( - area=Area.US, - app_id=os.environ["AGORA_APP_ID"], - app_certificate=os.environ["AGORA_APP_CERTIFICATE"], -) - -google_api_key = os.environ["GOOGLE_API_KEY"] -session = ( - Agent(client=client) - .with_stt(GeminiSTT(api_key=google_api_key, language_codes=["en-US"])) - .with_llm(Gemini(api_key=google_api_key, model="gemini-2.0-flash")) - .with_tts(GoogleTTS( - key=google_api_key, - voice_name="en-US-Chirp3-HD-Charon", - language_code="en-US", - )) - .create_session(channel="demo", agent_uid="1", remote_uids=["100"]) -) -agent_id = session.start() -``` - -`AsyncAgora` behaves the same, with `await session.start()`. - -Routing is session-scoped. Preview session calls use the preview host and pinned gate header; GA sessions created from the same client continue using the production regional endpoint. - -## The gate header - -The gateway routes preview traffic on a single request header: - -``` -agora-feature: gemini-live -``` - -| Constant | Value | -| ----------------------------- | --------------- | -| `PREVIEW_FEATURE_HEADER` | `agora-feature` | -| `PreviewFeatures.GEMINI_LIVE` | `gemini-live` | - -### The header is not overridable - -For preview sessions, the gate header is merged **after** caller-supplied client `headers`, so custom headers cannot drop or blank it: - -```python -client = Agora( - area=Area.US, - app_id=..., - app_certificate=..., - headers={"agora-feature": "", "x-custom": "kept"}, -) -# Requests still send agora-feature: gemini-live, and x-custom: kept -``` - -This ordering is deliberate and load-bearing. A preview request that loses the header is not rejected — it routes to the production environment, where the preview providers do not exist. - -The header rides every session verb, not just `start()` — `say`, `interrupt`, `think`, `update`, `get_history`, `get_info`, `get_turns`, and `stop`. The top-level `Agora.stop_agent()` and `AsyncAgora.stop_agent()` methods remain production-only because they do not carry session routing state. - -## Preview providers bypass request validation - -The generated request models mirror what production serves, so `asr.vendor = "gemini"` is not a member of the generated `Asr` union and pydantic rejects it. - -`_start_properties_from_mapping` catches that and, when `required_preview_features()` recognises the config, passes the mapping through unvalidated instead of raising. Production configs still get full validation — only configs the preview gateway understands take the bypass. - -This is worth knowing when adding a preview provider: if a new vendor is missing from the generated unions, register it in the detection sets in `preview/client.py` rather than editing generated code. - -## Intake node behavior - -The gateway decides where a request goes before it validates the body. That produces failure modes that look like outages but are routing problems. - -| Symptom | What it means | -| ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | -| `503` `{"reason":"ServiceUnavailable"}` on `POST .../join` | The gate header was not recognised. The request died before validation, so the body is irrelevant. | -| `401` `Missing authorization header` | Routing worked; auth did not. | -| `404` `no Route matched with those values` | The base URL path is wrong. | -| `400` validation error | You are past the gate. The header is fine and the body is the problem. | - -The 503 is the one that misleads. It reads as a partner-side outage and invites waiting it out, when the fix is usually a one-line header change. - -Observed on the `gemini-live` rollout in August 2026, when the gateway had not yet been configured to route on `agora-feature` and every request fell through to a 503. That was fixed server-side on 2026-08-09, so the 503 is not currently reproducible — the mapping is recorded here because it is the failure signature a newly provisioned preview family is most likely to hit first. - -### Diagnosing without starting a billable agent - -Two probes, neither of which allocates an agent: - -1. **`GET .../v2/projects/{appId}/agents`** — a `200` proves host, auth, and routing are all healthy. If this succeeds while `join` fails, the problem is specific to the start path. -2. **A deliberately invalid start body** — send properties with no `llm` and no `mllm` at all. A `400` means you are past the gate; a `503` means you are not. This is what separates "my config is wrong" from "my header is wrong". - -### Known gap (as of 2026-08-09) - -A missing gate header is _intended_ to route to the production environment, where a preview config would fail. In practice an ungated start currently **succeeds** against the preview host, so the fallback is not observable from the client side. - -The SDK cannot control the intake node, and this does not affect SDK users because the header is pinned. It matters only for callers hitting the REST API directly, who may get a request that appears to succeed while silently landing in the wrong environment. Flag it to the endpoint owner rather than working around it in SDK code. - -## Session routing detection - -`required_preview_features()` reads the resolved request body rather than the vendor classes, so hand-written configs and preset resolution are covered too. It keys on `asr.vendor` — the vendor names served only by the preview endpoint, listed as `_PREVIEW_ASR_VENDORS` in `preview/client.py`. - -## Preview vendors +Preview routing does not mutate the bound `Agora` or `AsyncAgora` client. A session that needs a preview feature +receives private generated clients configured with: -| Class | Wire vendor | Model | -| ----------- | ----------------------- | ---------------------------- | -| `GeminiSTT` | `asr.vendor = "gemini"` | `gemini-3.5-transcribe-live` | +- `https://partner.ai.agora.io/preview/api/conversational-ai-agent` as the base URL. +- `agora-feature` as the feature gate header. +- All custom headers, authentication settings, timeouts, and the supplied `httpx` client from the original client. -`GeminiSTT` is an ASR stage, so it needs an LLM and a TTS vendor alongside it. The sample above uses Gemini LLM and Google TTS with the same Google API key. Mixing in other vendors is still valid; preview routing triggers only on `asr.vendor`. +The gate header is applied after caller-provided headers, so it cannot be accidentally blanked or replaced. It is +kept on every request made through that session. Production sessions created from the same client continue using +the regional production endpoint. -### ASR language selection +## Adding a preview provider -Gemini Transcribe takes `params.language_codes`, an **array**, in place of the singular `params.language` other ASR vendors use. +Preview vendor classes should use the same `BaseSTT`, `BaseLLM`, `BaseMLLM`, `BaseTTS`, or `BaseAvatar` interfaces +as production vendors. Register the vendor and its feature gate in `_PREVIEW_FEATURES_BY_CATEGORY` in +`agentkit/preview/client.py`. -```python -# Auto-detect (the default) — language_codes is not sent at all -GeminiSTT(api_key=...) - -# Commit to one language -GeminiSTT(api_key=..., language_codes=["en-US"]) - -# Let the model choose between several -GeminiSTT(api_key=..., language_codes=["en-US", "es-ES"]) - -# Auto-detect, stated outright -GeminiSTT(api_key=..., language_codes=[]) -``` - -`language_codes` is omitted from the request unless you supply it, which is how the provider spells auto-detect. Omitting the field and sending `[]` mean the same thing. - -`GeminiSTT` takes **no `language` argument**. `Agent` always derives the top-level `asr.language` from the turn detection language — as it does for every STT vendor — so a vendor-level copy would be a no-op the builder overwrites. Set the interaction language on turn detection, and the transcription languages on `language_codes`; they are separate settings and neither feeds the other. The model forbids extra fields, so a stale `language=` argument raises rather than being silently dropped. - -| Setting | Where it belongs | What it controls | -| ----------------------- | ------------------------------ | --------------------------- | -| interaction language | turn detection `language` | top-level `asr.language` | -| transcription languages | `language_codes` on the vendor | `asr.params.language_codes` | - -`custom_vocabulary` biases recognition toward words the model would otherwise mis-hear — product names, jargon, proper nouns. It is omitted from the request entirely when unset. +The registry is keyed first by request category and then by the serialized vendor name: ```python -GeminiSTT(api_key=..., custom_vocabulary=["Agora", "Kubernetes"]) -``` - -`word_timestamp` is also omitted unless you set it explicitly. Gemini does not support enabled word timestamps together with `custom_vocabulary`, so `to_config()` raises `ValueError` if both are requested. Explicit `word_timestamp=False` remains compatible with custom vocabulary. - -```python -GeminiSTT(api_key=..., word_timestamp=True) -``` - -## The vendor class is not the whole wire shape - -A vendor class emitting the right dict is not proof of what ships, because `Agent.to_properties` **also writes into the vendor config** after the vendor is done with it — using the Agora schema spellings, which are correct for every GA provider but need not match what a preview route reads. - -Every field the shared builder injects is listed below. Each one is a candidate for a silent mismatch on a preview route: - -| Category | Field | Written from | When | -| -------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------- | -| `mllm` | `greeting_message` | agent-level `greeting` | only when the vendor left it unset (`setdefault`) | -| `mllm` | `failure_message` | agent-level `failure_message` | only when the vendor left it unset (`setdefault`) | -| `mllm` | `enable` | `with_mllm()` | always | -| `asr` | `language` | turn detection `language` | **always — overwrites any vendor value** | -| `llm` | `system_messages`, `greeting_message`, `greeting_configs`, `failure_message`, `max_history` | agent-level `instructions`, `greeting`, `greeting_configs`, `failure_message`, `max_history` | only when the vendor left them unset | - -So when a preview provider documents a field that appears in that table, putting it on the vendor class is not the whole fix. One of three things applies: - -- **The builder always overwrites it** (`asr.language`) — do not expose it on the vendor class at all; it would be an argument the builder silently discards. `GeminiSTT` takes no `language` for exactly this reason, and `extra="forbid"` turns a stale `language=` into an error rather than a no-op. -- **It is an Agora engine field rather than the provider's** (`failure_message`) — leave it in the schema spelling. -- **The preview route spells it differently** — the translation belongs in `preview/client.py`, applied at session start, so it disappears with `agentkit/preview/` at GA rather than leaving a vestigial hook in the shared builder. Nothing in this release needs one, but a future preview family may. - -### Verify against the request body, not the vendor output - -`to_config()` returning the right dict proves nothing about what ships, because the builder runs after it. Both checks are needed: - -1. A unit test on the vendor class, for the fields the vendor owns. -2. An **end-to-end test that starts a session against a mock transport and asserts on the captured request body** — the only check that sees the builder's injections. Every preview vendor has one in `tests/custom/test_preview.py`. - -The manual version is `debug=True`, which logs the fully resolved body. Diff it against the payload the provider documented, key by key. A value sitting under a name the route ignores fails **silently** — no error, no validation complaint, the agent simply never greets. That is the failure mode this whole section exists to catch, and it is invisible to type checking, to pydantic validation, and to any test that stops at the vendor class. - -Wire parity across the three SDKs is a hard requirement, so a change here lands in Python, TypeScript, and Go together, verified by diffing the serialized bodies. - -## Adding a future preview family - -Everything preview-only lives under `agentkit/preview/` so it can be deleted wholesale at GA. To add a family: - -1. Add an attribute to `PreviewFeatures` in `preview/client.py`. The value is what goes in the `agora-feature` header. -2. Add the vendor classes to `preview/vendors.py`, extending the same `BaseSTT` / `BaseMLLM` / `BaseLLM` bases as production vendors so the builder accepts them unchanged. -3. Register the detection keys — for an ASR family, the vendor name in `_PREVIEW_ASR_VENDORS` — so `required_preview_features()` recognises configs that need the new family. This is also what lets the config through the validation bypass described above. -4. Export from `preview/__init__.py`. Preview symbols stay in that subpackage rather than being re-exported from `agentkit/__init__.py`, which is what makes the GA deletion a single-directory change. -5. **Diff the resolved request body against the payload the provider documented**, not the vendor class output — see [The vendor class is not the whole wire shape](#the-vendor-class-is-not-the-whole-wire-shape). -6. Add an end-to-end test that starts a session and asserts on the captured body, alongside the vendor-class unit test. - -If the generated request models do not cover the new provider, rely on the bypass in `_start_properties_from_mapping` rather than editing generated code. Generated files are overwritten on the next Fern run; `.fernignore` protects `src/agora_agent/agentkit/`. - -At GA, delete `preview/` and move the vendor classes into `vendors/stt.py`. - -## Base URL - +_PREVIEW_FEATURES_BY_CATEGORY = { + "asr": {"new_vendor": "new-vendor-feature"}, +} ``` -https://partner.ai.agora.io/preview/api/conversational-ai-agent -``` - -Request paths append to it exactly as they do in production — `POST {base}/v2/projects/{appId}/join`. The service path segment is part of the base: `https://partner.ai.agora.io/preview/api/` alone returns `404 no Route matched with those values`. - -The preview host is a single partner endpoint with no regional replicas. The parent client's regional selection is not mutated; it remains available for GA sessions. - -## Debug output - -`debug=True` prints the resolved start request. The body is passed through `redact_secrets` first, which replaces vendor API keys, the RTC token, and the App ID with `[REDACTED]` while leaving model names, voices, and instructions readable. The same redaction is applied to the httpx request log. - -Empty strings are left visible on purpose: `""` is the signature of an unset environment variable, and hiding it would disguise the exact misconfiguration the debug output exists to surface. -## Related +Detection uses the fully resolved request body rather than the Python class, so hand-written configs and preset +resolution follow the same routing behavior. If Fern's production request union does not contain the preview +vendor yet, the registered config bypasses that generated union while retaining normal validation for production +providers. `None` values are removed before the preview request is sent. -- [Regional Routing](./regional-routing.md) — the production domain pool the preview client bypasses -- [Error Handling](./error-handling.md) — `ApiError` and API error handling +Add routing tests for both synchronous and asynchronous sessions when registering a provider. Tests should verify +the preview base URL, exact feature header, all lifecycle requests, caller-header precedence, and that the original +client remains configured for production. diff --git a/docs/guides/regional-routing.md b/docs/guides/regional-routing.md index a6051bd..ff6e4cc 100644 --- a/docs/guides/regional-routing.md +++ b/docs/guides/regional-routing.md @@ -40,7 +40,7 @@ If you omit `with_stt()`, AgentKit uses `FengmingSTT` by default for `Area.CN` c | Client area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes | |---|---|---|---|---|---| -| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` | +| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GeminiSTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` | | `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` | Global client example: diff --git a/docs/index.md b/docs/index.md index e2c2028..9178fcf 100644 --- a/docs/index.md +++ b/docs/index.md @@ -49,11 +49,11 @@ The Agora Conversational AI Python SDK lets you build voice-powered AI agents on | [MLLM Flow](./guides/mllm-flow.md) | Use a global or CN realtime multimodal model for end-to-end audio | | [Avatars](./guides/avatars.md) | Add a digital avatar with LiveAvatar, Akool, Anam, Generic Avatar, SenseTime, or Spatius | | [Regional Routing](./guides/regional-routing.md) | Route requests to the nearest region | +| [Preview Endpoint](./guides/preview-endpoint.md) | Understand session-scoped routing for preview providers | | [Error Handling](./guides/error-handling.md) | Handle API errors with ApiError | | [Pagination](./guides/pagination.md) | Iterate over paginated list endpoints | | [Advanced](./guides/advanced.md) | Raw response, retries, timeouts, custom httpx client | | [Low-Level API](./guides/low-level-api.md) | Generated REST APIs | -| [Preview Endpoint](./guides/preview-endpoint.md) | Session-scoped preview routing and the `agora-feature` gate header | | [Client Reference](./reference/client.md) | Full `Agora` / `AsyncAgora` API | | [Agent Reference](./reference/agent.md) | Full `Agent` builder API | | [Session Reference](./reference/session.md) | Full `AgentSession` / `AsyncAgentSession` API | diff --git a/docs/reference/vendors.md b/docs/reference/vendors.md index 38fee0e..fd3e501 100644 --- a/docs/reference/vendors.md +++ b/docs/reference/vendors.md @@ -20,7 +20,7 @@ Construct vendors directly from `agora_agent`, then bind a client with `Agent(cl | Area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes | |---|---|---|---|---|---| -| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` | +| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GeminiSTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` | | `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` | Global example: @@ -541,6 +541,36 @@ For `nova-2` and `nova-3`, omit `api_key` to use Agora-managed credentials. For | `model` | `str` | No | `None` | Recognition model | | `additional_params` | `Dict[str, Any]` | No | `None` | Additional parameters | +### `GeminiSTT` + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `api_key` | `str` | Yes | — | Google Gemini API key | +| `model` | `str` | No | `gemini-3.5-transcribe-live` | Gemini transcription model | +| `language` | `str` | No | `None` | Language code for speech recognition. Takes precedence over top-level `asr.language`. | +| `language_hints` | `List[str]` | No | `None` | Candidate transcription languages; `None` is omitted and an empty list is sent explicitly | +| `language_codes` | `List[str]` | No | `None` | Deprecated alias for `language_hints`; ignored when `language_hints` is provided | +| `custom_vocabulary` | `List[str]` | No | `None` | Words and phrases used to bias recognition | +| `word_timestamp` | `bool` | No | `None` | Include word-level timestamps; incompatible with non-empty `custom_vocabulary` and SMART mode | +| `mode` | `GeminiAsrParamsMode` | No | `None` | `SMART` or `VERBATIM`; `None` and an empty value are validated as VERBATIM and omitted from the request | +| `diarization` | `bool` | No | `None` | Include speaker labels; `None` is omitted and treated as false during validation; true is incompatible with SMART mode | +| `sample_rate` | `int` | No | `16000` | Audio sample rate in Hz | +| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Gemini ASR parameters | + +Construction fails when non-empty `custom_vocabulary` is combined with `word_timestamp=True`, or when SMART mode is combined with `word_timestamp=True` or `diarization=True`. `additional_params` remains an unchecked passthrough dictionary. +Existing imports from `agora_agent.agentkit.preview` remain supported and +resolve to the same production `GeminiSTT` class. + +```python +from agora_agent import GeminiSTT + +stt = GeminiSTT( + api_key="your-google-api-key", + language_hints=["en-US", "es-ES"], + custom_vocabulary=["Agora"], +) +``` + ### `AmazonSTT` | Parameter | Type | Required | Default | Description | diff --git a/src/agora_agent/__init__.py b/src/agora_agent/__init__.py index b132d4b..ca0fe07 100644 --- a/src/agora_agent/__init__.py +++ b/src/agora_agent/__init__.py @@ -45,6 +45,8 @@ ElevenLabsTTS, FishAudioTTS, Gemini, + GeminiSTT, + GeminiSTTModels, GeminiLive, GenericAvatar, GenericTTS, diff --git a/src/agora_agent/agentkit/__init__.py b/src/agora_agent/agentkit/__init__.py index 934dbd9..7f17cc0 100644 --- a/src/agora_agent/agentkit/__init__.py +++ b/src/agora_agent/agentkit/__init__.py @@ -180,6 +180,8 @@ Dify, FishAudioTTS, Gemini, + GeminiSTT, + GeminiSTTModels, GeminiLive, GenericAvatar, GoogleSTT, @@ -436,6 +438,8 @@ "MicrosoftSTT", "MicrosoftCNSTT", "OpenAISTT", + "GeminiSTT", + "GeminiSTTModels", "GoogleSTT", "AmazonSTT", "AssemblyAISTT", diff --git a/src/agora_agent/agentkit/agent.py b/src/agora_agent/agentkit/agent.py index a8dae1c..7097e20 100644 --- a/src/agora_agent/agentkit/agent.py +++ b/src/agora_agent/agentkit/agent.py @@ -202,11 +202,10 @@ class SessionOptions(typing_extensions.TypedDict, total=False): warn: typing.Callable[[str], None] - def _drop_none(value: typing.Any) -> typing.Any: """Recursively remove None-valued mapping entries.""" if isinstance(value, dict): - return {k: _drop_none(v) for k, v in value.items() if v is not None} + return {key: _drop_none(item) for key, item in value.items() if item is not None} if isinstance(value, list): return [_drop_none(item) for item in value] return value @@ -218,16 +217,10 @@ def _start_properties_from_mapping( try: return parse_obj_as(StartAgentsRequestProperties, dict(properties)) except Exception: - # Preview providers are absent from the generated unions by design — the - # schema models what production serves. A config the preview gateway - # does understand is passed through unvalidated rather than rejected. - # Imported lazily: preview.client imports the pool client, which imports - # this module. + # Preview providers may not exist in the production-generated union yet. from .preview.client import required_preview_features if required_preview_features(properties): - # The typed path serializes with exclude_none; strip None here so the - # bypass puts the same bytes on the wire instead of explicit nulls. return typing.cast(StartAgentsRequestProperties, _drop_none(dict(properties))) raise @@ -1051,10 +1044,6 @@ def to_properties( if is_mllm_mode: if self._mllm is not None: mllm_config = dict(self._mllm) - # These are production wire spellings. A route that spells one of them - # differently needs a rename entry in `preview/client.py`, or the value - # lands in a field the provider ignores and fails silently. See - # docs/guides/preview-endpoint.md#the-vendor-class-is-not-the-whole-wire-shape. if self._greeting is not None: mllm_config.setdefault("greeting_message", self._greeting) if self._failure_message is not None: @@ -1123,10 +1112,6 @@ def _resolve_asr_config(self, turn_detection_config: TurnDetectionInput) -> typi if not asr_config: area_scope = getattr(self._client, "area_scope", None) asr_config["vendor"] = "fengming" if area_scope == "cn" else "ares" - # Unconditional: turn detection is the single source of truth for the - # interaction language, so a vendor-level ``language`` would be silently - # discarded here. Do not add one to a vendor class — see - # docs/guides/preview-endpoint.md#the-vendor-class-is-not-the-whole-wire-shape. asr_config["language"] = self._field_value(turn_detection_config, "language") return asr_config diff --git a/src/agora_agent/agentkit/agent_session.py b/src/agora_agent/agentkit/agent_session.py index b5756e1..18a2bac 100644 --- a/src/agora_agent/agentkit/agent_session.py +++ b/src/agora_agent/agentkit/agent_session.py @@ -185,9 +185,7 @@ def _require_agent_management(self) -> typing.Any: def _bind_session_clients(self, features: typing.Sequence[str]) -> None: """Pin this session to production or preview without mutating its client.""" if features: - self._agents, self._agent_management = create_preview_session_clients( - self._client, features - ) + self._agents, self._agent_management = create_preview_session_clients(self._client, features) from .preview.client import PREVIEW_API_BASE_URL self._api_base_url = PREVIEW_API_BASE_URL @@ -195,9 +193,7 @@ def _bind_session_clients(self, features: typing.Sequence[str]) -> None: self._agents = self._client.agents self._agent_management = getattr(self._client, "agent_management", None) self._api_base_url = ( - self._client.get_current_url() - if hasattr(self._client, "get_current_url") - else None + self._client.get_current_url() if hasattr(self._client, "get_current_url") else None ) # ------------------------------------------------------------------ diff --git a/src/agora_agent/agentkit/preview/client.py b/src/agora_agent/agentkit/preview/client.py index 2203c2d..999104a 100644 --- a/src/agora_agent/agentkit/preview/client.py +++ b/src/agora_agent/agentkit/preview/client.py @@ -90,7 +90,9 @@ def create_preview_session_clients( #: ASR vendors served only by the preview endpoint. -_PREVIEW_ASR_VENDORS = frozenset({"gemini"}) +_PREVIEW_FEATURES_BY_CATEGORY: typing.Dict[str, typing.Dict[str, PreviewFeature]] = { + "asr": {}, +} def required_preview_features(properties: typing.Mapping[str, typing.Any]) -> typing.List[str]: @@ -100,11 +102,16 @@ def required_preview_features(properties: typing.Mapping[str, typing.Any]) -> ty hand-written configs are covered too. """ features: typing.List[str] = [] - - asr = properties.get("asr") - if isinstance(asr, dict) and asr.get("vendor") in _PREVIEW_ASR_VENDORS: - features.append(PreviewFeatures.GEMINI_LIVE) - + for category, vendors in _PREVIEW_FEATURES_BY_CATEGORY.items(): + config = properties.get(category) + if not isinstance(config, dict): + continue + vendor = config.get("vendor") + if not isinstance(vendor, str): + continue + feature = vendors.get(vendor) + if feature is not None and feature not in features: + features.append(feature) return features diff --git a/src/agora_agent/agentkit/preview/vendors.py b/src/agora_agent/agentkit/preview/vendors.py index d4ef4b7..b24ffd1 100644 --- a/src/agora_agent/agentkit/preview/vendors.py +++ b/src/agora_agent/agentkit/preview/vendors.py @@ -6,105 +6,6 @@ endpoint automatically. """ -from __future__ import annotations +from ..vendors.stt import GeminiSTT, GeminiSTTModels -from typing import Any, Dict, List, Optional - -from ..vendors.base import BaseSTT -from pydantic import ConfigDict, Field - -#: Rejects ``api_key=""``. ``Field(...)`` alone makes the key required but still -#: accepts the empty string, which would reach the provider as a blank -#: credential; the TypeScript and Go vendors both refuse it at construction. -_ApiKey = Field(..., min_length=1, description="Google API key") - - -class GeminiSTTModels: - """Gemini preview transcription models.""" - - TRANSCRIBE_35_LIVE = "gemini-3.5-transcribe-live" - - -class GeminiSTT(BaseSTT): - """Gemini 3.5 Transcribe ASR vendor (preview). - - Example:: - - agent = Agent(client=client).with_stt( - GeminiSTT(api_key=..., language_codes=["en-US"]) - ) - """ - - model_config = ConfigDict(extra="forbid") - - api_key: str = _ApiKey - model: Optional[str] = Field( - default=None, - description="Model name. Defaults to `gemini-3.5-transcribe-live`.", - ) - language_codes: Optional[List[str]] = Field( - default=None, - description=( - "Languages the model should transcribe, sent as `params.language_codes`. " - "Omitted from the request when unset, which is how the provider sets " - "auto-detect — the SDK does not pin a language the caller never asked for. " - "Pass one code to commit to a language, several to let the model choose " - "between them, or an explicit empty list to request auto-detect outright. " - "This is the only language setting on this vendor. The top-level " - "`asr.language` is supplied by `Agent` from turn detection, as it is for " - "every STT vendor." - ), - ) - custom_vocabulary: Optional[List[str]] = Field( - default=None, - description=( - "Words and phrases to bias recognition toward — product names, jargon, " - "proper nouns the model would otherwise mis-hear." - ), - ) - sample_rate: Optional[int] = Field( - default=None, - description="Audio sample rate in Hz. Defaults to 16000.", - ) - word_timestamp: Optional[bool] = Field( - default=None, - description=( - "Emit per-word timestamps in transcription results. Omitted unless explicitly set; " - "cannot be `true` when `custom_vocabulary` is set." - ), - ) - additional_params: Optional[Dict[str, Any]] = Field( - default=None, - description="Additional vendor-specific parameters.", - ) - - def to_config(self) -> Dict[str, Any]: - model = self.model if self.model is not None else GeminiSTTModels.TRANSCRIBE_35_LIVE - sample_rate = self.sample_rate if self.sample_rate is not None else 16000 - - # additional_params first so that explicit fields always win. - params: Dict[str, Any] = dict(self.additional_params or {}) - params["api_key"] = self.api_key - params["model"] = model - params["sample_rate"] = sample_rate - # Omitted unless the caller asked for it: no language_codes is how the - # provider spells auto-detect, and seeding it from ``language`` would pin - # every request to a language the caller never chose. - if self.language_codes is not None: - params["language_codes"] = list(self.language_codes) - if self.custom_vocabulary is not None: - params["custom_vocabulary"] = list(self.custom_vocabulary) - if self.word_timestamp is not None: - params["word_timestamp"] = self.word_timestamp - if "custom_vocabulary" in params and params.get("word_timestamp") is True: - raise ValueError("custom_vocabulary cannot be used with word_timestamp=true") - - # No top-level `language`: `Agent` sets it from turn detection, - # the same as every other STT vendor. - return {"vendor": "gemini", "params": params} - - -__all__ = [ - "GeminiSTTModels", - "GeminiSTT", -] +__all__ = ["GeminiSTTModels", "GeminiSTT"] diff --git a/src/agora_agent/agentkit/regional_agent.py b/src/agora_agent/agentkit/regional_agent.py index 0a91dd2..8a9b3f1 100644 --- a/src/agora_agent/agentkit/regional_agent.py +++ b/src/agora_agent/agentkit/regional_agent.py @@ -30,6 +30,7 @@ AresSTT, AssemblyAISTT, DeepgramSTT, + GeminiSTT, GoogleSTT, MicrosoftSTT, OpenAISTT, @@ -82,6 +83,7 @@ DeepgramSTT, MicrosoftSTT, OpenAISTT, + GeminiSTT, GoogleSTT, AmazonSTT, AssemblyAISTT, diff --git a/src/agora_agent/agentkit/vendors/__init__.py b/src/agora_agent/agentkit/vendors/__init__.py index c69669d..643ae17 100644 --- a/src/agora_agent/agentkit/vendors/__init__.py +++ b/src/agora_agent/agentkit/vendors/__init__.py @@ -41,6 +41,8 @@ AresSTT, AssemblyAISTT, DeepgramSTT, + GeminiSTT, + GeminiSTTModels, GoogleSTT, MicrosoftSTT, OpenAISTT, @@ -120,6 +122,8 @@ "MicrosoftSTT", "MicrosoftCNSTT", "OpenAISTT", + "GeminiSTT", + "GeminiSTTModels", "GoogleSTT", "AmazonSTT", "AssemblyAISTT", diff --git a/src/agora_agent/agentkit/vendors/catalog.py b/src/agora_agent/agentkit/vendors/catalog.py index 25c99df..6c63d08 100644 --- a/src/agora_agent/agentkit/vendors/catalog.py +++ b/src/agora_agent/agentkit/vendors/catalog.py @@ -22,6 +22,7 @@ AresSTT, AssemblyAISTT, DeepgramSTT, + GeminiSTT, GoogleSTT, MicrosoftSTT, OpenAISTT, @@ -74,6 +75,7 @@ def __init__( "deepgram": DeepgramSTT, "microsoft": MicrosoftSTT, "openai": OpenAISTT, + "gemini": GeminiSTT, "google": GoogleSTT, "amazon": AmazonSTT, "assemblyai": AssemblyAISTT, diff --git a/src/agora_agent/agentkit/vendors/namespaces.py b/src/agora_agent/agentkit/vendors/namespaces.py index 001b374..c5d7c89 100644 --- a/src/agora_agent/agentkit/vendors/namespaces.py +++ b/src/agora_agent/agentkit/vendors/namespaces.py @@ -19,6 +19,7 @@ AresSTT, AssemblyAISTT, DeepgramSTT, + GeminiSTT, GoogleSTT, MicrosoftSTT, OpenAISTT, @@ -53,6 +54,7 @@ class GlobalSTTVendors: deepgram = DeepgramSTT microsoft = MicrosoftSTT openai = OpenAISTT + gemini = GeminiSTT google = GoogleSTT amazon = AmazonSTT assemblyai = AssemblyAISTT diff --git a/src/agora_agent/agentkit/vendors/region.py b/src/agora_agent/agentkit/vendors/region.py index 721f7e3..0e5b473 100644 --- a/src/agora_agent/agentkit/vendors/region.py +++ b/src/agora_agent/agentkit/vendors/region.py @@ -22,6 +22,7 @@ "deepgram", "microsoft", "openai", + "gemini", "google", "amazon", "assemblyai", diff --git a/src/agora_agent/agentkit/vendors/stt.py b/src/agora_agent/agentkit/vendors/stt.py index 736390a..dc95564 100644 --- a/src/agora_agent/agentkit/vendors/stt.py +++ b/src/agora_agent/agentkit/vendors/stt.py @@ -1,6 +1,7 @@ import warnings from typing import Any, Dict, List, Optional +from ...types.gemini_asr_params_mode import GeminiAsrParamsMode from .base import BaseSTT from pydantic import BaseModel, ConfigDict, Field, model_validator @@ -196,6 +197,112 @@ def to_config(self) -> Dict[str, Any]: return config +class GeminiSTTModels: + """Gemini transcription models.""" + + TRANSCRIBE_35_LIVE = "gemini-3.5-transcribe-live" + + +class GeminiSTTOptions(BaseModel): + model_config = ConfigDict(extra="forbid") + + api_key: str = Field(..., min_length=1, description="Google Gemini API key") + model: Optional[str] = Field( + default=None, + description="Google Gemini transcription model", + ) + language: Optional[str] = Field(default=None, description="Language code for speech recognition") + language_hints: Optional[List[str]] = Field( + default=None, + description="Candidate transcription languages", + ) + language_codes: Optional[List[str]] = Field( + default=None, + description="Deprecated alias for language_hints", + deprecated="Use language_hints instead.", + ) + custom_vocabulary: Optional[List[str]] = Field( + default=None, + description="Words and phrases that bias speech recognition", + ) + word_timestamp: Optional[bool] = Field( + default=None, + description="Include word-level timestamps; cannot be true with custom_vocabulary or SMART mode", + ) + mode: Optional[GeminiAsrParamsMode] = Field( + default=None, + description="Transcription mode: SMART or VERBATIM; empty values use VERBATIM validation", + ) + diarization: Optional[bool] = Field( + default=None, + description="Include speaker labels; true cannot be combined with SMART mode", + ) + sample_rate: Optional[int] = Field(default=None, description="Audio sample rate in Hz") + additional_params: Optional[Dict[str, Any]] = Field(default=None) + + @model_validator(mode="before") + @classmethod + def _warn_deprecated_language_codes(cls, values: Any) -> Any: + if isinstance(values, dict) and "language_codes" in values: + warnings.warn( + "GeminiSTT.language_codes is deprecated; use language_hints instead.", + DeprecationWarning, + stacklevel=2, + ) + return values + + @model_validator(mode="after") + def _validate_transcription_options(self) -> "GeminiSTTOptions": + word_timestamp = self.word_timestamp is True + if self.custom_vocabulary and word_timestamp: + raise ValueError("custom_vocabulary cannot be used with word_timestamp=true") + + mode = self.mode + if mode is None or mode == "": + mode = "VERBATIM" + if not isinstance(mode, str) or mode not in {"SMART", "VERBATIM"}: + raise ValueError("GeminiSTT mode must be SMART or VERBATIM") + if mode == "SMART" and word_timestamp: + raise ValueError("GeminiSTT mode=SMART cannot be used with word_timestamp=true") + if mode == "SMART" and self.diarization is True: + raise ValueError("GeminiSTT mode=SMART cannot be used with diarization=true") + return self + + +class GeminiSTT(GeminiSTTOptions, BaseSTT): + def to_config(self) -> Dict[str, Any]: + model = self.model if self.model is not None else GeminiSTTModels.TRANSCRIBE_35_LIVE + sample_rate = self.sample_rate if self.sample_rate is not None else 16000 + params: Dict[str, Any] = dict(self.additional_params or {}) + params.update( + { + "api_key": self.api_key, + "model": model, + "sample_rate": sample_rate, + } + ) + if self.language is not None: + params["language"] = self.language + language_codes = self.__dict__.get("language_codes") + if language_codes is not None: + params["language_hints"] = list(language_codes) + if self.language_hints is not None: + params["language_hints"] = list(self.language_hints) + if self.custom_vocabulary is not None: + params["custom_vocabulary"] = list(self.custom_vocabulary) + if self.word_timestamp is not None: + params["word_timestamp"] = self.word_timestamp + if self.mode: + params["mode"] = self.mode + if self.diarization is not None: + params["diarization"] = self.diarization + + return { + "vendor": "gemini", + "params": params, + } + + class AmazonSTTOptions(BaseModel): model_config = ConfigDict(extra="forbid") diff --git a/tests/custom/test_debug.py b/tests/custom/test_debug.py new file mode 100644 index 0000000..aa91d23 --- /dev/null +++ b/tests/custom/test_debug.py @@ -0,0 +1,113 @@ +import json +from typing import List + +import httpx + +from agora_agent import Agent, Agora, Area, Gemini, GeminiSTT, GoogleTTS +from agora_agent.agentkit.debug import REDACTED, redact_secrets + +API_KEY = "test-google-api-key" +APP_ID = "0" * 32 +APP_CERTIFICATE = "1" * 32 + + +class _Recorder(httpx.MockTransport): + def __init__(self) -> None: + self.requests: List[httpx.Request] = [] + super().__init__(self._handle) + + def _handle(self, request: httpx.Request) -> httpx.Response: + self.requests.append(request) + return httpx.Response(200, json={"agent_id": "agent-1"}) + + +def test_redact_secrets_walks_nested_structures() -> None: + redacted = redact_secrets( + { + "appid": APP_ID, + "properties": { + "token": "007eJxTYKhsrH10", + "asr": {"vendor": "gemini", "params": {"api_key": API_KEY, "model": "m"}}, + "mcp_servers": [{"name": "a", "headers": {"authorization": "Bearer x"}}], + }, + } + ) + + assert redacted == { + "appid": REDACTED, + "properties": { + "token": REDACTED, + "asr": {"vendor": "gemini", "params": {"api_key": REDACTED, "model": "m"}}, + "mcp_servers": [{"name": "a", "headers": {"authorization": REDACTED}}], + }, + } + + +def test_redact_secrets_covers_query_keys_and_google_credentials() -> None: + redacted = redact_secrets( + { + "llm": { + "url": ( + "https://generativelanguage.googleapis.com/v1beta/models/" + f"gemini-2.0-flash:streamGenerateContent?alt=sse&key={API_KEY}" + ), + }, + "tts": {"params": {"credentials": API_KEY}}, + } + ) + + assert API_KEY not in json.dumps(redacted) + + +def test_redact_secrets_leaves_empty_values_visible() -> None: + assert redact_secrets({"params": {"api_key": ""}}) == {"params": {"api_key": ""}} + + +def test_redact_secrets_does_not_mutate_input() -> None: + original = {"properties": {"asr": {"params": {"api_key": API_KEY}}}} + redact_secrets(original) + + assert original["properties"]["asr"]["params"]["api_key"] == API_KEY + + +def test_debug_output_redacts_credentials_without_changing_request(capsys) -> None: + recorder = _Recorder() + client = Agora( + area=Area.US, + app_id=APP_ID, + app_certificate=APP_CERTIFICATE, + httpx_client=httpx.Client(transport=recorder), + ) + agent = ( + Agent(client=client) + .with_stt( + GeminiSTT( + api_key=API_KEY, + model="gemini-3.7-transcribe-live", + language="en-US", + word_timestamp=True, + ) + ) + .with_llm(Gemini(api_key=API_KEY, model="gemini-2.0-flash")) + .with_tts( + GoogleTTS( + key=API_KEY, + voice_name="en-US-Chirp3-HD-Charon", + language_code="en-US", + ) + ) + ) + + agent.create_session( + channel="debug-channel", + agent_uid="1", + remote_uids=["100"], + debug=True, + ).start() + + output = capsys.readouterr().out + assert API_KEY not in output + assert "gemini-3.7-transcribe-live" in output + + sent = json.loads(recorder.requests[0].content) + assert sent["properties"]["asr"]["params"]["api_key"] == API_KEY diff --git a/tests/custom/test_gemini_stt.py b/tests/custom/test_gemini_stt.py new file mode 100644 index 0000000..c4367d5 --- /dev/null +++ b/tests/custom/test_gemini_stt.py @@ -0,0 +1,355 @@ +import json +from typing import List + +import httpx +import pytest +from pydantic import ValidationError + +import agora_agent +from agora_agent import Agent, Agora, Area, Gemini, GeminiSTT, GeminiSTTModels, GoogleTTS +from agora_agent.agentkit.vendors.catalog import GLOBAL_VENDOR_NAMESPACE +from agora_agent.agentkit.vendors.namespaces import GlobalSTTVendors +from agora_agent.agentkit.vendors.region import GLOBAL_ASR_VENDORS + +API_KEY = "test-google-api-key" +MODEL = "gemini-3.7-transcribe-live" +APP_ID = "0" * 32 +APP_CERTIFICATE = "1" * 32 + + +class _Recorder(httpx.MockTransport): + def __init__(self) -> None: + self.requests: List[httpx.Request] = [] + super().__init__(self._handle) + + def _handle(self, request: httpx.Request) -> httpx.Response: + self.requests.append(request) + return httpx.Response(200, json={"agent_id": "agent-1"}) + + +def _gemini_stt(**kwargs) -> GeminiSTT: + options = { + "api_key": API_KEY, + "model": MODEL, + "language": "en-US", + "language_hints": ["en-US", "es-ES"], + "custom_vocabulary": ["Agora"], + "word_timestamp": False, + } + options.update(kwargs) + return GeminiSTT(**options) + + +def _complete_agent(client: Agora) -> Agent: + return ( + Agent(client=client) + .with_stt(_gemini_stt(mode="VERBATIM", diarization=True)) + .with_llm(Gemini(api_key=API_KEY, model="gemini-2.0-flash")) + .with_tts( + GoogleTTS( + key=API_KEY, + voice_name="en-US-Chirp3-HD-Charon", + language_code="en-US", + ) + ) + ) + + +def test_gemini_stt_serializes_fern_schema() -> None: + assert _gemini_stt().to_config() == { + "vendor": "gemini", + "params": { + "api_key": API_KEY, + "model": MODEL, + "sample_rate": 16000, + "language": "en-US", + "language_hints": ["en-US", "es-ES"], + "custom_vocabulary": ["Agora"], + "word_timestamp": False, + }, + } + + +def test_gemini_stt_serializes_optional_sample_rate() -> None: + config = _gemini_stt( + sample_rate=24000, + additional_params={"provider_option": "kept", "model": "overridden"}, + ).to_config() + + assert config["params"] == { + "provider_option": "kept", + "api_key": API_KEY, + "model": MODEL, + "sample_rate": 24000, + "language": "en-US", + "language_hints": ["en-US", "es-ES"], + "custom_vocabulary": ["Agora"], + "word_timestamp": False, + } + + +@pytest.mark.parametrize("options", [{}, {"api_key": ""}]) +def test_gemini_stt_requires_non_empty_api_key(options) -> None: + with pytest.raises(ValidationError): + GeminiSTT(**options) + + +def test_gemini_stt_preserves_preview_defaults() -> None: + config = GeminiSTT(api_key=API_KEY, model=None, sample_rate=None).to_config() + + assert config == { + "vendor": "gemini", + "params": { + "api_key": API_KEY, + "model": GeminiSTTModels.TRANSCRIBE_35_LIVE, + "sample_rate": 16000, + }, + } + + +def test_gemini_stt_preserves_explicit_false_word_timestamp() -> None: + config = GeminiSTT( + api_key=API_KEY, + model=MODEL, + word_timestamp=False, + ).to_config() + + assert config["params"]["word_timestamp"] is False + + +@pytest.mark.parametrize( + ("field", "wire_key"), + [ + ("language_hints", "language_hints"), + ("custom_vocabulary", "custom_vocabulary"), + ], +) +def test_gemini_stt_preserves_optional_list_semantics(field: str, wire_key: str) -> None: + omitted = GeminiSTT(api_key=API_KEY).to_config() + explicit_empty = GeminiSTT(api_key=API_KEY, **{field: []}).to_config() + + assert wire_key not in omitted["params"] + assert explicit_empty["params"][wire_key] == [] + + +def test_gemini_stt_maps_preview_parameters_to_production_extension() -> None: + config = GeminiSTT( + api_key=API_KEY, + language_hints=["en-US", "es-ES"], + custom_vocabulary=["Agora", "ConvoAI"], + ).to_config() + + assert config["params"]["language_hints"] == ["en-US", "es-ES"] + assert config["params"]["custom_vocabulary"] == ["Agora", "ConvoAI"] + + +def test_gemini_stt_deprecates_language_codes() -> None: + with pytest.warns(DeprecationWarning, match="use language_hints instead"): + config = GeminiSTT(api_key=API_KEY, language_codes=["en-US", "es-ES"]).to_config() + + assert config["params"]["language_hints"] == ["en-US", "es-ES"] + + +def test_gemini_stt_language_hints_take_priority_over_language_codes() -> None: + with pytest.warns(DeprecationWarning, match="use language_hints instead"): + config = GeminiSTT( + api_key=API_KEY, + language_codes=["en-US"], + language_hints=[], + ).to_config() + + assert config["params"]["language_hints"] == [] + + +@pytest.mark.parametrize( + ("options", "message"), + [ + ( + {"custom_vocabulary": ["Agora"], "word_timestamp": True}, + "custom_vocabulary cannot be used with word_timestamp=true", + ), + ( + {"mode": "SMART", "word_timestamp": True}, + "GeminiSTT mode=SMART cannot be used with word_timestamp=true", + ), + ( + {"mode": "SMART", "diarization": True}, + "GeminiSTT mode=SMART cannot be used with diarization=true", + ), + ], +) +def test_gemini_stt_rejects_invalid_parameter_combinations(options, message) -> None: + with pytest.raises(ValidationError, match=message): + GeminiSTT(api_key=API_KEY, **options) + + +@pytest.mark.parametrize( + "options", + [ + {"mode": "SMART", "custom_vocabulary": ["Agora"]}, + {"mode": "VERBATIM", "word_timestamp": True, "diarization": True}, + {"mode": None, "word_timestamp": True, "diarization": True}, + {"mode": "", "word_timestamp": True, "diarization": True}, + {"custom_vocabulary": [], "word_timestamp": True}, + ], +) +def test_gemini_stt_accepts_valid_parameter_combinations(options) -> None: + GeminiSTT(api_key=API_KEY, **options) + + +@pytest.mark.parametrize("mode", ["INVALID", "smart", 0, False]) +def test_gemini_stt_rejects_invalid_mode_at_construction(mode) -> None: + with pytest.raises(ValidationError, match="GeminiSTT mode must be SMART or VERBATIM"): + GeminiSTT(api_key=API_KEY, mode=mode) + + +def test_gemini_stt_omits_empty_mode_and_nil_diarization() -> None: + config = GeminiSTT(api_key=API_KEY, mode="", diarization=None).to_config() + + assert "mode" not in config["params"] + assert "diarization" not in config["params"] + + +def test_gemini_stt_does_not_validate_additional_params() -> None: + config = GeminiSTT( + api_key=API_KEY, + additional_params={ + "custom_vocabulary": ["Agora"], + "word_timestamp": True, + "mode": "SMART", + "diarization": True, + }, + ).to_config() + + assert config["params"]["custom_vocabulary"] == ["Agora"] + assert config["params"]["word_timestamp"] is True + assert config["params"]["mode"] == "SMART" + assert config["params"]["diarization"] is True + + +def test_gemini_stt_explicit_fields_override_additional_params() -> None: + config = GeminiSTT( + api_key=API_KEY, + model=MODEL, + sample_rate=24000, + language="en-US", + language_hints=["en-US"], + custom_vocabulary=["Agora"], + word_timestamp=False, + mode="SMART", + diarization=False, + additional_params={ + "api_key": "wrong-key", + "model": "wrong-model", + "sample_rate": 8000, + "language": "fr-FR", + "language_hints": ["fr-FR"], + "custom_vocabulary": ["wrong"], + "word_timestamp": True, + "mode": "VERBATIM", + "diarization": True, + "provider_option": "kept", + }, + ).to_config() + + assert config["params"] == { + "api_key": API_KEY, + "model": MODEL, + "sample_rate": 24000, + "language": "en-US", + "language_hints": ["en-US"], + "custom_vocabulary": ["Agora"], + "word_timestamp": False, + "mode": "SMART", + "diarization": False, + "provider_option": "kept", + } + + +def test_gemini_stt_rejects_unknown_fields() -> None: + with pytest.raises(ValidationError): + _gemini_stt(unknown_option=True) + + +def test_gemini_stt_is_available_from_standard_vendor_surfaces() -> None: + assert agora_agent.GeminiSTT is GeminiSTT + assert agora_agent.GeminiSTTModels is GeminiSTTModels + assert "GeminiSTT" in agora_agent.__all__ + assert "GeminiSTTModels" in agora_agent.__all__ + assert "gemini" in GLOBAL_ASR_VENDORS + assert GLOBAL_VENDOR_NAMESPACE.asr["gemini"] is GeminiSTT + assert GlobalSTTVendors.gemini is GeminiSTT + + +def test_gemini_stt_preserves_preview_import_paths_and_constructor_usage() -> None: + from agora_agent.agentkit.preview import GeminiSTT as PreviewGeminiSTT + from agora_agent.agentkit.preview import GeminiSTTModels as PreviewGeminiSTTModels + from agora_agent.agentkit.preview.vendors import GeminiSTT as PreviewVendorGeminiSTT + + assert PreviewGeminiSTT is GeminiSTT + assert PreviewVendorGeminiSTT is GeminiSTT + assert PreviewGeminiSTTModels is GeminiSTTModels + with pytest.warns(DeprecationWarning, match="use language_hints instead"): + config = PreviewGeminiSTT( + api_key=API_KEY, + model=None, + language_codes=["en-US"], + custom_vocabulary=["Agora"], + sample_rate=None, + word_timestamp=False, + additional_params={"provider_option": "kept"}, + ).to_config() + assert config == { + "vendor": "gemini", + "params": { + "provider_option": "kept", + "api_key": API_KEY, + "model": GeminiSTTModels.TRANSCRIBE_35_LIVE, + "sample_rate": 16000, + "language_hints": ["en-US"], + "custom_vocabulary": ["Agora"], + "word_timestamp": False, + }, + } + + +def test_retired_preview_helpers_remain_importable_without_preview_routing() -> None: + from agora_agent.agentkit.preview import PreviewFeatures, required_preview_features + + assert PreviewFeatures.GEMINI_LIVE == "gemini-live" + assert required_preview_features({"asr": {"vendor": "gemini"}}) == [] + + +def test_gemini_stt_uses_production_endpoint_and_fern_request_model() -> None: + recorder = _Recorder() + client = Agora( + area=Area.US, + app_id=APP_ID, + app_certificate=APP_CERTIFICATE, + httpx_client=httpx.Client(transport=recorder), + ) + + _complete_agent(client).create_session( + channel="gemini-channel", + agent_uid="1", + remote_uids=["100"], + ).start() + + request = recorder.requests[0] + body = json.loads(request.content) + assert str(request.url).startswith(client.get_current_url()) + assert body["properties"]["asr"] == { + "vendor": "gemini", + "language": "en-US", + "params": { + "api_key": API_KEY, + "model": MODEL, + "sample_rate": 16000, + "language": "en-US", + "language_hints": ["en-US", "es-ES"], + "custom_vocabulary": ["Agora"], + "word_timestamp": False, + "mode": "VERBATIM", + "diarization": True, + }, + } diff --git a/tests/custom/test_preview.py b/tests/custom/test_preview.py index 2fba9c4..7f416e6 100644 --- a/tests/custom/test_preview.py +++ b/tests/custom/test_preview.py @@ -1,48 +1,57 @@ -"""Preview endpoint tests. +"""Tests for provider-agnostic preview routing infrastructure.""" -Wire-shape expectations are copied from the TypeScript suite so the three SDKs -stay byte-identical on the wire. -""" - -import json +from typing import Dict, List import httpx import pytest +from pydantic import ConfigDict -from agora_agent import Agora, Area, AsyncAgora -from agora_agent.agentkit import Agent -from agora_agent.agentkit.debug import REDACTED, redact_secrets +from agora_agent import Agent, Agora, Area, AsyncAgora, Gemini, GoogleTTS from agora_agent.agentkit.preview import ( PREVIEW_API_BASE_URL, - GeminiSTT, + PREVIEW_FEATURE_HEADER, + create_preview_session_clients, required_preview_features, ) -from agora_agent.agentkit.vendors import Gemini, GoogleTTS +from agora_agent.agentkit.preview import client as preview_client +from agora_agent.agentkit.vendors import BaseSTT API_KEY = "test-google-api-key" -APP_ID = "test-app-id-0123456789abcdefghij" -APP_CERTIFICATE = "test-app-certificate-01234567890" +APP_ID = "0" * 32 +APP_CERTIFICATE = "1" * 32 +TEST_FEATURE = "future-asr" +TEST_VENDOR = "future_preview" -def _client(transport=None, **kwargs): - httpx_client = httpx.Client(transport=transport) if transport is not None else None - return Agora( - area=Area.US, - app_id=APP_ID, - app_certificate=APP_CERTIFICATE, - httpx_client=httpx_client, - **kwargs, - ) +class _PreviewSTT(BaseSTT): + model_config = ConfigDict(extra="forbid") + + def to_config(self) -> Dict[str, object]: + return { + "vendor": TEST_VENDOR, + "params": {"preview_option": True, "optional": None}, + } -def _with_preview_asr(agent): - """Complete an agent with the preview ASR plus Gemini LLM and Google TTS. +class _Recorder(httpx.MockTransport): + def __init__(self) -> None: + self.requests: List[httpx.Request] = [] + super().__init__(self._handle) - The preview ASR only reaches its provider through the preview endpoint, so - this is what the routing and gating tests need to have configured. - """ + def _handle(self, request: httpx.Request) -> httpx.Response: + self.requests.append(request) + return httpx.Response(200, json={"agent_id": "agent-1"}) + + +@pytest.fixture +def registered_preview_asr(monkeypatch): + monkeypatch.setitem(preview_client._PREVIEW_FEATURES_BY_CATEGORY["asr"], TEST_VENDOR, TEST_FEATURE) + + +def _complete_preview_agent(client) -> Agent: return ( - agent.with_stt(GeminiSTT(api_key=API_KEY, language_codes=["en-US"])) + Agent(client=client) + .with_stt(_PreviewSTT()) .with_llm(Gemini(api_key=API_KEY, model="gemini-2.0-flash")) .with_tts( GoogleTTS( @@ -54,230 +63,42 @@ def _with_preview_asr(agent): ) -class _Recorder(httpx.MockTransport): - """Captures every outgoing request and answers with a generic success body.""" - - def __init__(self): - self.requests = [] - super().__init__(self._handle) - - def _handle(self, request: httpx.Request) -> httpx.Response: - self.requests.append(request) - return httpx.Response(200, json={"agent_id": "agent-1", "data": {"list": []}}) - - -# --- Vendor wire shapes ----------------------------------------------------- - - -def test_gemini_transcribe_serialises_to_documented_asr_shape(): - config = GeminiSTT(api_key=API_KEY, language_codes=["en-US"]).to_config() - - assert config == { - "vendor": "gemini", - "params": { - "api_key": API_KEY, - "model": "gemini-3.5-transcribe-live", - "sample_rate": 16000, - "language_codes": ["en-US"], - }, - } - - -def test_gemini_transcribe_allows_overrides(): - config = GeminiSTT( - api_key=API_KEY, - model="not-a-real-model", - sample_rate=24000, - word_timestamp=False, - additional_params={"hotwords": ["Agora"]}, - ).to_config() - - assert config == { - "vendor": "gemini", - "params": { - "hotwords": ["Agora"], - "api_key": API_KEY, - "model": "not-a-real-model", - "sample_rate": 24000, - "word_timestamp": False, - }, - } - - -def test_emits_no_top_level_language_of_its_own(): - # Every STT vendor leaves asr.language to the Agent, which derives it from - # the turn detection language. A vendor-level copy would be a no-op the - # builder overwrites, so this one does not offer the option at all. - config = GeminiSTT(api_key=API_KEY).to_config() - - assert "language" not in config - assert "language" not in config["params"] - # Nor does it invent language_codes — absent means auto-detect. - assert "language_codes" not in config["params"] - +def test_gemini_is_not_registered_for_preview_routing() -> None: + assert required_preview_features({"asr": {"vendor": "gemini"}}) == [] -def test_language_is_not_an_accepted_option(): - # extra="forbid", so a stale `language=` argument fails loudly rather than - # being silently dropped. - with pytest.raises(Exception): - GeminiSTT(api_key=API_KEY, language="en-US") - -def test_language_codes_is_sent_verbatim_when_supplied(): - single = GeminiSTT(api_key=API_KEY, language_codes=["es-ES"]).to_config() - assert single["params"]["language_codes"] == ["es-ES"] - - multiple = GeminiSTT(api_key=API_KEY, language_codes=["en-US", "es-ES"]).to_config() - assert multiple["params"]["language_codes"] == ["en-US", "es-ES"] - - -def test_explicit_empty_language_codes_still_reaches_the_wire(): - # `[]` is the caller spelling auto-detect outright; both that and omitting - # the field mean the same thing to the provider. - config = GeminiSTT(api_key=API_KEY, language_codes=[]).to_config() - - assert config["params"]["language_codes"] == [] - - -def test_custom_vocabulary_is_sent_only_when_supplied(): - with_vocab = GeminiSTT(api_key=API_KEY, custom_vocabulary=["Agora", "Kubernetes"]).to_config() - assert with_vocab["params"]["custom_vocabulary"] == ["Agora", "Kubernetes"] - assert "word_timestamp" not in with_vocab["params"] - - without_vocab = GeminiSTT(api_key=API_KEY).to_config() - assert "custom_vocabulary" not in without_vocab["params"] - - -def test_word_timestamp_is_sent_only_when_explicitly_supplied(): - without_timestamp = GeminiSTT(api_key=API_KEY).to_config() - assert "word_timestamp" not in without_timestamp["params"] - - with_timestamp = GeminiSTT(api_key=API_KEY, word_timestamp=True).to_config() - assert with_timestamp["params"]["word_timestamp"] is True - - -def test_custom_vocabulary_rejects_enabled_word_timestamps(): - incompatible_options = [ - {"custom_vocabulary": ["Agora"], "word_timestamp": True}, - {"custom_vocabulary": [], "word_timestamp": True}, - { - "additional_params": { - "custom_vocabulary": ["Agora"], - "word_timestamp": True, - } - }, - ] - - for options in incompatible_options: - with pytest.raises( - ValueError, - match="custom_vocabulary cannot be used with word_timestamp=true", - ): - GeminiSTT(api_key=API_KEY, **options).to_config() - - -def test_custom_vocabulary_allows_explicitly_disabled_word_timestamps(): - config = GeminiSTT( - api_key=API_KEY, - custom_vocabulary=["Agora"], - word_timestamp=False, - ).to_config() - - assert config["params"]["custom_vocabulary"] == ["Agora"] - assert config["params"]["word_timestamp"] is False - - -# --- Routing ---------------------------------------------------------------- - - -def test_sends_requests_to_the_preview_endpoint_with_the_gate_header(): +def test_registered_provider_routes_session_to_preview(registered_preview_asr) -> None: recorder = _Recorder() - client = _client(transport=recorder) - production_url = client.get_current_url() - - ( - _with_preview_asr(Agent(client=client)) - .create_session(channel="preview-channel", agent_uid="1", remote_uids=["100"]) - .start() + client = Agora( + area=Area.US, + app_id=APP_ID, + app_certificate=APP_CERTIFICATE, + headers={PREVIEW_FEATURE_HEADER: "caller-value", "x-custom": "kept"}, + httpx_client=httpx.Client(transport=recorder), ) - - request = recorder.requests[0] - assert str(request.url) == f"{PREVIEW_API_BASE_URL}/v2/projects/{APP_ID}/join" - assert request.headers["agora-feature"] == "gemini-live" - # Exactly one header may carry the preview value — the gateway accepts other - # spellings that are not part of the public contract, and none may ship here. - carriers = [name for name, value in request.headers.items() if "gemini-live" in value] - assert carriers == ["agora-feature"] - assert client.get_current_url() == production_url - assert client._client_wrapper.get_base_url() == production_url - assert "agora-feature" not in client._client_wrapper.get_headers() - - -def test_ga_session_stays_on_production_endpoint(): - recorder = _Recorder() - client = _client(transport=recorder) production_url = client.get_current_url() - Agent(client=client).create_session( - channel="ga-channel", + _complete_preview_agent(client).create_session( + channel="preview-channel", agent_uid="1", remote_uids=["100"], - pipeline_id="pipeline-id", ).start() - assert str(recorder.requests[0].url).startswith(production_url) - assert "agora-feature" not in recorder.requests[0].headers - - -def test_keeps_the_gate_header_on_every_request(): - # A request that loses the header is routed to the production environment, - # where the preview providers do not exist — so each verb must carry it. - recorder = _Recorder() - client = _client(transport=recorder) - - session = _with_preview_asr(Agent(client=client)).create_session( - channel="preview-channel", agent_uid="1", remote_uids=["100"] - ) - session.start() - session.say("hello") - session.interrupt() - session.stop() - - assert len(recorder.requests) >= 4 - for request in recorder.requests: - assert request.headers.get("agora-feature") == "gemini-live", request.url - assert str(request.url).startswith(PREVIEW_API_BASE_URL) - - -def test_stop_agent_remains_production_only(): - recorder = _Recorder() - client = _client(transport=recorder) - production_url = client.get_current_url() - - client.stop_agent("agent-2") - - request = recorder.requests[0] - assert str(request.url).startswith(production_url) - assert "agora-feature" not in request.headers - - -def test_caller_headers_cannot_drop_the_gate(): - recorder = _Recorder() - client = _client(transport=recorder, headers={"agora-feature": "", "x-custom": "kept"}) - - ( - _with_preview_asr(Agent(client=client)) - .create_session(channel="preview-channel", agent_uid="1", remote_uids=["100"]) - .start() - ) - request = recorder.requests[0] - assert request.headers["agora-feature"] == "gemini-live" + assert str(request.url).startswith(PREVIEW_API_BASE_URL) + assert request.headers[PREVIEW_FEATURE_HEADER] == TEST_FEATURE assert request.headers["x-custom"] == "kept" + assert client.get_current_url() == production_url + assert client._client_wrapper.get_base_url() == production_url + custom_headers = client._client_wrapper.get_custom_headers() + assert custom_headers is not None + assert custom_headers[PREVIEW_FEATURE_HEADER] == "caller-value" + assert b'"preview_option":true' in request.content + assert b'"optional"' not in request.content @pytest.mark.asyncio -async def test_async_session_pins_the_preview_host_and_gate(): +async def test_registered_provider_routes_async_session_to_preview(registered_preview_asr) -> None: recorder = _Recorder() client = AsyncAgora( area=Area.US, @@ -285,119 +106,37 @@ async def test_async_session_pins_the_preview_host_and_gate(): app_certificate=APP_CERTIFICATE, httpx_client=httpx.AsyncClient(transport=recorder), ) - session = _with_preview_asr(Agent(client=client)).create_async_session( - channel="preview-channel", agent_uid="1", remote_uids=["100"] - ) - - await session.start() - await session.say("hello") - await session.interrupt() - await session.stop() - - assert len(recorder.requests) == 4 - assert all(str(request.url).startswith(PREVIEW_API_BASE_URL) for request in recorder.requests) - assert all(request.headers["agora-feature"] == "gemini-live" for request in recorder.requests) - assert "agora-feature" not in client._client_wrapper.get_headers() - - -# --- Preview support guard -------------------------------------------------- - -def test_required_preview_features_flags_the_gemini_asr_vendor(): - assert required_preview_features({"asr": {"vendor": "gemini"}}) == ["gemini-live"] - - -def test_required_preview_features_leaves_a_ga_pipeline_alone(): - assert required_preview_features({"asr": {"vendor": "microsoft"}}) == [] - - -# --- Debug redaction -------------------------------------------------------- - - -def test_redact_secrets_walks_nested_structures(): - redacted = redact_secrets( - { - "appid": "81190c52971d4004b7244bdcd93e2f34", - "properties": { - "token": "007eJxTYKhsrH10", - "asr": {"vendor": "gemini", "params": {"api_key": API_KEY, "model": "m"}}, - "mcp_servers": [{"name": "a", "headers": {"authorization": "Bearer x"}}], - }, - } + session = _complete_preview_agent(client).create_async_session( + channel="preview-channel", + agent_uid="1", + remote_uids=["100"], ) + await session.start() - assert redacted == { - "appid": REDACTED, - "properties": { - "token": REDACTED, - "asr": {"vendor": "gemini", "params": {"api_key": REDACTED, "model": "m"}}, - "mcp_servers": [{"name": "a", "headers": {"authorization": REDACTED}}], - }, - } + request = recorder.requests[0] + assert str(request.url).startswith(PREVIEW_API_BASE_URL) + assert request.headers[PREVIEW_FEATURE_HEADER] == TEST_FEATURE -def test_redact_secrets_covers_gemini_url_and_google_tts_credentials(): - redacted = redact_secrets( - { - "llm": { - "url": ( - "https://generativelanguage.googleapis.com/v1beta/models/" - f"gemini-2.0-flash:streamGenerateContent?alt=sse&key={API_KEY}" - ), - }, - "tts": {"params": {"credentials": API_KEY}}, - } +def test_preview_client_factory_supports_sync_and_async_clients() -> None: + sync_client = Agora( + area=Area.US, + app_id=APP_ID, + app_certificate=APP_CERTIFICATE, + httpx_client=httpx.Client(transport=_Recorder()), ) - - assert API_KEY not in json.dumps(redacted) - - -def test_redact_secrets_leaves_empty_values_visible(): - # "" is the signature of an unset env var and must stay diagnosable. - assert redact_secrets({"params": {"api_key": ""}}) == {"params": {"api_key": ""}} - - -def test_redact_secrets_does_not_mutate_input(): - original = {"properties": {"asr": {"params": {"api_key": API_KEY}}}} - redact_secrets(original) - - assert original["properties"]["asr"]["params"]["api_key"] == API_KEY - - -def test_debug_output_never_prints_a_live_credential(capsys): - recorder = _Recorder() - client = _client(transport=recorder) - - ( - Agent(client=client) - .with_stt(GeminiSTT(api_key=API_KEY)) - .with_llm(Gemini(api_key=API_KEY, model="gemini-2.0-flash")) - .with_tts( - GoogleTTS( - key=API_KEY, - voice_name="en-US-Chirp3-HD-Charon", - language_code="en-US", - ) - ) - .create_session(channel="c", agent_uid="1", remote_uids=["100"], debug=True) - .start() + async_client = AsyncAgora( + area=Area.US, + app_id=APP_ID, + app_certificate=APP_CERTIFICATE, + httpx_client=httpx.AsyncClient(transport=_Recorder()), ) - output = capsys.readouterr().out - assert API_KEY not in output - assert "gemini" in output # non-secret config stays readable - - # The request itself still carries the real key. - sent = json.loads(recorder.requests[0].content) - assert sent["properties"]["asr"]["params"]["api_key"] == API_KEY - - -# --- Vendor construction ---------------------------------------------------- - + sync_agents, sync_management = create_preview_session_clients(sync_client, [TEST_FEATURE]) + async_agents, async_management = create_preview_session_clients(async_client, [TEST_FEATURE]) -def test_rejects_an_empty_api_key(): - # Field(...) alone makes the key required but still accepts "", which would - # reach the provider as a blank credential. The TypeScript and Go vendors - # both refuse it at construction, so this one does too. - with pytest.raises(Exception): - GeminiSTT(api_key="") + for generated_client in (sync_agents, sync_management, async_agents, async_management): + wrapper = generated_client._raw_client._client_wrapper + assert wrapper.get_base_url() == PREVIEW_API_BASE_URL + assert wrapper.get_custom_headers()[PREVIEW_FEATURE_HEADER] == TEST_FEATURE