Repository navigation
feat: add opt-in SystemOne query model routing - #1218
Open
wujiaming-ai wants to merge 10 commits into
Open
wujiaming-ai wants to merge 10 commits into
wujiaming-ai wants to merge 10 commits into
Conversation
Route simple requests to manually listed low-cost models while preserving the Agent's configured model as the strong tier. Configuration is opt-in through a query_routing block in config.yaml or the Agent query_routing argument. Added conservative Decision-based classification, capability gates, ordered model selection, session upgrade stickiness, shadow mode, strong-model fallback, multilingual documentation, and unit/E2E coverage.
Exercise the real Responses API path with session pinning, high-stakes routing, shadow mode, and Decision failure fallback. Add exact threshold and context-limit boundary tests.
Resolve one concrete model before ADK preprocessing and preserve that binding through tool loops, nested AgentTools and native sub-agent transfers. Reclassify the next user turn instead of retaining session-level strong-model stickiness. Rebuild Responses chains from complete history when targets or protocols change, validate continuation prefixes, and retain full replay for expired response IDs. Support independent model connections and reject transport overrides or fallbacks that would violate the fixed-turn contract. Keep routing opt-in with shadow mode as the default and document both configuration and compatibility boundaries. Validation: - Post-sync regression: 8638 passed, 57 skipped, 4 expected failures. - All-files pre-commit: lint, format, secret scans and dependency policy passed. - Real SystemOne/model E2E: 8 passed. - Changed executable line coverage against origin/main: 98.40% (798/811). - Documentation type checks and production build passed. Live E2E uses stored Responses continuation with server-side caching disabled; cache-service entitlement and production routing quality/cost calibration remain operator-specific. No default behavior changes for unconfigured Agents.
Validate tool arguments with bounded correction under enforce routing, preserve Responses tool-call ordering across models, and clarify candidate capabilities. Add paired public-benchmark adapters, transport usage accounting, recovery ledgers and native scorer regression coverage. Off and shadow behavior remain unchanged. Verification: 8719-test full regression, 638-test affected suite, 98.67% changed production-line coverage, native adapter checks and real ADK HTTP campaigns; model quality and provider limits remain task-dependent.
Expose immutable DecisionConfig at the shared decision layer and Agent root. Remove nested consumer connection fields, former imports, and consumer-specific connection environment variables. Keep routing and content-safety policy and client state independent. Migrate both documentation languages and callers; add secret-safe rejection and credential-boundary regressions. Extend real HTTP evaluation with matched fixed controls and frozen routing policies for independent calibration.
Preserve cloud-drive create and append timestamps during offline replay so native scoring can verify the executed results exactly. Keep upstream utility/security checks and strict result comparison intact; no provider-specific changes. Verification: regression observed red then green; pinned native suite 15 passed; changed scorer lines 11/11 covered; full regression 8750 passed, 91 skipped, 4 xfailed; pre-commit passed. Evaluation-only change, with no public SDK contract impact.
Default complex reasoning, tool orchestration and long-context thresholds to 0.5. Keep high stakes at 0.05, explicit overrides authoritative, and custom-question defaults unchanged. Routing remains opt-in and shadow mode retains strong-model execution. Update both documentation languages and exercise the new boundaries and isolated overrides. Validation: 478 targeted tests; 8756 regression tests passed, 91 skipped, 4 xfailed; configuration module coverage 100%; pre-commit and docs checks passed. Live HTTP acceptance initially had 18 passes and 2 SystemOne timeout failures; both affected cases passed separate follow-up checks, with original timeout records retained.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Queries with different capability requirements currently use the same execution model. This adds opt-in SystemOne routing: select a strong or configured candidate model once per root user turn, then retain that choice through subsequent LLM calls and nested tools.
Changes
DecisionConfig, root-level YAMLdecision, andAgent(decision=...). Routing and content safety share connection settings while retaining independent policies and client state. Remove former consumer-specific settings and update both documentation languages.0.5; high stakes remains0.05. Explicit overrides take precedence. Routing is opt-in and defaults toshadow.Validation
Tested on 2026-11-12; final revision:
d629698c.GitHub CI: 25 checks passed, 4 release/publish jobs skipped as configured, 0 failed or pending. This includes Python 3.10/3.12, Codex smoke, license headers, pre-commit, secret scanning, CodeQL and release gates.
uv run --extra dev pytest -n 2 -m "not codex_smoke and not piagent_smoke": 8,697 passed, 73 skipped, 4 xfailed. Local test processes include loopback addresses inNO_PROXY/no_proxy; an initial run was interrupted after the host proxy blocked local HTTP requests. The 39 affected-module tests passed after this environment correction.doubao-seed-2-1-pro-260915anddoubao-seed-2-1-lite-260915.uv run --extra dev pre-commit run --all-files: pass, including Ruff and secret scanning..licenserc.yaml,header check: 0 invalid headers.npm --prefix docs run types:checkandnpm --prefix docs run build: pass.Live checks require explicit opt-in and credentials; ordinary CI skips them. They validate execution contracts, not universal routing quality or cost savings.
Compatibility and limits
Unconfigured routing preserves existing model execution. Users of the former content-safety-specific Decision connection settings must migrate to public
decision; no compatibility aliases are retained.The supported scope is ADK text turns. Applications must serialize concurrent turns in one session. Live bidirectional execution and external runtimes are excluded. Provider context limits, invalid tool calls and decision timeouts remain observable.
Cleanup and CI verification traceability
Comparison:
049d066d; PR base:38d3c6e6. Cleanup changes test scope and documentation, with license-only production edits; no additional runtime API or dependency changes.tests/e2e/test_query_routing_http_live.py.licenserc.yaml, failing License-Checkdocs/content/docs/framework/agent/model*.mdx