Skip to content

Send workspace model ids to routes:select unchanged - #1272

Draft
xsh310 wants to merge 4 commits into
databricks:mainfrom
xsh310:raw-route-options
Draft

xsh310 wants to merge 4 commits into
databricks:mainfrom
xsh310:raw-route-options

Conversation

@xsh310

@xsh310 xsh310 commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

What did you change, and why?

Change: ug now sends each workspace model id to routes:select exactly as the workspace lists it (for example system.ai.gpt-5-6-sol or anthropic-aigw-73ea02b2-system.ai.glm-5-2) and accepts only a pick it offered. It no longer rewrites ids into bare router names on the way out, and no longer maps the router's name back.

Why: AI Gateway now does this itself, so the rules live in one place for every client. On both routing paths it unwraps Anthropic gateway aliases, drops the [1m] hint, removes system.ai., lowercases, treats . and - as the same, drops models outside its canonical model list, keeps the first of duplicate spellings, and returns the option the caller sent.

Request, main vs this PR

main This PR
Codex option gpt-5-6-sol system.ai.gpt-5-6-sol
Claude option for a gateway alias glm-5-2 anthropic-aigw-73ea02b2-system.ai.glm-5-2
gpt-5.6-sol and system.ai.gpt-5-6-sol both listed one option both, as listed (the gateway keeps the first)
Router pick mapped back through ug's normalizer must be an offered model, otherwise routing fails open

The model ug launches, the routing notices, and requested_model in the decision log are unchanged, because they already used the workspace id. route_options and router_model in the decision log now hold workspace ids.

Commits

  1. Remove the unused fixed menu Claude routing path. claude_routing.request_routing_decision and route_pre_tool_use (the fixed claude-opus-4-8 and claude-sonnet-5 menu) have no callers, since Claude routing goes through smart_routing.v2. Their tests that still cover live behavior (decision log, task field, Task alias, legacy hook notice) move to the v2 path.
  2. Send workspace model ids unchanged. select_route drops its resolve callback and rejects a pick it did not offer. This removes v2._claude_router_model_id, the Codex GPT respelling _normalize_route_model (resolving its TODO), and the unused resolve_routed_model. routing.route_options_for builds both menus.
  3. Drop RoutingDecision.raw_model and the unused skip_arms. raw_model held the router's own name for its pick, so it now always equals model, and the decision log's router_model reads model. skip_arms matched on that name and has had no caller since Remove the GLM restriction for Codex subagents #273.
  4. Move normalize_model next to its only caller. Its one remaining use is the readable part of the routed Claude agent slug, so it moves from routing.py into v2.py. Its Codex use, _parse_gpt, only fed _model_strength, which nothing has called since it was added in Add opt-in Codex intelligent routing #251, so both are removed.

What stays, and why

Helper Why it stays
v2._canonical_claude_model_id Rewrites databricks-claude-* to system.ai.claude-* before routing. The gateway does not strip databricks-, so without it those models would be dropped as unlisted.
routing.unwrap_anthropic_gateway_model No longer used for the request. It turns the pick into the id for Claude's /model switch on the first prompt (#546), and shortens the model in the legacy hook notice.
v2._normalize_model Builds the readable part of the routed Claude agent slug.
codex_routing.codex_model_id Turns the pick into Codex's bundled catalog slug (gpt-5.6-sol) for the launch.

Depends on

Keep this in draft until these ship to every region:

  • databricks-eng/universe#2879869: the remote path drops a model service the caller cannot use, instead of failing the whole request.
  • databricks-eng/universe#2869774: removes restoreOriginalRouteOptions and enforceCanonicalModelList, making today's PROD-ALL behavior permanent. While the flag exists, turning restoreOriginalRouteOptions off would make the local routers return canonical names, which this PR treats as a pick it did not offer (routing fails open and the session keeps its model).

Notes

  • Open: a model service with more than one destination still fails the whole request on the remote path. That is not in #2879869.
  • ug does not cap the menu. The remote path rejects more than 20 distinct system.ai. services, so menus are expected to stay within that.
  • Older ug versions keep sending bare names, which the gateway still accepts.

How do you know it works?

  • Unit tests (the mocked gateway returns the option it was sent, as AI Gateway does):
    • tests/test_codex_routing.py: the request body carries the system.ai. ids; a pick that was not offered (a canonical name, as an older gateway returns) fails open; the decision log records the workspace id as router_model.
    • tests/test_codex_smart_routing_v2.py: ids pass through as given, and exact duplicates are sent once.
    • tests/test_claude_smart_routing_v2.py: gateway alias ids, and databricks-claude-* ids (rewritten to system.ai. as before), are offered unchanged; the decision log keeps the exact menu and router name; the legacy hook notice still shows the unwrapped model.
    • tests/test_claude_routing.py: the spawn tool detector, canary, and decision reconciliation tests that remain for that module.
    • Full unit suite on 9b56b29c: 5558 passed, 7 skipped, 4 failed. The 4 failures are tests/test_agents_init.py::TestInstallToolBinary::test_required_update_*, which fail the same way on clean main (ad128219) on my devbox. I left out tests/test_e2e_user_agent.py, which launches the local claude binary and also fails on clean main here. ruff and ty pass.
  • tests/e2e_cuj/test_cuj4_smart_routing.py compares offered and selected models through canonical_model, which accepts raw ids, so it needs no change.
  • Not done yet: a live run. Local router workspaces should already accept these requests, since both gateway flags are at PROD-ALL. The remote path (the Isaac workspace) needs #2879869 deployed first. Plan: a LiteSwap run on the remote path with the raw Codex catalog menu, the raw Claude menu with anthropic-aigw- ids, and one stale id, one forbidden id, and one service with several destinations.

This pull request and its description were written by Isaac.

xsh310 and others added 2 commits October 23, 2026 22:04
claude_routing.request_routing_decision and route_pre_tool_use offered a fixed
two model Claude menu, but no ug code calls them: Claude subagent and first
prompt routing go through smart_routing.v2. Keep the spawn tool detector, notice
config, and artifact helpers, and move the decision log, task field, Task alias,
and legacy notice checks onto the v2 path.

Co-authored-by: Isaac <no-reply@databricks.com>
AI Gateway now canonicalizes route option models itself (the system.ai. prefix,
Anthropic gateway aliases, the [1m] hint, case, and . vs - spellings), drops
models it does not support, collapses duplicate spellings, and returns the option
the caller sent. So ug offers each workspace model id as is and accepts only a
pick it offered, instead of normalizing names on the way out and mapping the
router's canonical name back.

This removes the Claude router id normalizer, the Codex GPT respelling (resolving
its TODO), and the unused resolve_routed_model.

Co-authored-by: Isaac <no-reply@databricks.com>
Comment thread src/ucode/smart_routing/routing.py Outdated
ROUTER_NAME_ENV_VAR = "SMART_ROUTER_NAME"
ROUTING_PATH = "/ai-gateway/routing/v1/routes:select"
REQUEST_TIMEOUT_S = 30.0
# The remote Smart Router rejects a request with more distinct model services.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not worry about this for now

Comment thread src/ucode/smart_routing/routing.py Outdated
@@ -185,10 +185,10 @@


def normalize_model(model: str) -> str:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function still needed?

return tail.lower()


def unwrap_anthropic_gateway_model(model: str) -> str:

@xsh310 xsh310 Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still needed?

xsh310 and others added 2 commits October 24, 2026 05:11
raw_model held the router's own name for its pick, which differed from the
workspace id only because ug sent normalized names. It now always equals model,
so the decision log's router_model reads model directly. skip_arms, the only
other reader, matched on that router name and has had no caller since databricks#273.

Co-authored-by: Isaac <no-reply@databricks.com>
Routing no longer normalizes model names. The one remaining use is the readable
part of the routed Claude agent slug, so the helper moves into v2.py. Its Codex
use, _parse_gpt, only fed _model_strength, which has had no caller since it was
added in databricks#251, so both are removed.

Co-authored-by: Isaac <no-reply@databricks.com>
@xsh310
xsh310 force-pushed the raw-route-options branch from 22301e0 to 9b56b29 Compare October 24, 2026 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant