fix(translation): return HTTP 502 when Responses generation fails

[elyasmnvidian]

{
  "id": "resp_failed",
  "object": "response",
  "model": "failed",
  "status": "failed",
  "error": {
    "code": "server_error",
    "message": "deterministic upstream failure"
  },
  "output": [],
  "usage": null
}

Request flow

An application sends a request to Switchyard. Switchyard forwards it to a model provider using the OpenAI Responses API, then returns the answer in the format the application requested: Chat Completions, Anthropic Messages, or Responses.

This PR covers a non-streaming response: the provider sends one JSON document with the generation status, output, and token usage. Switchyard reads the whole document before converting it; the code calls this a buffered response. A streaming response sends events as the model generates its answer and uses separate handling.

Problem

The JSON above is a response from the model provider to Switchyard. The provider sends HTTP 200, but the document says status: "failed" and contains no generated answer (output: []). HTTP 200 alone does not mean that generation succeeded.

Switchyard converts this failed response into an empty successful answer for applications using Chat Completions or Anthropic Messages. Applications using Responses receive status: "failed" in the JSON but still get HTTP 200. Switchyard also counts the failed model call as a success and does not try another model.

The result is an application receiving an empty answer without an HTTP error:

/v1/chat/completions: HTTP 200, content="", finish_reason="stop"
/v1/messages:         HTTP 200, text="", stop_reason="end_turn"
/v1/responses:        HTTP 200, status="failed"

Fix

OpenAiResponsesCodec::decode_response now checks the provider's generation status before converting the answer. It treats status: "failed" as a provider failure, which the HTTP client reports as HTTP 502 (Bad Gateway).

Switchyard counts the failed model call as an error and tries another model when the route supports fallback. If this failure reaches the application, Switchyard returns HTTP 502 on Chat Completions, Anthropic Messages, and Responses.

Applications using an OpenAI API receive the provider's message and nonempty string error code. Applications using Anthropic Messages receive type: "api_error" and the provider's message. The shared HTTP error mapper also preserves codes from ordinary upstream HTTP errors and uses upstream_error when the code is missing, empty, or not a string.

The HTTP client uses redact_forwarded_headers to remove echoed caller credentials before returning or logging the error. A failed status still counts as a provider failure when the error details are missing or malformed.

Verification

The local provider returns the JSON above. Switchyard uses this configuration:

schema_version = 1

[llm_clients.local]
format = "openai_responses"
base_url = "http://127.0.0.1:28081/v1"
max_retries = 0

[targets.failed]
id = "failed"
llm_client = "local"

[routes.failed]
id = "demo/failed"
type = "passthrough"
target = "failed"

/v1/chat/completions: HTTP 502, code="server_error"
/v1/messages:         HTTP 502, type="api_error"
/v1/responses:        HTTP 502, code="server_error"

Every response includes message: "deterministic upstream failure". The same three requests with a completed provider response return HTTP 200 and PONG, with 12 input tokens and 6 output tokens each.

Requests to a real Responses provider using openai/openai/gpt-5.6-sol also return HTTP 200 and PONG through all three endpoints, with 12 input tokens and 6 output tokens each. /v1/stats reports nine requests and three errors across the failed, completed, and real-provider cases.

Compatibility: When this failure reaches an application using Responses, the application now receives HTTP 502 instead of an HTTP-200 body with status: "failed". Successful and incomplete Responses keep their existing behavior.

Tests

cargo test -p switchyard-server --test server responses
cargo test -p switchyard-translation --test response_translation failed_responses_return_upstream_failure_with_provider_message

The HTTP tests check failed Responses, error counts, and fallback on all three public APIs. Chat Completions tests also cover missing or malformed error details, invalid or empty codes, and credential redaction. Translation tests check UpstreamFailure with the provider's message and retain coverage for incomplete responses.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6bf3b776-8895-4ee3-97ce-a6397c86202d

📥 Commits

Reviewing files that changed from the base of the PR and between fe76419 and f580610.

📒 Files selected for processing (5)

• CHANGELOG.md
• crates/libsy-llm-client/src/client.rs
• crates/switchyard-translation/src/codecs/responses/buffered.rs
• crates/switchyard-translation/src/error.rs
• crates/switchyard-translation/tests/response_translation.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

The translation layer now rejects failed Responses payloads as upstream failures. The client maps these failures to HTTP 502 responses while preserving provider error details. Tests cover translation targets and client integration behavior.

Changes

Failed Responses handling

Layer / File(s)	Summary
Upstream failure contract crates/switchyard-translation/src/error.rs	Adds TranslationError::UpstreamFailure with the provider error object and a stable error kind.
Failed Responses decoding crates/switchyard-translation/src/codecs/responses/buffered.rs, crates/switchyard-translation/tests/response_translation.rs	Rejects Responses payloads with status: "failed", preserves the provider error, and tests all supported target formats.
HTTP error mapping and integration crates/libsy-llm-client/src/client.rs, CHANGELOG.md	Maps upstream failures to HTTP 502 responses, preserves the error body, adds integration coverage, and documents fallback behavior.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to f5806

The failed-response handling change has no identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 4 files. (1 skipped: 1 u…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: returning HTTP 502 when an OpenAI Responses generation fails.

---

_{A rabbit found a failed reply
And stopped the empty success by
Passing errors clear and bright
To gateways set at five-oh-two
Fallback paths then hopped anew}

_{Comment @coderabbitai help to get the list of available commands.}