Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs-site/assets/kagent-docs/snippets/setup/responses-api-files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
| What the message includes | What the runtime sends |
| ------------------------- | ---------------------- |
| An `http://` or `https://` link with an `image/` media type | An `input_image` part with the link. |
| An `http://` or `https://` link with any other media type, such as a PDF | An `input_file` part with the link. |
| Inline image bytes | An `input_image` part with the bytes as a `data:` URL. |
| Inline bytes of a PDF, `.docx`, `.xlsx`, or `.pptx` file | An `input_file` part with the bytes as a `data:` URL. The bytes must match the media type. A file whose bytes do not match, such as a text file labeled `application/pdf`, reaches the model as a note that the file could not be read. When the inline PDF and Office files in the Session total more than 50 MB, the runtime sends the newest files and replaces each older file with a note. |
| Inline bytes of any other type | The file's text, or a note when the file cannot be read as text. |

The runtime refuses a link whose scheme is not `http` or `https`. When the newest user message includes such a link, the turn fails with `cannot send link "<uri>" with media type "<type>": this provider integration can only pass http and https links`. A link in an older message does not fail the turn, because the runtime sends links from older messages as bare URLs. A model with a fetch tool can still retrieve the file.

When a link or an inline PDF or Office file has a display name, the runtime adds a text part that reads `File: <name>` immediately before the file part. An inline image does not get this text part. For the file types and sizes that the UI accepts, see [Attach files to a message]({{< link path="observability/launch-ui#attach-files-to-a-message" >}}).
10 changes: 8 additions & 2 deletions docs-site/content/kagent/1.x/operations/upgrade-from-0x.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,11 +163,17 @@ The export splits into three groups: resources that carry forward with small cha

### Resources that carry forward

ModelConfig, ModelProviderConfig, and RemoteMCPServer keep every field that `v1alpha2` sets except one. `v1alpha3` renames the ModelConfig field `apiKeyPassthrough` to `tokenPassthrough`. The field's behavior does not change. `v1alpha3` also adds fields that 0.10.x did not serve, so these resources need no other change to their `spec`.
ModelConfig, ModelProviderConfig, and RemoteMCPServer keep most of the fields that `v1alpha2` sets. `v1alpha3` renames the ModelConfig field `apiKeyPassthrough` to `tokenPassthrough`, and the field's behavior does not change. `v1alpha3` removes the following ModelConfig fields. When a resource sets any of them, the API server rejects the resource and reports the field as unknown.

- `spec.openAI.tokenExchange`
- `spec.tls.caCertSecretRef` and `spec.tls.caCertSecretKey`
- `spec.tls.disableSystemCAs`

kagent 1.0 cannot pass a mounted credential or certificate file to an agent. In place of `tokenExchange`, use an endpoint that accepts a static API key. For what to use in place of a custom certificate authority bundle, see [Credentials that kagent rejects]({{< link path="setup/model-providers/about-model-providers#credentials-that-kagent-rejects" >}}).

1. Recreate the Secrets that your ModelConfigs name before you apply them, or the ModelConfigs resolve to nothing.

2. Convert these resources to `v1alpha3`. The export also holds Agents and other kinds that `v1alpha3` does not serve, so the command keeps only ModelConfig, ModelProviderConfig, and RemoteMCPServer resources. The command sets each `apiVersion`, renames the `apiKeyPassthrough` field, and removes the metadata that the 0.10.x API server set.
2. Convert these resources to `v1alpha3`. The export also holds Agents and other kinds that `v1alpha3` does not serve, so the command keeps only ModelConfig, ModelProviderConfig, and RemoteMCPServer resources. The command sets each `apiVersion`, renames the `apiKeyPassthrough` field, and removes the metadata that the 0.10.x API server set. The command does not delete the ModelConfig fields that `v1alpha3` removed, so delete them from the converted file before you apply it. Otherwise, the apply fails.
```bash
yq '.items |= map(select(.kind == "ModelConfig" or .kind == "ModelProviderConfig" or .kind == "RemoteMCPServer"))
| .items[].apiVersion = "api.kagent.dev/v1alpha3"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ Header injection accepts one shape of credential, a static string. A credential
| `AnthropicVertexAI` and `GeminiVertexAI` | A Google service account key is signed locally to obtain a token, and the `kagent` and `byo` harnesses also mount it as a file. | `Anthropic` or `Bedrock` for Claude models, and `Gemini` for Gemini models. See [Google Vertex AI]({{< link path="setup/model-providers/google-vertexai" >}}). |
| `SAPAICore` | OAuth2 client credentials are exchanged for a token before any request. | A provider that authenticates with an API key. See [SAP AI Core]({{< link path="setup/model-providers/sap-ai-core" >}}). |
| A `credentialRef` in a Harness `spec.env` entry | An arbitrary variable does not name a destination or a header to bind it to. | A ModelConfig or a RemoteMCPServer, each of which carries a destination. See [Agent harness]({{< link path="agents/agent-harness#configure-a-harness" >}}). |
| `openAI.tokenExchange` | The block reads a mounted service account file to acquire a token. | An endpoint that accepts a static API key. See [OpenAI]({{< link path="setup/model-providers/openai" >}}). |
| A custom certificate authority bundle | The bundle is mounted as a file, and `tls` defines no field for one. The API server rejects `caCertSecretRef` and `caCertSecretKey` as unknown fields. | An endpoint whose certificate chains to a public CA. Setting `tls.disableVerify: true` skips certificate verification entirely and belongs only in a test environment. |

A rejected credential reports one of two messages. A credential that cannot be injected reports `cannot use gateway header injection`, and one that needs a mounted file reports `ModelConfig requires volume mounts unsupported by Substrate ActorTemplate`.
Expand Down
14 changes: 2 additions & 12 deletions docs-site/content/kagent/1.x/setup/model-providers/azure-openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,21 +94,11 @@ spec:

## Send a file to the model

The `responses` format sends a file to Azure as a link or as inline bytes. Azure fetches a link only on the turn that sends it. In later turns, the runtime replaces the link with the bare URL as text.
In the `responses` format, the `kagent` runtime sends a file to Azure as a link or as inline bytes. Azure fetches a link only in the turn that sends it. In later turns, the runtime replaces the link with the bare URL as text.

A link reaches Azure regardless of its media type. Inline bytes reach Azure as a file only for an image, a PDF, or a `.docx`, `.xlsx`, or `.pptx` file. Azure documents file input only for PDF files. A `.docx`, `.xlsx`, or `.pptx` file might fail on Azure, although the runtime sends it.

| What the message includes | What the runtime sends |
| ------------------------ | ---------------------- |
| An `http://` or `https://` link with an `image/` media type | An `input_image` part with the link. |
| An `http://` or `https://` link with any other media type, such as a PDF | An `input_file` part with the link. |
| Inline image bytes | An `input_image` part with the bytes as a `data:` URL. |
| Inline bytes of a PDF, `.docx`, `.xlsx`, or `.pptx` file | An `input_file` part with the bytes as a `data:` URL. The bytes must match the media type. A file that does not, such as a text file labeled `application/pdf`, reaches the model as a note that it could not be read. When the files in the conversation total more than 50 MB, the runtime sends the newest files, and a note replaces each older file. |
| Inline bytes of any other type | The file's text, or a note when the file cannot be read as text. |

The runtime refuses a link on any other scheme. When the newest user message includes such a link, the turn fails with `cannot send link "<uri>" with media type "<type>": this provider integration can only pass http and https links`. A link in an older message does not fail the turn, because the runtime sends the links from older messages as bare URLs. A model with a fetch tool can still retrieve the file.

A link attachment with a display name gets an extra text part that reads `File: <name>` that is placed immediately before the file part. Azure rejects a filename beside a file URL, so the name reaches the model as ordinary text.
{{< reuse "kagent-docs/snippets/setup/responses-api-files.md" >}}

## Authentication

Expand Down
9 changes: 7 additions & 2 deletions docs-site/content/kagent/1.x/setup/model-providers/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,13 @@ The `openAI` block takes the following optional settings. For every field, inclu
| `n` | How many completions to request. |
| `timeout` | How long to wait on a request to the provider. |

> [!WARNING]
> The `openAI` block also accepts `tokenExchange`, which acquires a bearer token from a mounted service account file. That configuration mounts a credential file into the agent, which a Harness does not accept. For the full list of configurations that this affects, see [About model providers]({{< link path="setup/model-providers/about-model-providers" >}}).
The `openAI` block in `v1alpha3` does not include the `tokenExchange` field that 0.x accepted. When a ModelConfig sets `tokenExchange`, the API server rejects the ModelConfig and reports the field as unknown.

## Send a file to the model

When `apiFormat` is `responses`, the `kagent` runtime sends supported files through the Responses API without first extracting them as text. The provider fetches a link only in the turn that sends it. In later turns, the runtime replaces the link with the bare URL as text.

{{< reuse "kagent-docs/snippets/setup/responses-api-files.md" >}}

## Use the ModelConfig

Expand Down
Loading