diff --git a/content/docs/ingest-data/ai-agents/coding-assistants/claude-code.mdx b/content/docs/ingest-data/ai-agents/coding-assistants/claude-code.mdx new file mode 100644 index 0000000..e091b2c --- /dev/null +++ b/content/docs/ingest-data/ai-agents/coding-assistants/claude-code.mdx @@ -0,0 +1,126 @@ +--- +title: Claude Code +description: Send Claude Code events, metrics and traces directly to Parseable through OpenTelemetry +--- + +Claude Code exports usage events, metrics and beta traces through OpenTelemetry. Send each signal directly to a separate Parseable dataset to inspect sessions, model requests, token use, tool activity, cost, latency and errors. + +```text +Claude Code + |-- logs ----> Parseable: claudecode-logs + |-- metrics --> Parseable: claudecode-metrics + +-- traces ---> Parseable: claudecode-traces (optional beta) +``` + +Anthropic treats metrics and events as stable telemetry signals. Enhanced telemetry remains in beta and enables distributed traces. See [Monitoring Claude Code usage](https://code.claude.com/docs/en/monitoring-usage) for the current schema and version requirements. + +## Prerequisites + +- Claude Code installed and authenticated +- A Parseable instance reachable from the developer machine +- A Parseable API key with ingest access + +## Enable Claude Code telemetry + +Set separate OTLP endpoints and headers for logs and metrics before starting Claude Code: + +```bash +export PARSEABLE_URL="https://:8000" +export PARSEABLE_API_KEY="" + +export CLAUDE_CODE_ENABLE_TELEMETRY=1 +export OTEL_LOGS_EXPORTER=otlp +export OTEL_METRICS_EXPORTER=otlp + +export OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=http/json +export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT="${PARSEABLE_URL}/v1/logs" +export OTEL_EXPORTER_OTLP_LOGS_HEADERS="X-P-API-Key=${PARSEABLE_API_KEY},X-P-Stream=claudecode-logs,X-P-Log-Source=otel-logs" + +export OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/json +export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT="${PARSEABLE_URL}/v1/metrics" +export OTEL_EXPORTER_OTLP_METRICS_HEADERS="X-P-API-Key=${PARSEABLE_API_KEY},X-P-Stream=claudecode-metrics,X-P-Log-Source=otel-metrics" + +claude +``` + +For a self-hosted deployment that uses basic authentication, replace `X-P-API-Key=${PARSEABLE_API_KEY}` in each header value with `Authorization=Basic `. + +### Enable beta traces + +Add the trace exporter configuration before starting Claude Code: + +```bash +export CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 +export OTEL_TRACES_EXPORTER=otlp +export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json +export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="${PARSEABLE_URL}/v1/traces" +export OTEL_EXPORTER_OTLP_TRACES_HEADERS="X-P-API-Key=${PARSEABLE_API_KEY},X-P-Stream=claudecode-traces,X-P-Log-Source=otel-traces" +``` + +Claude Code requires both `CLAUDE_CODE_ENABLE_TELEMETRY=1` and `CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1` for traces. Anthropic may change the beta schema between releases. + +## Configure a team + +Administrators can place the same values in Claude Code managed settings. Replace the URL and API key placeholders before deployment: + +```json +{ + "env": { + "CLAUDE_CODE_ENABLE_TELEMETRY": "1", + "OTEL_LOGS_EXPORTER": "otlp", + "OTEL_METRICS_EXPORTER": "otlp", + "OTEL_EXPORTER_OTLP_LOGS_PROTOCOL": "http/json", + "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT": "https://:8000/v1/logs", + "OTEL_EXPORTER_OTLP_LOGS_HEADERS": "X-P-API-Key=,X-P-Stream=claudecode-logs,X-P-Log-Source=otel-logs", + "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL": "http/json", + "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT": "https://:8000/v1/metrics", + "OTEL_EXPORTER_OTLP_METRICS_HEADERS": "X-P-API-Key=,X-P-Stream=claudecode-metrics,X-P-Log-Source=otel-metrics" + } +} +``` + +Add the beta trace variables from the previous section if your team collects traces. Managed settings let administrators lock the organization's OTLP destination. Check Anthropic's [managed settings documentation](https://code.claude.com/docs/en/settings#settings-files) before rollout. + +## Verify ingestion + +Start Claude Code and submit one prompt. Then check: + +1. `claudecode-logs` contains a `claude_code.user_prompt` event. +2. `claudecode-metrics` contains `claude_code.session.count`. +3. If you enabled beta traces, `claudecode-traces` contains a `claude_code.interaction` root span. + +Run Claude Code with debug logging if no telemetry arrives: + +```bash +claude --debug +``` + +## Import the dashboard + +Download the [Claude Code Usage dashboard](https://github.com/parseablehq/dashboards/tree/main/claude-code-usage) and import its JSON file in Parseable. Map: + +- **Logs Dataset** to `claudecode-logs` +- **Metrics Dataset** to `claudecode-metrics` + +Use the dashboard to track cost, API requests, sessions, tokens, tools, errors and code activity. + +![Claude Code cost and token usage dashboard](/images/coding-assistants/claude-code-cost-and-tokens.png) + +## Protect sensitive content + +Claude Code redacts prompts, responses, tool details and tool content by default. Keep these defaults unless your data policy permits content capture. + +| Setting | Captured data | +| --- | --- | +| `OTEL_LOG_USER_PROMPTS=1` | User prompt text | +| `OTEL_LOG_ASSISTANT_RESPONSES=1` | Assistant response text | +| `OTEL_LOG_TOOL_DETAILS=1` | Tool parameters, commands, file paths and MCP names | +| `OTEL_LOG_TOOL_CONTENT=1` | Tool input and output content in trace events | + +## Troubleshooting + +- **No datasets appear:** confirm Claude Code can reach Parseable and that each signal has the correct endpoint, `X-P-Stream` and `X-P-Log-Source` value. +- **Metrics arrive but logs do not:** confirm `OTEL_LOGS_EXPORTER=otlp` and submit a prompt after restarting Claude Code. +- **Traces do not arrive:** set both beta variables and confirm your Claude Code version supports enhanced telemetry. +- **Parseable rejects exports:** verify the API key and use the Parseable ingestor URL, including its port. +- **The dashboard is empty:** map its dataset variables to `claudecode-logs` and `claudecode-metrics`. diff --git a/content/docs/ingest-data/ai-agents/coding-assistants/codex.mdx b/content/docs/ingest-data/ai-agents/coding-assistants/codex.mdx new file mode 100644 index 0000000..f94e832 --- /dev/null +++ b/content/docs/ingest-data/ai-agents/coding-assistants/codex.mdx @@ -0,0 +1,91 @@ +--- +title: OpenAI Codex +description: Send Codex logs, traces and metrics directly to Parseable through OpenTelemetry +--- + +Codex exports OpenTelemetry logs, traces and metrics from local coding sessions. Configure its native `[otel]` exporter to send each signal to a separate Parseable dataset without an OpenTelemetry Collector. + +```text +Codex + |-- logs ----> Parseable: codex-logs + |-- traces ---> Parseable: codex-traces + +-- metrics --> Parseable: codex-metrics +``` + +## Prerequisites + +- Codex CLI installed and authenticated +- A Parseable instance reachable from the developer machine +- A Parseable API key with ingest access + +## Configure telemetry + +Open `~/.codex/config.toml` and add: + +```toml +[otel] +environment = "production" +log_user_prompt = false + +exporter = { otlp-http = { endpoint = "https://:8000/v1/logs", protocol = "json", headers = { X-P-API-Key = "", X-P-Stream = "codex-logs", X-P-Log-Source = "otel-logs", Content-Type = "application/json" } } } + +trace_exporter = { otlp-http = { endpoint = "https://:8000/v1/traces", protocol = "json", headers = { X-P-API-Key = "", X-P-Stream = "codex-traces", X-P-Log-Source = "otel-traces", Content-Type = "application/json" } } } + +metrics_exporter = { otlp-http = { endpoint = "https://:8000/v1/metrics", protocol = "json", headers = { X-P-API-Key = "", X-P-Stream = "codex-metrics", X-P-Log-Source = "otel-metrics", Content-Type = "application/json" } } } +``` + +Set `environment` to a value such as `development`, `staging`, or `production`. Parseable stores it with the telemetry so you can separate environments in queries and dashboards. + +For a self-hosted deployment that uses basic authentication, replace `X-P-API-Key = ""` in each exporter with: + +```toml +Authorization = "Basic " +``` + +Restart Codex after changing `config.toml`. Existing processes do not reload telemetry settings. + +## Verify ingestion + +Start a new Codex session, submit a prompt and let Codex run one tool. Then check: + +1. `codex-logs` contains events from the session. +2. `codex-traces` contains spans for the agent run and its operations. +3. `codex-metrics` contains Codex measurements. + +If no data arrives, run Codex with strict configuration validation: + +```bash +codex --strict-config +``` + +Codex reports fields that the installed version does not support or recognize. + +## Import the dashboard + +Download the [Codex Observability dashboard](https://github.com/parseablehq/dashboards/tree/main/codex-observability) and import its JSON file in Parseable. Map: + +- **Logs Dataset** to `codex-logs` +- **Traces Dataset** to `codex-traces` +- **Metrics Dataset** to `codex-metrics` + +Use the dashboard to inspect sessions, turns, models, tokens, cache use, cost, latency, tools, MCP activity, approvals, errors, logs and traces. + +![Codex cost and FinOps dashboard](/images/coding-assistants/codex-cost-and-finops.png) + +## Protect prompt content + +Keep prompt logging disabled unless your data policy permits it: + +```toml +[otel] +log_user_prompt = false +``` + +Setting `log_user_prompt = true` can send source code, file paths, secrets, personal data and other prompt content to the logs dataset. Restrict dataset access and retention before enabling it. + +## Troubleshooting + +- **No datasets appear:** confirm Codex can reach the Parseable ingestor and verify each endpoint, API key, `X-P-Stream` and `X-P-Log-Source` value. +- **Only one signal arrives:** check that `exporter`, `trace_exporter` and `metrics_exporter` each use the matching `/v1/logs`, `/v1/traces`, or `/v1/metrics` endpoint. +- **Parseable rejects exports:** confirm `protocol = "json"`, include `Content-Type = "application/json"` and use the ingestor URL and port. +- **Configuration fails:** run `codex --strict-config` and compare the reported field with the configuration supported by your installed Codex version. diff --git a/content/docs/ingest-data/ai-agents/coding-assistants/github-copilot.mdx b/content/docs/ingest-data/ai-agents/coding-assistants/github-copilot.mdx new file mode 100644 index 0000000..b73489b --- /dev/null +++ b/content/docs/ingest-data/ai-agents/coding-assistants/github-copilot.mdx @@ -0,0 +1,108 @@ +--- +title: GitHub Copilot +description: Send GitHub Copilot agent traces directly to Parseable through OpenTelemetry +--- + +GitHub Copilot CLI exports OpenTelemetry traces for agent sessions. Each trace connects an `invoke_agent` root span to model calls and tool executions. Disable metric export and send the traces to Parseable. + +```text +GitHub Copilot CLI + | OTLP/HTTP + v +Parseable: copilot-traces +``` + +GitHub documents the emitted spans, attributes and supported environment variables in the [Copilot CLI command reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-command-reference#opentelemetry-monitoring). + +## Prerequisites + +- GitHub Copilot CLI installed and authenticated +- A Parseable instance reachable from the developer machine +- A Parseable API key with ingest access + +## Enable Copilot traces + +Set the Parseable OTLP endpoint and headers before starting Copilot CLI: + +```bash +export PARSEABLE_URL="https://:8000" +export PARSEABLE_API_KEY="" + +export COPILOT_OTEL_ENABLED=true +export OTEL_METRICS_EXPORTER=none +export OTEL_EXPORTER_OTLP_ENDPOINT="${PARSEABLE_URL}" +export OTEL_EXPORTER_OTLP_PROTOCOL=http/json +export OTEL_EXPORTER_OTLP_HEADERS="X-P-API-Key=${PARSEABLE_API_KEY},X-P-Stream=copilot-traces,X-P-Log-Source=otel-traces" +export OTEL_SERVICE_NAME=github-copilot + +copilot +``` + +Copilot appends `/v1/traces` to `OTEL_EXPORTER_OTLP_ENDPOINT`, so use the Parseable base URL. `OTEL_METRICS_EXPORTER=none` prevents metrics from using the trace dataset headers. + +For a self-hosted deployment that uses basic authentication, replace `X-P-API-Key=${PARSEABLE_API_KEY}` with `Authorization=Basic ` in `OTEL_EXPORTER_OTLP_HEADERS`. + +Copilot CLI supports `http/json` and `http/protobuf`. Parseable accepts the `http/json` configuration above. + +## Verify ingestion + +Start Copilot CLI and submit a request that causes one model call and one tool call. In Parseable: + +1. Open `copilot-traces` in the Traces view. +2. Find an `invoke_agent GitHub Copilot Chat` root span. +3. Open the trace and confirm it contains `chat` and `execute_tool` child spans. + +Useful trace fields include: + +- `span_trace_id`, `span_span_id` and `span_parent_span_id` +- `span_name`, `span_duration_ns` and `span_status_code` +- `gen_ai.request.model` and `gen_ai.response.model` +- `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` +- `gen_ai.tool.name` +- `github.copilot.git.repository` and `github.copilot.git.branch` + +## Import the dashboard + +Download the [GitHub Copilot Observability dashboard](https://github.com/parseablehq/dashboards/tree/main/github-copilot-observability) and import its JSON file in Parseable. Map **Traces Dataset** to `copilot-traces`. + +Use the dashboard to track trace volume, sessions, model and token usage, tools, latency, errors, repositories and recent traces. + +![GitHub Copilot models and tokens dashboard](/images/coding-assistants/github-copilot-models-and-tokens.png) + +## Configure the Copilot SDK + +If you use the Copilot SDK, pass the Parseable base URL to `TelemetryConfig`. For TypeScript: + +```typescript +import { CopilotClient } from "@github/copilot-sdk"; + +const client = new CopilotClient({ + telemetry: { + otlpEndpoint: "https://:8000", + otlpProtocol: "http/json", + captureContent: false, + }, +}); +``` + +Set `OTEL_METRICS_EXPORTER=none` and `OTEL_EXPORTER_OTLP_HEADERS` in the SDK process environment. GitHub provides equivalent options for Python, Go, .NET, Java and Rust in [OpenTelemetry instrumentation for Copilot SDK](https://docs.github.com/en/copilot/how-tos/copilot-sdk/observability/opentelemetry). + +## Protect sensitive content + +Copilot excludes prompts, responses and tool arguments by default. Keep content capture disabled for shared or production environments. + +Enable full message capture with: + +```bash +export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true +``` + +Content capture can include source code, file contents, user prompts, model responses and tool arguments. Apply Parseable access controls before enabling it. GitHub's [OpenTelemetry agent monitoring guide](https://docs.github.com/en/copilot/concepts/agents/opentelemetry) describes the privacy behavior. + +## Troubleshooting + +- **No spans arrive:** confirm Copilot CLI can reach Parseable, verify the API key and set `OTEL_LOG_LEVEL=DEBUG` to inspect exporter diagnostics. +- **The trace has no child spans:** generate a new session after setting the environment variables. Existing Copilot processes do not inherit later shell changes. +- **Metrics appear in the trace dataset:** set `OTEL_METRICS_EXPORTER=none` in the Copilot process environment. +- **Parseable rejects exports:** use the Parseable ingestor base URL and verify `X-P-Stream=copilot-traces` and `X-P-Log-Source=otel-traces`. +- **The dashboard is empty:** map **Traces Dataset** to `copilot-traces` and choose a time range that includes the test session. diff --git a/content/docs/ingest-data/ai-agents/coding-assistants/meta.json b/content/docs/ingest-data/ai-agents/coding-assistants/meta.json new file mode 100644 index 0000000..e79a989 --- /dev/null +++ b/content/docs/ingest-data/ai-agents/coding-assistants/meta.json @@ -0,0 +1,4 @@ +{ + "title": "Coding assistants", + "pages": ["claude-code", "github-copilot", "codex"] +} diff --git a/content/docs/ingest-data/ai-agents/index.mdx b/content/docs/ingest-data/ai-agents/index.mdx index fd75d55..d80b8a6 100644 --- a/content/docs/ingest-data/ai-agents/index.mdx +++ b/content/docs/ingest-data/ai-agents/index.mdx @@ -5,6 +5,7 @@ description: Send telemetry from agent frameworks, sandboxes, LLM runtimes, and import { IconBrandOpenai, + IconBrandGithub, IconBrain, IconCode, IconLink, @@ -83,6 +84,34 @@ Agent frameworks usually carry the highest-level application context. They can s +## Coding assistants + +Coding assistants emit agent-level telemetry from interactive development sessions. Use these guides to inspect model calls, token usage, tool execution, latency, errors, and session activity. + + + } + > + Export Claude Code events, metrics and optional traces to Parseable. + + } + > + Export GitHub Copilot agent traces to Parseable. + + } + > + Export Codex logs, traces and metrics directly to Parseable. + + + ## Sandboxes and runtimes Sandboxes and execution runtimes are where agent work becomes real system activity. These guides focus on telemetry from code execution, durable workflows, serverless jobs, and workflow orchestration layers. diff --git a/content/docs/ingest-data/ai-agents/meta.json b/content/docs/ingest-data/ai-agents/meta.json index 81c1731..b6fb7be 100644 --- a/content/docs/ingest-data/ai-agents/meta.json +++ b/content/docs/ingest-data/ai-agents/meta.json @@ -2,8 +2,9 @@ "title": "AI Infrastructure", "pages": [ "agent-frameworks", + "coding-assistants", "sandboxes-runtimes", "llms", "gateways" ] -} \ No newline at end of file +} diff --git a/public/images/coding-assistants/claude-code-cost-and-tokens.png b/public/images/coding-assistants/claude-code-cost-and-tokens.png new file mode 100644 index 0000000..6b180c7 Binary files /dev/null and b/public/images/coding-assistants/claude-code-cost-and-tokens.png differ diff --git a/public/images/coding-assistants/codex-cost-and-finops.png b/public/images/coding-assistants/codex-cost-and-finops.png new file mode 100644 index 0000000..6c9bd6f Binary files /dev/null and b/public/images/coding-assistants/codex-cost-and-finops.png differ diff --git a/public/images/coding-assistants/github-copilot-models-and-tokens.png b/public/images/coding-assistants/github-copilot-models-and-tokens.png new file mode 100644 index 0000000..03fa3d3 Binary files /dev/null and b/public/images/coding-assistants/github-copilot-models-and-tokens.png differ