feat(agent): per-command agent schema with response shapes - #883
Merged
Merged
Conversation
`pup agent schema` was all-or-nothing (~185 KB compact) and described inputs only, so agents paid ~50k tokens per lookup and still guessed response shapes. - `pup agent schema <path>` accepts `logs aggregate`, `logs.aggregate`, or a domain (`logs`); aliases resolve like clap. Unknown paths exit non-zero listing valid subcommands. `auth token` stays hidden. - `--search TERM` lists matching leaf commands (path + short description), optionally scoped to a path. - Filtered output adds a shared `envelope` contract and a `returns` entry on every leaf; hand-verified shapes for logs aggregate/search, traces aggregate/search, and metrics query, with `jq_root`, notes, and a worked example on single-command lookups. - Filtered output is minified: logs aggregate ~3.8 KB, logs ~16.8 KB. - No-argument and `--compact` output are unchanged. - Conformance test feeds recorded API bodies through `output::build_agent_envelope` and validates them against `returns`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In agent mode, `pup logs aggregate --help` resolved only the top-level token and returned the whole logs domain (~32 KB), with every entry's full_path built from an empty parent (so read_only could be wrong for nested commands) and no response shape. - Resolve the deepest command named on the command line (aliases map to canonical names; positional values and global flag values are skipped; `auth token` is never descended into). - Build the entry with its real parent path and attach `envelope` and per-leaf `returns`, as `pup agent schema <path>` does. - Keep global_flags, script_authoring and anti_patterns so a lone `--help` still carries the --no-agent guidance. Root `--help` is unchanged. - Replace `find_subcommand` with `help_command_path` and port its tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`pup security findings schema` downloads the whole findings reference
(~200 KB, ~50k tokens) on every call with no way to narrow it.
- Parse the reference's attribute tables into {path, type, section,
description} rows (all 618 rows in the current page).
- `--search TERM` keeps fields whose path or description match;
`--section NAME` keeps one namespace (nested headings included).
Filtered output goes through the formatter, so it honors agent mode,
--jq and --output. `--section advisory` is ~2 KB.
- An unknown section errors, listing the available ones; a page that
yields no parseable rows errors rather than returning an empty list.
- No flags: output unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without filters the command prints markdown, not an enveloped JSON value, so the generic contract misdescribed it. Document both modes, add a conformance fixture, and note in the envelope that some commands print plain text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
platinummonkey
marked this pull request as ready for review
October 1, 2026 20:51
platinummonkey
marked this pull request as draft
October 1, 2026 20:52
Contributor
There was a problem hiding this comment.
Three distinct issues were found: a timestamp unit mismatch in the metrics query contract (milliseconds vs. seconds), a missing blank-input guard in the section filter logic, and incorrect skipping of parent-local option values during help resolution.
🤖 Bits Code Review · Commit d1e2550 · @DataDog review to ask questions
Co-authored-by: datadog-prod-us1-3[bot] <266080212+datadog-prod-us1-3[bot]@users.noreply.github.com>
Co-authored-by: datadog-prod-us1-3[bot] <266080212+datadog-prod-us1-3[bot]@users.noreply.github.com>
`pup profiling --header "x: 1" services list --help` treated the header value as a subcommand name and fell back to the profiling domain, since only a hard-coded list of global flags was known to take values. Look each flag up in clap at the current level (plus root globals) instead, handling `--flag=value` and `-ovalue` forms. Also add a test for the blank --section guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit c50e909. The v1 metrics query response documents from_date/to_date as milliseconds since Unix epoch (datadog-api-client MetricsQueryResponse), so the contract's original "ms since epoch" was correct. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
platinummonkey
marked this pull request as ready for review
October 2, 2026 14:05
platinummonkey
enabled auto-merge
October 2, 2026 14:18
jack-edmonds-dd
approved these changes
Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pup agent schemareturns everything at once (~185 KB) and only describes what each command takes, not what it returns, so agents guess the output format. This adds lookups for a single command or domain that also show what the command returns.Changes
pup agent schema logs aggregate(orlogs.aggregate, or a domain likelogs) returns just that part, with areturnsshape for each command.--search TERMfinds commands by name or description.pup <cmd> --helpnow covers the exact command asked for, not its whole domain.pup security findings schema --search/--sectionreturns only the matching fields instead of the full ~200 KB reference.Testing
returnsshapes match real command output.Follow-ups
--jqnote in agent-mode output still conflicts with the published jq paths.🤖 Generated with Claude Code