feat: add AI agent skills inventory scanning support#162
Conversation
Scan for Agent Skills installed for AI coding agents at global and per-project scope, covering Claude Code skills and skills.sh-managed installs (including plugin-provided skills). For each skill the scan records metadata (name, agent, scope, source), a SHA-256 hash of its SKILL.md for drift detection, skills.sh provenance from lock files, and a stat-only census of the skill folder. Skill file contents are never read beyond SKILL.md and are never transmitted; local source paths from lock files are recorded by alias only. The ~/.claude.json project-registry discovery is shared with the MCP detector.
The home directory appears in the ~/.claude.json project registry whenever Claude Code has been run from $HOME. Treating it as a project re-scanned the global skill dirs (~/.claude/skills, ~/.agents/skills) and re-emitted every global skill as a project-scoped duplicate, also double-counting roots_scanned. Skip the home directory in project discovery — its dotfile skill dirs are the global roots. Adds a regression test covering a registry that lists both home and a genuine sub-project.
…s-inventory # Conflicts: # internal/featuregate/featuregate.go
skills.sh installs a skill once and symlinks it into each agent's own root, so enumeration emitted one record per root and inflated a single physical skill into N+1 records. Collapse those shadows into one record per physical skill dir, keyed on the symlink-resolved path: the real directory is canonical and the linked roots are recorded in a new symlink_sources field. Drop lock-only synthesis — a lock entry with no folder on disk is not an installed skill — and remove the now-constant is_symlink and present_on_disk fields. Also add Pi, Factory, Amp, and Copilot skill roots (global and project), including Factory's singular .agent/skills and Copilot's .github/skills.
The ~/.claude/plugins/{cache,repos} subtrees are no longer walked for
skill directories, and claude_plugin is removed from the source set.
Skills are discovered through the global, project, and skills.sh
lock-managed roots only; a plugin's skill is inventoried when it is
symlinked or installed into one of those roots.
The per-skill has_plugin_manifest census flag and the skills.sh lock's
plugin_name provenance are unchanged.
Also stop echoing the canonical record's own source into symlink_sources
when another member of its collapse group shares that source.
…nputs Truncation honesty: mark the scan Truncated on context cancellation or deadline and in the panic-recovery path, and enforce a 2000-record aggregate cap (matching the backend payload cap) through a finalizeSkills helper shared by the normal return and the recovery path, so a partial inventory is never reported as complete and a late panic cannot ship an uncapped record list. Input hardening: reject non-regular files (FIFO/socket/device) in findSkillMD, parseSkillMD, and loadLock — a reader-less FIFO would block the ctx-less os.ReadFile forever; sanitize skills-lock map keys before joining onto the install base so a traversal key in an untrusted project lock cannot stamp forged provenance onto an unrelated skill; rewrite splitFrontmatter line-based with a column-zero closing fence so a "---" inside a quoted value or block scalar no longer severs the frontmatter and silently drops fields such as hooks. Also map python venv paths up to their project dirs in the telemetry skills bridge (skills live under <project>/.claude/skills, not the venv), and render "Not scanned" vs "None detected" distinctly in pretty and HTML output when the skills scan did not run.
…scovery The agent-skills detector was the one filesystem walker not wired to the shared tcc.Skipper, so a project registered in ~/.claude.json under a TCC-protected directory (e.g. ~/Documents) was probed directly and could fire a macOS permission prompt, breaking the agent's "never triggers a TCC popup" contract. - Add tcc.Skipper.WithinProtected: an ancestor-aware check (a protected dir or any path nested under it) for callers that resolve a deep path directly, matched on equality or a "/" boundary so a sibling such as ~/Documents.backup is not swallowed. - Wire a skipper into SkillsDetector and guard project discovery before any stat; canonicalNoStat canonicalizes lexically so the check never triggers the EvalSymlinks that would itself prompt. Add defense-in-depth guards on global/project roots and symlink targets. - Guard lock-file paths before the stat in loadLock, closing the XDG_STATE_HOME-under-~/Library vector. - Wire the skipper at the community-scan and telemetry construction sites. A nil skipper (--include-tcc-protected) and non-macOS builds are unaffected.
has_code / code_file_count is a coarse "this skill could run code" signal that complements has_shell_injection and has_hooks. The census counted only .py/.js/.ts/.sh, so PowerShell, Ruby, ESM (.mjs/.cjs), .bash/.zsh, and Windows .bat/.cmd skills read as has_code=false, understating risk. Widen codeExtensions to the script/interpreter types an agent or the OS runs directly: Python (+.pyw), Node/JS/TS variants (.mjs/.cjs/.jsx/.tsx), shell family (.bash/.zsh/.fish), Windows scripts (.ps1/.psm1/.bat/.cmd), and other interpreters (.rb/.pl/.php/.lua). Compiled languages are intentionally excluded (build step required, prone to vendored-dep/build-artifact false positives). No logic change — the stat-only census keys off this map unchanged.
Add a TCC-safe recursive walk of the search dirs (default $HOME) that finds project-convention marker dirs (.claude, .agents, .gemini, .aider, …) and feeds their parents into discoverProjects, so projects never registered in ~/.claude.json and unseen by the node/python scanners are still inventoried. Registry and node/python roots stay authoritative — walk candidates fill only leftover maxProjects capacity and never evict a registered project. ReadDir is the walk's only filesystem primitive: roots are normalized, dropped if at or under a TCC-protected tree, and skipped if any path component is a symlink (classified from ancestor listings, never followed), so no prompt can fire under the default posture. Also add Gemini CLI (~/.gemini/skills, .gemini/skills) and Aider (.aider/skills) sources plus the Zed alias, and walk_dirs_visited / walk_roots_found counters.
…ccheck S1016
pipeDirEntry and pipeFileInfo are structurally identical (struct{ name string }), so use a type conversion instead of a field-by-field struct literal.
There was a problem hiding this comment.
Pull request overview
Adds a new filesystem-only detector that inventories “agent skills” (directories containing SKILL.md) across multiple AI coding agents and skills.sh-managed installs, and wires the results through community scan output, enterprise telemetry payloads, and both Pretty/HTML renderers.
Changes:
- Introduces
SkillsDetectorwith bounded discovery (global + project + optional home-walk), SKILL.md frontmatter parsing + hash, stat-only folder census, and optional skills.sh lock provenance enrichment. - Plumbs new
model.AgentSkill/model.AgentSkillScanInfointo scan + telemetry payloads and exposes counts via summary/metrics and Pretty/HTML output. - Shares Claude Code project-root discovery via new
discoverClaudeProjectsand adds a new TCC helper (WithinProtected) plus extensive unit tests.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| SCAN_COVERAGE.md | Documents the new “AI Agent Skills” inventory coverage and privacy posture. |
| internal/telemetry/telemetry.go | Adds agent-skills collection phase to enterprise telemetry and includes results + metrics in the payload. |
| internal/telemetry/phase_deadline.go | Allocates a phase budget for agent_skills_scan. |
| internal/telemetry/collect_project_roots_test.go | Tests telemetry-side project-root mapping for the skills detector. |
| internal/tcc/tcc.go | Adds WithinProtected and hasDirPrefix for pre-access protected-tree checks. |
| internal/tcc/tcc_darwin_test.go | Adds darwin-only tests covering WithinProtected. |
| internal/scan/scanner.go | Runs the skills detector in community scans and wires results into ScanResult + summary. |
| internal/output/pretty.go | Renders an “AGENT SKILLS” section and summary count in Pretty output. |
| internal/output/pretty_test.go | Adds Pretty output tests for the new section’s “not scanned / none / populated” states. |
| internal/output/html.go | Adds Agent Skills summary card + section/table to HTML output. |
| internal/output/html_test.go | Adds HTML output tests for the new Agent Skills section states. |
| internal/model/model.go | Adds AgentSkill, AgentSkillScanInfo, and scan summary count wiring in ScanResult/Summary. |
| internal/featuregate/featuregate.go | Adds and enables FeatureAgentSkillsScan. |
| internal/featuregate/featuregate_test.go | Extends override test coverage to include the new feature gate. |
| internal/executor/mock.go | Extends mock executor to simulate symlink dir entries and symlink resolution errors. |
| internal/detector/skills.go | Implements the main skills discovery pipeline (roots, project discovery, symlink collapsing, caps). |
| internal/detector/skills_test.go | Comprehensive unit tests for skills parsing, census, lock parsing, symlink collapsing, and detector behavior. |
| internal/detector/skills_tcc_darwin_test.go | Darwin-only tests ensuring the detector avoids TCC-protected trees without triggering access. |
| internal/detector/skills_lock.go | Parses skills.sh lock files, sanitizes keys, and enriches discovered skills with provenance. |
| internal/detector/skills_homewalk_test.go | Tests home-walk project discovery and its caps/determinism. |
| internal/detector/skills_frontmatter.go | Parses YAML frontmatter from SKILL.md, hashes content, and scans body for load-time shell exec markers. |
| internal/detector/skills_census.go | Implements the stat-only directory census (counts/sizes/mtime/plugin manifest detection). |
| internal/detector/mcp.go | Switches MCP project discovery to reuse discoverClaudeProjects. |
| internal/detector/claudeprojects.go | New shared helper to read project roots from ~/.claude.json. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Defense-in-depth: the walk is designed panic-free — every per-root and | ||
| // per-skill failure degrades to an Errors entry rather than a panic — but if | ||
| // one still escapes we must NOT leave AgentSkillScan nil. A nil scan info | ||
| // means "no information" and would strand the device's skill state; a non-nil | ||
| // info (even with partial or zero records) means "scan ran". Record the panic | ||
| // and finalize whatever we gathered. Registered after `defer cancel()` so it | ||
| // runs first (LIFO), recovering before the context is torn down; the recovery | ||
| // re-collapses whatever `discovered` accumulated, so partial discovery | ||
| // survives the unwind. Containing the panic here keeps a skills bug from | ||
| // failing the whole telemetry run via telemetry.Run. The recorded error also | ||
| // marks an early-panic "scan ran, 0 skills" result as partial rather than | ||
| // complete. | ||
| var discovered []discoveredSkill | ||
| defer func() { | ||
| if r := recover(); r != nil { | ||
| d.addError(info, fmt.Sprintf("panic in skills detect: %v", r)) | ||
| // A panic aborted the walk mid-flight — the inventory is partial. Mark it | ||
| // so the backend keeps the scan non-authoritative and suppresses deletions. | ||
| info.Truncated = true | ||
| skills = d.finalizeSkills(discovered, info) | ||
| info.SkillsFound = len(skills) | ||
| info.DurationMs = time.Since(start).Milliseconds() | ||
| } | ||
| }() |
There was a problem hiding this comment.
This recover() is intentional, and it's the one spot the skills detector deliberately diverges from the "detectors return values; telemetry.Run handles panics" convention. Two reasons specific to this detector:
-
Detectreturns(skills, info), whereinfois the "scan ran" sentinel. AnilAgentSkillScanmeans "no information," which the backend treats differently from "scan ran, found N." The local recover guarantees a non-nilinfo(with the partial inventory) even on a mid-walk panic. -
Blast radius. Without it, a panic here propagates to
telemetry.Run's top-level recover (telemetry.go:335), which marks the entire run failed and uploads no payload — discarding every other detector's results (IDEs, extensions, CLI tools, npmrc/pip audits, …) that already ran this pass. Containing it here keeps a bug in this new, default-on, most-complex detector from failing the whole telemetry run.
The failure isn't swallowed: the handler records the panic via addError(info, …) and sets info.Truncated = true, so the scan is surfaced as partial/non-authoritative (the backend then suppresses deletions for it) rather than silently succeeding. Given that, we'd prefer to keep it. Happy to revisit if we later add per-phase panic isolation in telemetry.Run — at that point this could move up to the phase runner and share the pattern with the other detectors.
| // Lock files: parse the global lock + each project lock and join skills.sh | ||
| // provenance onto matching on-disk records. A lock entry with no folder on | ||
| // disk is not an install and is dropped — the inventory is on-disk skills only. | ||
| discovered = d.applyLocks(discovered, projects, info) |
Summary
Adds a detector that inventories Agent Skills installed for AI coding agents, at both global and per-project scope. It covers Claude Code skills and skills.sh-managed installs (including plugin-provided skills).
For each discovered skill the scan records:
SKILL.md(for drift detection)Lock entries with no corresponding folder on disk are dropped — a lock entry is not an install, so the inventory is limited to on-disk skills.
Privacy
SKILL.md, and are never transmitted.SKILL.mdis recorded — never its contents.Notes
SKILL.mddiscovery is case-sensitive (exactSKILL.md).~/.claude.jsonproject-registry discovery is shared with the MCP detector.Testing
go build ./...,go vet ./..., andgofmtare cleango test ./...passes for the affected packages