Skip to content

feat: add AI agent skills inventory scanning support#162

Merged
ashishkurmi merged 12 commits into
step-security:mainfrom
raysubham:feat/ai-agent-skills-inventory
Jul 16, 2026
Merged

feat: add AI agent skills inventory scanning support#162
ashishkurmi merged 12 commits into
step-security:mainfrom
raysubham:feat/ai-agent-skills-inventory

Conversation

@raysubham

@raysubham raysubham commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a detector that inventories Agent Skills installed for AI coding agents, at both global and per-project scope. It covers Claude Code skills and skills.sh-managed installs (including plugin-provided skills).

For each discovered skill the scan records:

  • Metadata: skill name, agent, scope, and source
  • A SHA-256 hash of the skill's SKILL.md (for drift detection)
  • skills.sh provenance from lock files (source, ref, install/update timestamps)
  • A stat-only census of the skill folder (file counts and sizes — contents are not read)

Lock entries with no corresponding folder on disk are dropped — a lock entry is not an install, so the inventory is limited to on-disk skills.

Privacy

  • Skill file contents are never read beyond SKILL.md, and are never transmitted.
  • Only a single SHA-256 hash of each SKILL.md is recorded — never its contents.
  • For local skill sources, the on-disk source path is never serialized; only the lock alias is recorded.

Notes

  • SKILL.md discovery is case-sensitive (exact SKILL.md).
  • The ~/.claude.json project-registry discovery is shared with the MCP detector.
  • Enabled by default.

Testing

  • go build ./..., go vet ./..., and gofmt are clean
  • go test ./... passes for the affected packages

Scan for Agent Skills installed for AI coding agents at global and
per-project scope, covering Claude Code skills and skills.sh-managed
installs (including plugin-provided skills).

For each skill the scan records metadata (name, agent, scope, source),
a SHA-256 hash of its SKILL.md for drift detection, skills.sh provenance
from lock files, and a stat-only census of the skill folder. Skill file
contents are never read beyond SKILL.md and are never transmitted; local
source paths from lock files are recorded by alias only.

The ~/.claude.json project-registry discovery is shared with the MCP
detector.
@raysubham raysubham changed the title Add AI agent skills inventory scanning feature feat: add AI agent skills inventory scanning support Jul 9, 2026
raysubham and others added 10 commits July 9, 2026 19:17
The home directory appears in the ~/.claude.json project registry whenever
Claude Code has been run from $HOME. Treating it as a project re-scanned the
global skill dirs (~/.claude/skills, ~/.agents/skills) and re-emitted every
global skill as a project-scoped duplicate, also double-counting roots_scanned.

Skip the home directory in project discovery — its dotfile skill dirs are the
global roots. Adds a regression test covering a registry that lists both home
and a genuine sub-project.
…s-inventory

# Conflicts:
#	internal/featuregate/featuregate.go
skills.sh installs a skill once and symlinks it into each agent's own
root, so enumeration emitted one record per root and inflated a single
physical skill into N+1 records. Collapse those shadows into one record
per physical skill dir, keyed on the symlink-resolved path: the real
directory is canonical and the linked roots are recorded in a new
symlink_sources field. Drop lock-only synthesis — a lock entry with no
folder on disk is not an installed skill — and remove the now-constant
is_symlink and present_on_disk fields.

Also add Pi, Factory, Amp, and Copilot skill roots (global and project),
including Factory's singular .agent/skills and Copilot's .github/skills.
The ~/.claude/plugins/{cache,repos} subtrees are no longer walked for
skill directories, and claude_plugin is removed from the source set.
Skills are discovered through the global, project, and skills.sh
lock-managed roots only; a plugin's skill is inventoried when it is
symlinked or installed into one of those roots.

The per-skill has_plugin_manifest census flag and the skills.sh lock's
plugin_name provenance are unchanged.

Also stop echoing the canonical record's own source into symlink_sources
when another member of its collapse group shares that source.
…nputs

Truncation honesty: mark the scan Truncated on context cancellation or
deadline and in the panic-recovery path, and enforce a 2000-record
aggregate cap (matching the backend payload cap) through a
finalizeSkills helper shared by the normal return and the recovery
path, so a partial inventory is never reported as complete and a late
panic cannot ship an uncapped record list.

Input hardening: reject non-regular files (FIFO/socket/device) in
findSkillMD, parseSkillMD, and loadLock — a reader-less FIFO would
block the ctx-less os.ReadFile forever; sanitize skills-lock map keys
before joining onto the install base so a traversal key in an untrusted
project lock cannot stamp forged provenance onto an unrelated skill;
rewrite splitFrontmatter line-based with a column-zero closing fence so
a "---" inside a quoted value or block scalar no longer severs the
frontmatter and silently drops fields such as hooks.

Also map python venv paths up to their project dirs in the telemetry
skills bridge (skills live under <project>/.claude/skills, not the
venv), and render "Not scanned" vs "None detected" distinctly in pretty
and HTML output when the skills scan did not run.
…scovery

The agent-skills detector was the one filesystem walker not wired to the
shared tcc.Skipper, so a project registered in ~/.claude.json under a
TCC-protected directory (e.g. ~/Documents) was probed directly and could fire
a macOS permission prompt, breaking the agent's "never triggers a TCC popup"
contract.

- Add tcc.Skipper.WithinProtected: an ancestor-aware check (a protected dir or
  any path nested under it) for callers that resolve a deep path directly,
  matched on equality or a "/" boundary so a sibling such as ~/Documents.backup
  is not swallowed.
- Wire a skipper into SkillsDetector and guard project discovery before any
  stat; canonicalNoStat canonicalizes lexically so the check never triggers the
  EvalSymlinks that would itself prompt. Add defense-in-depth guards on
  global/project roots and symlink targets.
- Guard lock-file paths before the stat in loadLock, closing the
  XDG_STATE_HOME-under-~/Library vector.
- Wire the skipper at the community-scan and telemetry construction sites.

A nil skipper (--include-tcc-protected) and non-macOS builds are unaffected.
has_code / code_file_count is a coarse "this skill could run code" signal that
complements has_shell_injection and has_hooks. The census counted only
.py/.js/.ts/.sh, so PowerShell, Ruby, ESM (.mjs/.cjs), .bash/.zsh, and Windows
.bat/.cmd skills read as has_code=false, understating risk.

Widen codeExtensions to the script/interpreter types an agent or the OS runs
directly: Python (+.pyw), Node/JS/TS variants (.mjs/.cjs/.jsx/.tsx), shell
family (.bash/.zsh/.fish), Windows scripts (.ps1/.psm1/.bat/.cmd), and other
interpreters (.rb/.pl/.php/.lua). Compiled languages are intentionally excluded
(build step required, prone to vendored-dep/build-artifact false positives).

No logic change — the stat-only census keys off this map unchanged.
Add a TCC-safe recursive walk of the search dirs (default $HOME) that finds
project-convention marker dirs (.claude, .agents, .gemini, .aider, …) and feeds
their parents into discoverProjects, so projects never registered in
~/.claude.json and unseen by the node/python scanners are still inventoried.

Registry and node/python roots stay authoritative — walk candidates fill only
leftover maxProjects capacity and never evict a registered project. ReadDir is
the walk's only filesystem primitive: roots are normalized, dropped if at or
under a TCC-protected tree, and skipped if any path component is a symlink
(classified from ancestor listings, never followed), so no prompt can fire under
the default posture.

Also add Gemini CLI (~/.gemini/skills, .gemini/skills) and Aider (.aider/skills)
sources plus the Zed alias, and walk_dirs_visited / walk_roots_found counters.
…ccheck S1016

pipeDirEntry and pipeFileInfo are structurally identical (struct{ name string }), so use a type conversion instead of a field-by-field struct literal.
@ashishkurmi
ashishkurmi requested a review from Copilot July 16, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new filesystem-only detector that inventories “agent skills” (directories containing SKILL.md) across multiple AI coding agents and skills.sh-managed installs, and wires the results through community scan output, enterprise telemetry payloads, and both Pretty/HTML renderers.

Changes:

  • Introduces SkillsDetector with bounded discovery (global + project + optional home-walk), SKILL.md frontmatter parsing + hash, stat-only folder census, and optional skills.sh lock provenance enrichment.
  • Plumbs new model.AgentSkill / model.AgentSkillScanInfo into scan + telemetry payloads and exposes counts via summary/metrics and Pretty/HTML output.
  • Shares Claude Code project-root discovery via new discoverClaudeProjects and adds a new TCC helper (WithinProtected) plus extensive unit tests.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
SCAN_COVERAGE.md Documents the new “AI Agent Skills” inventory coverage and privacy posture.
internal/telemetry/telemetry.go Adds agent-skills collection phase to enterprise telemetry and includes results + metrics in the payload.
internal/telemetry/phase_deadline.go Allocates a phase budget for agent_skills_scan.
internal/telemetry/collect_project_roots_test.go Tests telemetry-side project-root mapping for the skills detector.
internal/tcc/tcc.go Adds WithinProtected and hasDirPrefix for pre-access protected-tree checks.
internal/tcc/tcc_darwin_test.go Adds darwin-only tests covering WithinProtected.
internal/scan/scanner.go Runs the skills detector in community scans and wires results into ScanResult + summary.
internal/output/pretty.go Renders an “AGENT SKILLS” section and summary count in Pretty output.
internal/output/pretty_test.go Adds Pretty output tests for the new section’s “not scanned / none / populated” states.
internal/output/html.go Adds Agent Skills summary card + section/table to HTML output.
internal/output/html_test.go Adds HTML output tests for the new Agent Skills section states.
internal/model/model.go Adds AgentSkill, AgentSkillScanInfo, and scan summary count wiring in ScanResult/Summary.
internal/featuregate/featuregate.go Adds and enables FeatureAgentSkillsScan.
internal/featuregate/featuregate_test.go Extends override test coverage to include the new feature gate.
internal/executor/mock.go Extends mock executor to simulate symlink dir entries and symlink resolution errors.
internal/detector/skills.go Implements the main skills discovery pipeline (roots, project discovery, symlink collapsing, caps).
internal/detector/skills_test.go Comprehensive unit tests for skills parsing, census, lock parsing, symlink collapsing, and detector behavior.
internal/detector/skills_tcc_darwin_test.go Darwin-only tests ensuring the detector avoids TCC-protected trees without triggering access.
internal/detector/skills_lock.go Parses skills.sh lock files, sanitizes keys, and enriches discovered skills with provenance.
internal/detector/skills_homewalk_test.go Tests home-walk project discovery and its caps/determinism.
internal/detector/skills_frontmatter.go Parses YAML frontmatter from SKILL.md, hashes content, and scans body for load-time shell exec markers.
internal/detector/skills_census.go Implements the stat-only directory census (counts/sizes/mtime/plugin manifest detection).
internal/detector/mcp.go Switches MCP project discovery to reuse discoverClaudeProjects.
internal/detector/claudeprojects.go New shared helper to read project roots from ~/.claude.json.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +153 to +176
// Defense-in-depth: the walk is designed panic-free — every per-root and
// per-skill failure degrades to an Errors entry rather than a panic — but if
// one still escapes we must NOT leave AgentSkillScan nil. A nil scan info
// means "no information" and would strand the device's skill state; a non-nil
// info (even with partial or zero records) means "scan ran". Record the panic
// and finalize whatever we gathered. Registered after `defer cancel()` so it
// runs first (LIFO), recovering before the context is torn down; the recovery
// re-collapses whatever `discovered` accumulated, so partial discovery
// survives the unwind. Containing the panic here keeps a skills bug from
// failing the whole telemetry run via telemetry.Run. The recorded error also
// marks an early-panic "scan ran, 0 skills" result as partial rather than
// complete.
var discovered []discoveredSkill
defer func() {
if r := recover(); r != nil {
d.addError(info, fmt.Sprintf("panic in skills detect: %v", r))
// A panic aborted the walk mid-flight — the inventory is partial. Mark it
// so the backend keeps the scan non-authoritative and suppresses deletions.
info.Truncated = true
skills = d.finalizeSkills(discovered, info)
info.SkillsFound = len(skills)
info.DurationMs = time.Since(start).Milliseconds()
}
}()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This recover() is intentional, and it's the one spot the skills detector deliberately diverges from the "detectors return values; telemetry.Run handles panics" convention. Two reasons specific to this detector:

  1. Detect returns (skills, info), where info is the "scan ran" sentinel. A nil AgentSkillScan means "no information," which the backend treats differently from "scan ran, found N." The local recover guarantees a non-nil info (with the partial inventory) even on a mid-walk panic.

  2. Blast radius. Without it, a panic here propagates to telemetry.Run's top-level recover (telemetry.go:335), which marks the entire run failed and uploads no payload — discarding every other detector's results (IDEs, extensions, CLI tools, npmrc/pip audits, …) that already ran this pass. Containing it here keeps a bug in this new, default-on, most-complex detector from failing the whole telemetry run.

The failure isn't swallowed: the handler records the panic via addError(info, …) and sets info.Truncated = true, so the scan is surfaced as partial/non-authoritative (the backend then suppresses deletions for it) rather than silently succeeding. Given that, we'd prefer to keep it. Happy to revisit if we later add per-phase panic isolation in telemetry.Run — at that point this could move up to the phase runner and share the pattern with the other detectors.

Comment on lines +202 to +205
// Lock files: parse the global lock + each project lock and join skills.sh
// provenance onto matching on-disk records. A lock entry with no folder on
// disk is not an install and is dropped — the inventory is on-disk skills only.
discovered = d.applyLocks(discovered, projects, info)
@ashishkurmi
ashishkurmi merged commit 3894108 into step-security:main Jul 16, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants