Skip to content

feat(skills): both name forms for tool→script binding + validate lint (#418) - #419

Merged
initializ-mk merged 2 commits into
mainfrom
feat/skill-validate-lint
Aug 21, 2026
Merged

feat(skills): both name forms for tool→script binding + validate lint (#418)#419
initializ-mk merged 2 commits into
mainfrom
feat/skill-validate-lint

Conversation

@initializ-mk

Copy link
Copy Markdown
Contributor

Closes #418.

Two Forge skill-registration behaviors failed silently — a tool was discarded at runtime with (at best) one error-log line, and no author/build-time signal. Customers had to reimplement Forge's parser as external lints and re-confirm them against every FORGE_VERSION. This makes Forge the source of truth.

1. Tool→script binding accepts both name forms

A ## Tool: some_name previously bound only to scripts/some-name.{sh,py,js} — underscores were forced to hyphens, so an author who named the file scripts/some_name.sh to match the tool got a silently-missing tool.

Now it binds to either scripts/some_name.<ext> or scripts/some-name.<ext> (and symmetrically for a hyphenated tool name). The hyphenated form keeps priority only if both files exist, so existing skills resolve identically. Priority order is otherwise unchanged: skill-local scripts/ before shared skills/scripts/, shell → python → node.

Resolution is refactored around one exported generator, runtime.SkillScriptCandidatePaths, shared by the runtime resolver (resolveSkillScript), the read-skill catalog (skillEntryHasScript), and the new validator — so the "what's a first-class tool" decision can't drift between them. Traversal hardening (.., path separators) is preserved.

2. forge skills validate surfaces the silent failures

validate now scans the whole skill tree (main SKILL.md and every skills/*/SKILL.md, not just the root as before) and reports, per ## Tool::

Check Level Why
Invalid **Input:** property key outside ^[a-zA-Z0-9_.-]{1,64}$ ERROR The provider rejects the entire request → the runtime drops the whole tool at startup (would 400 every LLM call).
No backing script (either name form) ERROR The tool never registers.
Orphan script with no ## Tool: heading WARN Reachable only via run_skill_script; dead weight if unreferenced.

The property-key check runs the real parser (InputSpecToSchema + InvalidSchemaPropertyKeys), so the lint can't approximate the rule wrong. validate exits non-zero on any ERROR, so CI can gate on it — replacing the external lints.

Sample output:

Tools:
  ERROR report / bad_scan: input property key(s) pod name violate ^[a-zA-Z0-9_.-]{1,64}$ — the tool is DROPPED at runtime (it would fail every LLM call)
  ERROR report / no_script: no backing script (scripts/no_script.{sh,py,js}) — the tool will not register
  WARN  report: orphan script skills/report/scripts/dead-helper.sh — no `## Tool:` heading binds it; reachable only via run_skill_script (dead weight if unreferenced)

Tests

Docs

skills-cli.md (new Validate section), writing-custom-skills.md (binding rule + silent-failure guard), skill-md-format.md (runtime table), cli-reference.md.

Follow-up (noted in #418)

Wire the same lint into the forge build validate stage so an invalid property key fails the build, not just validate.

…te lint (#418)

A `## Tool: some_name` previously bound ONLY to scripts/some-name.<ext>
(underscores forced to hyphens); a file named scripts/some_name.sh silently
did not register. Now it binds to EITHER scripts/some_name.<ext> or
scripts/some-name.<ext> (hyphenated wins only if both exist). Resolution is
refactored around a single SkillScriptCandidatePaths generator shared by the
runtime resolver and the validator so they can never drift.

`forge skills validate` now scans the whole skill tree (main SKILL.md +
skills/*/SKILL.md) and surfaces the registration failures the runtime
otherwise handles silently, each with a non-zero exit so CI can gate:
  - invalid **Input:** property key(s) outside ^[a-zA-Z0-9_.-]{1,64}$
    (the runtime drops the whole tool — it would 400 every LLM call)
  - a script-runtime `## Tool:` with no backing script (never registers)
  - orphan scripts with no `## Tool:` heading (dead weight; warning)

This replaces the external lints customers maintained as static
approximations of forge's parser. Docs updated (skills-cli, writing-custom-
skills, skill-md-format, cli-reference).
@initializ-mk
initializ-mk force-pushed the feat/skill-validate-lint branch from 78175dd to 2da8d80 Compare August 21, 2026 17:57

@initializ-mk initializ-mk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — both name forms for tool→script binding + validate lint (closes #418)

Clean, well-engineered fix. Traced to the branch source; CI all green (Build ×6, Integration, Lint, Test, Doc-link). Notable for preserving and extending the security invariants from the earlier skill-import arc rather than regressing them:

  • The #406 traversal fix survives the refactor and is correctly extended to both name forms. The filepath.IsLocal + separator guard now lives in the shared SkillScriptCandidatePaths and runs on every name variant before any os.Stat — see inline. I confirmed the -_ swap can't introduce an escape.
  • Single-source-of-truth is maintained — resolver, read-skill catalog (skillEntryHasScript), and the new validator all go through SkillScriptCandidatePaths, so "what's a first-class tool" can't drift across the three (now with a third consumer).
  • Existing skills resolve identically — hyphen-form candidates are ordered first within each extension, so the only behavior change is that a previously-silently-broken underscore-named file now registers (the #418 win).
  • The lint uses the real parser (InvalidSchemaPropertyKeys(InputSpecToSchema(...))) — no reimplementation — and the non-zero exit is properly wired (hasErrors gates the error return, composing with the existing env-error path), so "CI can gate on it" actually holds.
  • Orphan WARN is variant-aware (marks all candidate paths claimed) and correctly a WARN, not ERROR, so legitimate helper scripts don't fail validation.

One tiny scope note: the second commit (2da8d8, a default-deny PDP example in platform-policy.md) is unrelated to #418 — harmless docs, just flagging it's bundled. Approve-grade; nothing blocking.

// derived stem to be a bare, non-escaping path segment before it touches
// the filesystem. Mirrors the import-path traversal hardening.
for _, n := range names {
if !filepath.IsLocal(n) || strings.ContainsRune(n, filepath.Separator) || strings.ContainsRune(n, '/') {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified this preserves the #406 traversal hardening through the refactor, and correctly extends it to both name forms: the guard now runs on every variant (for _, n := range names) before any candidate path is built or os.Stated, and returns nil if any escapes. I checked that the -_ substitution can't sneak past it — neither char is a path separator or ., so filepath.IsLocal / separator results are identical for the hyphen and underscore forms; a ## Tool: ../../x fails on both. Because this generator is the single source shared by the resolver, skillEntryHasScript, and the new validator, the guard can't be bypassed by any one consumer. 👍

@initializ-mk
initializ-mk merged commit df3070e into main Aug 21, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skills: relax tool→script name binding + surface silent registration failures at forge skills validate

1 participant