Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ Use `.github/skills/code-review/SKILL.md` for repository-fit, differentiation,
provenance, validation, and editorial-quality review criteria. Apply the
deterministic file-format checks below as well.

Frontmatter must be valid YAML and satisfy the required fields, types, lengths,
and naming rules. Accept any valid YAML scalar style (plain, single-quoted,
double-quoted, literal, or folded); do not flag a value solely because it is not
single-quoted.

## README updates

- [ ] The new file should be added to the `docs/README.<type>.md`.
Expand Down Expand Up @@ -47,14 +52,23 @@ deterministic file-format checks below as well.

**Only apply to folders in the `skills/` directory**

Review `SKILL.md` against the [Agent Skills specification](https://agentskills.io/specification),
not just general YAML syntax. Valid YAML alone is insufficient: parsed values must
meet the specification's field types, structure, and constraints. Scalar style is
not itself a defect. The repository's 10-character description minimum is an
additional requirement beyond the specification's 1-character minimum.

- [ ] The skill folder contains a `SKILL.md` file.
- [ ] The SKILL.md has markdown front matter.
- [ ] The SKILL.md has a `name` field.
- [ ] The `name` field value is lowercase with words separated by hyphens.
- [ ] The `name` field is a string of 1-64 characters using lowercase letters, numbers, and hyphens, with no leading, trailing, or consecutive hyphens.
- [ ] The `name` field matches the folder name.
- [ ] The SKILL.md has a `description` field.
- [ ] The `description` field is not empty, at least 10 characters, and maximum 1024 characters.
- [ ] The `description` field value is wrapped in single quotes.
- [ ] The `description` field is a string, at least 10 characters and maximum 1024 characters, describing what the skill does and when to use it.
- [ ] If present, `license` is a string identifying a license or referencing a bundled license file.
- [ ] If present, `compatibility` is a string of 1-500 characters describing environment requirements.
- [ ] If present, `metadata` is a mapping from string keys to string values (for example, quote a version if needed to keep it a string).
- [ ] If present, `allowed-tools` is a space-separated string, not a YAML sequence (experimental).
- [ ] The folder name is lower case, with words separated by hyphens.
- [ ] Any bundled assets (scripts, templates, data files) are referenced in the SKILL.md instructions.
- [ ] Bundled assets are reasonably sized (under 5MB per file).
Expand Down
2 changes: 2 additions & 0 deletions .github/extensions/open-pr-board/lib/review-prompt.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ function sharedRules(repo, guidancePath) {
return [
"Work strictly read-only: inspect with `gh` and file reads only. Do not post comments, submit reviews, push commits, or modify the repository.",
`Review criteria, in priority order, live in \`.github/skills/code-review/SKILL.md\` and \`.github/copilot-instructions.md\` in ${repo}. Read the board's own procedure at ${guidancePath} first.`,
"Frontmatter must be valid YAML and meet the required metadata fields, types, lengths, and naming rules. Accept plain, single-quoted, double-quoted, literal, and folded scalars when they meet those requirements. Do not request changes solely because a value is not single-quoted.",
"For skills, review SKILL.md against the Agent Skills specification at https://agentskills.io/specification plus the repository's additional skill checks in `.github/copilot-instructions.md`. Valid YAML alone is not enough: check field types, naming constraints, lengths, and optional-field structure. In particular, `metadata` must map string keys to string values, and `allowed-tools` must be a space-separated string, not a YAML sequence. Do not impose a quote style.",
"For each pull request gather real evidence before deciding:",
" - `gh pr view <number> -R " + repo + " --comments` for the description and discussion.",
" - `gh pr diff <number> -R " + repo + "` for the actual change (use `--name-only` first if the diff is large).",
Expand Down
12 changes: 11 additions & 1 deletion .github/extensions/open-pr-board/review-guidance.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,17 @@ For every pull request:
without a source change is `close`.
- **Format checks** — run the deterministic checklist from `.github/copilot-instructions.md`
for whichever resource types the PR adds. These are objective and are the most common
reason for `request-changes`.
reason for `request-changes`. Frontmatter must be valid YAML and meet the required
metadata fields, types, lengths, and naming rules. Accept plain, single-quoted,
double-quoted, literal, and folded scalars when they meet those requirements.
Do not request changes solely because a value is not single-quoted.
- **Skill structure** — review `SKILL.md` against the
[Agent Skills specification](https://agentskills.io/specification) and the
repository's additional skill checks. Valid YAML alone is not enough: check
field types, naming constraints, lengths, and optional-field structure.
In particular, `metadata` maps string keys to string values, and `allowed-tools`
is a space-separated string, not a YAML sequence. These constraints do not
require a particular scalar quote style.
- **Duplication** — search the repo for an existing agent/instruction/skill covering the
same ground. Near-duplicates are `needs-insight` unless the overlap is total.
- **Repo fit** — non-developer domains (growth marketing, dating, career/LinkedIn, ad
Expand Down
12 changes: 12 additions & 0 deletions .github/skills/code-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,18 @@ deterministic checklists in `.github/copilot-instructions.md` first, then use
this skill for the editorial and repository-fit judgments that cannot be
reduced to schema validation.

Validate frontmatter as YAML and check the required metadata, not quote style.
Plain, single-quoted, double-quoted, literal, and folded scalars are all acceptable
when they parse to values that meet the field's type and content requirements.
Do not request changes solely because a value is not single-quoted.

For skills, use the [Agent Skills specification](https://agentskills.io/specification)
for `SKILL.md` structure and field constraints, plus the repository-specific
checks in `.github/copilot-instructions.md`. General YAML validity does not
replace the specification: for example, `metadata` must map strings to strings,
and `allowed-tools` must be a space-separated string, not a sequence. Check
parsed types and values without imposing a quote style.

## Review priorities

Review in this order:
Expand Down
24 changes: 13 additions & 11 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,24 +58,26 @@ npm run skill:create -- --name <skill-name>

All agent files (`*.agent.md`) and instruction files (`*.instructions.md`) must include proper markdown front matter. Agent Skills are folders containing a `SKILL.md` file with frontmatter and optional bundled assets. Hooks are folders containing a `README.md` with frontmatter and a `hooks.json` configuration file:

Frontmatter must be valid YAML and meet the metadata requirements below. Any valid YAML scalar style is accepted; single quotes are not required.

#### Agent Files (\*.agent.md)

- Must have `description` field (wrapped in single quotes)
- Must have `description` field
- File names should be lower case with words separated by hyphens
- Recommended to include `tools` field
- Strongly recommended to specify `model` field

#### Instruction Files (\*.instructions.md)

- Must have `description` field (wrapped in single quotes, not empty)
- Must have `description` field (not empty)
- Must have `applyTo` field specifying file patterns (e.g., `'**.js, **.ts'`)
- File names should be lower case with words separated by hyphens

#### Agent Skills (skills/\*/SKILL.md)

- Each skill is a folder containing a `SKILL.md` file
- SKILL.md must have `name` field (lowercase with hyphens, matching folder name, max 64 characters)
- SKILL.md must have `description` field (wrapped in single quotes, 10-1024 characters)
- SKILL.md must have `description` field (10-1024 characters)
- Folder names should be lower case with words separated by hyphens
- Skills can include bundled assets (scripts, templates, data files)
- Bundled assets should be referenced in the SKILL.md instructions
Expand All @@ -95,7 +97,7 @@ All agent files (`*.agent.md`) and instruction files (`*.instructions.md`) must

- Each hook is a folder containing a `README.md` file with frontmatter
- README.md must have `name` field (human-readable name)
- README.md must have `description` field (wrapped in single quotes, not empty)
- README.md must have `description` field (not empty)
- Must include a `hooks.json` file with hook configuration (hook events extracted from this file)
- Folder names should be lower case with words separated by hyphens
- Can include bundled assets (scripts, utilities, configuration files)
Expand All @@ -107,7 +109,7 @@ All agent files (`*.agent.md`) and instruction files (`*.instructions.md`) must

- Each workflow is a standalone `.md` file in the `workflows/` directory
- Must have `name` field (human-readable name)
- Must have `description` field (wrapped in single quotes, not empty)
- Must have `description` field (not empty)
- Contains agentic workflow frontmatter (`on`, `permissions`, `safe-outputs`) and natural language instructions
- File names should be lower case with words separated by hyphens
- Only `.md` files are accepted — `.yml`, `.yaml`, and `.lock.yml` files are blocked by CI
Expand Down Expand Up @@ -237,7 +239,7 @@ Before committing:

- Use proper front matter with required fields
- Keep descriptions concise and informative
- Wrap description field values in single quotes
- Use valid YAML scalar styles for frontmatter values
- Use lower-case file names with hyphens as separators

### JavaScript/Node.js Scripts
Expand Down Expand Up @@ -276,14 +278,14 @@ Before submitting your PR, ensure you have:
For instruction files (\*.instructions.md):

- [ ] Has markdown front matter
- [ ] Has non-empty `description` field wrapped in single quotes
- [ ] Has non-empty `description` field
- [ ] Has `applyTo` field with file patterns
- [ ] File name is lower case with hyphens

For agent files (\*.agent.md):

- [ ] Has markdown front matter
- [ ] Has non-empty `description` field wrapped in single quotes
- [ ] Has non-empty `description` field
- [ ] Has `name` field with human-readable name (e.g., "Address Comments" not "address-comments")
- [ ] File name is lower case with hyphens
- [ ] Includes `model` field (strongly recommended)
Expand All @@ -294,7 +296,7 @@ For skills (skills/\*/):
- [ ] Folder contains a SKILL.md file
- [ ] SKILL.md has markdown front matter
- [ ] Has `name` field matching folder name (lowercase with hyphens, max 64 characters)
- [ ] Has non-empty `description` field wrapped in single quotes (10-1024 characters)
- [ ] Has non-empty `description` field (10-1024 characters)
- [ ] Folder name is lower case with hyphens
- [ ] Any bundled assets are referenced in SKILL.md
- [ ] Bundled assets are under 5MB per file
Expand All @@ -303,7 +305,7 @@ For hook folders (hooks/\*/):

- [ ] Folder contains a README.md file with markdown front matter
- [ ] Has `name` field with human-readable name
- [ ] Has non-empty `description` field wrapped in single quotes
- [ ] Has non-empty `description` field
- [ ] Has `hooks.json` file with valid hook configuration (hook events extracted from this file)
- [ ] Folder name is lower case with hyphens
- [ ] Any bundled scripts are executable and referenced in README.md
Expand All @@ -314,7 +316,7 @@ For workflow files (workflows/\*.md):

- [ ] File has markdown front matter
- [ ] Has `name` field with human-readable name
- [ ] Has non-empty `description` field wrapped in single quotes
- [ ] Has non-empty `description` field
- [ ] File name is lower case with hyphens
- [ ] Contains `on` and `permissions` in frontmatter
- [ ] Workflow uses least-privilege permissions and safe outputs
Expand Down
19 changes: 19 additions & 0 deletions eng/open-pr-board.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,25 @@ const promptItem = {
};
const promptArgs = { instanceId: "board-1", repo: "example/repo", guidancePath: "guidance.md" };

test("initial and re-review agent briefs accept valid YAML scalar styles while preserving metadata checks", () => {
const prompts = [
buildReviewPrompt({ ...promptArgs, items: [promptItem], decisions: [] }),
buildRereviewPrompt({ ...promptArgs, item: promptItem }),
];
for (const prompt of prompts) {
const [, brief] = prompt.split("--- Agent brief ---");
assert.match(brief, /Frontmatter must be valid YAML/);
assert.match(brief, /required metadata fields, types, lengths, and naming rules/);
assert.match(brief, /Accept plain, single-quoted, double-quoted, literal, and folded scalars/);
assert.match(brief, /Do not request changes solely because a value is not single-quoted/);
assert.match(brief, /For skills, review SKILL\.md against the Agent Skills specification at https:\/\/agentskills\.io\/specification/);
assert.match(brief, /repository's additional skill checks/);
assert.match(brief, /Valid YAML alone is not enough/);
assert.match(brief, /`metadata` must map string keys to string values/);
assert.match(brief, /`allowed-tools` must be a space-separated string, not a YAML sequence/);
}
});

test("review prompts include vendor-funnel, author-batch, and recording checks", () => {
const prompts = [
buildReviewPrompt({ ...promptArgs, items: [promptItem], decisions: [] }),
Expand Down
31 changes: 31 additions & 0 deletions eng/yaml-parser.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,37 @@ description: A sample skill used to exercise bundled asset discovery.
# Sample skill
`;

for (const [style, scalar] of Object.entries({
plain: "A sample skill description.",
singleQuoted: "'A sample skill description.'",
doubleQuoted: '"A sample skill description."',
literal: "|-\n A sample skill description.",
folded: ">-\n A sample skill\n description.",
})) {
test(`parses ${style} frontmatter scalars without requiring single quotes`, () => {
const skillPath = createSkill({
"SKILL.md": `---\nname: sample-skill\ndescription: ${scalar}\n---\n`,
});
const metadata = parseSkillMetadata(skillPath);
assert.equal(metadata.name, "sample-skill");
assert.equal(metadata.description, "A sample skill description.");
});
}

test("rejects invalid YAML and missing required skill metadata regardless of scalar style", (t) => {
t.mock.method(console, "error", () => {});
t.mock.method(console, "warn", () => {});
for (const frontmatter of [
'name: sample-skill\ndescription: "Unclosed string',
"name: sample-skill\ndescription: Invalid: mapping",
'description: "A sample skill description."',
"name: sample-skill",
]) {
const skillPath = createSkill({ "SKILL.md": `---\n${frontmatter}\n---\n` });
assert.equal(parseSkillMetadata(skillPath), null);
}
});

test("lists bundled assets from the spec-defined subdirectories", () => {
const skillPath = createSkill({
"SKILL.md": SKILL_MD,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ The YAML frontmatter controls how Copilot discovers and executes your skill.
name: generate-tests
```

**description**: Brief summary of what the skill does and when to use it (10–1024 characters, wrapped in single quotes)
**description**: String summarizing what the skill does and when to use it (10–1024 characters; any valid YAML scalar style is accepted)
```yaml
description: 'Generate comprehensive unit tests for a component, covering happy path, edge cases, and error conditions'
```
Expand Down
Loading