Repository navigation
Enforce blocking quality gates for external plugin submissions and PRs - #5252
Merged
Merged
Conversation
Require manifest compliance and all applicable checks to pass for submissions and PRs. Fail the PR enforcement job when checks fail or cannot complete. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
🚦 Submission status: 👀 Ready for reviewRisk tier: Why this tier
Automated checks
Review
Commands
Updated for |
Contributor
|
🟡 Contributor Reputation Check: MEDIUM risk
Maintainers: please review this contributor before merging. |
github-actions
Bot
added
needs-review:MEDIUM
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Required-check integration and manifest selection still have correctness issues that can block valid PRs or plugin submissions.
3 open issues
What changed in this PR
This PR strengthens external-plugin validation across submission intake and listing-update PRs so failing or incomplete quality checks cannot be treated as successful.
Changes:
- Make manifest compliance blocking and require explicit passing aggregate results.
- Add a PR enforcement job and align bot messages and infrastructure-error reporting.
- Document stricter behavior and add regression coverage.
| File | Description |
|---|---|
eng/external-plugin-quality-gates.test.mjs |
Add manifest, skill-lint, and aggregate-status regression cases. |
eng/external-plugin-quality-gates.mjs |
Make manifest failures blocking and include compliance in overall status. |
eng/external-plugin-pr-quality-gates.mjs |
Require every changed plugin to pass aggregation. |
eng/external-plugin-intake.test.mjs |
Cover failed, successful, and incomplete intake checks. |
eng/external-plugin-intake.mjs |
Require explicit success before advancing intake and update messages. |
CONTRIBUTING.md |
Explain blocking gates and PR status enforcement. |
.github/workflows/external-plugin-pr-quality-gates.yml |
Add the enforcement job for unsuccessful or incomplete checks. |
.github/workflows/external-plugin-pr-quality-gates-writer.yml |
Remove non-blocking wording and display compliance failures accurately. |
.github/workflows/external-plugin-intake.yml |
Report compliance infrastructure errors consistently. |
.github/workflows/external-plugin-command-router.yml |
Align rerun infrastructure-error fallbacks with blocking compliance. |
Review effort: Max
Comment on lines
+187
to
+190
| enforce-quality-gates: | ||
| runs-on: ubuntu-latest | ||
| needs: [detect-changed-plugins, run-quality-gates] | ||
| if: always() |
|
|
||
| function inspectAgentPluginSpecCompliance(pluginRoot) { | ||
| export function inspectAgentPluginSpecCompliance(pluginRoot) { | ||
| const pluginJsonPath = findPluginJson(pluginRoot); |
| ? "This submission passed blocking quality checks and is ready for maintainer review, but it has non-blocking Agent Plugins spec compliance warnings." | ||
| : "This submission passed automated intake validation and quality checks and is ready for maintainer review."; | ||
| : qualityResult.overall_status !== "pass" | ||
| ? "This submission passed metadata validation, but the automated quality checks did not complete successfully. A maintainer should rerun intake or use the explicit override command after review." |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
npm startand verified thatREADME.mdis up to date.mainbranch for this pull request.Description
External plugin submissions and listing-update PRs should not advance while applicable quality checks are failing. Manifest compliance was previously non-blocking, and the PR workflow could report plugin failures without failing a status check.
vally lint(including Agent Skills spec and reference checks), installation smoke tests, and the other applicable gates.ready-for-reviewor PR aggregation succeeds. Update bot comments and infrastructure-error fallbacks to match.enforce-quality-gatesPR job that fails on unsuccessful detection, incomplete execution, or a non-passing aggregate result.Type of Contribution
Additional Notes
Validation
node --test --test-skip-pattern="pinned homepage dispatcher" eng\external-plugin-quality-gates.test.mjs eng\external-plugin-intake.test.mjs eng\external-plugin-intake-state.test.mjs- passed: 108 tests, 108 passed, 0 failed, 0 skipped as reported by Node. The command retains the exclusion pattern for a previously unreliable local network test.npm run build- passed; generated README and marketplace outputs remain unchanged. Existing external-listing metadata warnings were emitted.bash eng/fix-line-endings.sh- passed.git --no-pager diff --check- passed.npm startwas not run separately; it delegates to the successfulnpm run build. The new enforcement job has not yet run in GitHub Actions. Resource-specific and paid-service checklist items are not applicable to this automation change.Rollout: Configure
enforce-quality-gatesas a required status check in branch protection to prevent merges while it fails. Existing explicit maintainer intake overrides remain available.By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.