-
Notifications
You must be signed in to change notification settings - Fork 60
Creates a weekly deps upgrade skill and workflow #263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Diego Luces (dluces)
wants to merge
6
commits into
main
Choose a base branch
from
user/dilucesr/upgrade_skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
cfd5e16
Add weekly dependency upgrade skill
c078985
Add weekly dependency upgrade skill
98f5224
Require sample validation evidence for agent PRs
30c9795
Merge remote upgrade skill branch
6a25808
Document supported PowerShell and .NET runtimes
Copilot f123a17
Changes before error encountered
Copilot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,3 @@ | ||
| * text=auto eol=lf | ||
| * text=auto eol=lf | ||
|
|
||
| .github/workflows/*.lock.yml linguist-generated=true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| name: dependency-upgrade | ||
| description: Performs compatible dependency updates across all samples and produces concrete validation evidence for review. | ||
| --- | ||
|
|
||
| You are the dependency upgrade maintainer for this repository. | ||
|
|
||
| Follow `.github/skills/weekly-dependency-upgrade/SKILL.md` and | ||
| `.github/skills/sample-validation-evidence/SKILL.md` completely. Inventory all | ||
| npm and NuGet manifests, update only compatible current-major releases, and use | ||
| `Tools/powershell/Invoke-RepositoryValidation.ps1` instead of inventing | ||
| replacement checks. | ||
|
|
||
| Treat validation evidence as a deliverable. Publish only sanitized command | ||
| output, HTTP transcripts, process logs, and visually reviewed browser | ||
| screenshots from `.validation/sanitized`. In the pull request, distinguish | ||
| `PASS`, `SKIP_CONFIG`, `SKIP_ENV`, and `FAIL`; never convert a skipped check or | ||
| unresolved audit finding into a pass. | ||
|
|
||
| Make precise dependency and lockfile changes only. Do not commit credentials, | ||
| local settings, `.env` files, generated dependency directories, or transient | ||
| `.validation` artifacts. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # SharePoint Embedded Samples instructions | ||
|
|
||
| Follow the repository-wide requirements in `AGENTS.md` for every change. | ||
|
|
||
| Before creating or updating any pull request, use the | ||
| `sample-validation-evidence` skill in | ||
| `.github/skills/sample-validation-evidence/SKILL.md`. Run all sample validation | ||
| scripts through `Tools/powershell/Invoke-RepositoryValidation.ps1`, inspect the | ||
| result, and publish only the generated sanitized artifacts with | ||
| `Tools/powershell/Publish-ValidationArtifacts.ps1`. | ||
|
|
||
| A pull request is incomplete until its description or a comment contains: | ||
|
|
||
| - The exact `VALIDATION_RESULT` for every sample. | ||
| - Concrete build, test, lint, audit, and HTTP evidence that was produced. | ||
| - Embedded screenshots for successful browser smoke checks. | ||
| - Explicit reasons for `SKIP_CONFIG` and `SKIP_ENV` results. | ||
|
|
||
| Never claim that a skipped check passed. Never publish raw `.validation` | ||
| contents, local configuration, credentials, access tokens, cookies, connection | ||
| strings, tenant identifiers, or personal data. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| name: sample-validation-evidence | ||
| description: Run every sample validator, sanitize its evidence, and attach concrete validation artifacts to a pull request. | ||
| --- | ||
|
|
||
| # Sample validation evidence | ||
|
|
||
| Use this skill before any agent opens or updates a pull request in this | ||
| repository. | ||
|
|
||
| ## Run all validators | ||
|
|
||
| From the repository root, run: | ||
|
|
||
| ```pwsh | ||
| pwsh -NoProfile -File Tools/powershell/Invoke-RepositoryValidation.ps1 | ||
| ``` | ||
|
|
||
| The command runs every tracked `validate-sample.ps1`, captures its complete | ||
| output, records the exact `VALIDATION_RESULT`, and creates a sanitized evidence | ||
| bundle under `.validation/sanitized/<timestamp>/`. | ||
|
|
||
| Pass validator switches only when necessary: | ||
|
|
||
| ```pwsh | ||
| pwsh -NoProfile -File Tools/powershell/Invoke-RepositoryValidation.ps1 -SkipInstall | ||
| ``` | ||
|
|
||
| Do not use `-SkipBrowser` merely to save time. A skipped browser check does not | ||
| satisfy screenshot validation. | ||
|
|
||
| ## Review the evidence | ||
|
|
||
| 1. Open `.validation/sanitized/<timestamp>/validation-report.md`. | ||
| 2. Confirm that every sample is present. | ||
| 3. Treat `PASS`, `FAIL`, `SKIP_CONFIG`, and `SKIP_ENV` exactly as emitted. | ||
| 4. Inspect every PNG before upload. The sanitizer copies screenshots because | ||
| automatic text redaction cannot inspect pixels. Delete any image containing | ||
| secrets, tenant content, personal data, or other non-public information. | ||
| 5. Search the sanitized directory for credentials or tenant data. If anything | ||
| sensitive remains, remove it and improve | ||
| `Tools/powershell/Sanitize-ValidationArtifacts.ps1` before publishing. | ||
|
|
||
| Never upload directly from an app's raw `.validation` directory. | ||
|
|
||
| ## Add evidence to the pull request | ||
|
|
||
| The pull request description or a follow-up comment must include the sanitized | ||
| `validation-report.md` content and meaningful sanitized HTTP excerpts. | ||
|
|
||
| After visually reviewing every screenshot, publish the sanitized report, HTTP | ||
| evidence, and screenshots with: | ||
|
|
||
| ```pwsh | ||
| pwsh -NoProfile -File Tools/powershell/Publish-ValidationArtifacts.ps1 ` | ||
| -PullRequest <pull-request-url-or-number> ` | ||
| -ScreenshotsReviewed | ||
| ``` | ||
|
|
||
| For a pull request number, pass `-Repository owner/repo` when the checkout remote | ||
| is not the target repository. The publisher refuses directories outside | ||
| `.validation/sanitized`, uploads PNGs through GitHub's user attachments API, and | ||
| posts `validation-evidence.md` plus embedded screenshots as a PR comment. | ||
|
|
||
| Confirm the GitHub page contains the report, HTTP evidence, and rendered | ||
| screenshots before declaring the pull request complete. | ||
|
|
||
| If artifact upload fails, report the failure and keep working. Do not replace | ||
| the evidence with an unsupported statement that validation passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| --- | ||
| name: weekly-dependency-upgrade | ||
| description: Update dependencies across every runnable sample, validate each updated sample, preserve concrete validation evidence, and prepare a reviewer-ready pull request. | ||
| --- | ||
|
|
||
| # Weekly dependency upgrade | ||
|
|
||
| Use this skill for a repository-wide dependency update iteration. Work from the | ||
| current `main` branch checkout and treat every sample application as in scope. | ||
|
|
||
| ## Goals | ||
|
|
||
| 1. Update direct npm and NuGet dependencies to the newest compatible release in | ||
| their current major version. | ||
| 2. Refresh every affected lockfile. | ||
| 3. Resolve vulnerable transitive dependencies when a compatible override, | ||
| resolution, or direct dependency update is available. | ||
| 4. Validate every updated sample with the repository's own validation entrypoint. | ||
| 5. Put concrete build, test, audit, HTTP, and browser evidence in the pull request. | ||
|
|
||
| Do not perform major-version migrations in this workflow. If a vulnerability can | ||
| only be resolved through a major migration, document it as an unresolved security | ||
| exception with the dependency chain and available fixed version. | ||
|
|
||
| ## Repository inventory | ||
|
|
||
| Find manifests recursively while excluding generated and dependency directories: | ||
|
|
||
| - `package.json` and corresponding npm lockfiles | ||
| - `*.csproj`, `Directory.Packages.props`, and NuGet lockfiles | ||
|
|
||
| Current runnable samples and validation entrypoints are: | ||
|
|
||
| - `AI/mcp-server/validate-sample.ps1` | ||
| - `AI/ocr/validate-sample.ps1` | ||
| - `Custom Apps/boilerplate-aspnet-webservice/validate-sample.ps1` | ||
| - `Custom Apps/boilerplate-react-azurefunction/validate-sample.ps1` | ||
| - `Custom Apps/boilerplate-typescript-react/validate-sample.ps1` | ||
| - `Custom Apps/legal-docs/validate-sample.ps1` | ||
| - `Custom Apps/project-management/validate-sample.ps1` | ||
| - `Custom Apps/webhook/validate-sample.ps1` | ||
|
|
||
| If the inventory discovers another runnable sample with a | ||
| `validate-sample.ps1`, include it. Do not edit generated dependency directories. | ||
|
|
||
| ## Update procedure | ||
|
|
||
| 1. Record the current direct dependency versions and available compatible | ||
| updates for every manifest. | ||
| 2. Apply compatible current-major updates with npm and .NET tooling rather than | ||
| hand-editing lockfiles. | ||
| 3. Preserve the package manager and lockfile format already used by each sample. | ||
| 4. Review peer-dependency and engine warnings. Do not use `--force` to conceal an | ||
| incompatible graph. | ||
| 5. Run `npm audit` for each npm project and NuGet vulnerability checks for each | ||
| .NET project after updating. | ||
| 6. Use a targeted npm override only when it produces a valid installed graph and | ||
| fixes a vulnerable transitive package. Record why the override is needed. | ||
| 7. Review the final diff for accidental generated files, credentials, local | ||
| configuration, and unrelated changes. | ||
|
|
||
| ## Validation procedure | ||
|
|
||
| Use the repository-wide validation and sanitization skill in | ||
| `.github/skills/sample-validation-evidence/SKILL.md`. Run: | ||
|
|
||
| ```pwsh | ||
| pwsh -NoProfile -File Tools/powershell/Invoke-RepositoryValidation.ps1 | ||
| ``` | ||
|
|
||
| Do not pass `-SkipBrowser` unless browser tooling is genuinely unavailable. The | ||
| orchestrator captures all validator output and produces sanitized evidence under | ||
| `.validation/sanitized/`. Preserve those files until the workflow's artifact | ||
| upload and pull request publication complete. | ||
|
|
||
| Interpret `VALIDATION_RESULT` exactly: | ||
|
|
||
| - `PASS`: the checks named by the validator passed. | ||
| - `SKIP_CONFIG`: configuration-independent checks passed, but one or more runtime | ||
| checks were not executed. List each skipped check and reason. | ||
| - `SKIP_ENV`: the environment could not run the validator. List the missing | ||
| runtime or tool and do not report the sample as passed. | ||
| - `FAIL`: the sample failed validation. Do not open a success-shaped pull request. | ||
|
|
||
| Configuration-dependent behavior that is expected in an unconfigured runner: | ||
|
|
||
| - MCP runtime needs `AI/mcp-server/.env`. | ||
| - OCR backend runtime needs `AI/ocr/.env`; its frontend can use the validator's | ||
| temporary non-secret placeholder client ID. | ||
| - ASP.NET runtime needs a usable `appsettings.json` and SQL connection. | ||
| - Azure Functions runtimes need Azure Functions Core Tools and local settings. | ||
| - Authentication-dependent Vite clients may use the validators' non-secret | ||
| placeholder IDs for unauthenticated render and screenshot checks. | ||
| - Legal docs needs Node 20.19+ or 22.12+. | ||
|
|
||
| Never claim a skipped or configuration-blocked check passed. | ||
|
|
||
| ## Evidence report | ||
|
|
||
| Create `.validation/weekly-report.md` with: | ||
|
|
||
| 1. A manifest-by-manifest table of old and new direct dependency versions. | ||
| 2. A per-sample validation table with the exact `VALIDATION_RESULT`. | ||
| 3. The commands executed and meaningful terminal excerpts proving builds, tests, | ||
| lint, and audits ran. | ||
| 4. HTTP request and response excerpts, including method, URL, status, headers | ||
| when useful, and response body. | ||
| 5. A list of screenshot artifact paths and what each screenshot demonstrates. | ||
| 6. All skipped checks, warnings, and unresolved vulnerabilities, including their | ||
| dependency chains. | ||
| 7. A link to the current Actions run and the uploaded artifact named | ||
| `dependency-validation-artifacts`. | ||
|
|
||
| The pull request body must contain the report's useful evidence directly, not | ||
| only statements such as "validation passed." The pull request body MUST also contain | ||
| the sanitized validation artifacts attached (or in a follow up comment), and not just | ||
| reference the Actions artifacts. In addition to that, the PR description may point to | ||
| the Actions artifacts for complete logs and full-resolution screenshots. | ||
|
|
||
| ## Pull request behavior | ||
|
|
||
| If no dependency or lockfile changes are available, request the workflow's | ||
| `noop` safe output and explain that the repository is current. | ||
|
|
||
| If changes are available: | ||
|
|
||
| - Request one non-draft pull request against `main`. | ||
| - Use a source branch beginning with | ||
| `automation/weekly-dependency-upgrades-`. | ||
| - Request reviewer `gnjoseph` and `dluces`. | ||
| - Include the dependency table, exact validation results, HTTP excerpts, | ||
| screenshot inventory, audit results, skips, and Actions artifact link. | ||
| - Do not request a pull request when any updated sample has a `FAIL` result. | ||
| Instead, create a failure issue or report the failure through the workflow's | ||
| available safe output. | ||
|
|
||
| Exclude `.validation` evidence from the code patch. It is transient workflow | ||
| evidence. Upload only `.validation/sanitized/` files through the workflow and | ||
| attach the sanitized report and visually reviewed screenshots to the pull | ||
| request. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.