-
Notifications
You must be signed in to change notification settings - Fork 0
workflows, ci_scripts: add automation logic from wheel_builder #100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
threexc
wants to merge
12
commits into
main
Choose a base branch
from
tgamblin/migrate_automation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
8deed4e
LICENSE-Apache-2.0: add
threexc dcd6659
ci_scripts: add audit.sh
threexc 1f925d0
ci_scripts: add check_versions.py
threexc 9862170
ci_scripts: check_patch.py: add
threexc 6548483
ci_scripts: audit_report: add
threexc a0fe6f2
ci_scripts: migrate deprecated.txt
threexc 37be7fa
ci_scripts: migrate packages.txt
threexc 4ba8556
ci_scripts: add list-packages.sh
threexc 380a04a
workflows: pr-trigger: add
threexc 77db550
workflows: pr-checks: add
threexc 5079651
workflows: nightly: add
threexc ccbf97d
revertme: add pull_request to nightly workflow for testing
threexc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| # SPDX-FileCopyrightText: 2026 The RISE Project | ||
| # SPDX-License-Identifier: MIT | ||
|
|
||
| name: Run nightly package version and status checks | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, edited, synchronize, reopened] | ||
| schedule: | ||
| - cron: '0 2 * * *' | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| actions: write | ||
| issues: read | ||
|
|
||
| env: | ||
| PIP_EXTRA_INDEX_URL: https://gitlab.com/api/v4/projects/56254198/packages/pypi/simple | ||
|
|
||
| jobs: | ||
| check_versions: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3' | ||
|
|
||
| - name: Install Python dependencies | ||
| run: pip install requests packaging | ||
|
|
||
| - name: Check versions and open PRs | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: python3 ci_scripts/check_versions.py --summary --create-prs | ||
|
|
||
| check_deprecated_packages: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install curl and jq | ||
| run: sudo apt-get update -qq && sudo apt-get install -qq -y curl jq | ||
|
|
||
| - name: Verify riscv64 wheels still published | ||
| run: | | ||
| ret=0 | ||
| while IFS= read -r pkg; do | ||
| [[ -z "$pkg" || "$pkg" =~ ^[[:space:]]*# ]] && continue | ||
|
|
||
| if ! output=$(curl -sf "https://pypi.org/pypi/${pkg}/json"); then | ||
| echo "Failed to fetch $pkg, skipping..." | ||
| continue | ||
| fi | ||
|
|
||
| if ! filenames=$(echo "$output" | jq -r '.urls[].filename'); then | ||
| echo "Failed to parse JSON for $pkg, skipping..." | ||
| continue | ||
| fi | ||
|
|
||
| if ! echo "$filenames" | grep -qi riscv; then | ||
| echo "$pkg does not distribute riscv64 wheel anymore!" | ||
| ret=1 | ||
| fi | ||
| done < ci_scripts/deprecated.txt | ||
| exit $ret | ||
|
|
||
| check_installation: | ||
| runs-on: ubuntu-24.04-riscv | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install packages inside riscv64 container | ||
| working-directory: ci_scripts | ||
| run: | | ||
| sed -Ei 's/^scipy.*$//' packages.txt | ||
| sed -Ei 's/^patchelf.*$//' packages.txt | ||
| ./list-packages.sh python-wheels | ||
|
|
||
| pip_audit: | ||
| runs-on: ubuntu-24.04-riscv | ||
| continue-on-error: true | ||
| permissions: | ||
| contents: read | ||
| issues: write | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Run pip-audit | ||
| working-directory: ci_scripts | ||
| run: ./audit.sh python-wheels | ||
|
|
||
| - name: Report vulnerabilities and open issues | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: python3 ci_scripts/audit_report.py ci_scripts/audit-report.json |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # SPDX-FileCopyrightText: 2026 The RISE Project | ||
| # SPDX-License-Identifier: MIT | ||
|
|
||
| name: PR verification checks | ||
|
|
||
| on: | ||
| pull_request: | ||
|
|
||
| jobs: | ||
| check_commit_messages: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Reject revertme / DO NOT MERGE commits | ||
| env: | ||
| BASE_SHA: ${{ github.event.pull_request.base.sha }} | ||
| HEAD_SHA: ${{ github.event.pull_request.head.sha }} | ||
| run: | | ||
| set -euo pipefail | ||
| range="$BASE_SHA..$HEAD_SHA" | ||
| if git log --pretty=format:%s "$range" | grep -i -e '^revertme' -e '^revert me'; then | ||
| echo "::error::Merge request contains at least a commit starting with a 'revert me' tag. Fix it before merging." | ||
| exit 1 | ||
| fi | ||
| if git log --pretty=format:%s "$range" | grep -i -e '^DO NOT MERGE'; then | ||
| echo "::error::Merge request contains at least a commit starting with a 'DO NOT MERGE' tag. Review it." | ||
| exit 1 | ||
| fi | ||
|
|
||
| check_patches: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3' | ||
|
|
||
| - name: Validate Upstream-Status in added/modified patches | ||
| env: | ||
| BASE_SHA: ${{ github.event.pull_request.base.sha }} | ||
| HEAD_SHA: ${{ github.event.pull_request.head.sha }} | ||
| run: python ci_scripts/check_patch.py "$BASE_SHA" "$HEAD_SHA" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| # SPDX-FileCopyrightText: 2026 The RISE Project | ||
|
threexc marked this conversation as resolved.
|
||
| # SPDX-License-Identifier: MIT | ||
|
threexc marked this conversation as resolved.
|
||
|
|
||
| name: PR package build trigger | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, edited, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| actions: write | ||
| pull-requests: read | ||
|
|
||
| jobs: | ||
| dispatch-triggers: | ||
| runs-on: ubuntu-latest | ||
| if: contains(github.event.pull_request.body, 'Trigger:') | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| package_version: ${{ github.event.pull_request.head.sha }} | ||
|
|
||
| - name: Dispatch build/test workflows for Trigger directives | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| PR_BODY: ${{ github.event.pull_request.body }} | ||
| HEAD_REF: ${{ github.event.pull_request.head.ref }} | ||
| HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }} | ||
| BASE_REPO: ${{ github.repository }} | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| if [ "$HEAD_REPO" != "$BASE_REPO" ]; then | ||
| echo "PR from fork ($HEAD_REPO); cannot dispatch workflows against fork ref. Skipping." | ||
| exit 0 | ||
| fi | ||
|
|
||
| triggers=$(printf '%s\n' "$PR_BODY" | grep -oE '^Trigger:[[:space:]]*[^[:space:]]+' || true) | ||
|
|
||
| if [ -z "$triggers" ]; then | ||
| echo "No Trigger: directives found." | ||
| exit 0 | ||
| fi | ||
|
|
||
| printf '%s\n' "$triggers" | while IFS= read -r line; do | ||
| directive=${line#Trigger:} | ||
| directive=${directive# } | ||
| package_name=${directive%:*} | ||
| package_version=${directive#*:} | ||
|
|
||
| if [ -z "$package_name" ] || [ -z "$package_version" ] || [ "$package_name" = "$package_version" ]; then | ||
| echo "Skipping malformed directive: $line" | ||
| continue | ||
| fi | ||
|
|
||
| # Normalize: build workflows prepend `v` to the version themselves | ||
| # (e.g. `ref: v${{ env.NUMPY_VERSION }}`). Strip any leading v/V | ||
| # in the directive so `Trigger: numpy:v2.5.1` and | ||
| # `Trigger: numpy:2.5.1` behave identically. | ||
| package_version=${package_version#[vV]} | ||
|
|
||
| build_wf=".github/workflows/build-${package_name}.yml" | ||
| test_wf=".github/workflows/test-${package_name}.yml" | ||
|
|
||
| if [ ! -f "$build_wf" ]; then | ||
| echo "No build workflow for $package_name at $build_wf; skipping." | ||
| continue | ||
| fi | ||
|
|
||
| echo "Dispatching build-${package_name}.yml on $HEAD_REF with version=$package_version" | ||
| gh workflow run "build-${package_name}.yml" --ref "$HEAD_REF" -f "version=$package_version" | ||
|
|
||
| if [ -f "$test_wf" ]; then | ||
| echo "Dispatching test-${package_name}.yml on $HEAD_REF with version=$package_version" | ||
| gh workflow run "test-${package_name}.yml" --ref "$HEAD_REF" -f "version=$package_version" | ||
| fi | ||
| done | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.