Skip to content

Commit 7b8e4e9

Browse files
devopvoiddevopvoid
authored
ci: harden the release workflow and keep native caches warm (#302)
* ci: harden the release workflow Pass the release notes and the development version to the scripts through the environment instead of expanding them into the script text. The notes are commit subjects, so a quote or a backtick in one broke the release step and ran as shell code. Take the release version from the pom, which is what release:prepare tags, instead of letting git-cliff bump it from the commit types. The GitHub release now always matches the Maven tag, and --verify-tag makes a mismatch fail instead of creating an unbuilt tag on main. Grant write access only to the jobs that push, and keep two releases from running at once. * ci: keep the native build caches warm Build on every push to main that can change a build, not only on source changes. A PR can restore caches only from its own ref and from main, so when main skipped a build change, every following PR compiled WebRTC from scratch under a key that already existed on another PR's ref. Pom, CMake and submodule changes now also deploy a snapshot. Save the WebRTC and FFmpeg caches right after the build instead of in the post step of actions/cache, which saves only when the whole job succeeds; a failing test threw the finished native build away. Cancel a PR's running build when a newer push supersedes it, and give the workflow a read-only token. * ci: push a release only once it is prepared Fold the changelog job into prepare-release and commit CHANGELOG.md locally, before release:prepare. The changelog commit, the release commit, the tag and the development commit now go out in one atomic push at the end, so a failure on the way leaves main untouched instead of carrying a changelog for a release that never happened. Check the input before anything else: developmentVersion must be a SNAPSHOT that differs from the release version, the release tag must not exist yet, and release:prepare must choose the version the changelog was written for. Hand the pushed tag to the platform and publish jobs as an output instead of having each of them guess the newest tag in the repository. Run release:prepare with "clean validate" as its preparation goals. The default "clean verify" compiled WebRTC and FFmpeg on an uncached runner, about 35 of the 52 minutes of the last release, only for the platform jobs to build the tagged tree again. * ci: skip docs-only native builds and make test failures debuggable Leave docs-only PRs out of the native build and build the docs site on PRs that touch it instead, without deploying, so a broken VitePress build shows up before the merge. Cache the Chromium Clang package and stop listing every file it extracts, keep Homebrew from updating itself before each install, and drop the two disabled steps of the Windows preparation. Give every test run a 30 minute Surefire timeout, and test-natives a job timeout, so a hung test no longer holds a runner for six hours. Composite action steps have no timeout-minutes, and macOS has no timeout command. Upload the Surefire reports when a job fails.
1 parent 3501136 commit 7b8e4e9

11 files changed

Lines changed: 330 additions & 177 deletions

File tree

‎.github/actions/build-macos-x86_64/action.yml‎

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,13 @@ runs:
2727
run: echo "WEBRTC_CACHE_BRANCH=$(sed -n 's|.*<webrtc.branch>branch-heads/\([^<]*\)</webrtc.branch>.*|\1|p' webrtc-jni/pom.xml)" >> "$GITHUB_ENV"
2828
shell: bash
2929

30-
- name: Set up WebRTC cache
31-
uses: actions/cache@v6
30+
# Restored here, saved right after the build below: the post step of
31+
# actions/cache only saves when the whole job succeeds, so a failing test
32+
# would otherwise throw away a WebRTC build that took the better part of an
33+
# hour.
34+
- id: webrtc-cache
35+
name: Restore WebRTC cache
36+
uses: actions/cache/restore@v6
3237
with:
3338
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
3439
key: webrtc-${{ env.WEBRTC_CACHE_BRANCH }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-jni/src/main/cpp/dependencies/webrtc/CMakeLists.txt') }}-${{ hashFiles('webrtc-jni/pom.xml') }}
@@ -38,8 +43,9 @@ runs:
3843
run: echo "FFMPEG_VERSION=$(sed -n 's|.*<ffmpeg.version>\([^<]*\)</ffmpeg.version>.*|\1|p' webrtc-java-media/pom.xml)" >> "$GITHUB_ENV"
3944
shell: bash
4045

41-
- name: Set up FFmpeg cache
42-
uses: actions/cache@v6
46+
- id: ffmpeg-cache
47+
name: Restore FFmpeg cache
48+
uses: actions/cache/restore@v6
4349
with:
4450
path: ~/ffmpeg
4551
key: ffmpeg-${{ env.FFMPEG_VERSION }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-java-media/src/main/cpp/dependencies/ffmpeg/CMakeLists.txt') }}-${{ hashFiles('webrtc-java-media/pom.xml') }}
@@ -80,10 +86,34 @@ runs:
8086
run: mvn package -DskipTests -Pmacos-cross-x86_64
8187
shell: bash
8288

89+
- name: Save WebRTC cache
90+
if: steps.webrtc-cache.outputs.cache-hit != 'true'
91+
uses: actions/cache/save@v6
92+
with:
93+
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
94+
key: ${{ steps.webrtc-cache.outputs.cache-primary-key }}
95+
96+
- name: Save FFmpeg cache
97+
if: steps.ffmpeg-cache.outputs.cache-hit != 'true'
98+
uses: actions/cache/save@v6
99+
with:
100+
path: ~/ffmpeg
101+
key: ${{ steps.ffmpeg-cache.outputs.cache-primary-key }}
102+
83103
- name: Test
84-
run: JAVA_HOME="${{ steps.jdk-x64.outputs.path }}" mvn -B jar:jar surefire:test -Pmacos-cross-x86_64
104+
run: JAVA_HOME="${{ steps.jdk-x64.outputs.path }}" mvn -B jar:jar surefire:test -Pmacos-cross-x86_64 -Dsurefire.timeout=1800
85105
shell: bash
86106

107+
# What failed, for runners that cannot be reproduced locally.
108+
- name: Upload test reports
109+
if: failure()
110+
uses: actions/upload-artifact@v7
111+
with:
112+
name: test-reports-${{ inputs.platform-name }}
113+
path: '**/target/surefire-reports/'
114+
if-no-files-found: ignore
115+
retention-days: 7
116+
87117
- name: Deploy
88118
if: ${{ github.event_name != 'pull_request' && github.repository == 'devopvoid/webrtc-java' }}
89119
env:

‎.github/actions/build/action.yml‎

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,13 @@ runs:
2727
run: echo "WEBRTC_CACHE_BRANCH=$(sed -n 's|.*<webrtc.branch>branch-heads/\([^<]*\)</webrtc.branch>.*|\1|p' webrtc-jni/pom.xml)" >> "$GITHUB_ENV"
2828
shell: bash
2929

30-
- name: Set up WebRTC cache
31-
uses: actions/cache@v6
30+
# Restored here, saved right after the build below: the post step of
31+
# actions/cache only saves when the whole job succeeds, so a failing test
32+
# would otherwise throw away a WebRTC build that took the better part of an
33+
# hour.
34+
- id: webrtc-cache
35+
name: Restore WebRTC cache
36+
uses: actions/cache/restore@v6
3237
with:
3338
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
3439
key: webrtc-${{ env.WEBRTC_CACHE_BRANCH }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-jni/src/main/cpp/dependencies/webrtc/CMakeLists.txt') }}-${{ hashFiles('webrtc-jni/pom.xml') }}
@@ -38,8 +43,9 @@ runs:
3843
run: echo "FFMPEG_VERSION=$(sed -n 's|.*<ffmpeg.version>\([^<]*\)</ffmpeg.version>.*|\1|p' webrtc-java-media/pom.xml)" >> "$GITHUB_ENV"
3944
shell: bash
4045

41-
- name: Set up FFmpeg cache
42-
uses: actions/cache@v6
46+
- id: ffmpeg-cache
47+
name: Restore FFmpeg cache
48+
uses: actions/cache/restore@v6
4349
with:
4450
path: ~/ffmpeg
4551
key: ffmpeg-${{ env.FFMPEG_VERSION }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-java-media/src/main/cpp/dependencies/ffmpeg/CMakeLists.txt') }}-${{ hashFiles('webrtc-java-media/pom.xml') }}
@@ -74,11 +80,35 @@ runs:
7480
fi
7581
shell: bash
7682

83+
- name: Save WebRTC cache
84+
if: steps.webrtc-cache.outputs.cache-hit != 'true'
85+
uses: actions/cache/save@v6
86+
with:
87+
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
88+
key: ${{ steps.webrtc-cache.outputs.cache-primary-key }}
89+
90+
- name: Save FFmpeg cache
91+
if: steps.ffmpeg-cache.outputs.cache-hit != 'true'
92+
uses: actions/cache/save@v6
93+
with:
94+
path: ~/ffmpeg
95+
key: ${{ steps.ffmpeg-cache.outputs.cache-primary-key }}
96+
7797
- name: Test
7898
if: ${{ inputs.platform-name != 'linux_arm' && inputs.platform-name != 'linux_arm64' && inputs.platform-name != 'windows_arm64' }}
79-
run: mvn -B jar:jar surefire:test
99+
run: mvn -B jar:jar surefire:test -Dsurefire.timeout=1800
80100
shell: bash
81101

102+
# What failed, for runners that cannot be reproduced locally.
103+
- name: Upload test reports
104+
if: failure()
105+
uses: actions/upload-artifact@v7
106+
with:
107+
name: test-reports-${{ inputs.platform-name }}
108+
path: '**/target/surefire-reports/'
109+
if-no-files-found: ignore
110+
retention-days: 7
111+
82112
- name: Deploy
83113
if: ${{ github.event_name != 'pull_request' && github.repository == 'devopvoid/webrtc-java' }}
84114
env:

‎.github/actions/prepare-linux/action.yml‎

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -30,20 +30,35 @@ runs:
3030

3131
- name: Install required packages
3232
run: |
33-
sudo apt update
33+
sudo apt-get update -q
3434
# nasm is what FFmpeg assembles its x86 code with.
35-
sudo apt install -y binutils cmake git locales lsb-release nasm ninja-build pipewire pipewire-pulse pkg-config python3 python3-setuptools rsync unzip wget xz-utils
35+
sudo apt-get install -y -q binutils cmake git locales lsb-release nasm ninja-build pipewire pipewire-pulse pkg-config python3 python3-setuptools rsync unzip wget xz-utils
36+
shell: bash
37+
38+
# Chromium Clang to be used with the clang toolchain file. The JNI code
39+
# compiles against the libc++ headers of the WebRTC build, which only
40+
# support the Clang that WebRTC branch ships with, so this must be the
41+
# package pinned by CLANG_REVISION and CLANG_SUB_REVISION in
42+
# tools/clang/scripts/update.py of the webrtc.branch set in
43+
# webrtc-jni/pom.xml. Update it whenever that branch changes.
44+
- name: Select the Chromium Clang package
45+
run: echo "CLANG_PACKAGE=clang-llvmorg-23-init-19482-g53d18800-1.tar.xz" >> "$GITHUB_ENV"
46+
shell: bash
3647

37-
# Chromium Clang to be used with the clang toolchain file. The JNI code
38-
# compiles against the libc++ headers of the WebRTC build, which only
39-
# support the Clang that WebRTC branch ships with, so this must be the
40-
# package pinned by CLANG_REVISION and CLANG_SUB_REVISION in
41-
# tools/clang/scripts/update.py of the webrtc.branch set in
42-
# webrtc-jni/pom.xml. Update it whenever that branch changes.
43-
CLANG_PACKAGE=clang-llvmorg-23-init-19482-g53d18800-1.tar.xz
48+
- name: Cache the Chromium Clang package
49+
uses: actions/cache@v6
50+
with:
51+
path: ~/.cache/chromium-clang
52+
key: chromium-clang-${{ env.CLANG_PACKAGE }}
53+
54+
- name: Install Chromium Clang
55+
run: |
56+
mkdir -p ~/.cache/chromium-clang
57+
if [ ! -f ~/.cache/chromium-clang/$CLANG_PACKAGE ]; then
58+
wget -q -O ~/.cache/chromium-clang/$CLANG_PACKAGE https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/$CLANG_PACKAGE
59+
fi
4460
sudo mkdir -p /opt/clang
45-
wget https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/$CLANG_PACKAGE
46-
sudo tar -xvf $CLANG_PACKAGE -C /opt/clang
61+
sudo tar -xf ~/.cache/chromium-clang/$CLANG_PACKAGE -C /opt/clang
4762
shell: bash
4863

4964
- name: Install required packages for x86-64

‎.github/actions/prepare-macos/action.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,11 @@ description: 'Installs required packages for macOS builds.'
55
runs:
66
using: "composite"
77
steps:
8+
# No Homebrew self-update: it takes minutes and neither package needs it.
89
- name: Install required packages
10+
env:
11+
HOMEBREW_NO_AUTO_UPDATE: '1'
12+
HOMEBREW_NO_INSTALL_CLEANUP: '1'
913
run: |
1014
# Required on macos-14
1115
brew install python-setuptools

‎.github/actions/prepare-windows/action.yml‎

Lines changed: 0 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -5,35 +5,6 @@ description: 'Frees up disk space and installs required packages for Windows bui
55
runs:
66
using: "composite"
77
steps:
8-
- name: Set up Python 3.11
9-
if: false
10-
uses: actions/setup-python@v7
11-
with:
12-
python-version: "3.11"
13-
14-
- name: Disk cleanup
15-
if: false
16-
run: |
17-
Get-PSDrive
18-
# Docker Images
19-
docker rmi $(docker images -q -a)
20-
# Android SDK
21-
if ($Env:ANDROID_HOME) {
22-
Remove-Item -Recurse -Force $Env:ANDROID_HOME -ErrorAction Ignore
23-
}
24-
if ($Env:ANDROID_NDK_HOME) {
25-
Remove-Item -Recurse -Force $Env:ANDROID_NDK_HOME -ErrorAction Ignore
26-
}
27-
# JVM
28-
if ($Env:JAVA_HOME_11_X64) {
29-
Remove-Item -Recurse -Force $Env:JAVA_HOME_11_X64 -ErrorAction Ignore
30-
}
31-
if ($Env:JAVA_HOME_8_X64) {
32-
Remove-Item -Recurse -Force $Env:JAVA_HOME_8_X64 -ErrorAction Ignore
33-
}
34-
Get-PSDrive
35-
shell: powershell
36-
378
- name: Install required packages
389
run: |
3910
choco install ninja

‎.github/actions/release-macos-x86_64/action.yml‎

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,13 @@ runs:
1919
run: echo "WEBRTC_CACHE_BRANCH=$(sed -n 's|.*<webrtc.branch>branch-heads/\([^<]*\)</webrtc.branch>.*|\1|p' webrtc-jni/pom.xml)" >> "$GITHUB_ENV"
2020
shell: bash
2121

22-
- name: Set up WebRTC cache
23-
uses: actions/cache@v6
22+
# Restored here, saved right after the build below: the post step of
23+
# actions/cache only saves when the whole job succeeds, so a failing test
24+
# would otherwise throw away a WebRTC build that took the better part of an
25+
# hour.
26+
- id: webrtc-cache
27+
name: Restore WebRTC cache
28+
uses: actions/cache/restore@v6
2429
with:
2530
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
2631
key: webrtc-${{ env.WEBRTC_CACHE_BRANCH }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-jni/src/main/cpp/dependencies/webrtc/CMakeLists.txt') }}-${{ hashFiles('webrtc-jni/pom.xml') }}
@@ -30,8 +35,9 @@ runs:
3035
run: echo "FFMPEG_VERSION=$(sed -n 's|.*<ffmpeg.version>\([^<]*\)</ffmpeg.version>.*|\1|p' webrtc-java-media/pom.xml)" >> "$GITHUB_ENV"
3136
shell: bash
3237

33-
- name: Set up FFmpeg cache
34-
uses: actions/cache@v6
38+
- id: ffmpeg-cache
39+
name: Restore FFmpeg cache
40+
uses: actions/cache/restore@v6
3541
with:
3642
path: ~/ffmpeg
3743
key: ffmpeg-${{ env.FFMPEG_VERSION }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-java-media/src/main/cpp/dependencies/ffmpeg/CMakeLists.txt') }}-${{ hashFiles('webrtc-java-media/pom.xml') }}
@@ -67,10 +73,34 @@ runs:
6773
run: mvn package -DskipTests -Pmacos-cross-x86_64
6874
shell: bash
6975

76+
- name: Save WebRTC cache
77+
if: steps.webrtc-cache.outputs.cache-hit != 'true'
78+
uses: actions/cache/save@v6
79+
with:
80+
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
81+
key: ${{ steps.webrtc-cache.outputs.cache-primary-key }}
82+
83+
- name: Save FFmpeg cache
84+
if: steps.ffmpeg-cache.outputs.cache-hit != 'true'
85+
uses: actions/cache/save@v6
86+
with:
87+
path: ~/ffmpeg
88+
key: ${{ steps.ffmpeg-cache.outputs.cache-primary-key }}
89+
7090
- name: Test
71-
run: JAVA_HOME="${{ steps.jdk-x64.outputs.path }}" mvn -B jar:jar surefire:test -Pmacos-cross-x86_64
91+
run: JAVA_HOME="${{ steps.jdk-x64.outputs.path }}" mvn -B jar:jar surefire:test -Pmacos-cross-x86_64 -Dsurefire.timeout=1800
7292
shell: bash
7393

94+
# What failed, for runners that cannot be reproduced locally.
95+
- name: Upload test reports
96+
if: failure()
97+
uses: actions/upload-artifact@v7
98+
with:
99+
name: test-reports-${{ inputs.platform-name }}
100+
path: '**/target/surefire-reports/'
101+
if-no-files-found: ignore
102+
retention-days: 7
103+
74104
# Maven Central publishes a version as one atomic bundle, so this job must not
75105
# deploy. The native library jar is handed to the aggregating publish-central
76106
# job instead, which attaches every platform to a single deployment.

‎.github/actions/release/action.yml‎

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,13 @@ runs:
1818
run: echo "WEBRTC_CACHE_BRANCH=$(sed -n 's|.*<webrtc.branch>branch-heads/\([^<]*\)</webrtc.branch>.*|\1|p' webrtc-jni/pom.xml)" >> "$GITHUB_ENV"
1919
shell: bash
2020

21-
- name: Set up WebRTC cache
22-
uses: actions/cache@v6
21+
# Restored here, saved right after the build below: the post step of
22+
# actions/cache only saves when the whole job succeeds, so a failing test
23+
# would otherwise throw away a WebRTC build that took the better part of an
24+
# hour.
25+
- id: webrtc-cache
26+
name: Restore WebRTC cache
27+
uses: actions/cache/restore@v6
2328
with:
2429
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
2530
key: webrtc-${{ env.WEBRTC_CACHE_BRANCH }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-jni/src/main/cpp/dependencies/webrtc/CMakeLists.txt') }}-${{ hashFiles('webrtc-jni/pom.xml') }}
@@ -29,8 +34,9 @@ runs:
2934
run: echo "FFMPEG_VERSION=$(sed -n 's|.*<ffmpeg.version>\([^<]*\)</ffmpeg.version>.*|\1|p' webrtc-java-media/pom.xml)" >> "$GITHUB_ENV"
3035
shell: bash
3136

32-
- name: Set up FFmpeg cache
33-
uses: actions/cache@v6
37+
- id: ffmpeg-cache
38+
name: Restore FFmpeg cache
39+
uses: actions/cache/restore@v6
3440
with:
3541
path: ~/ffmpeg
3642
key: ffmpeg-${{ env.FFMPEG_VERSION }}-${{ inputs.platform-name }}-${{ hashFiles('webrtc-java-media/src/main/cpp/dependencies/ffmpeg/CMakeLists.txt') }}-${{ hashFiles('webrtc-java-media/pom.xml') }}
@@ -62,11 +68,35 @@ runs:
6268
fi
6369
shell: bash
6470

71+
- name: Save WebRTC cache
72+
if: steps.webrtc-cache.outputs.cache-hit != 'true'
73+
uses: actions/cache/save@v6
74+
with:
75+
path: ~/${{ env.WEBRTC_INSTALL_FOLDER }}
76+
key: ${{ steps.webrtc-cache.outputs.cache-primary-key }}
77+
78+
- name: Save FFmpeg cache
79+
if: steps.ffmpeg-cache.outputs.cache-hit != 'true'
80+
uses: actions/cache/save@v6
81+
with:
82+
path: ~/ffmpeg
83+
key: ${{ steps.ffmpeg-cache.outputs.cache-primary-key }}
84+
6585
- name: Test
6686
if: ${{ inputs.platform-name != 'linux_arm' && inputs.platform-name != 'linux_arm64' && inputs.platform-name != 'windows_arm64' }}
67-
run: mvn -B jar:jar surefire:test
87+
run: mvn -B jar:jar surefire:test -Dsurefire.timeout=1800
6888
shell: bash
6989

90+
# What failed, for runners that cannot be reproduced locally.
91+
- name: Upload test reports
92+
if: failure()
93+
uses: actions/upload-artifact@v7
94+
with:
95+
name: test-reports-${{ inputs.platform-name }}
96+
path: '**/target/surefire-reports/'
97+
if-no-files-found: ignore
98+
retention-days: 7
99+
70100
# Maven Central publishes a version as one atomic bundle, so this job must not
71101
# deploy. The native library jar is handed to the aggregating publish-central
72102
# job instead, which attaches every platform to a single deployment.

‎.github/actions/test-natives/action.yml‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -121,5 +121,15 @@ runs:
121121
# The native build is off: these natives were cross compiled by another
122122
# job, and this runner is here to run them, not to build them again.
123123
- name: Test
124-
run: mvn -B -pl webrtc,webrtc-java-media test -DskipNativeBuild -P${{ inputs.profile }}
124+
run: mvn -B -pl webrtc,webrtc-java-media test -DskipNativeBuild -P${{ inputs.profile }} -Dsurefire.timeout=1800
125125
shell: bash
126+
127+
# What failed, for runners that cannot be reproduced locally.
128+
- name: Upload test reports
129+
if: failure()
130+
uses: actions/upload-artifact@v7
131+
with:
132+
name: test-reports-native-${{ inputs.platform-name }}
133+
path: '**/target/surefire-reports/'
134+
if-no-files-found: ignore
135+
retention-days: 7

0 commit comments

Comments
 (0)