Repository navigation
Conversation
The nightly GraalVM jobs split 248 modules over 20 Kokoro builds for each of the three GraalVM versions and run their tests as native images. Since 2023, the issues filed about these jobs show no native image defect that was specific to one generated module, apart from the memory use of the largest ones. Each of the 60 builds is also a chance for a Kokoro infrastructure failure to turn the nightly red. Test nine modules instead, in one build per GraalVM version: - java-aiplatform (gRPC) and java-compute (REST), the two largest generated modules. Their native image builds need the most memory. - java-ftp, a small REST client with google.longrunning operations, numeric enums and the Locations mixin. java-compute has none of these. - java-containeranalysis, java-dns, java-notification, java-resourcemanager, java-samples and java-translate. They contain handwritten code with integration tests and have no GraalVM job of their own. Of the 248 modules, only seven have integration tests (IT*.java), and all seven are in the list. The nightly therefore keeps running every integration test that it runs today. The list is graalvm_nightly_modules in .kokoro/common.sh, and the new graalvm-nightly job type tests it. After this change, no build uses the old graalvm job type or its sharding code. That code stays for now so that this change is only the switch, and a follow-up can remove it. Each nightly job keeps one child build instead of running the tests itself. Kokoro retries a child build that it could not build from the parent job, and that also covers a child build that never started. A job without child builds gets no retry in that case. Do not merge before cl/1025754480. That change reduces each nightly job to one child build and moves it to machines with 32 GB of memory. Until it lands, the nightly still starts the other 19 child builds per GraalVM version, and this change deletes their build configs.
There was a problem hiding this comment.
Code Review
This pull request introduces a new graalvm-nightly job type to run nightly GraalVM checks on a fixed list of modules, consolidating the nightly GraalVM sub-jobs by removing redundant configuration files and updating the remaining configurations. A test is also added to verify that the specified nightly modules exist in the root POM and contain native tests. Feedback was provided regarding .kokoro/common_test.sh, where using find ... | grep -q . in a pipeline under pipefail can lead to flaky test failures due to SIGPIPE when grep exits early, and the non-portable -quit flag should be avoided.
| local root_modules | ||
| root_modules=$(extract_pom_modules pom.xml) | ||
| local module | ||
| for module in "${graalvm_nightly_modules[@]}"; do | ||
| if [[ ! " ${root_modules} " == *" ${module} "* ]]; then | ||
| echo "graalvm_nightly_modules has ${module}, which is not a module of the root pom.xml" | ||
| exit 1 | ||
| fi | ||
| if ! find "${module}" -path '*/src/test/java/*' \( -name 'IT*.java' -o -name '*ClientTest.java' \) -print -quit 2>/dev/null | grep -q .; then | ||
| echo "graalvm_nightly_modules has ${module}, which has no tests that the native profile runs" | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
Using grep -q in a pipeline (find ... | grep -q .) when pipefail is enabled can lead to flaky test failures. As soon as grep -q finds a match, it exits successfully and closes the pipe. If find attempts to write further output, it will receive a SIGPIPE (exit status 141), causing the entire pipeline to fail with a non-zero exit code under pipefail.
Additionally, -quit is a GNU find extension and is not portable (e.g., it will fail on macOS/BSD find unless GNU find is explicitly installed).
To make this check robust, portable, and safe under pipefail, you can use command substitution instead of a pipeline to check if any matching files exist.
| local root_modules | |
| root_modules=$(extract_pom_modules pom.xml) | |
| local module | |
| for module in "${graalvm_nightly_modules[@]}"; do | |
| if [[ ! " ${root_modules} " == *" ${module} "* ]]; then | |
| echo "graalvm_nightly_modules has ${module}, which is not a module of the root pom.xml" | |
| exit 1 | |
| fi | |
| if ! find "${module}" -path '*/src/test/java/*' \( -name 'IT*.java' -o -name '*ClientTest.java' \) -print -quit 2>/dev/null | grep -q .; then | |
| echo "graalvm_nightly_modules has ${module}, which has no tests that the native profile runs" | |
| exit 1 | |
| fi | |
| local root_modules | |
| root_modules=$(extract_pom_modules pom.xml) | |
| local module | |
| local has_tests | |
| for module in "${graalvm_nightly_modules[@]}"; do | |
| if [[ ! " ${root_modules} " == *" ${module} "* ]]; then | |
| echo "graalvm_nightly_modules has ${module}, which is not a module of the root pom.xml" | |
| exit 1 | |
| fi | |
| has_tests=$(find "${module}" -path '*/src/test/java/*' \( -name 'IT*.java' -o -name '*ClientTest.java' \) 2>/dev/null) | |
| if [[ -z "${has_tests}" ]]; then | |
| echo "graalvm_nightly_modules has ${module}, which has no tests that the native profile runs" | |
| exit 1 | |
| fi |
The nightly GraalVM jobs split 248 modules over 20 Kokoro builds for each of the three GraalVM versions and run their tests as native images. Since 2023, the issues filed about these jobs show no native image defect that was specific to one generated module, apart from the memory use of the largest ones. Each of the 60 builds is also a chance for a Kokoro infrastructure failure to turn the nightly red.
This change tests nine modules instead, in one build per GraalVM version:
java-aiplatform(gRPC),java-compute(REST)java-ftpgoogle.longrunningoperations, numeric enums and the Locations mixin.java-computehas none of thesejava-containeranalysis,java-dns,java-notification,java-resourcemanager,java-samples,java-translateOf the 248 modules, only seven have integration tests (
IT*.java), and all seven are in the list. The nightly keeps running every integration test that it runs today.Changes
.kokoro/common.sh: the list,graalvm_nightly_modules, with the reason for each module..kokoro/build.sh: agraalvm-nightlyjob type that tests the list..kokoro/nightly/graalvm-sub-jobs/native-{a,b,c}: use the new job type, drop the sharding variables, and delete the build configs of child builds 2 to 20..kokoro/common_test.sh: checks that each listed module is in the rootpom.xmland has tests that thenativeprofile runs.After this change, no build uses the old
graalvmjob type or its sharding code. That code stays for now so that this change is only the switch. A follow-up can remove it.Why one child build remains
Each nightly job keeps one child build instead of running the tests itself. Kokoro retries a child build that it could not build from the parent job, and that also covers a child build that never started. A job without child builds gets no retry in that case.
Merge order
Merge after cl/1025754480. That change reduces each nightly job to one child build and moves it to machines with 32 GB of memory. Until it lands, the nightly still starts the other 19 child builds per GraalVM version, and this change deletes their build configs.
Testing
.kokoro/common_test.shpasses locally. CI does not run it for this change: the workflow that runs it only does so for changes undersdk-platform-java/orgoogle-auth-library-java/.JOB_TYPE=graalvm-nightly .kokoro/build.shwithmvn,gcloud,terraformandjqreplaced by stubs. It sets up Terraform for the five listed modules that have a.clouddirectory, and runsmvn test -Pnativeon the 31 submodules of the nine modules.main, and a run of this branch needs cl/1025754480 first.b/580634620
b/580635246