Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
63078b6
Add serial test execution support
giordano Apr 2, 2026
b974dc8
Improve tests for serial jobs
giordano Apr 2, 2026
46c1afc
Simplify implementation
giordano Apr 2, 2026
0af9ec8
Add documentation for serial execution
giordano Apr 2, 2026
c25ed29
[docs] Use `--verbose` in serial tests examples
giordano Apr 2, 2026
aa83bc3
Cap number of parallel jobs to number of parallel tests
giordano Apr 2, 2026
7dc5834
Add some more tests for serial jobs
giordano Apr 2, 2026
f6f0796
Add test for case of parallel tests less than requested jobs
giordano Apr 3, 2026
415de6b
Improve test for serial names filtered by positional args
giordano Apr 3, 2026
2a9d3e3
Fix case of all tests serial
giordano Apr 3, 2026
3d1a844
Reuse same worker for all the serial tests
giordano Apr 3, 2026
6e002f3
Add tests for checking serial tests run alone
giordano Apr 3, 2026
0fa5f49
Slightly simplify some tests
giordano Apr 3, 2026
029a261
Also test serial tests run after are alone
giordano Apr 3, 2026
7b2ea8f
Use `@show_if_error` macro in more places
giordano Apr 3, 2026
13c2c5f
Merge branch 'main' into mg/serial-test
giordano Apr 5, 2026
d69b22d
Stop parallel workers when serial tests are at the end
giordano Apr 7, 2026
5406c95
Remove redundant test
giordano Apr 7, 2026
266274b
Merge branch 'main' into mg/serial-test
giordano Jun 21, 2026
da01182
Fix some indentations in tests
giordano Jun 21, 2026
c2f4c92
Merge remote-tracking branch 'origin/main' into mg/serial-test
giordano Jul 15, 2026
16b9b15
Merge commit '3efd40e801a8546498e082a5c80be8a4c3792336' into mg/seria…
christiangnrd Jul 28, 2026
3caadb9
Merge branch 'main' into mg/serial-test
christiangnrd Jul 28, 2026
cc581c5
Merge branch 'main' into mg/serial-test
christiangnrd Jul 29, 2026
e24a2ce
Add tests for --quickfail with serial phases
giordano Aug 1, 2026
ec3edda
Merge branch 'main' into mg/serial-test
christiangnrd Aug 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions docs/src/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,53 @@ end # hide
```
The `init_worker_code` is evaluated once per worker, so all definitions can be imported for use by the test module.

## Serial Tests

Some tests cannot safely run in parallel with other tests — for example, tests that allocate very large arrays and would exhaust memory if multiple ran simultaneously.
The `serial` keyword argument to [`runtests`](@ref) lets you designate specific tests to run one at a time, while the remaining tests still run in parallel.

```@example mypackage
using ParallelTestRunner
using MyPackage

testsuite = Dict(
"big_alloc" => quote
# This test allocates ~4 GB and should not overlap with other tests
@test true
end,
"huge_matrix" => quote
@test true
end,
"fast_unit" => quote
@test 1 + 1 == 2
end,
"fast_integration" => quote
@test true
end,
)

# "big_alloc" and "huge_matrix" run one at a time; the rest run in parallel
runtests(MyPackage, ["--verbose"]; testsuite, serial=["big_alloc", "huge_matrix"])
```

By default serial tests run **before** the parallel batch.
Use `serial_position=:after` to run them after instead:

```@example mypackage
runtests(MyPackage, ["--verbose"]; testsuite, serial=["big_alloc", "huge_matrix"], serial_position=:after)
```

Serial tests participate in the same ordering logic as parallel tests (sorted by historical
duration, longest first) and their results appear in the same overall summary.

!!! tip
With automatic test discovery via [`find_tests`](@ref), the `serial` names are the same
keys that appear in the testsuite dictionary (e.g. `"subdir/memory_test"`).

!!! note
If the user filters tests via positional arguments (e.g. `julia test/runtests.jl unit`),
any serial test names that were filtered out are silently removed from the serial list.

## Custom Workers

For tests that require specific environment variables or Julia flags, you can use the `test_worker` keyword argument to [`runtests`](@ref) to assign tests to custom workers:
Expand Down Expand Up @@ -254,3 +301,5 @@ function jltest {
Having few long-running test files and other short-running ones hinders scalability.

1. **Use custom workers sparingly**: Custom workers add overhead. Only use them when tests genuinely require different configurations.

1. **Use `serial` for resource-intensive tests**: If a test allocates significant memory or uses exclusive hardware resources, mark it as serial rather than reducing `--jobs` globally. This keeps the rest of your suite running in parallel.
5 changes: 3 additions & 2 deletions docs/src/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,13 @@ execute
parent(::ParallelTestRunner.AbstractTestRecord)
```

## Internal Types
## Internal Functionalities

These are internal types, not subject to semantic versioning contract (could be changed or removed at any point without notice), not intended for consumption by end-users.
These are internal types or functions, not subject to semantic versioning contract (could be changed or removed at any point without notice), not intended for consumption by end-users.
They are documented here exclusively for `ParallelTestRunner` developers and contributors.

```@docs
ParsedArgs
WorkerTestSet
partition_tests
```
7 changes: 7 additions & 0 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,13 @@ Pkg.test("MyPackage"; test_args=`--verbose --jobs=4 integration`)
Tests run concurrently in isolated worker processes, each inside own module.
`ParallelTestRunner` records historical tests duration for each package, so that in subsequent runs long-running tests are executed first, to improve load balancing.

### Serial Test Support

Certain tests (e.g. memory-hungry tests) may need to run one at a time.
The `serial` keyword argument to [`runtests`](@ref) lets you designate specific tests
for sequential execution, either before or after the parallel batch.
See [Serial Tests](@ref) in the advanced usage guide for details.

### Real-time Progress

The test runner provides real-time output showing:
Expand Down
Loading