Support vitest 5#86
Conversation
GuillaumeLagrange
changed the title
| } | ||
| } | ||
|
|
||
| function patchRunBenchmarks(): void { |
There was a problem hiding this comment.
By the way, you can do that just in a setup file without overriding the runner. This should also make it support the browser(?)
If there is any feedback, I would love to address them on vitest side; we would also like to use codspeed after Vitest 5 is out in our own repo
GuillaumeLagrange
force-pushed
the
cod-2931-prepare-compatibility-with-vitest-5
branch
4 times, most recently
from
c48683e to
25ef927
Compare
Vitest 5 reworked the benchmark backend: the dedicated `NodeBenchmarkRunner` and the `vitest/runners` / `vitest/suite` entrypoints are gone, benchmarks now run inside `test()` through the unified `TestRunner`, and tinybench moved to v6 (stats moved from `result.benchmark` to `task.result.latency`, `includeSamples` became `retainSamples`). Detect the installed Vitest generation and select the integration seam behind a `VitestBackend` abstraction so the rest of the plugin never inspects the version: - v3/4 keep the custom benchmark runner per instrument mode (analysis/walltime). - v5 installs instrumentation from a setup file that patches the shared `TestRunner.runBenchmarks` static. A setup file (rather than a custom `test.runner`) leaves the runner untouched for non-benchmark tests and also applies to the browser pool.
GuillaumeLagrange
force-pushed
the
cod-2931-prepare-compatibility-with-vitest-5
branch
from
25ef927 to
e04f0da
Compare
Now that the plugin's own dev dependency tracks Vitest 5, add a dedicated Vitest 4 example so the legacy (v3/4) benchmark seam keeps explicit coverage alongside the existing with-vitest-v3 example. Mirrors that example, pinning vitest ^4.1.9. Refs COD-2931 Co-Authored-By: Claude <noreply@anthropic.com>
Merging this PR will degrade performance by 59.17%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | test_recursive_fibo_20 |
19 B | 65,624 B | -99.97% |
| ❌ | Memory | long body |
960 B | 26,344 B | -96.36% |
| ❌ | Simulation | wait 500ms |
10.2 ms | 13.3 ms | -23.09% |
| ❌ | Simulation | wait 1sec |
20.8 ms | 24.5 ms | -14.92% |
| ❌ | WallTime | switch 1 |
144 ns | 168 ns | -14.29% |
| ❌ | WallTime | switch 2 |
168 ns | 192 ns | -12.5% |
| ❌ | WallTime | test_recursive_cached_fibo_10 |
1 µs | 1.2 µs | -10.31% |
| ⚡ | Simulation | switch 1 |
313.8 µs | 105.3 µs | ×3 |
| ⚡ | Memory | wait 500ms |
64.1 KB | 23 KB | ×2.8 |
| ⚡ | WallTime | test sync baz 10 |
228 ns | 192 ns | +18.75% |
| ⚡ | WallTime | test_iterative_fibo_10 |
276 ns | 240 ns | +15% |
| 🆕 | WallTime | iterative fibo 15 |
N/A | 840 ns | N/A |
| 🆕 | WallTime | iterative fibo 20 |
N/A | 816 ns | N/A |
| 🆕 | WallTime | recursive fibo 15 |
N/A | 22.3 µs | N/A |
| 🆕 | WallTime | recursive fibo 20 |
N/A | 239.8 µs | N/A |
| 🆕 | Memory | iterative fibo 15 |
N/A | 0 B | N/A |
| 🆕 | Memory | iterative fibo 20 |
N/A | 0 B | N/A |
| 🆕 | Memory | recursive fibo 15 |
N/A | 19 B | N/A |
| 🆕 | Memory | recursive fibo 20 |
N/A | 19 B | N/A |
| 🆕 | Simulation | iterative fibo 15 |
N/A | 17 µs | N/A |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing cod-2931-prepare-compatibility-with-vitest-5 (6791750) with main (1999ec9)
Footnotes
-
53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
2 participants
No description provided.