feat: add coverage merge command - #1760
Draft
marcossevilla wants to merge 14 commits into
Draft
marcossevilla wants to merge 14 commits into
marcossevilla wants to merge 14 commits into
Conversation
The upcoming `coverage merge` command must report coverage exactly like `very_good test`. Moving the metrics, threshold and uncovered-lines logic into a standalone function lets both commands share it.
Merging sharded reports needs to sum hits per line, function and branch and recompute the summaries. package:lcov_parser can't be used for this: it stops at the first blank line, keeps \r, splits SF paths on every ':' (breaking Windows absolute paths), throws on tags like VER or FNL, and doesn't export its detail models, so records can't be rebuilt from it.
Reports from different runners or packages must key the same file the same way: separators become '/', absolute paths under the current directory become relative, and relative paths can be rebased onto their package so lib/a.dart from two packages stay distinct. Absolute paths outside the current directory are kept and surfaced to the caller.
Sharded and recursive runs leave one lcov report per shard or package, with no way to enforce --min-coverage on the whole suite. 'very_good coverage merge' unions the given reports (files or globs, expanded by the CLI so quoted patterns work on every shell), writes the result and enforces the threshold with the same output as 'very_good test'. --min-coverage, --exclude-coverage and --show-uncovered fall back to the existing test.* and then dart.test.* keys in very_good.yaml. The 'coverage/' rule in .gitignore, meant for generated reports, also matched the new command and test directories, so they are re-included.
After 'very_good test --recursive --coverage' every package has its own coverage/lcov.info. With no arguments, 'coverage merge' now finds them the same way --recursive finds packages and rebases their relative source paths onto their package, so lib/a.dart from two packages stay distinct. The --output report is skipped, with a warning, so merging again doesn't count the previous merge.
The --min-coverage + sharding error now names 'very_good coverage merge' instead of the vague 'merge the lcov reports'. A min_coverage inherited from very_good.yaml is still skipped while sharding, but no longer silently: 'test' and 'dart test' warn once per run, without changing the exit code.
The sharding caution told users to merge reports with lcov and pass --ignore-errors empty. It now points at 'very_good coverage merge', documents the very_good.yaml min_coverage warning, and shows the full matrix, upload, download and merge workflow. A new Coverage page covers the command, its merge semantics and the config keys it reads.
Runs a Flutter fixture unsharded and in 2 shards, then checks that 'coverage merge' rebuilds the unsharded report and fails a 100% threshold on it like 'very_good test' would.
checkCoverage now takes the computed CoverageMetrics instead of lcov_parser records, so callers choose how the metrics are built. CoverageMetrics.fromLcovRecords folds per-file summaries through a private _fromFiles, ready to be shared by other record sources. Also renames coverage_reporter.dart to coverage_check.dart to match its only function, and replaces the list-based branch comparator in LcovRecord.toLcov with a plain _compareBranches helper.
coverage merge wrote the merged report and then re-read it with package:lcov_parser to enforce --min-coverage. That parser splits lines on ":" and keeps the second field, so the external Windows source paths that merge deliberately keeps (e.g. "C:/runner/lib/a.g.dart") were read back as "C": --exclude-coverage globs never matched them and --show-uncovered grouped them all under "C". The threshold is now checked on the merged records in memory, through a new CoverageMetrics.fromLcov, so the merge only goes through parseLcov.
Discovery already left the --output report out, but glob inputs did not: running `very_good coverage merge '**/lcov.info'` twice merged the previous coverage/lcov.info into the new one, doubling its hit counts. Glob matches now skip the --output report too, with the same warning, and fail like an empty glob when it was their only match. A report passed by its exact path is still merged, so it can be overwritten on purpose. Explicit paths are also kept relative to the current directory, like glob matches, so the same report given as an absolute path and through a glob is merged once.
marcossevilla
marked this pull request as draft
October 1, 2026 16:59
- Report file system errors from coverage merge instead of crashing. - Keep an lcov record left without end_of_record before the next SF. - Rebase every package coverage/lcov.info report onto its package and skip glob matches in platform, build and tool directories. - Match exclude_coverage globs against package-relative paths. - Share the --recursive pubspec filter with lcov package discovery. - Parse lcov with parseLcov in the test runner and drop CoverageMetrics.fromLcovRecords. - Move the coverage check helpers into coverage_check.dart. Addresses FINDING-01 to FINDING-08 from review. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Rename lcov_merger.dart to lcov.dart. - Make LcovRecord read-only and sum hits in a private builder. - Read the lcov 2 FN:<line>,<end line>,<name> layout, reject functions without a name, and pin extra BRDA fields as ignored. - Expand lcov globs in the CLI layer, and skip the --output report and ignored directories in one explicit step. - Read coverage options from a single very_good.yaml section. - Compare the e2e merged report per source file. - Link the sharding docs to the coverage CI workflow. Addresses FINDING-09 to FINDING-19 from review. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status
READY
Description
Closes #1754.
Adds
very_good coverage merge [files or globs], which merges the lcov reports from sharded (--shard-index/--total-shards) or recursive (--recursive) test runs into one report. It then checks--min-coverage,--exclude-coverageand--show-uncoveredagainst that report, with the same output and exit codes asvery_good test.DA), per function (FNDA) and per branch (BRDA).LF/LH/FNF/FNH/BRF/BRHare recalculated from those details instead of summed. A 0% padded record (from--collect-coverage-from all) merges cleanly with real hits for the same file. The parser accepts CRLF line endings, blank lines, and:/,in paths.coverage/lcov.info(skipping the same directories astest -r) and prefixes each source path with its package directory, so two packages'lib/files stay separate. A glob or discovered file that matches the--outputreport is skipped. Passing that report explicitly still merges it./. Absolute paths under the current directory become relative. Paths outside it are left unchanged, with a warning.test.*, thendart.test.*invery_good.yaml. No new config keys.--min-coveragecombined with sharding now points tovery_good coverage merge. Sharded runs also warn whenvery_good.yamlsets amin_coveragethat won't be checked. The exit code doesn't change.checkCoverage, which bothtestandcoverage mergecall. The output oftestis unchanged.Docs: new
site/docs/commands/coverage.md; thetest.mdsharding section now walks through matrix → upload → download → merge;configuration.mdlists the keysmergereads. Includes an e2e test that compares merged shard coverage with an unsharded run.Type of Change