feat(stats): add --all-projects and --aggregate flags

[glours]

What this PR does

Adds two opt-in flags to docker compose stats. Without them the command is unchanged: current project, one line per container.

• --all-projects: one line per container for every project, with a PROJECT column
• --aggregate: one line per project, summing CPU, memory, network, block I/O and PIDs of its containers. Combined with --all-projects it covers all projects; alone it reports on the current project

Both support table and json (one object per line, like docker stats) and refresh until interrupted unless --no-stream is set. They can't be combined with a SERVICE, and --all-projects can't be combined with --project-name.

docker/cli doesn't export its stats collection, so containers are sampled through the Engine API and the figures are computed the way docker stats does.

Related issue

Closes #12321

(not mandatory) A picture of a cute animal, if possible in relation to what you did

[docker-agent]

Assessment: 🟢 APPROVE

[codecov]

Codecov Report

❌ Patch coverage is 86.41975% with 22 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
cmd/compose/stats_projects.go	87.89%	13 Missing and 6 partials ⚠️
cmd/compose/stats.go	40.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[docker-agent]

Assessment: 🟡 NEEDS ATTENTION

[ndeloof]

Nice feature, and the code is clean: up-front validation (flags, format), deterministic ordering, JSON one object per line like docker stats. The unit tests pass locally and go vet is clean. I did not run it against a daemon. A few points to address:

1. CPU on older engines. CPU% relies on the daemon returning the previous sample (IncludePreviousSample). If an engine ignores the option, PreCPUStats stays at zero, cpuDelta and systemDelta are both positive, and the result is a since-start average rather than an instantaneous value. Could you check this on the engine versions Compose supports, or document it?
2. Unbounded concurrency. collectContainerStats starts one goroutine per container on every refresh (2s). With --all-projects on a busy host that can be hundreds of concurrent calls; eg.SetLimit(...) would bound it.
3. One failure stops the whole loop. Any error other than NotFound on a single container cancels the other samples and exits. Rendering that row without usage (as for stopped containers) would be more robust for a refresh loop.

Minor:

• With --all, a stopped container shows 0B / 0B; as far as I remember docker stats shows --.
• The main loop of runStatsProjects (--no-stream, screen clear, refresh) isn't covered by tests, only its building blocks.
• The formulas follow docker stats for Linux only. A link to the upstream functions in a comment would help limit drift, and Windows hosts get no warning.

_{Drafted with Claude Code; read and validated before posting.}

[glours]

Thanks for the review.

1. CPU on older engines

Compose doesn't send anything new here: IncludePreviousSample only drops one-shot=true, so the request is a plain stream=false. In that case the daemon discards the first frame (noStreamFirstFrame in daemon/stats.go, present in v19.03.15, v20.10.0 and v25.0.0), so precpu_stats is populated. On 29.7.2: stream=false returns it (after ~1s), one-shot=true leaves it at zero. I read the daemon code for the versions above and ran 29.7.2 only; I did not run older engines.

2. Unbounded concurrency

Fixed: at most 32 samples at a time (statsConcurrency). A sample takes about a second on the daemon side, so a refresh of n containers now lasts at least n/32 seconds.

3. One failure stops the loop

Fixed: a container that can't be sampled is listed without usage and the others are still refreshed. An interrupted command is still reported as such rather than rendered as containers without usage.

Minor

• 0B / 0B vs --: docker stats -a --no-stream (CLI 29.8.1) prints 0.00% 0B / 0B 0.00% 0B / 0B 0B / 0B 0 for a stopped container, same as here (the daemon returns empty stats for a non-running container without streaming). I did not check the streaming mode. Left as is.
• Main loop: added TestRunStatsProjectsNoStream (rows of all projects, then --aggregate). The refresh part (screen clear, interval) is still not covered.
• Formulas: containerUsage now points to the docker/cli helpers it follows. No Windows warning.

Pushed in 71bb455.

[docker-agent]

This review covers only the commits pushed since 25e882ada344.

Assessment: 🟢 APPROVE

The concurrency-bounding refactor (SetLimit(32), silent-continue on sampling failures, ctx.Err() cancellation check) is correct and well-tested. The four new tests cover the key scenarios cleanly.

[docker-agent]

[low] Exact peak-concurrency assertion may fail spuriously under extreme CPU starvation

TestCollectContainerStatsBoundsConcurrency asserts peak.Load() == int32(statsConcurrency) — i.e., exactly 32 goroutines must have been running simultaneously. The test creates 40 tasks (32 + 8) each sleeping 20 ms, relying on the Go scheduler filling all 32 errgroup slots before any of the first batch finishes.

In practice the 20 ms sleep provides a large margin, but on an extremely CPU-starved CI runner the scheduler could delay goroutine startup long enough that some of the first batch complete before all 32 slots are filled, causing peak to land below 32 and the test to fail spuriously.

The meaningful invariant is the upper bound (≤ statsConcurrency), not the exact value. If also testing that real concurrency occurred, a lower-bound check (> 1) is sufficient:

Suggested change

	assert.Equal(t, peak.Load(), int32(statsConcurrency))
	assert.Assert(t, peak.Load() <= int32(statsConcurrency), "peak concurrency %d exceeded limit %d", peak.Load(), statsConcurrency)

Confidence	Score
🟡 moderate	62/100

[ndeloof]

LGTM, thanks for the fixes. I checked the points in the reply:

• CPU on older engines: confirmed in moby. noStreamFirstFrame is in daemon/stats.go in v19.03.15 (unconditional), v20.10.0 and v25.0.0 (!config.OneShot), and on master. Compose doesn't send one-shot, so precpu_stats is populated. I did not run an older engine either.
• Bounded concurrency and rows kept when a sample fails (cancellation still reported): correct, and covered by the new tests.
• 0B / 0B for a stopped container matches docker stats -a --no-stream, fair enough.

Two non-blocking suggestions:

• sampleStats errors are dropped silently, so a container that failed to sample looks like a stopped one. A logrus.Debugf would help diagnose it.
• TestCollectContainerStatsBoundsConcurrency asserts peak == statsConcurrency, which depends on scheduling (docker-agent flagged it too). Having each call wait until statsConcurrency are in flight (with a timeout) would make it deterministic.

_{Drafted with Claude Code; read and validated before posting.}