Skip to content

Reuse batchesBuf across series via BufferCarrier - #7782

Open
PaurushGarg wants to merge 1 commit into
cortexproject:masterfrom
PaurushGarg:fix/buffer-carrier
Open

Reuse batchesBuf across series via BufferCarrier#7782
PaurushGarg wants to merge 1 commit into
cortexproject:masterfrom
PaurushGarg:fix/buffer-carrier

Conversation

@PaurushGarg

@PaurushGarg PaurushGarg commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What this PR does:
Each mergeIterator allocates its own batchesBuf (~1200 bytes) at creation time. For engines that allocate all iterators upfront (without series batching), this adds to the heap - while only one buffer is actively used at a time since series are processed sequentially within a shard.

This PR adds a bufProvider interface that allows an upstream caller to pass a shared buffer through s.Iterator(carrier). The first series allocates the buffer and writes it back to the carrier; all subsequent series reuse it. When no carrier is passed (nil), each iterator allocates its own buffer as before.

The companion promql-engine PR passes a BufferCarrier from the matrixSelector, enabling the reuse.

Benchmark
Added BenchmarkNewChunkMergeIterator_BufferCarrier which compares:

  • no_carrier: creates 10K iterators passing nil (current behavior — each allocates own buffer)
  • with_carrier: creates 10K iterators passing a shared carrier (new behavior — one buffer reused)

Results here shows -13.1% memory, -10K allocations, no speed regression. Savings are ~1280 bytes/series.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
  • docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags

Signed-off-by: Paurush Garg <paurushg@amazon.com>
@PaurushGarg PaurushGarg changed the title reuse batchesBuf across series via BufferCarrier Reuse batchesBuf across series via BufferCarrier Aug 19, 2026
@PaurushGarg

Copy link
Copy Markdown
Contributor Author
 
dev-dsk-paurushg-2a-87a5ee42 %   go test ./pkg/querier/batch/ -bench=BenchmarkNewChunkMergeIterator_BufferCarrier -run=^$ -benchmem -count=5

goos: linux
goarch: amd64
pkg: github.com/cortexproject/cortex/pkg/querier/batch
cpu: AMD EPYC 7R13 Processor
BenchmarkNewChunkMergeIterator_BufferCarrier/no_carrier-32         	       1	1238926825 ns/op	97765928 B/op	  820017 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/no_carrier-32         	       1	1241601111 ns/op	97760288 B/op	  820006 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/no_carrier-32         	       1	1231510838 ns/op	97760448 B/op	  820004 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/no_carrier-32         	       1	1228693033 ns/op	97765912 B/op	  820010 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/no_carrier-32         	       1	1235855774 ns/op	97760560 B/op	  820005 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/with_carrier-32       	       1	1237800866 ns/op	84961656 B/op	  810007 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/with_carrier-32       	       1	1235080586 ns/op	84961672 B/op	  810006 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/with_carrier-32       	       1	1233754924 ns/op	84961560 B/op	  810005 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/with_carrier-32       	       1	1230165453 ns/op	84961448 B/op	  810004 allocs/op
BenchmarkNewChunkMergeIterator_BufferCarrier/with_carrier-32       	       1	1223676402 ns/op	84961448 B/op	  810004 allocs/op
PASS
ok  	github.com/cortexproject/cortex/pkg/querier/batch	12.359s

(26-08-19 22:42:08) <0> [~/workplace/bench/cortex] 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant