Skip to content

Use a block-reduction search in TensorPrimitives.IndexOfMin/Max/MinMagnitude/MaxMagnitude - #133969

Open
Mrnikbobjeff wants to merge 3 commits into
dotnet:mainfrom
Mrnikbobjeff:argmin-blocks
Open

Mrnikbobjeff wants to merge 3 commits into
dotnet:mainfrom
Mrnikbobjeff:argmin-blocks

Conversation

@Mrnikbobjeff

@Mrnikbobjeff Mrnikbobjeff commented Sep 15, 2026

Copy link
Copy Markdown

Summary

IndexOfMinMaxCore (shared by IndexOfMin, IndexOfMax, IndexOfMinMagnitude, IndexOfMaxMagnitude) tracks a running best value plus a vector of indices per lane. That costs a compare and two selects per element, needs the Size4Plus/Size2/Size1 variants so indices fit in lanes, and on AVX-512 hardware the blends of the running value go through mask-register conversions. IndexOfMin<int> over 65536 elements runs at about 5 GElem/s on a Zen 4 box, no faster than the .NET 10 package.

This PR replaces the loop with a two-pass block reduction, written against the operator interface so all four searches use it:

  • Pass 1 reduces each block of 32 vectors to its best element with a pure vector loop (one load and one lane-wise Reduce per vector, two independent accumulators), then a horizontal Aggregate, and keeps the first block whose result beats the running result under the operator's strict Compare.
  • Pass 2 scans only the winning block for the first element the result does not beat. That is the operator's own tie rule (first equal element; -0 before +0 for min; the sign preference for equal magnitudes), so no bitwise match is needed.

IIndexOfMinMaxOperator<T> gains Reduce (scalar and per vector width). Each operator forwards it to its aggregation operator (Vector.Min / Max / MinMagnitude / MaxMagnitude), which the previous code already used for its final horizontal step and bitwise match, so tie behavior is unchanged. Those reductions implement IEEE minimum/maximum and propagate NaN, so a block containing a NaN reduces to NaN and the first NaN of that block is returned; the per-vector NaN checks go away. Indices never live in vector lanes, so the per-element-size variants and ElementWiseSelect are removed (536 lines added, 597 removed).

Performance

BenchmarkDotNet 0.15.8, AMD Ryzen 7 7800X3D, .NET 10.0.5, IndexOfMin<int>, N = 65536 (256 KB), GElem/s:

Build Increasing Sparse hits Random Decreasing
main (this core, unmodified) 5.0 5.2 5.1 5.2
main, DOTNET_PreferredVectorBitWidth=256 8.0 7.9 9.1 8.7
This PR 36.7 36.7 36.6 35.6

The block loop compiles to two vpminsd zmm with memory operands per iteration; the result is independent of the input pattern and matches a hand-written Vector256 block-min argmin on the same machine (the L2 fill bandwidth for this working set). Inputs shorter than one vector use the scalar fallback.

Tests

  • Adds multi-block length, NaN and signed-zero cases for all four searches (Helpers.TensorLengths stops at 256 elements, which is a single block at 256 bits).
  • System.Numerics.Tensors.Tests filtered to *IndexOf* passes on the live net11.0 testhost, and the same test sources compiled against the net10.0 build pass on the 512-, 256- and 128-bit paths and with hardware intrinsics disabled (DOTNET_PreferredVectorBitWidth=256, DOTNET_EnableAVX2=0, DOTNET_EnableHWIntrinsic=0), for all 17 element types.

…gnitude/MaxMagnitude

IndexOfMinMaxCore tracked a running best value and a vector of indices per lane, which costs a compare and two selects per element and needs per-element-size variants so that indices fit in lanes. On AVX-512 hardware the blends of the running value also go through mask-register conversions, and IndexOfMin<int> over 65536 elements runs at about 5 GElem/s on a Zen 4 box, no faster than the .NET 10 package.

Replace it with a two-pass block reduction:

- Pass 1 reduces each block of 32 vectors to its best element with a pure vector loop (one load and one lane-wise Reduce per vector, two accumulators), then a horizontal Aggregate, and keeps the first block whose result beats the running result under the operator's strict Compare.
- Pass 2 scans only the winning block for the first element the result does not beat, which is the operator's own tie rule (first equal element; -0 before +0 for min; the sign preference for equal magnitudes).

IIndexOfMinMaxOperator<T> gains Reduce (scalar and per vector width); each operator forwards it to its aggregation operator (Vector.Min/Max/MinMagnitude/MaxMagnitude), the same operators the previous code used for its final horizontal step and bitwise match, so the tie behavior is unchanged. Because those reductions implement IEEE minimum/maximum and propagate NaN, a block containing a NaN reduces to NaN and the first NaN of that block is returned; the per-vector NaN checks are no longer needed. Indices never live in vector lanes, so the Size4Plus/Size2/Size1 variants and ElementWiseSelect are removed.

Measured with BenchmarkDotNet on an AMD Ryzen 7 7800X3D (.NET 10.0.5, N = 65536 ints): IndexOfMin<int> goes from 5.1 GElem/s to 35.6-36.7 GElem/s on increasing, random, decreasing and sparse-hit inputs, which is the L2 bandwidth ceiling for that working set. The hot loop is two vpminsd with memory operands per iteration.

Tests: adds multi-block length, NaN and signed-zero cases for all four searches (the existing lengths stop at 256 elements, one block). The IndexOf* tests pass for all element types on the 512-, 256- and 128-bit and scalar paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Sep 15, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Mrnikbobjeff and others added 2 commits September 15, 2026 19:23
Add debug-only checks to IndexOfMinMaxCore: after each block reduction, no element of the block beats the reduction result under the operator's Compare and no NaN survived the reduction (the properties the final scan and the NaN handling rely on), the element size is one the lane extraction supports, and pass 1 selected a block.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Numerics community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant