Repository navigation

feat(attention): cuDNN FROST attention backend for head_dim in (256, 512], with context parallelism - #3527

Open

nvegesna-netizen wants to merge 116 commits into

NVIDIA:mainfrom

nvegesna-netizen:nvegesna/te-frost-d512-cp

Open

feat(attention): cuDNN FROST attention backend for head_dim in (256, 512], with context parallelism#3527
nvegesna-netizen wants to merge 116 commits into
NVIDIA:mainfrom
nvegesna-netizen:nvegesna/te-frost-d512-cp

Commits on Sep 16, 2026
- feat(attention): cuDNN FROST kernel wrapper for head_dim in (256, 512]
  
  nvegesna-netizen
  and
  claude
  committed
  7765b76
- feat(attention): select and dispatch FrostAttention from DotProductAttention
  
  nvegesna-netizen
  and
  claude
  committed
  3319826
- feat(attention): context parallelism for FROST across p2p, all_gather and a2a
  
  nvegesna-netizen
  and
  claude
  committed
  d60b3fd
- test(attention): CP coverage for FrostAttention at head_dim 512
  
  nvegesna-netizen
  and
  claude
  committed
  62ffe39
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  
  pre-commit-ci[bot]
  authored and
  nvegesna-netizen
  committed
  fe72e4a
- test(attention): run the FrostAttention CP configs from pytest
  
  nvegesna-netizen
  and
  claude
  committed
  a8957ae
- fix(attention): gate FROST on the cuDNN Frontend version and key plans by device
  
  nvegesna-netizen
  and
  claude
  committed
  bdbb36a
- fix(attention): repair the FROST plan-cache key arity and harden the version gate
  
  nvegesna-netizen
  and
  claude
  committed
  e30ea42
- fix(attention): reject a v that does not match k, and refine the version probe
  
  nvegesna-netizen
  and
  claude
  committed
  0957f71
- fix(attention): update the mixed-THD backend unpack for the new return value
  
  nvegesna-netizen
  and
  claude
  committed
  438b4da
- fix(attention): bind a cuDNN stream and close the remaining silent-wrong-answer paths
  
  nvegesna-netizen
  and
  claude
  committed
  c0d0713
- fix(attention): JIT-compile FROST plans under the device their handle names
  
  nvegesna-netizen
  and
  claude
  committed
  7504bff
- test(attention): anchor FROST numerics to an fp32 reference, not to itself
  
  nvegesna-netizen
  and
  claude
  committed
  85a0f51
- feat(attention): honour deterministic on the FROST path, and document the backend
  
  nvegesna-netizen
  and
  claude
  committed
  c5f9cd2
- docs(attention): scope the FROST exclusivity claim to context parallelism
  
  nvegesna-netizen
  and
  claude
  committed
  a97496e
- fix(attention): drop a duplicate deterministic parameter on the fused p2p wrapper
  
  nvegesna-netizen
  and
  claude
  committed
  5a675c2
- fix(attention): decline FROST when determinism is required
  
  nvegesna-netizen
  and
  claude
  committed
  e82981f
- test(attention): make the FROST oracle float64, since an fp32 one is not a reference
  
  nvegesna-netizen
  and
  claude
  committed
  3edba85
- feat(attention): express FROST masking as a diagonal band, adding sliding window
  
  nvegesna-netizen
  and
  claude
  committed
  064396e
- fix(attention): carry the sliding window through a2a, and decline it for p2p
  
  nvegesna-netizen
  and
  claude
  committed
  441dee4
- test(attention): cover the sliding window in backward, at its boundary, and per cp_comm_type
  
  nvegesna-netizen
  and
  claude
  committed
  9770ca5
- fix(attention): let the CP sliding-window asserts know FROST exists
  
  nvegesna-netizen
  and
  claude
  committed
  ad9dfdc
- docs(attention): correct the claimed cuDNN import-ordering hazard
  
  nvegesna-netizen
  and
  claude
  committed
  3eea2f9
- test(attention): apply the window for every mask type in the reference
  
  nvegesna-netizen
  and
  claude
  committed
  e062e8d
- docs(attention): justify the p2p sliding-window decline from the ring, not precedent
  
  nvegesna-netizen
  and
  claude
  committed
  dd0033c
- fix(attention): bind the FROST flag on the ONNX path, decline what was silently ignored
  
  nvegesna-netizen
  and
  claude
  committed
  a82903b
- fix(attention): read qkv_type from attention_params, not the rebound local
  
  nvegesna-netizen
  and
  claude
  committed
  591955d
- test(attention): cover the ONNX-export branch on hardware that can run it
  nvegesna-netizen
  committed
  6832a9b
Commits on Sep 17, 2026
Commits on Sep 21, 2026
Commits on Sep 22, 2026
- Merge remote-tracking branch 'origin/main' into nvegesna/te-frost-d512-cp
  nvegesna-netizen
  committed
  c5d8825
Commits on Oct 1, 2026
- refactor(attention): extract the shared cuDNN pygraph plumbing
  
  nvegesna-netizen
  and
  claude
  committed
  7bbccff
- feat(attention): let the flex cuDNN graphs carry a diagonal-band mask
  
  nvegesna-netizen
  and
  claude
  committed
  42fa333
- fix(attention): stop preparing the FROST graphs twice
  
  nvegesna-netizen
  and
  claude
  committed
  084f528
- fix(attention): keep the flex builder call shape, and frost's cudnn handle
  
  nvegesna-netizen
  and
  claude
  committed
  1f43e88
- fix(attention): enable the FROST engines whichever backend imports cuDNN first
  
  nvegesna-netizen
  and
  claude
  committed
  469bad9
- test(attention): cover the flex mask_spec path on CPU
  
  nvegesna-netizen
  and
  claude
  committed
  626bde2
- fix(attention): say why a pinned cuDNN engine declined the graph
  
  nvegesna-netizen
  and
  claude
  committed
  40b8a2a
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  812d475
- test(attention): skip the new cuDNN-frontend tests when the package is absent
  
  nvegesna-netizen
  and
  claude
  committed
  c981a0d
- fix(attention): stop flex graphs running on a FROST engine
  
  nvegesna-netizen
  and
  claude
  committed
  73a205b
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  c46dffc
- test(attention): skip the FROST switch test where it cannot detect anything
  
  nvegesna-netizen
  and
  claude
  committed
  2cfd6ff
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  636d7a6
Commits on Oct 4, 2026
- Merge branch 'main' into nvegesna/te-frost-d512-cp
  nvegesna-netizen
  authored
  8ae1663
- fix(attention): index the FROST p2p results by the alternating slot
  
  nvegesna-netizen
  and
  claude
  committed
  69a253b
Commits on Oct 6, 2026
- refactor(attention): make FROST a FusedAttention sub-backend
  
  nvegesna-netizen
  and
  claude
  committed
  7a55819
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  9eb866d
- fix(attention): decline FROST where the diagonal anchor is ambiguous
  
  nvegesna-netizen
  and
  claude
  committed
  0d1c4bf
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  91a37f1
- refactor(attention): make frost_attention.py standalone again
  
  nvegesna-netizen
  and
  claude
  committed
  521c9a8
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  76f59c7
- fix(attention): return max_logit by index from the CP p2p fused step
  
  nvegesna-netizen
  and
  claude
  committed
  94577ea
- fix(attention): put the new CP grad slot at the right index
  
  nvegesna-netizen
  and
  claude
  committed
  2cb9db1
- test(attention): count FROST as a comparable fused sub-backend
  
  nvegesna-netizen
  and
  claude
  committed
  d400eea
- Merge branch 'main' into nvegesna/te-frost-d512-cp
  nvegesna-netizen
  authored
  0f2c781
- test(attention): drop the comment on the sub-backend list
  
  nvegesna-netizen
  and
  claude
  committed
  dbc298e
- fix(attention): bar the cuDNN FROST engines from flex attention graphs
  
  nvegesna-netizen
  and
  claude
  committed
  aec8b4b
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  5d506d8
- refactor(attention): trim the FROST comments to the repo's register
  
  nvegesna-netizen
  and
  claude
  committed
  df5ae6e
Commits on Oct 7, 2026
- refactor(attention): shorten the max_logit unpack comment
  
  nvegesna-netizen
  and
  claude
  committed
  597e593
- feat(attention): serve asymmetric head_dim on the FROST sub-backend
  
  nvegesna-netizen
  and
  claude
  committed
  6b1a6d8
- revert(attention): move the flex FROST engine bar out of this change
  
  nvegesna-netizen
  and
  claude
  committed
  2fca15d
- docs(attention): drop the stale symmetric head_dim claim for FROST
  
  nvegesna-netizen
  and
  claude
  committed
  d04ece9
- fix(attention): resolve the FROST diagonal anchor the way the dispatcher does
  
  nvegesna-netizen
  and
  claude
  committed
  0c4601a
- refactor(attention): give the cuDNN pygraph state one owner
  
  nvegesna-netizen
  and
  claude
  committed
  c1c3a86
- refactor(attention): describe the FROST tensors instead of permuting them
  
  nvegesna-netizen
  and
  claude
  committed
  2e6787d
- refactor(attention): drop the FROST kernel API, keep the sub-backend one
  
  nvegesna-netizen
  and
  claude
  committed
  7bc01fb
- refactor(attention): share the graph cache mechanism and the per-tensor key
  
  nvegesna-netizen
  and
  claude
  committed
  314a91d
- fix(attention): scope the uncacheable graph build to its device too
  
  nvegesna-netizen
  and
  claude
  committed
  2931c95
- fix(attention): bar the cuDNN FROST engines by default, not on request
  
  nvegesna-netizen
  and
  claude
  committed
  e0f2690
- docs(attention): bring the new docstrings into the repo's register
  
  nvegesna-netizen
  and
  claude
  committed
  631d8be
- fix(attention): decline asymmetric head_dim again, measured on B200
  
  nvegesna-netizen
  and
  claude
  committed
  6654a68
- fix(attention): close two guards the fused path was bypassing
  
  nvegesna-netizen
  and
  claude
  committed
  16915e9
- refactor(attention): move the diagonal-band translation to cudnn_pygraph
  
  nvegesna-netizen
  and
  claude
  committed
  86609ba
- Revert "refactor(attention): move the diagonal-band translation to cudnn_pygraph"
  
  nvegesna-netizen
  and
  claude
  committed
  639cdad
Commits on Oct 8, 2026
- refactor(attention): share the graph execution step
  
  nvegesna-netizen
  and
  claude
  committed
  0d25aa3
- docs(attention): mark the FROST engine bar as a workaround with an expiry
  
  nvegesna-netizen
  and
  claude
  committed
  8c6c36a
- docs(attention): say what actually verifies the FROST engine bar
  
  nvegesna-netizen
  and
  claude
  committed
  f01a0ee
- test(attention): let a lane require the flex cuDNN tests to actually run
  
  nvegesna-netizen
  and
  claude
  committed
  5019bf3
- test(attention): trim the flex required-guard comment
  
  nvegesna-netizen
  and
  claude
  committed
  598db0d
- test(attention): trim the FROST guard comments to match the siblings
  
  nvegesna-netizen
  and
  claude
  committed
  8176c30
- test(attention): cut the FROST suite's CI cost where it buys nothing
  
  nvegesna-netizen
  and
  claude
  committed
  a88556b
- test(attention): put the cp_comm_type selector test back
  
  nvegesna-netizen
  and
  claude
  committed
  c01f48b
- Revert "test(attention): put the cp_comm_type selector test back"
  
  nvegesna-netizen
  and
  claude
  committed
  adbc951
- test(attention): put the CP sequence length back, it saved nothing
  
  nvegesna-netizen
  and
  claude
  committed
  6c66bca
- test(attention): trim the CP seqlen comment
  
  nvegesna-netizen
  and
  claude
  committed
  dbb9500
- test(attention): drop the CP seqlen note
  
  nvegesna-netizen
  and
  claude
  committed
  029e0d7
- test(attention): point the single a2a+p2p case at the shortest model
  
  nvegesna-netizen
  and
  claude
  committed
  f23a850
- refactor(attention): build the cuDNN SDPA graphs through cudnn_pygraph
  
  nvegesna-netizen
  and
  claude
  committed
  b66c078
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  5c099bf
- refactor(attention): drop the FROST layout checks cuDNN already makes
  
  nvegesna-netizen
  and
  claude
  committed
  b163269
- fix(attention): restore the FROST rank check, which cuDNN cannot make
  
  nvegesna-netizen
  and
  claude
  committed
  7364535
- docs(attention): cite the engine capability for the FROST symmetry rule
  
  nvegesna-netizen
  and
  claude
  committed
  8a680d5
- fix(attention): check the whole softmax_lse shape, not just its leading dims
  
  nvegesna-netizen
  and
  claude
  committed
  80672e3
- docs(attention): shorten the cudnn_pygraph module docstring
  
  nvegesna-netizen
  and
  claude
  committed
  0effc6c
Commits on Oct 10, 2026
- refactor(attention): derive FusedAttnBackend.FROST instead of hardcoding 3
  
  nvegesna-netizen
  and
  claude
  committed
  c395186
- test(attention): fold the FROST forward test into the backward one
  
  nvegesna-netizen
  and
  claude
  committed
  8f2416e
- refactor(attention): let CP use the selected backend directly
  
  nvegesna-netizen
  and
  claude
  committed
  8eaa51a
- test(attention): keep one unwindowed bottom-right forward case
  
  nvegesna-netizen
  and
  claude
  committed
  e726f4e
- refactor(attention): act on the review round, trims and two shared helpers
  
  nvegesna-netizen
  and
  claude
  committed
  24e430c
- [pre-commit.ci] auto fixes from pre-commit.com hooks
  pre-commit-ci[bot]
  committed
  b8c2eee
- fix(attention): match the C++ fused_attn contract for the LSE and the backward return
  
  nvegesna-netizen
  and
  claude
  committed
  869ef39
Commits on Oct 14, 2026
- to-be-reviewed: minor fixes
  cyanguwa
  committed
  7c56bcb
Commits on Oct 15, 2026
- refactor(attention): fold the backend shims into cudnn_pygraph
  
  nvegesna-netizen
  and
  claude
  committed
  6d65eb6
- refactor(attention): share the SDPA plan cache key, drop the last flex shim
  
  nvegesna-netizen
  and
  claude
  committed
  42eaaab
- fix(attention): keep the qkv formats in flex's plan cache key
  
  nvegesna-netizen
  and
  claude
  committed
  a66f88f
- docs(attention): match the attention subtree's commenting conventions
  
  nvegesna-netizen
  and
  claude
  committed
  e72c5cd
- fix(attention): correct two FROST decline messages
  
  nvegesna-netizen
  and
  claude
  committed
  e88dc70
- docs(attention): name NVTE_FROST_ATTN in the module docstring
  
  nvegesna-netizen
  and
  claude
  committed
  3453ec1
- test(attention): drop the FROST k/v mismatch test
  
  nvegesna-netizen
  and
  claude
  committed
  f3f77a3

Commits

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 21, 2026

Commits on Sep 22, 2026

Commits on Oct 1, 2026

Commits on Oct 4, 2026

Commits on Oct 6, 2026

Commits on Oct 7, 2026

Commits on Oct 8, 2026

Commits on Oct 10, 2026

Commits on Oct 14, 2026

Commits on Oct 15, 2026