Add Inkling attention support to vLLM runtime

[meenchen]

What does this PR do?

Type of change: new feature

Adds ModelOpt attention support for vLLM 0.26's model-specific InklingAttention path:

• extends the Triton attention kernel with Inkling relative-position logits and causal left-window masking while preserving GQA and paged KV-cache support;
• adds block-16 NVFP4 K-cache on-write fake quantization and wires Inkling Q/K/P/V attention quantizers into the vLLM runtime adapter;
• composes NVFP4 attention with 2:4 prefill sparsity, while keeping N:M sparsity disabled during decode to match the existing vLLM contract;
• adds focused GPU/runtime tests and a launcher smoke configuration;
• packages the project metadata needed for editable installation inside launcher jobs.

The root cause was that Inkling uses a model-specific module rather than vLLM's regular Attention subclass. It also performs fused Q/K/V preparation and calls a private relative-attention implementation, so the existing ModelOpt discovery and adapter paths could not install attention quantization or sparsity.

Usage

from modelopt.torch.sparsity.attention_sparsity.plugins.vllm_runtime import (
    install_vllm_nvfp4_attention,
)

report = install_vllm_nvfp4_attention(model_runner, sparse_cfg="checkpoint")

A standalone GPU smoke workload is available at
tools/launcher/examples/smoke/inkling_attention.yaml.

Testing

Validated with vLLM 0.26.0 on NVIDIA GB300:

• 29 Triton attention kernel tests passed.
• 28 vLLM plugin/runtime tests passed.
• Published Inkling 64Q/8KV global and 64Q/16KV local GQA geometries matched a PyTorch reference.
• Dense relative-bias/window attention, paged KV, NVFP4 K/P/V QDQ, Q quantizer integration, and active 2:4 prefill sparsity were exercised.
• Smoke result: INKLING_ATTENTION_SANITY_OK 0.26.0 NVIDIA GB300.
• All pre-commit hooks passed.

Full 66-layer Inkling checkpoint serving and benchmark evaluation have not yet been run; this PR is intentionally a draft.

Before your PR is "Ready for review"

• Is this change backward compatible?: ✅
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: ✅
• Did you update Changelog?: ❌ Pending before ready for review.
• Did you get Claude approval on this PR?: N/A

Additional Information

Branch commit: 9c56d882de1c9f1a2686e3d6d44446d8bf0229e7.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 085ae644-3e1e-44eb-99b8-d55293559b1b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}