Repository navigation
[Docs] Clarify that the draft KV-cache dtype is not restricted / [Docs] 澄清 draft KV-cache dtype 不受限制 - #4640
Open
nvpohanh wants to merge 1 commit into
Open
[Docs] Clarify that the draft KV-cache dtype is not restricted / [Docs] 澄清 draft KV-cache dtype 不受限制#4640nvpohanh wants to merge 1 commit into
nvpohanh wants to merge 1 commit into
Conversation
…cache dtype The draft-precision rule limits draft weight, activation, and compute precision. Update CONTRIBUTING.md and verifier Check 13 so any upstream-supported draft KV-cache dtype is allowed, including one set independently of the target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The draft-precision rule in
CONTRIBUTING.mdand verifier Check 13 said that the draft KV-cache dtype must be the framework default or match the target's KV-cache dtype. That was a mistake. The rule is meant to limit draft weight, activation, and compute precision only. No rule restricts the draft KV-cache dtype.The wording is causing false Check 13 failures. For example, #4524 sets
--speculative-draft-kv-cache-dtype fp8_e4m3while the target KV cache stays at the default, and the verifier rejected it.Changes:
CONTRIBUTING.md("Draft-model precision"): any upstream-supported draft KV-cache dtype is allowed, including one set independently of the target. The forbidden dtype overrides are now limited to weight and compute dtypes..github/codeowner-signoff-verify-prompt.md(Check 13): the same clarification, and Check 13 must not FAIL or WARN because of the draft KV-cache dtype. The KV-cache dtype override is removed from the FAIL list and from the inspection leads.Draft weights, activations, and computation must still run as shipped.
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOEstays prohibited.中文
CONTRIBUTING.md中的 draft 精度规则和 verifier Check 13 曾要求 draft KV-cache dtype 必须是框架默认值，或与 target 的 KV-cache dtype 一致。这是一个错误。该规则只用于限制 draft 的权重、激活和计算精度，目前没有任何规则限制 draft KV-cache dtype。这段措辞导致了 Check 13 的误判。例如 #4524 设置了
--speculative-draft-kv-cache-dtype fp8_e4m3，而 target KV cache 保持默认，verifier 因此拒绝了该 PR。改动：
CONTRIBUTING.md（"Draft-model precision"）：允许任何上游支持的 draft KV-cache dtype，包括独立于 target 设置的 dtype。被禁止的 dtype override 现在仅限于权重和计算 dtype。.github/codeowner-signoff-verify-prompt.md（Check 13）：同样的澄清，并规定 Check 13 不得因 draft KV-cache dtype 而判为 FAIL 或 WARN。KV-cache dtype override 已从 FAIL 列表和检查线索中移除。draft 的权重、激活和计算仍须按原样运行。
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE仍然禁止。AI model disclosure
Prepared with Claude Code using
claude-opus-5-5at the human author's request.🤖 Generated with Claude Code