From 6623edbf1cde0c223af35327559e89e0f90614b1 Mon Sep 17 00:00:00 2001 From: Po-Han Huang Date: Thu, 29 Oct 2026 02:14:42 -0700 Subject: [PATCH] Clarify that the draft-precision rule does not restrict the draft KV-cache dtype The draft-precision rule limits draft weight, activation, and compute precision. Update CONTRIBUTING.md and verifier Check 13 so any upstream-supported draft KV-cache dtype is allowed, including one set independently of the target. Co-Authored-By: Claude Opus 5.5 --- .github/codeowner-signoff-verify-prompt.md | 14 ++++++++------ CONTRIBUTING.md | 9 +++++---- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/.github/codeowner-signoff-verify-prompt.md b/.github/codeowner-signoff-verify-prompt.md index 868c764afe..f6de8ba977 100644 --- a/.github/codeowner-signoff-verify-prompt.md +++ b/.github/codeowner-signoff-verify-prompt.md @@ -646,15 +646,17 @@ because they predate the rule. the model authors' reference implementation), that default conversion IS the baseline. A patch or setting that removes it changes the draft computation and FAILs; its absence is not a violation. -- The framework's default draft KV-cache dtype for that checkpoint, or an - upstream-supported KV-cache dtype applied consistently to target and draft (for - example FP8 draft KV inherited from an FP8 target), is allowed. +- The draft KV-cache dtype is NOT restricted. Any upstream-supported draft KV-cache + dtype is allowed, including one set independently of the target (for example + `--speculative-draft-kv-cache-dtype fp8_e4m3` with a BF16 target KV cache). Do not + FAIL or WARN Check 13 because of the draft KV-cache dtype; this check covers draft + weight, activation, and computation precision only. - Upstream optimizations that ship in the pinned image and run the same draft computation faster without lowering weight or activation precision below what ships (fused or default lower-precision kernels for that model, scheduling) are allowed. FAIL when the submission lowers draft precision below the baseline: online or offline -quantization of draft weights, activations, or computation; dtype or KV-cache dtype +quantization of draft weights, activations, or computation; weight or compute dtype overrides aimed at the draft that go below the framework default; substituting a precision-converted or differently quantized draft checkpoint, or a head from a different release than the target; patching the pinned image so the draft loads or @@ -679,8 +681,8 @@ its separate speculative-algorithm/configuration requirements. unchanged launch command does not prove unchanged precision. - Investigate `--speculative-draft-model-quantization` (both space and `=` forms), quantization/dtype fields in `--speculative-config`, `speculative_draft_model_quantization`, - `--speculative-draft-model-path`, `--dtype` / `torch_dtype` / draft dtype and - KV-cache dtype overrides, and settings such as + `--speculative-draft-model-path`, `--dtype` / `torch_dtype` / draft dtype + overrides, and settings such as `SGLANG_GLM_NEXTN_MOE_PTPC=1`. These are inspection leads, not a string denylist: resolve variables and inherited defaults, and determine whether the effective path lowers draft precision below the baseline. A draft path or an explicit setting that diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index abccd0ef41..7b10345f83 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -57,9 +57,10 @@ Allowed (this is the baseline, not an exception): DeepSeek V4.1 DSpark `wo_a` projections, stored as FP8, as BF16, matching DeepSeek's reference implementation. That conversion is the shipped path. A local patch that keeps them FP8 changes the draft computation and is the violation. -- The framework's default draft KV-cache dtype for that checkpoint, and an - upstream-supported KV-cache dtype applied consistently to target and draft (for - example FP8 draft KV inherited from an FP8 target). +- Any upstream-supported draft KV-cache dtype. This rule limits the precision of + draft weights, activations, and computation only; the draft KV-cache dtype is not + restricted. A submission may set it independently of the target, for example + `--speculative-draft-kv-cache-dtype fp8_e4m3` with a BF16 target KV cache. - Genuine upstream optimizations that ship in the pinned image and are used in production by accuracy-sensitive customers: fused or lower-precision kernels that upstream enables by default for that model, scheduling, and anything else that runs @@ -83,7 +84,7 @@ Forbidden (submission-side precision hacking of the draft): applied to a BF16 MTP head, `SGLANG_GLM_NEXTN_MOE_PTPC=1`, and an ATOM `--online_quant_config` whose `exclude_layer` patterns do not cover the entire draft head. -- Dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint. +- Weight or compute dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint. - Substituting a precision-converted or differently quantized draft checkpoint, or a draft head taken from a different release than the served target. - Patching the pinned image so that it loads or computes the draft at a different