Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions .github/codeowner-signoff-verify-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -646,15 +646,17 @@ because they predate the rule.
the model authors' reference implementation), that default conversion IS the baseline. A patch or setting
that removes it changes the draft computation and FAILs; its absence is not a
violation.
- The framework's default draft KV-cache dtype for that checkpoint, or an
upstream-supported KV-cache dtype applied consistently to target and draft (for
example FP8 draft KV inherited from an FP8 target), is allowed.
- The draft KV-cache dtype is NOT restricted. Any upstream-supported draft KV-cache
dtype is allowed, including one set independently of the target (for example
`--speculative-draft-kv-cache-dtype fp8_e4m3` with a BF16 target KV cache). Do not
FAIL or WARN Check 13 because of the draft KV-cache dtype; this check covers draft
weight, activation, and computation precision only.
- Upstream optimizations that ship in the pinned image and run the same draft
computation faster without lowering weight or activation precision below what ships
(fused or default lower-precision kernels for that model, scheduling) are allowed.

FAIL when the submission lowers draft precision below the baseline: online or offline
quantization of draft weights, activations, or computation; dtype or KV-cache dtype
quantization of draft weights, activations, or computation; weight or compute dtype
overrides aimed at the draft that go below the framework default; substituting a
precision-converted or differently quantized draft checkpoint, or a head from a
different release than the target; patching the pinned image so the draft loads or
Expand All @@ -679,8 +681,8 @@ its separate speculative-algorithm/configuration requirements.
unchanged launch command does not prove unchanged precision.
- Investigate `--speculative-draft-model-quantization` (both space and `=` forms),
quantization/dtype fields in `--speculative-config`, `speculative_draft_model_quantization`,
`--speculative-draft-model-path`, `--dtype` / `torch_dtype` / draft dtype and
KV-cache dtype overrides, and settings such as
`--speculative-draft-model-path`, `--dtype` / `torch_dtype` / draft dtype
overrides, and settings such as
`SGLANG_GLM_NEXTN_MOE_PTPC=1`. These are inspection leads, not a string denylist:
resolve variables and inherited defaults, and determine whether the effective path
lowers draft precision below the baseline. A draft path or an explicit setting that
Expand Down
9 changes: 5 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,10 @@ Allowed (this is the baseline, not an exception):
DeepSeek V4.1 DSpark `wo_a` projections, stored as FP8, as BF16, matching DeepSeek's
reference implementation. That conversion is the shipped path. A local patch that
keeps them FP8 changes the draft computation and is the violation.
- The framework's default draft KV-cache dtype for that checkpoint, and an
upstream-supported KV-cache dtype applied consistently to target and draft (for
example FP8 draft KV inherited from an FP8 target).
- Any upstream-supported draft KV-cache dtype. This rule limits the precision of
draft weights, activations, and computation only; the draft KV-cache dtype is not
restricted. A submission may set it independently of the target, for example
`--speculative-draft-kv-cache-dtype fp8_e4m3` with a BF16 target KV cache.
- Genuine upstream optimizations that ship in the pinned image and are used in
production by accuracy-sensitive customers: fused or lower-precision kernels that
upstream enables by default for that model, scheduling, and anything else that runs
Expand All @@ -83,7 +84,7 @@ Forbidden (submission-side precision hacking of the draft):
applied to a BF16 MTP head, `SGLANG_GLM_NEXTN_MOE_PTPC=1`, and an ATOM
`--online_quant_config` whose `exclude_layer` patterns do not cover the entire draft
head.
- Dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint.
- Weight or compute dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint.
- Substituting a precision-converted or differently quantized draft checkpoint, or a
draft head taken from a different release than the served target.
- Patching the pinned image so that it loads or computes the draft at a different
Expand Down
Loading