Skip to content

[4/5] refactor(attention): use varlen operators for vision and audio - #1720

Draft
sufubao wants to merge 1 commit into
refactor/attention-03-backendsfrom
refactor/attention-04-varlen
Draft

sufubao wants to merge 1 commit into
refactor/attention-03-backendsfrom
refactor/attention-04-varlen

Conversation

@sufubao

@sufubao sufubao commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

视觉/音频调用改为VarlenAttentionOp.forward(inputs,out=...),迁移FA3/Triton/SDPA/xFormers的选择、延迟初始化和fallback,删除旧attention_vit入口;--vit_att_backend改为单一字符串。文本模型的State和Graph路由仍在下一步切换。

验证:H200中间层provider/旧attention/varlen/视觉/音频合计277项通过;缺少DeepSeek-V4官方参考processor/tower的120项跳过。中间版本Qwen3.5-0.8B BF16 TP1/DP1 native MTPstep1,prefill/decodeGraph及GPU/CPU/本地DP/PD缓存恢复64次logits比较最大误差0,分配器回收通过。

由已关闭的 #1700 拆分,本PR为第4/5步。依赖顺序:

  1. [1/5] refactor(attention): define execution contracts and device workspace #1717 — API与设备/工作区
  2. [2/5] refactor(attention): relocate kernels without changing execution #1718 — kernel目录迁移
  3. [3/5] refactor(attention): add provider backends and explicit selection #1719 — provider backend与选择
  4. [4/5] refactor(attention): use varlen operators for vision and audio #1720 — 视觉/音频Varlen
  5. [5/5] refactor(attention): integrate model bindings and state lifecycle #1721 — 模型/State/Graph运行时

当前base为#1719 的分支,Files changed只展示本步增量。请按序合入;前序合入main后,将后续PR的base依次改为main并更新分支。所有步骤保持Draft。

每层已运行仓库固定版本Black/Flake8。仓库CI仅监听base=main,因此第2–5步当前以本地lint和实验记录为准,retarget后再检查CI。实验记录位于/data6/exp_logs/runs。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant