Skip to content

[5/5] refactor(attention): integrate model bindings and state lifecycle - #1721

Draft
sufubao wants to merge 1 commit into
refactor/attention-04-varlenfrom
refactor/attention-05-runtime
Draft

sufubao wants to merge 1 commit into
refactor/attention-04-varlenfrom
refactor/attention-05-runtime

Conversation

@sufubao

@sufubao sufubao commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

接入模型LayerAttentionBinding、BatchBuilder、State布局/快照和Graph执行生命周期。初始化先选择backend及预留workspace,再规划KV容量;模型通过binding.context(infer_state)显式取得context,State继续拥有存储与恢复。同步所有模型调用、LLM启动参数、离线验证工具和中英文文档,删除旧backend及过渡wrapper。

五步叠加后的Git tree与原已验证提交d649b22d8完全相同,没有额外功能改动。复用该源码树的验证:attention/resource/Graph等2421个独立case最终通过;linear专项122通过;最新KDA与启动专项286通过(各组有重叠)。Qwen3.5-0.8B native/replay MTP各64次、普通replay32次logits比较误差0。

旧native MTP test_mtp_verify_path[1]精确相等失败仍在已知范围,合并前kernel/test对照也可复现,未放宽断言。跨节点PD、完整多模态模型及性能未验收。

由已关闭的 #1700 拆分,本PR为第5/5步。依赖顺序:

  1. [1/5] refactor(attention): define execution contracts and device workspace #1717 — API与设备/工作区
  2. [2/5] refactor(attention): relocate kernels without changing execution #1718 — kernel目录迁移
  3. [3/5] refactor(attention): add provider backends and explicit selection #1719 — provider backend与选择
  4. [4/5] refactor(attention): use varlen operators for vision and audio #1720 — 视觉/音频Varlen
  5. [5/5] refactor(attention): integrate model bindings and state lifecycle #1721 — 模型/State/Graph运行时

当前base为#1720 的分支,Files changed只展示本步增量。请按序合入;前序合入main后,将后续PR的base依次改为main并更新分支。所有步骤保持Draft。

每层已运行仓库固定版本Black/Flake8。仓库CI仅监听base=main,因此第2–5步当前以本地lint和实验记录为准,retarget后再检查CI。实验记录位于/data6/exp_logs/runs。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant