Skip to content

fix: support InfiniRT metadata views - #828

Draft
voltjia wants to merge 3 commits into
masterfrom
fix/infinirt-metadata-views
Draft

fix: support InfiniRT metadata views#828
voltjia wants to merge 3 commits into
masterfrom
fix/infinirt-metadata-views

Conversation

@voltjia

@voltjia voltjia commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adapt shared operator metadata handling to InfiniRT borrowed shape and stride views.
  • Preserve owning metadata only where values outlive their source or require mutation.
  • Convert Python tensor metadata directly into Tensor::Shape and Tensor::Strides, and allow the ATen bridge to consume generic ranges.
  • Keep generated base headers synchronized with scripts/generate_torch_ops.py and extend installed C++ consumer regression coverage.

Motivation

InfiniTensor/InfiniRT#38 changes lvalue TensorView::shape() and strides() accessors to borrowed metadata views and replaces exact std::vector aliases with inline-capacity owning containers. Existing InfiniOps assumptions about exact std::vector types fail at mutable, conditional-expression, pybind, and ATen boundaries. This PR keeps transient paths borrowed and makes ownership explicit only where required.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

InfiniRT #38 @ 1824269fc72a69d9a6276a28af41034661960588
Release CPU + NVIDIA build/install: passed
Generated ATen build: 492 ops / 643 overloads, wheel built successfully
pytest tests/test_generate_torch_ops.py -q: 12 passed
pytest tests/test_cpp_api.py -q: 5 passed
pytest tests/test_silu_and_mul.py -m smoke -q --devices cpu nvidia: 4 passed, 44 deselected
pytest tests -m smoke -q --devices cpu nvidia --ignore=tests/test_cpp_api.py: 101 passed, 8 skipped, 22328 deselected
Direct rank-8/rank-9 add on CPU and CUDA: passed

InfiniRT current master @ b548ff5a1475a3945009b101c1aadc13306ae32c
Release CPU + NVIDIA build/install: passed
CPU + NVIDIA + Torch smoke wheel: built successfully
pytest tests/test_generate_torch_ops.py -q: 12 passed
pytest tests/test_cpp_api.py -q: 5 passed
pytest tests/test_silu_and_mul.py -m smoke -q --devices cpu nvidia: 4 passed, 44 deselected
pytest tests -m smoke -q --devices cpu nvidia --ignore=tests/test_cpp_api.py: 93 passed, 16 skipped, 12965 deselected
Direct rank-8/rank-9 add on CPU and CUDA: passed

Ruff 0.15.22 check + format --check: passed
clang-format 21.1.8 dry-run: passed
git diff --check: passed

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
CPU Yes smoke passed with both InfiniRT revisions Full suite not run; installed C++ and rank-8/rank-9 checks passed
NVIDIA Yes smoke passed with both InfiniRT revisions Full generated ATen build passed with the exclusion noted below; full pytest not run
Iluvatar Yes not run Hardware/toolchain unavailable; reviewer routing requested below
MetaX Yes not run Hardware/toolchain unavailable; reviewer routing requested below
Cambricon Yes not run Hardware/toolchain unavailable; reviewer routing requested below
Moore Yes not run Hardware/toolchain unavailable; reviewer routing requested below
Ascend Yes not run Hardware/toolchain unavailable; reviewer routing requested below
Hygon Yes not run Hardware/toolchain unavailable; reviewer routing requested below

Benchmark / Performance Impact

N/A - compatibility fix. Borrowed views remain borrowed on transient paths, explicit copies are limited to owned or mutable metadata, and the Python conversion avoids an intermediate std::vector.

Notes for Reviewers

  • This is a Draft because the shared wrapper changes still need platform-owner validation on Iluvatar, MetaX, Cambricon, Moore, Ascend, and Hygon. @InfiniTensor/infiniops-core-reviewers, please route those checks to reviewers with access.
  • The branch is compatible with both current InfiniRT master and the metadata-view implementation from InfiniRT refactor: make data type mappings and shared CUDA headers device-aware #38.
  • InfiniRT refactor: make data type mappings and shared CUDA headers device-aware #38 changes TensorView layout, so matching InfiniRT headers and libinfinirt.so must be rebuilt and deployed together.
  • The unrestricted all-ATen build reaches the pre-existing std::optional<const at::Tensor> assignment error in src/torch/ops/flash_attn_with_kvcache/flash_attn.cc. The successful generated ATen build excludes only that handwritten implementation while still compiling all 492 generated ops; this PR does not modify the unrelated failure.
  • tests/test_cpp_api.py requires an installed prefix. The reported five tests were run with INFINI_OPS_INSTALL_PREFIX set for each InfiniRT revision.
  • Generated base headers and scripts/generate_torch_ops.py should be reviewed together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant