OpenVINO: LeRobot policies that do not run yet, and what blocks them #24325

We exported the LeRobot policies to ExecuTorch and ran them with the OpenVINO and TensorRT backends. This issue lists what still keeps policies from working on OpenVINO, so it can be picked up.

Results

Milliseconds per action chunk, batch size 1. Every number matches eager PyTorch on the same weights, except the two marked *.

• OpenVINO: Intel Xeon Platinum 8339HC server CPU, 16 threads, OpenVINO 2026.4.1, with the fixes in Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24294.
• TensorRT: NVIDIA H100, Jetson AGX Thor, Jetson Orin Nano 8 GB.
• On the Orin Nano, pi0 and xvla use a low-memory export, and pi05 and groot are split into several programs.
• - does not export or run yet (fastwam on Jetson Thor is not measured yet). x does not fit in 8 GB.

* tdmpc is sensitive to bf16 (eager PyTorch in bf16 also differs). pi0_fast in fp32 matches a float32 eager reference.

Already fixed or in review

• Fix crashes, wrong outputs and lowering failures in several backends and in exir #24222 (in main; its release/1.6 pick Fix crashes, wrong outputs and lowering failures in several backends and in exir #24298 is still open): loading a .pte from a file no longer segfaults with OpenVINO 2026.4, the graph is converted only after input types are known (fixes bool-mask and bf16 inputs for smolvla, vla_jepa, evo1 and eo1), and stray assert nodes no longer break xvla, molmoact2 and eo1.
• Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24294 (in main; its release/1.6 pick Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24377 is still open): 0-d tensors, float32 by default on CPU (including ARM, where precision is fixed at compile time), outputs with data-dependent sizes, and the ExecuTorch-side workarounds for items 2 and 4.

What blocks OpenVINO

1. Boolean-mask indexing does not convert (groot, vla_jepa)

Conversion stops with Dynamic rank for aten::index input is not supported. Both policies use the Qwen3-VL DeepStack code in transformers, which writes and then reads the same masked rows on several layers in a row (_deepstack_process):

local_this = hidden_states[visual_pos_masks, :] + visual_embeds
hidden_states[visual_pos_masks, :] = local_this

On the second layer, the boolean mask reaches aten.index with a dynamic rank. The cause is in the FX decoder: TorchFXPythonDecoder.get_shape_for_value reads only node.meta["tensor_meta"], which is None after ExecuTorch's to_edge, and does not fall back to node.meta["val"] as get_found_shape already does. A fallback to meta["val"] fixes it; a small Qwen3-VL text model with DeepStack inputs then converts and matches PyTorch.

2. cumsum over a bool tensor wraps after 255 (pi0, pi0.5, pi0_fast)

Partly fixed. The pi0 family builds position ids with a cumulative sum over a bool padding mask (pi0, pi0.5):

position_ids = torch.cumsum(pad_masks, dim=1) - 1

PyTorch returns int64 here. The OpenVINO frontend keeps the narrow input type for the accumulation, so the count wraps after 255 and prompts longer than 255 tokens get wrong position ids. ExecuTorch now casts the input of cumsum and prod to int64 (#24294), but aten::cumsum over bool or integer inputs, and aten::prod over integer inputs, should follow PyTorch's type promotion in the frontend itself.

3. int64 reductions are computed in 32 bits on the CPU plugin

Even with the int64 cast from item 2, the CPU plugin of OpenVINO 2026.4.1 computes prod, cumsum and sum in 32 bits, so a result past the int32 range comes back wrong with no error. For example, prod of the int32 tensor [50000, 50000], cast to int64 first, returns -1794967296 instead of 2500000000.

4. Integer cumsum over constants is wrong on the CPU plugin

Partly fixed. Masks built from an int64 or int32 cumsum over a constant can come back wrong on CPU from the second inference on: they keep only what earlier calls allowed. We saw this on 2025.4.1, 2026.0.0 and 2026.4.1, but not on the nightly. The pi0 family builds its masks this way. ExecuTorch now computes these cumsums during lowering (#24294). Without that step, pi0.5 stopped matching eager PyTorch after a few action chunks. With it, pi0.5 matches on 16 action chunks in a row. The plugin bug remains in released versions.

5. HETERO devices

Lowering for HETERO:CPU crashes in export_model(), and for AUTO:CPU the same call fails with Not Implemented. Also, OpenVINO rejects set_property on a HETERO: device string (2025.1.0 to 2026.4.1), which is how the ExecuTorch runtime applies inference_precision. Setting the hint on each listed device works.

6. A failed run gives no reason

Partly fixed. Without #24294, molmoact2 lowered to two OpenVINO partitions plus CPU ops, then the first delegate call failed with CALL_DELEGATE execute failed at instruction 0: 0x1. With #24294 it runs and matches eager PyTorch. Failures in OpenvinoBackend::execute still do not log OpenVINO's own message (ov_get_last_err_msg()), and some, such as a failure to set an input or output tensor, return with no log at all.

7. Lowering is slow for large models (wall_x, flux3, lingbot_vla_v2)

wall_x now finishes, but lowering alone takes about 8.3 hours on this CPU, with other jobs sharing it (a 61,000-node graph, 12 OpenVINO partitions). lingbot_vla_v2 exports, but OpenVINO lowering does not finish in 12 hours in either precision. flux3 lowers for about 9 hours and then fails in conversion. AtenIndexPutReplacer fails on an aten.index_put whose indices are a list of tensors: while building the index it adds an int64 tensor to an int32 one, and OpenVINO rejects the mixed types:

While validating node 'opset1::Add Add_18585 (opset1::Add Add_18584[0]:i64[1,1,1,1],
  opset1::Unsqueeze aten_unsqueeze_copy_default_23/...[0]:i32[128,1]) -> (dynamic[...])'

Part of the time is ExecuTorch partitioning, which slows down sharply for very large graphs. A profile of the OpenVINO conversion step would show how much is on the OpenVINO side.

Outside OpenVINO

• fastwam and lawam: eager PyTorch already fails on CPU in the policy's own dtype.
• lingbot_va: its KV cache allocator branches on tensor sizes, so it cannot be exported as written.
• flux3: uses custom attention ops (NATTEN) with no ExecuTorch kernels, which blocks TensorRT. On OpenVINO it exports and gets as far as conversion (item 7).

cc @suryasidd @daniil-lyakhov @anzr299