sim: fix eval bugs, add pioneer-portable RTC driver for flow-matching policies - #286
Merged
Merged
Conversation
…w-matching policies Two real bugs found while researching SO-101 ACT/SmolVLA success rate: LocalLeRobotPolicy.reset() was a no-op (leaked policy state across eval episodes), and RTC's guidance step needs a live backward pass that torch.inference_mode() permanently blocks. Also adds env-var research knobs (n_action_steps override, temporal ensembling, RTC config) and a new embodiment-agnostic RTCDrivenPolicy (src/il/humanoid_il/rtc_driver.py) that drives flow-matching policies (pi0/pi0.5/SmolVLA) via proper prefix-guided chunk replanning instead of lerobot's naive truncate-and-replace select_action() path. Regression-tested against SO-101 (SO101RTCPolicy adapter in lerobot_eval_rtc.py) with no behavior change. Research log in docs/research/so101_manipulation_learning.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012R6u6wWkwgvVuC3KgKse49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Deliverables from an SO-101 ACT/SmolVLA manipulation research session (full log:
docs/research/so101_manipulation_learning.md):lerobot_interface.py:LocalLeRobotPolicy.reset()was a no-op — policy action-queue/temporal-ensembler state leaked across episode boundaries in every multi-episode eval run.torch.autograd.gradinsidetorch.enable_grad()); running it undertorch.inference_mode()permanently blocks that even with a nestedenable_grad(). Fixed the RTC driving path to usetorch.no_grad()instead.src/il/humanoid_il/rtc_driver.py— a generic, embodiment-agnosticRTCDrivenPolicythat drives flow-matching policies (pi0/pi0.5/SmolVLA) via proper prefix-guided chunk replanning (predict_action_chunk()+ hand-rolledActionQueue), instead of lerobot'sselect_action(), which asserts against RTC entirely. This is the piece needed to port RTC to pioneer — it knows nothing about SO101/cameras/joints, only LeRobot-schema obs frames and action tensors.scripts/lerobot_eval_rtc.py— SO-101 eval script exercising the new driver via a thinSO101RTCPolicyadapter (the only robot-specific code left in the RTC path).scripts/lerobot_eval_video.py— video-capturing eval variant, useful for demoing/debugging policy behavior.lerobot_interface.py(ACT_N_ACTION_STEPS_OVERRIDE,ACT_TEMPORAL_ENSEMBLE_COEFF,RTC_EXECUTION_HORIZON,RTC_MAX_GUIDANCE_WEIGHT) for quick architecture experiments without retraining.Test plan
reset()fix: multi-episode runs no longer show stale-state artifacts across episode boundaries.RTCDrivenPolicy) validated against SO-101 across 4 regression trials post-refactor: success rate stayed in the same 30-45% band as pre-refactor runs (noise consistent with known Isaac Lab GPU non-determinism), confirming the generic driver is behaviorally equivalent to the original SO-101-specific RTC implementation.max_guidance_weight=1.0as a genuine (non-monotonic) improvement over lerobot's default of 10.0 for this task.Full context, dataset composition, and what-worked/what-didn't findings are in
docs/research/so101_manipulation_learning.md.🤖 Generated with Claude Code
https://claude.ai/code/session_012R6u6wWkwgvVuC3KgKse49