Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin for Real-World Robot Policy Evaluation

- Incorporate previous work embodied gaussians as a novel representation of the real world
- using gaussian splatting and warp to reconstruct the scene in simulation

#arxiv #simulation #reconstruction

X Overview, Project Website, Embodied Gaussians

FixupX

Jad Abou-Chakra (@jadachakra)

1/ Real2Sim and Sim2Real are well-known paradigms in robotics🤖. Is there another paradigm where sim and the real world are more tightly interlinked? We introduce Real-IS-Sim: a framework featuring an always-in-the-loop, correctable simulator for behavior…

Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning

- two stage training for in-hand dexterous manipulation
- using residual action to refine the manipulation

#CVPR2025 #dexterous

X Overview, Project Link

FixupX

2:00

Siyuan Huang (@siyuanhuang95)

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25).

🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using…

PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos

- utilize scene reconstruction to create digital twin
- using warp as the simulation pipeline

#arxiv #defromable #reconstruction #simulation

X Overview, Project Link, Arxiv Link

FixupX

Hanxiao Jiang (@jiang_hanxiao)

🚀 How can we create interactive Physical Digital Twins from videos?

Thrilled to share our latest work: PhysTwin! 🌟 Using inverse physics optimization, we generate photo-realistic, physically accurate, and real-time interactive virtual replicas. 🔥

🔗http…

Empirical Analysis of Sim-and-Real Cotraining Of Diffusion Policies For Planar Pushing from Pixels

- detailed analysis of co-training on DP sim-and-real

#arxiv #manipulation

X Overview, Arxiv Link

FixupX

Adam Wei (@adamwei_)

Learning from both sim+real data could scale robot imitation learning. But what are the scaling laws & principles of sim+real cotraining?

We study this in the first focused analysis of sim+real cotraining spanning 250+ policies & 40k+ evals https://arxi…

NavRL: Learning Safe Flight in Dynamic Environments

- good reward design
- randomized collision environment in IsaacSim and combination of perception in actor-critic network

#RAL2025 #navigation #RL

GitHub Link, Paper Link

GitHub

GitHub - Zhefan-Xu/NavRL: [IEEE RA-L'25] NavRL: Learning Safe Flight in Dynamic Environments (NVIDIA Isaac/Python/ROS1/ROS2)

[IEEE RA-L'25] NavRL: Learning Safe Flight in Dynamic Environments (NVIDIA Isaac/Python/ROS1/ROS2) - Zhefan-Xu/NavRL

Rapidly Adapting Policies to the Real-World via Simulation-Guided Fine-Tuning

- efficient RL algorithm to improve rollout performance

#ICLR2025 #RL #sim2real #refinement

X Overview, Project Website

🧵 Thread • FixupX

0:49

Patrick Yin (@patrickhyin)

Current RL finetuning methods are too inefficient to make autonomous real world robot learning tractable. We propose Simulation-Guided Fine-Tuning (SGFT) - a simple, general sim2real framework that extracts structured exploration priors from sim to accelerate…

BaB-ND: Long-Horizon Motion Planning with Branch-and-Bound and Neural Dynamics

- insightful idea of using neural dynamics to assist planning

#ICLR2025 #manipulation

X Overview, Project Website

🧵 Thread • FixupX

Keyi (@Keyi_Shen_)

🎯Can robots do better on planning involving neural dynamics?

Introducing our recent work, BaB-ND: https://robopil.github.io/bab-nd/.

Existing sampling- and gradient-based methods may fail short on complex planning problems over non-convex neural dynamics…

DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset

- benchmark with precise calibration
- it is mind-blowing to see how modern vision model could help to calibrate automatically

#arxiv #dataset

X Overview, Project Website

🧵 Thread • FixupX

Zubair Irshad (@mzubairirshad)

Introducing ✨Posed DROID✨, results of our efforts at automatic post-hoc calibration of a large-scale robotics manipulation dataset.

We provide:
🤖 ~36k calibrated episodes with good quality extrinsic calibration
🦾 ~24k calibrated multi-view episodes with…

π0.5: a VLA with Open-World Generalization

- following work of pi-0
- utilize co-train and use VLM to identify low-level tasks from high-level command

#arxiv #manipulation

X Overview, Project Website, pi-0

🧵 Thread • FixupX

3:10

Chelsea Finn (@chelseabfinn)

Introducing π-0.5!

The model works out of the box in completely new environments.

Here the robot cleans new kitchens & bedrooms. 🤖

Detailed paper + videos in more than 10 unseen rooms:
http://physicalintelligence.company/blog/pi05

A short thread 🧵

Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model

- using RL with residual policy to fine-tune online policy
- similar to TRANSIC

#ICLR2025 #RL #refinement

X Overview, Project Website, TRANSIC

🧵 Thread • FixupX

0:17

Tongzhou Mu 🤖 @ ICLR 2025 (@tongzhou_mu)

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough:

1️⃣ Special designs (usually for multimodal action distributions) in…

SRSA: Skill Retrieval and Adaptation for Robotic Assembly Tasks

- following work of IndustReal, AutoMate
- NVIDIA assembly

#ICLR2025 #assembly

X Overview, Project Website, IndustReal, AutoMate

FixupX

NVIDIA Robotics (@NVIDIARobotics)

Dive into this #ICLR2025 robotics research paper from #NVIDIAResearch, which focuses on skill retrieval and adaptation for #robotic assembly tasks.

The framework teaches robots new tasks using preexisting skill libraries, boosting zero-shot success rates…

HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots

- universal controller for humanoids utilizing co-train with mask
- detailed benchmark

#RSS2025 #humanoid

X Overview, Project Website

🧵 Thread • FxTwitter / FixupX

Tairan He (@TairanHe99)

🤖 Humanoids are expected to be the ultimate embodiment for general-purpose robots, but reaching that potential means adapting to diverse tasks—navigation, tabletop manipulation, loco-manipulation—each needing unique control modes.

🤔 Can one controller handle…

Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation

- good engineering using AR
- slow+fast diffusion procedure

#RSS2025 #contact_rich #tactile

X Overview, Project Website

🧵 Thread • FixupX

1:41

Han Xue (@HanXue012)

Humans can easily perform complex contact-rich tasks with vision and touch, but these tasks remain challenging for robots. How can we resolve this from both the algorithm side and the data side?

Introducing Reactive Diffusion Policy (RDP),a slow-fast imitation…