HIL-SERL: Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning

- sparse reward classifier
- human in the loop online RL training

#arxiv #RL

Project Website

IMLE Policy: Fast and Sample Efficient Visuomotor Policy Learning via Implicit Maximum Likelihood Estimation

- state of art BC comparing to diffusion
- faster inference speed

#RSS0225 #BC

X Overview, Project Website

🧵 Thread • FixupX

0:41

Krishan Rana (@krshnrana)

Are Diffusion and Flow Matching the best generative modelling algorithms for behaviour cloning in robotics?

✅Multimodality
❌Fast, Single-Step Inference
❌Sample Efficient

💡 We introduce IMLE Policy, a novel behaviour cloning approach that can satisfy all…

Scalable Real2Sim: Physics-Aware Asset Generation Via Robotic Pick-and-Place Setups

- automatically generated simulation objects from the real world
- detailed analysis of object parameter estimation procedure

#arxiv #real2sim #reconstruction

X Overview, Project Website

🧵 Thread • FixupX

Nicholas Pfaff (@NicholasEPfaff)

New Paper: "Scalable Real2Sim: Physics-Aware Asset Generation via Robotic Pick-and-Place Setups"! 🤖

We introduce a fully automated pipeline that generates simulation-ready assets for real-world objects—no manual intervention needed!

🌐 Website: https://scalable…

PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers

- diffusion model in motion generation
- combine RL in data augmentation

#SIGGRAPH2025 #diffusion #RL

X Overview, Project Website

🧵 Thread • FixupX

Michael Xu (@mxu_cg)

Interested in simulated characters traversing complex terrains?

PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers

Project page: https://michaelx.io/parc/index.html

with: @YiShi_333, @KangkangYin, and @xbpeng4

@siggraph…

Visuomotor Policies to Grasp Anything with Dexterous Hands

- grasping based on stero input
- engineering example of teacher-student, sim2real, and geometric fabrics

#arxiv #dexterous #grasp #sim2real

X Overview, Project Website, Geometric Fabrics

🧵 Thread • FixupX

Ritvik Singh (@ritvik_singh9)

Excited to announce our latest work: DextrAH-RGB where we successfully train visuomotor policies in sim to perform dexterous grasping of arbitrary objects end-to-end from RGB input!

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

- combination of sim and real could help train robust grasping policy
- engineering example of pi-0 policy (VLM + flow matching)

#arxiv #grasp #simulation #sim2real #flow_matching

X Overview, Project Website, NVidia GraspGen

🧵 Thread • FixupX

Tuo Liu (@TuoLiuRobot)

Galbot（银河通用）has just dropped GraspVLA, the world’s first end-to-end embodied grasping model! Trained on over a billion “visual-language-action” pairs, GraspVLA is ready to handle a wide range of tasks, even the unexpected ones.

And with a little post-training…

FunGrasp: Functional Grasping for Diverse Dexterous Hands

- detailed explanation of engineering
- demonstrate the importance of contact signal

#IROS2025 #dexterous #grasp

X Overview, Project Website

🧵 Thread • FixupX

Hui Zhang (@Hui_Zhang_eth)

People grasp objects with different poses for different tasks. Can dexterous robots possess similar functional grasping capabilities? Our recent work FunGrasp enables this for various unseen objects, diverse task-specific poses, and different robot hands.…

FALCON: Learning Force-Adaptive Humanoid Loco-Manipulation

- combines upper and lower body for policy training
- curriculum training for force adaptation

#arxiv #humanoid #locomanip

X Overview, Project Website

🧵 Thread • FixupX

1:50

Yuanhang Zhang (@Yuanhang__Zhang)

🦾How can humanoids unlock real strength for heavy-duty loco-manipulation?

Meet FALCON🦅: Learning Force-Adaptive Humanoid Loco-Manipulation.

🌐: https://lecar-lab.github.io/falcon-humanoid/

See the details below👇:

Steerable Scene Generation with Post Training and Inference-Time Search

- use a diffusion-based framework to generate scalable scenes on procedural data
- use a Monte Carlo Tree Search to find physical feasibility and satisfy downstream objective

#arxiv #diffusion #generative

X Overview, Project Website

🧵 Thread • FixupX

0:39

Nicholas Pfaff (@NicholasEPfaff)

Want to scale robot data with simulation, but don’t know how to get large numbers of realistic, diverse, and task-relevant scenes?

Our solution:
➊ Pretrain on broad procedural scene data
➋ Steer generation toward downstream objectives

🌐 https://steerable…

AMO:
Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control

- framework that integrates sim-to-real reinforcement learning (RL) with trajectory optimization for real-time, adaptive whole-body control

#RSS2025 #humanoid

X overview, Project Website

🧵 Thread • FixupX

1:39

Xuxin Cheng (@xuxin_cheng)

Meet 𝐀𝐌𝐎 — our universal whole‑body controller that unleashes the 𝐟𝐮𝐥𝐥  kinematic workspace of humanoid robots to the physical world.
AMO is a single policy trained with RL + Hybrid Mocap & Trajectory‑Opt.

Accepted to #RSS2025.
Try our open models & more…

Unified World Models: Coupling Video and Action Diffusion
for Pretraining on Large Robotic Datasets

- combine video and action diffusion in one transformer, can be train from robot trajectories and action-free videos.

- represents a policy, a forward dynamics model, an inverse dynamics model, and a video prediction model in a unified framework.

#RSS2025 #diffusion
X overview, Project Website

🧵 Thread • FixupX

Chuning Zhu (@chuning_zhu)

Scaling imitation learning has been bottlenecked by the need for high-quality robot data, which are expensive to collect. But are we utilizing existing data to the fullest extent? A thread (1/11)

TWIST Teleoperated Whole-Body Imitation System

- single network for whole-body control

#arxiv #humanoid

X Overview, Project Website

🧵 Thread • FixupX

1:30

Yanjie Ze (@ZeYanjie)

🤖Introducing TWIST: Teleoperated Whole-Body Imitation System.

We develop a humanoid teleoperation system to enable coordinated, versatile, whole-body movements, using a single neural network.

This is our first step toward general-purpose robots.

🌐http://humanoid…

PyRoki A Modular Toolkit for Robot Kinematic Optimization

- better performance than cuRobo
- GPU-assisted trajectory optimization toolbox

#arxiv #toolbox #trajopt

X Overview, Project Website

🧵 Thread • FixupX

Chung Min Kim (@ChungMinKim)

Excited to introduce PyRoki ("Python Robot Kinematics"): easier IK, trajectory optimization, motion retargeting... with an open-source toolkit on both CPU and GPU

VideoMimic Visual imitation enables contextual humanoid control

- used human video to learn policy for humanoid
- real-to-sim-to-real tendency

#arxiv #real2sim2real #humanoid #reconstruction

X Overview, Project Website

🧵 Thread • FixupX

1:35

Arthur Allshire (@arthurallshire)

our new system trains humanoid robots using data from cell phone videos, enabling skills such as climbing stairs and sitting on chairs in a single policy

(w/ @redstone_hong @junyi42 @davidrmcall)

Instant Policy: In-Context Imitation Learning via Graph Diffusion

- formulate in-context imitation learning as a graph generation problem
- demonstrate impressive one-shot learning performance

#ICLR2025 #manipulation #in_context

X Overview, Project Website

🧵 Thread • FixupX

Edward Johns (@Ed__Johns)

Happy to announce that "Instant Policy" was accepted at ICLR 2025!

Below is a single uncut video: the robot is learning several novel tasks instantly, after just one demonstration each ...

We've achieved in-context learning in robotics!

http://www.robot…

Vision Language Models are In-Context Value Learners

- formulating value learning as an autoregressive prediction task over shuffled sequence of the input video
- suitable across different tasks and in-context learning

#ICLR2025 #manipulation #in_context

X Overview, Project Website

🧵 Thread • FixupX

Jason Ma (@JasonMa2020)

Excited to finally share Generative Value Learning (GVL), my @GoogleDeepMind project on extracting universal value functions from long-context VLMs via in-context learning!

We discovered a simple method to generate zero-shot and few-shot values for 300+…

Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation

- Combined ABD with IPC and implemented in Warp
- Good implementation of tactile sensor

#arxiv #tactile #warp

X Overview, Project Website

🧵 Thread • FixupX

2:27

Siyuan Huang (@siyuanhuang95)

🤖🤖🤖 Following RoboVerse, we introduce another work focused on Robotic Tactile Simulation - Taccel Simulator. Taccel is a high-performance simulation platform for vision-based tactile sensors and robots.

🚀🚀🚀 Boosted by Nvidia Warp, we optimize Taccel with…

Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin for Real-World Robot Policy Evaluation

- Incorporate previous work embodied gaussians as a novel representation of the real world
- using gaussian splatting and warp to reconstruct the scene in simulation

#arxiv #simulation #reconstruction

X Overview, Project Website, Embodied Gaussians

FixupX

Jad Abou-Chakra (@jadachakra)

1/ Real2Sim and Sim2Real are well-known paradigms in robotics🤖. Is there another paradigm where sim and the real world are more tightly interlinked? We introduce Real-IS-Sim: a framework featuring an always-in-the-loop, correctable simulator for behavior…

Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning

- two stage training for in-hand dexterous manipulation
- using residual action to refine the manipulation

#CVPR2025 #dexterous

X Overview, Project Link

FixupX

1:59

Siyuan Huang (@siyuanhuang95)

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25).

🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using…

PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos

- utilize scene reconstruction to create digital twin
- using warp as the simulation pipeline

#arxiv #deformable #reconstruction #simulation

X Overview, Project Link, Arxiv Link

FixupX

Hanxiao Jiang (@jiang_hanxiao)

🚀 How can we create interactive Physical Digital Twins from videos?

Thrilled to share our latest work: PhysTwin! 🌟 Using inverse physics optimization, we generate photo-realistic, physically accurate, and real-time interactive virtual replicas. 🔥

🔗http…