Recent interesting robotics papers
HIL-SERL: Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning

- sparse reward classifier
- human in the loop online RL training

#arxiv #RL

Project Website
arxiv RL
Scalable Real2Sim: Physics-Aware Asset Generation Via Robotic Pick-and-Place Setups

- automatically generated simulation objects from the real world
- detailed analysis of object parameter estimation procedure

#arxiv #real2sim #reconstruction

X Overview, Project Website
🧵 Thread • FixupX
Nicholas Pfaff (@NicholasEPfaff)
Nicholas Pfaff (@NicholasEPfaff)
New Paper: "Scalable Real2Sim: Physics-Aware Asset Generation via Robotic Pick-and-Place Setups"! 🤖

We introduce a fully automated pipeline that generates simulation-ready assets for real-world objects—no manual intervention needed!

🌐 Website: https://scalable…
arxiv real2sim reconstruction
Visuomotor Policies to Grasp Anything with Dexterous Hands

- grasping based on stero input
- engineering example of teacher-student, sim2real, and geometric fabrics

#arxiv #dexterous #grasp #sim2real

X Overview, Project Website, Geometric Fabrics
🧵 Thread • FixupX
Ritvik Singh (@ritvik_singh9)
Excited to announce our latest work: DextrAH-RGB where we successfully train visuomotor policies in sim to perform dexterous grasping of arbitrary objects end-to-end from RGB input!
arxiv dexterous grasp sim2real
GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

- combination of sim and real could help train robust grasping policy
- engineering example of pi-0 policy (VLM + flow matching)

#arxiv #grasp #simulation #sim2real #flow_matching

X Overview, Project Website, NVidia GraspGen
🧵 Thread • FixupX
Tuo Liu (@TuoLiuRobot)
Tuo Liu (@TuoLiuRobot)
Galbot（银河通用）has just dropped GraspVLA, the world’s first end-to-end embodied grasping model! Trained on over a billion “visual-language-action” pairs, GraspVLA is ready to handle a wide range of tasks, even the unexpected ones.

And with a little post-training…
arxiv grasp simulation sim2real flow_matching
FALCON: Learning Force-Adaptive Humanoid Loco-Manipulation

- combines upper and lower body for policy training
- curriculum training for force adaptation

#arxiv #humanoid #locomanip

X Overview, Project Website
🧵 Thread • FixupX
Yuanhang Zhang (@Yuanhang__Zhang)
🦾How can humanoids unlock real strength for heavy-duty loco-manipulation?

Meet FALCON🦅: Learning Force-Adaptive Humanoid Loco-Manipulation.

🌐: https://lecar-lab.github.io/falcon-humanoid/

See the details below👇:
arxiv humanoid locomanip
Steerable Scene Generation with Post Training and Inference-Time Search

- use a diffusion-based framework to generate scalable scenes on procedural data
- use a Monte Carlo Tree Search to find physical feasibility and satisfy downstream objective

#arxiv #diffusion #generative

X Overview, Project Website
🧵 Thread • FixupX
Nicholas Pfaff (@NicholasEPfaff)
Want to scale robot data with simulation, but don’t know how to get large numbers of realistic, diverse, and task-relevant scenes?

Our solution:
➊ Pretrain on broad procedural scene data
➋ Steer generation toward downstream objectives

🌐 https://steerable…
arxiv diffusion generative
TWIST Teleoperated Whole-Body Imitation System

- single network for whole-body control

#arxiv #humanoid

X Overview, Project Website
🧵 Thread • FixupX
Yanjie Ze (@ZeYanjie)
🤖Introducing TWIST: Teleoperated Whole-Body Imitation System.

We develop a humanoid teleoperation system to enable coordinated, versatile, whole-body movements, using a single neural network.

This is our first step toward general-purpose robots.

🌐http://humanoid…
arxiv humanoid
PyRoki A Modular Toolkit for Robot Kinematic Optimization

- better performance than cuRobo
- GPU-assisted trajectory optimization toolbox

#arxiv #toolbox #trajopt

X Overview, Project Website
🧵 Thread • FixupX
Chung Min Kim (@ChungMinKim)
Excited to introduce PyRoki ("Python Robot Kinematics"): easier IK, trajectory optimization, motion retargeting... with an open-source toolkit on both CPU and GPU
arxiv toolbox trajopt
VideoMimic Visual imitation enables contextual humanoid control

- used human video to learn policy for humanoid
- real-to-sim-to-real tendency

#arxiv #real2sim2real #humanoid #reconstruction

X Overview, Project Website
🧵 Thread • FixupX
Arthur Allshire (@arthurallshire)
our new system trains humanoid robots using data from cell phone videos, enabling skills such as climbing stairs and sitting on chairs in a single policy

(w/ @redstone_hong @junyi42 @davidrmcall)
arxiv real2sim2real humanoid reconstruction
Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation

- Combined ABD with IPC and implemented in Warp
- Good implementation of tactile sensor

#arxiv #tactile #warp

X Overview, Project Website
🧵 Thread • FixupX
Siyuan Huang (@siyuanhuang95)
🤖🤖🤖 Following RoboVerse, we introduce another work focused on Robotic Tactile Simulation - Taccel Simulator. Taccel is a high-performance simulation platform for vision-based tactile sensors and robots.

🚀🚀🚀 Boosted by Nvidia Warp, we optimize Taccel with…
arxiv tactile warp
Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin for Real-World Robot Policy Evaluation

- Incorporate previous work embodied gaussians as a novel representation of the real world
- using gaussian splatting and warp to reconstruct the scene in simulation

#arxiv #simulation #reconstruction

X Overview, Project Website, Embodied Gaussians
FixupX
Jad Abou-Chakra (@jadachakra)
1/ Real2Sim and Sim2Real are well-known paradigms in robotics🤖. Is there another paradigm where sim and the real world are more tightly interlinked? We introduce Real-IS-Sim: a framework featuring an always-in-the-loop, correctable simulator for behavior…
arxiv simulation reconstruction
PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos

- utilize scene reconstruction to create digital twin
- using warp as the simulation pipeline

#arxiv #deformable #reconstruction #simulation

X Overview, Project Link, Arxiv Link
FixupX
Hanxiao Jiang (@jiang_hanxiao)
🚀 How can we create interactive Physical Digital Twins from videos?

Thrilled to share our latest work: PhysTwin! 🌟 Using inverse physics optimization, we generate photo-realistic, physically accurate, and real-time interactive virtual replicas. 🔥

🔗http…
arxiv deformable reconstruction simulation
Empirical Analysis of Sim-and-Real Cotraining Of Diffusion Policies For Planar Pushing from Pixels

- detailed analysis of co-training on DP sim-and-real

#arxiv #manipulation

X Overview, Arxiv Link
FixupX
Adam Wei (@adamwei_)
Adam Wei (@adamwei_)
Learning from both sim+real data could scale robot imitation learning. But what are the scaling laws & principles of sim+real cotraining?

We study this in the first focused analysis of sim+real cotraining spanning 250+ policies & 40k+ evals https://arxi…
arxiv manipulation
DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset

- benchmark with precise calibration
- it is mind-blowing to see how modern vision model could help to calibrate automatically

#arxiv #dataset

X Overview, Project Website
🧵 Thread • FixupX
Zubair Irshad (@mzubairirshad)
Introducing ✨Posed DROID✨, results of our efforts at automatic post-hoc calibration of a large-scale robotics manipulation dataset.

We provide:
🤖 ~36k calibrated episodes with good quality extrinsic calibration
🦾 ~24k calibrated multi-view episodes with…
arxiv dataset
π0.5: a VLA with Open-World Generalization

- following work of pi-0
- utilize co-train and use VLM to identify low-level tasks from high-level command

#arxiv #manipulation

X Overview, Project Website, pi-0
🧵 Thread • FixupX
Chelsea Finn (@chelseabfinn)
Introducing π-0.5!

The model works out of the box in completely new environments.

Here the robot cleans new kitchens & bedrooms. 🤖

Detailed paper + videos in more than 10 unseen rooms:
http://physicalintelligence.company/blog/pi05

A short thread 🧵
arxiv manipulation
 
 
Back to Top