The Dex team at NVIDIA is defining the bleeding edge of sim2real dexterity. Take a look below 🧵 There's a lot happening at NVIDIA in robotics, and we’re looking for good people! Reach out if you're interested. We have some big things brewing (and scaling :)
We find keypoint trajectories to be a powerful interface between VLM planning & RL control
VLM: Generates object + hand motion plan from a task prompt & RGB-D image (perception + commonsense)
RL policy: Conditioned on the plan, learns low-level dexterous control (0-shot sim2real)
We find keypoint trajectories to be a powerful interface between VLM planning & RL control
VLM: Generates object + hand motion plan from a task prompt & RGB-D image (perception + commonsense)
RL policy: Conditioned on the plan, learns low-level dexterous control (0-shot sim2real)
Teleoperation is slow, expensive, and difficult to scale. So how can we train our robots instead?
Introducing X-Sim: a real-to-sim-to-real framework that trains image-based policies 1) learned entirely in simulation 2) using rewards from human videos.
portal-cornell.github.io/X-Sim
Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input.
The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.
40 Followers 4K FollowingExpert Manager👩
Forex Trader📈
Digital Marketing📡 Over 6 Years Experience, Dm me and start making extra money every week
Weekly Payouts 100% Guaranteed
111 Followers 395 FollowingResearch Scientist @Apple | Sharing notes on @EmbodiedAIRead | Prev @Uber_ATG ,@CMU_Robotics, @Tsinghua_Uni l Robot Learning, Embodied AI | Opinions are my own
3K Followers 366 FollowingResearch Scientist at BIGAI Working on #3d_scene_understanding and #embodied_ai Ph.D. in Statistics from @UCLA Former intern at @DeepMind and @MetaAI
644 Followers 319 FollowingResearch scientist at @rai_inst and fresh PhD in computer science from @crl_ethz, @eth_en. Legged robotics, character animation, optimal control and RL.
2K Followers 730 FollowingSr. Research Scientist at @NVIDIAAI. Foundation models for robotics. Previously PhD at @CMU_Robotics, @Berkeley_EECS, @MetaAI, AWS
6K Followers 1K FollowingAssistant Professor @CMU_Robotics @SCSatCMU, leading the @LeCARLab. @Amazon Scholar. I build generalist robots with agility. Ph.D. from @Caltech.