Looking for 25Fall Ph.D.!; Undergraduate @SJTU1896 | @CMU_Robotics; @UCSD. Learn to learn representations and learn to search within and beyond. 🤖🧠👁️theknight-z.github.io Pittsburgh, PAJoined July 2023
90% success rate in unseen environments.
No new data, no fine-tuning.
Autonomously.
Most robots need retraining to work in new places.
What if they didn’t?
Robot Utility Models (RUMs) learn once and work anywhere... zero-shot.
A team from NYU and Hello Robot built a set of…
🏓🤖 Our humanoid robot can now rally over 100 consecutive shots against a human in real table tennis — fully autonomous, sub-second reaction, human-like strikes.
The new humanoid whole-body teleop & policy learning demo from @BostonDynamics & @ToyotaResearch is so impressive. The telelop system includes BD's MPC system as controllers, VR headset, base stations, and trackers on chest & feet. The policy learning architecture is similar to…
🎉 Very happy to have our paper on predictive red teaming accepted to @corl_conf!
Writing a first-author paper again was a good reminder of all the hard work students put in on details that I've forgotten about over the years.
Project page:
rb.gy/buih3o
A large part of Robot Learning is systems research, but it doesn't quite fit the existing paradigms: think of CoRL, RSS, SOSP, and MLSys. It draws from all but is inherently full-stack --- hardware, control, compute, ml algos. There are nice works here and there but we likely…
Another one. Already a powerful painting, but moving around it yourself gives a totally different feeling.
Jacques Louis David's "The Death of Socrates" => #Genie3
World modeling for robotics is incredibly hard because (1) control of humanoid robots & 5-finger hands is wayyy harder than ⬆️⬅️⬇️➡️ in games (Genie 3); and (2) object interaction is much more diverse than FSD, which needs to *avoid* coming into contact. Our GR00T Dreams work was…
World modeling for robotics is incredibly hard because (1) control of humanoid robots & 5-finger hands is wayyy harder than ⬆️⬅️⬇️➡️ in games (Genie 3); and (2) object interaction is much more diverse than FSD, which needs to *avoid* coming into contact. Our GR00T Dreams work was…
What if you could not only watch a generated video, but explore it too? 🌐
Genie 3 is our groundbreaking world model that creates interactive, playable environments from a single text prompt.
From photorealistic landscapes to fantasy realms, the possibilities are endless. 🧵
Excited to open-source GMR: General Motion Retargeting. Real-time human-to-humanoid retargeting on your laptop. Supports diverse motion formats & robots. Unlock whole-body humanoid teleoperation (e.g., TWIST).
video with 🔊
Excited to see that particle-based dynamics model learning + MPC gets applied in cloth manipulation with multifingered hands!
The idea of predicting object motions by representing them as dense particles shares the same insight as our recent paper, PGND…
Excited to see that particle-based dynamics model learning + MPC gets applied in cloth manipulation with multifingered hands!
The idea of predicting object motions by representing them as dense particles shares the same insight as our recent paper, PGND…
We’ve been building quietly — starting tomorrow, we go live.
Here’s a teaser of what we did before Skild AI. It has shaped what’s coming next.
07/29. Stay tuned.
My Takeaways from GR3: Insights on Vision-Language-Action (VLA) Models
Although I wasn’t directly involved in the GR3 project, I gained valuable insights into Vision-Language-Action (VLA) models through discussions and observing their performance, particularly their failure…
My Takeaways from GR3: Insights on Vision-Language-Action (VLA) Models
Although I wasn’t directly involved in the GR3 project, I gained valuable insights into Vision-Language-Action (VLA) models through discussions and observing their performance, particularly their failure…
🚀 New Open-Source Release! PyTorchTNN 🚀
A PyTorch package for building biologically-plausible temporal neural networks (TNNs)—unrolling neural network computation layer-by-layer through time, inspired by cortical processing. PyTorchTNN naturally integrates into the…
Large robot datasets are crucial for training 🤖foundation models. Yet, we lack systematic understanding of what data matters.
Introducing MimicLabs
✅System to generate large synthetic robot 🦾 datasets
✅Data-composition study 🗄️ on how to collect and use large datasets
🧵1/
When facing a new physical manipulation problem, robots should think creatively and solve them in only a few attempts.
We make first steps towards this MacGyver robot in our work Causal-PIK. It enables smart exploration of the massive action space - sometimes faster than humans.
When facing a new physical manipulation problem, robots should think creatively and solve them in only a few attempts.
We make first steps towards this MacGyver robot in our work Causal-PIK. It enables smart exploration of the massive action space - sometimes faster than humans.
🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts.
At TRI, we’ve been researching how LBMs can…
🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts.
At TRI, we’ve been researching how LBMs can… https://t.co/IjzvkhmBwP
Many roboticists focus on designing human-like hands, but we took a closer look at the fingers. Human fingers are soft, responsive, and packed with sensitive sensory points—perfect for handling even fragile objects. What if robots had such fingers?
Check out our work on FORTE!
Many roboticists focus on designing human-like hands, but we took a closer look at the fingers. Human fingers are soft, responsive, and packed with sensitive sensory points—perfect for handling even fragile objects. What if robots had such fingers?
Check out our work on FORTE!
1K Followers 2K FollowingNSF AI Institute with researchers from @UTAustin, @UW, @WichitaState, @MSFTResearch, @UCBerkeley, @UCLA, @sfiscience, @Stanford, @Caltech, @ASU
28 Followers 286 FollowingUndergraduate researcher at Tsinghua University, focusing on World Models and Robotics. Current intern @StanfordSVL. My site: https://t.co/R9bicUDDdk
129 Followers 160 FollowingPhD student at UC Berkeley | Robotics: Science and Systems Pioneer | Qualcomm AI Fellow | World Artificial Intelligence Conference Rising Star
17K Followers 6K FollowingNeurodivergent physics student with a keen interest in multisensory integration and emergent perception. Exploring research on a proposed ‘sixth sense’. Δ
403 Followers 411 FollowingAI Researcher. Visiting researcher in @Mila_Quebec. Previously: MS at @kaist_ai. Interested in conversational agents & intersection between XR + AI nowadays.
1K Followers 3K FollowingVITA Group @UTAustin w/ Prof Atlas Wang | https://t.co/Wi3tJXf1mg Run by VITA students (PI is busy changing diapers😄). Tweets only reflect personal views
124 Followers 11 FollowingResearch lab @UofT @UHN. Our mission is to reverse-engineer the brain to understand the computational principles that drive intelligence. Led by @DrLaschowski
2K Followers 162 FollowingComputational Neuroscientist. Building neural networks to reverse-engineer the brain and human intelligence. Assistant Professor @UofT Research Scientist @UHN
3K Followers 22 FollowingThe Center for Brains, Minds and Machines is a multi-institutional NSF Center dedicated to the study of the science and engineering of intelligence.
20K Followers 3K Following@Cornell's NYC grad school.
Developing leaders, building ventures, and creating technologies for the Al era.
All at #CornellTech
2K Followers 433 FollowingThe Office of the Vice President for Research, Scholarship & Creative Endeavors helps @UTAustin researchers turn bold ideas into discoveries. #TEXASResearch
1K Followers 605 FollowingRobotics Research Scientist @ Robotics and AI Institute (RAI) | Making robots smarter for everyone | CS PhD from @BrownUniversity 🤖
1K Followers 2K FollowingNSF AI Institute with researchers from @UTAustin, @UW, @WichitaState, @MSFTResearch, @UCBerkeley, @UCLA, @sfiscience, @Stanford, @Caltech, @ASU
3K Followers 95 FollowingTalkRL Podcast is All Reinforcement Learning, All the Time.
Follow for interviews with brilliant folks from across the world of RL.
Host @robinc. DMs open.
7K Followers 868 FollowingPrinceton neuro prof. But Twitter is an absurd platform for professional communication so I strive to use it most unprofessionally.
2K Followers 1K FollowingMember of Technical Staff @GoodfireAI; Previously: Postdoc / PhD at Center for Brain Science, Harvard and University of Michigan
559 Followers 1 FollowingSharing daily personal notes on selected interesting Embodied AI papers, blogs and talks | Maintained by @yilun_chen_ | Opinions are my own.
364K Followers 8 FollowingVercel provides the developer tools and cloud infrastructure to build, scale, and secure a faster, more personalized web. Creators of @nextjs, @v0, and @aisdk.
1K Followers 315 FollowingBringing good stuff to the world.
CMU MLD phd. cooked with TPUs at Google Brain.
Leading Tree and Rock AI Lab (TRAIL) at NUS (Singapore)
1K Followers 341 FollowingRobotics & Computer Science Professor, Director of USC School of Advanced Computing, Executive Vice Dean at USC Viterbi School of Engineering. Amazon Scholar.
11K Followers 2K FollowingML meets Neuroscience #NeuroAI, Full Professor at the Institute of Cognitive Science (@UniOsnabrueck), prev. @DondersInst, @Cambridge_Uni #neuroconnectionism
No recent Favorites. New Favorites will appear here.