Le et al., "Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels"
Feed-forward mapping of material property from CLIP, powered by synthetic data. Can be distilled to 3D representations (eg NeRF/3DGS) to do some cool physics stuff.
Check out our recent work on fast physical property prediction for multi-material objects from images, led by our amazing student @LongLeRobot ! We’ve also released our annotations and data generation pipeline (github.com/vlongle/pixie) for 3D objects with physical material labels…
Check out our recent work on fast physical property prediction for multi-material objects from images, led by our amazing student @LongLeRobot ! We’ve also released our annotations and data generation pipeline (github.com/vlongle/pixie) for 3D objects with physical material labels…
Those who works in robot sim2real knows that the visual gap is very real and non-trivial 😤
so it was very exciting to see Pixie zero-shot generalize to real-world scenes out of the box!
(P.S. I also had a lot of fun making these web visualization @ pixie-3d.github.io)
Those who works in robot sim2real knows that the visual gap is very real and non-trivial 😤
so it was very exciting to see Pixie zero-shot generalize to real-world scenes out of the box!
(P.S. I also had a lot of fun making these web visualization @ pixie-3d.github.io) https://t.co/WAitAi3HYP
The first step in “world models” has been visual modeling—great for flashy demos, but robots need more than watching the world, they need to act in it. Visuals may aid semantics & planning, yet it’s physics that dictates complex object interactions. Excited to see @LongLeRobot…
The first step in “world models” has been visual modeling—great for flashy demos, but robots need more than watching the world, they need to act in it. Visuals may aid semantics & planning, yet it’s physics that dictates complex object interactions. Excited to see @LongLeRobot…
As promised: three commercially valuable tasks continuously demoed live for all attendees
Public demos are a big deal -- there's no reshoots or camera tricks to hide flaws
Our stack is simply robust enough to set up multiple bots and showcase valuable manipulation skills live
As promised: three commercially valuable tasks continuously demoed live for all attendees
Public demos are a big deal -- there's no reshoots or camera tricks to hide flaws
Our stack is simply robust enough to set up multiple bots and showcase valuable manipulation skills live https://t.co/ZfQrCXciQb
"Pixie: Physics from Pixels"
TL;DR: NeRF, GS w/ physics; neural network mapping pretrained visual features (i.e., CLIP) to dense material fields of physical properties in a single forward pass, enabling real‑time physics simulations.
Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels
Contributions:
1. Novel Framework for 3D Physics Prediction: We introduce PIXIE, a unified framework that predicts discrete material types and continuous physical parameters (Young’s modulus, Poisson’s…
617 Followers 685 FollowingPerception for "embodied AI" at StackAV. Visiting Researcher @CMU_robotics. Formerly @motionaldrive @argoai. Opinions are my own.
303 Followers 1K FollowingComputer Vision, Graphics, and ML | @unccs PhD Candidate | Student Researcher at @Google | Interested in many, many things. Optimist.
538K Followers 17K FollowingThe best from AI community | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future | Silicon Valley robots, holodecks, BCIs, and startups.
5K Followers 1K FollowingDocumenting the exciting progression of radiance field based technologies, including but not limited to Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting
617 Followers 685 FollowingPerception for "embodied AI" at StackAV. Visiting Researcher @CMU_robotics. Formerly @motionaldrive @argoai. Opinions are my own.
303 Followers 1K FollowingComputer Vision, Graphics, and ML | @unccs PhD Candidate | Student Researcher at @Google | Interested in many, many things. Optimist.
538K Followers 17K FollowingThe best from AI community | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future | Silicon Valley robots, holodecks, BCIs, and startups.
14K Followers 519 FollowingYour guide to radiance fields | Host of the podcast @ViewDependent | DM open for business inquiries | https://t.co/llYGWliKUv | discord: https://t.co/lrl64WGvlD
2K Followers 266 FollowingComputer Vision Research Scientist at @simulon, music lover , fond of scientific/musical/geeky/useless stuff. I'm posting papers on whatever I found amazing :)
58 Followers 18 FollowingResearch associate professor at the University of Pennsylvania. Specializing in lifelong machine learning for robotics and medicine. @LifelongML_Penn @GRASPlab
5K Followers 1K FollowingDocumenting the exciting progression of radiance field based technologies, including but not limited to Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting
19K Followers 3K FollowingMostly posting about robots.
currently AI @agilityrobotics
prev embodied AI @AIatMeta, @NVIDIAAI. All views my own.
writing: https://t.co/iNLA4djfZo
7K Followers 6K FollowingProduct Lead | Google Gemini
Prev: Launched @aws Trainium, @alexa99 Echo Show 5
Tweets are my own. Retweets are not endorsements.
Joyful Learning Machines
107 Followers 89 FollowingI am a PhD student in Computer Vision and Machine Learning @PRS @ETH. I love working on the intersection of 2D and 3D generative models.
No recent Favorites. New Favorites will appear here.