The full PBDL book is available in a single PDF now arxiv.org/pdf/2109.05237, and has grown to 451 pages 😳 Enjoy all the new highlights on generative models, simulation-based constraints and long term stability with diffusion & flow matching 😁
🚨Announcing our #ICLR2025 Oral!
🔥Diffusion LMs are on the rise for parallel text generation! But unlike autoregressive LMs, they struggle with quality, fixed-length constraints & lack of KV caching.
🚀Introducing Block Diffusion—combining autoregressive and diffusion models…
This is interesting as a first large diffusion-based LLM.
Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to…
This is interesting as a first large diffusion-based LLM.
Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to…
Excited to share that I’ve been working on scaling up diffusion language models at Inception. A new generation of LLMs with unprecedented capabilities is coming!
Excited to share that I’ve been working on scaling up diffusion language models at Inception. A new generation of LLMs with unprecedented capabilities is coming!
This is really insane. They took all the bet and scaled up discrete diffusion model to llama-7B scale.
IIRC nobody dared to do this at this scale but these madlads done it. They even fine-tuned it to be a dialogue model.
This is really frontier-level shit that is genuinely new…
Happy to see that our work, LLM-SR, has been accepted to #ICLR 2025! 🇸🇬
If you’re interested in learning how LLMs can be used for scientific equation discovery, check this out 👇🏻
Happy to see that our work, LLM-SR, has been accepted to #ICLR 2025! 🇸🇬
If you’re interested in learning how LLMs can be used for scientific equation discovery, check this out 👇🏻
Introducing NeuralOperator 1.0: a Python library that aims at democratizing neural operators for scientific applications by providing all the tools for learning neural operators in PyTorch : state-of-the-art models, built-in trainers for quick starting and modular neural operator…
Excited to present our #NeurIPS2024 paper: "Simple and Effective Masked Diffusion Language Models" on Thurs at 11:30 a.m. in Hall A-C (#2505)
🔥 Our method, MDLM, almost surpasses AR models in text generation
📕 arxiv.org/abs/2406.07524
💻 github.com/kuleshov-group…
📒…
Here's one to read on your flight to #NeurIPS2024! A flow-matching transformer model in function space! This model has all the advantages of neural fields: resolution-free generation and domain-agnostic architecture, while obtaining strong results on ImageNet-256 and Objaverse!
Here's one to read on your flight to #NeurIPS2024! A flow-matching transformer model in function space! This model has all the advantages of neural fields: resolution-free generation and domain-agnostic architecture, while obtaining strong results on ImageNet-256 and Objaverse!
A common question nowadays: Which is better, diffusion or flow matching? 🤔
Our answer: They’re two sides of the same coin. We wrote a blog post to show how diffusion models and Gaussian flow matching are equivalent. That’s great: It means you can use them interchangeably.
I'm excited to share our APEBench paper arxiv.org/abs/2411.00180 and code github.com/tum-pbs/apeben…, to be presented at #NeurIPS. Congratulations Felix and Simon 😀 👍 APEBench features a lightning-fast ⚡️ fully differentiable spectral solver with a huge range of different PDEs
Super hyped to share NeuralDEM -- the first real-time simulation of industrial particulate flows. NeuralDEM replaces Discrete Element Method (DEM) routines and coupled (CFD-DEM) multiphysics simulations. 🧵
📜: arxiv.org/abs/2411.09678
🖥️: nx-ai.github.io/NeuralDEM/
In #AI+#Science, data is often given on meshes or irregular grids.
For such domains, it is strongly encouraged to avoid referring to them as "Graphs", and use GNNs (models for graphs) on these "non-graph" data.
It is highly encouraged to look at them as what they are, that is…
Our new paper in @NatMachIntell tells a story about how, and why, ML methods for solving PDEs do not work as well as advertised.
We find that two reproducibility issues are widespread. As a result, we conclude that ML-for-PDE solving has reached overly optimistic conclusions.
10K Followers 10K FollowingA multidisciplinary, #openaccess journal devoted to the application and development of #machinelearning for the sciences. Published by @IOPPublishing.
1K Followers 2K FollowingAssistant Professor at Stony Brook University.
Geometric Deep Learning, AI4Science, Scientific Machine Learning, LLMs for Science.
155 Followers 343 FollowingUndergrad@KAIST
Research interests:
Stochastic Differential Geometry(ricci flow), Complex Geometry(optimal transport) and Symplectic Geometry(hamiltonian) in AI
3K Followers 4K FollowingStaff Research Scientist at Google DeepMind. Former adjunct assistant prof at @NYU_Courant. PhD at @mldcmu. ML for Bio/Chem (Prev. NLP).
All opinions my own.
2K Followers 5K FollowingAssistant Professor @McCormackUMass | Isenberg School of Management | Pursuing a Master's in AI | Bourbon lover 🥃 | 2 Timothy 4:7 | Isaiah 42:3
12K Followers 1K FollowingFounder of https://t.co/9KM4uFScMi, Associate Professor at Columbia. Making ai agent design and deployment easy and fast!
Forbes 30 under 30.
11K Followers 417 FollowingCode, AI, and 3D printing. Opinions are mostly my own, sometimes my computer's. Husband of @thesamnichol. Co-creator of DALL-E 2. Researcher @openai.
43K Followers 3K FollowingWe're in a race. It's not USA vs China but humans and AGIs vs ape power centralization.
@deepseek_ai stan #1, 2023–Deep Time
«C’est la guerre.» ®1
3K Followers 178 FollowingResearch Scientist at Google Deepmind. Previous PhD with @wellingmax at the Univ. of Amsterdam, Research intern at Qualcomm AI Research and Google Brain.
3K Followers 276 FollowingIncoming Assistant Professor @imperialcollege and @Mila_Quebec Affiliate member. Into Geometry ∩ Generative Models and AI4Science. Ex-@Mila_Quebec, @UofT.
6K Followers 2K FollowingCo-founder @SilurianAI. Training big neural nets to simulate the world. Previously @MSFTResearch, @Google Brain, @Theteamatx | ML PhD from @Cambridge_Uni
1K Followers 258 FollowingResearch Scientist, FAIR at Meta.
PhD from @MetaAI and Université Gustave Eiffel with Yann LeCun and Laurent Najman.
Ex MVA and @ESIEEPARIS
23K Followers 680 FollowingProfessor and Head of Machine Learning Department at @CarnegieMellon. Board member @OpenAI and @Qualcomm. Chief Technical Advisor @GraySwanAI.
6K Followers 2K FollowingAss. prof. of Machine Learning. PI of Generative Memory Lab (@DondersInst). Statistical physics, generative diffusion, memory, and generalization.