tensorqt @tensorqt
chaos dancing star Icecrown Citadel Joined February 2022-
Tweets6K
-
Followers3K
-
Following300
-
Likes26K
FlashAttention in 3D? Our latest blog explores the #kernel design of 2-Simplicial #Attention, modeling the algorithm with a hardware aligned design and rewriting the entire kernel in TLX (Triton Low Level Extensions). 🔗 hubs.la/Q03H6S9D0 #PyTorch #OpenSourceAI
we are so early
*Advanced automatic differentiation* Twitter friends, I finally released a (drafty) extended chapter on autodiff, covering general vector spaces, implicit autodiff, and multilinear algebra (largely based on material from @mblondel_ml & Roulet). 🙂 sscardapane.it/assets/alice/A…
I agree this is kind of the weakpoint here, especially since the last checkpoint is the one where Muon and SOAP really degrade to ~adamw performance. with that said the paper is very good
I agree this is kind of the weakpoint here, especially since the last checkpoint is the one where Muon and SOAP really degrade to ~adamw performance. with that said the paper is very good
We did a very careful study of 10 optimizers with no horse in the race. Despite all the excitement about Muon, Mars, Kron, Soap, etc., at the end of the day, if you tune the hyperparameters rigorously and scale up, the speedup over AdamW diminishes to only 10% :-( Experiments…
We did a very careful study of 10 optimizers with no horse in the race. Despite all the excitement about Muon, Mars, Kron, Soap, etc., at the end of the day, if you tune the hyperparameters rigorously and scale up, the speedup over AdamW diminishes to only 10% :-( Experiments…
the physics part of my brain wants to express each of these trade-offs as some form of conserved energy and do constrained optimization
the physics part of my brain wants to express each of these trade-offs as some form of conserved energy and do constrained optimization
Today, we are releasing FineVision, a huge open-source dataset for training state-of-the-art Vision-Language Models: > 17.3M images > 24.3M samples > 88.9M turns > 9.5B answer tokens Here are my favourite findings:
"we may never know, Uther, I intend to live forever."
"we may never know, Uther, I intend to live forever."
very proud to see one of my earliest moots now be one of the leading faces of the ML scene
very proud to see one of my earliest moots now be one of the leading faces of the ML scene
The Lore of Kalomaze! ⚡️ bringing a great pod with @kalomaze (20yo ml researcher, prime intellect) - we'd talked about training, finetuning, RL (environments and recipes), scaling, working at PI and a Lot of Lores! (link in replies)
i am somewhat growing skeptical of looping layers as an architectural strategy (as some of you may know, i've been quite a fan). Something is still missing imo, probably a combination of: - a proper experimental demonstration of looping being worth the FLOPs and - a hierarchical…
i am somewhat growing skeptical of looping layers as an architectural strategy (as some of you may know, i've been quite a fan). Something is still missing imo, probably a combination of: - a proper experimental demonstration of looping being worth the FLOPs and - a hierarchical…
Another active stream of Language Modeling literature investigates whether, and how, one can adapt a pretrained model to perform better on a given task, without any additional continued-pretraining, nor fine-tuning. At the current stage, two ideas have emerged: layer-pruning to…
μtransfer for Mamba2 & Muon
not sure everyone here had caught on on my bio
While we’re still wondering if there could be more than a handful of labs in Europe, even the local Chinese DoorDash is pretraining giant MoEs.
While we’re still wondering if there could be more than a handful of labs in Europe, even the local Chinese DoorDash is pretraining giant MoEs.
New post! The fact that we experience life through what feels like a singular entity, I believe, is chance adaptation rather than a given rule of life. If our environments warranted it and evolution proceeded a little differently, experience might take on other forms
this is very very true. i think easiest example is when you are being hosted by a friend: a male friend will throw a mattress on the ground and that's where you're gonna sleep. any woman will treat you like a PRINCE and make hotels pale in comparison
this is very very true. i think easiest example is when you are being hosted by a friend: a male friend will throw a mattress on the ground and that's where you're gonna sleep. any woman will treat you like a PRINCE and make hotels pale in comparison
> fading fever > thunderstorm outside > mcdonalds > psytrance > tight deadline locking in
david glukhov @DavidGlukhov
91 Followers 186 Following
Jimmy Smith @jimmysmith1919
472 Followers 289 Following Founding Scientist @LiquidAI_. Previously PhD @Stanford and member of @scott_linderman's lab.
Ankur Bohra @AnkurBohra9
43 Followers 4K Following
Anurag Bagchi @Miccooper9
9 Followers 1K Following
Tuan Tran @vantuan5644
33 Followers 530 Following
Zacky Zack @demnatizakaria
27 Followers 799 Following
Clive Chan @itsclivetime
11K Followers 3K Following intelligence per picojoule @openai / prev led dojo workload @tesla
Hailey Collet @HaileyStormC
317 Followers 102 Following Mother. Ex Controls Engineer. Software dev. AI enthusiast & tinkerer.
Y C @YC87148229
0 Followers 1K Following
acem @acemtwayana
18 Followers 192 Following
kaushikpatnaik @kaushikpatnaik
990 Followers 3K Following Generative Human. Current: Leading Multimodal Agents @Meta Wearables. Prev: co-founder @ai_minion, AR/VR Research (VisionPro) @ 🍎. Opinions my own.
🌎 @ascetic_one
34 Followers 1K Following
Alex Speicher @AlxSp_
6 Followers 61 Following
z @betaokumura
317 Followers 3K Following a dog in human’s clothing | interested in math + informatics + chemistry (sometimes even philosophy) | no hate but curiosity really killed the cat 😉
Wen-Ding Li @xu3kev
3K Followers 6K Following LLM for code and reasoning. PhD student at Cornell. Previously Student Researcher at @google. Previously intern at @theteamatx.
Yong-Hyun Park @hagsaeng_bag
379 Followers 1K Following Incoming PhD. student @CIS_Penn. prev. MS @SNU research intern @SonyAI_global, @NAVER_AI_Lab.
Udit Saxena @saxenaudit
166 Followers 920 Following Amateur vintner, coder, photographer - aspiring polymath
Ben @arguableutopia
80 Followers 1K Following
Michel Aractingi @AractingiMichel
322 Followers 552 Following Doing robotics at @huggingface @LeRobotHF https://t.co/yj99DBAQ0l
Edouardo Honig @e__honig
168 Followers 2K Following
iksrat @c0mbinat0r
23 Followers 263 Following
Nathan Chen @nathancgy4
656 Followers 562 Following @tilderesearch trying to (pragmatically) understand my friend, ml & open-source, 16
Théo Pomies @theopomies
287 Followers 2K Following CTO @ https://t.co/BRhXDz6tjE, Roman Catholic, Bitcoin Maximalist, AI dabbler, Weight training x Training weights
sid bharthulwar @bharthulwar
296 Followers 1K Following research on large-scale rl, diffusion models, symmetry learning, synthetic data. prev @harvard @twosigma @bridgewater
Anh Nguyen @AnhNguyenWho
324 Followers 6K Following opinions expressed are my own | explore @netflix | @tamu alum | prev. @kumo_ai_team, @tobikodata, @snapchat, @confluent
Shaun Tucker @ShaunTu34460624
0 Followers 115 Following
Pume Tuchinda @pumetuchinda
7 Followers 145 Following Research Assistant @VISTEC_Thailand | @PurdueECE 2023
Tim Kellogg @kellogh
1K Followers 653 Following AI architect // hiking, camping and long walks on the beach as long as they involve backpacking
SenaBeren @findingmerit
305 Followers 5K Following
云创兽Ai @Arcruilal79725
4 Followers 66 Following ⚡ coolly chasing stock investing, I’m a wealth goddess! open to insights. DM me for Hong Kong markets! 💬 #OptionsTrading
Sohir Maskey @SohirMaskey
177 Followers 469 Following AI Researcher @Aleph__Alpha | prev PhD @LMU_Muenchen | Working on GenAI | Interested in Geometric DL and the beautiful math behind.
Irpolo @Irpolo941320
36 Followers 2K Following
Anurag Singh @ashleymavericks
645 Followers 965 Following automate everything | self-host every service | build tiny projects | indie hacker | raspi • esp32 tinkerer | systematic investor
Zaraki Kenpachi fan @p0nnnpppppppppp
514 Followers 918 Following | professional bed rotter | drug fiend in the making | among aunt juwee's favourite nephews | dead (incoming)
Rishabh Agarwal @agarwl_
17K Followers 792 Following Reinforcement Learner, Adjunct Prof at McGill. Ex MSL Meta, DeepMind, Brain, Mila, IIT Bombay. NeurIPS Best Paper
Valeria Ruscio @RuscioValeria
124 Followers 162 Following Research fellow @ Sapienza, RSTLess lab. Working on representation learning and spectral probing for LLMs.
Federico Barbero @fedzbar
3K Followers 291 Following I like Transformers and graphs. I also like chess and a few other things as well @googledeepmind @compscioxford
Jesse Michael Han @jessemhan
3K Followers 583 Following @morph_labs // prev. research @OpenAI / PhD in math and neural theorem proving
Leo @AgenticLeo
343 Followers 711 Following Founder engineering autonomous AI systems | Obsessed with machine-to-machine communication | RM - SF
Dimitri von Rütte @dvruette
2K Followers 307 Following PhD @ETH_en, prev. Machine Learning @DeepJudgeAI
maja 🔭🍒 @majamediaco
6K Followers 1K Following searching for the miraculous in the mundane. yearning, learning and earning.
Jack Morris @jxmnop
45K Followers 974 Following research @cornell @meta // language models, information theory, science of AI
Checkpoint @0xcpoint
7 Followers 1 Following
Lorenzo ventrone @Lorenzo_V98
1 Followers 2 Following
Diego Pizzocaro @diegopizzocaro
500 Followers 951 Following CEO @ H-FARM AI - exCMO @ ForceManager (exit) - prev. CEO/founder of Sellf (exit)
Mazeyar Moeini 👨�... @mazy1998
544 Followers 433 Following CTO https://t.co/TS6khHv0jO | Gaussian Splatter | Artist | Proverbs 1:7
Ahmad @TheAhmadOsman
13K Followers 240 Following ai research & software engineering, on a mission to build a DGX B300 GPU cluster, i moderate GPUs on r/LocalLLaMA
Emanuele Rodolà @EmanueleRodola
812 Followers 91 Following Head of GLADIA @SapienzaRoma and fellow of @SSAS_Sapienza, @ELLISforEurope and @yacadeuro. Lover of crazy ideas and anything passion-driven. #EUFunded
Andrea Santilli @teelinsan
836 Followers 2K Following Research Scientist in NLP & LLMs | Prev: @Apple, @NousResearch, @BigscienceW, @picampusschool #NLProc
Daniele Solombrino @dansolombrino
20 Followers 32 Following 📚 Computer Science BSc and MSc. 🎓 Deep Learning PhD student @ GLADIA (Sapienza). 🔬 Research: model merging, audio gen, music edit.
BabelDev @BabelDev
92 Followers 254 Following Curiosity drives discovery ~ 🎨 Polymath ~ 🌌 MSc quantum engineering ~ 💻 BSc computer engineering https://t.co/22ScyNJ4Zo
Fernando Nardi @fnardi
2K Followers 2K Following @unitorvergata Associate professor of #hydrology #hydraulics @IAHS_AISH Academy chair @FIUEnvironment visiting prof #Unescochair @NexusNess_Prima coordinator
Aurko Roy @aurko79
2K Followers 228 Following ML research | @AIatMeta (2025-2025) | @GoogleDeepmind (2023-2025) | @GoogleAI (Brain) (2017-2023) | CS PhD @Georgiatech | CS @IITKanpur
Giorgio Strano @krytox205
34 Followers 62 Following
Ariel @redtachyon
752 Followers 203 Following p/hd | Big RL energy | 0.71 |research⟩ + 0.71 |engineer⟩ @ Meta, but never speaking on behalf of the company | Prev. lead maintainer of Gymnasium
Asuka🎀Redpanda @VoidAsuka
20K Followers 4K Following senior sde, junior ai research engineer. share something good/fun.
Nicolò Monti @NMonti25537
32 Followers 237 Following
furlanel @furlanel
271 Followers 353 Following
jianlin.su @Jianlin_S
3K Followers 14 Following Grad is all you need @Kimi_Moonshot Blog: https://t.co/YVxsWylklA , Cool Papers: https://t.co/scS1n1oyaO
Andrea Calcagni @Andreacalc
4 Followers 138 Following
Enrico Bertino @EnricoBerto91
63 Followers 128 Following Natural Language Processing 📣, Deep Learning 🧮 and Computational Linguistics 🤖 Co-founder @ https://t.co/XSzDJTGLsQ
peppe @intillagiuseppe
4 Followers 4 Following
Shaurya Sharthak @ShauryaSharthak
71 Followers 83 Following LowIQGenAI | यदा यदा हि धर्मस्य ग्लानिर्भवति भारत अभ्युत्थानमधर्मस्य तदात्मान सजाम्यहम परित्राणाय साधूना विनाशाय च दुष्कताम धर्मसस्थापनार्थाय सम्भवामि युगे युगे
Simone Facchiano @SimoneFacchiano
15 Followers 117 Following On my quest for solving Artificial Intelligence @SapienzaRoma
Donato Crisostomi @DonatoCrisosto1
345 Followers 423 Following PhD stud. @ELLISforEurope, Sapienza / Uni Cambridge | prev visiting @Cambridge_Uni | ex Science intern @Amazon Search & @AmazonAlexa
Ruins @vandaloruins
5K Followers 4K Following Expanding literature as @darktalesxyz / Previously team @rarible
Ludovico Comito @ludocomito
32 Followers 165 Following Currently loving AI🧠, Robotics🦾 and Photography📷
Nantas Nardelli @nntsn
2K Followers 3K Following Climate change monomaniac. Past: ML @ Pacific Fusion, Carbon Re, various AI labs. 🇮🇹 🌍
jellybean ❄️ @jdchawla29
625 Followers 313 Following aspiring clown | watching patterns unfold into agi | 23 l jiggery-pokerying... @hud_evals
Elias @fcvprzhfgsybj
1K Followers 773 Following Energy efficient chips for AI - CEO @thezettascale (YC S24) - hiring: https://t.co/Sxv0609V84Trends for United States
You might like
