Arindam @halg0rithmist
22 Aspiring theoretician Interested in decision making algorithms India Joined January 2025-
Tweets46
-
Followers33
-
Following1K
-
Likes751
I wrote this last year, and it doesn't hurt to share it again. A phone-formatted ELBO TL;DR.
This is the syllabus of the course @geoffreyhinton and I taught in 1998 at the Gatsby Unit (just after it was founded). Notice anything?
Asking a non-expert in AI but as a (discrete) mathematician: does anyone really know how deep learning works mathematically?
LLMs can be programmed by backprop 🔎 In our new preprint, we show they can act as fuzzy program interpreters and databases. After being ‘programmed’ with next-token prediction, they can retrieve, evaluate, and even *compose* programs at test time, without seeing I/O examples.
Wrapped up Stanford CS336 (Language Models from Scratch), taught with an amazing team @tatsu_hashimoto @marcelroed @neilbband @rckpudi. Researchers are becoming detached from the technical details of how LMs work. In CS336, we try to fix that by having students build everything:
Our interpretability team recently released research that traced the thoughts of a large language model. Now we’re open-sourcing the method. Researchers can generate “attribution graphs” like those in our study, and explore them interactively.
It was a dream come true to teach the course I wish existed at the start of my PhD. We built up the algorithmic foundations of modern-day RL, imitation learning, and RLHF, going deeper than the usual "grab bag of tricks". All 25 lectures + 150 pages of notes are now public! 🧵
L1 regularization for sparse solutions - as usually taught - is actually terrible in practice! I’m always surprised how few people know this. To get good results, retrain with the sparsity pattern found from the initial L1 run, but without the regularizer. Works much better.
L1 regularization for sparse solutions - as usually taught - is actually terrible in practice! I’m always surprised how few people know this. To get good results, retrain with the sparsity pattern found from the initial L1 run, but without the regularizer. Works much better.
We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name - system prompt learning? Pretraining is for knowledge. Finetuning (SL/RL) is for habitual behavior. Both of these involve a change in parameters but a lot of human…
And to finish off, Lectures 21 - 23: - AI for Mathematics: youtu.be/ToY57HgQKXA - Multimodal I (CLIP / Llava): youtu.be/5uI5WOpq8LQ - Multimodal II (VQVAE / Chameleon): youtu.be/VismiXpCs_Y
And to finish off, Lectures 21 - 23: - AI for Mathematics: youtu.be/ToY57HgQKXA - Multimodal I (CLIP / Llava): youtu.be/5uI5WOpq8LQ - Multimodal II (VQVAE / Chameleon): youtu.be/VismiXpCs_Y
first time someone talking about 2045 AGI timeline
first time someone talking about 2045 AGI timeline
Today, we're announcing our $50M Series A and sharing a preview of Ember - a universal neural programming platform that gives direct, programmable access to any AI model's internal thoughts.
This paper also recommended for understanding GRPO. TLDR is that the output-length normalization in DeepSeek GRPO is making models not penalize repetitive behaviors while rewarding shorter correct responses. Same intuition as the last RL paper I posted. Writeup soon.
This paper also recommended for understanding GRPO. TLDR is that the output-length normalization in DeepSeek GRPO is making models not penalize repetitive behaviors while rewarding shorter correct responses. Same intuition as the last RL paper I posted. Writeup soon.
@natolambert I know you all uploaded GGUFs, but also just uploaded other GGUF formats + dynamic 4bit bitsandbytes and general 4bit BnB versions! Dynamic 4bit BnB: huggingface.co/unsloth/OLMo-2… 4bit BnB: huggingface.co/unsloth/OLMo-2… GGUFs: huggingface.co/unsloth/OLMo-2… Fantastic true open source model!
My Gemma-3 analysis: 1. 1B text only, 4, 12, 27B Vision + text. 14T tokens 2. 128K context length further trained from 32K 3. Removed attn softcapping. Replaced with QK norm 4. 5 sliding + 1 global attn 5. 1024 sliding window attention 6. RL - BOND, WARM, WARP Detailed analysis:…
Kathleen @kathleengalenti
287 Followers 3K Following
Ma @ma_hardy_
283 Followers 3K Following
Eunice monicaro @AdkKoba25755
108 Followers 2K Following Disponible pour les plans occasionnels de sexes payantes❤️😘🤑
Deborah @d_tison14
273 Followers 3K Following
Angela @martinangela79
252 Followers 3K Following
Doris @s_doris11
305 Followers 3K Following
EmpressMiaWilson @bmf3MXm4P93924
3 Followers 640 Following Living life on my terms, chasing my goals.
Orhauji @Orhauji303824
31 Followers 1K Following
Connor Treacy @theconnortreacy
12K Followers 13K Following
Markus Brunnermeier @MarkusEcononmis
68 Followers 371 Following Professor @PrincetonEcon , Director of @PrincetonBCF , Research on Macro, Money, and Finance, Author The Resilient Society and A Crash Course on Crises
Jayden Teoh @jayden_teoh_
127 Followers 1K Following research intern @MSFTResearch nyc, undergrad cs, looking for PhD in ML starting fall 2026
21 sahil katiyar @21_katiyar
7 Followers 108 Following Data Scientist student Github - https://t.co/23WO09bZJc
Anna @EleanoraB87046
96 Followers 4K Following
Shashwat Kumar @iamshashwatk
371 Followers 339 Following Quantum Computing | PhD Candidate @Princeton | Student Researcher at @GoogleQuantumAI | BS-MS Physics @IiserMohali | @EPFL | Travelling & Hiking 🏔
Michal Valko @misovalko
8K Followers 8K Following Building something new · Chief Models Officer @ Stealth Startup & Inria & MVA - Ex: Llama @AIatMeta Gemini and BYOL @GoogleDeepMind
Priyesh Vijayan @vijayan_priyesh
23 Followers 492 Following Ph.D. student @Mila_Quebec | Ex @rbc_dsai_iitm @iitmadras #Areas: Reinforcement Learning and Graph-based Learning
Aayush Karan @aakaran31
475 Followers 748 Following PhD student @Harvard and @GoogleDeepMind | Algorithmic insights for generative machine learning | @PDSoros 2024 | Prev @citsecurities, @Apple
Jack Mayo @JackJMayo1
298 Followers 686 Following ML theory PhD with @tverven. Sequential data, parameter-free learning, bandits and RL. Building @kurtos_ai.
Mirco Mutti @mirco_mutti
631 Followers 641 Following Postdoc @TechnionLive. PhD from @polimi. Reinforcement learning, but without rewards.
Mary Johnson @MaryJohnso27431
106 Followers 2K Following
Klee @xiaonando
6 Followers 727 Following
Honam Wong @MH2023ML
405 Followers 1K Following CS PhD @Penn | Prev @HKUST🇭🇰 | Theory and Empirical Science of Deep Learning
Daniel Samanez @DanielSamanez3
2K Followers 6K Following consciousness accelerationist - ai non determinist computing physics philosophy… trying to never forget that in our infinite ignorance we are all equal -popper-
Jason Lee @jasondeanlee
18K Followers 4K Following Associate Professor at UC Berkeley. Former Research Scientist at Google DeepMind. ML/AI Researcher working on foundations of LLMs and deep learning.
Pragya Srivastava @Pragya2k
108 Followers 282 Following Researcher @GoogleDeepMind Prev @MSFTResearch, Undergrad @iitdelhi
Bharath Kumar @bharathkumar__1
544 Followers 518 Following CTO @mayaresearch_ai, MSCE @nyuniversity | Ex - @Oracle
Dylan HadfieldMenell @dhadfieldmenell
4K Followers 2K Following Associate Prof @MITEECS working on value (mis)alignment in AI systems; @[email protected]; he/him
idan shenfeld @IdanShenfeld
239 Followers 100 Following PhD student @MIT, Student Researcher @GoogleDeepMind
Dmytro Mishkin 🇺�... @ducha_aiki
24K Followers 703 Following Marrying classical CV and Deep Learning. I do things, which work, rather than being novel, but not working.
Sanja Fidler @FidlerSanja
16K Followers 489 Following Associate Professor @UofT, Vice President of AI Research @nvidia, founding member of @VectorInst. Computer vision, deep learning, 3D. Opinions are my own.
Justin Johnson @jcjohnss
20K Followers 587 Following Cofounder @theworldlabs. Building Spatial Intelligence.
Ehsan Adeli @eadeli
2K Followers 718 Following Researcher & Faculty @Stanford @StanfordMed @StanfordPSY; @StanfordSVL, @StanfordAILab. AI, Computer Vision, Neuroscience, Healthcare, Medical Image Analysis.
Harvard Department of... @HarvardMath
6K Followers 64 Following The Department of Mathematics at Harvard University is one of the world's leading centers for research and education in pure mathematics. #harvardmath
Charlie Snell @sea_snell
8K Followers 6K Following PhD student @berkeley_ai; research @cursor_ai; prev @GoogleDeepMind. My friend told me to tweet more. I stare at my computer a lot and make things
Lenore Blum @BlumLenore
2K Followers 381 Following
Trenton Bricken @TrentonBricken
12K Followers 2K Following Trying to figure out what makes minds and machines go "Beep Bop!" @AnthropicAI
Ada Fang @AdaFang_
4K Followers 162 Following PhD Student @Harvard | AI/ML for Chemical Biology @marinkazitnik lab | currently at @GoogleDeepMind
Shirley Ho @cosmo_shirley
4K Followers 375 Following Group Leader @FlatironCCA @SimonsFdn 🌟 Professor at @NYUPhysics @NYUDataScience @Princeton ML/AI-accelerated Science, @PolymathicAI HongKonger
François Rozet @FrancoisRozet
677 Followers 136 Following datamancer, generative models, bayesian inference, dynamical systems, open-source software, phd with @glouppe
PolymathicAI @PolymathicAI
3K Followers 68 Following The Polymathic AI Collaboration. Shared account.
Sanjana Srivastava @sanjana__z
554 Followers 591 Following AI PhD student @StanfordAILab @StanfordSVL | Previously @DeepMind @MITCSAIL @mitbrainandcog @MIT_CBMM
Aditya Parameswaran @adityagp
5K Followers 710 Following associate prof @ucberkeley, co-director @ucbepic, cofounder @ponderdata (acq. @snowflakeDB) | on a mission to make data science effortless at scale | he/him
Maya Arce @maya_m_arce
333 Followers 30 Following
Dileep George @dileeplearning
12K Followers 1K Following AGI research @DeepMind. Ex cofounder & CTO @vicariousai (acqd by Alphabet) and @Numenta. Triply EE (BTech IIT-Mumbai, MS&PhD Stanford). #AGIComics
Mahi Shafiullah 🏠�... @notmahi
3K Followers 616 Following Trying to understand the emergence of generally intelligent robotic behavior at @berkeley_ai @AIatMeta. Previously @CILVRatNYU @MIT & @Apple AI/ML fellow.
Meng Jiang @Meng_CS
1K Followers 526 Following Frank M. Freimann Collegiate Professor at Notre Dame CSE | Data Mining | NLP | AI
Noah Ziems @NoahZiems
1K Followers 1K Following Visiting Researcher @MIT_CSAIL. PhD student @NotreDame advised by @Meng_CS. Creator of Arbor RL library for @DSPyOSS
Jiantao Jiao @JiantaoJ
2K Followers 115 Following Director of Research & Distinguished Scientist at @NVIDIA, Professor at UC Berkeley EECS and Statistics. Building AGI/ASI
Zhaocheng Zhu @zhu_zhaocheng
3K Followers 393 Following Senior Research Scientist @nvidia. PhD @Mila_Quebec. BSc @PKU1898. Reasoning, LLMs, ML systems. Photographer. Opinions are my own.
François Charton @f_charton
4K Followers 26 Following AI for mathematics and theoretical physics Tomorrow's problems on yesterday's machines Axiom - École nationale des ponts et chaussées
Scott Yih @scottyih
2K Followers 844 Following Research Scientist at Meta Fundamental AI Research (FAIR)
Ziwei Liu @liuziwei7
11K Followers 1K Following Associate Professor @ NTU - Vision, Learning and Graphics.
Yi Ma @YiMaTweets
101K Followers 490 Following Chair Prof. in AI, HKU; Visiting Prof. of EECS, UCB New book on Principles of Intelligence: https://t.co/leZlkURb7j
Rachel Rudinger @rachelrudinger
3K Followers 648 Following Assistant Professor of Computer Science at University of Maryland, College Park. NLP, CompLing, AI. she/her
clem 🤗 @ClementDelangue
155K Followers 5K Following Co-founder & CEO @HuggingFace 🤗, the open and collaborative platform for AI builders
Iain Dunning @iaindunning
6K Followers 456 Following Head of AI @ HRT (@wehrtyou), Chairman @ New York Transit Museum
elvis @omarsar0
263K Followers 670 Following Building with AI agents @dair_ai • Prev: Meta AI, Galactica LLM, Elastic, PaperswithCode, PhD • I share insights on how to build with AI Agents ↓
Aran Nayebi @aran_nayebi
4K Followers 3K Following Asst Prof @CarnegieMellon Machine Learning (@mldcmu @SCSatCMU) | @BWFUND CASI Fellow | Building a Natural Science of Intelligence 🧠🤖 | Prev: @MIT, @Stanford
Kyle🤖🚀🦭 @KyleMorgenstein
16K Followers 5K Following Full of childlike wonder. Teaching robots manners. UT Austin PhD candidate. 🆕 RL Intern @ Apptronik. Past: Boston Dynamics AI Institute, NASA JPL, MIT ‘20.
vicki @vboykis
55K Followers 1K Following lots of lists of floats. Founding ml engineer in recsys/personalization. building ✨I like Nutella.
Georgia Gkioxari @georgiagkioxari
11K Followers 453 Following Assistant professor in Computing + Mathematical Sciences @Caltech 🏛️ ∙ Computer vision enthusiast 🤖 ∙ Part-time at @metaai 👩🏻💻∙ From 🇬🇷
Yonglong Tian @YonglongT
3K Followers 244 Following Research Scientist @OpenAI. Prev RS@GoogleDeepMind, PhD@MIT. Opinions are my own.
Huazhe Harry Xu @HarryXu12
4K Followers 984 Following Hi, I like reinforcement learning, robots, and video games:) I am an amateur pianist. Assistant Prof at Tsinghua; Postdoc at Stanford; Ph.D. at Berkeley
Xingang Pan @XingangP
3K Followers 418 Following Assistant Professor at Nanyang Technological University @NTUsg @MMLabNTU - Computer Vision, Deep Learning, Computer Graphics
Xiaolong Wang @xiaolonw
18K Followers 1K Following Associate Professor @UCSDJacobs Postdoc @berkeley_ai PhD @CMU_Robotics
Elliott / Shangzhe Wu @elliottszwu
8K Followers 878 Following Assistant Professor @Cambridge_Eng, working on 3D computer vision and inverse graphics, previously postdoc @StanfordSVL, PhD @Oxford_VGG
Silvio Savarese @silviocinguetta
11K Followers 45 Following Executive Vice President, Chief Scientist @salesforce. Adjunct Professor of Computer Science @Stanford University. Faculty co-director @StanfordSVL. #AI
Joy Hsu @joycjhsu
3K Followers 302 Following CS PhD-ing @stanford & @knighthennessy. Studying visual reasoning, neuro-symbolic learning, and visual concepts @stanfordailab & @stanfordsvl.
Isabelle Augenstein @IAugenstein
12K Followers 1K Following Full Professor @CopeNLU @uni_copenhagen. Formerly @ucl_nlp, @SheffieldNLP. Explainable AI, Natural Language Processing, ML.Trends for United States
You might like
