Fusheng Liu @mathlfs
PhD student @ National University of Singapore [email protected] mathematicallfs.github.io Singapore Joined September 2022-
Tweets51
-
Followers19
-
Following97
-
Likes530
Fusheng Liu retweeted
🚀 Introducing our new tech report: Muon is Scalable for LLM Training We found that Muon optimizer can be scaled up using the follow techniques: • Adding weight decay • Carefully adjusting the per-parameter update scale ✨ Highlights: • ~2x computational efficiency vs AdamW…
84
307
2K
749K
1K
Download Image
Fusheng Liu retweeted
🚀 Introducing DeepSeek-V3! Biggest leap forward yet: ⚡ 60 tokens/second (3x faster than V2!) 💪 Enhanced capabilities 🛠 API compatibility intact 🌍 Fully open-source models & papers 🐋 1/n
Highly recommend this user-friendly project if you start with LM pretraining and want to build your own model/optimizer. The repo is easy to understand, easy to edit and easy to implement new ideas with minimum workloads. Well done Keller! Looking forward to your records on VIT:)
Highly recommend this user-friendly project if you start with LM pretraining and want to build your own model/optimizer. The repo is easy to understand, easy to edit and easy to implement new ideas with minimum workloads. Well done Keller! Looking forward to your records on VIT:)
Fusheng Liu retweeted
Fixed a bug which caused all training losses to diverge for large gradient accumulation sizes. 1. First reported by @bnjmn_marie, GA is supposed to be mathematically equivalent to full batch training, but losses did not match. 2. We reproed the issue, and further investigation…
Fusheng Liu retweeted
Harm's Law of Smol Models (HLSM) tells us how much we need to scale up the data size (k_D) as we scale down the model size (k_N), if we wish to preserve the loss of a Chinchilla-optimal model. harmdevries.com/post/model-siz…
Fusheng Liu retweeted
Afraid of sum-of-squares (SOS) relaxations? Read this new blog post for a smooth ride in the Fourier domain. francisbach.com/sums-of-square…
Quentin Berthet @qberthet
3K Followers 2K Following Research Scientist at Google DeepMind Machine Learning - Paris
Julien Siems @julien_siems
305 Followers 663 Following PhD student advised by Frank Hutter working on linear RNNs and state-tracking.
Jing Zhang @JingZhang133376
1 Followers 6 Following
𝔻𝕃 @dlb07
197 Followers 7K Following
leloy! @leloykun
7K Followers 4K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Brandon Amos @brandondamos
20K Followers 2K Following research scientist @MetaAI (FAIR) | optimization, machine learning, control, transport | PhD from @SCSatCMU
Du Mingzhe @spirit__song
25 Followers 138 Following PhD Student in CS @NTUsg / @NUSingapore / @UniMelb
Yingnan Liu @yninthefuture
11 Followers 49 Following Machine learning PhD student from National University of Singapore. Work in Out-of-Distribution generalization and AI for healthcare.
Zhiyuan @ZhiyuanCS
397 Followers 203 Following PhD student in @NUSingapore Visiting Researcher in @MIT
Taco Cohen @TacoCohen
27K Followers 3K Following Post-trainologer at FAIR. Into codegen, RL, equivariance, generative models. Spent time at Qualcomm, Scyfer (acquired), UvA, Deepmind, OpenAI.
Alan Jeffares @ ICML ... @Jeffaresalan
2K Followers 735 Following Multiplying matrices @Cambridge_Uni & @MSFTResearch | PhD student in Machine Learning | Previously MSc @ucl & BSc @ucddublin
Claudio Gallicchio @claudiogallicc1
783 Followers 903 Following Associate Professor of ML at the University of Pisa (Italy). Deep Randomized Neural Networks, Reservoir Computing, Stable Architectures, Deep Learning 4 Graphs
Tan Minh Nguyen @TanNguyen689
1K Followers 740 Following Assistant Professor of Mathematics (Presidential Young Professor) at the National University of Singapore (@NUSingapore). #DeepLearning, #RobustAI, #ScalableAI
Bingchen Wang @BingchenWang1
7 Followers 55 Following Oxford MPhil Econ 21' | Columbia Math-Stats 19' | CityUHK Computing Math 19'
Ayaka @ayaka14732
2K Followers 311 Following Ex-JAX maintainer @Google, LLM, low-resource languages | Japanese account @ayaka4f
Rough Researcher @SapkotaTsuman
123 Followers 246 Following Rough AI blogger. Writing poems with code. Neural Networks. Currently on Interpretability, Sparsity, Continual Learning, Programmable NNs.
Thang Luong @lmthang
27K Followers 95 Following Lead Superhuman Reasoning team @GoogleDeepMind. AI IMO Gold. Co-led #DeepThink, #AlphaGeometry, #Bard (now Gemini) Multimodality, #MeenaBot. LuongAttention.
Fuzhao Xue (Frio) @XueFz
5K Followers 682 Following Research Scientist @GoogleDeepmind | Scaling & Distillation #Gemini | Zero-shot Cooking Learner🧑🍳
Shai Shalev-Shwartz @shai_s_shwartz
3K Followers 174 Following Computer Science @HebrewU; CTO @Mobileye #selfdrivingcars #ai #machinelearning Opinions are my own.
Shengjia Zhao @shengjia_zhao
52K Followers 231 Following Chief Scientist @ Meta MSL. Formerly MTS @ OpenAI, PhD @ Stanford. I train models. All opinions my own.
Nino Scherrer @ninoscherrer
994 Followers 2K Following Research Scientist at @Google | Rigorous evaluations, cognitive science & causality | Ex: {@PatronusAI, @VectorInst, @Mila_Quebec, @MPI_IS, @ETH_en}
Johannes Oswald @oswaldjoh
1K Followers 641 Following Research Scientist, Paradigms of Intelligence Team, Google Zurich
Tairan He @TairanHe99
5K Followers 768 Following Robotics&AI PhD Student @CMU_Robotics Research Intern at @NVIDIA Prev: @MSFTResearch @sjtu1896 Emboddied AI; Humanoid; Robot Learning
Quentin Berthet @qberthet
3K Followers 2K Following Research Scientist at Google DeepMind Machine Learning - Paris
Riccardo Grazzi @riccardograzzi
107 Followers 131 Following Researcher at MSR Cambridge, previously at IIT in Genoa, Italy. Working on principled, efficient optimization for machine learning and on LLMs' expressivity.
Julien Siems @julien_siems
305 Followers 663 Following PhD student advised by Frank Hutter working on linear RNNs and state-tracking.
Nazneen Rajani @nazneenrajani
5K Followers 2K Following building @collinearAI 🧪 | MIT 35u35 | UN AI Advisory Body | Featured in NYT, Quanta, Science, MIT TR| Previously: @huggingface 🤗, @SFResearch, PhD @utcompsci
jianlin.su @Jianlin_S
3K Followers 14 Following Grad is all you need @Kimi_Moonshot Blog: https://t.co/YVxsWylklA , Cool Papers: https://t.co/scS1n1oyaO
Nancy Pelosi Stock Tr... @PelosiTracker_
1.2M Followers 522 Following Highlighting Politicians' trades so we can invest alongside Goal: get them banned from trading. $800,000,000 invested on @joinautopilot_ so far
Noam Razin @noamrazin
618 Followers 325 Following Postdoctoral Fellow at @PrincetonPLI | Past: Computer Science PhD @TelAvivUni & Apple Scholar in AI/ML | Interested in the foundations of deep learning
Jing Zhang @JingZhang133376
1 Followers 6 Following
DeepSeek @deepseek_ai
972K Followers 0 Following Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism.
Jimmy Smith @jimmysmith1919
474 Followers 290 Following Founding Scientist @LiquidAI_. Previously PhD @Stanford and member of @scott_linderman's lab.
Scott Linderman @scott_linderman
5K Followers 822 Following Assistant Professor @Stanford Statistics and @StanfordBrain. Computational Neuroscience, Machine Learning, Bayesian Statistics. Tweets are my own.
leloy! @leloykun
7K Followers 4K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Brandon Amos @brandondamos
20K Followers 2K Following research scientist @MetaAI (FAIR) | optimization, machine learning, control, transport | PhD from @SCSatCMU
Du Mingzhe @spirit__song
25 Followers 138 Following PhD Student in CS @NTUsg / @NUSingapore / @UniMelb
Yingnan Liu @yninthefuture
11 Followers 49 Following Machine learning PhD student from National University of Singapore. Work in Out-of-Distribution generalization and AI for healthcare.
Institute of Data Sci... @NUS_IDS
16 Followers 33 Following 🌐 https://t.co/sEoYviavrz ✍ https://t.co/ttD0XrMTj9
Zhiyuan @ZhiyuanCS
397 Followers 203 Following PhD student in @NUSingapore Visiting Researcher in @MIT
Taco Cohen @TacoCohen
27K Followers 3K Following Post-trainologer at FAIR. Into codegen, RL, equivariance, generative models. Spent time at Qualcomm, Scyfer (acquired), UvA, Deepmind, OpenAI.
Simons Institute for ... @SimonsInstitute
9K Followers 311 Following The world's leading venue for collaborative research in theoretical computer science. Follow us at https://t.co/KvcuGI7WM0.
Alexander Wei @alexwei_
24K Followers 193 Following Reasoning @OpenAI. Co-built CICERO @MetaAI | @Berkeley_AI PhD '23 | @Harvard '20
Daniela Massiceti @dannimassi
2K Followers 406 Following Machine learning researcher at Microsoft Research | based in Sydney | PhD @UniofOxford | South African 🇿🇦 | @DeepIndaba 🐘 | maker of things
Alan Jeffares @ ICML ... @Jeffaresalan
2K Followers 735 Following Multiplying matrices @Cambridge_Uni & @MSFTResearch | PhD student in Machine Learning | Previously MSc @ucl & BSc @ucddublin
Andrej Karpathy @karpathy
1.4M Followers 1K Following Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets.
William Merrill @lambdaviking
5K Followers 667 Following Incoming Assistant Prof, Toyota Technical Institute at Chicago @TTIC_Connect Recruiting PhD students (start 2026) 👀 Will irl - TC0 enthusiast
Zhiyuan Li @zhiyuanli_
2K Followers 317 Following Assistant Professor @TTIC_Connect. Previously Postdoc @Stanford and PhD @PrincetonCS. Deep Learning Theory.
Sepp Hochreiter @HochreiterSepp
14K Followers 376 Following Pioneer of Deep Learning and known for vanishing gradient and the LSTM.
Durk Kingma @dpkingma
50K Followers 404 Following @AnthropicAI. Prev. @Google Brain/DeepMind, founding team @OpenAI. Computer scientist; inventor of the VAE, Adam optimizer, and other methods. ML PhD.Trends for United States
21,2 B posts
5.145 posts
2.316 posts
1.789 posts
42,9 B posts
21,7 B posts
28,5 B posts
19,8 B posts
7.989 posts
80,2 B posts
53,4 B posts
52,3 B posts
16,5 B posts
243 B posts
44,6 B posts
1.328 posts
14 B posts
48,7 B posts
You might like
Elon Musk
@elonmusk
225.8M Followers
Barack Obama
@BarackObama
130.7M Followers
Cristiano Ronaldo
@Cristiano
115.8M Followers
Donald J. Trump
@realDonaldTrump
109.1M Followers
Justin Bieber
@justinbieber
108.9M Followers
Rihanna
@rihanna
107.6M Followers
Taylor Swift
@taylorswift13
93.7M Followers
NASA
@NASA
87.9M Followers
Lady Gaga
@ladygaga
81.7M Followers
YouTube
@YouTube
79.0M Followers
Kim Kardashian
@KimKardashian
74.4M Followers
The Ellen Show
@EllenDeGeneres
73.2M Followers
X
@X
68.9M Followers
Virat Kohli
@imVkohli
67.9M Followers
Bill Gates
@BillGates
66.5M Followers
Selena Gomez
@selenagomez
64.9M Followers
Neymar Jr
@neymarjr
63.9M Followers
CNN Breaking News
@cnnbrk
63.9M Followers
CNN
@CNN
63.7M Followers
Justin Timberlake
@jtimberlake
59.1M Followers