Marc Sun @_marcsun
Machine Learning Engineer @huggingface Open Source team New york Joined February 2023-
Tweets629
-
Followers2K
-
Following450
-
Likes3K
i was reading the torchao paper today and came across a pleasant surprise 😳 contributions were super minimal but kinda cool that the folks over at torchao mentioned me 🤗 defo wanted to contribute more impactfully but didn't know how to write triton nor fully understood…
New in-depth blog post - "Inside vLLM: Anatomy of a High-Throughput LLM Inference System". Probably the most in depth explanation of how LLM inference engines and vLLM in particular work! Took me a while to get this level of understanding of the codebase and then to write up…
Day 14 of 14 Days of Distributed! We've got a number of cool people still that are talking since we started this list, so today we're going to rapid fire them all (in no particular order)! Let's buckle up and go! @winglian @FerdinandMom @m_sirovatka @mervenoyann @charles_irl
pip install -U mlx
Sometimes you just need to enable a few existing options to get SotA when fine-tuning for long context lengths.
Sometimes you just need to enable a few existing options to get SotA when fine-tuning for long context lengths.
GPT-OSS bug fixes + Flex Attention support is here! 1. Fixed float16 infinite losses (>65504 overflows) 2. SWA=128 Flex default uses 129 tokens (extra 1) 3. Fixed MXFP4 inference swiglu_limit=7.0 not set 4. Sink token moved to index 0 5. FA3 doesn't have attn sink dX Details:…
GPT-OSS bug fixes + Flex Attention support is here! 1. Fixed float16 infinite losses (>65504 overflows) 2. SWA=128 Flex default uses 129 tokens (extra 1) 3. Fixed MXFP4 inference swiglu_limit=7.0 not set 4. Sink token moved to index 0 5. FA3 doesn't have attn sink dX Details:… https://t.co/rUbvvjGW7W
Happy to participate in the online course by my mentor @TheZachMueller ! The topic of my talk will be efficient distributed inference
Happy to participate in the online course by my mentor @TheZachMueller ! The topic of my talk will be efficient distributed inference
🚀 LLM Compressor v0.7.0 is here! This release brings powerful new features for quantizing large language models, including transform support (QuIP, SpinQuant), mixed precision compression, improved MoE handling with Llama4 support, and more. Full blog: developers.redhat.com/articles/2025/……
Context parallelism in 🤗 transformers Trainer? Training models on 100k+ sequence length has never been easier 🚀
Wrote a blogpost. Hopefully it's the first of many to come. Feedback is welcome 🤗 gau-nernst.github.io/fa-5090/
It's time again for our last (now yearly) celebration extravaganza of the year. GPU MODE is meeting IRL again in downtown San Francisco on Friday October 24 from 10am to 10pm to hack all day
We just published an example-led blog on quantization, showing how it reduces memory use and speeds up inference with minimal accuracy loss. Example: moving from FP16 → FP8 halves VRAM, freeing space for larger KV caches and parallel workloads. A thread (1/9):
WE ARE SO BACK!!! huggingface.co/deepseek-ai/De…
Transformers v4.55.1 was just released, with the mission to make it dead simple to run OpenAI gpt-oss on consumer GPUs – including 3090, Colab, Kaggle, or HF Spaces. Just uv run this 👇
You can now fine-tune OpenAI gpt-oss for free with our notebook! Unsloth trains 1.5x faster with -70% VRAM, 10x longer context & no accuracy loss. 20b fits in 14GB & 120b in 65GB GPU. Guide: docs.unsloth.ai/basics/gpt-oss GitHub: github.com/unslothai/unsl… Colab: colab.research.google.com/github/unsloth…
BOOOOM! You can now run @OpenAI gpt-oss 20B natively in @GoogleColab T4 for FREE! 🔥 Powered by Transformers ⚡ The setup takes a bit since everything is bleeding edge, but once done it should work as expected Link to our cookbook in comments 👇
This is by far my favorite feature so far this year. Unlocks a lot of exciting possibilities for post-training larger models in the @huggingface ecosystem such as improved multi-node and slurm support.
This is by far my favorite feature so far this year. Unlocks a lot of exciting possibilities for post-training larger models in the @huggingface ecosystem such as improved multi-node and slurm support.
We’ve made N-D parallelism training simpler in accelerate ! It was a pleasure to collaborate with @axolotl_ai on this feature.
We’ve made N-D parallelism training simpler in accelerate ! It was a pleasure to collaborate with @axolotl_ai on this feature.
It seems the closed-source vs open-weights landscape has been leveled. GPT-5 is just 10% better at coding than an open-weight model you can run on a consumer desktop and soon laptop. If Anthropic cannot come up with a good model, then we will probably not see AGI for a while.
Rewrote gpt-oss's experts forward pass (bfloat16 weights) in transformers to only use active experts + simple gemv MoE Triton kernel: 36 tokens/sec -> 165 tokens/sec (20B on the RTX6000 Pro) MXFP4 weights next, decoding should run faster 👀
Tarek Masryo @TarekMasryo
1 Followers 37 Following ML Engineer @ MMC Global | Data Science, Machine Learning, Deep Learning | Generative AI | Python & SQL
Aman Swar @AmanSwar_
2 Followers 141 Following MLSys. Hacking on CUDA kernels, compilers,and LLM infra. Pushing performance
Sourik @Sourik24
256 Followers 2K Following Making GPUs and CPUs go Brrrrr @ https://t.co/CXXbtt3IPU , GPU tinkerer, Compiler Fanatic, Code Slinger, Harry Potter and Star Trek Nerd, Full-Time LEGO Connoisseur
Sam Foreman @saforem2
2K Followers 5K Following 🏡 https://t.co/oi6qzoIAB8 | making rocks think @argonne / @argonne_lcf | Physics PhD | https://t.co/P3VfVDmZRX | he/him
daryl martis @realdarylmartis
337 Followers 2K Following
myron koch @myronkoch
572 Followers 2K Following Saxophone | Technology | Film | Blockchain & AI Research
Kazuki Fujii @okoge_kaz
3K Followers 2K Following Tokyo Tech CS Master (Rio Yokota Lab → Jun Sakuma Lab) Swallow LLM Project: Distributed Training, Sytems for Machine Learning, Low Precision Training
ELONMUSKTESLA @elonneurlink
32 Followers 797 Following Live life to the fullest, keep things simple, truthful & filter the noise. I am a long term investor MAGA🚀🇺🇸
Jannik @JannikHWX
15 Followers 1K Following
ye dongxi @YDongxi
45 Followers 1K Following A data set, data annotation sales, selling high-quality annotation solutions similar to AI for science/autonomous driving/lean4 data topics。
Stewart Caitlin @StewartCaitlin3
24 Followers 337 Following
Sai Vignan @vignan_sai
104 Followers 2K Following ML Engineering @Microsoft, prev ML @sprinklr, CS @iitdelhi, Interested in ML, Bio Informatics
sandya mannarswamy @sandyasm
1K Followers 7K Following Natural Language Processing Researcher. https://t.co/oYoCTKS2Ho
Rahul ✨ @Geek4PM
59 Followers 763 Following Helping fashion studios & brands replace $400/SKU photoshoots with custom, consistent, and scalable AI-generated visuals. Book a call with 8M Studio
Kashish Jagga @kashish_jagga
23 Followers 3K Following
Hamid Soorghali @soorghali
1K Followers 5K Following Connecting space-based infrastructure and services to the downstream industries and markets | Strategy @SatAppsCatapult | @SOAS & @InterpolAber alumnus
Wen-Ding Li @xu3kev
3K Followers 6K Following LLM for code and reasoning. PhD student at Cornell. Previously Student Researcher at @google. Previously intern at @theteamatx.
Vivian🐧 – e/acc @vynride
122 Followers 529 Following 19 • I build stuff • hackathons • cs ug • learning ml • oss https://t.co/RYniU5JfTC • arch linux fanboy🐧• (will code for pizza 🍕)
Nina @Majorg09355
22 Followers 1K Following
Ilias Miraoui @iliasmiraoui
691 Followers 1K Following Hacking with LLMs⚒️ https://t.co/Q9bogacIgC https://t.co/CTcyYe1bmv
Christian Lim @christian_lim_
158 Followers 1K Following VP of Engineering, Arklex AI (@ArklexAI) | Adjunct, Columbia (@Columbia) | Director of Internships, ICPC Foundations (@icpcnews) | Stanford (BS ‘11 MS ‘13)
PinkGigi @Daqrui17777
14 Followers 1K Following "Life is unpredictable, but good medicine and a compassionate heart never fail."
Utaiwi @Utaiwi36604
93 Followers 1K Following
Daniel Bis @danielbis01
105 Followers 759 Following LLMs at Amazon AI | prev Samsung, RMS | Opinions expressed are my own
Sergio Soage @Sergio_Soage
878 Followers 6K Following artificial intelligence, math. Random stuff @ https://t.co/tqV9OIPsWE
AlexandeR Aguilar @04l3x4nd3r0
134 Followers 2K Following
Dylan Beadle @Dylan_Beadle
1K Followers 2K Following
Oldman Jin Tiger @JinOldman
0 Followers 94 Following
ゆーき @yukiharada1228
576 Followers 1K Following
Kayli Weimann @kayli61646
24 Followers 2K Following
Nauijep @Nauijep3677582
20 Followers 2K Following
DayTradeAlerts🇺�... @Ubalgeef196526
30 Followers 2K Following 15-30% Monthly | 2 High-Conviction Stocks.Short-Term Gains: 15-20% in Days/Weeks.DM "JOIN" for WhatsApp Alerts. Live Trade Signals • Market Analysis
Lereehfou @Lereehfou84930
21 Followers 1K Following
Léo Aparisi de Lanno... @LeoAparisidL
1K Followers 4K Following PhD student @UChi_Economics. Economics/History/Politics/Whatever.
Ehtisham Sadiq @EhtishamSadiq10
78 Followers 1K Following Machine Learning Engineer | Tech Explorer | Data Whisperer | Code Creator | Crafting Tomorrow's Digital World
chelley @chelley1941117
34 Followers 277 Following
Irwan Bello @IrwanBello
7K Followers 3K Following Supercomputers & Friends AGI research & products founding team @reflection_ai ex @OpenAI, founding team @character_ai
Christopher De Sa @chrismdesa
496 Followers 23 Following
Bert Maher @tensorbert
3K Followers 342 Following I’m a software engineer building high-performance kernels and compilers at Anthropic! Previously at Facebook/Meta (PyTorch, HHVM, ReDex)
Romain Huet @romainhuet
33K Followers 8K Following Head of Developer Experience @OpenAI. Empowering builders with GPT-5, Codex, gpt-oss, and more. Previously, Product Lead @Stripe.
Stuart Sul @stuart_sul
1K Followers 122 Following ml research @cursor_ai, cs @Stanford, mlsys @HazyResearch
Zeyuan Allen-Zhu, Sc.... @ZeyuanAllenZhu
20K Followers 452 Following physics of language models @ Meta (FAIR, not GenAI) 🎓：Tsinghua Physics — MIT CSAIL — Princeton/IAS 🏅：IOI x 2 — ACM-ICPC — USACO — Codejam — math MCM
Eric Hartford @QuixiAI
17K Followers 574 Following We make AI models Dolphin and Samantha BTC 3ENBV6zdwyqieAXzZP2i3EjeZtVwEmAuo4 https://t.co/3ri2GbWU13 https://t.co/zH0F3pSLuq @dphnAI
Wanchao Liang @wanchao_
1K Followers 225 Following building @thinkymachines ex-PyTorch @ Meta. Author of PyTorch DTensor and TorchTitan. Opinions are my own
Rémi Ouazan @gpus_go_brrr
53 Followers 14 Following Crafting cutting-edge GPU kernels at Hugging Face 🤗
Dan Saunders @djsaunde
493 Followers 2K Following ML eng @axolotl_ai making open source llm training tools. miniature Aussie haver, sheep wrangler, remote worker, extremely quacked
Cheng @zcbenz
3K Followers 90 Following creator of @electronjs, check https://t.co/ZDJujd4Nql for the open source things I built. currently sponsored to write a CUDA backend for MLX.
Federico Cassano @ellev3n11
2K Followers 235 Following training @cursor_ai prev @neu_prl, @scale_AI, @Roblox, @trailofbits
Christian Szegedy @ChrSzegedy
41K Followers 3K Following #deeplearning, #ai research scientist. Opinions are mine.
Harry Mellor @hmellor_
170 Followers 32 Following ML Engineer @huggingface maintaining @vllm_project, prev @graphcoreai, @uniofoxford
LMSYS Org @lmsysorg
8K Followers 167 Following Large Model Systems Organization: Join our Slack: https://t.co/mSPNyKTLTS We developed SGLang https://t.co/jEqIJcGwGA, Chatbot Arena (now @lmarena_ai), and Vicuna!
Vasiliy Kuznetsov @vkuzo
39 Followers 27 Following
turboderp @turboderp_
766 Followers 36 Following
Vijay @__tensorcore__
2K Followers 515 Following MLIR, CUTLASS,Tensor Core arch @NVIDIA. Mechanic @hpcgarage. Exercise of any 1st amendment rights are for none other than myself.
Prime Intellect @PrimeIntellect
45K Followers 26 Following find compute. train models. contribute to open superintelligence. https://t.co/ZRZOsRRbwr
joe @official_j3rck
249 Followers 203 Following training things @pytorch @aiatmeta | previously language research at @usc_isi
fal @FAL
32K Followers 5 Following the generative media cloud. hiring https://t.co/JrbUk989MN. for support/discounts, e-mail us at [email protected].
Alex Zhang @a1zhang
13K Followers 587 Following phd student @MIT_CSAIL + @SakanaAILabs, ugrad @Princeton, 🫵🏻 go participate in the @GPU_MODE kernel competitions!
Jeff Rasley @jeffra45
877 Followers 1K Following @Snowflake AI Research Team. @DeepSpeedAI co-founder, @BrownCSDept PhD, @uwcse alum
Hao AI Lab @haoailab
4K Followers 345 Following Hao AI Lab at UCSD. Our mission is to democratize large machine learning models, algorithms, and their underlying systems.
kalomaze @kalomaze
18K Followers 2K Following ML researcher (@primeintellect), speculator • extremely silly jester
You Jiacheng @YouJiacheng
8K Followers 2K Following a big fan of TileLang 关注TileLang喵！关注TileLang谢谢喵！ https://t.co/utshC0jrCO 十年老粉
ℏεsam @Hesamation
36K Followers 575 Following ai engineer | rigorously overfitting on a learning curve
Sergio Paniego @SergioPaniego
3K Followers 2K Following Machine Learning Engineer @huggingface 🤗 AI PhD. Technology enables us to be more human. 🏳️🌈
Taishi Nakamura @Setuna7777_2
2K Followers 6K Following CS MS at @sciencetokyo_en Intern @SakanaAILabs
Mira Murati @miramurati
365K Followers 573 Following Now building @thinkymachines. Previously CTO @OpenAI
zhyncs @zhyncs42
3K Followers 519 Following 🌁 OPINIONS ARE MY OWN, Homepage https://t.co/saCowtppUm, Just for fun @lmsysorg SGLang, Prev @basetenco @meituan @Baidu_Inc
Eldar Kurtić @_EldarKurtic
729 Followers 619 Following Principal Research Scientist @RedHat_AI & @ISTAustria
Gabriel Martín Bláz... @gabrielmbmb_
688 Followers 655 Following Founding Engineer @SupersonikAI | ex-Hugging Face 🤗
Alina Lozovskaya @ailozovskaya
717 Followers 294 Following ML Engineer at @huggingface 🤗 | linguist by eduction, engineer by profession | take photos and play music
Alpay Ariyak @AlpayAriyak
3K Followers 3K Following Post-Training Lead @ Together AI | OpenChat Project Lead (#1 7B LLM on Arena for 2+ months, 2M+ downloads) | DeepCoder, DeepSWE
m_ric @AymericRoucher
6K Followers 244 Following ✨ Stella Duce ⚙️ Agents lead @huggingface ✍️ Writing a book on the coming Ultra-intelligence
Trends for United States
You might like
