Matthew Douglas @mattkdouglas
ML Engineer 🤗 @huggingface | Engineering Lead @ bitsandbytes | Opinions are my own. Indianapolis, IN Joined May 2011-
Tweets315
-
Followers319
-
Following507
-
Likes421
You! Do you like CUDA or Triton? Do you have free time this week? Would you like your work to bring happiness to thousands of people? The open-source community needs you to write a backward kernel for MXFP4!
Update: `mxfp4` is now supported in transformers for Turing, Ampere, and Ada GPUs. Which means you can run openai/gpt-oss-20b on Google Colab 🥳(and other consumer GPUs!) Link to notebook in comments.
We recently released PEFT v0.17.0 🔥 This release enables LoRA to be applied to Mixture of Expert (MoE) layers, as for example found in the new open gpt-oss model by #OpenAI. More information in the 🧵
The new GPT-OSS models are Mixture of Experts (MoEs), with 20B and 120B parameters. Since expert weights make up ~90% of the model, OpenAI decided to quantize them to 4 bits during post-training using the MXFP4 standard. Quantizing these to MXFP4 enables the larger model to…
Thank you @Google for the ML and Systems Junior Faculty Award! This award is for work on sparsity, and I am excited to continue this work focusing on mixture of experts. We might bring big MoEs to small GPUs quite soon! Stay tuned! Read more here: cs.cmu.edu/news/2025/dett…
Super excited to share SmolLM3, a new strong 3B model. SmolLM3 is fully open, we share the recipe, the dataset, the training codebase and much more! > Train on 11T token on 384 H100 for 220k GPU hours > Support long context up to 128k thanks to NoPE and intra document masking >…
"The great unbloating" of transformers continues. Over the past few weeks, 10+ PRs were merged, aiming to simplify code across the library. This brought in refactors for Attention, the Cache, a new linter. We're improving type hints everywhere, and are checking type checkers.
I have bittersweet news to share. Yesterday we merged a PR deprecating TensorFlow and Flax support in transformers. Going forward, we're focusing all our efforts on PyTorch to remove a lot of the bloating in the transformers library. Expect a simpler toolkit, across the board.
Ever wonder how Hugging Face moves so fast? Here’s one reason: 0 hours of meetings on my June calendar. An entire month to build, think, and ship. And it's not just me, it's our culture.
Bitsandbytes latest works with `torch.compile(fullgraph=True)` and you should put it to good use 🔥 For example, when applied to Flux, it beefs up the performance quite a bit. Code: gist.github.com/sayakpaul/0db9… Enjoy 🔥
New Blog Post! 🚀 Explore how quantization backends in Diffusers make large diffusion models like Flux run with less VRAM without sacrificing (much) quality! Run Flux with bitsandbytes-4bit using under 18 GB VRAM and in just 15 seconds! blogpost: huggingface.co/blog/diffusers… Can you…
We've just revamped the @huggingface Quantization docs! 🥳 Understand concepts better & choose the right technique for your needs with these key updates: - Explanations of quantization fundamentals (schemes, int4, FP8). huggingface.co/docs/transform… New Selection Guide: Choose the…
The fused architecture of Llama 4 isn’t ideal for quantization. In 🤗 Transformers, we swapped the fused experts with a list of MLPs to make inference with quantized models possible. You can now find 8-bit and 4-bit quantized models using bitsandbytes in the bnb-community ! 🧠⚡…
Check out the latest release of 🤗 Accelerate! This update is packed with exciting features, including the much-anticipated FSDPv2 integration. Discover why FSDPv2 is a game-changer in the thread below. 🧵
Check out the latest release of 🤗 Accelerate! This update is packed with exciting features, including the much-anticipated FSDPv2 integration. Discover why FSDPv2 is a game-changer in the thread below. 🧵
‼️Sentence Transformers v4.0 is out! You can now train and finetune reranker (aka cross-encoder) models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also prove that finetuning on your domain helps much more than you might think. Details in 🧵
We teamed up with @huggingface to release a free notebook for fine-tuning Gemma 3 with GRPO! Learn to: • Enable reasoning in Gemma 3 (1B) • Prepare/understand reward functions • Make GRPO work for tiny LLMs Notebook: colab.research.google.com/github/unsloth… Details: huggingface.co/reasoning-cour…
🚀 Big news! We’ve launched a Hugging Face Space to make bitsandbytes quantization easier than ever! 🎉 🔹 No complex setups 🔹 No headaches 🔹 Just plug, quantize & go! With bnb, LLMs that once needed high-end GPUs can now run on consumer hardware—making AI more accessible!…
We're collaborating with open source maintainers to make sure Gemma models are well supported in day-0 when a new model is out! For Gemma 3: ✅HF transformers/bitsandbytes ✅Ollama ✅VLLM ✅llama.cpp ✅Unsloth ✅MLX ✅TGI ✅ONNX with transformers.js Who are we missing?
Exciting times for open-source!
📚 Exploring CUTLASS / CuTe: A Learning Journey 📚 While exploring DeepSeek’s open-source work, I noticed they rely heavily on CUDA kernels implemented with CUTLASS for matrix operations. However, there aren’t many structured resources to learn CUTLASS & CuTe. CUTLASS (CUDA…
Besher @mr_besher
34 Followers 526 Following
hamed @thehamedmp
1K Followers 2K Following building, exploring ai safety in my free time | ex- (@posthog 🦔 @Cursorlens)
Majid Dadashi @MajidDadashi
2 Followers 54 Following
Artificially Intellig... @ArtiIntelligent
240 Followers 2K Following Insanity is doing the same thing over and over and expecting different results...
Kunal Suri @suri_kunal_ml
54 Followers 828 Following Solo dev. Learning LLMs and how to function on 3 hours of sleep. Posting proof of life (and progress) daily.
Pedro Cuenca @pcuenq
6K Followers 928 Following ML Engineer at 🤗 Hugging Face | Co-founder at LateNiteSoft (Camera+). I love AI and photography.
Matt McClean @matthewmcclean
548 Followers 4K Following Kiwi bloke. Into Cloud Computing, DevOps and Agile/Lean for a living and Philosophy for the soul. Solutions Architect at Amazon Web Services
priya joseph @ayirpelle
5K Followers 8K Following geek, entrepreneur, 'I strictly color outside the lines!', opinions r my own indeed. @ayirpelle , universal handle at this time
Charchit Sharma @charchits7
98 Followers 881 Following Hey, thanks for stopping by. I am interested in Computer Vision..and multi-modal learning. Please have a look at my profile: https://t.co/E6bhdAOwjk
Spiral Dalat @spiraldalat
17 Followers 599 Following
Kadir Nar @kadirnardev
697 Followers 687 Following AI Research Engineer 🤖 Building Omni & TTS Models 👨🍳 at Vyvo
Antonio J. Dominguez @antferdom
516 Followers 2K Following Head of AI @DataCrunch_io. Ph.D in Large Language Models & Efficient AI @unisevilla. Inference. Programming language theory.
The()Graduate @firexgamerx
497 Followers 3K Following PhD in cancer research. Still studying cancer. Half the time, I tweet on my investment interests (synbio and biotech).
KittyMore @RniAPy6r06eoug
128 Followers 4K Following
Prabhjit Dhillon @PrabhjitDh45118
4 Followers 64 Following Currently building a supply chain management system to reduce production waste. Also an avid open source contributor and blockchain enthusiast.
Dan Saunders @djsaunde
494 Followers 2K Following ML eng @axolotl_ai making open source llm training tools. miniature Aussie haver, sheep wrangler, remote worker, extremely quacked
L @CodeTitanium
101 Followers 5K Following
Nam Cao @simoncao1207
7 Followers 144 Following
Tim Hickle @timhickle
2K Followers 2K Following Fractional CMO on a mission to become the best AI-enabled marketing leader in the world. Let's build cool shit. Pacers/Colts/IUBB/IUFB
Nicolasa Alia @AliaNicola14467
7 Followers 645 Following
Karl Weinmeister @kweinmeister
2K Followers 4K Following Cloud Engineering @ Google. AI/ML/Data, Blue Devil & Longhorn, wanna-be at home improvement. Opinions are my own.
Raymond @P7NBKbhHXpbj0
66 Followers 7K Following
Ada Owen @OwenAda19881
194 Followers 2K Following
Dorian Quelle @dorian_quelle
277 Followers 601 Following PhD Candidate Quantitative Network Science @ UZH, Research Associate @ Oxford Internet Institute, Data Scientist @ https://t.co/4c9KfrSAfB
Hina Dixit @hinadixit
3K Followers 758 Following Founder & CEO @DecomputeAI , Ex - Investment Partner @Microsoft, SWE AI Leader @Apple, VC @SamsungNext, AI @stanford
CCLD @ccldarjun
7 Followers 267 Following
Barry @Barry3578072
65 Followers 2K Following
Nikos Antoniou @nikosanto0
21 Followers 799 Following
Michael M. Pieler @MichaelMPieler
373 Followers 2K Following
Pedro Cuenca @pcuenq
6K Followers 928 Following ML Engineer at 🤗 Hugging Face | Co-founder at LateNiteSoft (Camera+). I love AI and photography.
Kimi.ai @Kimi_Moonshot
50K Followers 98 Following Built by Moonshot AI to empower everyone to be superhuman.
Harry Mellor @hmellor_
170 Followers 32 Following ML Engineer @huggingface maintaining @vllm_project, prev @graphcoreai, @uniofoxford
Michael Goin @mgoin_
1K Followers 376 Following inference optimization @RedHat_AI building @vllm_project | you can call me misha
Eldar Kurtić @_EldarKurtic
726 Followers 619 Following Principal Research Scientist @RedHat_AI & @ISTAustria
Hamel Husain @HamelHusain
38K Followers 2K Following Evals evals evals https://t.co/Zrmp6LRd9c About Me: https://t.co/P6WyeKkyTa
DailyPapers @HuggingPapers
5K Followers 3 Following Tweeting interesting papers submitted at https://t.co/rXX8x0HzXV. Submit your own at https://t.co/QhbJKXBd4Q, and link models/datasets/demos to it!
Hot Aisle @HotAisle
3K Followers 2K Following Dell + AMD AI developer cloud. We welcome all developers to build your next generation AI utility on our secure hardware. SOC2 / HIPAA
Soumith Chintala @soumithchintala
250K Followers 1K Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Aleksa Gordić (水�... @gordic_aleksa
25K Followers 232 Following getting us to singularity with friends x @GoogleDeepMind @Microsoft tensor core maximalist
Simon Willison @simonw
115K Followers 6K Following Creator @datasetteproj, co-creator Django. PSF board. Hangs out with @natbat. He/Him. Mastodon: https://t.co/t0MrmnJW0K Bsky: https://t.co/OnWIyhX4CH
Ostris @ostrisai
9K Followers 285 Following AI / ML researcher and developer. Creator of AI Toolkit - https://t.co/Thqof0Gxpj Support my work - https://t.co/Isg2EXrP7s
AI at AMD @AIatAMD
47K Followers 106 Following Advancing AI innovation together. Built with devs, for devs. Supported through an open ecosystem. Powered by AMD. #TogetherWeAdvance
Dan Saunders @djsaunde
494 Followers 2K Following ML eng @axolotl_ai making open source llm training tools. miniature Aussie haver, sheep wrangler, remote worker, extremely quacked
Axolotl AI @axolotl_ai
2K Followers 56 Following Axolotl is the premier open source LLM fine tuning framework. find us on discord https://t.co/wlcE2wlJa9 or email us at [email protected]
Python Software Found... @ThePSF
686K Followers 127 Following The nonprofit organization behind the Python programming language. For help with Python code: https://t.co/XDHPttz2Xv On Mastodon: @[email protected]
Vikram @msharmavikram
2K Followers 588 Following @NVIDIA Sr. Research Scientist | UIUC PhD All opinions and tweets are personal. Tweets about AI Inference, CUDA and GPU systems.
cohere @cohere
108K Followers 4 Following Cohere builds secure, scalable, and private enterprise-grade AI solutions for real-world business problems. Join us: https://t.co/Yb2xItMObl
👋 Jan @jandotai
11K Followers 975 Following Jan is an open source ChatGPT-alternative that runs 100% offline. Built by @menloresearch. Community: https://t.co/gXXor3poY5
vLLM @vllm_project
17K Followers 20 Following A high-throughput and memory-efficient inference and serving engine for LLMs. Join https://t.co/lxJ0SfX5pJ to discuss together with the community!
Mira Murati @miramurati
365K Followers 572 Following Now building @thinkymachines. Previously CTO @OpenAI
Sara Hooker @sarahookr
49K Followers 9K Following I lead @Cohere_Labs. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, ML reliability. Changing spaces where breakthroughs happen.
Ben Clavié @bclavie
6K Followers 1K Following regressing linearly on a daily basis. wife guy who does retrieval. research @mixedbreadai, prev answerdotai
NVIDIA AI Developer @NVIDIAAIDev
81K Followers 321 Following All things AI for developers from @NVIDIA. Additional developer channels: @NVIDIADeveloper, @NVIDIAHPCDev, and @NVIDIAGameDev.
DeepSeek @deepseek_ai
973K Followers 0 Following Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism.
Vijay @__tensorcore__
2K Followers 515 Following MLIR, CUTLASS,Tensor Core arch @NVIDIA. Mechanic @hpcgarage. Exercise of any 1st amendment rights are for none other than myself.
Nathan Lambert @natolambert
56K Followers 853 Following Figuring out AI @allen_ai, open models, RLHF, fine-tuning, etc Contact via email. Writes @interconnectsai Wrote The RLHF Book Mountain runner
Red Hat AI @RedHat_AI
8K Followers 2K Following Deliver AI value with the resources you have, the insights you own, and the freedom you need.
NVIDIA @nvidia
2.4M Followers 47 Following The official handle for NVIDIA. Blog: https://t.co/JAn5eKOTBT Support: https://t.co/6ln5FVnA2o All our social media: https://t.co/Uc56dL57Dh
Charles 🎉 Frye @charles_irl
14K Followers 3K Following gpu enjoyer at @modal. he/him. ex @full_stack_dl, @weights_biases (acq. @CoreWeave), phd Berkeley @Redwood_Neuro. try https://t.co/SYWVMCazZ3
Alina Lozovskaya @ailozovskaya
717 Followers 294 Following ML Engineer at @huggingface 🤗 | linguist by eduction, engineer by profession | take photos and play music
Clémentine Fourrier ... @clefourrier
5K Followers 398 Following Evals @HuggingFace 🐍✨ "The future is already here, it’s just not very evenly distributed" (Gibson) Not an AGI believer, LLMs are good at form not substance
Benjamin Bossan @BenjaminBossan
227 Followers 80 FollowingTrends for United States
You might like
