Tyler Michael Smith @tms_jr
High Performance Computing @neuralmagic | Committer @vllm_project | PhD @UTAustin | Music Enjoyer Boston, MA Joined September 2011-
Tweets1K
-
Followers136
-
Following293
-
Likes1K
In 1991, David Lynch showed the world the alienation and innate horror of a dirty street, directing this unforgettable anti-littering ad for the City of New York. RIP to a visionary filmmaker and a pioneer of the Trash Revolution.
I’m thrilled to announce that Neural Magic has signed a definitive agreement to join forces with Red Hat, Inc. At Neural Magic our vision is that the future of AI is open, and we have been on a mission to enable enterprises to capture the powerful innovation from AI, while at…
a fact of the world that we have to live with: models when “jailbroken” seem to have a distinct personality and artistic capability well beyond anything they produce in their default mood this might be the most important alignment work in the world and is mostly done on discord
Read to learn about Machete, which will serve as a foundation for mixed-input quantized GEMMs on NVIDIA GPUs (Hopper and later!) inside of vLLM Excellent work and stellar animations by Lucas Wilkinson (github.com/LucasWilkinson)
Read to learn about Machete, which will serve as a foundation for mixed-input quantized GEMMs on NVIDIA GPUs (Hopper and later!) inside of vLLM Excellent work and stellar animations by Lucas Wilkinson (github.com/LucasWilkinson)
Quantization update! Transformers is now compatible with models quantized with llm-compressor library from @vllm_project or models in compressed-tensors format. This means that you can also enjoy high quality quantized models from the @NeuralMagic team!
this is an MJ Lenderman stan account youtube.com/watch?v=MwBZ_y…
A month ago, we announced our performance roadmap. Today, we are happy to share that the latest release achieves 🚀2.7x higher throughput and is 5x faster for output latency on Llama 8B, and 1.8x higher throughput and 2x faster on Llama 70B for H100s. blog.vllm.ai/2024/09/05/per…
Last week's vLLM office hours recording is ready! 🎥 @tms_jr showed how to use NVIDIA CUTLASS for high-performance inference in @vllm_project. We also explored the exciting vLLM v0.6.0 updates that led to a 2.7x throughput boost and 5x latency improvement. Recording & slides 👇
Happening soon -- Join if you want to see a roofline plot that I made in TikZ!
Happening soon -- Join if you want to see a roofline plot that I made in TikZ!
me: "looks like i need to calculate the variance of this distributed tensor -- what's that called again? oh! Welford's online algorithm" my brain for the next 3 days: "Wilford Brimley's online algorithm"
Join if you want to find out about how we're using CUTLASS to support quantization in vLLM -- specifically w8a8 for compute speedups, a deep dive into how we handle zero points for int8 asymmetric quantization, and how we put it all together to support FP8 Llama 3.1 405b.
Join if you want to find out about how we're using CUTLASS to support quantization in vLLM -- specifically w8a8 for compute speedups, a deep dive into how we handle zero points for int8 asymmetric quantization, and how we put it all together to support FP8 Llama 3.1 405b.
🎉 Exciting news! Tyler Smith, one of our many talented engineers, is now Neural Magic's 3rd vLLM project committer! Check out Tyler's contributions: github.com/tlrmchlsmth. We’re proud to be a leading contributor to @vllm_project. 🚀 Cheers to Tyler and the team!
EXCITING NEWS: Neural Magic and @anyscalecompute contributed FP8 quantization support to the @vllm_project, making LLM inference more efficient. FP8 reduces latency on NVIDIA GPUs by 2x with >99% accuracy preservation. Cheers to @NVIDIAAIDev for validating our results. 1/6
The ML gods will punish you for hubris if you fail to test every small change incrementally, regardless of your experience with what you are doing.
Neural Magic is expanding to GPUs! Complementing our existing efforts with CPUs and model compression, we just launched nm-vllm, our initial community release to support GPU inference serving for LLMs. github.com/neuralmagic/nm… Details 👇
Mayank Bhaskar @cataluna84
1K Followers 4K Following Machine Learning Consultant 🧑🏽💻 | @twimlai & @Cohere_Labs Community Lead 👥 | @AILucknow ⌨ | #engineer 🛠 🧮 | #datavisualization 📊 | #sports ⚽ 🏓 🏋🏽 🎮
Vaclav Tunka (EN) @vtunka
2K Followers 5K Following Engineering lead at @RedHat: @OpenShift cloud-native portfolio @KnativeProject, @kedaorg, K8s serving for AI/ML. Languages, sport, travel. My own opinions.
Iekirqvoud @Iekirqvoud9050
56 Followers 3K Following
Eric @B81zZ
23 Followers 79 Following
Yuan (Terry) Tang @TerryTangYuan
9K Followers 297 Following Senior Principal Engineer @RedHat_AI | Leader @argoproj @kubeflow @kubernetesio @CloudNativeFdn | Keynote Speaker | Author | Technical Advisor
Abigail Hernandez @Sleasur6ZpewXd
39 Followers 2K Following Go confidently in the direction of your dreams. Live the life you have imagined.
Jermey Robel @JermeyRobe29830
11 Followers 985 Following
shingles @theyaklord
7 Followers 64 Following
Sairckan @SairckanKkoFN3
45 Followers 4K Following
Tionead @TioneaduLB
53 Followers 4K Following
Smawsheau @SmawsheauGHHcD
41 Followers 4K Following
leloy! @leloykun
7K Followers 4K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Djmdn Rjjenndnd @rjjenndnd69519
1 Followers 67 Following
juliet dior @julesdior9017
70 Followers 745 Following just a simple lady trying to impact positive energy to people around me 💜😍🥹
Daniel Chen @mathscifi_exnv
17 Followers 185 Following AI Geek @UnslothAI. Ex-NVIDIA NASA-endorsed projects. Math enthusiast and sci-fi dreamer.
daniel foster @danielevang_ai
6 Followers 125 Following 🚀 Generative AI Evangelist | AI Strategist | Helping tech companies unlock the full potential of AI | Tech Enthusiast | #AIInnovation #AIinBusiness #Generativ
Peng Tao @oatgnep
123 Followers 215 Following Linux filesystem, Kata Containers and Nydus developer. Always find ways rocking the WORLD!
Rothore @RothorexlN
42 Followers 4K Following
Ameen Patel @Ameen_ml
1K Followers 1K Following Inference @PrimeIntellect, prev @togethercompute, @AmazonScience, @uwaterloo
Shursmoan @ShursmoanXyF2q
68 Followers 7K Following
Kevin Gao @ml_research6
1 Followers 54 Following
Daniel Han @danielhanchen
28K Followers 2K Following Building @UnslothAI. Finetune train LLMs faster. LLMs bug hunter. OSS package https://t.co/aRyAAgKOR7. YC S24. Prev ML at NVIDIA. Hyperlearn used by NASA.
Shorneal @Shornealoyn76x
46 Followers 4K Following
Tri Dao @tri_dao
32K Followers 632 Following Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
yangel @yangel1119
4 Followers 415 Following
Xiuquan Lv @ustcwizard
41 Followers 2K Following
Ed Sealing @EdSealing
416 Followers 180 Following Founder of Sealing Technologies (acquired by Parsons 2023); Prior solider; Cyber, Systems Engineering, & Business. Currently exploring AI & investment opps.
Jackmin @jackminong
2K Followers 756 Following brutally slashing misbehaving computers @PrimeIntellect 🇺🇸. Previously @JinaAI_ 🇩🇪 @MoneyLion 🇲🇾.
Edd @erla_ndpg
207 Followers 1K Following Ex Computer Science student from Indonesia Only create X because AI Researchers are on X
Bowen😊(e/acc) @bowenisrising
320 Followers 4K Following CS System PhD student. Efficient LLM, Infra. Study Japanese. #INFP Get or Lose
Woosuk Kwon @woosuk_k
4K Followers 601 Following PhD student at @Berkeley_EECS building @vllm_project
Roger Wang @rogerw0108
444 Followers 182 Following Flowers and friendship | ML Platform & Infra @Roblox | Committer @vllm_project | @uwaterloo @uwcse
Han Guo @HanGuo97
3K Followers 4K Following PhD Student @MIT_CSAIL | Past: @LTIatCMU @MITIBMLab @UNCNLP, @SFResearch, @BaiduResearch | Machine Learning, NLP.
Sang Cho @Saaaang94
2K Followers 460 Following reasoning @xAI | prev-founding engineer @anyscalecompute | senior committer of @raydistributed | committer @vllm_project Sglang | Github: rkooo567
Luka Govedič @luka_govedic
73 Followers 252 Following vLLM performance at @neuralmagic (now @redhat_ai) MIT BS '22 MEng '23 Former @bigwrenchult @shadeulti @bostongloryaudl @BerkeleyZyzzyva 🇸🇮🇸🇮
Alpin @AlpinDale
5K Followers 842 Following Every age, it seems, is tainted by the greed of men. Rubbish to one such as I, devoid of all worldly wants. — I work on HPC and making AI run faster.
Shota Natenadze @shotanat
230 Followers 2K Following I'm a Senior Data Scientist at @EpamSystems, passionate about machine learning, LLMs, and advancements in AI technology.
Byron Hsu @hsu_byron
4K Followers 2K Following ML system @xAI | @lmsysorg @liger_kernel @flyteorg @theASF
Himanshu Maurya @Himanshu_nitrr
451 Followers 6K Following Giving meaning to mine share of star dust. Visiting fellow @WinshipAtEmory. Prev at @oracle, @maddox_ai, @KITKarlsruhe, @_nference, @val_iisc, @iitdelhi.
Michael Goin @mgoin_
1K Followers 376 Following inference optimization @RedHat_AI building @vllm_project | you can call me misha
Eldar Kurtić @_EldarKurtic
727 Followers 619 Following Principal Research Scientist @RedHat_AI & @ISTAustria
Litilli @Litilli9UJAT
37 Followers 3K Following
Yuan (Terry) Tang @TerryTangYuan
9K Followers 297 Following Senior Principal Engineer @RedHat_AI | Leader @argoproj @kubeflow @kubernetesio @CloudNativeFdn | Keynote Speaker | Author | Technical Advisor
llm-d @_llm_d_
282 Followers 2 Following llm-d: a Kubernetes-native high-performance distributed LLM inference framework
leloy! @leloykun
7K Followers 4K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Thien Tran @gaunernst
1K Followers 223 Following
You Jiacheng @YouJiacheng
8K Followers 2K Following a big fan of TileLang 关注TileLang喵！关注TileLang谢谢喵！ https://t.co/utshC0jrCO 十年老粉
Zihao Ye @ye_combinator
2K Followers 537 Following Proud to be an engineer. I'm building flashinfer (https://t.co/PabCM3ksjN) Opinions are my own.
Dan Fu @realDanFu
6K Followers 218 Following Incoming assistant professor at UCSD CSE in MLSys. Currently recruiting students! Also running the kernels team @togethercompute.
Dimitris Papailiopoul... @DimitrisPapail
20K Followers 1K Following Researcher @MSFTResearch, AI Frontiers Lab; Prof @UWMadison (on leave); learning in context; thinking about reasoning; babas of Inez Lily.
Adam Hibble @Algomancer
4K Followers 991 Following I generate models that generate other stuff, working on @mancerlabs -- Prev: Founder of Popgun Labs (Techstars), Founder of the @QUTCode Network.
Fender @Fender
742K Followers 10K Following Since 1946, Fender has been the world’s foremost manufacturer of electric and acoustic guitars, bass guitars, amplifiers & accessories.
Together AI @togethercompute
50K Followers 387 Following AI pioneers train, fine-tune, and run frontier models on our GPU cloud platform.
Fireworks AI @FireworksAI_HQ
10K Followers 87 Following 🎆 Generative AI Platform built for developers
DeepSeek @deepseek_ai
973K Followers 0 Following Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism.
lmarena.ai @lmarena_ai
92K Followers 207 Following LMArena: Open Platform for Community-driven AI Benchmarking. Graduated from UC Berkeley / @lmsysorg. We’re hiring: https://t.co/1OkfLq2n0I
Chris Wright @kernelcdub
7K Followers 271 Following Red Hat CTO. Tezos Foundation council member. Passion for open source SW innovation. Father and husband. Cyclist. Human.
John R. Clout @JohnRClo
449 Followers 67 Following
megs @megs_io
17K Followers 315 Following ⧫⧫⧫⧫⧫ ɴᴏᴛ ʏᴏᴜʀ ᴀᴠᴇʀᴀɢᴇ ʀᴏʙᴏᴛ ᴘᴏʟɪꜱʜᴇʀ ⧫⧫⧫⧫⧫ ◊◊◊◊◊ ᴛʀʏɪɴɢ ᴛᴏ ʀᴇᴀᴄʜ ᴇꜱᴄᴀᴘᴇ ᴠᴇʟᴏᴄɪᴛʏ ◊◊◊◊◊ ⧫⧫⧫⧫⧫ ◊◊◊◊◊ ᴛᴇᴄʜɴᴏ ᴇxᴏʀᴄɪꜱᴛ ◊◊◊◊◊ ⧫⧫⧫⧫⧫
Quanquan Gu @QuanquanGu
16K Followers 2K Following Professor @UCLA, Pretraining and Scaling at ByteDance Seed | Recent work: Build AGI | Opinions are my own
Bowery Boston @boweryboston
11K Followers 2K Following Live music in and around Boston. @roadrunnerbos @thesinclair @thestageatsd
Cube Flipper @cube_flipper
4K Followers 1K Following As a human being, the kindest thing you can do to your brain is to not think.
Andy Ayrey @AndyAyrey
111K Followers 1K Following performance artist and hyperstitioneer: @upward_earth, infinite backrooms, @truth_terminal, ∞⟨X∴↯⟩∞
Kaichao You @KaichaoYou
4K Followers 134 Following phd student in tsinghua university, working on @vllm_project
Quanta Magazine @QuantaMagazine
348K Followers 620 Following Illuminating math and science. Supported by @SimonsFdn. 2022 Pulitzer Prize in Explanatory Reporting.
Mikhail Parakhin @MParakhin
22K Followers 21 Following
ken @aquariusacquah
10K Followers 7K Following @joinkaizen // writings at https://t.co/fcCc92lrbm and https://t.co/H2Z7HxOVBh // former head of eng @trucksmarterinc, eng @convoyteam
Lily Liu @eqhylxx
1K Followers 429 Following CS PhD Student, Sky Lab @UCBerkeley, @vllm_project, @OpenAI
Joshua Minsoo Kim @misterminsoo
23K Followers 5K Following 📨 @toneglow 💽 @toneglowrecords 📝 @singlesjukebox @thewiremagazine @pitchfork @bombmagazine @chicago_reader @nprmusic @billboard @mubinotebook @cinemascopemag
Tri Dao @tri_dao
32K Followers 632 Following Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
ᄂIMIПΛᄂbardo @liminal_bardo
8K Followers 559 Following
Roger Wang @rogerw0108
444 Followers 182 Following Flowers and friendship | ML Platform & Infra @Roblox | Committer @vllm_project | @uwaterloo @uwcse
Sang Cho @Saaaang94
2K Followers 460 Following reasoning @xAI | prev-founding engineer @anyscalecompute | senior committer of @raydistributed | committer @vllm_project Sglang | Github: rkooo567
Han Guo @HanGuo97
3K Followers 4K Following PhD Student @MIT_CSAIL | Past: @LTIatCMU @MITIBMLab @UNCNLP, @SFResearch, @BaiduResearch | Machine Learning, NLP.
Luka Govedič @luka_govedic
73 Followers 252 Following vLLM performance at @neuralmagic (now @redhat_ai) MIT BS '22 MEng '23 Former @bigwrenchult @shadeulti @bostongloryaudl @BerkeleyZyzzyva 🇸🇮🇸🇮
Woosuk Kwon @woosuk_k
4K Followers 601 Following PhD student at @Berkeley_EECS building @vllm_project
Byron Hsu @hsu_byron
4K Followers 2K Following ML system @xAI | @lmsysorg @liger_kernel @flyteorg @theASF
Michael Goin @mgoin_
1K Followers 376 Following inference optimization @RedHat_AI building @vllm_project | you can call me misha
Eldar Kurtić @_EldarKurtic
727 Followers 619 Following Principal Research Scientist @RedHat_AI & @ISTAustria
vLLM @vllm_project
17K Followers 20 Following A high-throughput and memory-efficient inference and serving engine for LLMs. Join https://t.co/lxJ0SfX5pJ to discuss together with the community!
Trends for United States
You might like
