Simon Mo @simon_mo_
@vllm_project Joined July 2018-
Tweets120
-
Followers1K
-
Following342
-
Likes143
It has been 1+ month of intense work! Now time to get some sleep 😴
It has been 1+ month of intense work! Now time to get some sleep 😴
I didn't expect the first section "KV-cache hit rate is the single most important metric for a production-stage AI agent" but 🤯
I didn't expect the first section "KV-cache hit rate is the single most important metric for a production-stage AI agent" but 🤯
Long time in the making and I'm beyond excited about the future of vLLM!
Long time in the making and I'm beyond excited about the future of vLLM!
Announcing the first Codex open source fund grant recipients: ⬩vLLM - inference serving engine @vllm_project ⬩OWASP Nettacker - automated network pentesting @iotscan ⬩Pulumi - infrastructure as code in any language @PulumiCorp ⬩Dagster - cloud-native data pipelines @dagster…
😲 super cool !!! Reminded me of Kevin's thesis "Structured Contexts For Large Language Models" and this is such a natural continuation of the idea.
😲 super cool !!! Reminded me of Kevin's thesis "Structured Contexts For Large Language Models" and this is such a natural continuation of the idea.
🙏 @deepseek_ai's highly performant inference engine is built on top of vLLM. Now they are open-sourcing the engine the right way: instead of a separate repo, they are bringing changes to the open source community so everyone can immediately benefit! github.com/deepseek-ai/op…
Having been at every single vLLM meetup, I won't miss this one :D Looking forward to meet all the vLLM users in Boston!
Having been at every single vLLM meetup, I won't miss this one :D Looking forward to meet all the vLLM users in Boston!
Landed my first PR in @vllm_project 1 year ago today (github.com/vllm-project/v…) 38K LOC and 100+ PRs later and we are just getting started
Robert and I started contributing to vLLM around the same time and today is my turn. Back then vLLM had only about 30 contributors. One year later, today the project has received contributions from 800+ community members! and we're just getting started github.com/vllm-project/v…
Robert and I started contributing to vLLM around the same time and today is my turn. Back then vLLM had only about 30 contributors. One year later, today the project has received contributions from 800+ community members! and we're just getting started github.com/vllm-project/v…
We landed the 1st batch of enhancements to the @deepseek_ai models, starting MLA and cutlass fp8 kernels. Compared to v0.7.0, we offer ~3x the generation throughput, ~10x the memory capacity for tokens, and horizontal context scalability with pipeline parallelism.
Finally, I want to give a special thanks to the @vllm_project team (@KaichaoYou @woosuk_k @simon_mo_ @zhuohan123) for their invaluable support in debugging NCCL weight transfer issues. They made our 70 RLVR weight transfer 45x faster and 405B RLVR even possible! See…
Our biggest milestone yet! I'm particularly excited how the vLLM contributor community organized from many organization to deliver a high quality V1 engine core. We are just getting started 🚀
Our biggest milestone yet! I'm particularly excited how the vLLM contributor community organized from many organization to deliver a high quality V1 engine core. We are just getting started 🚀
1/6 🚀 Introducing Sky-T1-32B-Preview, our fully open-source reasoning model that matches o1-preview on popular reasoning and coding benchmarks — trained under $450! 📊Blog: novasky-ai.github.io/posts/sky-t1/ 🏋️♀️Model weights: huggingface.co/NovaSky-AI/Sky…
Yuming DU @yuming_du
1K Followers 4K Following Research Scientist at @Meta GenAI. | PhD in Computer Vision @ImagineEnpc | Opinions are my own
Paige Beane @BeanePaige
0 Followers 15 Following
Yahya @AliSherGhumman_
0 Followers 42 Following Parallel Programming | Distributed Systems | Deep Learning
Augustas Macijauskas @augustasmac
28 Followers 569 Following ML engineer at Spike Technologies. Previously, a machine learning master's student at @Cambridge_Uni. Excited about opportunities in deep-learning and LLMs.
Bryce Schmidtchen @_bschmidtchen
755 Followers 1K Following building something new · ex- Vision Pro · SF 🌁
!.! @xypyth
45 Followers 4K Following
Adam Falls @AdamFalls172137
53 Followers 4K Following
Eduardo Slonski @EduardoSlonski
775 Followers 687 Following AI Researcher | LLM Reasoning and Scaling
GLASS_PhD @joybeingfree99
7K Followers 7K Following "...A free man's wisdom is, to meditate not on death but on life".
Kirill Borisov @kiborisov
194 Followers 2K Following I look at data. ai multilinguality at meta. TSFM (cohort-1)
paul @paul_okewunmi
1K Followers 4K Following ML/AI Engineer | MLH Fellow'23 @ Meta | Drone Hobbyist
Fatih⏩⤴️ @taskinfatih
660 Followers 5K Following Lover of all novel and hard concepts: especially machine learning and systems theory
SocialWatch @SocialWatchNew
985 Followers 3K Following Retail Investor. I do NOT provide financial advice. My messages are my personal opinions only. In my opinion, there is no such thing called DUMB MONEY.
Lohith Javali @lohithjavali
383 Followers 3K Following tech • makesomethingwonderful • putsomethingback
ashish ranajn @RanajnAshi63328
1 Followers 495 Following
Nihal Potdar @NihalPotdar
44 Followers 160 Following Inference Performance @DbrxMosaicAI | @UWaterloo Alum | Prev @Citadel
Swayam Singh @swayaminsync
1K Followers 1K Following ML Research @MSFTResearch | Core Maintainer @numpy_team (QuadDType)
Murali Nandan @muralinandann
3 Followers 179 Following
Taha Magdy @tahamagdy_
4 Followers 61 Following
blakemcelheny @blakemcelheny
253 Followers 4K Following Community member participating in civic life.
- @monkeyfighter98
339 Followers 595 Following
Alejandro Camacho Mas... @alejandrocamam
265 Followers 6K Following
andrew lee @ahlee22
0 Followers 29 Following
Moummar @MoummarNawafleh
2K Followers 923 Following founder @ https://t.co/tQ6TFdKqeO - prev. sold $30M+ in semiconductors
Mathieu Bertolo @MathieuBRTL
11K Followers 2K Following CEO & Explorer @briefstory ⚡️ @mesaj_io 💬 Metaverse Walker 🧑🏻🚀 #JamaisSansElles
Su @BilgeSuuuu
1K Followers 4K Following
Wilton Gorske ⛓️ @wiltongorske
51K Followers 288 Following marketing at @linera_io. running @torproject relays on @raspberry_pi and with @emeraldonion. free and open-source soft/hardware.
nikhil tayal @Alloutnikhil
3K Followers 5K Following I love to build products and services that people want to use. Trying my best to be max useful to the humanity.
Edward Ochoa @edwardochoaphd
0 Followers 4K Following
Shicheng Liu @ShichengGLiu
194 Followers 177 Following CS Phd @StanfordNLP @StanfordOVAL RS Intern @meta
James Alcorn @JamesAlcorn94
1K Followers 2K Following AI/infra at Lightspeed. Prev Zetta VC, Spectrum Equity, UCBerkeley 🇦🇺 🇺🇸
Character.AI @character_ai
150K Followers 13 Following Unleash your imagination—chat, create, explore.
Ion Stoica @istoica05
5K Followers 20 Following Professor at UC Berkeley, co-founder of Databricks, Anyscale, LMArena, Conviva.
SemiAnalysis @SemiAnalysis_
34K Followers 16 Following
NovaSky @NovaSkyAI
3K Followers 15 Following Next-generation Open Vision and AI @BerkeleySky Contact: [email protected]
OpenAI @OpenAI
4.3M Followers 3 Following OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6Lg202
Yichao 'Peak' Ji @peakji
25K Followers 167 Following Co-founder & Chief Scientist of Manus @ManusAI_HQ
Letta @Letta_AI
4K Followers 0 Following Stateful agents that remember and learn https://t.co/4upATozfXj
Hao Zhang @haozhangml
6K Followers 474 Following Asst. Prof. @HDSIUCSD and @ucsd_cse running @haoailab. Cofounder and runs @lmsysorg. 20% with @Snowflake
Liangsheng Yin （尹... @lsyincs
257 Followers 258 Following B.E. @sjtu1896 Maintainer @sgl_project Incoming Ph.D. Student @UCBerkeley
Michael Luo @michaelzluo
540 Followers 213 Following Project Lead @Agentica_ | Prev. Researcher @GoogleDeepMind | PhD at UC Berkeley @berkeley_ai
Amog Kamsetty @AmogKamsetty
328 Followers 218 Following AI Systems @BerkeleySky← Building @raydistributed + founding ML eng @anyscalecompute ← ML research @ucbrise
Zihao Ye @ye_combinator
2K Followers 537 Following Proud to be an engineer. I'm building flashinfer (https://t.co/PabCM3ksjN) at @NVIDIA Opinions are my own.
Kaichao You @KaichaoYou
4K Followers 134 Following phd student in tsinghua university, working on @vllm_project
Justin Wong @justinywong_
228 Followers 226 Following CS PhD Student working on diverse AI generation advised by @profjoeyg and Sanjit Seshia. formerly @aiatmeta, @nuro, @columbia
Tianle Cai @tianle_cai
8K Followers 5K Following ML PhD @Princeton. Life-long learner, hacker, and thinker. Tech consultant & angel investor. Prev @togethercompute @GoogleDeepMind @MSFTResearch @citsecurities.
Myle Ott @myleott
2K Followers 541 Following research @thinkymachines, previously @GoogleDeepMind, @character_ai and @MetaAI
Bowen Liang @LiangBowen
276 Followers 221 Following Research Engineer @GoogleDeepMind, formerly @character_ai
Tian Xie @tianxie233
307 Followers 447 Following Research Scientist @GoogleDeepMind | previously @character_ai @SFResearch @USC_ISI
Stephen Roller @stephenroller
5K Followers 1K Following MoTS @thinkymachines. previously pre-training @googledeepmind ,@character_ai, and @aiatmeta.
Ying Xiao @YingXiao
820 Followers 298 Following ML Researcher at @character_ai; previously @googIeresearch, @Twitter Cortex, and @GoldmanSachs. Making haste slowly.
Sam Shleifer @sam_shleifer
3K Followers 856 Following Founding Team Thinking Machines. Previously @c.ai. I like zsh aliases, audiobooks, and running. This does not reflect the opinions of my future self.
Xiangyi Li 李向一 @xdotli
2K Followers 1K Following chat with any agent in iMessage. backed by @fdotinc. made @benchflow_ai @lemmahq
Kenneth Ding @kd78277
142 Followers 124 Following AI Research Engineer @LeptonAI | Creator of GPTFast | Math & CS @ NYU Courant
Jason Wei @_jasonwei
98K Followers 635 Following ai researcher @meta superintelligence labs, past: openai, google 🧠
Yi Tay @YiTayML
46K Followers 81 Following research scientist @googledeepmind ✨♊, model co-lead/captain of gemini deepthink imo gold medal 🥇, opinions are my own.
Hyung Won Chung @hwchung27
38K Followers 302 Following AI Research Scientist @Meta Superintelligence Labs. Past: @OpenAI / @Google Brain / PhD @MIT
Kai Fricke @kaifri
9 Followers 53 Following
lmarena.ai @lmarena_ai
92K Followers 207 Following LMArena: Open Platform for Community-driven AI Benchmarking. Graduated from UC Berkeley / @lmsysorg. We’re hiring: https://t.co/1OkfLq2n0I
Romil Bhardwaj @bromil101
391 Followers 181 Following Building SkyPilot @skypilot_org | PhD AI+Systems @Berkeley_EECS @ucbrise
Bill Gurley @bgurley
720K Followers 2K Following VC @benchmark, Investor: @grubhub, @zillow, @uber, @stitchfix, @nextdoor, @glassdoor, @hacker0x01, @solvhealth, @instawork Trustee @sfiscience, Podcast: @bg2pod
Xingyao Wang @xingyaow_
5K Followers 1K Following Co-founder @allhands_ai, building OpenHands | PhD candidate @IllinoisCDS | BS @UMichCSE ('22) | Ex Intern @GoogleAI @Microsoft | Opinions are my own
Ben Firshman @bfirsh
8K Followers 1K Following @replicate / https://t.co/P3bIJyMtkU / https://t.co/vt6LozqLGX / previously: @docker, @guardian, @govuk
Ji Lin @jilin_14
6K Followers 944 Following Research @Meta Superintelligence Lab | Prev: Research @OpenAI; PhD @MIT
Lisa Dunlap @lisabdunlap
1K Followers 278 Following messin around with model evals @berkeley_ai | prev @lmarena_ai
Zhihao Jia @JiaZhihao
3K Followers 686 Following Assistant professor of Computer Science at Carnegie Mellon University. Research on systems and machine learning.
Dacheng Li @DachengLi177
1K Followers 778 Following Student Lead @NovaSkyAI, PhD @BerkeleySky, @lmsysorg, @lmarena_ai | Prev: @Nvidia @Google @SCSatCMU | @istoica05 @profjoeyg @songhan_mit @haozhangml @ericxing
Alex Krentsel @AlexKrentsel
249 Followers 180 Following Senior Sys Research Eng @ Google, Systems PhD Student @Berkeley_EECS. Prev. @YouTube, @Meta, Lecturer @HowardU read my writing: https://t.co/mxRccOCFDp
Shiyi Cao @shiyi_c98
1K Followers 595 Following Student Lead @NovaSkyAI, PhD student @UCBerkeley @BerkeleySky, MSc @ETH, B.S @sjtu1896, llm and system
Dylan Patel @dylan522p
94K Followers 941 Following SemiAnalysis Boutique AI & Semiconductor Research and Consulting DMs are open for consulting, quotes, or to talk shop
Mistral AI @MistralAI
156K Followers 0 Following Frontier AI in your hands. https://t.co/VdyEwpQsiy Apps: https://t.co/1vZA5XdBYo https://t.co/rj5G4u5sHuTrends for United States
You might like
