Zhaoyue Cheng @joey__cheng

joeycheng.me Joined February 2011

Tweets

117
Followers

185
Following

2K
Likes

216

Aleksa Gordić (水平问题) @gordic_aleksa

5 days ago

New in-depth blog post - "Inside vLLM: Anatomy of a High-Throughput LLM Inference System". Probably the most in depth explanation of how LLM inference engines and vLLM in particular work! Took me a while to get this level of understanding of the codebase and then to write up…

55 369 2K 248K 3K

Download Image

Deedy @deedydas

a month ago

This is the best public resource on scaling hardware for AI, and its free. "How to Scale Your Model" is the bible from Google DeepMind that covers the math, systems and scaling laws for LLM training and inference workloads. Approachable yet thorough. Absolute Must-read.

19 116 1K 71K 1K

Download Image

九原客 @9hills

2 months ago

节选： 1. 开源了就意味着第一方再也不能用各种 hack 的方式粉饰效果，必须拿出足够通用、任何第三方拿到同样的 weights 都要能很简单地复现出你的效果才行。 2. 绝大多数 Agent 产品，离了 Claude 以后，什么都不是。

🐻熊狸 @bigeagle_xd

2 months ago

34 112 755 289K 478

5 10 114 30K 35

Tri Dao @tri_dao

2 months ago

Getting mem-bound kernels to speed-of-light isn't a dark art, it's just about getting the a couple of details right. We wrote a tutorial on how to do this, with code you can directly use. Thanks to the new CuTe-DSL, we can hit speed-of-light without a single line of CUDA C++.

Wentao Guo @WentaoGuo7

2 months ago

13 73 332 77K 204

Download Image

7 57 522 48K 362

Jeff Dean @JeffDean

2 months ago

Very cool thread about the CS336 Language Models from Scratch course at Stanford taught by @percyliang et al. Makes me wish I was a student again!

Percy Liang @percyliang

3 months ago

Very cool thread about the CS336 Language Models from Scratch course at Stanford taught by @percyliang et al. Makes me wish I was a student again!

45 585 5K 659K 7K

19 85 979 111K 603

Nando de Freitas @NandoDF

3 months ago

Ilya Sutskever, U of T honorary degree recipient, June 6, 2025 youtu.be/zuZ2zaotrJs?si… via @YouTube This is a must watch speech, the wisest words you could hear. Congratulations 🙌 @ilyasut @UofT was very fortunate to have you and so many other amazing students and…

6 26 188 29K 124

Dan Roy @roydanroy

3 months ago

Dear PhD students now regretting taking offers at US schools: If you turned down PhD offers in Canada, but want to rethink that, email the professors who were trying to recruit you. They might be able to pull some strings. Your sane neighbor to the north, Canada

22 66 812 89K 55

Sasha Rush @srush_nlp

4 months ago

Strong recommend for this book and the JAX/TPU docs, even if you are using Torch / GPUs. Clean notation and mental model for some challenging ideas. github.com/jax-ml/scaling… github.com/jax-ml/scaling… docs.jax.dev/en/latest/note…

9 160 1K 76K 1K

Download Image

LMSYS Org @lmsysorg

4 months ago

🚀 Breaking: SGLang provides the first open-source implementation to serve @deepseek_ai V3/R1 models with large-scale expert parallelism and prefill-decode disaggregation on 96 GPUs. It nearly matches the throughput reported by the official DeepSeek blog, achieving 52.3K input…

11 84 400 160K 178

Download Image

Chujie Zheng @ChujieZheng

4 months ago

The SGLang guys @lmsysorg are always doing such incredible work. This is also what the open-source community has made possible! 🚀

3 14 142 9K 45

Download Image

Sundar Pichai @sundarpichai

4 months ago

4/ Our AI leadership was on display at @googlecloud Next, where we launched Ironwood, our most powerful TPU yet — 10X compute boost, optimized for inference at scale. And we’re first to bring NVIDIA’s next-gen Blackwell GPUs to customers. Now with tools for building multi-agent…

3 12 392 52K 17

Aravind Srinivas @AravSrinivas

9 months ago

Full talk of Ilya here 👇

Vincent Weisser @vincentweisser

9 months ago

Full talk of Ilya here 👇

83 761 3K 785K 5K

Download Video

17 93 753 93K 676

theseriousadult @gallabytes

10 months ago

> go to Claude store > ask the man at the counter if it is TPU Claude or Trainium Claude > he doesn't understand > pull out illustrated diagram explaining the differences between TPU and Trainium > it's a good Claude sir > get the membership > Trainium Claude

3 3 146 9K 12

François Chollet @fchollet

10 months ago

When a product is 10x better, it doesn't matter if it starts from a small niche and has no marketing. It will grow steadily, and eventually it will win. That's how I feel about the current trajectory of JAX.

40 52 920 91K 236

Ilya Sutskever @ilyasut

11 months ago

Congratulations to @geoffreyhinton for winning the Nobel Prize in physics!!

209 619 12K 827K 360

Jeff Dean @JeffDean

a year ago

I gave a talk at the MLSys conference in May this year, touching on various topics, including large-scale ML training systems, abstractions for embedding ML choices in computer systems, and CO2e emissions of language model training. mlsys.org/virtual/2024/i…

12 52 337 44K 222

Yangqing Jia @jiayq

a year ago

An index of my llama3.1 405b related posts this week: - Memory size matters for inference: x.com/jiayq/status/1… - Lepton endpoint release: x.com/jiayq/status/1… - Measurement and qualitative analysis of the model: x.com/jiayq/status/1… - Running 405b on a poor 4090? A fun…

Yangqing Jia @jiayq

a year ago

7 34 152 136K 76

Download Image

1 23 121 28K 133

xjdr @_xjdr

a year ago

they did it. these crazy bastards actually released a full and detailed technical report. the llama 3 paper is going to be up there with the deepseekv2 paper in terms of detail and quality

4 18 464 22K 77

xjdr @_xjdr

a year ago

The llama3 paper is just as, if not more, important than the model and was completely unexpected. congratulations @AIatMeta and the whole llama3 team. This is outstanding work.

9 18 410 26K 31

Jan Leike @janleike

a year ago

To all OpenAI employees, I want to say: Learn to feel the AGI. Act with the gravitas appropriate for what you're building. I believe you can "ship" the cultural change that's needed. I am counting on you. The world is counting on you. :openai-heart: