Muru Zhang @zhang_muru

First-year PhD @nlp_usc | Student Researcher @GoogleDeepmind | bsms @uwcse | Prevs. @togethercompute @AWS nanami18.github.io Joined August 2021

Tweets

74
Followers

564
Following

306
Likes

322

Kaiyue Wen @wen_kaiyue

3 days ago

(1/n) Check out our new paper: "Fantastic Pretraining Optimizers and Where to Find Them"! >4000 models to find the fastest optimizer! 2× speedups over AdamW? Unlikely. Beware under-tuned baseline or limited scale! E.g. Muon: ~40% speedups <0.5B & only 10% at 1.2B (8× Chinchilla)!

11 76 370 132K 190

Download Image

XINYUE CUI @xinyue_cui411

a month ago

Can we create effective watermarks for LLM training data that survive every stage in real-world LLM development lifecycle? Our #ACL2025Findings paper introduces fictitious knowledge watermarks that inject plausible yet nonexistent facts into training data for copyright…

3 8 38 3K 9

Download Image

Robin Jia @robinomial

a month ago

I’ll be at ACL 2025 next week where my group has papers on evaluating evaluation metrics, watermarking training data, and mechanistic interpretability. I’ll also be co-organizing the first Workshop on LLM Memorization @l2m2_workshop on Friday. Hope to see lots of folks there!

0 5 47 4K 4

Muru Zhang @zhang_muru

2 months ago

I'm at #ICML2025, presenting Ladder-Residual (arxiv.org/abs/2501.06589) at the first poster session tomorrow morning (7/15 11am-1:30pm), looking forward to seeing you at West Exhibition Hall B2-B3 #W-1000!

0 1 22 1K 2

Download Image

Chenghao Yang @chrome1996

2 months ago

Have you noticed… 🔍 Aligned LLM generations feel less diverse? 🎯 Base models are decoding-sensitive? 🤔 Generations get more predictable as they progress? 🌲 Tree search fails mid-generation (esp. for reasoning)? We trace these mysteries to LLM probability concentration, and…

1 28 91 15K 33

Download Video

Matthew Finlayson @mattf1n

3 months ago

I didn't believe when I first saw, but: We trained a prompt stealing model that gets >3x SoTA accuracy. The secret is representing LLM outputs *correctly* 🚲 Demo/blog: mattf1n.github.io/pils 📄: arxiv.org/abs/2506.17090 🤖: huggingface.co/dill-lab/pils-… 🧑‍💻: github.com/dill-lab/PILS

3 23 97 10K 57

Download Image

Johnny Tian-Zheng Wei @johntzwei

3 months ago

Hi all, I'm going to @FAccTConference in Athens this week to present my paper on copyright and LLM memorization. Please reach out if you are interested to chat about law, policy, and LLMs!

Johnny Tian-Zheng Wei @johntzwei

6 months ago

Hi all, I'm going to @FAccTConference in Athens this week to present my paper on copyright and LLM memorization. Please reach out if you are interested to chat about law, policy, and LLMs!

2 7 38 6K 9

1 4 25 2K 0

Harvey Yiyun Fu @harveyiyun

3 months ago

LLMs excel at finding surprising “needles” in very long documents, but can they detect when information is conspicuously missing? 🫥AbsenceBench🫥 shows that even SoTA LLMs struggle on this task, suggesting that LLMs have trouble perceiving “negative space” in documents. paper:…

11 32 159 26K 69

Download Image

Piotr Nawrot @p_nawrot

3 months ago

We built sparse-frontier — a clean abstraction that lets you focus on your custom sparse attention implementation while automatically inheriting vLLM’s optimizations and model support. As a PhD student, I've learned that sometimes the bottleneck in research isn't ideas — it's…

9 52 320 41K 218

Download Image

varepsilon @var_epsilon

3 months ago

read the first letter of every name in the gemini contributors list

38 112 3K 212K 338

Download Image

Hao Xu @xuhaoxh

3 months ago

Wanna 🔎 inside Internet-scale LLM training data w/o spending 💰💰💰? Introducing infini-gram mini, an exact-match search engine with 14x less storage req than the OG infini-gram 😎 We make 45.6 TB of text searchable. Read on to find our Web Interface, API, and more. (1/n) ⬇️

6 23 63 21K 33

Download Image

Mickel Liu @mickel_liu

3 months ago

🤔Conventional LM safety alignment is reactive: find vulnerabilities→patch→repeat 🌟We propose 𝗼𝗻𝗹𝗶𝗻𝗲 𝐦𝐮𝐥𝐭𝐢-𝐚𝐠𝐞𝐧𝐭 𝗥𝗟 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 where Attacker & Defender self-play to co-evolve, finding diverse attacks and improving safety by up to 72% vs. RLHF 🧵

5 28 105 29K 57

Download Image

Muru Zhang @zhang_muru

3 months ago

After a year of internship with amazing folks at @togethercompute, I will be interning at @GoogleDeepMind this summer working on language model architecture! Hit me up and I will get you a boba at the bayview rooftop of my Emeryville apartment 😉

7 3 272 19K 34

Download Image

Yuqing Yang @yyqcode

3 months ago

🧐When do LLMs admit their mistakes when they should know better? In our new paper, we define this behavior as retraction: the model indicates that its generated answer was wrong. LLMs can retract—but they rarely do.🤯 arxiv.org/abs/2505.16170 👇🧵

5 26 115 14K 55

Download Image

Jingyu Liu @Jingyu227

3 months ago

Ever get bored seeing LLMs output one token per step? Check out HAMburger (advised by @ce_zhang), which smashes multiple tokens into a virtual token with up to 2x decoding TPS boost + reduced KV FLOPs and storage while maintaining quality! github.com/Jingyu6/hambur…

1 2 8 950 1

Download Video

Muru Zhang @zhang_muru

3 months ago

Extremely fun read that unifies many scattered anecdotes on RLVR together and conclude with a set of beautiful experiments and explanations :))

Stella Li @StellaLisy

3 months ago

Extremely fun read that unifies many scattered anecdotes on RLVR together and conclude with a set of beautiful experiments and explanations :))

73 348 2K 691K 1K

Download Image

0 2 14 1K 0

Deqing Fu @DeqingFu

4 months ago

Textual steering vectors can improve visual understanding in multimodal LLMs! You can extract steering vectors via any interpretability toolkit you like -- SAEs, MeanShift, Probes -- and apply them to image or text tokens (or both) of Multimodal LLMs. And They Steer!