Hao Peng @haopeng_nlp

🧩New blog: From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones Do LLMs learn new skills through RL, or just activate existing patterns? Answer: RL teaches the powerful meta-skill of composition when properly incentivized. 🔗:husky-morocco-f72.notion.site/From-f-x-and-g…

So many works talking about entropy, but what is the **mechanism** of entropy in RL for LLMs? 🤔 Our work gives a principled understanding, as well as two tricks that get entropy **controlled** 🧵

Can entropy minimization alone improve LLM performance? And how far can they go without any labeled data? This work answers both: yes, and surprisingly far 🐮 At inference EM can beat GPT4o Claude 3 opus & Gemini 1.5 pro on challenging scientific coding w/o any data/model update

🚨 Paper Alert: “RL Finetunes Small Subnetworks in Large Language Models” From DeepSeek V3 Base to DeepSeek R1 Zero, a whopping 86% of parameters were NOT updated during RL training 😮😮 And this isn’t a one-off. The pattern holds across RL algorithms and models. 🧵A Deep Dive

💡We find that models “think” 💭 in English (or in general, their dominant language) when processing distinct non-English or even non-language data types 🤯 like texts in other languages, arithmetic expressions, code, visual inputs, & audio inputs ‼️ 🧵⬇️arxiv.org/abs/2411.04986

🚨 I’m on the job market this year! 🚨 I’m completing my @uwcse Ph.D. (2025), where I identify and tackle key LLM limitations like hallucinations by developing new models—Retrieval-Augmented LMs—to build more reliable real-world AI systems. Learn more in the thread! 🧵

I'm on the academic job market! I develop autonomous systems for: programming, research-level question answering, finding sec vulnerabilities & other useful+challenging tasks. I do this by building frontier-pushing benchmarks and agents that do well on them. See you at NeurIPS!

Wanna train PRMs but process labels, annotated manually or automatically, sound too expensive to you😖? Introduce Implicit PRM🚀 – Get your model free process rewards by training an ORM on the cheaper response-level data, with a simple parameterization at no additional cost💰!

Curious whether video generation models (like #SORA) qualify as world models? We conduct a systematic study to answer this question by investigating whether a video gen model is able to learn physical laws. Three are three key messages to take home: 1⃣The model generalises…

What If LLMs can cite the pre-training source(s) supporting their parametric knowledge? Won't this dramatically improve verifiability and trustworthiness? We aimed to answer this during my internship @allen_ai Paper: arxiv.org/abs/2404.01019 To be presented at #COLM Thread👇👇

🎯 Introducing SOLO, a single Transformer architecture for unified vision-language modeling. SOLO accepts both raw image patches (in pixels) and texts as inputs, without using a separate pre-trained vision encoder. Paper: arxiv.org/abs/2407.06438 Code: github.com/Yangyi-Chen/SO…

Language models excel at undergraduate exams, but how do they fare in research? SciCode challenges models with real research coding problems. Even the best models solve less than 5%. Very proud of @MinyangTian1 and @luyu_gao for leading the charge!

Minyang Tian @MinyangTian1

a year ago

Language models excel at undergraduate exams, but how do they fare in research? SciCode challenges models with real research coding problems. Even the best models solve less than 5%. Very proud of @MinyangTian1 and @luyu_gao for leading the charge!

13 62 262 54K 143

Download Image

I'm joining the UIUC @UofIllinois this fall as an Assistant Professor in the iSchool, with an affiliation in Computer Science! My research passion lies in the intersection of NLP and the medical domain. I'm recruiting students for 2025! Check more info: yueguo-50.github.io.

From Claude100K to Gemini10M, we are in the era of long context language models. Why and how a language model can utilize information at any input locations within long context? We discover retrieval heads, a special type of attention head responsible for long-context factuality

Want to train an aligned LM in a new language 🌏 but don’t have preference data for training the reward model (RM)? 💡 Just use a RM for another language: it often works well, sometimes even BETTER than if you had a RM in your target language! 🤯 arxiv.org/abs/2404.12318

SWE-agent is our new system for autonomously solving issues in GitHub repos. It gets similar accuracy to Devin on SWE-bench, takes 93 seconds on avg + it's open source! We designed a new agent-computer interface to make it easy for GPT-4 to edit+run code github.com/princeton-nlp/…

Very proud of Eurus. A huge shoutout to @lifan__yuan and @charlesfornlp for leading this!

Lifan Yuan @lifan__yuan

a year ago

Very proud of Eurus. A huge shoutout to @lifan__yuan and @charlesfornlp for leading this!

7 61 316 65K 228

Download Image

Very proud of Eurus. A huge shoutout to @lifan__yuan and @charlesfornlp for leading this!

Lifan Yuan @lifan__yuan

a year ago

Very proud of Eurus. A huge shoutout to @lifan__yuan and @charlesfornlp for leading this!

1 0 12 2K 0

Frontier models all have at least 100k context length, Gemini 1.5 has even 1m context. What about research and open source? Introducing Long Context Data Engineering, a data driven method achieving the first 128k context open source model matching GPT4-level Needle in a…

Large Language Model (LLM) agents promise to free us from mundane tasks, but how should they best interact with our world? Introducing CodeAct, an agent {framework, instruction-tuning dataset, model}, employs executable Python code to unify the actions of LLM agents. 🧵1/