Shashank Gupta @shashank_bits

Researcher at Ai2 || Work on NLP, LLMs, Reasoning, Agents, AI4Code || Prev: Microsoft AI, Univ. of Illinois (UIUC), Max Planck, IIT-Bombay || @shashanknlp 🟦sky shashankgupta.info Seattle, WA Joined December 2010

Tweets

604
Followers

472
Following

1K
Likes

6K

Ai2 @allen_ai

2 weeks ago

As part of Asta, our initiative to accelerate science with trustworthy AI agents, we built AstaBench—the first comprehensive benchmark to compare them. ⚖️

3 14 103 9K 38

Download Image

Ai2 @allen_ai

2 weeks ago

Introducing Asta—our bold initiative to accelerate science with trustworthy, capable agents, benchmarks, & developer resources that bring clarity to the landscape of scientific AI + agents. 🧵

9 48 217 293K 97

Download Video

Vaish Shrivastava @VaishShrivas

3 weeks ago

Test-time scaling w/ GRPO boosts accuracy, but also adds “filler tokens” increasing length w/o real progress. We present Group Filtered Policy Optimization (GFPO):🧵 1️⃣ Sample more per prompt 2️⃣ Rank by token efficiency (reward ÷ length) 3️⃣ Train on top-k 4️⃣ 🚀 Cut 80% of…

4 48 331 58K 278

Download Image

Ai2 @allen_ai

3 weeks ago

With fresh support of $75M from @NSF and $77M from @nvidia, we’re set to scale our open model ecosystem, bolster the infrastructure behind it, and fast‑track reproducible AI research to unlock the next wave of scientific discovery. 💡

34 74 735 271K 89

Download Image

Dimitris Papailiopoulos @DimitrisPapail

3 weeks ago

Thinking Less at test-time requires Sampling More at training-time! GFPO is a new, cool, and simple Policy Opt algorithm is coming to your RL Gym tonite, led by @VaishShrivas and our MSR group: Group Filtered PO (GFPO) trades off training-time with test-time compute, in order…

19 41 361 93K 273

Download Image

Ai2 @allen_ai

2 months ago

🚨 We're hiring a #ResearchScientist in #AI for Scientific Discovery at Ai2! Are you passionate about intelligent agents, data-driven discovery, and AI systems that accelerate science? Join us in shaping the future of research. 🧬🧠 Apply now: job-boards.greenhouse.io/thealleninstit…

0 8 50 5K 12

Download Image

Ai2 @allen_ai

2 months ago

Introducing SciArena, a platform for benchmarking models across scientific literature tasks. Inspired by Chatbot Arena, SciArena applies a crowdsourced LLM evaluation approach to the scientific domain. 🧵

12 64 408 71K 210

Download Image

Ai2 @allen_ai

2 months ago

Today we’re releasing a prototype of Genesys, an autonomous multi-agent LLM discovery system that aims to discover new types of language model architectures. We found Genesys can discover novel architectures competitive with the industry-standard transformer. 🧵

5 36 251 21K 135

Download Image

Unnat Jain @unnatjain2010

3 months ago

✨New edition of our community-building workshop series!✨ Tomorrow at @CVPR, we invite speakers to share their stories, values, and approaches for navigating a crowded and evolving field, especially for early-career researchers. Cheeky title🤭: How to Stand Out in the…

Anand Bhattad @anand_bhattad

3 months ago

6 19 146 111K 74

Download Image

3 15 66 12K 15

Pushmeet Kohli @pushmeet

4 months ago

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in @GoogleDeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

49 337 2K 201K 930

Download Video

Greg Brockman @gdb

4 months ago

We've just released HealthBench — a new eval for AI systems for health. Developed with 262 physicians who have practiced in 60 countries.

OpenAI @OpenAI

4 months ago

We've just released HealthBench — a new eval for AI systems for health. Developed with 262 physicians who have practiced in 60 countries.

179 490 4K 2.1M 1K

71 206 2K 320K 641

Parshin Shojaee @ParshinShojaee

5 months ago

Scientific discovery with LLMs has so much potential yet is underexplored. Our new benchmark **LLM-SRBench** enable rigorous evaluations of equation discovery with LLMs! 🧠Key takeaway: Even SOTA discovery models with strong LLM backbones still fail to discover mathematical…

4 32 199 37K 117

Download Image

Naman Jain @StringChaos

5 months ago

Excited to release R2E-Gym - 🔥 8.1K executable environments using synthetic data - 🧠 Hybrid verifiers for enhanced inference-time scaling - 📈 51% success-rate on the SWE-Bench Verified - 🤗 Open Source Data + Models + Trajectories 1/

15 63 257 45K 115

Download Image

Ai2 @allen_ai

6 months ago

Announcing OLMo 2 32B: the first fully open model to beat GPT 3.5 & GPT-4o mini on a suite of popular, multi-skill benchmarks. Comparable to best open-weight models, but a fraction of training compute. When you have a good recipe, ✨ magical things happen when you scale it up!

29 161 671 90K 220

Download Image

Ai2 @allen_ai

7 months ago

Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on…

153 378 2K 446K 678

Download Image

Devi Parikh @deviparikh

7 months ago

Excited to share a sneak peek into what we've been building at Yutori! What you see below is our trained model and internal prototype — multiple agents running in parallel in the background, completing tasks of varying complexity, relevant information and cues to step in being…

45 52 465 220K 187

Download Video

Nouha Dziri @nouhadziri

8 months ago

Interested in knowing more about LLMs agents and in contributing to this topic?🚀 📢We're thrilled to announce REALM: The first Workshop for Research on Agent Language Models 🤖 #ACL2025NLP in Vienna 🎻 We have an exciting lineup of speakers 🗓️ Submit your work by *March 1st*

2 36 173 38K 58

Download Image

Ai2 @allen_ai

8 months ago

Can AI really help with literature reviews? 🧐 Meet Ai2 ScholarQA, an experimental solution that allows you to ask questions that require multiple scientific papers to answer. It gives more in-depth, detailed, and contextual answers with table comparisons, expandable sections…

14 74 221 41K 134

Download Image

Shashank Gupta @shashank_bits

8 months ago

Excited to share that LLM-SR will appear at ICLR'25 🥳🎉🇸🇬

Parshin Shojaee @ParshinShojaee

a year ago

Excited to share that LLM-SR will appear at ICLR'25 🥳🎉🇸🇬

1 8 75 32K 35

Download Gif

0 1 12 947 0

Sayash Kapoor @sayashk

8 months ago

How expensive are the best SWE-Bench agents? Do reasoning models outperform language models? Can we trust agent evaluations? 📢 Announcing HAL, a Holistic Agent Leaderboard for evaluating AI agents, with 11 benchmarks, 90+ agents, and many more to come.