Second Year CS Ph.D. student at Princeton University (@princeton_nlp), previously CS undergrad at IIT Bombayadithyabh.github.io Princeton, NJJoined June 2023
Are AI scientists already better than human researchers?
We recruited 43 PhD students to spend 3 months executing research ideas proposed by an LLM agent vs human experts.
Main finding: LLM ideas result in worse projects than human ideas.
🤔 Recent mech interp work showed that retrieval heads can explain some long-context behavior. But can we use this insight for retrieval?
📣 Introducing QRHeads (query-focused retrieval heads) that enhance retrieval
Main contributions:
🔍 Better head detection: we find a…
Introducing COMPACT: COMPositional Atomic-to-complex Visual Capability Tuning, a data-efficient approach to improve multimodal models on complex visual tasks without scaling data volume. 📦
arxiv.org/abs/2504.21850
1/10
Want to train large vision-language models but drowning in data? arxiv.org/abs/2501.00654
Introducing ICONS - we demonstrate how to select only 20% of training samples while maintaining 98.6% of the performance, and 60% of training samples to achieve 102.1% of the performance.
Have you ever wondered why we don’t use multiple visual encoders for VideoLLMs? We thought the same!
Excited to announce our latest work MERV, on using Multiple Encoders for Representing Videos in VideoLLMs, outperforming prior works with the same data. 🧵
Past work observed that DPO often decreases the probability of preferred responses. So where does the probability go? 🧐
We investigate the causes for this counter-intuitive phenomenon and show that it can lead to surprising failures in alignment!
📰 arxiv.org/abs/2410.08847
🧵
I'll be at ACL 2024!
I'd love to chat with about interpretability, preference optimization, science of LM, or any NLP topics -- feel free to reach out!
Oh, and I'll present The Heuristic Core (arxiv.org/abs/2403.03942) both as an oral (Aug 13 10:30) and a poster (Aug 12 14:00).
How can we understand neural chatbots in terms of interpretable, symbolic mechanisms? To explore this question, we constructed a Transformer that implements the classic ELIZA chatbot algorithm (with @Abhishek_034 and @danqi_chen). Paper: arxiv.org/abs/2407.10949 (1/6)
My new blog post argues from first principles how length normalization in preference learning objectives (e.g., SimPO) can facilitate learning from model-annotated preference data. Check it out! cs.princeton.edu/~smalladi/blog…
85 Followers 228 FollowingResearch @ MATS, CS @ Princeton, working on multi-agent safety, long-context language models, and efficient inference techniques.
315 Followers 3K Following📎 Learning & Research: Deep Learning, Computational Protein Design, Protein Language Models.
📎 PhD Student at Drexel University.
📎 Becoming an avid reader.
558 Followers 7K FollowingFuturist philosophy, molec neuro/immuno, pathophys, software eng, AI enjoyer
Made an Apache/MIT `tree` util with tokens, lines, and module components
5K Followers 6K FollowingHTML & JS since 1999, PHP and Java since 2003, HLSL shaders since 2007, WebGL and glsl shaders since day one, university CS diploma, C#/.NET developer by trade.
10K Followers 2K FollowingCS PhD candidate @PrincetonCITP. I tweet about AI agents, AI evals, AI for science.
AI as Normal Technology: https://t.co/5amOkqKDf2
Book: https://t.co/DabpkhNrcM
50K Followers 3K FollowingAI alignment + LLMs at Anthropic. On leave from NYU. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
14K Followers 519 FollowingAsst. Prof. of CS at Stanford, Google DeepMind. Prev: Anthropic, Google Brain. Co-Creator of MoEs, AlphaChip, Test Time Scaling Laws.
85 Followers 228 FollowingResearch @ MATS, CS @ Princeton, working on multi-agent safety, long-context language models, and efficient inference techniques.
3K Followers 1K FollowingResearch Engineering Lead at @StanfordCRFM. Previously co-founder at Semantic Machines ⟶ MSFT. Lead developer of Levanter and Marin @[email protected]