Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
Author's Explanation:
x.com/itayoush/statu…
Overview:
Puzzle introduces a distillation-based neural architecture search framework that significantly optimizes LLM inference on specific hardware, achieving a 2.17x…
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
Author's Explanation:
x.com/itayoush/statu…
Overview:
Puzzle introduces a distillation-based neural architecture search framework that significantly optimizes LLM inference on specific hardware, achieving a 2.17x… https://t.co/MWHT8TKfbw
🚀 @NeurIPSConf Spotlight! 🥳 Imagine fine-tuning an LLM with just a sparsity mask! In our latest work, we freeze the LLM and use 2:4 structured sparsity to learn binary masks for each linear layer. Thanks to NVIDIA Ampere’s 2:4 sparsity, we can achieve up to 2x compute…
We're launching EnIGMA, our state-of-the-art AI agent for offensive cybersec!
It uses tools like Ghidra & pwntools, can debug, connect to servers, and exploit vulnerabilities to solve CTF challenges.
Built with researchers from Princeton, NYU, and TAU.
enigma-agent.github.io
🚀 Exciting news! We’ve just released a new LLM:
Llama-3.1-Nemotron-51B = LLaMa-70B-Instruct + Block Distillation + NAS + Logics Distillation;
Powered by a single H100 GPU with nearly the same accuracy! ⚡ This gives a 2.2x inference speed-up with MT Bench 8.99 ➡️ 8.94.…
📢 New Benchmark: SUPER for Setting UP and Executing tasks from Research repositories
Reproducibility is crucial in science. We introduce SUPER to evaluate LLMs' capabilities in autonomously running experiments from research repositories. ⬇️
arxiv.org/pdf/2409.07440
🚀 Our team is hiring! Join to Advance Efficiency in Deep Learning at NVIDIA! 🚀
🔗 Apply here: bit.ly/nvdler-job
Our team, Deep Learning Efficiency Research (nv-dler.github.io) at NVIDIA Research, is about a year old, and we are expanding. We're looking for…
🌟 The best 8B Base model via pruning and distillation!
🚀 Introducing Mistral-NeMo-Minitron-8B-Base model we derived from the recent Mistral-NeMo-12B.
Our recipe: finetune teacher on 100B tokens, prune to 8B params, run teacher-student distillation on <400B tokens.
Result: the…
741 Followers 1K FollowingLong document understanding, Multilingual Evals and efficient models mainly, but other #NLProc applications in free time | vim enthusiast
2K Followers 1K FollowingResearch Scientist @Apple MLR on #machine_learning understanding and robustness. @ELLISforEurope member. Previously at ServiceNow and Element AI in Montréal.
861 Followers 411 FollowingSr. Deep Learning Research Engineer @NVIDIAAI. MSCS'18 @UICCS. Multi-domain Deep Learning researcher and library developer. All opinions are my own.
1K Followers 516 FollowingHi, I am a PI at ELLIS Institute Tübingen and MPI-IS. Was RS NIF @UniofOxford, JRF @SomervilleOx, postdoc @UTAustin, and PhD @Data_AI_TUe.
1K Followers 7K FollowingI'm a technical imagineer:
If MyBrain ideas + great Scientists meet, I & Scientists make New things, if MyBrain ideas can link(implant) Neuro into AI quantum 📩
6K Followers 1K FollowingResearch scientist at @GoogleDeepMind, working on generative models, deep learning, RL. PhD from @stanford. Gemini Diffusion lead.
4K Followers 2K FollowingTech 💙 People.
AI Lead @Shopify | FE Architect | https://t.co/OvoaBU5xlH | https://t.co/XqSWnmvTqM
Analog astronaut @Oewf space simulations 🚀
10K Followers 2K FollowingCS PhD candidate @PrincetonCITP. I tweet about AI agents, AI evals, AI for science.
AI as Normal Technology: https://t.co/5amOkqKDf2
Book: https://t.co/DabpkhNrcM
29K Followers 1K FollowingAI, national security, China. Part of the founding team at @CSETGeorgetown (opinions my own). Author of Rising Tide on substack: https://t.co/LKAoyL00iB
8K Followers 246 FollowingResearch @Meta Superintelligence Labs, I led Llama 2, built post-training from scratch. Also Toolformer, GAIA, Llama-3.0, CodeLlama, Galactica
2K Followers 330 FollowingAI Product Director @Meta (again) leading Generative AI open source ex-Google, ex @PyTorch leader ex-Amazon - Love building community around AI and Open Source.
2K Followers 205 FollowingResearch Scientist @GoogleDeepMind working on Gemini Thinking and post-training. Drove Gemini 2.5 Pro launch. Co-created Deep Think. PhD from @StanfordAILab.
7K Followers 103 FollowingResearch scientist at @openai working on AI agents and Deep Research. Co-creator of ChatGPT agent. Ex-@Stanford CS PhD. My words do not represent my employer's.
46K Followers 2K FollowingSenior correspondent covering AI @WIRED • Subscribe to my newsletter https://t.co/jxLAFHz8UP • Robison (rah-beh-son) not Robinson • Send tips on Signal @ kylie.01