We will be presenting Self-Distilled Sparse Drafters at #ICML this afternoon at the Efficient Systems for Foundation Models workshop. Come chat with us about acclerating speculative decoding! 🚀
@CerebrasSystems@UCalgaryML
I am excited to share that I will present two of my works at #ICML2025 workshops!
If you are interested in AI security and XAI, feel free to come by and chat - I'm looking forward to all feedbacks and discussions!
Excited to be presenting our work at ICML next week! If you're interested in loss landscapes, weight symmetries, or sparse training, come check out our poster — we'd love to chat.
📍 East Exhibition Hall A-B, #E-2106.
📅 Tue, July 15 | 🕚 11:00 a.m. – 1:30 p.m PDT.
Excited to be presenting our work at ICML next week! If you're interested in loss landscapes, weight symmetries, or sparse training, come check out our poster — we'd love to chat.
📍 East Exhibition Hall A-B, #E-2106.
📅 Tue, July 15 | 🕚 11:00 a.m. – 1:30 p.m PDT.
@UCalgaryML will be at #ICML2025 in Vancouver🇨🇦 next week: our lab has 6 different works being presented by 5 students across both the main conference & 6 workshops!
We are also recruiting fully-funded ML PhD/MSc students🧑🎓If attending, reach out to me on Whova app to meet 📅
Want faster #LLM inference without sacrificing accuracy? 🚀
Introducing SD², a novel method using Self-Distilled Sparse Drafters to accelerate speculative decoding. We show that self-data distillation combined #sparsity can be used to train low-latency, well-aligned drafters.🧵
🎉 Excited to share our latest work has been accepted at #ICML2025.
We explain how Lottery Ticket masks can be extended to arbitrary random initializations by leveraging permutations of weight symmetries achieving strong generalization.
Check out the details:…
🎉 Excited to share our latest work has been accepted at #ICML2025.
We explain how Lottery Ticket masks can be extended to arbitrary random initializations by leveraging permutations of weight symmetries achieving strong generalization.
Check out the details:…
✨Our paper is now officially published in Transactions on Machine Learning Research (TMLR)!
We explore how knowledge #distillation (KD) impacts fairness & bias in AI models, across both group and individual fairness.
I'm proud that the @UCalgaryML lab will have 6 different works being presented by 6 students across #NeurIPS2024, in workshops (@unireps, @WiMLworkshop, MusiML) and the main conference! 🎉
Hope to see you at our posters/talks 🤓, full schedule at calgaryml.com
🧵(1/4)
✨Our new @unireps paper tries to answer why the Lottery Ticket Hypothesis (LTH) fails to work for different random inits through the lens of weight-space symmetry.
We improve the transferability of LTH masks to new random inits leveraging weight symmetries.
🧵(1/6)
Happy to share our work on Knowledge Distillation temperature🌡️ and fairness⚖️. Must-read for AI practitioners given the prevalence of distilled models! KD is crucial in deploying AI, e.g. limited hardware, real-time applications
🙏@sarahookr@mikelasby for feedback! @UCalgaryML
Happy to share our work on Knowledge Distillation temperature🌡️ and fairness⚖️. Must-read for AI practitioners given the prevalence of distilled models! KD is crucial in deploying AI, e.g. limited hardware, real-time applications
🙏@sarahookr@mikelasby for feedback! @UCalgaryML
Knowledge #distillation is a widely used model compression method. We explore the nuanced impact of temperature on distilled models' #fairness ⚖️. Our findings show distilled students are less fair than their teachers at typical temps, but can be more fair in some instances…🧵👇
"Dynamic Sparse Training with Structured Sparsity" (openreview.net/forum?id=kOBkx…) was accepted at ICLR 2024! DST methods learn state-of-the-art sparse masks, but accelerating DNNs with unstructured masks is difficult. SRigL learns structured masks, improving real-world CPU/GPU timings
181 Followers 927 FollowingAbdul Rahman is from Gaza and is now displaced and the situation is very tragic. I have lost work and cannot provide food. I ask you to donate and help with the
801 Followers 696 FollowingSchulich Research Chair & Assistant Professor, Calgary ML Lab (@UCalgaryML), PhD U. Cambridge. Sparse Neural Networks/Efficient Deep Learning. 🇨🇦🇬🇧🇨🇾
2K Followers 673 FollowingResearch scientist at Google DeepMind. Ph.D. in Machine Learning from the University of Toronto / Vector Institute. @[email protected]
114 Followers 381 FollowingMachine learning researcher, engineer, & enthusiast | PhD Candidate at UofC | When I'm not at the keyboard, I'm outside hiking, biking, or skiing.
34K Followers 627 Following‘sharq-zada’: Struck by the East. Rumi translator & Sufi teacher.
My Rumi circle + Persian dinner in Manhattan:
https://t.co/wz7aMUAodA
50K Followers 3K FollowingDeveloper Experience Lead at @GoogleDeepMind
Building Gemini API, Gemma, AI Studio and more AI products. My views
ex-Chief Llama Officer @huggingface 🇵🇪🇲🇽
42K Followers 865 FollowingFR/US/GB AI/ML Person, Director of Research at @GoogleDeepMind, Honorary Professor at @UCL_DARK, @ELLISforEurope Fellow. All posts are personal.
58K Followers 619 FollowingDistinguished Professor (Emeritus), Oregon State Univ.; Former President, Assoc. for the Adv. of Artificial Intelligence; Robust AI & Comput. Sustainability
62K Followers 12K FollowingAI policy researcher, wife guy in training, fan of cute animals and sci-fi, Substack writer, stealth-ish non-profit co-founder
42K Followers 1K FollowingSecular Bayesian.
Professor of Machine Learning @Cambridge_CL. Talent aficionado at https://t.co/RbJkoLguey
Alum of @Twitter, Magic Pony and @Balderton
56K Followers 853 FollowingFiguring out AI @allen_ai, open models, RLHF, fine-tuning, etc
Contact via email.
Writes @interconnectsai
Wrote The RLHF Book
Mountain runner
1K Followers 601 FollowingLearning about robots learning. Researcher, Robotics at Google DeepMind. Co-author of Foundations of Deep Reinforcement Learning. https://t.co/JgfxI9VJAh
2K Followers 29 FollowingCo-Founder, CTO, @reflection_ai
DQN, AlphaGo, AlphaZero, MuZero, Gemini RLHF
Prev Senior Staff RS and founding eng @GoogleDeepMind
AGI one PR at a time
16K Followers 16K FollowingProfessor of AI @lborouniversity
Neural Information Processing, Retrieval & Modelling, NLP, LLM Learning & Unlearning, AI Ethics, AI in Health &Social Care
26K Followers 876 FollowingResearch Scientist Director in Meta FAIR. Reasoning, Optimization and Understanding LLM. Novelist in spare time. PhD in @CMU_Robotics.
287 Followers 305 FollowingAssistant professor of Machine Learning @mbzuai, Ph.D. @KAUST_News, former intern @metaAI, @samsungresearch, and @amazon. Co-organizer of @flow_seminar.
10K Followers 4K Followingsth new // ex Gemini RL+Inference @GoogleDeepMind // Chat AI @Meta // RL Agents @EA // ML+Information Theory @MIT+@Harvard+@GeorgiaTech // زن زندگی آزادی
31K Followers 877 FollowingVP GenAI @Databricks. Former CEO/cofounder MosaicML & Nervana/IntelAI. Neuro + CS. I like to build stuff that will eventually learn how to build other stuff.
67K Followers 1K Following🇵🇸✍🎙⚽Palestinian journalist and commentator, mainly football, from Deir al-Balah in Gaza covering Israel's war on the Gaza Strip
📩[email protected]