Super exciting that my former student @edward_milsom is starting his lab at the University of Bath. Ed has led super-fundamental work on representation learning and learning dynamics. And he’s great fun to work with, so I definitely recommend collaborating!
Super exciting that my former student @edward_milsom is starting his lab at the University of Bath. Ed has led super-fundamental work on representation learning and learning dynamics. And he’s great fun to work with, so I definitely recommend collaborating!
Excited to announce I'll be starting this September 2025 as a Lecturer (Assistant Professor) at the University of Bath! I will continue my research on deep learning foundations, and am open to ideas for collaborations. (Pictured: Bath. Not pictured: University of Bath)
I talked to a lot of people about "a weight decay paper from Wang and Aitchison" at ICLR, which is officially been accepted at #ICML2025 . Laurence summarized the stuff in our paper in the post, here I will talk about the connection with a *broad* collection of existing works 1/
I talked to a lot of people about "a weight decay paper from Wang and Aitchison" at ICLR, which is officially been accepted at #ICML2025 . Laurence summarized the stuff in our paper in the post, here I will talk about the connection with a *broad* collection of existing works 1/
I'm at #ICLR, presenting our work on multi-layer SAEs for language-model interpretability tomorrow (Sat 26 Apr) from 10AM at Hall 3 + Hall 2B #519: iclr.cc/virtual/2025/p…
#ICLR2025 I will hold two talks on KBLaM (my internship project at MSR Cambridge w/ @jameshensman) at Microsoft’s booth: Thursday and Saturday 4 - 4:30,
As well as a poster at Poster Session 5 on Saturday morning.!
#ICLR2025 I will hold two talks on KBLaM (my internship project at MSR Cambridge w/ @jameshensman) at Microsoft’s booth: Thursday and Saturday 4 - 4:30,
As well as a poster at Poster Session 5 on Saturday morning.!
Second, we trained SAEs on transformers with randomized parameters, finding that auto-interpretability scores do not always distinguish them from trained models. This underscores the difficulty of automating feature interpretation and the importance of appropriate baselines! 3/
There's a lot to process here, but I was pleased to see that Anthropic's 'Circuit Tracing' paper cites three of our recent contributions to the interpretability literature! 1/
There's a lot to process here, but I was pleased to see that Anthropic's 'Circuit Tracing' paper cites three of our recent contributions to the interpretability literature! 1/
Really happy to have this paper out on arXiv! Scalable GPU-based Bayesian inference for hierarchical models without requiring gradients wrt model parameters (unlike e.g. VI).
arxiv.org/abs/2503.08264
Really happy to have this paper out on arXiv! Scalable GPU-based Bayesian inference for hierarchical models without requiring gradients wrt model parameters (unlike e.g. VI).
arxiv.org/abs/2503.08264
Our paper on the best way to add error bars to LLM evals is on arXiv! TL;DR: Avoid the Central Limit Theorem -- there are better, simple Bayesian (and frequentist!) methods you should be using instead. Super lightweight library: github.com/sambowyer/baye… 🧵👇
🚨NEW PAPER ALERT 🚨
SAEs can give us insight into the representations of LLMs. But what about the LLMs' computations?
If we want to understand LLMs, we don't just need sparse SAE activations, but also a sparse computational graph connecting them.
So how do we get them? A 🧵
Our paper "Function-Space Learning Rates" is on arXiv! We give an efficient way to estimate the magnitude of changes to NN outputs caused by a particular weight update. We analyse optimiser dynamics in function space, and enable hyperparameter transfer with our scheme FLeRM! 🧵👇
4K Followers 6K Following@InstituteGC (Science & Tech Policy). ex. UK Govt Sovereign AI Unit Advisor. AI Policy. State Capacity. R&D. Bridging tech and policy worlds @txp_io
1K Followers 371 FollowingCarpe espresso ☕
Centre for AI FUNdamentals
Department of Computer Science
The University of Manchester
Deep Thinker.
Posts/reposts might be non-deep.
620 Followers 1K FollowingPhD,CEng | @turinginst Research Fellow | opinions may not reflect those of my employer | Decisions under uncertainty #JuliaLang 🔴🟢🟣| #FinoAllaFine ⚪⚫️ | 🇪🇺
4K Followers 6K FollowingMathematician (algebraic geometry, motives & friends, singularities in statistics and ML).
'Geometry is successful magic' (R. Thom)
University of Amsterdam.
6K Followers 3K FollowingAbsurdist humor, AI stuff, recsys, machine learning, mythology, meditation. Tech Emmy & book author. Work on robots. eigenhector on bsky and subst4ck
459 Followers 751 FollowingSenior Lecturer in Economics @unimelb. Working on developing new pedagogical tools and techniques. Teaching Macro, Finance, Time Series & Metrics. NYU alum. 🌈
2K Followers 883 FollowingSr Research Scientist @GoogleDeepMind, Thinking with Gemini ♊ | Polyglot (5) | ex-@TU_Muenchen @LMU_Muenchen - Opinions are my own
9K Followers 422 FollowingLover of birds and beards || መፍቀሬ ሥዕላት || Ground Hornbill Artist fan || descriptions are sometimes silly || no politics, just love
"ሠዐሉ፡ ለነ፡ ቅዱሳን፡"
8K Followers 124 FollowingSeminar series on machine learning for protein engineering. Details for seminars and slack are on our website. DM us if you're interested in presenting!
4K Followers 893 FollowingMolecular cell biology and statistics, mostly scRNA-seq in the immune system. Principal Scientist at Tahoe Tx. Some photography as well.
9K Followers 998 FollowingDeveloper of next-generation interactive entertainment experiences born from infinite wisdom.
Currently working on Kowloon's Curse.
4K Followers 461 FollowingFollow for AI in Digital Biology and Drug Discovery @NVIDIA, ex Insilico Medicine, ex Yale, PhD UMaryland, views are mine, DM for collabs
457 Followers 401 FollowingResearch scientist at ByteDance Seed. I do ml for human language & protein. Recent: Diffusion & LLMs for generative protein modeling. | Opinions are my own.
855 Followers 363 FollowingML x Chemistry research @valence_ai | Creating Machine Learning Algorithms for Science | Opinions my own | 🦋 https://t.co/g8iq4GoIa6
2K Followers 2K FollowingJunior fellow at the Society of Fellows at @Harvard and @iaifi_news fellow, incoming Assistant Professor at @Harvard and the @KempnerInst views my own
1K Followers 1K FollowingBiologist that navigates in the oceans of diversity through the space-time |
MSc in Biochem/Bioinfo @ibt_unam 🇲🇽 |
Protein evo, metagenomics & AI/ML/DL
6K Followers 2K FollowingModelling and simulation using quantum chemistry, stat mech and neural network potentials #compchem #theochem Researcher at Orbital Materials
4K Followers 2K FollowingSenior Research Scientist @IsomorphicLabs building AI for drug discovery. Prev: CS PhD @berkeley_ai, @PrescientDesign, @insitro, @UofT. 🇨🇦
2K Followers 4 Followingmolecular design and simulation tools for scientists. (also see our literature-posting account @RowanReads.)
(cover image from @owl_posting)
13K Followers 716 FollowingMaking a brand new game for Sony PlayStation 1.
I also post stuff there: https://t.co/uN5dT3c6Yt
Main website: https://t.co/gT9thpM66f
9K Followers 20 FollowingAdvancing humanity's understanding of AI through interpretability research. Building the future of safe and powerful AI systems.