Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

ML Engineer @FactoredAI. Prev @Columbia Physics. Code and funnies please garysnotebook.com Medellín, Colombia Joined February 2019

Tweets

287
Followers

49
Following

454
Likes

3K

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

2 days ago

Tip for debugging with LLM: instead of asking it to just stare at the code to guess what’s wrong, ask it to create a script to print out everything you need to diagnose the issue and then give it the output. Suddenly your LLM becomes 10x better at debugging.

0 0 1 49 0

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

a year ago

One consequence of much faster internet may be that LLMs will move more and more to the frontend (browser) — better privacy, UX and no serving cost. WebGPU is already working pretty well and one major bottleneck (other than storage) is speed to download these giant models.

0 0 0 96 0

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

a year ago

Been contemplating the fact that in LLMs, the outlier values are often the most important values. If you remove these outliers, your model performance degrades drastically. It's quite poetic and affirms my belief that "the average is overrated".

0 0 0 83 0

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

2 years ago

One cool thing from Gemini’s technical report no one is talking about is that it can take audio signal natively (as opposed to converting audio to text). This means it can potentially capture speech tones?

0 0 0 96 0

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

3 years ago

LLMs are to small deep learning models what small deep learning models are to hard-code logic

0 0 3 146 0

Yizhong Wang @yizhongwyz

3 years ago

Is it possible to solve NLP tasks by simply following instructions that define the tasks? How can we measure the progress? Excited to announce Natural Instructions v2, a collection of 1600+ diverse language tasks and their expert-written instructions! 📜arxiv.org/abs/2204.07705

4 80 361 0 92

Horace He @cHHillee

4 years ago

Everybody wants their models to run faster. However, researchers often cargo cult performance without a solid understanding on the underlying principles. To address that, I wrote a post called "Making Deep Learning Go Brrrr From First Principles". (1/3) horace.io/brrr_intro.html

26 392 2K 0 977

Download Image

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

4 years ago

Was wondering why these model checkpoints files are so big (~GB). Aren’t they just a bunch of floats (~4 bytes each)? Then realized roberta-large is 355M parameters 🤯🤯🤯

0 0 2 0 0

Gary’s Notebook 🧙‍♂️ @gary_doesnt_lai

4 years ago

github.com/tonsky/FiraCode @FiraCode thank you for this font. Immediately noticing a reduction in cognitive load after switching my text editor to it!

0 0 2 0 0

Cristian Garcia @cgarciae88

4 years ago

We need a dedicated collection of Toy Datasets for Machine Learning: 1. They can be more interesting than real datasets, specially if designed to be hard for certain algorithms. 2. They are more useful for teaching / learning. Maybe @huggingface / @kaggle can help with this?

10 9 102 0 24

Eric Jang @ericjang11

4 years ago

Got hit-and-run by a white work van on Atlantic and Hellman on 9/24, 7:21pm. Neck is a bit sore but otherwise I'm ok. I'm offering $1k cash or $2k to charity of your choice for first person to send me dash cam footage clearly showing the van's license plate (ends in 5G)

7 24 105 0 1

Download Image

Ethan Caballero is busy @ethanCaballero

4 years ago

Stanford's ~entire AI Department has just released a 200 page 100 author Neural Scaling Laws Manifesto. They're pivoting to positioning themselves as #1 at academic ML Scaling (e.g. GPT-4) research. "On the Opportunities and Risks of Foundation Models" arxiv.org/abs/2108.07258