Researcher focusing on LLMs: https://t.co/iVZDFdIQiE Previously, dev tools and infra for ML. Ex @Github, @Airbnb. @fastdotai core contributor.hamel.dev Portland, ORJoined September 2012
It's @huggingface Accelerate release time and there are a TONof exciting features to get through: new optimizers, FP8 fixes, DataLoader improvements, documentation, and so much more!
For a quickread, check out the full notes: github.com/huggingface/ac…
Otherwise let's dig in🧵
Maven's top AI course just added a ton of new guest speakers.
Incredible talent convening on Maven to teach LLM Fine-Tuning:
- Wing Lian: Creator of Axolotl library for LLM fine-tuning
- Shreya Shankar: LLMOps and LLM Evaluations researcher
- Zachary Mueller: Lead maintainer…
Maven's top AI course just added a ton of new guest speakers.
Incredible talent convening on Maven to teach LLM Fine-Tuning:
- Wing Lian: Creator of Axolotl library for LLM fine-tuning
- Shreya Shankar: LLMOps and LLM Evaluations researcher
- Zachary Mueller: Lead maintainer…
brushed up my personal site and brain dumped a post on 🎶musicgen-songstarter-v0.2🎶
It covers:
- 🧠my thought process/motivation behind it
- ✏️notes on my previous experiments over the last 9 months
- 👀 training deets, @weights_biases logs w/ hparams
nateraw.com/posts/training…
Classic example of overfitting to the validation set re: LLMs, when I started working with @_cartermp I found few-shot examples from the validation set in the prompt (we fixed it!).
There are lots of reasons for a separate eval set. Overfitting can come in many forms.
Has someone created materials around “fundamentals of ML for AI Engineers”, not focused on building models but things like evaluations, error analysis, etc
Maybe something already exists? I don’t want to do it lol - looking for a resource I can share with people
📺Tune in next week as @rasbt and I riff on "Developing and Training LLMs From Scratch" in a live podcast recording for @VanishingData 💫
lu.ma/build-llms-fro…
This will likely be a sprawling convo in which we tell you everything you need to know about LLMs, but were too…
NVIDIA has just added CUDA checkpointing functionality via: github.com/NVIDIA/cuda-ch…
which should allow CRIU to do application-level checkpointing, that includes GPU state save/restore.
Thank you for addressing this long-outstanding request, @NVIDIAAI
Discovered via this…
I’ve tried 7+ AI note takers and my favorite one by far is @circlebackai
Here is some code I have been playing with to automate the tedious process of writing up consulting proposals based on meeting summaries using circleback webooks + @modal_labsgist.github.com/hamelsmu/ac72d…
I’m getting lots of questions about why this is a bad idea.
Repeatedly peeking at the validation set in the process optimizing anything makes that validation set very biased
It’s very bad hygiene to intermingle your validation and test/eval set. The consequences of this…
I’m getting lots of questions about why this is a bad idea.
Repeatedly peeking at the validation set in the process optimizing anything makes that validation set very biased
It’s very bad hygiene to intermingle your validation and test/eval set. The consequences of this…
Here's the 256k (262k) version built on OSS tools so that anyone can reproduce on their own. Trained using PoSE further extending our previous 64k version at the original RoPE theta. Per our previous experiments, I expect this should handle passkey retrieval up to 512k.
🤗Model:…
Here's the 256k (262k) version built on OSS tools so that anyone can reproduce on their own. Trained using PoSE further extending our previous 64k version at the original RoPE theta. Per our previous experiments, I expect this should handle passkey retrieval up to 512k.
🤗Model:…
Happy to say that @huggingface accelerate has hit 100 MILLION downloads today!
It's been so much fun enabling so many users to have their code just run on any system with as minimal friction as possible. Here's to 200M 🚀🚀🚀
Finetuning Embeddings:
Most people don't know that if you had any production-ready data, you should be able to fine-tune and outperform OpenAI.
1. With even 2,000 examples, you can fine-tune an embedding.
2. By using the Hugging Face Inference Server and Modal Labs, we showed…
There's a new bill, SB-1047 "Safe and Secure Innovation for Frontier Artificial Intelligence Models Act".
I think it could do a great deal of harm to startups, American innovation, open source, and safety. So I've written a response to the authors: 🧵
answer.ai/posts/2024-04-…
267K Followers 885 FollowingMachine learning & AI researcher writing at https://t.co/A0tXWzG1p5. LLM research engineer @LightningAI. Previously stats professor at UW-Madison.
25K Followers 554 FollowingResources to take your Machine Learning skills to the next level
🧪 Senior Data Scientist, RecSys @NVIDIAAI
🏫 @fastdotai trained DL Eng
📝 https://t.co/By87iXx5Pu
38K Followers 3K FollowingBuilding @modal_labs when I'm not posting bangers about data and software. Previously built the music rec sys at Spotify and ran the eng team at Better.
55K Followers 1K FollowingPhD at 19 |
Founder and CEO at @MedARC_AI |
Research Director at @StabilityAI |
@kaggle Notebooks GM |
Biomed. engineer @ 14 |
TEDx talk➡https://t.co/xPxwKTq6Qb
10K Followers 395 Following🤗 Technical Lead for the Accelerate Project | Passionate about Open Source | Nerd who enjoys touching the grass | #ADHD | He/Him
59K Followers 2K Following✨Keep it simple, make it scale. AI should be about empowering people, building understanding, & making dreams realities. 👩💻GenAI @GoogleDeepMind ex-@GitHub
47K Followers 1K FollowingCo-founder and CTO at @huggingface 🤗. ML/AI for everyone, building products to propel communities fwd. @Stanford + @Polytechnique
48K Followers 2K FollowingChief AI & Co-founder @AnacondaInc; invented @pyscript_dev, @PyData @Bokeh @Datashader. Former physicist. A student of the human condition. bsky: @wang.social
35K Followers 1K FollowingMachine learning and language models R&D. Builder. Writer. Visualizing AI, ML, and LLMs one concept at a time. @Cohere. https://t.co/TquuQXlLOJ
62K Followers 16K FollowingNewsletter exploring AI & ML
- Weekly trends
- LLM/FM insights
- Unicorn spotlights
- Global dynamics
- History
Led by @kseniase_
Elevate your AI game 👇🏼
119 Followers 147 FollowingTechnical Founder of Stealth AI. PhD (CS, NLP and Knowledge Graph Embeddings). MLE for 8+ years. Most interested in decision making under uncertainty
3K Followers 1K FollowingFounder of @slitehq, we put knowledge management on autopilot 🛰️
Also building AskX, 1 entry point to all your team knowledge 🔮
148 Followers 2K FollowingML expert, tech solution architect. Built NLP and Computer Vision based SaaS enterprise products.
Currently building something in Gen AI, SaaS and next gen IT
162 Followers 335 Followingtweeting about AI, human progress, and exponential tech while traveling | techno-optimist | my newsletter: https://t.co/GXZorFToFo | 🌱 DM Open
67 Followers 405 FollowingSecurity Product Management @ Microsoft | Ex-Amazon | Passionate about Security, XAI, Privacy-Preserving ML, and Humanity! All views are mine.
482 Followers 1K FollowingFounder of Elephant in the Room. Previously: SE @GetYourStake, SE @Workday, PM @Trufl_App, MD @USCLavaLab, SE Zuma AI. CSBA @USC.
295 Followers 676 Followingtrying to photograph airplanes. trying to relax at the beach. trying to Take Pictures with Smartphone. cant See airplane on Smartphone Screen. Love you. #ai #py
267K Followers 885 FollowingMachine learning & AI researcher writing at https://t.co/A0tXWzG1p5. LLM research engineer @LightningAI. Previously stats professor at UW-Madison.
25K Followers 554 FollowingResources to take your Machine Learning skills to the next level
🧪 Senior Data Scientist, RecSys @NVIDIAAI
🏫 @fastdotai trained DL Eng
📝 https://t.co/By87iXx5Pu
38K Followers 3K FollowingBuilding @modal_labs when I'm not posting bangers about data and software. Previously built the music rec sys at Spotify and ran the eng team at Better.
187K Followers 887 FollowingCofounded and lead @PyTorch at Meta.
Also dabble in robotics at NYU.
AI is delicious when it is accessible and open-source.
380K Followers 77 FollowingTensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundation
55K Followers 1K FollowingPhD at 19 |
Founder and CEO at @MedARC_AI |
Research Director at @StabilityAI |
@kaggle Notebooks GM |
Biomed. engineer @ 14 |
TEDx talk➡https://t.co/xPxwKTq6Qb
10K Followers 395 Following🤗 Technical Lead for the Accelerate Project | Passionate about Open Source | Nerd who enjoys touching the grass | #ADHD | He/Him
59K Followers 2K Following✨Keep it simple, make it scale. AI should be about empowering people, building understanding, & making dreams realities. 👩💻GenAI @GoogleDeepMind ex-@GitHub
3K Followers 345 FollowingCreator of the OpenWebText and OpenGPT2. @PyTorch Core Reviewer. PhD Student at @Cornell (interning at @MosaicML) Previously at @FacebookAI and @BrownUniversity
675 Followers 477 FollowingStatistician. Creator of Chrome extensions "XCoach" for timed @X sessions and daily stats, and "TextLinks" for mnemonic URL bar quick links. Free in the store.
720 Followers 343 Followingbuilding @circlebackai (yc w24). previously built things @stripe and @twitter, a campervan, https://t.co/IaKhUNj4yx, watchai.
2K Followers 982 FollowingCo-Founder at Phonic. Previously @Stanford CS PhD Dropout, @MosaicML, CS @MIT. I tend to be wrong, but the learning process makes it enjoyable. 🇵🇰🇺🇲
2K Followers 652 FollowingA 501(c)(3) shared community space promoting and encouraging technical, scientific and artistic skills through individual projects, collaboration and education.
463 Followers 1K FollowingPast: Data+ML @lyft and Slew of Things in 🇺🇸🇮🇱🇨🇳. Alum @wharton @HopkinsEngineer @Yale Please consider buying my data kthx @procurefyi @frontier_optic