This is Providence before/after it removed the I-195 highway, which is one of the best I've seen.
It's not unrealistic to think Louisville and the Portland area could all look like this someday if we #8664.
At this point, you should stop using Adam in favor for Muon/Shampoo et al. For context, this was a vqgan run and the training was significantly more stable and had faster convergence
At this point, you should stop using Adam in favor for Muon/Shampoo et al. For context, this was a vqgan run and the training was significantly more stable and had faster convergence
New essay exploring why experts so strongly disagree about existential risk from ASI, and why focusing on alignment as a primary goal may be a fundamental mistake
The field of model collapse can largely be explained as "doing stupid & unrealistic things with synthetic data yields stupid & unrealistic outcomes"
But those papers are on the front cover of Nature and spotlights at ICLR and I'm on Arxiv. We are not the same 😵💫
3/3
nature cover - july 2024: synthetic data causes model collapse
nature cover - august 2021: synthetic data leads to improved accuracy in protein prediction (and wins nobel in 2024)
----
in other words: if your model collapses with synthetic data... it's probably a skill issue.
Isn't the Strong Model Collapse paper basically impossible to be correct since synthetic data is a huge part of frontier model training already?
> results show that even the smallest fraction of synthetic data (e.g., as little as 1% [...]) can still lead to model collapse
???
The online discussion seems to have largely missed this key point. The complaint wasn't "they didn't cite us wahhhh" but that prior work was selectively omitted to tell a misleading narrative that ANY amount of synthetic data can be toxic
1/3
The online discussion seems to have largely missed this key point. The complaint wasn't "they didn't cite us wahhhh" but that prior work was selectively omitted to tell a misleading narrative that ANY amount of synthetic data can be toxic
1/3
50K Followers 5K FollowingCofounder and Head of Post Training @NousResearch, prev @StabilityAI
Github: https://t.co/LZwHTUFwPq
HuggingFace: https://t.co/sN2FFU8PVE
4K Followers 991 FollowingI generate models that generate other stuff, working on @mancerlabs
-- Prev: Founder of Popgun Labs (Techstars), Founder of the @QUTCode Network.
985 Followers 905 Followingacc/e instead of e/acc.
Accelerate everything.
AGI will come from video games.
The best video games will come from FOSS.
A = B = C
AGI will come from FOSS.
1K Followers 892 FollowingInterested in things that generalize. SuperIntelligence @Meta, Prev: Science of Scaling co-TL @GoogleDeepmind. PhD Student at UC Berkeley. 🇺🇸🇨🇦
46K Followers 2K FollowingSenior correspondent covering AI @WIRED • Subscribe to my newsletter https://t.co/jxLAFHz8UP • Robison (rah-beh-son) not Robinson • Send tips on Signal @ kylie.01
1.2M Followers 518 FollowingHighlighting Politicians' trades so we can invest alongside Goal: get them banned from trading. $800,000,000 invested on @joinautopilot_ so far
24K Followers 706 FollowingMember of the technical staff @ Anthropic. Most (in)famous for inventing diffusion models. AI + physics + neuroscience + dynamics.
29K Followers 4K FollowingGTM at @OpenAI. A fan of NY sports, memes, tech & nice people. My opinions are my own & I will never intentionally leak info. 1-800-CHATGPT for a good time.
798 Followers 790 Followingresearch @openai || ESM3 @evoscaleai || Princeton PhD || MIT BS/MEng || former ai resident @google, intern @nvidia || Bay Area native
125K Followers 36 FollowingSharing the cringe that is LinkedIn | Follow @jovvvian THE LinkedIn Lunatic | post unhinged memes on LinkedIn and go viral https://t.co/z7o93BEBb0
8K Followers 679 FollowingPhD student @MIT • Research on Generative Models and Geometric Deep Learning for Biophysics • BA @CambridgeUni • Former @TwitterResearch, @DEShawGroup and @IBM
243 Followers 442 FollowingProduct leadership for Google Cloud AI/ML. Plays a pivotal role in shaping the future of AI acceleration with a focus on specialized AI infrastructure (TPUs).
61K Followers 3K FollowingCo-founder and GP @coalitionvc, investing $500K in early stage startups. Former comms pro @Glossier, @SocialCapital, @Box. Co-host of @greatchatpod.
2K Followers 1K FollowingResearch at @OpenAI; Reinforcement Learning; PhD from UT Austin. Previously FAIR Paris @AIatMeta, @CMU_Robotics @NVIDIAAI @UberATG.