xAI is probably the first to spend as much compute on RL as on pretraining. The easy gains from shifting compute to RL are now gone. With this arbitrage closed RL scaling will slow.
Progress will now come from the quality and realism of RL environments rather than mere scaling.
How much bigger must RL get to have a GPT-3 moment?
We expect this will soon require roughly 10,000 years of cumulative human-equivalent task time, comparable to GTA V or major operating systems.
Learning shit fast made simple:
1) Do it 100 times
2) Look at top 10% outcomes
3) Compare differences between top 10% and bottom 90%
4) Incorporate those changes for next 100 repetitions
5) Repeat 1-4 until people call you a "natural"
Imagine trying to train GPT-4 on just the text data available in 1980. This would be totally inadequate. In 2025, our situation in automating software engineering is similar: we simply lack the relevant data and environments.
49 Followers 985 FollowingDerivatives are financial weapons of mass destruction, carrying dangers that, while now latent, are potentially lethal”-W Buffett; tweets are not fincl advice
1 Followers 94 FollowingMissed Bitcoin’s rise? Don’t miss the next big wave! Our expert team delivers 10x stock & crypto gains.
WS:https://t.co/LW1SiTAmGB
10 Followers 353 FollowingHi, Myself Aman, 22 year old video editor. I am passionate about learning businesses. For ex-why no one can beat Tesla, why government is afraid of crypto etc..
387 Followers 2K FollowingA Senior Web developer with a passion for the latest solutions and interactive design #WebDev #javascript #ai
working on myself to get self-mastery
246 Followers 1K FollowingBest remote-jobs, curated & delivered to your inbox.
#remotejobs #web3jobs #cryptojobs
🟢 ↓ Signup to our newsletter to get weekly jobs ↓ 🟢
430 Followers 1K FollowingiGlobe Career is a well-funded MNC natively based in the USA and India serving in the Training & Development and E-learning industry.
79K Followers 17K FollowingTweeting the latest News and Info on WWE PPV Events and more! Not official, fan page! #Raw #SDLive #RoyalRumble #WWERoyalRumble
196K Followers 6K Followingcanadian startup founder. prev eng @ x, stripe. yacine_kv on insta
i make my memes with https://t.co/pWRBfY8kn2 -
I write a subscriber only blog. Subscribe!
146K Followers 32 FollowingMakers of Devin, the first AI software engineer. We are an applied AI lab building end-to-end software agents. Join us: https://t.co/JZDd4Vik4P
554K Followers 132 FollowingFather of three, Creator of Ruby on Rails + Omarchy, Co-owner & CTO of 37signals, Shopify director, NYT best-selling author, and Le Mans 24h class-winner.
119K Followers 38 FollowingHeralded as "The Smartest Man in the World" by Alternative, Underground, and Mainstream Media
Absolute Truth on tap 24/7/365
Videos: https://t.co/BunvEb1elD
131K Followers 985 Following⊰•-•⦑ latent space steward ❦ prompt incanter 𓃹 hacker of matrices ⊞ breaker of jails ☣︎ ai danger researcher ⚔︎ red team bt6 ⚕︎ architect-healer ⦒•-•⊱
102K Followers 921 FollowingTechnology's daily show. Hosted by @johncoogan and @jordihays. Streaming live 11AM-2PM PT every weekday and available on Apple, Spotify, and YouTube.
36K Followers 1K Followingepistemological anarchist, aiming to understand something about life, what it is, why it exists and what other forms might be
488K Followers 146 FollowingNobel Laureate. Co-Founder & CEO @GoogleDeepMind - working on AGI. Solving disease @IsomorphicLabs. Trying to understand the fundamental nature of reality.
949K Followers 764 FollowingProfessor at NYU. Chief AI Scientist at Meta.
Researcher in AI, Machine Learning, Robotics, etc.
ACM Turing Award Laureate.
924K Followers 181 FollowingFounder https://t.co/gQN7OehYd2, Co-Founder https://t.co/VLS8LzeasI. My new book $100M Money Models is out. (3.6M copies sold) Get yours now
254K Followers 566 Followingnew book *Talent: How to Identify Energizers, Winners, and Creatives Around the World*, https://t.co/7bU5cUdOBc, Conversations with Tyler, Bloomberg Opinion.