in about 7min I’ll show you the most common deep learning research pattern like:
- simplification
- continuum
- merging
- scaling
- flat out stealing an idea
fun stuff check it out
"Linear Algebra for Machine Learning: From Scratch to Mastery" series:
there will be 40 lectures of around 1 hour each for linear algebra, syllabus in the comments.
Series Philosophy:
This lecture series is designed to be the definitive, one-stop resource for students,…
"Linear Algebra for Machine Learning: From Scratch to Mastery" series:
there will be 40 lectures of around 1 hour each for linear algebra, syllabus in the comments.
Series Philosophy:
This lecture series is designed to be the definitive, one-stop resource for students,… https://t.co/mzlkFALGPM
Sometime in Dec 2021, I got to talk to Ilya Sutskever, for my book WHY MACHINES LEARN (it was before the ChatGPT era; doubt I'd be able to do so now).
Ilya said this about the math of deep learning that he encountered in the first papers he read on the subject (given to him by…
This is how they vibe code at a FAANG:
>"get enough stakeholders to agree"
>"design review"
>"first few weeks of documentation"
>"PMs & TPMs breaking down tasks"
>3 months passed, then finally *vibe coding*!
What kind of hell is it to work like this?
they also dropped fsdp2 optimized muon. though they don't use muon for 2.6b dense model, i think it's just beginning and they are preparing larger one. they pipeline muon's comm-comp with calc flops and the code is neat. not sure if it's existing method.
huggingface.co/Motif-Technolo…
they also dropped fsdp2 optimized muon. though they don't use muon for 2.6b dense model, i think it's just beginning and they are preparing larger one. they pipeline muon's comm-comp with calc flops and the code is neat. not sure if it's existing method.
huggingface.co/Motif-Technolo… https://t.co/teujTtyin2
OpenAI took a bit of a detour, as we were the pioneers in AI coding research but didn’t prioritise it enough after ChatGPT took off
Knowing what research is cooking, I’m quite confident in our coding progress over the next year
OpenAI took a bit of a detour, as we were the pioneers in AI coding research but didn’t prioritise it enough after ChatGPT took off
Knowing what research is cooking, I’m quite confident in our coding progress over the next year
"You Are NOT Lazy, You Just Lack a Habit" is the first of 106 passages in Advice on Upskilling.
Here's the latest table of contents: https://t.co/mPwfj2jubR
Yesterday once more.
I was the first people to enable MacBook GPU training, getting it to run at about one-quarter the speed of a P100 in 2016–2017 for fine-tuning models. After that, it was all mediocre politics with no real technical vision. I developed PTSD and took half a…
Yesterday once more.
I was the first people to enable MacBook GPU training, getting it to run at about one-quarter the speed of a P100 in 2016–2017 for fine-tuning models. After that, it was all mediocre politics with no real technical vision. I developed PTSD and took half a…
@bingxu_ I was ambitious about making a change when joking Apple, excited for the last year’s announcement. Now I’m also taking a recover at msl, hoping that can really make a difference this time
cool method to learn math, physic or exercise heavy topic is the green, yellow, red method™️
in a nutshell it’s a iterative elimination method that focus the theoretical learning on getting exercise done.
as time goes on you focus almost exclusively on the hard problem.
When I was just getting started in ML two and a half years ago, I didn’t understand LeCunn’s points here and assumed eventually I’d get it.
Now I understand his points and think he is both obviously correct and completely missing the point.
I've written the full story of Attention Sinks — a technical deep-dive into how the mechanism was developed and how our research ended up being used in OpenAI's new OSS models.
For those interested in the details:
hanlab.mit.edu/blog/streaming…
this person wrote an entire unet training script in pure CUDA, i remember seeing this repo before i started learning about writing GPU kernels and being absolutely mind blown.
Reading through the @__tinygrad__ source with all the work they've done since the last time I looked and realising everything else is super bloated. Imagine needing pytorch and triton and accelerate and a thousand other flaky things etc, just make your own JIT nubs, it's like…
5 Followers 94 FollowingTo all my incredible fans—your love, your energy, your unwavering support mean the world to me. I see you, I feel you, and I love you more than words can say ❤️
490 Followers 2K FollowingBelieve in your dreams, even when they seem impossible. Embrace the journey, learn from setbacks, and celebrate every small victory along the way! 🎉💪
20K Followers 44 FollowingGlobal Stock markets related news, Companies Earnings, Global Economy Business, Economic events, Commodity, Political and War real-time news updates.
66 Followers 34 FollowingTrading, mostly swings and a few long-terms.
No investment advice to be found here; I got 0 qualifications to give any.
Barely surviving the market myself tbh!
3K Followers 840 Following40x #Salesforce Dev/Architect. Inventor of Dumpster__c. Always take the stage like it's the last day of your life. #TrailblazerCommunity #salesforceArchitect
4K Followers 643 FollowingChairman No Fallen Heroes Foundation | CEO | Navy TOPGUN Fighter Pilot | Keynote Speaker | Best Selling Author | MAX Afterburner Pod | Sacred Warrior Fellowship
2K Followers 956 FollowingPostdoc @Stevens1lab | Interested in Alzheimer’s disease, all things single cell, and glia, glia, glia | Dad 👧👶🏻 | #scicomm #rstats | NEU ‘10 & UCI ’16
5K Followers 303 FollowingI love building things. AppliedAI/ChatGPT @openai. Formerly, eng @airbnb, founder @fabric_app. Creator of the first @facebook Timeline, Memories, See Friendship
34K Followers 463 FollowingAttempting to lay down Consensus views to better inform decision making. Often sarcastic, and often ranting. Do your own due diligence.
1K Followers 62 FollowingI invest in biotechs. I often tweet about companies I have positions in. Nothing I say should be considered investment advice.
20K Followers 6K Followingyour favorite pro’s favorite pro 👷🏼 i went to @stanford then worked @google @morganstanley @baincapital and now I build decks (real ones)
19K Followers 641 FollowingAttempting to make this acct more professional; microcap PM Artko Capital; CFO/corp fin consultant/board member; aging trail runner; love dogs; rarely serious