introducing tiny-tpu, a tiny TPU, supporting both training and inference, ENTIRELY on chip!
here is an overview of how I got started and how this was built:
I spent my summer building TinyTPU : An open source ML inference and training chip.
it can do end to end inference + training ENTIRELY on chip.
here's how I did it👇:
i remade tiny-tpu to support both inference and training!
we successfully tested our architecture on the classic XOR problem.
here's what i learned throughout the process:👇
i remade tiny-tpu to support both inference and training!
we successfully tested our architecture on the classic XOR problem.
here's what i learned throughout the process:👇 https://t.co/upJoqqH4I0
🚨 BREAKING: Detailed list of all 44 people in Meta's Superintelligence team.
— 50% from China
— 75% have PhDs, 70% Researchers
— 40% from OpenAI, 20% DeepMind, 15% Scale
— 20% L8+ level
— 75% 1st gen immigrants
Each of these people are likely getting paid $10-$100M/yr.
Tokenization has been the final barrier to truly end-to-end language models.
We developed the H-Net: a hierarchical network that replaces tokenization with a dynamic chunking process directly inside the model, automatically discovering and operating over meaningful units of data
looking for my next thing! thinking about dropping out.
would love to learn more about opportunities within hardware acceleration or interpretability.
dms are open. happy to chat. would love to hear what makes you excited!
A Personal Update
By Vijay Pande
Today, I’m sharing that after more than a decade, it’s both exciting and bittersweet that I’ll be stepping away from my role at Andreessen Horowitz. This has been one of the most meaningful chapters of my life, and I’m filled with gratitude as I…
this winter, i interned at @trykino in sf to build the cursor for video editing!
yesterday, i gave a talk to share some of my work and a lot of the engineering details into building the best video search & editing software for filmmakers!
7K Followers 498 FollowingCo-founder & President of Technology @OscarHealth; former Bridgewater, McKinsey, Eigentrust, Vostu, Stanford CS visiting scholar, U Hannover.
246K Followers 1K FollowingAt Greylock, we are the first partner to consumer and enterprise software entrepreneurs. Newsletter: https://t.co/4tHdH9xmvk
17K Followers 1K Followingapplied AI @openai. I work with the world's leading startups and developers to bring the benefits of safe AI to every human. views my own 🇮🇳 @dukeu
56K Followers 853 FollowingFiguring out AI @allen_ai, open models, RLHF, fine-tuning, etc
Contact via email.
Writes @interconnectsai
Wrote The RLHF Book
Mountain runner
2.4M Followers 47 FollowingThe official handle for NVIDIA. Blog: https://t.co/JAn5eKOTBT Support: https://t.co/6ln5FVnA2o All our social media: https://t.co/Uc56dL57Dh