Alex Gajewski @apagajewski

trying to figure out sample efficiency, prev founder @sfcompute, @ExaAILabs alexgajewski.org San Francisco Joined June 2014

Tweets

89
Followers

2K
Following

876
Likes

886

Alex Gajewski @apagajewski

7 months ago

I wonder what you would get if you trained something Cycle-GAN-like between images and music. Probably possible today with the quality of generative models we have!

0 0 4 1K 0

Alex Gajewski @apagajewski

7 months ago

The new google image model is quite good except for the fact that it doesn't like to draw physicists:

0 0 1 1K 1

Download Image

Alex Gajewski @apagajewski

7 months ago

This one seems like a good idea to me, increasingly I think datasets and RL environments are the limiting factor:

Y Combinator @ycombinator

7 months ago

This one seems like a good idea to me, increasingly I think datasets and RL environments are the limiting factor:

9 16 138 113K 86

Download Video

1 0 2 1K 1

Alex Gajewski @apagajewski

7 months ago

I hope that somebody starts a company to make an AI-native smartwatch. It feels to me like the ideal form factor for most of what I want a language model to do.

0 0 0 590 0

Feels like a good time to start a computer control startup. The methods are generally known (RL on top of base models), and it probably doesn't require that much compute, just thoughtful environment design. I would probably start with a text-only representation of websites.

1 0 7 678 2

Alex Gajewski @apagajewski

7 months ago

Very excited for this new cluster. Big enough to train R1, but it's running our combinatorial auction so the prices should be rational

evan conrad @evanjconrad

7 months ago

Very excited for this new cluster. Big enough to train R1, but it's running our combinatorial auction so the prices should be rational

35 38 686 92K 136

Download Video

0 0 17 1K 1

Alex Gajewski @apagajewski

7 months ago

very excited the weights of o1 finally arrived

2 0 13 927 0

Download Image

Alex Gajewski @apagajewski

8 months ago

One part of SF Compute we haven’t talked about very much yet is that post-AGI (presumably soon), the models will want to train more models. (Really, people will ask the first models to train more models, or perhaps to solve tasks that would benefit from, say, some custom RL).…

0 0 9 1K 1

Alex Gajewski @apagajewski

8 months ago

Has anyone tried “sub-token attention”? Artificially increase the sequence length by including K copies of each token next to each other (say, each linearly projected by a different map), and let the different copies attend to each other. True self-attention :P (And then at the…

0 0 3 478 0

Alex Gajewski @apagajewski

8 months ago

I wonder if companies training creative models will be more durable than companies training models to perform well-defined tasks. Because it's easier for competitors to hill-climb on the evals for the tasks, but training a creative model requires the founders to have good taste,…

0 0 4 621 1

Alex Gajewski @apagajewski

8 months ago

I wonder how much compute it would take to reproduce the entire published set of deep learning knowledge. if you were efficient about it.

2 0 2 547 0

Alex Gajewski @apagajewski

8 months ago

very interesting talk on the future of mathematics given language models: youtu.be/vYCT7cw0ycw?si… one thing this made me realize is that the next decade or so of math is going to be incredibly fun, as the reasoning models crank through all sorts of new and interesting questions

0 1 7 592 5

Alex Gajewski @apagajewski

8 months ago

this is a surprisingly creative (very) short story to have been written by an LM!

Yuxi on the Wired @layer07_yuxi

8 months ago

this is a surprisingly creative (very) short story to have been written by an LM!

2 0 14 2K 8

0 0 1 667 1

Raffi Hotter @raffi_hotter

8 months ago

Why are MRIs so damn heavy? They weigh over 10,000 pounds, or about 3 cars' worth. MRI design knowledge is hidden inside large companies, so I tried to work it out from first principles. 🧵 (1/12)