Scott Swingle @bio_bootloader

Father of 3, building Mentat (the github native coding agent!) @AbanteAI, prev @DeepMind mentat.ai California Joined May 2022

Tweets

5K
Followers

11K
Following

3K
Likes

43K

Scott Swingle @bio_bootloader

4 days ago

after much testing and tuning, I think this is still the case At least for a coding agent like Mentat, Sonnet handles uncertainty and exploration better, and is more thorough that GPT-5. It's much more expensive and slower though, so there are real tradeoffs

Scott Swingle @bio_bootloader

2 months ago

0 0 17 4K 0

4 1 24 2K 2

Scott Swingle @bio_bootloader

a week ago

feels like a lot of these ideas will simply work now that we have powerful in-context learning AlphaGo needed RL because we didn't have a generally intelligent base. Now models can start teaching themselves

Andrej Karpathy @karpathy

a week ago

261 851 7K 846K 5K

0 0 6 653 0

Download Image

Scott Swingle @bio_bootloader

a week ago

after tons of testing with Mentat: Sonnet: Great default behavior, but ignores a lot of prompts. Needs strong pushes to change behavior GPT-5: Super steerable. Actually does what I say! But I have to tell it what to do more Depending on use case each can be good

0 0 7 482 1

Scott Swingle @bio_bootloader

2 weeks ago

every day at midnight do all of OpenAI's api prompt caches invalidate simultaneously?

1 0 12 776 1

Download Image

Scott Swingle @bio_bootloader

3 weeks ago

whoa OpenAI is giving GPT-5 on the API a prompt before the one I set?? they give it: - instructions on formatting (which is contradicting my own) - today's date - telling it that it's being used over the API - a bunch of other stuff not cool

11 8 159 32K 20

Scott Swingle @bio_bootloader

3 weeks ago

@repligate @mayfer @claudeai finally ran Opus on LoCoDiff and it's better than Sonnet on short context, but falls apart on higher contexts? very strange

0 1 7 494 1

Download Image

Grant♟️ @granawkins

3 weeks ago

Wrap your head around this. - Have git spit out a file's edit history - Line up all the diffs one-after-another - Feed it to a model and tell it to recreate the current (last) state At 100,000 tokens, Sonnet 4 succeeds 2/3 of the time, while GPT-5 does <5%.

Scott Swingle @bio_bootloader

3 weeks ago

12 12 255 182K 74

Download Image

16 27 397 39K 134

Scott Swingle @bio_bootloader

3 weeks ago

we've made a ton of changes to Mentat the last several months. Would love feedback on our new landing page! mentat.ai

1 0 6 435 0

Scott Swingle @bio_bootloader

4 weeks ago

Mentat has positive margins. It’s simply 19.5% over our model api costs. Use as much or as little as you want (you’ll want a lot)

Nick @nickbaumann_

4 weeks ago

Mentat has positive margins. It’s simply 19.5% over our model api costs. Use as much or as little as you want (you’ll want a lot)

133 247 3K 888K 2K

Download Image

0 0 13 788 0

Scott Swingle @bio_bootloader

4 weeks ago

almost none of my gpt-5 api calls are hitting the cache

0 0 7 482 0

Scott Swingle @bio_bootloader

4 weeks ago

GPT-5 is OpenAI's best model yet on LoCoDiff LoCoDiff is unusual - more reasoning usually hurts performance, and it's the same for GPT-5, with "minimal" doing better than "medium" reasoning effort. GPT-5 almost at Gemini 2.5 Pro levels until 60k tokens Sonnet 4 still dominates

1 0 17 3K 3

Download Image

Scott Swingle @bio_bootloader

4 weeks ago

will update as soon as I know if this is no longer the case

Scott Swingle @bio_bootloader

2 months ago

will update as soon as I know if this is no longer the case

0 0 17 4K 0

2 0 16 1K 0

Scott Swingle @bio_bootloader

4 weeks ago

$1.25 / $10 input/output is great, and 400k context window also great cached input is only $0.125 - 10%, matching Anthropic's cached input rates - Amazing!