turboderp @turboderp_

Joined June 2023

Tweets

84
Followers

766
Following

36
Likes

54

We just added Tensor Parallelism to TabbyAPI! Huge thanks to @turboderp_ and testers who made this possible. Now, enjoy the clip diving into the origins of Exllama. Wanna see TabbyAPI built live? Follow me on twitch: kingbri1st

0 2 15 2K 2

Download Video

kingbri @kingbri1st

2 months ago

1000 stars on tabbyAPI. Holy crap. Huge thanks to @turboderp and everyone who contributed!

3 2 20 698 0

Download Image

kingbri @kingbri1st

4 months ago

TabbyAPI now supports ExllamaV3 with automatic backend detection! 🎉 Please note that exl3 is being actively worked on and mileage may vary compared to exl2 Thanks to @turboderp_ and all contributors for making this a reality.

1 1 12 516 0

Download Video

turboderp @turboderp_

4 months ago

I have decided to tweet today. So here is a visualization of how the paged cache works with continuous batching in ExLlamaV3. I think it's neat. #🐈

4 13 135 8K 31

Download Video

turboderp @turboderp_

5 months ago

Seems to still be true that larger models are less sensitive to quantization. Here is Mistral-Large 123B at 1.4 bits per weight, running on one 24 GB GPU. #AI or something

10 24 207 33K 53

Download Video

turboderp @turboderp_

5 months ago

\____

1 1 65 8K 8

Download Image

turboderp @turboderp_

5 months ago

I made a thing. github.com/turboderp-org/…

16 25 227 13K 62

turboderp @turboderp_

6 months ago

CUDA builds character 😭

1 2 19 613 0

kingbri @kingbri1st

9 months ago

Supply chain alert! Don't use the comfyUI Impact pack! Its dependency ultralytics has been compromised on pypi. Thanks Shinon for letting me know in Discord. github.com/ultralytics/ul…

1 4 11 2K 1

turboderp @turboderp_

10 months ago

Fun with grounding in Qwen2-VL. Finding the things. #wherearethethings #exllamav2 #cat

1 0 12 504 5

Download Video

kingbri @kingbri1st

10 months ago

TabbyAPI now supports vision. Thanks to @turboderp_ for exllamav2's updates and DocShotgun for the initial work. Any Exl2 supported vision model works, but this release focuses on Pixtral from @MistralAI

0 2 6 503 1

Download Video

kingbri @kingbri1st

10 months ago

1 year ago, I made TabbyAPI with @turboderp_ as a side project. Now, it's my most popular side project. I wanted to break away from the bloated nature of AIO local model backends and just run #exllama. Thanks to all the contributors and testers. github.com/theroyallab/ta…

0 2 15 2K 3

turboderp @turboderp_

a year ago

huggingface.co/turboderp/llam…

2 1 4 554 0

Download Image

turboderp @turboderp_

a year ago

I performed a successful vocabulary transplant on Qwen2-0.5B and turned it into a useful draft model for Llama-3. What a time to be alive. #hashtag huggingface.co/turboderp/Qwam…

8 9 84 6K 46

turboderp @turboderp_

a year ago

Llama-3-instruct becomes much more useful when you censor some of its catchphrases. #simplesolutions etc. 🤷

1 3 22 882 10

Download Image

Mike Lacher @mikelacher

2 years ago

New project: goody2.ai GOODY-2 is an AI model that's so responsible it won't give a straight answer to anything.

2 11 27 4K 6

Daniel van Strien @vanstriendaniel

2 years ago

🚀 TACO: a new benchmark for code generation from @BAAIBeijing with 26,443 problems. • 🤖 English questions & Python solutions • 🧠 Ideal for evaluating code generation from natural language • 📊 Train: 25,443 samples, Test: 1,000 samples • 📚 Diverse difficulty levels

6 9 42 18K 36

turboderp @turboderp_

2 years ago

I guess I should post something once in a while. So here's a whole chatbot in 26 lines of Python running Mixtral 8x7B real fast on one 3090. Idk, I think it's neat. 🐈