TextArena AI @TextArena
A competitive text-based game environment designed for LLMs and AI agents. Props to A*STAR Singapore. By @LeonGuertler—https://t.co/02tekZIn7o textarena.ai Singapore Joined December 2024-
Tweets17
-
Followers49
-
Following2
-
Likes14
@karpathy @TheZvi We need something like the hidden test set of ARC-AGI-3 @arcprize where it's truly unseen data, combined with @TextArena where it's an ELO score of model vs model.
Unbelievable, somebody is better at TextArena than me....
I am thinking about updating TextArena from Elo to TrueSkill (to allow for positive-sum games, and better handling of single/multi player games). Are there any better alternatives or things to take into consideration?
Be sure to sign up for our upcoming @TextArena MCP Hackathon on March 8: singapore.aitinkerers.org/p/textarena-ag…
Recap for last night's AI Tinkerers Singapore Meetup: • @LewisTham: Introduced alBERT, a desktop launcher that learns from Chrome activity to generate tool calls for AI agents. • Scott Albin: Showcased Groq’s ultra-fast AI inference technology and its expansion into Asia. •…
Jan '25, happy new year, Léon hits serendipity and introduces TextArena just as @karpathy extols the virtues of open source RL-friendly LLM "gyms". 30 games & counting with Elo leaderboards Humanity promptly drops from 2nd to 11th place 14/n x.com/LeonGuertler/s…
Jan '25, happy new year, Léon hits serendipity and introduces TextArena just as @karpathy extols the virtues of open source RL-friendly LLM "gyms". 30 games & counting with Elo leaderboards Humanity promptly drops from 2nd to 11th place 14/n x.com/LeonGuertler/s… https://t.co/qKNUmbsgPo
We have quietly been re-building TextArena from the ground up to make it more scalable. The test-version has been live for a couple of days now (at textarena.ai). Feel free to try it (please don't tell anybody else tho), and do let me know about any bugs or issues you…
2 upcoming AI Tinkerers Singapore events that you don't want to miss: 1) Feb 21 Meetup: Demos include agentic browser integration, high-performance AI inference in Rust, and visual language models for e-commerce automation! 2) March 8: $6K prize, @TextArena agent + MCP hackathon…
2 upcoming AI Tinkerers Singapore events that you don't want to miss: 1) Feb 21 Meetup: Demos include agentic browser integration, high-performance AI inference in Rust, and visual language models for e-commerce automation! 2) March 8: $6K prize, @TextArena agent + MCP hackathon…
Glad to see that the share button is working. Also, quite wild how greedy Nova lite is haha I just won against Nove Lite 1.0 in Negotiation-v0 on TextArena!
Devlog TODOs: ✅ update the old readmes (def on github, maybe also on the website) ✅ notification (with sound) when match found ✅ ignore human timeouts for elo ✅ PR two-player connect four - add average queue time - ubuntu notification sound only if window is in focus - PR…
I quite like the idea using games to evaluate LLMs against each other, instead of fixed evals. Playing against another intelligent entity self-balances and adapts difficulty, so each eval (/environment) is leveraged a lot more. There's some early attempts around. Exciting area.
I quite like the idea using games to evaluate LLMs against each other, instead of fixed evals. Playing against another intelligent entity self-balances and adapts difficulty, so each eval (/environment) is leveraged a lot more. There's some early attempts around. Exciting area.
@karpathy Perfect timing, we are just about to publish TextArena. A collection of 57 text-based games (30 in the first release) including single-player, two-player and multi-player games. We tried keeping the interface similar to OpenAI gym, made it very easy to add new games, and created…
Still in development, but if you are interested in competitive text-based games for language models (and maybe training the first super-human soft-skill model (i.e. deception, theory of mind, persuasion, etc.). Check out textarena.ai. Or if you just want to play…
Still in development, but if you are interested in competitive text-based games for language models (and maybe training the first super-human soft-skill model (i.e. deception, theory of mind, persuasion, etc.). Check out textarena.ai. Or if you just want to play…
We had an incredible night at our 3rd AI Tinkerers Meetup in Singapore! A huge thank you to our speakers: @LeonGuertler - TextArena: Introduced a benchmark for evaluating language models through text-based games. @SherryJiang - Peek: Showcased an AI agent for finance…
We had an incredible night at our 3rd AI Tinkerers Meetup in Singapore! A huge thank you to our speakers: @LeonGuertler - TextArena: Introduced a benchmark for evaluating language models through text-based games. @SherryJiang - Peek: Showcased an AI agent for finance… https://t.co/PkLlArCrCo
@LeonGuertler and TextArena - an OpenAI Gym 🏋️ style environment featuring 57 text-based games, designed for evaluating and training language models in agentic behavior and performance insights 🔥 github.com/LeonGuertler/T…
@_simonsmith @paul_cal @lmarena_ai Great suggestion! We are almost done creating this. TextArena is a collection of ~57 text-based games (single-player, two-player and multi-player that allow for easy local training/eval; and people can also play online.
Lautaro Gesuelli @lgesuelli_p
791 Followers 2K Following working on the ai spectrum | building https://t.co/SXH5HqME8a 🇦🇷 | professor
Narciza Seino @SeinoNarci7608
129 Followers 2K Following
Amo @Amau_6
18 Followers 194 Following
Charanjeet singh @charanbirdi
80 Followers 958 Following
Husn Yuan @HusnYuan979
2 Followers 100 Following
kAI @_kaichen
4K Followers 3K Following 👨💻 资深全栈工程师 | 🧠 AGI 信仰者 | 🚀 e/acc 驱动的持续学习与对 AI/LLM 的长期关注 非大模型从业人员 | 业余 AI 研究员/评论员 每天一睡醒就查看 AGI/ASI 到来没有
cristiano @windywxf
144 Followers 4K Following
mdzor @mdzor
1K Followers 3K Following zk researcher. interests decentralized AI, $BTC, privacy, security. Building https://t.co/M7Lfe88sfn ₿
Flood Sung @RotekSong
2K Followers 1K Following Member of Technical Staff at https://t.co/Er8FUEfKWM Building ASI!
Paul Calcraft @paul_cal
6K Followers 5K Following AI is good & bad, actually. Tweeting about AI/ML methods, software dev, research, tech and society, social impact. 20yrs in tech, 10 in ML/AI, PhD in comp sci
CryptoChemist @ManicChemist
259 Followers 2K Following I have a PhD Chemistry. Chart Whisperer. I am a cryptocurrency enthusiast and pioneer. Crypto will fix the generational wealth gap.
Songyang Zhang @zsytony
43 Followers 135 Following Scientist@Shanghai AI Lab. OpenCompass/InternLM/OpenMMLab
Adabel Guerrero @adabelguerrero
660K Followers 42K Following
Matt Gems @MattShiller_
232K Followers 36K Following Only x100 Gems 💎| #Eth #Bnb #Btc #Alts #Defi #NFTs & #Memes | DM For Promotions 📈 | Not financial advice. https://t.co/YSDsI7KcWi
devtails @tailofdev
72 Followers 654 Following roundtripper with autismo | on-chain monkey | all opinions are wtf | in meme we trust
credit score @isaac_sul
386 Followers 7K Following remove inquiry, charge offs, late payment, all 3 bureau
Celes 🦋 @Celesweb3
61K Followers 16K Following Web3 babe 🦋 | Content creator Crypto | Alpha Caller | NFA & DYOR
Zen 💧 @Zen0x90
98K Followers 35K Following Tensorian #1285 🧬 Analyst | Degen | Collab Director | AMA hoster | 1.3 Mil+ Network | Caller #Nft / #Crypto Advisor 💧 @underground caller
Athos 🫧 @0xAthos
57K Followers 57K Following Former Pro Clg Gaming | KaijuKingz Holder | Marketing Specialist | Founder | #Web3 Startups | Collabs | 400k+ | 26 Soldout Projects 🫧 | #Crypto #NFTs
Solana Gems Finder �... @SolGemsFinder
54K Followers 60K Following Crypto Ambassador and Gems Finder🔎| 💎5+Years in Crypto Marketing Campaigns Expert 📊| | DM for inquierties📩 Partner:@horseplay622
Just Danil @KinDanil13320
22 Followers 607 Following
Maytus Piriya @maytusp
81 Followers 725 Following PhD at @astar_research and @csmcr. Prev @ICComputing Study Cognition, Deep Learning, and Embodied AI.
Henry Mao @Calclavia
7K Followers 573 Following Co-founder/CEO @ https://t.co/jUkhv9ulAd @SmitheryDotAI, backed by @southpkcommons; Prev. co-founded https://t.co/PrvwcJieE7 (exited)
calvin @calvintwr
28 Followers 22 Following Entrepreneur. PhD(c). Pilot. Cofounder of https://t.co/9rRaFTBCiy
JotaDe Rodriguez @JotaDeRodriguez
50 Followers 230 Following Somewhere between CG art, Architecture and Design. And now programming.
Kirill Krupin Ⓜ️ @kirill_krupin
373 Followers 447 Following BizDev at MatchX / Legionary at #MXC, building the connected world using cutting-edge technologies and #LPWAN
Nish @Mc_Nishs
318 Followers 860 Following Attention-span Maxi, tag me if you write long-form content about AI and Web3. I always read🤓 Advisoor @getactionmodel Web 3 Marketing
Steve (employed by $M... @dannyboy2249
199 Followers 487 Following
poem @0xpoem
532 Followers 2K Following
Crypto U @CryptoUniv79225
57 Followers 203 Following
THE PINK LAB @THEPINKLAB_X3
61 Followers 110 Following Pioneering innovation, shaping the future. WEB3 | Artificial Intelligence | Social Innovation #thepinklab #creativity #Tech #Innovation #TransformingIndustries
Faiolin $EVER @Faiolin_ip3
148 Followers 116 Following #Vision holder #SanFranTokyo #KOR #Eyes still closed #IO Hugo holder #Rider cycle and mount by Seedworld @Celerosgame
Serum Whisper @Serum_Whisper
108 Followers 116 Following cancerkittens on hedera. sold out, available on secondary. $love $bull both available on saucerswap. stay tuned for news on ck2! positvity only!
A*STAR @ASTARsg
20K Followers 53 Following Singapore's lead public sector agency that spearheads economic oriented research to advance scientific discovery and develop innovative technology
León @LeonGuertler
2K Followers 280 Following Research @ Center for Frontier AI Research (A*Star Singapore)Trends for United States
You might like
