BigScience Large Model Training @BigScienceLLM
Follow the training of "BLOOM 🌸", the @BigScienceW multilingual 176B parameter open-science open-access language model, a research tool for the AI community. bigscience.notion.site/BigScience-176… JeanZay supercomputer (France) Joined March 2022-
Tweets129
-
Followers9K
-
Following1
-
Likes50
The Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100
Crosslingual Generalization through Multitask Finetuning 🌸 Demo: huggingface.co/bigscience/blo… 📜 arxiv.org/abs/2211.01786 💻github.com/bigscience-wor… We present BLOOMZ & mT0, a family of models w/ up to 176B params that follow human instructions in >100 languages zero-shot. 1/7
The super-fast inference solutions are finally here for all to use:
The super-fast inference solutions are finally here for all to use:
What do @StabilityAI @EMostaque #stablediffusion & @BigscienceW Bloom - aka the coolest new models ;) - have in common? They both use a new gen of ML licenses aimed at making ML more open & inclusive while keeping it harder to do harm with them. So cool! huggingface.co/blog/open_rail
The Technology Behind BLOOM Training🌸 Discover how @BigscienceW used @MSFTResearch DeepSpeed + @nvidia Megatron-LM technologies to train the World's Largest Open Multilingual Language Model (BLOOM): huggingface.co/blog/bloom-meg…
BLOOM is here. The largest open-access multilingual language model ever. Read more about it or get it at bigscience.huggingface.co/blog/bloom hf.co/bigscience/blo…
🌸@BigscienceW BLOOM's intermediate checkpoints have already shown some very cool capabilities! What's great about BLOOM is that you can ask it to generate the rest of a text - and this even if it is not yet fully trained yet! 👶 🧵 A thread with some examples
🌸@BigscienceW BLOOM's intermediate checkpoints have already shown some very cool capabilities! What's great about BLOOM is that you can ask it to generate the rest of a text - and this even if it is not yet fully trained yet! 👶 🧵 A thread with some examples https://t.co/7WcCE2YAjY
For 111 days, we've enjoyed world-class hardware stability and throughput thanks to the hard work of our friends at @Genci_fr, @INS2I_CNRS, Megatron & DeepSpeed. Having reached our objective earlier than expected, we'll keep training for a few more days. Stay tuned, more soon ;)
merve @mervenoyann
56K Followers 4K Following open-sourceress at @huggingface 🧙🏻♀️ proud mediterrenean 🍋 I do TL;DR on ML papersHugging Face @huggingface
345K Followers 189 Following The AI community building the future. https://t.co/VkRPD0VKaZ #BlackLivesMatter #stopasianhateJeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @StanfordJulien Chaumond @julien_c
47K Followers 1K Following Co-founder and CTO at @huggingface 🤗. ML/AI for everyone, building products to propel communities fwd. @Stanford + @Polytechniqueclem 🤗 @ClementDelangue
91K Followers 5K Following Co-founder & CEO @HuggingFace 🤗, the open and collaborative platform for AI buildersOmar Sanseviero @osanseviero
32K Followers 2K Following Chief Llama Officer @huggingface 🦙 Founder @AI_Learners. Xoogler (SWE @Google Assistant, 20% PM TF Graphics). 100% Hacker Llama🇵🇪🇲🇽Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | PianistThomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-scienceMMitchell @mmitchell_ai
80K Followers 1K Following Interdisciplinary researcher focused on shaping AI towards long-term positive goals. ML & Ethics. Same content in the Sky, Threads, & the Prehistoric ElephantSasha Luccioni, PhD �.. @SashaMTL
19K Followers 4K Following AI & Climate @HuggingFace, Board Member of @WiMLworkshop and @ClimateChangeAI. @techreview 35 Innovators under 35, @TEDTalks speaker. She/her/Dr/ 🦋Richard Socher @RichardSocher
101K Followers 971 Following CEO @youSearchEngine Investing at @aixventuresHQ Before: Stanford Adj Prof in AI/NLP, Chief Scientist at Salesforce, MetaMindStella Biderman @BlancheMinerva
15K Followers 748 Following Open source LLMs and interpretability research at @BoozAllen and @AiEleuther. My employers disown my tweets. She/herSam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGzRoss Wightman @wightmanr
18K Followers 1K Following Computer Vision @ 🤗. Ex head of Software, Firmware Engineering at a Canadian 🦄. Currently building ML, AI systems or investing in startups that do it better.Nate Raw @_nateraw
7K Followers 1K Following machine learning hacker. previously @huggingface @lightningaiSara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.rohan anil @_arohan_
12K Followers 2K Following Principal Engineer, @GoogleDeepMind Gemini. prev PaLM-2. Tinkering with optimization and distributed systems. opinions are my own.Zach Mueller @TheZachMueller
10K Followers 393 Following 🤗 Technical Lead for the Accelerate Project | Passionate about Open Source | Nerd who enjoys touching the grass | #ADHD | He/HimRob Osouro @robosouro
100 Followers 644 FollowingKumara _Baran @barankumara3010
1 Followers 97 Followingshivam madnoorkar @shivammadn491
2 Followers 26 FollowingNoman Tanveer @NomanTa98551465
2 Followers 96 Following Interested in Deep Generative Models and Multimodal research!Mr. Z @pbhalesain
115 Followers 282 Following मेत्तानच सब्बलोकास्मीन, माणसं भावये अपरिमानं - करणीय मेत्त सुत्तItqdevs Softwares @itq_devs
23 Followers 357 Following Itqdevs is your one-stop service provider for all your business technology needs. Custom softwares, exceptional design services, data analytics & cybersecurityViaViaPersoneel @ViaViaPersoneel
32K Followers 13K Following Bent u op zoek naar de juiste kandidaat voor uw vacature? Dan bent u bij ons aan het juiste adres!! Bekijk snel onze website! http://t.co/IF04LJ5JWCJeff Lee @JeffLee88939390
18 Followers 119 FollowingErich Steinbüchel @steinbuchel
130 Followers 508 Following #InternetofThings, #IoT, #blockchain, #apis, #tennislila @fatemme97
26 Followers 97 Following a programmer. solving a math problem; It's about living forever, without sadness, without disease, without die.erwin86 @erwin8644975808
4 Followers 47 FollowingItxaso Baskero Dorrea.. @IDorreak
12 Followers 365 FollowingCoenraad Loubser 🇿.. @dagelf
2K Followers 3K Following If you love an idea, set it free: share it and help it find those who will refine it and help it take root in a community that you can be part of.Xya_cerX_!3 @3Cerx
5 Followers 215 Followingsoon @soon29980677710
3 Followers 15 FollowingGastón Antonio Zapat.. @AZV4800
85 Followers 779 FollowingGustavo Ochoa @checatavo
69 Followers 198 FollowingZaid Shariff @zaidshariff1
16 Followers 446 FollowingGIOVANNI @apuliasalus
245 Followers 5K FollowingAIQUEST @ProAiquest
101 Followers 454 Following Exploring the latest in AI tools and technologies. Join me on a journey into the future of innovation and automation. #AI #chatgpt #TechEnthusiast刘智国 @liuzdcq416
3 Followers 64 FollowingChamnan Muon (មួ�.. @chamnanmuon
1K Followers 3K Following 🎯 ICT & Digital Marketing Consultant 🏆 #LocalGuides Summit Alumni @GoogleMaps ✈️ Traveler 🇰🇭🇺🇲🇰🇷🇸🇬🇹🇭🇻🇳🇱🇦 ...🌍 📖 Lifelong Learner 📗 📊Lover📁Daniela Francisca @djarafreire
229 Followers 2K Following Madre, hija,nieta,sobrina y profesora de biología.🦋🐞🐹🐦🌾🌹🏫👩🏼💻paulorcf @paulorcf
425 Followers 3K Followingweedge @weege_007
3 Followers 80 FollowingHadoop @Hadoop02277010
543 Followers 4K FollowingB Business @business_curio
75 Followers 249 FollowingAbhinav @abhimeeofficial
26 Followers 172 Following Software Engineer @ AlphawatchAI | Crafting code with a touch of LLM magic | Hyderabadwayjayblue @gemeng
151 Followers 3K Following Soon will be the break of day. Sitting here in Blue Jay Way.Pritom Bose @_pritom_bose
40 Followers 201 FollowingTonya McCulloch @tonyamariel1663
5 Followers 77 Followingrany @ray_rain_sky
2 Followers 199 Following中推挖掘机Chines.. @ZTProspector
303 Followers 5K FollowingThomas Villanueva @ThomasVill88515
1 Followers 12 Followingcyk @ychai1224
5 Followers 398 FollowingDion Hinchcliffe @dhinchcliffe
58K Followers 7K Following Thinker, strategist, enterprise architect, keynote speaker, analyst, book author, futurist on IT, #cloud, #digitalworkplace, #Web3, #cio. @Constellationr @ZDNetMohammad Amin Abbasza.. @MAAZ__98
52 Followers 111 Following CSE graduate student @ Polytechnic University of MilanKhipu Kamayuq @KamayuqKhipu
239 Followers 864 Following waranqa wataq unayachun ama chinkachunchu qichwa rimayninchik ch'uwalla kakuchunRizz Romeo @rizz_romeo
57 Followers 136 Following Young fella with great talents 🌹🌹🌹 Music (Afro pop) & Modeling Instagram: https://t.co/t8H305HD9d%M @DS0987654321197
0 Followers 666 FollowingBigScience Research W.. @BigscienceW
15K Followers 1 Following A research workshop on large language model gathering 1000+ researchers around the world Follow the training of the 176B multilingual model live @BigScienceLLMThe Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100
BLOOM paper! An international collaboration to train a large 176B-param open-source language model, with multi-lingual language modeling the main aim. Paper: arxiv.org/abs/2211.05100 Hugging Face page: huggingface.co/bigscience/blo… 12/18
The Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100
print("Hello world! 🎉") Excited to announce the BigCode project led by @ServiceNowRSRCH and @huggingface! In the spirit of BigScience we aim to develop large language models for code in an open and responsible way. Join here: bigcode-project.org/docs/about/joi… A thread with our goals🧵
What do @StabilityAI @EMostaque #stablediffusion & @BigscienceW Bloom - aka the coolest new models ;) - have in common? They both use a new gen of ML licenses aimed at making ML more open & inclusive while keeping it harder to do harm with them. So cool! huggingface.co/blog/open_rail
As AI language skills grow, so do scientists' concerns| More about the @BigScienceLLM Open-science Open-access Multilingual Language Model nbcnews.com/tech/tech-news… via @NBCNews
🌸BLOOM (the-model-formerly-known-as-@BigScienceLLM ) has been trained on 46 natural languages and 13 programming languages ⚙️ It took a little under 4 months of training on 384 A100 GPUS (80GB) ⚖️ It's open-access under the @BigscienceW RAIL license
BLOOM is here. The largest open-access multilingual language model ever. Read more about it or get it at bigscience.huggingface.co/blog/bloom hf.co/bigscience/blo…
@BigScienceLLM Bloom launches a new #GPT3 competitor that is much more than just another big language model. #ai #artificialintelligence mixed-news.com/en/bloom-is-a-…
The Technology Behind BLOOM Training🌸 Discover how @BigscienceW used @MSFTResearch DeepSpeed + @nvidia Megatron-LM technologies to train the World's Largest Open Multilingual Language Model (BLOOM): huggingface.co/blog/bloom-meg…
@CNRS_Villejuif @CNRS_Paris @CNRSIdFSud @CNRS_Centre_Est @CnrsAlpes @CNRS_dr12 @CNRS_OccitaniE @CNRS_Toulouse @CNRS_dr17 #PressRelease 🗞️| BLOOM is the largest multilingual language model to be trained 100% openly and transparently. It handles 46 human languages, and lets scientists from all horizons freely explore how language models work, in order to improve them. ➡️cnrs.fr/en/release-lar…
Inside a radical new project to democratize AI A group of over 1,000 AI researchers has created a multilingual large language model bigger than GPT-3—and they’re giving it out for free. technologyreview.com/2022/07/12/105… @techreview @BigScienceLLM
Roughly 10% of BLOOMs training data has been code. So it is pretty good at coding🙂 Since it is multilingual you can even prompt it in arabic!
Pretty cool!! 0.155 Pass@1 is also pretty cool for a non-explicit code model
Le projet @BigscienceW dévoile #Bloom, le plus gros modèle #TAL #NLP entraîné sur le #supercalculateur #JeanZay de manière complètement ouverte et transparente. Il gère plus de 46 langues et revêt un caractère #openscience @INS2I_CNRS @CNRS @huggingface bit.ly/3yyuijJ
🌸@BigscienceW BLOOM's intermediate checkpoints have already shown some very cool capabilities! What's great about BLOOM is that you can ask it to generate the rest of a text - and this even if it is not yet fully trained yet! 👶 🧵 A thread with some examples
A milestone soon to be reached 🚀💫 Can't wait to see the capabilities and performance of this long-awaited checkpoint! What about you? Have you already prepared some prompts that you want to test? ✏️
First talk of day 2 of the #cardiffnlpworkshop! 🤗 Teven Le Scao (@Fluke_Ellington) talking about @BigScienceLLM #NLProc
It's a wrap! Thank you to all speakers, authors, organizers, volunteers and to ~100 people who joined us online and in Dublin for the afternoon workshop talks! 🌸
At @BigScienceLLM we have been running into an intermittent deadlock issue in the pytorch DataLoader during multi-dataset validation. Suspecting this bug: github.com/pytorch/pytorc… For now we used a num_workers=0 workaround for validation to overcome it. github.com/bigscience-wor…
So... the participants of BigScience have voted on the future name of the 176B language model currently being trained and it will be called 🌸 BLOOM 🌸 For the "BigScience Language Open-source Open-access Multilingual", a bit out-of-order but you get the idea ;)
Halfway there! 1000+ researchers, 176 billion parameters, 46 languages in the training dataset, 1.5 TB of text, 350 billion tokens. The sheer scale of it all is really humbling and I cannot wait to prompt it!
@BigScienceLLM Maybe add some more punctuation? BigScience Language: Open-source, Open-access, Multilingual
If you’re at #ICLR2022, hope you’ll check out our spotlighted poster: “Multitask Prompted Training Enables Zero-Shot Task Generalization.” arxiv.org/abs/2110.08207 Poster session 5, Tue 1:30-3:30 ET