🚨New model alert🚨 We're super excited to release OpenHathi-Hi-v0.1, the first Hindi LLM from our OpenHathi series of models. This model is trained under compute and data constraints to show that we can get GPT-3.5-like performance on Indic languages with a frugal budget. 1/5

We build on top of Llama2-7B and extend its tokenizer to 48K tokens. We divide our training into 2 phases: (1) embedding alignment: aligns the randomly initialised Hindi embeddings, & (2) bilingual language modeling: teaches the model to attend cross-lingually across tokens. 2/5

@SarvamAI To be precise, an even earlier high-performing open source Hindi LLM was our Hi-NOLIN from more than a month ago... 😀

Nolano.ai @NolanoOrg

2 years ago

@SarvamAI To be precise, an even earlier high-performing open source Hindi LLM was our Hi-NOLIN from more than a month ago... 😀

6 24 156 32K 46

Download Image

@SarvamAI haathi mera saathi

@SarvamAI This is awesome! do you know if there's any plan to do a conversational/"chatty" fine-tune of it?

@SarvamAI Congratulations!

@SarvamAI Any hosted versions to try out?

@SarvamAI LFG we ship!!

@SarvamAI I recall the days in the early 2000s we did not have Unicode fonts and publishing content in Indian languages was a challenge. And now a Hindi LLM! The technological progress we are seeing in the Indic space over the last few years is phenomenal.

@SarvamAI Proud to contribute to the ecosystem through the launch of OpenHathi models @NandanNilekani @PeoplePlusAI @pramodkvarma @ai4bharat @Shankar4EkStep @abhish18

@SarvamAI I think what we achieved here is better tokeniser and context awareness on the Hindi language. Very good work!! @SarvamAI team. Long way to go💪

@SarvamAI any chance you're building something for kannada? cc @scam_adi

@SarvamAI @RajanAnandan Congratulations Team @SarvamAI

@SarvamAI Great to see LLMs come out of India! Congratulations! Can this model recognize other prominent Indian languages like Tamil, Telugu, etc.?

@SarvamAI does it support Roman script Hindi?

@SarvamAI Awesome work! Would love to see Urdu support for this too

@SarvamAI

@SarvamAI @uncensored_ai 's app rips with hinglish, bantai

@SarvamAI 🤩🤩

@SarvamAI Hi, Is the instruct model available?

@SarvamAI For anyone looking to chat with the model can goto chat.sarvam.ai However currently the access is limited for people 🥲.

@SarvamAI Great release. Is it proficient in understanding Hinglish context as well.

@SarvamAI Congratulations @pratykumar @vivek_raghavan and team Sarvam AI! Look forward to seeing exciting new applications that leverage OpenHathi to dramatically increase access to AI across India

@SarvamAI Post to contribute to the Indian AI ecosystem and continue to tradition of frugal innovation @Rajeev_GoI @AshwiniVaishnaw @_BHASHINI

@SarvamAI @gareebscientist

@SarvamAI @realpolity101

@SarvamAI Nice

@SarvamAI Great approach and super results !! Following this team closely from the AI4Bharat days. Any timeline to create other Indic language bi-lingual specific models ? Furthermore, any open recipe to create such language specific corpora or instruction set ?

@SarvamAI why did you choose to spend resources here vs elsewhere?

@SarvamAI Congratulations @Team SarvamAI

@SarvamAI Do you think if we build a model from scratch for Hindi the token creation will be different than how OpenAI approached ? Because for Hindi that working could be fully different.

@SarvamAI Congratulations on releasing the first Hindi LLM and thank you for releasing the model weights.

@SarvamAI 🤩