Sarvm AI Collaborates with Microsoft to Bring Indian Languages to Azure

Sarvam AI collaborates with Microsoft to bring its Indic voice large language model (LLM) to Azure

The collaboration aims to enable Sarvam AI to leverage Azure AI and Azure Infrastructure to build and deploy their voice LLM stack

It also aims to enable developers to build generative AI apps at scale and empower enterprises to adopt generative AI quickly and responsibly

Indian generative AI startup Sarvam AI has announced that it is working with Microsoft to make its Indic voice large language model (LLM) available on Azure. The collaboration reinforces Microsoft’s commitment to enabling AI-driven growth and innovation in India.

Sarvam AI is building generative AI models targeting Indic languages and contexts. The startup aims to make the development and deployment of generative AI apps in India more accurate and cost effective.

Sarvam AI’s Indic voice LLM, which intends to provide a natural voice-based interface to LLMs, will initially be available in Hindi. Sarvam AI is working to expand coverage to more Indian languages while ensuring support for colloquial language use.

Pratyush Kumar, co-founder of Sarvam AI, said, “We are very excited to collaborate with Microsoft to make advanced AI technology accessible and impactful for everyone in India. This partnership embodies our vision of ‘Sarvam’ – meaning ‘all’ – by enhancing AI’s reach across various Indian languages and sectors.”

Voice is one of the most natural interfaces for generative AI applications in Indian languages and can be applied in sectors such as education, finance, healthcare, and customer service. By working to make Sarvam AI’s Indic voice LLM available on Azure, Microsoft is laying the foundations for more India-focused developers to build real-time, voice-based generative AI apps at scale.

The companies will also collaborate to help enterprises adopt generative AI quickly and responsibly. Vivek Raghavan, co-founder of Sarvam AI, said, “We see great synergy in combining the deep-tech expertise at Sarvam AI with Microsoft’s leadership in frontier models and the Copilot stack to empower enterprises to be successful in Generative AI”.

Sarvam AI will use Microsoft’s cutting-edge cloud and AI infrastructure – including Azure OpenAI Service and Azure Machine Learning – to train, host, and scale its LLMs quickly and efficiently. As part of the collaboration, Sarvam AI and Microsoft are also researching ways to better support Indian languages in Microsoft’s Generative AI frontier language models and to deploy them at scale.

“At Microsoft, we are committed to enabling AI for everyone, empowering India’s transformation into an AI-first nation,” said Puneet Chandok, president of Microsoft India & South Asia. “Through our collaboration with Sarvam AI, we are not just supporting homegrown innovation – we are fostering a future where every individual, regardless of their language or background, can benefit from the power of voice-driven AI solutions. Together, we are taking a significant step toward enabling India’s people, communities, and organizations to achieve more.”

Click here to learn more about Satya Nadella’s visit and how Microsoft is empowering organizations in India with AI.

About Sarvam AI

Sarvam AI is on a mission to lead transformative research in AI to make the development, deployment, and distribution of generative AI applications in India more robust, better performing, and cheaper. The company aims to develop a full stack of generative AI solutions, including efficient large-scale Indic language models and an enterprise-grade platform for building generative AI apps. Sarvam AI is also committed to fostering a generative AI ecosystem through open-source contributions and large-scale data curation for the public good.

Bengaluru-based Sarvam AI Releases 1st Hindi LLM 'OpenHathi' with GPT-3.5 Like Performance

Bengaluru-based Sarvam AI has recently announced the released OpenHathi-Hi-v0.1, the first Hindi LLM from its OpenHathi series of models. Trained under compute and data constraints, this Generative AI model can get GPT-3.5-like performance on Indic languages with a frugal budget, claims the 5 months old Sarvam AI, which recently raised funds from Lightspeed Ventures, Peak XV and Khosla Ventures.

The OpenHathi series of work at Sarvam AI is to make contributions to the ecosystem with open models and datasets to encourage innovation in Indian language AI.

OpenHathi is developed by Sarvam AI in partnership with AI4Bharat, a research lab at IIT Madras which works on developing open-source datasets, tools, models and applications for Indian languages.

AI4Bharat has contributed language resources and cross-lingual benchmark.

Build on top of Meta AI's Llama2-7B, the AI model used by Sarvam AI has a 48,000-token extension of Llama2-7B’s tokenizer and undergoes a two-phase training process. and extend its tokenizer to 48K tokens –

1) Embedding alignment: aligns the randomly initialised Hindi embeddings

2) Bilingual language modeling: teaches the model to attend cross-lingually across tokens.

The Tokenizion of Hindi

Tokenization refers to splitting text into smaller parts for easier machine analysis, helping machines understand human language.

To add Hindi skills to Llama-2, Sarvam AI firstly decreases the fertility score (the average number of tokens a word is split into) of its tokeniser on Hindi text. This made both training and inferencing faster and more efficient. It then train a sentence-piece tokeniser from a subsample of 100K documents from the Sangraha corpus, created at AI4Bharat, with a vocabulary size of 16K. It then merge this with the Llama2 tokeniser and create a new tokeniser with a 48K vocabulary (32K original vocabulary plus our added 16K).

For datasets of the OpenHathi's base model, Sarvam AI partnered – VerSe, hindi social media platform 'Koo' , and Kissan AI, (previously known as KissanGPT) an advanced multilingual AI chatbot engineered to help Indian farmers.

This open-sourced base model of OpenHathi has been trained with bilingual language modelling and thus needs fine-tuning on tasks for it to be used as an instruction-following model. It has also not been aligned and thus can occasionally generate inappropriate content seen in its original pretraining.

The company is inviting people to innovate on top of this latest release of OpenHathi series, by building fine-tuned models for different use-cases. Sarvam AI will additionally release enterprise-grade models on its full stack GenAI platform, which will launch soon. The base model is at HuggingFace – Here

The start-up company claims that latest release of OpenHathi model works as well as, if not better than GPT-3.5 on various Hindi tasks while maintaining its English performance. Along with standard NLG tasks, we also evaluate on a bunch of non-academic, real-world tasks.