NVIDIA and IIT Madras AI4Bharat lab introduced Nemotron-pretraining-Indic-v1, a synthetic pretraining corpus of about 2.8 trillion tokens across 22 Indian languages, alongside the open GPU-accelerated pipeline used to build it. The corpus holds 2,601.0 billion native-script tokens and 224.6 billion romanized tokens counted with the Gemma-4 tokenizer. For business teams building products for India, this matters because lack of training data has kept Indian-language technology behind English.
How the corpus and generation pipeline work
The 22 languages are those listed in India's Constitution, described as first languages for 97% of Indians, nearly 1.2 billion citizens at the 2011 Census. Across 34 language-script configurations, all 22 languages receive native-script data and twelve also receive romanized text in Latin script. The twelve dual-script languages are Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu and Urdu, while ten have native-script data only.
Generation starts from documents in 15 Nemotron-CC-v2 subsets covering web text, question-answer material, mathematics, code and specialized content. NeMo Data Designer runs translation, quality checks and targeted repair, with source identifiers retained so each output traces to its original document. Every record links to a curated source document and each output is checked against its input, with prompts, LLM workflows and quality checks published for adaptation to other source material.
For the twelve dual-script languages, full documents are translated with Google's Gemma-4-31B-it, selected after comparing models on complete documents. Prompts separate prose to translate from content to preserve, including code, equations, URLs and formatting. The same model scores each output against the English source on meaning, fluency, terminology, style and formatting, retaining only scores of at least 75 out of 100 and repairing lower scores from feedback for reassessment, a loop that recovered 438.4 billion tokens.
What this means for companies using AI
Accepted native-script translations become the source for romanized data through script conversion, which human comparison found to produce fewer hallucinated additions and less residual English than direct romanization from English. The other ten languages use Indic Translate from Bodhan AI, with Gemma-4-31B-it as assessor and the same 75-point threshold but without feedback-based rewriting. Santali uses overlapping windows for long documents, and Manipuri is translated into Bengali script before conversion to Meitei Mayek.
Early model evidence comes from Nemotron 3.5 Super VL, where a 100-billion-token continued-pretraining stage allocated 10 billion tokens to synthetic and natural Indic text. Scores rose on five Indic benchmarks — MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy and Indic GSM8K — by 4.98 to 9.82 percentage points. Against Nemotron 3 Super, the new model leads on five of six task benchmarks and on all four IndiVibe dimensions, while instruction following on IFEval 1.5 dipped 2.04 points.
Alongside the corpus, AI4Bharat introduced Indic-LLM-Eval-Suite, an open collection across eight categories including chat, instruction following, mathematics, STEM, reasoning, knowledge, code and translation. It includes MILU-1.5 and IndicIFEval-1.5 after audits fixed incomplete questions, missing figures, translation errors and evaluation-code faults, and integrates with NeMo-Gym. Dataset tags such as hi-Deva-IN and hi-Latn-IN support streaming inspection on Hugging Face, while the Nemotron Bharat plugin supplies prompts, rewriting, windowing and remapping tools.
