NVIDIA and AI4Bharat, the Indian-language lab at IIT Madras, unveiled Nemotron-pretraining-Indic-v1 on October 16, 2026: a pretraining corpus of roughly 2.8 trillion tokens spanning the 22 languages listed in India's constitution, to be released together with the open data-generation pipeline that produced it. For teams building multilingual models, the point is not one more dataset. Low-resource languages lack exactly what this release targets — training material that is large-scale, traceable, and quality-checked — and here the production method is part of the package.
How the 2.8 trillion tokens are composed
Start with the breakdown published on the Hugging Face blog. About 2.601 trillion tokens are written in native scripts, and about 224.6 billion are romanized, across 34 language-script configurations. The 22 languages are the first languages of roughly 97% of Indians, nearly 1.2 billion people at the 2011 census. The sources are not scraped Indic web text: the team selected English material from 15 subsets of NVIDIA's Nemotron-CC-v2 — web text, question answering, mathematics, and code — and translated it at document level into each target language. Every record keeps its source-document identifier, each output is checked against its input, and any generated text can be traced back to the original.
AI4Bharat head Mitesh Khapra states the industry's bottleneck plainly: Indian language technology trails English because there is not enough data. This corpus addresses that gap, but not by volume alone.
Translate, score, and send failures back for repair
The pipeline runs two routes. Twelve languages take the full route: Gemma-4-31B-it translates whole documents while preserving code, equations, and formatting; the same model then scores each translation on a rubric inspired by the Multidimensional Quality Metrics framework — meaning, fluency, terminology, style, and formatting — with 75 out of 100 as the pass mark. Failures are repaired using the scoring feedback and rescored; the repair loop alone recovered about 438.4 billion tokens. A subset of accepted native-script data is then converted into romanized spelling, because Indian users routinely write their languages in Latin letters. The other ten languages, where data is scarcer, start from the dedicated Indic Translate document-level model and face the same 75-point bar, minus the feedback-repair loop. Everything runs on NVIDIA's open NeMo Data Designer, and the translation plugin's prompts and configuration are public, so other teams can feed in their own source documents and generate data for other languages the same way.
The corpus already has a first piece of usage evidence: Nemotron 3.5 Super VL, released earlier by NVIDIA, mixed 10 billion Indic tokens — drawn from a twelve-language subset plus natural text — into its continued pretraining stage and gained roughly 5 to 9.8 percentage points across five Indic knowledge and reasoning benchmarks.
The unproven parts are stated up front
The caveats come from the authors themselves. First, the corpus is mostly translated synthetic data, and the blog is explicit that it does not replace text originally written in Indian languages; local knowledge and cultural context still require natural corpora. Second, the measured gains come from a single continued-pretraining stage using a mixture of synthetic and natural data, so they cannot be credited to this corpus alone; the full corpus, including repaired data and the other ten languages, has not yet had its controlled evaluation. Third, the corpus itself is still listed as releasing soon — the pipeline is open now, the data is not fully out yet.
Even so, the signal is clear: multilingual competition is shifting from whose model has more parameters to who holds language data that can be produced at scale and audited. For other non-English ecosystems, the borrowable asset is not the 2.8-trillion figure but the pipeline that turns translation, scoring, repair, and provenance into a standard process.
