NVIDIA previews 2.8 trillion Indic tokens, with an open recipe to make more
NVIDIA and AI4Bharat describe synthetic training data for 22 Indian languages and a reusable translation pipeline. The corpus is still listed as releasing soon; the public plugin can already be inspected.
A large data announcement, with availability still pending
NVIDIA and AI4Bharat at IIT Madras have outlined Nemotron-pretraining-Indic-v1, approximately 2.8 trillion synthetic tokens spanning all 22 constitutionally scheduled Indian languages. The October 16 announcement describes native-script data for every language and romanized subsets for twelve, alongside the recipe used to generate and check the text.
The availability distinction matters: NVIDIA says the corpus is releasing soon. This is an announced training resource, not confirmation that builders can download the complete collection today. The public translation plugin is already inspectable in NVIDIA’s DataDesignerPlugins repository.
Translate the document, then judge and repair it
The corpus starts from curated English material, including web text, mathematics, code and question-answer documents. NVIDIA says it retains source identifiers so an output can be traced to its input. Native-script here describes the writing system, not originally authored Indian-language text. Romanized records contain Indic-language text in Latin script and can repeat content represented in native script.
For twelve languages, the reported workflow uses Gemma-4-31B-it for document translation, quality assessment and targeted repair. Outputs need a score of at least 75 out of 100 to enter the corpus. That is a translation-error rubric score, not a 75% accuracy measurement. The other ten languages use Bodhan AI’s Indic Translate for translation and Gemma for evaluation, with additional handling for long Santali documents and Manipuri script conversion.
The announcement inventories 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer. Those are retained corpus counts, including accepted rewrites and script variants, rather than unique English documents or the token budget of a model-training run.
The reusable part is a plugin, not included inference
The repository documents installation with uv add data-designer data-designer-nemotron-indic-translation. Its package requires Python 3.10 or newer and Data Designer 0.9.4 or newer. Users configure translator and evaluator model aliases; the Apache-2.0 plugin license does not supply model access, compute or rights to the forthcoming corpus.
The blog’s links still name a nemotron-bharat-translation directory, which returned 404 during review. The working English documentation is under data-designer-nemotron-indic-translation, linked in this story’s sources. Its configuration reference also separates generation from acceptance: users choose their filtering policy, and optional rewrites are not enabled by default.
More text does not eliminate the quality problem
NVIDIA acknowledges that assessment is primarily automated, human review coverage varies by language, and the twelve-language translator also acts as its own judge. Errors can survive. Translated English also cannot replace naturally authored text for local knowledge and culture, while source-ID checks do not semantically deduplicate similar passages.
Reported improvements during Nemotron’s Indic continued-pretraining stage combine synthetic and natural text; they do not isolate this corpus’s effect. AITrending reviewed the English announcement and pinned plugin documentation, but did not run the pipeline, inspect the full corpus or reproduce its evaluations.
