Compose
Sample characters, settings, objects, and moral themes.
Slot-based promptsarXiv:2603.14563 · cs.CL
Tiny stories.
Many voices.
A synthetic combinatorial corpus of children's stories in 17 Indic languages, built to help small language models learn coherent language from clean, simple narratives.
132,942
stories
93.9M
tokens
17
languages
3M+
prompt combinations
Statistics reported in the paper release
01
The premise
Why this corpus exists
Small models need
small, clear worlds.
The web is noisy, structurally complex, and unevenly distributed across languages. For underrepresented Indic languages, that makes foundational model training especially difficult.
Multilingual TinyStories replaces that noise with concise narratives written for a five-year-old's reading level. The constrained vocabulary helps compact models learn syntax, semantics, and basic narrative reasoning without being overwhelmed by jargon.
Native scripts5–8 sentences80–200 tokensClean JSONLCC BY 4.0
From prompt to corpus
A hybrid pipeline,
built for variety.
Seven languages are generated natively with Sarvam-M. A validated Gujarati split then becomes the pivot for expansion into ten more languages.
Instantiate
Build a fresh instruction from millions of combinations.
15+ archetypesGenerate
Sarvam-M writes short native-script stories with controlled decoding.
7 native languagesValidate
Check length, deduplicate, and remove Latin-script leakage.
Programmatic QAExpand
Translate the clean Gujarati split into ten additional languages.
17 total languagesNative generation · gu, kn, ml, or, pa, ta, te
Cross-lingual expansion · as, doi, gom, mai, mni, ne, sa, sat, sd, ur
Combinatorial prompt engineering
One simple equation.
Millions of stories.
50characters
×
40settings
×
50objects
×
30themes
= 3,000,000 base combinations
Prompt laboratory
Generated instruction
Adapted from the problem-solving template in Appendix B.
Paper release by language
A corpus with
17 distinct voices.
Native Sarvam-M generation
Gujarati-pivot translation
Bar length represents token count
Diversity analysis
Different words.
Different worlds.
Distinct-2 measures the share of unique two-word sequences. Native-generation splits generally show the strongest lexical variety, while translated splits remain meaningfully diverse.
0.25–0.69Distinct-2 range
0.11–0.38Average similarity
10M–200MTarget model size
Lexical diversityDistinct-2 · higher is better
Native Expanded
Source: Table 1 in the paper.
Figure 2 · Samples from the dataset
Native-script corpus
Stories that
look like home.
Every split is localized to its native writing system. Language-specific Unicode filters remove Latin characters while preserving punctuation and narrative flow.
17
script-aware filters tailored to every language split
Built to be used
A foundation for
smaller, local models.
Pre-train compact models
Use short, constrained narratives to teach grammatical structure and basic coherence.
Study cross-lingual transfer
Compare a shared narrative distribution across 17 low-resource language settings.
Build educational tools
Prototype story completion, continuation, and child-oriented language experiences.
Open research
Read it.
Load it. Build.
The dataset is published on Hugging Face under CC BY 4.0. Generation and filtering scripts are available in the companion repository.
Python · Hugging Face Datasets
from datasets import load_dataset
# Load the Gujarati split
stories = load_dataset(
"deeponh/multilingual-tinystories",
split="gu"
)
BibTeX citation
@article{halder2026multilingual,
title={Multilingual TinyStories: A Synthetic
Combinatorial Corpus of Indic Children's Stories
for Training Small Language Models},
author={Halder, Deepon and Mukherjee, Angira},
journal={arXiv preprint arXiv:2603.14563},
year={2026}
}