Our research, explained
Articles on how we build and evaluate scientific foundation models and scientific machine learning tools.
-
Tokenizers
The price of a token
Every token costs compute, context and money. We measured how many tokens twelve tokenizers need for the same Indian-language, English, maths and code text, with an interactive token explorer.
-
Training
Small models for the physical world: how we trained Dhīr from scratch
Why we built our own foundation models, where small models win, and how we pre-trained Dhīr-2B and Dhīr-5B MoE from random weights on Google Cloud TPU v6e.
-
Post-training
Reinforcement learning on TPUs: lessons from training Dhīr
How we used reinforcement learning to shape Dhīr-2B and Dhīr-5B MoE on Google Cloud TPU v6e: scoring 2,048 answers per step, rewarding the right language, and catching the model gaming the score.
-
Edge deployment
Running Dhīr on edge devices: mobiles, laptops and embedded boards
Measured speeds for Dhīr 2B and Dhīr 5B MoE on mobiles, laptops, embedded boards and data-centre GPUs: why the mixture-of-experts model decodes faster for one user, when each model serves many users better, and how much memory each needs.
Discuss the science behind our models
Technical architecture, evaluation approach and benchmark materials, shared with qualified partners under NDA.