HomeLearnCoursesHackathonsAccount
Test-Time Compute & Reasoning Models
Two Ways to Make a Model Smarter · 1/2

The training-time paradigm

For most of the last decade, the dominant way to improve a language model was to scale it up during training: more parameters, more training data, more compute spent once, upfront, to produce the model's weights. This is often described through scaling laws, the observed pattern that model performance improves fairly predictably as you increase model size, dataset size, and training compute together. Once training finishes, the resulting model is fixed, and every query it answers afterward uses roughly the same amount of computation regardless of how hard the question is.

This paradigm has an appealing simplicity. You pay a large, one-time cost to train a better model, and then every user gets the benefit of that improvement for free, at the same speed, whether they ask it to summarize an email or solve a differential equation. The tradeoff is that the model can't adapt its effort to the difficulty of what's in front of it. A simple lookup and a genuinely hard multi-step problem get the same fixed pass through the network.