A concept · Inside the Machine · Computing
Scaling Laws
Capability bought by scale: more data, more compute, the same recipe.
Every few months a new model arrives that does something the last one could not, and the headlines call it a breakthrough. We wait for the story of the clever idea behind it and it never quite comes. The company mostly talks about how many chips it bought and how much text it used. If the leap came from a new idea, we would expect to hear it. What if there was no new idea?
What Scaling Laws means
Scaling laws are the measured finding that a model's performance improves in a smooth, predictable way as we give the same training method more data, more computing power and more internal settings. The curve can be drawn before the model is built, which is why the labs plan spending years ahead. Owning this changes how we read the news and plan around it. We stop waiting for a breakthrough to explain each new capability and start asking what the next order of magnitude will buy. We stop assuming the current tool's limits are permanent, and we stop assuming they are about to vanish for free. Before: someone must have invented something. After: someone bought the next point on a curve, and the curve has a price.
Scaling Laws examples: where it shows up
- At work: the plan that says "the tool cannot do this yet, so we are safe" has a shelf life measured in the next model's budget.
- With money: the enormous sums flowing into chips and data centres are bets on a curve that has held so far, and a curve is not a guarantee.
- In the news: when a lab announces a model ten times larger, the story is not the size; it is that the size was chosen because the gain was predicted.
Where it sits
Concept 4 of 4 in Inside the Machine, a journey in the course Understand AI.
See where Scaling Laws sits on the sky →
Nearest on the sky
A filled dot marks a star that is already open to walk.
Knowing it isn't owning it.
Connect Scaling Laws to your mental models and apply it to your world today.
The first one is on us. No signup, no card.