A concept · How AI Thinks · Computing
The Generation Loop
A model never writes a paragraph; it guesses the single next token, again and again.
We ask a chatbot the same question twice and get two different answers, both fluent, both sure. A colleague asks it to write a report and the first half is sharp and the second half slowly goes off the rails. When we ask it to explain the reasoning behind an answer, it gives one, but the answer came first and the reasoning after. If it were writing the way we write, none of this would happen. So how is it writing?
What The Generation Loop means
The generation loop is the fact that a language model produces text one small piece at a time: it works out the most likely next fragment, adds it to the page, and starts again from the whole page, with no plan for the sentence after. Owning this changes how we use the tools. We stop reading a confident paragraph as a finished thought and start reading it as a chain of guesses that could have gone another way. We start giving the loop a good start, because everything after follows from it, and we stop asking for the reasoning behind an answer as if it had been kept somewhere. Before: it wrote that. After: it arrived there, step by step, and could go elsewhere tomorrow.
The Generation Loop examples: where it shows up
- At work: the long document a tool drafted for us started well and drifted, because each paragraph was built from the one before and small slips compounded.
- At home: the same question to the same chatbot gives our partner a different answer, and neither answer is the "real" one.
- In the news: stories about a model that "made up" a court case or a citation are stories about a loop that kept producing the likeliest next words past the point where any facts remained.
Where it sits
Concept 3 of 3 in How AI Thinks, a journey in the course Understand AI.
See where The Generation Loop sits on the sky →
Nearest on the sky
A filled dot marks a star that is already open to walk.
Knowing it isn't owning it.
Connect The Generation Loop to your mental models and apply it to your world today.
The first one is on us. No signup, no card.