A concept · Inside the Machine · Computing
The Training Pipeline
From raw text-predictor to helpful assistant, grown stage by stage.
The chatbot at work is unfailingly polite, declines certain requests, and formats every answer as a tidy list. It reads like a well-briefed employee. Then someone finds a way of phrasing a question that gets it to say something it would never say to us, and the polish falls away in one sentence. A person's manners do not switch off like that. What was the politeness sitting on top of?
What The Training Pipeline means
The training pipeline is the sequence of stages that turns a blank network into an assistant: first a long stage of predicting the next word across a huge slice of the internet, then shorter stages that teach it a format and shape it towards answers people rate well. Nearly everything it knows comes from the first stage; nearly everything about how it behaves comes from the later ones. Owning this changes what we expect. We stop treating the assistant's manners as evidence of its knowledge, and its knowledge as evidence of its manners, because the two were installed separately. We start noticing that a refusal is a trained reflex rather than a judgement. Before: a helpful expert. After: a vast predictor wearing a trained layer of helpfulness, and the layer has seams.
Not to be confused with
The training pipeline is not the inference pipeline. Training is the months of building the model, stage by stage, and it happens once. Inference is what happens every time we type a message: the finished model producing an answer, unchanged by it.
The Training Pipeline examples: where it shows up
- At work: the tool that answers our customers was made agreeable in its last stage of training, and agreeable is not the same as right.
- In the news: a story about a model that was "jailbroken" is a story about the first stage showing through the later ones.
- In our own careers: a tool that seems to know our field learned it from public text long before it learned to be helpful, so its knowledge stops where the public text stopped.
Where it sits
Concept 3 of 4 in Inside the Machine, a journey in the course Understand AI.
See where The Training Pipeline sits on the sky →
Nearest on the sky
A filled dot marks a star that is already open to walk.
Knowing it isn't owning it.
Connect The Training Pipeline to your mental models and apply it to your world today.
The first one is on us. No signup, no card.