In plain English
Instead of training a small model from scratch, you let it learn from a big, capable one. The small model picks up much of the big one's skill while staying fast and cheap to run.
Example
A company has its large model answer a million customer questions, then trains a small model on those answers. The small one handles most questions for a fraction of the cost.
Not the everyday meaning
Purifying a liquid by boiling it and collecting the vapor.
Why it matters
Distillation is a big reason small, cheap models keep getting better. It's also controversial: AI labs' terms often forbid using their models' answers to train a rival model.
Common confusion
Distillation vs. fine-tuning. Fine-tuning trains a model on new examples, often written by people. Distillation specifically trains a smaller model to imitate a bigger one. Both are kinds of extra training.