Think of a large language model like a giant mixing board in a recording studio. It has billions of tiny knobs, and each one shifts the final sound a little. I just watched a video where Anthropic researcher Jeremy Berman sits down with Matthew Berman (no relation) and explains how turning those knobs gets you to models like ChatGPT and Claude.
The part that surprised me most is that the expert is self-taught. He doesn’t have a PhD. He followed his curiosity down rabbit holes until he understood how the whole system works.
🎛️ The Mixing Board, Mapped to Real Steps
- The knobs are parameters. A model is just a file of numbers plus a small Python program that wires them together. A “2 trillion parameter model” really does mean 2 trillion knobs.
- The sound check is loss. Jeremy starts with a toy example: predicting New York apartment prices from location, size and number of baths. The model makes a guess and compares it to the real price. The gap between the two is called “loss.”
- The auto-engineer is backpropagation. Tools like PyTorch track how much each knob affected the final answer. Then they nudge every knob a little in the right direction. The “learning rate” sets how big each nudge is. Repeat that thousands of times and the error shrinks.
📚 Recording the Whole Internet (Pre-training)
A language model works the same way, but it predicts the next word instead of a price. Its last layer gives a probability to every possible next token. Each training step makes the correct word slightly more likely.
Train it on the internet and you get a “base model,” which is pure autocomplete. When the creator typed “Matthew Berman said,” the model made up a quote Matthew never said. It isn’t trying to be truthful. It’s only guessing what usually comes next.
🎤 Teaching It to Be the Assistant (Post-training)
Researchers noticed they could trick a base model by typing “User: … Assistant:” and letting it fill in the rest. So they wrote lots of example chats in a special format and ended each one with a “stop token.” After training on those chats, the model takes on an assistant persona and learns when to stop talking.
Here’s something I didn’t know: your chat messages still get converted into that formatted text before the model sees them.
🧠 Practice Sessions (Reinforcement Learning)
Instruct models still blurt out answers. One was asked 398 × 923 + 2901 and confidently said about 37,000. The real answer is roughly 370,000.
Adding “Let’s think step by step” helped. The bigger jump came with OpenAI’s o1 and DeepSeek’s R1 paper. R1 used a method called GRPO, which works like this:
- The model tries the same problem many times, say 100.
- Each answer gets a score for following the format and getting the right result.
- Each score is compared to the group average. That difference is the “advantage.”
- Above-average answers get made more likely and below-average ones get made less likely.
The model is learning from its own answers, which is called “on-policy” training. That means the changes are gentle and it forgets less of what it already knew.
🚀 How to Try It Yourself
- Open a free Google Colab notebook with a free T4 GPU, which is exactly what the expert used.
- Load a small open-weights Qwen model (about 0.5B parameters).
- Give the same prompt to the base version and the instruct version and compare the answers.
- Ask Claude or ChatGPT to write a PyTorch training loop and explain it line by line.
💡 Use Cases
- Explain AI to your team without jargon. The knob analogy works really well.
- Pick the right tool. For a single number like a price, a plain neural network gets more out of the same compute than an LLM.
- Understand hallucinations. They make sense once you see the autocomplete engine underneath.
- Think about what’s next. Jeremy bets that models trained to reason over long stretches will discover new knowledge, the way Einstein built new ideas step by step.
The full video goes deeper on every stage and includes live code demos. If you’ve wanted to understand how these models work from the ground up, it’s worth watching.