Learn AI Layer by Layer
An interactive guide to understanding AI from first principles.
Subscribe to messyprogress.substack.com to find out when new chapters get released.
Introduction
What this is and how to use it
Computation
Everything is numbers
Optimization
The power of incremental improvement
Neural Networks
Building a brain
Vectors
Describing the world with numbers
Embeddings
From words to meanings
Next-Word Prediction
Understanding by predicting
Attention
Letting words look at each other
Positional Encoding
Where am I?
Transformers
One architecture to rule them all
Post-Training
Teaching a model to be useful
Conversations
One long document, pretending to be a chat
Reasoning
Thinking by talking to yourself
Agents
Getting things done
Matrix Math
Thinking by rotating
Coming soon
Making Training Work
Why training almost doesn't work
Coming soon
Mixture of Experts
Only wake the specialists you need
Coming soon
Long Context
Remembering a million words
Coming soon
Inference and Hardware
Running models fast
Coming soon
Interpretability
Looking inside the mind
Coming soon
Reinforcement Learning
Learning from experience
Coming soon
Self-Play
Getting better by beating yourself
Coming soon
Alignment
Teaching AI right from wrong
Coming soon
Distillation and Synthetic Data
Models teaching models
Coming soon
Image Comprehension
Teaching machines to see
Coming soon
Image Generation
Drawing pictures
Coming soon
World Models
Simulating reality
Coming soon
Audio
Listening, speaking, and singing
Coming soon
Hallucination and Grounding
Making stuff up
Coming soon
Context Management
Getting the right information
Coming soon