Scaling Book Solutions
Typst transcriptions of detailed solutions to the exercises in
How to Scale Your Model
.
Chapter 1: A Brief Introduction to Roofline Analysis
Chapter 2: How to Think About TPUs
Chapter 3: Sharded Matrices and How to Multiply Them
Chapter 4: All the Transformer Math You Need to Know
Chapter 5: How to Parallelize a Transformer for Training
Chapter 6: Training LLaMA 3 on TPUs
Chapter 7: All About Transformer Inference
Chapter 8: Serving LLaMA 3 on TPUs