Build a Large Language Model (from Scratch)

Sebastian Raschka

Understand LLMs by building one. Code the tokenizer, self-attention, and transformer blocks yourself, pretrain a small model, then fine-tune it.

A hands-on book that walks through implementing a GPT-style LLM in PyTorch step by step: tokenization, attention, the transformer block, pretraining, and fine-tuning for classification and instruction following.