Building LLM From Scratch

This section describes my ongoing journey of building a Large Language Model from scratch. I'm currently working through the core stages—dataset preprocessing, tokenization, architecture design, and setting up training pipelines. This hands-on project is helping me dive deep into the inner workings of transformers, language modeling objectives, and model scaling. As I progress, I aim to get a thorough understanding of what drives today’s generative AI systems.

---

Table of contents

• Part 1: How LLMs Read Text: Tokenization, BPE, and Word Embeddings Explained
• Part 2: Self Attention Explained without Trainable Weights
• Part 3: Attention From Scratch: Self-Attention to Multi-Head Attention