Build a tiny language model from scratch in JavaScript.
Work through bite-sized coding exercises that teach core attention mechanics directly. You write real code, run deterministic tests, and build one model step by step from basic token counting to a mini GPT.
Curriculum scope is grounded in Attention Is All You Need (Vaswani et al., 2017) and lessons are written to stay faithful to the paper.
Attention Flow
Tokens -> Q/K/V projections -> scores -> softmax weights -> context vectors.
Course at a glance
Total Modules
8
Total Lessons
32
Estimated Duration
6.6h
Learning Style
Code + Tests
What You Will Learn
Tokenization and text-to-ID pipelines.
Unigram and bigram language model baselines.
Embeddings, context windows, and trainable neural LMs.
Attention (Q/K/V), masking, and multi-head design.
Transformer blocks, training loops, and tiny GPT generation.
How The Platform Works
Read the lesson README in plain English.
Implement the target function in Monaco editor.
Run tests and inspect case-level feedback.
Save attempts locally and track completion in curriculum.
Iterate toward clean passes and deeper intuition.
Who This Is For
Built for builders who want practical LLM intuition, not just definitions.
If you are a student, self-taught engineer, or backend/frontend developer curious about transformers, this course gives you a concrete path from zero ML background to implementation confidence.
You can move at your own pace, repeat lessons, compare your code with starter scaffolds, and revisit failed attempts as you sharpen your understanding.
Every unit focuses on one major concept so you always know what changed, why it changed, and how to validate it with tests.
What You Can Build By The End
Explain how token IDs become vectors and how context windows shape the training signal.
Implement and debug self-attention, masking, and multi-head attention without magic helper code.
Train a compact GPT-like model and generate text with sampling to observe model behavior.
Read tensor shapes and test failures quickly so you can fix logic errors with confidence.
Course Roadmap
Each module builds on the previous one.
Module 01
First Wins (Coding Basics)
4 lessons
Module 02
First Working LM (Bigram)
4 lessons
Module 03
Trainable Neural LM
4 lessons
Module 04
Embeddings and Better Context
4 lessons
Module 05
Attention Fundamentals
4 lessons
Module 06
Transformer Block
4 lessons
Module 07
Tiny GPT Training Loop
4 lessons
Module 08
Capstone Quality and Interpretability
4 lessons
Frequently Asked Questions
Quick answers for learners deciding whether this curriculum fits their goals.