The Context Window

My notes on AI — rewritten from first principles, in code, with interactive demos.

Abhishek Bisht
Abhishek Bisht
Applied AI Research Engineer. Building and writing about the things that matter.
Deep Learning

Neurons All the Way Down

One neuron. Then a network. Then GPT-2 — built by hand.

8 tutorials · micrograd → GPT-2
NLP

Tokens, Transformers & Truth

How language models work, misbehave, and get steered.

14 tutorials
NLP

The Transformer Deep Cuts

The internals most courses skip — efficiency, alignment, multimodal.

14 tutorials · 5 arcs
Architectures

Architecture Deltas: 2017 → 2026

What each landmark model actually invented — 2017 to now.

21 milestones · 6 deep dives · single page
Architectures

The Architecture Atlas

Fifty-one LLM diagrams. Zoom, search, compare side by side.

51 architectures · gallery + compare
Architectures

The Evolving Transformer

One codebase. Paper by paper. Watch the architecture grow.

16 architectures · 55+ deltas · code evolution
Pre-Training

The Training Ground

Inside the pretraining forward pass — every layer, every FLOP.

1 tutorial · dense transformer deep-dive
Deep Learning

Decoding DeepSeek

MLA, MoE, MTP, FP8 — rebuilt from first principles.

12 tutorials · 6 arcs · from transcript to blueprint
RL + Reasoning

Reasoning from Scratch

From chain-of-thought to GRPO. Teach a model to think.

12 tutorials · 4 arcs · CoT → RL → GRPO → R1-Zero
Inference

Beyond the Forward Pass

Serving at scale — paging, batching, speculation, VRAM.

1 tutorial · vLLM deep-dive
Search

Finding Needles

From BM25 to agents that search for themselves.

19 tutorials · 6 arcs
Interview

The Gauntlet

Seventy-five interview questions. Reveal, track, quiz yourself.

75 questions · 16 topics · self-test
Research

The Paper Trail

Every paper cited here — searchable, tagged, one click away.

92 papers · 9 blog series · searchable & paginated
Python

Python Under the Hood

Python that actually matters for ML — interactive, in the browser.

16 chapters
Python

The Sandbox

A Python REPL in your tab. No install.

Pyodide · instant
Archives

The Archives

Early notes, 2018–2024. The path that led here.

76 posts · 2018 – 2024 · year-wise
Course

AI Researcher Course

Six months. Ten phases. Applied AI research, end to end.

10 phases · 6 months · syllabus + tracker