Karpathy AI Courses: A Learning Roadmap from micrograd to nanochat
Build understanding step by step, from basic computation and text prediction to an end-to-end workflow. This illustrates the article’s suggested learning route, not a new official Karpathy curriculum or a guarantee of learning outcomes.
Start with basic Python and derivatives, then work through micrograd, makemore, and GPT before exploring nanochat. If you do not code yet, begin with the concepts rather than a full training run.
The difficult part of following Andrej Karpathy's AI materials is often choosing a starting point. Should you learn Python first, jump into a GPT tutorial, or run every repository you find?
This guide recommends a sequence based on the official materials. It is an editorial learning roadmap, not a newly announced Karpathy curriculum or a promise of a fast career change. Leave autoresearch until you can evaluate experiments yourself.
Are Karpathy's AI courses suitable for complete beginners?
Neural Networks: Zero to Hero starts with neural-network fundamentals, but its first lecture assumes basic Python and a rough recollection of high-school calculus. Starting from the foundations of neural networks does not mean starting without any programming knowledge. Official course outline
Readers without an engineering background can begin conceptually: how data becomes numbers, how a model makes predictions, and why fluent output can still be wrong. If you want to train models yourself, first check whether you can:
- Read Python functions, loops, lists, and classes.
- Explain a slope with a simple example and describe how changing an input affects an output.
- Read an error message and find the line where a program stopped.
- Pause a video to attempt an exercise instead of simply watching to the end.
This is a suggested readiness checklist, not an official admissions requirement. If one item is unfamiliar, work on that gap. You do not need to finish an entire mathematics textbook before beginning.
The article suggests starting with Python and derivatives, then studying micrograd, makemore/GPT, and nanochat. Consider autoresearch as an optional extension only after developing evaluation skills. This is not a new official Karpathy curriculum or a guarantee of learning outcomes.
Step 1: Use micrograd to understand how a model learns
micrograd is a small automatic differentiation engine. It exposes the computation graph and backpropagation through scalar operations, with a small neural-network library built on top. Its compact implementation makes individual operations easier to follow; the repository explicitly identifies educational use as a possible purpose. micrograd repository
Focus on three questions: How do we measure a model's error? What does a gradient tell us? How do parameters change in response to it? Take a simple function, predict the effect of changing an input on paper, then compare your prediction with the program's output.
A useful milestone is being able to explain one cycle of prediction, loss calculation, gradient calculation, and parameter update without memorizing the implementation. If you can only copy and run the code, stay here a little longer before moving to larger models.
Step 2: Learn language modeling with makemore before building GPT
The official sequence introduces character-level language modeling through makemore, progressing from a bigram model to multilayer perceptrons while developing familiarity with tensors, gradients, and training. The GPT lecture recommends earlier makemore lessons to establish the autoregressive language-modeling and PyTorch foundations it assumes. Zero to Hero lesson sequence
Do more than ask whether the model generates text. Record whether training and validation data are separate, how samples change as the loss improves, and whether a result can be reproduced after you change a setting.
When you reach GPT, focus on how context influences the next-token prediction and what attention contributes. Draw your own diagram of text entering the model and becoming a prediction. Being able to explain that diagram is a more useful milestone than recognizing every technical term.
Step 3: Should you still learn nanoGPT? Know its deprecated status
Older guides often recommend nanoGPT as the next stop. Its README now includes a November 2025 notice describing the repository as old and deprecated and directing users to nanochat. The code remains available for historical reference, including its GPT model definition and training loop. Do not treat an older tutorial as current implementation guidance without checking. nanoGPT deprecation notice
If a course you are already following uses nanoGPT, it can still serve as reading material. If your next goal is to understand a more complete language-model pipeline, prioritize nanochat. When an old command fails, check the tutorial's date and software versions before assuming that you misunderstood the lesson.
Step 4: Connect training, evaluation, and inference with nanochat
nanochat covers tokenization, pretraining, finetuning, evaluation, and inference. Once you understand a model in isolation, it provides a way to examine how the stages fit together. Its documentation also includes a reduced-scale CPU and Apple Silicon example, while warning that this approach will not produce strong results. nanochat repository
Start by mapping how data and model files move through the pipeline; you do not need to rent a GPU immediately. Before running training, check hardware requirements, expected runtime, data sources, and a spending limit. A model that produces conversational responses does not automatically have the quality, safety, or operational readiness of a commercial chat service.
A useful milestone here is explaining each stage's input, output, and evaluation method. One impressive response is not sufficient evidence that the whole system works well.
Leave autoresearch until you can judge experimental results
The autoresearch program.md describes a bounded experiment: changes are limited to train.py, evaluation remains fixed, and each training run has a five-minute budget excluding startup and compilation. It also requires establishing a baseline first. autoresearch experiment specification
Hardware requirements differ, too. Upstream autoresearch currently requires a single NVIDIA GPU and has been tested on H100. Do not assume that nanochat's CPU/MPS support carries over to it. autoresearch platform requirements
The transferable lesson is experimental discipline: state a hypothesis, control a small number of variables, keep records, and compare results. If an agent performs the work, also define a total budget, an allowed modification scope, and stopping conditions while retaining environment permissions and safety controls. A five-minute training budget does not mean the entire experiment takes five minutes or authorize an unlimited sequence of runs.
Choose a starting point for your background
If you have never programmed: Begin by understanding the difference between training and inference while learning Python. Aim to explain a small micrograd example rather than immediately build a chatbot.
If you code but are new to deep learning: Start with micrograd, continue through makemore and GPT, and practice recording model performance. Being able to read code is not a reason to skip gradients or the purpose of validation data.
If you already train models: You can begin by reading nanochat, identify unfamiliar stages, and return to the relevant lessons. Consider autoresearch-style automation only once you can assess whether an evaluation is meaningful.
After choosing a starting point, complete one small, checkable task: calculate a gradient by hand, explain a prediction, or draw part of a training pipeline. Measure progress by what you can independently explain and inspect, not by the number of repositories you have bookmarked.
FAQ: Do you need a GPU or have to complete every lesson?
Can I start without a GPU? Yes. Reading the materials, working through small examples on paper, and understanding the model pipeline are useful first steps. Choose a specific exercise before deciding what hardware it requires.
Must I complete everything in order? The suggested sequence builds foundations, but it also works as a self-assessment checklist. Move ahead when you can clearly explain a stage. Return to the missing concept when you find yourself copying code without understanding the result.
Can AI help me write the exercises? It can help explain errors or suggest a next step. First make your own prediction, then compare it with the answer. Finally, try again without hints to check what you have actually learned.