Prepare for the interview that asks how fast your kernel should run.
GPU performance interviews are unlike the ones most engineers have practised for. You may be asked to write a kernel on a shared screen with no compiler, then say how fast it ought to go before anyone has measured it. You may be shown a profile and asked what you would do next. You may be asked why a model's decode speed barely changed when the team moved to a GPU with twice the FLOP rate.
Algorithm practice does not prepare you for this. Neither do CUDA tutorials, which teach you how to make a kernel work but rarely how to reason about why it is slow.
This book starts from the arithmetic: what a kernel can reach on a named GPU, and why. Then it builds each kernel from a correct naive version to a fast one, one justified step at a time, because those steps are what interviewers want to hear.
Inside you'll find:
- The performance model: roofline, arithmetic intensity, latency hiding, occupancy and the numbers to carry into the room, with worked estimation drills
- CUDA in depth: execution, coalescing, shared memory and bank conflicts, warp-level programming, atomics, streams, tensor cores, and what Hopper and Blackwell changed
- Triton: the programming model, how to make it fast, and when to switch to CUDA
- Version ladders for the kernels interviewers ask for: reduction, scan, transpose, matrix multiplication, softmax, LayerNorm, FlashAttention and more
- Profiling and debugging: a workflow for Nsight Systems and Nsight Compute, correctness bugs, and 20 worked debugging scenarios
- System-level performance: training and inference arithmetic, MFU, decode floors, and 8 open-ended design questions
- Interview practice: over 240 questions with answers, rubrics of strong signals and red flags, and preparation plans from three days to eight weeks
- Reference appendices: hardware tables, CUDA and Triton quick reference, a glossary and further reading
Who it's for: engineers preparing for GPU performance, kernel, inference and ML-compiler roles, including CUDA developers moving into AI, ML engineers moving down the stack, and systems programmers moving across to GPUs. You should be able to write C++ and Python. Book Two of The AI Infrastructure Library; it can be read on its own.
