cs249r_book

The book addresses the gap between training models in research environments and running them reliably in production covering topics including ML hardware (CPUs, GPUs, TPUs, NPUs), model compression and quantization, efficient inference, on-device ML for edge and embedded systems, and the software systems (compilers, runtimes, serving frameworks) that connect model weights to real-world applications.

The curriculum is organized to follow the full ML system stack: from the silicon and memory hierarchies that determine throughput, through the frameworks and compilers that transform model graphs into optimized executables, to the serving infrastructure and monitoring systems that keep models performant after deployment.

Special attention is given to TinyML deploying ML models on microcontrollers and embedded devices with milliwatt power budgets reflecting the growing importance of AI at the edge in IoT, medical devices, and autonomous systems.

The textbook is freely available online and is used in university courses globally as a companion to traditional ML theory curricula.

It is particularly valuable for ML engineers who understand how to train models but want deeper knowledge of the systems decisions that govern inference cost, latency, and reliability in production.

The open-access model reflects a commitment to making ML systems education accessible beyond institutions with expensive textbook budgets.