Supercomputing for Artificial Intelligence: Foundations, Architectures, and Scaling Deep Learning Workloads (Paperback) (ISBN-13: 9798319328359)

The definitive practical guide to running Artificial Intelligence at scale.
Learn how to train deep learning models and LLMs on GPUs, clusters, and supercomputers with real tools like PyTorch, CUDA, and SLURM.

Are you ready to go beyond the basics of AI and truly harness the power of large-scale computing? Supercomputing for Artificial Intelligence is your practical guide to mastering the infrastructures, tools, and techniques needed to scale deep learning systems-from neural networks to large language models (LLMs).

Designed for graduate students, AI researchers, data scientists, and engineers, this book bridges the gap between high-performance computing (HPC) and real-world AI applications. Whether you're working in academia, industry, or exploring advanced AI on your own, you'll find clear explanations and hands-on examples built around tools like PyTorch, CUDA, MPI, SLURM, and multi-GPU distributed training.

With over 650 pages of rigorously tested content, this book takes you on an end-to-end journey through:

• The foundations of supercomputing and its role in AI workloads

• Practical GPU programming with CUDA and distributed systems

• Parallel programming with MPI on modern clusters

• Efficient training of neural networks, CNNs, and Transformers

• Performance optimization for deep learning at scale

• Distributed training with PyTorch DistributedDataParallel (DDP)

• Building and scaling LLMs using real biomedical and NLP datasets

• Jupyter, Google Colab, and Hugging Face workflows

• Deployment and inference strategies for modern LLMs

All source code, configuration files, and job scripts are available in a public GitHub repository. The material is field-tested through years of teaching and research at the Barcelona Supercomputing Center, and can be applied on local GPU setups, cloud platforms, and HPC clusters.

This book is ideal for:

• Instructors looking for practical material for AI and HPC courses

• Students and professionals wanting to learn how to run AI at scale

• Engineers transitioning from standard AI workflows to distributed environments

• Researchers working on LLMs and interested in reproducible pipelines

Do I need a supercomputer to use this book? Not at all. While some examples are run on large systems like MareNostrum, some code is designed to scale-from a single GPU to a full HPC node. You'll find guidance for running experiments in Google Colab and containerized environments.

Whether you're teaching AI, training models at scale, or simply curious about the invisible infrastructure powering today's most powerful AI systems, this book is your companion to understanding and leveraging supercomputing for artificial intelligence.

Start scaling your deep learning models today-this is not just a book, it's your gateway to HPC for AI.

What's inside:

• 650+ pages of real-world content tested in supercomputing classrooms

• Hands-on examples with PyTorch, CUDA, MPI, and SLURM

• Full GitHub access with ready-to-run scripts and datasets

• Workflows adapted for Google Colab, and HPC clusters