From Box to Cluster:
Building a Personal AI Supercomputer
with NVIDIA DGX Spark Bundle — from bare hardware to a production inference cluster running large language models on Kubernetes
Preface
Why We Wrote This Book
In early 2026, NVIDIA released the DGX Spark — a personal AI supercomputer the size of a Mac mini, packing a Grace Blackwell GB10 GPU with 128GB of unified memory into a single quiet desktop unit. For the first time, running frontier-scale AI workloads from a home lab or a startup office became genuinely practical.
We bought two of them.
What followed was weeks of hard-won lessons: first-boot quirks, ARM64 compatibility landmines, networking configurations that broke SSH, container images that silently ran on the wrong architecture, and tensor parallelism tuning that only worked after we understood the interplay between NCCL, Ray, and the k3s networking layer. None of this was fully documented in a single place.
This book is the documentation we wish we had when we started.
What This Book Covers
- Hardware setup — physical connections, first-boot wizard, static IP, SSH, Docker
- System updates — OS, CUDA 13.0, and driver updates via the DGX Dashboard
- Kubernetes — k3s, GPU Operator, Helm, namespace architecture
- Distributed inference — KubeRay, cross-node Ray cluster, ARM64 gotchas
- vLLM — tensor parallelism across both nodes, HuggingFace secrets
- AIBrix — AI gateway for routing, multi-tenancy, agent lifecycle
- Monitoring — Prometheus, Grafana, GPU metrics via DCGM Exporter
Who This Book Is For
This book assumes you are comfortable with a Linux terminal, have basic Kubernetes familiarity, and have purchased or are evaluating the NVIDIA DGX Spark Bundle. You do not need to be a DevOps engineer or CUDA expert — we explain every decision and command, including why certain approaches were tried and abandoned.
— Mohinish Shaikh & Sanwi Sarode, July 2026
Table of Contents
Part I — Node Setup
| Chapter | Title | Key Topics |
|---|---|---|
| 1 | Introduction — What You Are Building | Architecture, software stack, namespace layout |
| 2 | Hardware Setup and First Boot | Physical setup, static IPs, SSH, Docker |
| 3 | CUDA and System Updates | DGX Dashboard, CUDA 13.0, driver updates |
| 4 | Kubernetes Cluster with k3s | k3s, worker join, Helm, GPU Operator |
Part II — Model Serving
| Chapter | Title | Key Topics |
|---|---|---|
| 5 | KubeRay for Distributed AI | KubeRay operator, ARM64 image, RayCluster |
| 6 | vLLM Inference Engine | Tensor parallelism, API verification, models |
| 7 | AIBrix AI Gateway | Routing, multi-tenancy, ModelAdapter |
| 8 | Cluster Overview and Monitoring | Grafana, GPU metrics, roadmap |
Back Matter
| Section | Contents |
|---|---|
| Command Cheatsheet | Quick-reference kubectl, vLLM, port-forward, iptables commands |
| Troubleshooting | SSH loss, GPU visibility, ARM64 errors, model download failures |
| FAQ | Common questions about DGX Spark, models, and cluster setup |
| About the Authors | Mohinish Shaikh and Sanwi Sarode |