Chapter 2 · Part I

Hardware Setup and First Boot

Hardware Specifications

ComponentSpecification
GPUNVIDIA GB10 Blackwell (Grace Blackwell Superchip)
GPU Memory128GB unified (shared with CPU)
CPU20-core ARM64 (Grace CPU)
StorageNVMe SSD
NetworkConnectX-7 (QSFP, RDMA-capable)
OSDGX OS (Ubuntu 24.04.4 LTS)
Kernel6.17.0-1018-nvidia

Physical Setup

  1. Connect both Sparks to power — they power on automatically when plugged in
  2. Connect a monitor to Spark 1 only via HDMI
  3. Connect a USB-C hub to Spark 1 and attach keyboard + mouse
  4. Connect both Sparks to your router via Ethernet
  5. Connect the QSFP cable between the ConnectX-7 ports on each unit
USB-C Note
The DGX Spark has USB-C ports only. A USB-C to USB-A hub is required for standard keyboards and mice.

First Boot Setup Wizard

  1. Select keyboard layout
  2. Accept the license agreement
  3. Create your user account (username: moonlit used in this guide)
  4. Connect to network (Ethernet recommended)
  5. System downloads and installs complete software image automatically
Warning
Do not power off during the software download step. The download cannot be resumed if interrupted. Allow 15–45 minutes.

Static IP Assignment

Configure static IPs on both Sparks before any Kubernetes setup. Dynamic IPs will break the cluster between sessions.

Spark 1 (master): 192.168.86.30 | Spark 2 (worker): 192.168.86.26

On Spark 1:

sudo nmcli con mod "$(nmcli -g NAME con show --active | head -1)" \
  ipv4.addresses 192.168.86.30/24 \
  ipv4.gateway 192.168.86.1 \
  ipv4.dns 8.8.8.8 \
  ipv4.method manual
sudo nmcli con up "$(nmcli -g NAME con show --active | head -1)"

On Spark 2:

sudo nmcli con mod "$(nmcli -g NAME con show --active | head -1)" \
  ipv4.addresses 192.168.86.26/24 \
  ipv4.gateway 192.168.86.1 \
  ipv4.dns 8.8.8.8 \
  ipv4.method manual
sudo nmcli con up "$(nmcli -g NAME con show --active | head -1)"

Persistent iptables Rules

Add these on both Sparks before k3s installation to prevent SSH from being blocked after k3s restarts:

sudo iptables -I INPUT -s 192.168.86.0/24 -j ACCEPT
sudo iptables -I FORWARD -s 192.168.86.0/24 -j ACCEPT
sudo apt install iptables-persistent -y
sudo netfilter-persistent save

SSH Setup on Spark 2

# On Spark 2 — switch from socket-activated to persistent SSH
sudo systemctl stop ssh.socket
sudo systemctl disable ssh.socket
sudo systemctl enable ssh
sudo systemctl start ssh

Enable password auth in /etc/ssh/sshd_config: set PasswordAuthentication yes, then sudo systemctl restart ssh.

# From Spark 1 — verify connection
ssh moonlit@192.168.86.26

Docker Configuration

Run on both Sparks — configures cgroup v2 mode and NVIDIA runtime:

sudo python3 -c "
import json, os
path = '/etc/docker/daemon.json'
d = json.load(open(path)) if os.path.exists(path) else {}
d['default-cgroupns-mode'] = 'host'
d['default-runtime'] = 'nvidia'
d['runtimes'] = {'nvidia': {'path': 'nvidia-container-runtime', 'args': []}}
json.dump(d, open(path, 'w'), indent=2)
"
sudo systemctl restart docker
sudo usermod -aG docker $USER
newgrp docker
Previous← Ch 1: Introduction