Question 1

What is the NVIDIA DGX Spark Bundle?

Accepted Answer

The NVIDIA DGX Spark Bundle is two DGX Spark personal AI supercomputers sold together. Each unit has a Grace Blackwell GB10 GPU with 128GB of unified memory and a 20-core ARM64 CPU. Combined, the bundle provides 256GB of total GPU memory and 40 CPU cores, connected via a high-speed QSFP RDMA cable using ConnectX-7.

Question 2

How do you set up a k3s cluster on DGX Spark?

Accepted Answer

Install k3s on Spark 1 (master) with: curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--write-kubeconfig-mode 644 --disable=traefik' sh -. Then get the join token from /var/lib/rancher/k3s/server/node-token and join Spark 2 as a worker with K3S_URL and K3S_TOKEN environment variables set. Do not use the --docker flag; k3s uses containerd natively.

Question 3

Which Docker image should I use for Ray on DGX Spark (ARM64)?

Accepted Answer

Use nvcr.io/nvidia/vllm:25.09-py3 — not the standard rayproject/ray image, which is x86-only and will fail with 'exec format error' on DGX Spark's ARM64 architecture. The NVIDIA vLLM image is ARM64-native, includes Ray 2.49.2, vLLM 0.10.1.1, and CUDA 13.0.

Question 4

How long does vLLM take to start on DGX Spark?

Accepted Answer

First startup takes approximately 25 minutes: about 20 minutes to download the 7B model weights from HuggingFace, then 5 minutes for model loading and tensor parallel initialization. Subsequent startups with a cached model take 3-5 minutes.

Question 5

What is AIBrix and why should I use it with vLLM?

Accepted Answer

AIBrix is an open-source AI inference gateway that sits between your AI applications and vLLM. It provides request routing, multi-tenancy with namespace isolation, agent lifecycle management, and GPU utilization-aware scheduling. This allows multiple applications to share one vLLM instance safely, with per-namespace quota enforcement.

Question 6

What models can you run on two DGX Spark units?

Accepted Answer

With 256GB of combined unified memory across two DGX Sparks, you can run models including Qwen2.5-7B-Instruct, Qwen2.5-72B-Instruct, Llama 3.3 70B Instruct, Qwen3-235B-A22B-NVFP4 (quantized), Nemotron-3-Super-120B, and Llama 405B in quantized form. vLLM distributes model weights across both GPUs using tensor parallelism.

Question 7

How do I fix SSH being blocked after k3s restarts on DGX Spark?

Accepted Answer

k3s modifies iptables rules on restart which can block SSH. Flush the rules with: sudo iptables -F && sudo iptables -X && sudo iptables -P INPUT ACCEPT && sudo iptables -P FORWARD ACCEPT && sudo iptables -P OUTPUT ACCEPT. To prevent this permanently, add persistent rules before k3s installation: sudo iptables -I INPUT -s 192.168.86.0/24 -j ACCEPT and save with sudo netfilter-persistent save.

Question 8

How do you install the NVIDIA GPU Operator on k3s?

Accepted Answer

Install with Helm: helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update && helm install gpu-operator nvidia/gpu-operator --namespace gpu-operator --create-namespace. The operator automatically installs the NVIDIA Container Toolkit, Device Plugin, DCGM Exporter, and GPU Feature Discovery on every node.

Question 9

Why use k3s instead of full Kubernetes?

Accepted Answer

k3s provides the full Kubernetes API in a single lightweight binary, eliminating the operational complexity of managing separate control-plane components and etcd. For a 2-node personal cluster, the tradeoff is that Flannel CNI (k3s default) does not support RDMA/RoCE, so NCCL falls back to TCP for cross-GPU communication. A future upgrade to RKE2 + Cilium will enable full RDMA support.

Frequently Asked Questions