Kubernetes Cluster with k3s

Install k3s on Spark 1 (Master)

curl -sfL https://get.k3s.io | \
  INSTALL_K3S_EXEC="--write-kubeconfig-mode 644 --disable=traefik" \
  sh -

Configure kubectl

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" >> ~/.bashrc

Get the Join Token

sudo cat /var/lib/rancher/k3s/server/node-token

Join Spark 2 as Worker

curl -sfL https://get.k3s.io | \
  K3S_URL=https://192.168.86.30:6443 \
  K3S_TOKEN=<TOKEN_FROM_SPARK1> \
  sh -

Assign Worker Role Label

kubectl label node spark-7229 node-role.kubernetes.io/worker=worker

Verify Cluster

kubectl get nodes
# Expected:
# NAME         STATUS   ROLES                  VERSION
# spark-720e   Ready    control-plane,master   v1.35.5+k3s1
# spark-7229   Ready    worker                 v1.35.5+k3s1

Install Helm

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm version

Shell Completions & kubectl Aliases

Enable tab-completion and a small set of kubectl/helm aliases you will lean on for the rest of the book. Completions are guarded by command -v, so lines for tools you haven't installed yet (gh, uv, poetry) are no-ops until those tools land on PATH.

Option A — One-liner (recommended)

Download and run the idempotent installer script (setup-completions.sh) on both Sparks — it installs both completions and aliases:

curl -fsSL https://mohnishbasha.github.io/dgx-spark-bundle/books/from-box-to-cluster/scripts/setup-completions.sh | bash

Safe to re-run — the script uses two marker blocks (completions, aliases) and skips whichever is already installed.

Option B — Manual: completions

{
  echo 'command -v kubectl >/dev/null && source <(kubectl completion bash)'
  echo 'command -v helm    >/dev/null && source <(helm completion bash)'
  echo 'command -v gh      >/dev/null && source <(gh completion -s bash)'
  echo 'command -v uv      >/dev/null && eval "$(uv generate-shell-completion bash)"'
  echo 'command -v poetry  >/dev/null && source <(poetry completions bash)'
} >> ~/.bashrc

source ~/.bashrc

Option C — Manual: kubectl / helm aliases

The k alias is worth its weight in gold — you will type it thousands of times. The complete -F __start_kubectl k line rewires kubectl's completion so k Tab works exactly like kubectl Tab.

{
  echo "alias k='kubectl'"
  echo "alias kgp='kubectl get pods'"
  echo "alias kgs='kubectl get svc'"
  echo "alias kgn='kubectl get nodes'"
  echo "alias kga='kubectl get all'"
  echo "alias kd='kubectl describe'"
  echo "alias kl='kubectl logs'"
  echo "alias kx='kubectl exec -it'"
  echo "alias kaf='kubectl apply -f'"
  echo "alias kdel='kubectl delete'"
  echo "alias kns='kubectl config set-context --current --namespace'"
  echo "alias h='helm'"
  echo 'command -v kubectl >/dev/null && complete -o default -F __start_kubectl k'
  echo 'command -v helm    >/dev/null && complete -o default -F __start_helm    h'
} >> ~/.bashrc

source ~/.bashrc

Cheat sheet:

Alias	Expands to	Use
k	kubectl	Everything.
kgp	kubectl get pods	kgp -A for all namespaces.
kgn	kubectl get nodes	Check spark-720e / spark-7229 readiness.
kga	kubectl get all	Everything in the current namespace.
kd	kubectl describe	kd pod <name>.
kl	kubectl logs	kl -f <pod> to tail.
kx	kubectl exec -it	kx <pod> -- bash.
kaf	kubectl apply -f	Apply a manifest.
kdel	kubectl delete	Delete a resource.
kns	kubectl config set-context --current --namespace	kns core-services — persists across sessions.
h	helm	Same pattern as k.

Verify with k TabTab, kgp, and kns kube-system. Repeat on Spark 2 if you plan to run kubectl from there as well.

NVIDIA GPU Operator

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update

helm install gpu-operator nvidia/gpu-operator \
  --namespace gpu-operator \
  --create-namespace

# Verify both GPUs visible
kubectl get nodes -o json | grep '"nvidia.com/gpu":'
# Expected: "nvidia.com/gpu": "1"  (one per node)

kubectl get nodes -o json | grep "nvidia.com/gpu.family"
# Expected: "nvidia.com/gpu.family": "blackwell"

Create Namespaces

kubectl create namespace core-services
kubectl create namespace monitoring