Media Generation — ComfyUI and GPU Time-Sharing

The cluster has one GPU. Layer 10 gave it to Ollama for LLMLarge Language ModelA model trained to predict text, served behind a chat or completion endpoint. On Frank these run locally on the GPU node rather than against a hosted provider. inference. This layer adds a second consumer — ComfyUI for diffusion-based media generation — and a mechanism to share the hardware between them.

    flowchart LR
  subgraph GPU[gpu-1 — RTX 5070 Ti, 16GB]
    Ollama[ollama<br/>replicas: 1, default]
    Comfy[comfyui<br/>replicas: 0, on demand]
  end
  subgraph Switcher[GPU Switcher — 192.168.55.214]
    Go[Go web app<br/>50m CPU, 32Mi mem]
    RBAC[ClusterRole<br/>patch deployments]
  end
  subgraph ArgoCD[ArgoCD]
    ID[ignoreDifferences<br/>spec.replicas]
  end

  Go -->|scale up| Comfy
  Go -->|scale down| Ollama
  ID -->|doesn't fight| Go
  

The constraint: the RTX 5070 Ti has 16GB of GDDR7Graphics Double Data Rate 7The memory standard on current NVIDIA consumer cards. It sets how much model weight the card can hold at once — the 16GB figure that decides which quantisation a model has to run at.. LTX-2.3 needs 8-12GB. Ollama with a 9B model uses 6-7GB. Both cannot run simultaneously.

The solution is time-sharing: scale one workload to zero, let the other use the full GPU, swap when needed. Both Deployments request nvidia.com/gpu: 1, so Kubernetes will not schedule them concurrently.

The Three Apps

ComfyUI

ComfyUI is a node-based visual editor for diffusion model pipelines. Text-to-video (LTX-2.3), text-to-image (SDXL, Flux), text-to-audio (Stable Audio). Exposes both a visual graph editor and a REST API.

containers:
  - name: comfyui
    image: ghcr.io/ai-dock/comfyui:latest
    resources:
      requests:
        nvidia.com/gpu: 1
      limits:
        nvidia.com/gpu: 1

Key decisions:

• 100Gi PVCPersistentVolumeClaimA Kubernetes request for durable storage. The pod names a claim and the storage layer — Longhorn on Frank — binds real disk behind it, so the data outlives the pod. on Longhorn gpu-local — models are large (LTX-2.3 ~4GB, SDXL ~7GB). Mounts at /workspace.
• Starts at 0 replicas — Ollama is the default. ComfyUI only runs when switched via the GPU Switcher.
• Node affinity to gpu-1.

GPU Switcher

A custom Go web application that manages time-sharing. It reads a WORKLOADS env var defining managed workloads:

WORKLOADS=ollama:ollama:ollama,comfyui:comfyui:comfyui

Format: name:namespace:deployment. On each status check, it queries the K8s API for each Deployment’s replica count. Activating a workload scales it to 1 and all others to 0.

The ArgoCD Problem

ArgoCD’s self-heal normally detects drift between Git and the live cluster. If Git says replicas: 0 for ComfyUI but the Switcher just scaled it to 1, ArgoCD would scale it back.

The fix: ignoreDifferences on spec.replicas in both Application CRsCustom ResourceAn object of a type Kubernetes did not ship with, added by a CRD. Frank's ArgoCD Applications, Rollouts and Tekton Pipelines are all CRs.:

spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas

This tells ArgoCD that the GPU Switcher, not Git, is the authority for replica counts.

RBAC

The Switcher’s ServiceAccount needs cross-namespace access via ClusterRole:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gpu-switcher
rules:
  - apiGroups: ["apps"]
    resources: ["deployments", "deployments/scale"]
    verbs: ["get", "list", "patch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list"]

Building the Image

Cross-compiling a Go binary for amd64 from an arm64 Mac:

# Cross-compile natively (no QEMU)
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o gpu-switcher-linux-amd64 .

# Package into amd64 distroless runtime
docker buildx build --platform linux/amd64 \
  -t ghcr.io/derio-net/gpu-switcher:v0.1.1 \
  --push .

First attempt used Docker’s --platform on the full multi-stage build, which ran the Go compiler under QEMUQuick EmulatorThe emulator and virtualiser underneath most Linux virtualisation, including KubeVirt's virtual machines. — and crashed with a SIGSEGVSignal Segmentation ViolationThe signal sent when a process touches memory it does not own. Nearly always a bug in the program, not in its configuration. in the GC. The working approach: compile natively with GOARCH=amd64, then use a single-stage Dockerfile that copies the pre-built binary.

Second attempt pushed an image with arm64 in its OCIOpen Container InitiativeThe body behind the standard image and runtime formats. "OCI registry" means any registry speaking that standard, not a specific vendor's. manifest despite containing an amd64 binary — Docker inherits manifest platform from the build host. Explicit --platform linux/amd64 fixed it.

Model Downloads

ComfyUI models must be downloaded into the PVC after first deployment:

kubectl exec -it -n comfyui deploy/comfyui -- bash
cd /workspace/ComfyUI/models
# LTX-2.3 video model
wget -P video_models/ https://huggingface.co/Lightricks/LTX-Video/resolve/main/ltx-video-2b-v0.9.5.safetensors
# SDXL base
wget -P checkpoints/ https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

Missteps

Recovery Path

References

• ComfyUI — Node-based diffusion pipeline editor
• GPU Switcher source — Custom Go dashboard
• LTX-Video — Video diffusion model

Next: Hopping Through the Portal — Hop Edge Cluster