Multi-tenancy — Disposable Kubernetes Clusters with vCluster

Every experiment on a shared cluster carries risk. Install a CRDCustomResourceDefinitionThe object that teaches the Kubernetes API a new resource type. Install a CRD and the API server starts serving a kind it has never heard of, with validation and RBAC like any built-in. that conflicts with production. Deploy a Helm chart that creates cluster-scoped resources you did not expect. Run a fuzz test that fills all available memory. On a homelab with one cluster, the blast radius is everything.

Layer 14 adds vCluster — virtual Kubernetes clusters that run inside Frank. Each one has its own API server, its own namespaces, its own resources. From the inside it looks like a real cluster. From the outside it is a StatefulSet in a namespace.

    flowchart LR
  subgraph Host[Host Cluster — Frank]
    NS[vcluster-experiments<br/>namespace]
    ST[StatefulSet<br/>experiments-0]
    PVC[5Gi Longhorn<br/>backing store]
  end
  subgraph Virtual[Virtual Cluster — experiments]
    API[API Server]
    CM[Controller Manager]
    SQL[SQLite<br/>embedded]
    NSv[Namespaces<br/>kube-system, default, ...]
  end
  subgraph Tenant[Tenant Workloads]
    Pod[nginx pod<br/>scheduled on host]
  end

  NS --> ST
  ST --> API
  API --> SQL
  API --> CM
  CM -->|sync| Pod
  Pod -->|scheduled on| NS
  

What vCluster Actually Is

vCluster runs a virtual Kubernetes control plane (API server + controller manager + backing store) as a StatefulSet. The virtual cluster has its own API endpoint, its own etcd (or SQLite), and its own namespaces. Workloads created inside the virtual cluster get synced to the host cluster for actual scheduling — the virtual cluster does not run its own kubelet or container runtime.

Key properties:

• API isolation — a tenant can install CRDs, create cluster-scoped resources, and run kubectl without affecting the host
• Resource isolation — quotas and limit ranges bound what the tenant can consume
• Network isolation — network policies restrict traffic between virtual cluster pods and the host
• Lifecycle simplicity — delete the namespace, everything is gone

The Template Pattern

Adding a vCluster should be as simple as adding a Helm values file and an ArgoCD Application CRCustom ResourceAn object of a type Kubernetes did not ship with, added by a CRD. Frank's ArgoCD Applications, Rollouts and Tekton Pipelines are all CRs.. Values are split into two layers:

apps/vclusters/
  template/values.yaml        # Base defaults — all vClusters inherit
  experiments/values.yaml     # Instance-specific overrides

The ArgoCD Application CR loads both files in order — Helm deep-merges them:

helm:
  valueFiles:
    - $values/apps/vclusters/template/values.yaml
    - $values/apps/vclusters/experiments/values.yaml

To create a new vCluster: copy the Application CR, point it at a new values file, push.

Template Defaults

Deploying

One ArgoCD app per vCluster. The experiments instance:

# apps/vclusters/experiments/application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: vcluster-experiments
  namespace: argocd
spec:
  project: infrastructure
  sources:
    - repoURL: https://charts.loft.sh
      chart: vcluster
      targetRevision: 0.32.1
      helm:
        releaseName: experiments
        valueFiles:
          - $values/apps/vclusters/template/values.yaml
          - $values/apps/vclusters/experiments/values.yaml
    - repoURL: <git-repo>
      targetRevision: main
      ref: values
  destination:
    namespace: vcluster-experiments

Chart Schema Gotchas

The vCluster chart v0.32.1 has a strict JSON schema. Three things the initial plan got wrong:

1. isolation does not exist — it is policies (with resourceQuota, limitRange, networkPolicy)
2. networking.service does not exist — the chart does not expose a top-level service type override
3. Case sensitivity matters — configMaps not configmaps, persistentVolumeClaims not persistentvolumeclaims

Any schema violation produces a template error during ArgoCD sync. The error message is clear, but discovering the correct field names required helm show values against the actual chart.

Verify

Inside the virtual cluster:

$ kubectl get namespaces
NAME              STATUS   AGE
default           Active   3m
kube-node-lease   Active   3m
kube-public       Active   3m
kube-system       Active   3m

$ kubectl get nodes
NAME     STATUS   ROLES           AGE   VERSION
mini-3   Ready    control-plane   3m    v1.35.2

$ kubectl run nginx --image=nginx:alpine
pod/nginx created

$ kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          10s

On the host, the nginx pod appears in vcluster-experiments with a mangled name — the syncer translates between virtual and host namespaces. The pod is scheduled normally by the host’s kubelet.

Missteps

Recovery Path

References

• vCluster Documentation — Installation, configuration, sync rules
• vCluster Helm Chart — Chart repository
• apps/vclusters/template/values.yaml — Base defaults
• apps/vclusters/experiments/ — Instance-specific config

Next: Paperclip — AI Agent Orchestrator