Debugging Kubernetes
Understand what a healthy status actually proves, and find the checks that matter.
Building Frank
Follow the implementation, from bare metal to working services.
Building the Foundation — Talos, Nodes, and Cilium
Bootstrapping a Talos Linux cluster with Omni, configuring node labels and zones, and replacing Flannel with Cilium's eBPF networking.
GitOps Everything with ArgoCD
Migrating from Flux to ArgoCD with an App-of-Apps pattern — adopting existing workloads without downtime.
Observability — Metrics and Logs with VictoriaMetrics
Deploying a resource-efficient observability stack with VictoriaMetrics, VictoriaLogs, and Grafana — and the four gotchas that made it interesting, including a scrape that was configured and inert for 148 days.
Health Monitoring — Feature Probes, Heartbeats, and Telegram Alerts
Adding feature-level health monitoring to the cluster — Blackbox probes for endpoints, Pushgateway for cron heartbeats, and Grafana alerting that fires to Telegram when things go silent.
Health Bridge — Closing the Loop from Grafana Alerts to GitHub Issues
A lightweight Go service that receives Grafana webhook alerts and automatically updates GitHub Project lifecycle states — turning monitoring signals into actionable project board updates.
Edge Observability — Watching Frank's Edge Without Watching Frank's Edge Burn
Collectors on Hop, backend on Frank, AI alert enrichment that knows when 12x baseline is a scraper and when it is Hacker News.
The Metrics API — Teaching Frank to Read His Own Vitals
kubectl top was dead for months because nothing served metrics.k8s.io. The fix was one Talos flag and a decision I almost got wrong: build the fancy thing, or the right thing.
Operating on Frank
Check health, investigate failures, and recover the system.
Operating on Cluster & Nodes
Day-to-day commands for checking cluster health, managing Talos nodes, and debugging Cilium networking on Frank — including the real failure patterns the gotcha docs cover.
Operating on GitOps
Day-to-day commands for managing ArgoCD applications, syncing, debugging drift, handling degraded apps, and navigating the gotchas that have bitten Frank.
Operating on Observability
Day-to-day commands for querying metrics and logs, checking the etcd scrape, managing Grafana dashboards, debugging alert delivery failures, and fixing the observability pipeline on Frank.
Operating on Health Monitoring
Day-to-day commands for managing feature health probes, heartbeat metrics, file-provisioned Grafana alerts, and Telegram notifications — including the silent delivery failures we've fixed.
Operating on Health Bridge
Day-to-day commands for managing the health-bridge service — testing webhooks, managing alert labels, recovering stranded board tiles, and auto-closing healed bug issues.
Operating on ArgoCD Drift
How 20 of 52 ArgoCD apps were permanently OutOfSync, why it was seven different bugs, and how fixing the noise unmasked a 21-day crashloop.
Operating Edge Observability
Querying the blog log stream, banning a scraper before lunch, tuning Falco out of kube-system noise, and triggering a digest by hand.
Operating the Metrics API — kubectl top, HPAs, and the Silent-Empty Failure
How to check the resource Metrics API is healthy, add a CPU/mem HPA, and diagnose the one failure mode where the pod is Ready but top is empty.
Operating on Green
Five ways a healthy dashboard lies — wrong artifact, unconsumed signal, out of scope, stale view, never applied — and the command that checks each
The Frank Papers
Compare approaches, examine the evidence, and understand the trade-offs.
Immutable OS & Declarative Machines
Immutable OS for Kubernetes nodes is a four-job problem — read-only root, transactional updates, declarative config, and (at the strict end) no SSH at all — and the six 2026…
GitOps at Small Scale
GitOps on Kubernetes is a four-jobs-in-one bundle — reconcile, drift- detect, gate write access, audit — and the six contenders in 2026 (ArgoCD, Flux v2, Jenkins X,…
The Observability Stack, Honestly
Cluster observability is a five-job problem — metrics, logs, traces, dashboards, alerts — and the six contenders in 2026 (Grafana+Prometheus+Loki, VictoriaMetrics, Mimir,…