A documented comparison of running Kubernetes across every major hosting model — cloud managed, self-managed on cloud, private cloud, and bare metal at home. The goal is a honest, practical reference for each environment: what it costs you in time and money, where the rough edges are, and how the networking story differs between them.
The thread running through all of it is Talos Linux — an immutable, API-driven OS built specifically for Kubernetes. No SSH, no shell, no config drift. The same OS everywhere means the operational model stays consistent regardless of what is running underneath.
| Environment | Approach | |
|---|---|---|
| OpenStack — Cleura | Talos & Terraform | draft exists |
| OpenStack — Cleura | Talos, with Omni | maybe ? |
| OpenStack — ElastX | Talos & Terraform | draft exists |
| OpenStack — ElastX | Talos, with Omni | maybe ? |
| Homelab — bare metal | Talos + Pixieboot + Omni | draft exists |
| Homelab — bare metal | Talos + Pixieboot without Omni | maybe ? |
| Homelab — OpenStack | OpenStack on bare metal, Talos running on top | (stretch) |
| Homelab — OpenStack | Talos on bare metal, OpenStack inside cluster | (stretch) |
| AWS | Talos on EC2 | (stretch) |
| Azure | Talos on VMs | (stretch) |
| GCP | Talos on Compute Engine | (stretch) |
Stretch goals
AWS, Azure, GCP — same Talos approach, different underlying infrastructure. Interesting eventually, but not the priority.
Omni
Omni is Sidero’s managed control plane for Talos clusters — worth documenting both with and without it. Without Omni gives you the full picture of what Talos management looks like manually; with Omni shows what the managed layer buys you.
Homelab provisioning
Nodes provisioned via Pixieboot — no USB sticks, no manual installations. A node powers on, boots from the network, and registers. The goal is a fully reproducible cluster from scratch with minimal human steps.
Scope
- Cluster provisioning and bootstrap for each environment
- Networking — CNI choices, ingress, cross-cluster connectivity
- Storage — what you get managed vs what you have to bring yourself
- Operational differences — upgrades, node management, observability
- Cost and trade-off summary across environments
Making it usable
Getting a cluster running is the easy part. Making it usable is where environments diverge. Each environment needs an answer for ingress, DNS, and storage — and the answer varies significantly depending on what the underlying platform provides.
On managed cloud you can lean on load balancers and block storage from the provider. On OpenStack you have those options if the provider exposes them. On bare metal at home you are on your own — MetalLB or similar for load balancer IPs, a local DNS solution, and either local storage or something like Rook/Ceph. Same Kubernetes, very different operational story underneath.
Notes exist in various states — pulling them together, testing, and documenting properly is the work.