Goodbye Linode Kubernetes, Hello… Azure Kubernetes?

BillHomelab & Nerding

On to plan C!

After a fair amount of googling and noodling, I’ve come to the conclusion that Linodes LKE Kubernetes service can’t do what I want it to, at least in a way that doesn’t feel hacky and get expensive. My goals – and working on this migration has helped sharpen these:

  • Migrate my OhanaAlpine VPS docker workloads over to Kubernetes.
  • Do so in a way that can run comfortably on a single node, or scale up to 3-5 for testing  / research / upgrades.
  • Not be cost-prohibitive. << $50/mo, the lower the better.
  • Not have a single point of failure, even in the single node config. That means that if the single node got recycled that it’d be able to reconstitute itself including data (DB, app data, files) from backup as part of the rebuild if necessary.
  • Be flexible for other deployments.

LKS hit all but #4, and I could not for the life of me figure out a way to do that that wasn’t kludgey. Here’s why:

  • Persistent storage doesn’t persist for node recycles. That is, if I put MariaDB tables out on a PV/PVC block storage volume, it doesn’t get re-attached if the node is recycled and built from scratch.
  • Linode doesn’t offer RWX access for PV. That is, a block storage volume can only be attached to one volume at a time.
  • Related, there’s no easy, obvious way to do shared storage across nodes / pods easily. I looked into Longhorn that might do the trick, it depends on at least 1 node in the cluster being running. I know that should be the norm, but that violates #4
  • I thought about S3 object storage, either as primary shared storage (I don’t think RWX is required for that) and as backup storage to store backups for Longhorn to bootstrap with.  It all felt overly complicated and rickety to set up. Yandex S3 was the lead option, and while S3 kinda got close, it wasn’t really a proven option. I may circle back to this one day.

What I really wanted was a file storage service from Linode, sorta like EFS from AWS. If I could reliably and securely mount an NFS share in a pod or across pods, that would have solved most of my problems, or at least been a non-hacky way to achieve my goals. Why doesn’t Linode offer this?  Oh, I could have spun up my own, but that’s more cost and complexity. Not out of the question in the future but feels like too heavy of a lift for now.

So, what’s a hacker to do? Without changing my requirements (Im looking at you, item #4…) looking at alternatives Kubernetes hosting is the next step. Looking at Digital Ocean, AWS, GCP, and others, it seems like Azure (AKS) is the best way forward. I think its a super-capable platform, but I’m not totally crazy about it because it looks expensive with even a minimal cluster. It comes with a $200 credit to use in the first month and a bunch of free services for the first 12mos, and that should give me time to get built and see what steady-state costs are going to be. I might yet fail on #3 but at least I’m learning, right?