this post was submitted on 26 Aug 2023
3 points (100.0% liked)

Selfhosted

39254 readers
177 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

Many of the posts I read here are about Docker. Is anybody using Kubernetes to manage their self hosted stuff? For those who've tried it and went back to Docker, why?

I'm doing my 3rd rebuild of a K8s cluster after learning things that I've done wrong and wanted to start fresh, but when enhancing my Docker setup and deciding between K8s and Docker Swarm, I decided on K8s for the learning opportunities and how it could help me at work.

What's your story?

top 11 comments
sorted by: hot top controversial new old
[–] [email protected] 2 points 1 year ago (1 children)

Kubernetes is useful if you have gone full cattle over pets. And that is very uncommon in home setups. If you only own one or two small machines you cannot destroy infra easily in a "cattle" way, and the bloatware that comes with Kubernetes doesn't help you neither.

In homelabs and home servers the pros of Kubernetes are not very useful: high availability, auto-scaling, gitops integrations, etc: Why would you need autoscaling and HA for a SFTP used only by you? Instead you write a docker-compose.yml and call it a day.

[–] [email protected] 1 points 1 year ago

This mostly, I haven't seen a compelling reason to leave my docker setup.

[–] [email protected] 1 points 1 year ago (1 children)

I was looking into converting my docker services into a cluster to get high availability and to learn it for work, but while investigating it, I read that kubernetes is actually meant for scalability and just a single service per cluster.

Also read that docker swarm is actually what is recommended for my homelab use case. So I'm right now on my way to convert everything to docker stacks. What do you think?

[–] [email protected] 0 points 1 year ago (1 children)

I'm not sure what you mean by that.

It provides high availability if you have multiple nodes and pods.

Also what do you mean by single service per cluster? Because that's not the idea at all.

[–] [email protected] 1 points 1 year ago

Of course high availability always requires multiple nodes.

Its just that while choosing how to set up my cluster I looked up several options (proxmox, swarm, kubernetes...) and I noticed that kubernetes is generally meant for bigger deployments.

I only need a single replica for each of my containers and they can all run on a single node, so kubernetes is overkill just to get high availability For my use case

[–] [email protected] 0 points 1 year ago (2 children)

Seems a bit overkill for a personal use selfhosting set-up.

Personally, I don't need anything that requires multiple replicas and loadbalencers.

Do people who have homelabs actually need them? Or is it just for learning?

[–] [email protected] 1 points 1 year ago

I find mine useful as both a learning process and as a thing need. I don't like using cloud services where possible so I can set things up to replace having to rely on those such as next loud for storage, plex and some *arr servers for media etc. And I think once you put the hardware and power costs vs what I'd pay for all the subs (particularly cloud storage costs) it comes out cheaper at least with hardware I'm using.

[–] [email protected] 1 points 1 year ago (1 children)

A lot of people thought this was the case for VMs and docker as well, and now it seems to be the norm.

[–] [email protected] 1 points 1 year ago

A lot of people thought this was the case for VMs and docker as well, and now it seems to be the norm.

Yes, but docker does provide features that are useful at the level of a hobbyist self-hosting a few services for personal use (e.g. reproducibility). I like using docker and ansible to set up my systems, as I can painlessly reproduce everything or migrate to a different VPS in a few minutes.

But kubernetes seems overkill. None of my services have enough traffic to justify replicas, I'm the only user.

Besides learning (which is a valid reason), I don't see why one would bother setting it up at home. Unless there's a very specific use-case I'm missing.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

I use k8s at work and have built a k8s cluster in my homelab... but I did not like it. I tore it down, and currently using podman, and don't think I would go back to k8s (though I would definitely use docker as an alternative to podman and would probably even recommend it over podman for beginners even though I've settled on podman for myself).

  1. K8s itself is quite resource-consuming, especially on ram. My homelab is built on old/junk hardware from retired workstations. I don't want the kubelet itself sucking up half my ram. Things like k3s help with this considerably, but that's not quite precisely k8s either. If I'm going to start trimming off the parts of k8s I don't need, I end up going all the way to single-node podman/docker... not the halfway point that is k3s.
  2. If you don't use hostNetworking, the k8s model of traffic routes only with the cluster except for egress is all pure overhead. It's totally necessary with you have a thousand engineers slinging services around your cluster, but there's no benefit to this level fo rigor in service management in a homelab. Here again, the networking in podman/docker is more straightforward and maps better to the stuff I want to do in my homelab.
  3. Podman accepts a subset of k8s resource-yaml as a docker-compose-like config interface. This lets me use my familiarity with k8s configs iny podman setup.

Overall, the simplicity and lightweight resource consumption of podman/docker are are what I value at home. The extra layers of abstraction and constraints k8s employs are valuable at work, where we have a lot of machines and alot of people that must coordinate effectively... but I don't have those problems at home and the overhead (compute overhead, conceptual overhead, and config-overhesd) of k8s' solutions to them is annoying there.

[–] [email protected] -1 points 1 year ago

I'd suggest Podman over docker if someone is starting fresh. I like Podman running as rootless, but moving an existing docker to Podman was a pain. Since the initial docker setup was also a pain, I'd rather have only done it once :/

For me the use case of K8s only makes sense with large use cases (in terms of volume of traffic and users). Docker / Podman is sufficient to self-host something small.