this post was submitted on 08 Mar 2024
44 points (95.8% liked)

Selfhosted

40006 readers
563 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
44
Proxmox vs. TrueNAS Scale (lemmy.world)
submitted 8 months ago* (last edited 7 months ago) by [email protected] to c/[email protected]
30 comments fedilink hide all child comments
 

I'm in the process of migrating my system to some new hardware. I was curious on everyone's thoughts about Proxmox vs. TrueNAS Scale.

Here is some background - I'm currently running a mini-computer, with Debian, attached to an external hard drive. I host Plex, -arr suite, PhotoPrism/Photo backup space, Syncthing and some other apps. It runs fine, but could probably use some more memory. I also haven't had a lot of luck backing up all my family's data (on and off different cloud services) in one place in a way that avoids duplicates. My 4TB HDD is at about 80% full now. I have an offsite synology that I back up to using Syncthing. Syncthing has been having some problems lately, so I'm looking at some other options for that too.

I've been wanting to move my storage to an internal HDD, so I bought a larger used computer and a hard drive so that I can clean my setup a bit. It has an i3 8100, 500GB M2, 256 SSD, 8TB HDD and 24GB ram. To experiment, I've been running Proxmox and set up a few VMs including TrueNAS.

Proxmox has been pretty amazing. I thought I would have a TrueNAS VM, my Debian-based Plex/-arr VM, and then another Debian vm where I could just test different software that I wanted to host. I haven't really experimented with the LXMs yet.

I started testing out TrueNAS and saw that it also offers virtualization. If so, I probably wouldn't need Proxmox for my purposes.

With all that, here are some questions -

  1. What do you think about Proxmox vs. TrueNAS? Any reason to prefer one over the other?
  2. What do you think about having a Debian VM to host my Plex and -arr suite? What are the pros and cons of that method vs. hosting the apps on my TrueNAS or Proxmox as containers? I think mainly it would just be portability and isolation.
  3. Currently, my external HDD is formatted so you could also plug it into Windows and read the contents. If something happens to me, I would like my family to be able to easily access the data. I need to figure out a good way to ensure it is easily accessible to them.

Thanks in advance!

Edit for posterity: after this post, I tried TrueNas, but was annoyed because the HDD was constantly being accessed. I tried unRAID after that, but had a similar problem with HDD access noise. I tried several cache drive configurations , but I couldn't escape the constant 5-second access pattern. I finally went back to Proxmox and will cobbler together my own NAS setup. We'll see how it goes.

all 31 comments
sorted by: hot top controversial new old
[–] [email protected] 14 points 8 months ago* (last edited 8 months ago) (3 children)

Their use cases are a bit different, no? Proxmox is a general hypervisor. You can run whatever you want on it. NAS is one workload that could be run on top of Proxmox. TrueNAS is a NAS first solution, hypervisor second. And that's the overlap with Proxmox. You could think of your core use case:

  • Do you want mainly a NAS that can run a few services too?
    • Yes
      • Perhaps use TrueNAS
    • No
      • Perhaps use Proxmox and roll your own NAS on top
[–] [email protected] 4 points 8 months ago (1 children)

Well fuckin thank you for that concise explanation, because I've been planning to build a NAS and Jellyfin box, and have been wondering

[–] [email protected] 3 points 8 months ago (1 children)

You definitely want to go the truenas route for a media server.

[–] [email protected] 5 points 8 months ago

I disagree in this case. Other posters are pointing out how Trunas support is aggressive and unhelpful. I also feel like a true hypervisor gives you more flexibility if your needs change later. I run a Jellyfin media server on proxmox, and it's been rock solid.

[–] [email protected] 1 points 8 months ago (1 children)

Apparently people run TrueNAS in a VM on Proxmox and it works fine, it's even supported by both Proxmox and TrueNAS.

I've tested both - the virtualization in TrueNAS is rather limited, so I moved over to Proxmox - I need good VM and Container support, in addition to storage management. I'll probably go the TrueNAS on Proxmox route once I have real storage capability.

[–] [email protected] 1 points 8 months ago

Can you explain how it is limited?

I'm assuming that TrueNas has basic features, and that I only need basic features, so Proxmox is probably overkill for me. What would I be missing if I only used TrueNas for virtualization? Before they talk about backups. It's there anything else?

[–] [email protected] 1 points 8 months ago (1 children)

Yes, this is my thinking. I mainly need a NAS that will let me use some services. I'm just not sure what I'll be missing if I forgo Proxmox. I'm assuming it is a lot of advanced settings I don't need, but other users mentioned backups as a positive for Proxmox.

[–] [email protected] 2 points 8 months ago* (last edited 8 months ago) (1 children)

Can't say. Personally, I'm running vanilla Ubuntu LTS and rolling my own ZFS, NFS, containers, desktop and so on but "I know what I'm doing." I hardly see a reason to do TrueNAS outside of UI. With that said I would highly recommend to ensure your data sits on ZFS because it protects it from silent data corruption. If I had to choose between Proxmox and TrueNAS and one ensured my data is on ZFS, I'd choose that solution, and then think about other use cases.

[–] [email protected] 2 points 8 months ago

I think both are on zfs, so that should be good.

I'm definitely a "no clue what I'm doing" user, so GUIs are helpful.

Thanks for the feedback, it's helpful.

[–] [email protected] 7 points 8 months ago* (last edited 8 months ago) (1 children)

I use Truenas scale.

For virtualisation, most people use a community run set of apps called Truecharts. I will say that documentation/support for this is rubbish. They have a discord server only really there. Very hostile community in general.

Not sure why kubernetes is used. Docker is being phased out and will stop being supported in the future.

Also, the latest version of scale broke onedrive backups, which were handled by the gui, and now you're on the own to run rclone via the cli. Definitely, the devs are not working on a fix as stated in forum posts. This is a pretty fundamental requirement of an at-home NAS for anyone using onedrive for photo backup say.

Ix system forum devs are rude. I haven't posted there but searching through other people's threads for solutions to my problems show dismissive unhelpful answers by the devs/users.

All this is to say Truenas virtualisation is compromised, poorly documented, and run by a hostile community of devs both in true charts and the ix systems forums.

I regret not trying a different nas os, but I'm a bit invested now.

[–] [email protected] 2 points 8 months ago (1 children)

The comments on the community and OneDrive implemention are helpful. I do remember seeing one post from someone asking about running only one hdd in TruNAS. You're right, the comments were very rude. I guess it's a bigger problem than I realized.

I don't have any experience with Kubernetes, I'll check on that.

If you had to start over, what NAS software would you look at?

[–] [email protected] 2 points 8 months ago (1 children)

I think the IX Systems would rather Truenas scale be an enterprise OS, and have short patience for people learning the ropes.

Kubernetes to me is a lot more complicated than Docker, but I'm sure in an enterprise environment where you have many systems to administrate it is superior. Docker would be a better, simpler solution for a person at home with one computer being used for their personal virtualisation.

I think going back in time I would go for Unraid, and use Docker containers. Apparently it is better documented, more beginner-friendly. It is a one-off payment, but it is reasonably cheap. Community seems much friendlier too.

Bear in mind I haven't used unraid, so potentially there is a grass is always greener situation going on here.

That said, I have thought about running a VM in TrueNAS so I don't need to muck around with kubernetes and using a discord chat for troubleshooting.

All the best!

[–] [email protected] 2 points 8 months ago* (last edited 8 months ago) (1 children)

FYI, unRAID is moving to a subscription, so if you want to try it out, it might be good now before they change.

Also, I thought I read that docker/portainer was possible on TrueNas. You might check it out if Kubernetes is giving you trouble. I might be wrong, but I can't imagine why it wouldn't work on the base Debian install.

[–] [email protected] 2 points 8 months ago

Oh damn, that's disappointing about the subscription.

Yep Docker is currently possible, but there's plenty of threads discussing that it is being phased out.

But yeah, I suppose the solution could be a VM running Debian and then Docker within that.

[–] [email protected] 7 points 8 months ago (1 children)

Just my 2 cents:

  1. Proxmox. Flexibility for both new services via VM/LXC and backups (just install proxmox backup server alongside and you get incremental backups with nice retention settings, file-restore capabilities as well as backup consistency checks)

  2. If it's in a VM/container you don't need to worry about backups, see 1.

  3. In this case isn't it sufficient to be able to access the data via Windows network?

[–] [email protected] 4 points 8 months ago (1 children)

By a backup server, is that an additional component to my proxmox install? Could I back things up to the offsite Synology? I assume it isn't a separate box on the network?

In what ways is proxmox virtualization more flexible than TrueNas? I thought it was fairly similar for basic things (my use case). I do realize they are built for different purposes, I just don't need a lot of virtualization.

For some reason I've never had luck setting up network shares for Windows on my network. I should really figure that out.

Thanks for the response. I'll check it the backup server in more detail.

[–] [email protected] 4 points 8 months ago* (last edited 8 months ago) (1 children)

You setup Proxmox Backup Server on separate hardware and then you add it as a storage option in your Proxmox Virtualization Server.
I haven't dived into it but I imagine you could run the Proxmox Backup Server as a VM in your Synology NAS.
https://www.proxmox.com/images/download/pbs/docs/proxmox-backup-3-1.pdf

edit: Unofficial PBS Docker github: https://github.com/ayufan/pve-backup-server-dockerfiles

[–] [email protected] 3 points 8 months ago

Thanks for the clarification and the other information. It's helpful.

[–] [email protected] 3 points 8 months ago* (last edited 8 months ago)

I use Proxmox and don't use Truenas. My setup is basically to install Cockpit on the host server via apt-get and then the 45 Drives cockpit-sharing plugin. This provides the NFS and Samba sharing I need and use. I host Home Assistant in a VM and Docker containers in a few LXC containers which host about 10 containers each. Then, in combination with https://tteck.github.io/Proxmox/ you can set up pretty much anything you need from there.

This is on in computer terms, ancient; a 13 year old Dell Optiplex 990 with 16gb Ram and software such as Authentik and Vaultwarden from different dedicated LXC containers. Never have any issues with overload of the system resources or running out of memory. It's pretty much rock solid.

[–] [email protected] 3 points 8 months ago (1 children)

I recently installed Proxmox and TrueNAS on the same box in kind of an ouroboros styles, following this guide: https://github.com/enigmacurry/d.rymcg.tech

So now I have both. I run VMs in proxmox and use truenas only for nas.

[–] [email protected] 2 points 8 months ago (3 children)

So is your TrueNas in a Proxmox VM? Did you try VMs in TrueNas?

I guess I'm trying to see the benefits of VMs in Proxmox versus VMs in TrueNas Scale. My use case is not very complicated, so I'm wondering if I can simplify the setup by just using TrueNas.

[–] [email protected] 3 points 8 months ago

Proxmox has a built in backups system, and also handles LXC containers.

Also proxmox can run ZFS just like TrueNAS, so there is little reason to use TrueNAS instead of Proxmox unless you're just wanting a pure NAS with nothing else.

[–] [email protected] 2 points 8 months ago (1 children)

I currently only use proxmox for VMs. Proxmox hosts a TrueNAS VM, TrueNAS controls all but the main (small) drive on the box, proxmox then has access to the other drives through TrueNAS. Kind of neat.

But I think it would indeed be simpler to only have TrueNAS and use it for both nas and VMs. I have no experience with TrueNAS' VMs.

[–] [email protected] 1 points 8 months ago

Thanks for the feedback. More to think about...

[–] [email protected] 2 points 8 months ago

TrueNAS isn't supported as a VM. This doesn't prevent people from doing it anyway, but it might be relevant for you.

I've seen performance comparisons on the web and there is a slight advantage to Proxmox, but nearly insignificant. It really boils down to what you're more comfortable maintaining.

Note that your can run docker containers for the services you run in your Debian server. Docker will run faster than any VM.

[–] [email protected] 3 points 8 months ago* (last edited 8 months ago)

As mentioned by others, if your primary use case is storage/NAS with only one or two VMs, then for sure go with TrueNAS SCALE.

Where Proxmox wins is the easy backups for VMs/containers locally to disk, local NAS or remote NAS, easy management when you have a lot of VMs (I feel like there’s too much clicking through menus in the TN UI which gets annoying fast), more recent kernel and ability to use LXC.

I use both in my homelab and I keep TrueNAS as a NAS and Docker host via Jailmaker script (for services that benefit from direct/local file bind mounts) because I really dislike the way the TrueNAS Apps feature is handled.

In my experience, TN Apps are just not stable and seem to get randomly stuck ‘deploying’ for no good reason after being restarted or updated. Combine that with the general hostility of the forums and of TrueCharts, and so I decided to not have anything to do with the Apps feature. IX changed a few things to do with app storage that then forced TrueCharts to change how they do things, so there’s been a few occasions where the only solution has been to delete and recreate containers which pissed off a lot of users.

Jailmaker lets me use Docker Compose inside a systemd-nspawn container. It’s kind of funny how this nested containerisation method ended up being a hell of a lot more reliable than TN Apps. I don’t want this to sound like I’m ungrateful for the good things they’ve done for TN by making services easy to run, but their reading their posts, their behaviour and tone online just always makes me shake my head.

Sorry this turned into a rant.

[–] [email protected] 2 points 8 months ago (1 children)

I've had a proxmox machine running for about a year now, and just set up a standalone Frankenstein NAS running truenas scale (only because I was having trouble with HBA firmware on truenas core)

If you're new to zfs, truenas is an easier entrypoint because its GUI is easier to understand. Proxmox has been rough for me, and I'm often jumping into the cli to manually do things that I think should be easy to do in the GUI but aren't.

That said, proxmox is more full-featured and (I think) more stable and lightweight than truenas scale. If you're comfortable with zfs, or learning how to manage it via cli, then I think proxmox is a better workhorse.

Long-term, though, do you foresee expanding storage? A single 8TB hdd seems unlikely to fit your needs for very long.

[–] [email protected] 1 points 8 months ago

Thanks for the response.

This is my first experience with zfs. So far I've been okay with the Proxmox UI, but I also haven't tried anything other than vanilla VMs. I've enjoyed using Proxmox and learning it, but it also seems like an extra layer that I'm going to have to keep updated and maintained.

I might expand storage, but as it is, I'm doubling my current available storage, I have enough offsite backup capacity for a long time, and I'll have the original 4tb external HDD if I want to shuck it. The main issue is getting all my data centralized with as little duplication as possible.

[–] [email protected] 2 points 8 months ago

This is a great post with lots of info that I'll need to dig into once I have time.

I've been running truenas CORE for like 3 years with adguard, emby server, and jellyfin running on it. 2 6tb HDDs in my zfs pool and a backup 6tb drive. As well as smb shares for the network.

Running all this on my gaming PC I built 12 years ago with 16gb of ram and 3770k processor.

Truenas has been awesome and I've learned a lot.

Recently picked up some thin clients so I've been learning proxmox and lxc containers (plus Linux in general) but haven't quite figured out how to have proxmox zfs stuff take over for truenas for local network shares.Someone has already linked to the setup installing truenas on a VM on proxmox, which I started but haven't made much progress yet.

Running game servers in docker inside lxc containers is pretty dang cool though

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
LTS Long Term Support software version
LXC Linux Containers
NAS Network-Attached Storage
ZFS Solaris/Linux filesystem focusing on data integrity

4 acronyms in this thread; the most compressed thread commented on today has 7 acronyms.

[Thread #583 for this sub, first seen 8th Mar 2024, 14:35] [FAQ] [Full list] [Contact] [Source code]