this post was submitted on 04 Dec 2023
110 points (99.1% liked)

Selfhosted

40183 readers
766 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

Unless you are missed the news. Bug found in OpenZFS causing data corruption, FreeBSD, Linux and illumos are all affected.

Good description of the bug

Some good news from bug tracker on OpenZFS

OpenZFS 2.2.2 and 2.1.14 released with fix in place. If you don't get your OpenZFS direct from here, point your vendor at it.

Patch your systems!!

top 13 comments
sorted by: hot top controversial new old
[–] [email protected] 26 points 11 months ago (1 children)

For all Proxmox users it looks like the new ZFS kernel module with the patch is included in the opt-in kernel 6.5.11-6-pve for now.

The kernel 6.5 actually became the default in Proxmox 8.1, so a regular dist-upgrade should bring it in. Run "zpool --version" after rebooting and double check you get this:

zfs-2.2.0-pve4 zfs-kmod-2.2.0-pve4

As this versions are patched for bug.

[–] [email protected] 2 points 11 months ago

Thank you for this info!

[–] [email protected] 10 points 11 months ago (1 children)

TrueNAS iX Systems statement with interim mitigation steps.

[–] [email protected] 8 points 11 months ago* (last edited 11 months ago)

The statement is very informative. The bug happens under increased read/write operations to the same file causing a race condition.

I also found interesting:

Despite the bug being present in OpenZFS for many years, this issue has not been found to impact any TrueNAS systems. The bug fix is scheduled to be included in OpenZFS 2.2.2 within the next week

[–] [email protected] 9 points 11 months ago* (last edited 11 months ago) (1 children)

It looks like due to the older coreutils Ubuntu 22.04 LTS is unlikely to be affected as much. So if you're on Ubuntu LTS and have not compiled it yourself, you might have dodged data corruption.

Would this corruption be caught by the checksumming?

No

[–] [email protected] 9 points 11 months ago* (last edited 11 months ago) (1 children)

This seems to affect ZFS >=2.2.0. Ubuntu 22.04 LTS is on ZFS 2.1.5

Nope!

OpenZFS through 2.1.13 and 2.2.x through 2.2.1 contain this bug.

This issue occurs less often in version 2.2.1, and in versions before 2.1.4, because of the default configuration in those versions.

From here

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago) (1 children)

Weird. Reading the issue on GitHub I got the impression that it affects 2.2.0+.

Also I glanced something that the version of coreutils also contributes to the interaction.

E: Seems like Ubuntu LTS is unlikely to be affected a lot due to the older coreutils.

[–] [email protected] 1 points 11 months ago (1 children)

Jep, it seems like Damocles Sword was hanging above us the whole time (⊙_⊙)

[–] [email protected] 2 points 11 months ago* (last edited 11 months ago)

At this point I think I'm just gonna wait for the patch to land in Ubuntu.

[–] [email protected] 7 points 11 months ago* (last edited 11 months ago)

FreeBSD has the patch pushed, corrected versions are:

14.0-p1

13.2-p6

12.4-p8

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
LTS Long Term Support software version
RAID Redundant Array of Independent Disks for mass storage
ZFS Solaris/Linux filesystem focusing on data integrity

3 acronyms in this thread; the most compressed thread commented on today has 13 acronyms.

[Thread #335 for this sub, first seen 4th Dec 2023, 15:25] [FAQ] [Full list] [Contact] [Source code]

[–] [email protected] 3 points 11 months ago

But my filesystem has RAID and checksums and I use ECC memory, I don't need backups /s

[–] [email protected] 2 points 11 months ago

Oh shit,

Welp work is going to be fun