Open Source

31024 readers

916 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Posts must be relevant to the open source ideology
No NSFW content
No hate speech, bigotry, etc

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago

MODERATORS

[email protected]

TIL XZ has built-in checksumming (and now threaded decompression?) (lemmy.ml)

submitted 11 months ago by [email protected] to c/[email protected]

8 comments fedilink hide all child comments

I need to transport multiple very large files over an unstable and untrusted network, and the file contents are outputted as a data stream. I wanted to use OpenSSL for streaming authenticated encryption, but they purposefully don't support that and are preachy about it.

Well, it turns out that XZ has checksumming built-in! It even has different algorithms (CRC32, CRC64, and SHA256). It's part of the same file, within/before the encryption, and automatically verified by the decompression tool. I'm already using XZ for compression before encryption, so this is just super convenient and useful. Also, it seems like XZ supports threaded decompression now, when it didn't before. Thanks XZ devs!

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago)

Possibly the source of any confusion here is when the encryption and when the compression takes place? Maybe some more details about how you are using xz and encryption would help.

As far as I can tell, xz doesn't do anything with signatures or encryption, but it does perform checksums like you stated, which is very cool and I'm glad you shared this.

Edit: I am re-reading your post above. You are compressing with xz, then encrypting, got it. So yes, if any part of the payload is tampered with, then it would be detected by the decryption, depending on the algorithm, or by the decompression because of the checksums like you said. Sorry for the confusion! You've got it all straight lol.