this post was submitted on 23 May 2024
14 points (88.9% liked)

Selfhosted

40113 readers
870 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

My use-case: streaming video to a Linux virtual mount and want compression of said video files on the fly.

Rclone has an experimental remote for compression but this stuff is important to me so that's no good. I know rsync can do it but will it work for video files, and how I get rsync to warch the virtual mount-point and automatically compress and move over each individual file to rclone for upload to the Cloud? This is mostly to save on upload bandwidth and storage costs.

Thanks!

Edit: I'm stupid for not mentioning this, but the problem I'm facing is that I don't have much local storage, which is why I wanted a transparent compression layer and directly push everything to the Cloud. This might not be worth it though since video files are already compressed. I will take a look at handbrake though, thanks!

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 3 points 5 months ago (1 children)

Unless you've got raw uncompressed video, any kind of transparent compression like you describe is only going to cost you in energy bills for no benefit. Most video is already compressed with specialised video compression as part of the file format, you can't keep compressing stuff and getting smaller files.

The alternative is a lossy compression, which you could automate with some scripts or a transcoding tool like tdarr. This would reduce the quality of the video in order to reduce the file size

[–] [email protected] 1 points 5 months ago (1 children)

The problem is that I don't have the local storage to maintain a watch folder for continually streaming video. I want to write semi-directly to the Cloud, which is why I'm looking for a transparent reencoding layer. Can handbrake do this?

[–] [email protected] 3 points 5 months ago (1 children)

I'm not sure about transparently, that's more in the tdarr wheelhouse I'd say. You'd dump the files into a monitored folder and it will replace it with a version transcoded to your specification.

Transcoding video takes a fair bit of time and energy too FWIW, so you're going to need enough local storage to handle both the full size and smaller one.

I have to question the idea though, cloud storage is always more expensive than local for anything remotely non-temporary, and transcoding a load of video all the time is going to increase your energy bills. If you have any kind of internet bandwidth restrictions that's gonna factor in too.

I'd say it would be better to save up for a cheap external hard drive to store your video on. For a year's subscription to a cloud storage service that would provide enough space for a media library, you could probably get twice the amount of storage forever.

[–] [email protected] 1 points 5 months ago (1 children)

I'm going to be using an SBC for this, which doesn't have the capacity for an extra storage drive. Also, I'm planning to move in a couple of months, and I wouldn't want to deal with storage in the middle of all of this. The cloud isn't as insanely expensive as I initially thought; B2 is $6/TB, and I hope that with reencoded streams at an OK resolution I wouldn't go beyond 1.5TB a month (I'll be deleting stuff with bucket policies, of course).

I'll take a look at tdarr alongside ffmpeg, thanks!

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago) (1 children)

Okay fair play, if you're doing this super short term it could make sense. Though I question what SBC you're using that's capable of transcoding video but not the ability to plug in an external drive.

$12/m for your 2TB of usage would make sense for maybe 5 months before it would be cheaper to buy an external disk—and of course that storage is gone once that time is up, Vs a hard disk which will probably last you a decade or so

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago) (1 children)

I was considering the usual BananaPi/OrangePi/Raxda/Pine64 SBCs, are those not enough horsepower? I'd like to stay under $80 for my SBC purchase, and it will be doing double duty with managing some services like DNS and music scrobbling alongside uploading to the cloud

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago) (1 children)

Hardware transcoding on SBCs is generally not fantastic, you're gonna want to look for one that has VAAPI/VDPAU support or you're gonna be looking at 100% CPU for half a day to transcode a film, which will make your other services effectively unavailable at the time.

I used to run my Plex server on a Pi4 with 4GB of ram and it basically crashed any time transcoding kicked in, I swapped to an intel NUC so I could get QuickSync for transcoding.

I'll point out though, every SBC you've listed has usb, which is all you need for an external disk. If you're worried about size, I've got a 5tb external drive that's about 5cm², which is basically the footprint of any SBC you could use in this scenario

[–] [email protected] 2 points 5 months ago

Thank you for the tip, I'll keep a look out for VAAPI/VDPAU support. Thanks!