this post was submitted on 04 Nov 2023
102 points (100.0% liked)
Technology
37699 readers
320 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
PeerTube will not replace youtube. it cannot compete in either scale or creator compensation.
i don’t think people realize just how insane your infrastructure has to be to handle 30,000 hours of video being uploaded every hour.
Taking some simple napkin math, I have a 1min 1080p video downloaded from YT. It clocks in at 15MB.
So, Gamer's Nexus has 2.6k videos. (That's insane, btw, but fairly large channel, not even LTT size though).
Assuming just 1080p, and let's say about 10min average per video. (Some are less, some are 40+), that's 150MB per 10min video, and that means it's 390,000Mb (or 380.86GB) for their collection. Assuming I'm wrong and the average is even half of that, and the average GN video is only 5 minutes that's still 190GB. And that isn't counting 4k, or the multiple other formats to optimize streaming (720, 480, 360, misc bitrates, etc)
And that's just storage, not even taking into account compute! (Or egress, or transcriptions, or scaling, or..)
Really for something like Peertube to take off it will require each channel to spin up their own instance, which honestly is just another expense for them, one that Youtube does for them for free, plus Youtube offers to pay them. Which, would cut down on some of the chaff (only people who want to do it would do it), but yeah, I don't think it's going to replace YT at any point. Smaller channels can combine for sure, but there is definitely a threshold where it becomes extremely costly.
I'm all for the fediverse, but video streaming is freaking costly and expensive. There's definitely a reason youtube has a monopoly on it. Now this isn't to discourage, but more for anyone who may be thinking "yeah why doesn't peertube just replace it?)
You are vastly overestimating the amount of storage you need since you are looking at some download which itself has to choose the encoding (which is independent of whatever youtube does: youtube absolutely crushes the quality).
Most estimates assume that youtube has 1 exabyte of storage, let's say we buy this in bulk from retail (which we wouldn't do: you wait as long as possible since storage prices are going down and retail stores would give you the finger if you ordered and exabyte worth).
Let's take that number and run with it:
Buying retail, you can get Seagate Exos X20 20TB drives for 280€, 1 exabyte is 1Mio terabyte, meaning we have 1_000_000/20 * 280 = 14 Mio € (you'd need machines to put those into but you also wouldn't buy the entire thing upfront, and using retail prices either).
Compute also isn't that big of a deal if you do it correctly: the expensive part in video hosting is usually video encoding since to get small video sizes you need to spend compute beforehand to compress it.
However, you can shift this in significant parts to the user by implementing the transcoding in WASM and running this clientside (see e.g. https://www.w3.org/2021/03/media-production-workshop/talks/qiang-fu-video-transcoding.html) in that case users would compress locally in the browser before uploading (this presumably wouldn't even take longer than normal uploads for most people since you trade off transcoding time against upload time).
There are still other compute expenses but those are much more limited.
These mechanisms don't (at least to my knowledge) exist in peertube yet, but would be possible.
The actually expensive part is always the actual networking: Networking is one of the few things that actually get more expensive at scale due to the complexity explosion, rather than cheaper (e.g. having dedicated transcoding hardware drops in price per user since you have higher utilization).
Networking quickly runs into bottlenecks where you have to account for all the covariances between datasets in the network.
Basically to increase the amount of e.g. storage available everything in the network needs to be increased (from the local machines connections, over the cables and switches up to routers and outgoing connections) due to you increasing the density at one point, you have to increase the network everywhere.
That's why networking dwarfs everything: you just get crushed by networking being the bottleneck between your increasingly dense devices.
The clue behind peertube is that this is not as extreme of an effect due to
The latter is the important part: instead of having network cost rising (super) linearly to the amount of users you have it rise linearly to the amount of simultaneous unique videos.
This is a much smaller number which means you do not need to compete in that space, which is the dominant cost factor. (if you have a method where one user can retain the video and share it without actively watching that same video, you can probably get real-world sublinear scaling)
Mind you, the costs involved here are still large, but not insurmountably large, especially considering there is not one unique organisation that would have to pay for the entire thing and its not an upfront expense. Fundamentally though the system is built such that it won't be crushed as users flood into the network.