this post was submitted on 06 Nov 2023
185 points (95.6% liked)

Open Source

31175 readers
165 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago
MODERATORS
 

Most of the problems in the current internet landscape is caused by the cost of centralized servers. What problems are stopping us from running the fediverse on a peer to peer torrent based network? I would assume latency, but couldn't that be solved by larger pre caching in clients? Of course interaction and authentication should be handled centrally, but media sharing which is the largest strain on servers could be eased by clients sending media between each other. What am I missing? Torrenting seems to be such an elegant solution.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 29 points 1 year ago* (last edited 1 year ago)

The short answer is that while torrents show great possibility for content distribution (as an alternative to CDNs for example), they inherently rely on some centralized resources and don't make sense for a lot of use cases. Most websites are a bunch of small files, torrenting is really much more useful for offloading large bandwidth loads. On small files, the overhead for torrents is a waste. That's why your favorite linux ISO has a torrent but your favourite website doesn't.

One major issue is difficulty in accurately tracking the contribution of each member of the swarm. I download a file and I seed it to the next person, sounds great right? But what if the next person doesn't come along for a long time? Do I keep that slot open for them just in case? How long? How I prove I actually "paid my dues" whether that was waiting for peers or actually delivering to them? How do we track users across different swarms? Do we want a single user ID to be tracked across all content they've ever downloaded? When you get into the weeds with these kinds of questions you can see how quickly torrenting is not a great technology for a number of use cases.

Being somewhat centralized, by the way, is how BitTorrent solved the spam issue which plagued P2P networks prior to it. Instead of searching the entire network and everything it contains (and everything every spammer added it to it), you instead rely on a trusted messenger like a torrent index to find your content. The torrent file or magnet link points to a link in a DHT and there you go, no need to worry about trusting peers since you are downloading a file by hash not by name. And you know the hash is right because some trusted messenger gave it to you. Without some form of centralization (as in previous P2P networks), your view of the network was whatever your closest peers wanted it to be, which you essentially got assigned at random and had no reason to trust or not trust. You couldn't verify they were accurately letting you participate in the wider network. Even a 100% trustworthy peer was only as good as the other peers they were connected to. For every one peer passing you bad data, you needed at least two peers to prove them wrong.

Blockchain gets us close to solving some of these problems as we now have technology for establishing distributed ledgers which could track things like network behavior over time, upload/download ratio, etc. This solves the "who do I trust to say this other peer is a good one?" problem: you trust the ledger. But an underlying problem to applying Blockchain to solve this problem is that ultimately people are just going to be self-reporting their bandwidth. Get another peer to validate it, you say? Of course! But how do we know that peer is not the same person (how do we avoid sybil attacks)? Until we have a solid way to do "proof of bandwidth" or "proof of network availability", that problem will remain. There are many people working on this problem (they've already solved proof of storage so perhaps this could be solved in a similar way) but as of right now I know of no good working implementation that protects against sybil attacks. Then again, if you can use blockchain or some other technology to establish some kind of decentralized datastore for humanity, you don't need torrents at all as you would instead be using that other base layer protocol for storage and retrieval.

IPFS was intended as a decentralized replacement for much of the way the the current internet works. It was supposed to be this "other protocol", but the system is byzantinely complex and seems to have suffered from a lack of usability, good leadership, and promotion. When you have an awesome technology and nobody uses it, there are always good reasons for lack of adoption. I don't know enough about those reasons to really cover them here, but suffice to say they actually do exist. Then again, IPFS has been around for a while now (15 years?) and people use it for stuff so clearly it has some utility.

That said, if you want to code on this problem and contribute to helping solve data storage/transmission problems, there are certainly many OSS projects which could use your help.