[Suggestion] IPFS/distributed Mangadex.

[sillymangareader]

If this has been suggested before, please close/delete this thread.

With the recent DDoS attacks and generally high volume of traffic Mangadex has been seeing, it would be a good time to consider a peer-to-peer, distributed approach to this site. I don't know how much the current code would allow for such a thing, but the benefits are enticing: Mangadex would become resistant to further DDoS attacks without the need for expensive extra servers, bandwidth and Cloudflare protections. These are always welcome additions and I'm not suggesting to get rid of them, just a sort of "adding another layer to the cake" thing.

Without going into a lot of detail about the backend, how plausible is this?

 

[ZipFile]

IPFS is cool in theory but on practice it is pain in the ass to implement it right.
Here's some problems to deal with:
1. You need to deliver content to IPFS nodes. IPFS does not make your files magically available for everyone. Someone must have all those files.
2. Browsers do not support IPFS nativelly. Means, you need to setup a HTTP gateways. Unless node has something already stored, it works as caching layer between IPFS and outer web.
3. Based on restrictions from pt.2 you need to setup load and traffic balancing. Since vast majority of traffic would be over HTTP, p2p nature of IPFS would be thrown away. Load speed matters, so we'll end up with nodes that serve both as IPFS data node and as a getaway. Some gateways are fast here, but slow over the ocean, some has full mirror, some might have partial copy of the site's content. You need to keep track who is who and who posses which content in order to deliver smooth experience for end user.
4. IPFS gateways still need to deal with DDoS. Means, everyone should bother protecting their own servers now. Also pay for servers and bandwidth.
Other words, you need to reinvent your own CDN on top of IPFS. It's not trivial, error prone and require highly skilled (and expensive) developers. Cloudflare does all of this, plus some bonuses.
Or you can skip entire CDN stuff and just link to local IPFS gateway. There is still a problem with delivering content to solve (new stuff being added, old updated). But this way regular MD user won't benefit much from IPFS.
There is also H@H around (not IPFS-powered), guys behind it pulled something similar, but never released backend server code afaik.

 

[Holo]

As we continue to grow, bandwidth will continue to be a problem in future. CloudFlare helps out a lot, but even on the business plan, there are limits somewhere. We are looking at other cost effective ways of mitigating this problem, but the cost will continue to rise... that is inevitable.

People are looking at alternative ways of dealing with this issue, like the examples mentioned above, but to every genius idea there is always some problem with how to implement it and how it would actually work in reality.

 

[h]

I don't know how you guys store manga, but an IPFS implementation would be a directory with all manga available in the site (an example: https://ipfs.infura.io/ipfs/QmP8Wjiu6CpcqNif1TRwTctLmok1BW59px1zWPPZ2HgptC, like this but with folders) with IPFS running in the background. And I'm guessing the images would be links to an IPFS gateway..

PS: Did someone mention me by mistake?

 

[Yorple]

You don't need to manage an IPFS gateway yourself; you could just use ipfs.io or whatever. But really, you should just use ipfs: protocol links and have people use the IPFS Companion extension to open those links with either their local IPFS instance or an appropriate gateway.

You'd want to add a checkbox in the site settings to use ipfs: protocol links vs gateway links.

 

[Yorple]

'Course I'm just talking about distributing images and static metadata with IPFS, not the whole website.