Long term strategies to address bandwidth issues and running costs: MangaDex@Home + Affiliates + BTC

[SakamotoDeath]

Old chapters can't be loaded, did something gone terribly wrong?

 

[wedge1001]

do you have mangadex @\home active?
https://mangadex.org/md_at_home/options

if yes, try disabling it.
There seems to be some problems here (and it will also try only the same server - until now)

 

[Holo]

@wedge1001

Yes, we do wildcard subdomains as you have suggested, and it is all automated via the CF api.

The stats will be global and per region, similar to H@H.

@KazuyaKan

If you have a dynamic IP, just enter your current IP. It doesn't matter if it changes later. We only need to this to know the location of your client.

 

[ilx]

This is a great idea to reduce server cost!

There seems to be some stability issues with the MD@Home network though.
Some servers report an internal server error (HTTP 500), and some fail to establish a TLS connection with the error `SSL_ERROR_RX_RECORD_TOO_LONG`.
Regarding the TLS error, it seems that the server is reporting an HTTP error response in plain text instead of completing the TLS handshake.
Relevant tcpdump:

	0x0000:  4500 00d3 dbd1 4000 3506 dd33 2530 5c52  E.....@.5..3%0\R
	0x0010:  0a89 0015 04b0 9d70 d2c4 ca73 2dc5 a4da  .......p...s-...
	0x0020:  5018 00ed 2676 0000 4854 5450 2f31 2e31  P...&v..HTTP/1.1
	0x0030:  2035 3033 2053 6572 7669 6365 2055 6e61  .503.Service.Una
	0x0040:  7661 696c 6162 6c65 0d0a 636f 6e74 656e  vailable..conten
	0x0050:  742d 7479 7065 3a20 7465 7874 2f68 746d  t-type:.text/htm
	0x0060:  6c3b 2063 6861 7273 6574 3d55 5446 2d38  l;.charset=UTF-8
	0x0070:  0d0a 636f 6e74 656e 742d 6c65 6e67 7468  ..content-length
	0x0080:  3a20 3735 0d0a 0d0a 5468 6973 2073 6572  :.75....This.ser
	0x0090:  7665 7220 6973 2065 7870 6572 6965 6e63  ver.is.experienc
	0x00a0:  696e 6720 6120 7375 7267 6520 696e 2063  ing.a.surge.in.c
	0x00b0:  6f6e 6e65 6374 696f 6e73 2e20 506c 6561  onnections..Plea
	0x00c0:  7365 2074 7279 2061 6761 696e 206c 6174  se.try.again.lat
	0x00d0:  6572 2e                                  er.

Tapping to reload doesn't seem to try requesting the image from another server if the last image fetch failed. Maybe because there's no other client to serve that image from or is this planned to be implemented?

I have a few questions regarding contributions:
- Will the implementation of the client and potentially the server be made open source?
- Is there an bug tracker to collect issues in a structured way, or is this exclusively done through the forum?

 

[berubetto]

SEA region is underserved? Well, I thought, that can't be my airport's code SEA, which is SEA-TAC Airport serving my area's Seattle & Tacoma, Washington, USA, the region where Microsoft is headquartered and dot-com bubbles burst merrily everywhere in the late 1990s.

Searching DuckDuckGo lead me to the SEA region = Southeast Asia, which I think is correct.

Sorry if this was covered earlier, but I'm posting this just in case and I'm too tired to read the whole shebang.

 

[BraveDude8]

@ilx The client will be open-sourced as soon as it's stable enough to not require two builds a day, and the bug tracking is currently internal. I've passed your post along.

 

[knabber]

I second @wedge1001 point about using wildcard certificates. This would allow you to assign the same certificate to multiple clients (say 5 or 10), reducing the amount of certificates needed.

edit: missed holo's post, it's already planned.

 

[iDeSmI]

This is the beginning of a new era.

 

[bozunee]

Works for doujinshi so it should work for regular manga right?
Really hoping so joking aside

 

[Holo]

@ilx

Client will be open source once it's stable and out of RC.
I suspect bug tracking etc will be on the github page once it's open source.

The backend will not be open source.

 

[Seneca7C]

Regarding MD@Home, I think that's a good strategy and I'm sure many people are willing to help, but asking for a 80Mbps up/down minimum sounds like a very lofty goal for us SEA folks here. 😬😅
For me personally from Malaysia we have a 100 Mbps plan as advertised but all I ever get is 50~80 Mbps down & 30~50 Mbps up (yes it fluctuates a lot). Just another day of monopolies being monopolies.. 😢 Aside from that though are easy requirements to meet.

 

[Lolipop_Candy]

I have 100 Mbps Upload & Download but don't have dedicated static IP, can I join the Hentai@Home I mean..... MangaDex@Home ? Or can I use DynamicDNS so I can provide the link as dedicated static IP ?

About the server, is Raspberry Pi enough as the machine ?

 

[wedge1001]

@Lolipop_Candy:
Dynamic ip is okay - just put your momentary public IP in there. But it must not be natted. ISP-Nat won't work.
You don't need to setup a dynamic DNS.

A raspberry pi has enough cpu to power the client.
But you need to provide a fast enough access to files - using the flash-drive will be too slow;
Using a dedicated hdd via usb2 should do the trick.


at SEA:
I've already added a 1gbit-system for Malaysia - but it's still pending

 

[Lolipop_Candy]

@wedge1001
Hmm..... thanks I note that

 

[iencm]

Is it intended that server drop connection after one request?

 

[iencm]

@vexcat

I had similar problem in my code

(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')))

I manually install "Let’s Encrypt Authority X3 (IdenTrust cross-signed)" from https://letsencrypt.org/certificates/
And error dissapeared

My guess that was some issue with LetsEncrypt certificates

Probably browser update fix this for everyone. I do not have your error in FF nor chromium

 

[CatWEax]

@TheDragonLord
Apparently, images are retrieved from md@\h nodes simply via HTTPS, which makes sense since every browser supports it.

I just checked a node that served a chapter I wanted to read and got these encryption parameters: TLS 1.2, ECDHE-RSA-AES128-GCM-SHA256, Curve X25519, certificate by Let’s Encrypt. Looks good to me.

From your provider’s perspective it’s just normal HTTPS and all they can get is the hostname from SNI. Some nodes run on ports other than 443, so if your provider throttles all traffic on uncommon ports, you’re out of luck.

 

[TheDragonLord]

@CatWEax the downfall of unregulated, private multinational monopolistic ISPs.

Thanks to you both for the info! <3

 

[frogcrush]

@sid_cypher you said it yourself, because it requires an IPFS client.

@izuzu how the fuck? How is Microsoft your ISP?

 

[takosama]

@frogcrush
Just a guess but prb a Microsoft hosted server?