Improvements to data saver option, servers and recruitment for a dev to work on...

[OnlyAlphanumericCharacters]

...to reduce bandwidth, data saver is now default on for guests...

Does this affect the API too (for chapter info / getting images)? Or is it just something for the reader?

 

[Holo]

This affects the API. You would have to add ?saver to toggle data saver via the API

@SolVic Yes, we are still waiting on delivery of 1 more server. And yes, if more people used data saver, it would help massively.

 

[TheGodEmperor]

@h has only himself to blame for such a name to get pinged so often

 

[Thoraxe41]

Well if we are going for an E-Hentai style, does that mean we will have a Sad Panda as well?

 

[pikokola]

@Holo @ixlone
It is, and a credit sink for whatever comes out as a reward of hosting the server.
If you take a close look at their community, many starts running the client solely to fund their endless torment playing the game.

I mention the game partially as a joke, but let me elaborate more.
Sure there are individuals or group who are willing to freely give huge bandwidth to host stuff (this site, those xdcc guys on irc, - you know the guys), but afaik most of them still come from the west or having their server in the west due to reasons.
My point is, expecting people run the client solely for the sake of it could only go to that extend.

If you're expecting a reliable hoster, moreover to ease the load from asia, you'll want those who own or are willing to rent a vps located closer to that area to host the client over people who use their home connection (those home connection should also help but surely you understand how most asia individual-consumer-grade internet are.)
It wouldn't hurt to start considering creating incentives to those people to host and actually use a reliable server. Finally, you know how weak asian are against achievement, watching endless increasing numbers, or a way to show off their big fat e-peen. that's why I mention the game.
Here, you could make something totally different as an incentive, but when designing that, I believe it's worth looking at the relationship between their game as credit sink to their user-run server which indirectly generate credits - and, if you want to look further, the "moderation power" given to the user which partially derived from how much they help hosting the site.

But of course, you are the one who knows how big and how wide the coverage from extra servers needed. If you consider that the people who said that they are willing to host should be enough, and passive approach to increase the servers should be fine; then there's nothing more I need to add.

 

[beritbunny]

@cyberbeing 's comments sound very well-informed... I am definitively having bad load times on "old" chapters, which seem like 94% of what I want to read. Whether it's catching up, refreshing, or trying something new...

I feel really sad and fed up. I don't really feel like I want to or that it is FAIR to complain to any admins, Devs, owners or other MD staff about my opinions and beefs, more like just to the universe in general. I feel like I am enduring on all other fronts and just want this of all things to "work for me". Maybe I'll feel more equanimity if I dig up more in depth info regarding it. As always, thanks for your work, and please keep in mind the importance of "old" chapters' accessibility.

 

[demonicbeast696]

wow this is great

 

[NodiX]

I'm been wondering this before, but is Mangadex is tied with Path Network through sponsorship? I've been suggesting some other CDN services, though multi-CDN is still an option, but doesn't receive any response from admin.

I'm addressing this because I failed to find Path network providing any sort of feature like origin shield when browsing through their site, which if it was the case, then it's reasonable to see Mangadex is vulnerable against large spike of traffic, especially if large number of chapters have 15mb++ of total size.

Cloudflare do have a paid service for this, called Argo Tiered Caching, though many other CDN companies do provide it for free if using their services.

https://docs.fastly.com/en/guides/shielding
https://www.limelight.com/origin-shield/
https://www.limelight.com/blog/reducing-origin-server-load/
https://www.keycdn.com/support/origin-shield
https://www.stackpath.com/products/cdn/origin-shield/
https://bunnycdn.com/blog/origin-shield-released/
https://www.cdnplanet.com/guides/origin-shield/

Edit:
Upgrading servers won't help much in the long run, in my opinion, at least not when Mangadex doesn't have this sort of feature sitting between their original servers and cache networks.

 

[syahriz]

thank you for the feature. this way i can enjoy a lot more mangas

 

[Halo]

@NodiX

Just to give you an idea of just how amazing this is, consider that the service that they’re affording us for free for a year, 3Gbps worth of bandwidth for commercial grade DDoS mitigation and caching services, is worth $600,000 annually. You read that correctly, folks. I’m still not sure if Marshal Webb and his team realize how grateful we at MD are. It still surprises us sometimes how generous people are in helping us grow.

- Original announcement

For those who are not aware, our image server traffic is handled by Path - they serve around 700TB of our image data per month.

- NA servers announcement

Unless you have an extra $600,000 (it's more now) to donate annually, suggesting alternative cdn companies isn't that useful.

 

[iencm]

@Holo

Please migrate from jpg/png to webp - it will save 50%+ storage and a lot of bandwidth WITHOUT loosing image quality

It will cost some CPU hours

And you even can make this on fly with nginx plugins

 

[leaflady]

@iencm As others have already mentioned, WebP doesn't play nice with MacOS, and about 20% of the userbase is on an Apple product.

 

[NodiX]

@Halo

I'm not saying they should burn the bridge with Path. I'm just asking they have an agreement or some sort to not using another paid CDN services as to promote Path. You know, multi-cdn is a valid strategy to cut cost and performance, considering Mangadex is stacking Cloudflare on top of their service. Just because a certain website use another CDN on their network doesn't mean their cost would suddenly bloated 600k annually. They could just use those extra caching servers to patch on the lacking area as to not overloading their origin server. A lower tier CDN wouldn't bill you if a CDN on top of it doesn't have a cache miss and has to fetch content from the CDN below it.

 

[NodiX]

@leaflady

I already mentioned this before, but you can still serve webp without having to mess with iOS users. I did that on my website I posted on a comment I posted previously. Some company also provided unlimited on the fly webp optimization as low as $9.5/month.

 

[iencm]

@leaflady
FF on apple devices is fine
(edit: duh! I was wrong. Apple locked browsers on iOS
This really bad)

I dunno about
>20%
Even so, that means 80% is not on that OS

(and how many from that 20% is not using apple browser?)

And even more this can be done on the fly

 

[Holo]

@NodiX

Simply put, CDNs are expensive. Unless we had an income of 6 figures+, we can't use a proper CDN. And even then, given the nature of this site, due to opsec, we can't pay anyone properly. A proper CDN company isn't going to accept payment in Bitcoins anytime soon, lol. This is also the reason why we aren't on Cloudflare, because we would have to pay - we use too much data for the free tier (which has a unspoken limit of 1 PB /month, and we probably pump over 2 PBs a month these days)

 

[NodiX]

@Holo

I see. It seems I haven't paid close attention and been referring to old post about Cloudflare. If you're the one who said the option is unviable then I probably should stop here, thought I certainly do have something more to say about it in mind.

 

[leaflady]

@Holo

Out of curiosity - and this isn't really the thread for this, sorry - has an opt-in ad system been considered? I already allow Brave to serve me ads; I'd be perfectly happy to whitelist MD on my various devices. It's the least I could do, considering how many hours I've spent on here.

@iencm

20% is a significant number, though. And yeah, there is a workaround for desktops and laptops, but not for mobile or tablets - tbh, it seems like pointless extra work for users, with not a lot of benefit. I say that as a noted Google fan with a Pixel phone and nothing in particular against webP other than that it's not great enough to warrant the switch (plus, JPEG XL is performing better than webP in terms of file size so far).

 

[RoadMovieToBerlin]

@Holo There’s a problem I’ve encountered with the data saver option, when you go to the next page (at least on the most recent version of safari for iPad) it doesn’t remove the last page from being viewed/rendered/printed on the screen meaning every pages look like the first page and to read it you have to refresh it every page

 

[Holo]

@leaflady
An opt in ad system would require a significant amount of people opting in to be generate any significant income. Ads have been considered and are still a last resort as we have not exhausted all possible options.

@RoadMovieToBerlin link me a chapter with this issue and we'll investigate.