MangaDex Development Update - API Abuse Enforcement

[BestBoy]

They were the ones who made MD great, and we haven't heard a peep out of them in quite a while.

Ya make Zephyrus start modding the forum again

 

[Tscheddov]

But hey! At least Advanced Search is back for guest users, that's something.

The functionality of the searchbar for quicksearch is still limited though
(It originally used to show titles groups and authors when typing something to search)
And the issue with the blank pages for titles with filtered content is back
(only checked for mature as filtered so far)

 

[Tscheddov]

Ah, maybe you haven't seen it then. A shame. Let's just say that when it comes to venture capital, Hal comes from a school of thought that I like to call "bankman-friedism"

Could you explain this in more detail please ?
I am intrigued

 

[shinya78]

The functionality of the searchbar for quicksearch is still limited though
(It originally used to show titles groups and authors when typing something to search)

Might as well use advanced search at that point, probably why it became available for guests again in the first place lol

 

[finalm]

Could you explain this in more detail please ?
I am intrigued

You can probably guess what I'm hinting at if you take into account what the guy I responded to said. There were also some older monologues by the person in question that could be googled. Going into details here is likely to get someone getting hot and bothered and trying to subpoena me, again.

 

[wiz_gmc]

Not technically savvy to understand the exact workings of rate limiting, but I often go through the updates list, and open titles in their own tabs as I go, which tend to result in a lot of tabs. It so happens that sometimes I walk away from the pc for some reason, or take a nap or something, and when I come back and click one of the tabs to check it out, suddenly all the tabs on the mangadex domain refresh? I think that triggers it as I vaguely recall getting a few 'too many requests' in the past. Which is strange as the 'browser.tabs.unloadOnLowMemory' is at default value in firefox(true) so the tabs should be asleep if not accessed for a time?

That's why I often just bookmark the titles then close them and, and go through the bookmark instead when checking them...

That's also how I use the site, on a PC it just makes sense to open many tabs. I have also had those times where every MangaDex tab would reload all at once, seemingly at random, and I would be blocked from the site for a period.

I can only theorise, but I think there is a part of the service worker's javascript that can force every MangaDex tab to reload in order for an updated service worker to be installed. Because service workers only really make sense for progressive web app functionality, I just disabled them altogether.

I haven't had a forced reload of tabs since I disabled service workers, but because my theory would only happen when an update is made, which appears random in practice, I can't actually confirm that it was the cause.

 

[Chizan]

That's also how I use the site, on a PC it just makes sense to open many tabs. I have also had those times where every MangaDex tab would reload all at once, seemingly at random, and I would be blocked from the site for a period.

I can only theorise, but I think there is a part of the service worker's javascript that can force every MangaDex tab to reload in order for an updated service worker to be installed. Because service workers only really make sense for progressive web app functionality, I just disabled them altogether.

I haven't had a forced reload of tabs since I disabled service workers, but because my theory would only happen when an update is made, which appears random in practice, I can't actually confirm that it was the cause.

Thanks. I've disabled service workers in firefox, let's see if that solves it. Based on searches, it shouldn't cause much trouble when disabled, and the benefits of it seem almost nonexistent, at least on desktop. Didn't even know such a service/option existed..

So for anyone else who wants to disable it, go to Firefox's about:config page and paste

dom.serviceWorkers.enabled

in the searchbox and click enter, and it will come up. The default value is 'true', change it to 'false'.

 

[bwemk]

hi people, it's all local so you can do the crazy thing of deleting cookies.

 

[wiz_gmc]

Thanks. I've disabled service workers in firefox, let's see if that solves it.

Fingers crossed you don't see anymore forced reloads. If it does happen to you again, maybe you could send me an update? If my theory is wrong, I would like to go back to finding the true cause.

It's maybe important to note that disabling service workers for all sites will probably increase your browser's uniqueness for fingerprinting by quite a bit. Though using firefox is already puts you in a narrow range, so it's probably not worth worrying about any more than that is.

hi people, it's all local so you can do the crazy thing of deleting cookies.

I had a look since you said this. It appears to it is stored in local storage rather than cookies.

It doesn't appear to be bound checked either, it just +1s the existing value when a chapter loads, so rather than clearing site data every 10 chapters, it's probably more efficient to manually set it to -9007199254740991 (the lowest integer in JavaScript numbers) and forget about it until you've read 9 quadrillion chapters.

 

[bwemk]

Fingers crossed you don't see anymore forced reloads. If it does happen to you again, maybe you could send me an update? If my theory is wrong, I would like to go back to finding the true cause.

It's maybe important to note that disabling service workers for all sites will probably increase your browser's uniqueness for fingerprinting by quite a bit. Though using firefox is already puts you in a narrow range, so it's probably not worth worrying about any more than that is.

I had a look since you said this. It appears to it is stored in local storage rather than cookies.

It doesn't appear to be bound checked either, it just +1s the existing value when a chapter loads, so rather than clearing site data every 10 chapters, it's probably more efficient to manually set it to -9007199254740991 (the lowest integer in JavaScript numbers) and forget about it until you've read 9 quadrillion chapters.

Thanks for taking the time to look into it. I assumed it was cookies and not local storage because it was session bound without considering how incognito could be affecting my tests. Anyway, this really shows how this only affects regular users and is an attempt for something bigger. This doesn't even affect those scrapers, because I'm an ordinary user with limited web development knowledge and I could find this out after a good 3 minutes of trying to bypass it.

 

[Foostini]

Yeah nah fuck this site. Either an intrusive "become a supporter" haunting every page or a reading limit, what a choice.

 

[exLightning]

Is this why I'm unable to access the site while my VPN is on?

 

[Angry_Panda]

Is this why I'm unable to access the site while my VPN is on?

Unlikely - this particular update affects chapter loading and advanced search, but doesn't do anything to restrict access to the main page.

 

[seekermoc]

Is this why I'm unable to access the site while my VPN is on?

What country is your VPN server located in? MD is blocked in a lot of European and east Asian countries, so try a different server location.