The Sunday maintenance did something to HakuNeko's ability to access the site

[MarqFJA87]

Some of you are probably familiar with HakuNeko as a common option for downloading manga from this site. Well, I don't know what your developers did during this past Sunday's maintenance, but whatever it was, HakuNeko suddenly ceased to be able to connect to MangaDex afterwards on my end. Strangely enough, this problem is not affecting all users, as when I raised the issue on the app's Github repository, its developer said that everything is completely fine on their end, and nobody else appears to have reported the same problem (though that doesn't necessarily mean I'm the only one).

All attempts to debug the issue were met with failure after failure, until today I got hit with an idea after Pixiv suffered a seemingly selective pseudo-crash that left a seemingly arbitrary selection of users unable to access the site in anyway unless they used a VPN to spoof their geolocation. The idea: use ProtonVPN - which I had used to get around Pixiv's pseudo-crash - to mask my geolocation and see if that does anything. And to my surprise, it actually did: as long as the VPN connection is on, HakuNeko can access MangaDex like normal. Using GoodbyeDPI as an alternative (by adding mangadex.org to the blacklist) also works.

Keep in mind that during all this, I was able to access MangaDex on desktop just fine, though the same could not be said on mobile; neither app nor browser can access the site.

Oh, if you're wondering where the hell I am to be facing all this trouble, I'm in the Arabian Peninsula.

So the question that I wish to pose to the staff is this: WTH did this Sunday maintenance do that now requires geolocation-spoofing to make HakuNeko (and presumably mobile phones) able to access your site? It's fine if you wish to keep the details classified for security reasons; I just hope that you will have your devs look into the problem and see if it's fixable.

 

[tristan9]

WTH did this Sunday maintenance do that now requires geolocation-spoofing to make HakuNeko (and presumably mobile phones) able to access your site?

The answer will disappoint you but… nothing?

The maintenance Sunday was for our internal LAN (ie the network over which our api calls our databases, for example), nothing to do with public networking.

And since you write, in that github issue, the following

Namely, I activated ProtonVPN to spoof my geolocation.

You should know that VPNs, aside from geolocation (which MD does not care about and never filters for), also very specifically prevent your ISP from tampering with your connection.
Which is what is actually solving your issue in our case.
And which suggests that it is indeed your ISP’s fault.

The fact that GoodbyeDPI also fixes it is even further indication of it, since DPI is not something websites like MD do… because we have no use for it. We are after all the recipient of the data that DPI is meant to spy on, so we needn’t do anything to receive it…

All-in-all, I’m sorry your ISP increased their censorship, but this is none of our doing…

 

[MarqFJA87]

Well, that is a very odd coincidence that this apparent ISP change happened around the time that you performed your maintenance. And weird how it only affects HakuNeko and mobile access, and not desktop browser access.

 

[tristan9]

Well, that is a very odd coincidence that this apparent ISP change happened around the time that you performed your maintenance.

True, but not much we can do. Everyday users connect to us via thousands of different ISPs, so one of them having an issue at nearly the same time as a maintenance is not that uncommon, alas.

All I can say is that we don't block any incoming IP address, save for a few /24 blocks in some shady russian datacenters that were used to DDoS us regularly in the past, and that list hasn't changed for months and months :/

 

[BayuAH]

Well, that is a very odd coincidence that this apparent ISP change happened around the time that you performed your maintenance. And weird how it only affects HakuNeko and mobile access, and not desktop browser access.

Probably Desktop and Mobile have different ISP. This is happens alot in my country. The Desktop accessed from state own ISP that well-know doing funny things and Mobile from private one.

About NakuNeko, probably ISP specifically block the API end point, or User-Agent detection. Although I not sure User-Agent detection by ISP possible in HTTPS connection.

Another possibility is the Desktop still have cached IP address of the Desktop not like NakuNeko and Mobile, so still use actual IP address. ISPs usually just change IP address of the domain to "block" certain website.

Update: After I read more carefully, it seems NakuNeko also access its own domain to get manga list from MangaDex. Probably that one is also blocked.

 

[Doctor_Who]

Is HakuNeko still being maintained? The latest version available on Github is from 2020.

EDIT: I tried the program and just with Mangadex it returns this error message:

Failed to update manga list for MangaDex

Failed to receive content from "https://api.mangadex.org/manga?limit=100&order%5BcreatedAt%5D=asc&createdAtSince=2018-03-07T05%3A44%3A10&contentRating%5B%5D=safe&contentRating%5B%5D=suggestive&contentRating%5B%5D=erotica&contentRating%5B%5D=pornographic" (status: 400)-

EDIT 2: The nightly build fixes the problem.

 

[MarqFJA87]

Is HakuNeko still being maintained? The latest version available on Github is from 2020.

EDIT: I tried the program and just with Mangadex it returns this error message:

Failed to update manga list for MangaDex

Failed to receive content from "https://api.mangadex.org/manga?limit=100&order%5BcreatedAt%5D=asc&createdAtSince=2018-03-07T05%3A44%3A10&contentRating%5B%5D=safe&contentRating%5B%5D=suggestive&contentRating%5B%5D=erotica&contentRating%5B%5D=pornographic" (status: 400)-

EDIT 2: The nightly build fixes the problem.

Yes, it is still being maintained. I was also confused, but it turns out the master(?) is regularly updated, which isn't reflected by the releases section.

And yes, I did use the nightly build (I even reinstalled it just to be sure), but to no avail.

 

[doppler]

Just an off question: If the ISP is blocking MD. If done by DNS denial wouldn't using opendns or goggle dns fix the problem ? Or are they denying the IP's at there Border Gateway ?

 

[RogueKitsune]

If it is only a DNS block then yes changing your DNS settings will fix the problem

 

[MarqFJA87]

Do the DNS options have any significant differences in performance?

 

[doppler]

Do the DNS options have any significant differences in performance?

YES/NO I use optonline at one time there DNS sucked. I switched to Opendns and got better big time.

Want 1000% better performance use "pi-hole". It has a bad player "blocking lists" and caching of gotten DNS's requests. In other words. Get the unknown DNS once (becomes cached). Next time it comes off your local network, at a much lower lag time. Performance of cache hits can see 1000% improvement. Especially if you live in an area like S.E.A. Notorious for bad internet access.

DNS servers are meant to be super servers. But during high demand times they will degrade. Your local pi-hole, if you do it right won't suffer performance issues.

 

[tristan9]

If done by DNS denial wouldn't using opendns or goggle dns fix the problem ? Or are they denying the IP's at there Border Gateway ?

As per Roki's answer, if it's only poisoned DNS then switching it fixes your issues yes.
In practice, SNI filtering is at play sometimes, but still quite rare (you can find out more on this here: https://blog.cloudflare.com/encrypted-client-hello/ ).
As for IP filtering, to the best of our knoweldge it is not in widespread use against us at the moment. Since IPs are typically shared between multiple websites due to intermediaries like Cloudflare, it is not a very reasonable blocking method in this day and age. (mind you, sometimes govts/ISPs are unreasonable and will still do it).