I have verizon fios. I run my traffic through cloudflare warp to bypass Verizon’s peering shenanigans.
Cloudflare definitely doesn't use its premium peering for Warp usage, and you can be sure that neither does Verizon to Warp.
But if you can provide MTR measurements to help us investigate what is going on, please do (remember to black out the first couple of hops that may contain your own IP address).
so the issue is between mangadex and the iCloud Private Relay.
Well, same question. It's possible we have poor connectivity to Apple's servers. Honestly not something we get a lot of chances to look at.
Having to wait minutes for parts of chapters to still not load is abnormal.
Indeed, it isn't. Loading a chapter (try in an incognito tab just in case) should really be a few seconds at most, typically.
it loads so quickly that I would think the server was in my living room.
Yeah, when it works, it works. However in practice we observed multiple people having dogshit load times on different CDNs and locations (even for the same exact CDN brand-wise) over the years. There's just a lot of factors at the end of the day.
They have to pay for transit to avoid congested links. They have not admitted to paying for prioritization, which only matters when the links are congested.
The thing is, crappy ISPs ensure their public links are congested, on purpose:
- Deutsche Telekom in DECIX (Frankfurt, Germany) => Every single day at evening time, there's at least 5% of packet loss.
- Telin in Indonesia, which (when I looked anyway) had a total of 40Gbps total public connectivity in their main DC (they do have multiple, but the point stands), for a population of nearly 300 million of which 80% or so are their customers. Obviously this link was congested to high heavens pretty much 24/7.
Or they make extremely subpar routing decisions by prioritizing transit costs over latency:
- Most Indian ISPs route their customers to western europe unnecessarily instead of southeast asia, because India<>Europe is cheaper than India<>Singapore even if a CDN is present there. So, around the globe they go.
- Australian ISPs routinely route their customers to Singapore even if the target CDNs also has presence in Australia (this one is to ensure nobody can ever meet their peering requirements which have a minimum local average traffic clause, as most such requirement lists do)
And all of these have the same exact solution "buy direct connectivity to us where you want, of course at multiple times market price".
They would never word it as "prioritization" because that'd be either illegal, against net neutrality where applicable, or just bad PR.
Finally, the US especially has much deeper problems in general with regards to internet connectivity that border on comedy.
We had multiple issues in the past being caught in the crossfire of fights, like between AT&T and Tata communications around Seattle (connection would just drop for 2h a day every day) or [some ISP I forget] with NTT in Florida (something like 40% packet loss all of a sudden).
But also things like NYC <> Los Angeles traffic costing more than NYC <> London even though the latter goes under an entire ocean.
Point is, not many ISPs actually do the simple, obvious, and sane thing of "I will pick the shortest route and that's it", because then they can't monetize faster connectivity to your service for their customers. They only accept that Netflix etc install cache servers directly on their network because it also saves them some cost as their infra couldn't handle it actually pulling data live.
It occurs to me that you could use subdomains to support multiple CDNs. Either you could have the entire site duplicated on a subdomain or you could have the subdomains host images only and then have a toggle in the interface to allow the user to select the CDN used to deliver images. That would work around the one size does not fit all problem that occurs with CDNs. As long as cloudflare is not the default, you might not incur its throttling problem.
We actually used to do that and have some dedicated rules for certain IP subgroups. It didn't work very well, and was a major hassle for us to maintain.
using 1 CDN to feed the other CDNs
This is usually not necessary, because CDNs typically have good pull speeds. All of what I wrote applies near-exclusively to residential ISPs, because people often don't have a choice (or don't know any better) so there's little competition. There's often a dozen ISPs in a datacenter, so they're forced to not do dumb shit to their professional customers or risk losing them fast.
Anyway, we have no plans to maintain multiple entrypoints to the site, so it'd be best if you really want this fixed to get measurements between different setups (Warp + iCloud, Warp-only, nothing) and see what works best or send over some MTR measurements for each and we can always ask our service providers if they can do something, but don't hope too much on that.
