Cloudflare DDOS protection + RSS

[nosongforyou]

The RSS feeds have become inaccessible from dedicated readers, as they are tripped up by the Cloudflare anti-DDOS features.

Since RSS already requires login, could you please do something about it?

 

[galinkinlin]

Bumping this. Sometimes my rss feeds work, but most of the time they do not.

 

[nekou]

My RSS-feed completely quit on me 3 days ago. Changes done on Cloudflare DDOS protection? The RSS bot I'm using can no longer find the RSS feed URL for my Followed series. Also there is another conversation for this under Bugs: https://mangadex.org/thread/72578

 

[shroomz]

yea hopefully they will change DDOS protection anytime soon, cant grab followed titles either and its kind of a bother

 

[mochi]

+1

 

[BzzBzz]

Why don't request your rss-reader's devs for cloudflare support?

 

[galinkinlin]

As a concept, BzzBzz, I'm not sure that makes much sense. An rss-reader is essentially a bot. The DDOS protections are specifically designed to block bots. Even if they could get around the protections sometimes, they couldn't get around them consistently.

They could turn off the protections for their RSS (especially since it requires login). They could add rate-limiting. They could ban accounts that hit feeds more than 10k times a day. They could enforce that you can only be logged in from one box at a time. They could also tune their threat threshold lower, so that cloudflare isn't constantly checking everyone. Hell, they could maybe do a whitelist of people who are using RSS responsibly.

There are many options, but updating the reader is probably the hardest and one of the options least likely to be successful. There are a couple readers, I think, that work fine, but the 3 I've tested with all fail.

 

[WildCrow]

Same here can't get my RSS feed either, I mean I have the update frequency every 120mins IF i have my rss client open on the morning or afternoon , that's not a bot for sure

 

[BzzBzz]

> They could turn off the protections for their RSS

They can't actually. For bots it's irrelevant what service to use to overload server.

 

[nunopcampos]

Hi, I'm using feedly to follow different titles and it works fine for the one's I've already added.
However, I've been unable to add new rss feeds from MangaDex for some time now.
Seems like it could be related to the DDoS protection.

Any ideas?

 

[aquamarin]

This seems to be an unfortunate trade-off of using Cloudflare. You can't just exclude RSS pages because they are just as vulnerable to attack.

There is no easy solution. Apparently Cloudflare rate limiting can work, but that comes at a cost.

We're poor and this is the poor man's solution.

 

[wedge1001]

that's really annoying, though.

But since i'm not here to only bash around:
What about the following proposal:
We / you create a reverse proxy wich will use rss.mangadex.org as domain and is not protected by cloudflare. (we can go for an ovh/online/whatever is cheap and has a little bit of DDOS-Protection)
This will "just" be a reverse proxy to the mangadex/rss site (either points at the not CF-Protected IP or do a whitelisting in CF).
We then can define a rate-limit for each IP - e.g. ~5/10/... requests per Minute per IP (this should effectivly be nothing to the backend-System)
Also we can add caching for the rss-feeds (personally i don't care if i have to wait 1 hour to get my feed updated or not - i just want to get the update) in order to minimize the requests to the backend.

I can even think about sponsoring and maintaining this rss-system for you

 

[Raneer]

This includes the "recent updates list", right? Because 90% of the time, the it only shows half of the first page or nothing at all.

 

[marshmallowcat]

Bumping as I've hit this exact same issue using Feeder. Maybe a suggested new feature for automated delivery of new entries in our Follows pages (perhaps in bulk or delayed intervals to ease the bandwidth on your side) to user email addresses? I'd assume that would bypass CF since data is only being sent from your side, and that would also be a nice supporter or "pro" feature I'd be willing to donate for.

SMS delivery and/or an actual standalone mobile app would also work wonders, but those are understandably much more work to implement.

 

[Holo]

We are considering what upgrades to make next. We want the next upgrade to give the site at least 6 months of room for growth.

 

[Shinonome]

I've got the same trouble with my rss client (quite-rss). Can't get updates anymore for older/already added feeds and can't add any new one's. Granted I'm not using my login/pass in the client and not sure that would solve the problem.

 

[Shinonome]

Update: since this morning seems quiterss (my client) works again, and I just added a new feed as well.
Thanks ^^

P.S. I didn't do/touch anything on my net/client so it's not on my side I guess.