What is the status on RSS news feeds? [Rejected]

[doppler]

The last of the old board to happen. Or did it happen? I don't see links for the feed(s).

RSS is so 90's. But I like it.

 

[RogueKitsune]

Anything that was on the old site will eventually be reimplemented. Quality of life features, such as RSS, are the lowest priority.

 

[doppler]

eventually be reimplemented.

I can't remember a single old board feature still needing to be re implemented. I agree it's a lowest priority. But I like it to be informed of good stuff I am watching. Not everything I watch needs immediate attention. Solo leveling was one of those phenomenon's, I always wanted to view, ASAP. Mairimashita! Iruma-kun IMHO is one of the few worth while to me.

 

[kouyo]

I can't remember a single old board feature still needing to be re implemented.

Aside from RSS, view count of titles, chapters, and groups are still missing. We have a larger volume of people asking for those, so that's the next big thing devs are working on.

 

[doppler]

Aside from RSS, view count of titles, chapters, and groups are still missing.

Ok, the more knowledge you collect the more power you have. BwaaHaaHaa, I am now more powerful.

 

[shwiftyshfifty]

Aside from RSS, view count of titles, chapters, and groups are still missing. We have a larger volume of people asking for those, so that's the next big thing devs are working on.

+1 for can has RSS pllssssss.

 

[kirbo]

+1 for can has RSS pllssssss.

I concur

 

[doppler]

@shwiftyshfifty @kirbo
I concur too, but other things are not finished yet. (they are working on it). As told by devs, RSS wil most likely be last thing done. RSS is ancient, just like IRC. Both are throws backs to the BBS and text message boards days.

RSS up and running is no small en devour.

 

[kirbo]

@shwiftyshfifty @kirbo
I concur too, but other things are not finished yet. (they are working on it). As told by devs, RSS wil most likely be last thing done. RSS is ancient, just like IRC. Both are throws backs to the BBS and text message boards days.

RSS up and running is no small en devour.

Depending on the implementation (https://mangadex.dev/mangadex-v5-infrastructure-overview/) , the feed page is a dump of SQL table stuff. All you need to do is parse the last 25 feed items into a very basic xml (https://www.w3schools.com/xml/xml_rss.asp) and in the user info tables attach rss file path for each user. which you can link to on the profile or something. Obviously make it so that the file path is pseudo random to stop people like, sequentially going through feeds. At least thats how I'd do it on a small scale server. I know that they're not really dealing with that and security is top priority. But like, stop acting like this shit is hard. Now, how a server acts to having to update 60,000 rss feeds cause a popular manga drops would be interesting however its already kind of already tasking the server every time somebody checks their update page cause its asking for new stuff.

My apologies for the rant, I'm sure the devs could do it in 3 days + code review + qa. But yeah they prioritizing other, much harder stuff but not soley like incrementing a counter or two on a page click or on the flag of a manga closing (just with catching duplicates from the same users).

 

[Fenatix]

I'm just going to necro this. I need my feeds back!

 

[kouyo]

Hi everyone, unfortunately there aren't any plans to bring back RSS, please stay tuned.

 

[boursk]

I was certain it was on the list of planned features before. I guess this post above was meant to show that this feature has been removed from the list, given the edit on planned features post being close to the post above? I'm also not sure what to stay tuned for if there are no plans implement it (anymore). In case it goes back on the planned features list?

This feature is the reason why I ended up creating my account: just so I could make and use my own server app that produces an RSS feed for newly uploaded English translated chapters of the manga I'm reading. It's fairly easy to do if you use the MangaDex API. I'd rather have an RSS feed provided by MangaDex myself as well, but there's at least a way to get this if you really want it...

 

[kouyo]

Hi everyone.

To clarify, the overhead required to support it is quite high relative to its usage. We’ve decided to prioritize other features, making an RSS rollout unlikely.

 

[boursk]

To clarify, the overhead required to support it is quite high relative to its usage.

Can you expand on that a little bit more?

Is there a concern that people will setup the RSS feed in their RSS client and never make use of it (i.e. click on a link from the RSS feed)? That would be a situation where MangaDex would incur the overhead for no reason. But then you could do something like have every RSS link reset a "last accessed" value and disable the RSS feed when the last accessed value is longer than a certain period.

Outside of that, if people don't subscribe to the RSS feed, they're not polling the server, so there shouldn't be any (meaningful) overhead. Or is the concern that the overhead per RSS subscription so high that no type of caching could keep the total overhead low enough for MangaDex to handle the strain (but then I don't understand the "relative to its usage" part)?

 

[kouyo]

The reality is that RSS is a "legacy" technology. The overhead isn't just server load, it's development cost. To implement your suggestion of a "lease" or "auto-disable" system, the devs would have to write, test, and maintain a custom tracking engine just to keep a niche feature from crashing the servers.

In a world where 99% of users are using the API or the site, spending that much engineering time on the 1% who use RSS readers is hard to justify.

 

[boursk]

But if you're just talking about development cost, we're just going back to the demand issue (which was what your original post talked about before it was deleted). Then again, I'm kind of sensing that you're also hinting at some prejudice against the technology itself by starting out to emphasize that RSS is a "legacy" technology?
I'm obviously not trying to attack you, but it does give the impression that it wouldn't matter if there was widespread demand for the feature or not; it's not going to be implemented regardless. Which would feel a bit unreasonable if true, but it'd be a stance you're entitled to take.

So in the end, is it about upvotes in this thread (since theoretically given enough demand, it would justify whatever development cost) or is there a clear stance that it won't be implemented, regardless of the demand for the feature?

a custom tracking engine

That's kind of blowing up the situation. 😅

All you'd have to do is add a column in the user table (or any other low cost resource to retrieve data from to avoid changing the user table from being static data to dynamic data) that keeps track of the last accessed dtime, then add an endpoint specifically for the RSS links in the feed that will update the field and redirect to the selected manga chapter. The feed link itself would check the last updated field before doing the expensive call and reject it if it's past a certain period.

It's neither complicated nor prohibitively expensive to implement/test/maintain.

 

[kouyo]

I understand your thoughts about this decision, but:

I'm kind of sensing that you're also hinting at some prejudice against the technology itself by starting out to emphasize that RSS is a "legacy" technology?

When I say "legacy", I’m referring to the architectural mismatch. Modern web scaling relies heavily on granular API calls and push-based notifications. Adding an RSS layer, even a "simple" one, introduces a different caching and polling pattern that we have to monitor, secure, and maintain forever.
When 99% of our traffic is optimized for the API/Site, it’s hard to justify shifting focus to a protocol that the broader industry has largely moved away from in favor of more efficient delivery methods.
Mind you I have plenty of respect for RSS, but in the context of a platform at MangaDex's scale, "legacy" is shorthand for "high maintenance for low reach".

All you'd have to do is add a column in the user table (or any other low cost resource to retrieve data from to avoid changing the user table from being static data to dynamic data) that keeps track of the last accessed dtime

Even a "simple" database column for tracking last-accessed dates adds write-load and complexity to our schema. It’s less about a dislike for the tech and more about the fact that every hour spent on RSS is an hour taken away from features that the vast majority of our users are asking for.

 

[boursk]

@kouyo
I appreciate you taking the time to writing a response. And thanks for giving me the opportunity to have a dialog with you about this matter. When I wrote my previous post, I didn't notice that the thread title changed to reflect that the suggestion was in fact rejected. That would've given me a clearer indication of the state of the suggestion than what I read before. So while it's clear to me now that a clear decision has been made, I still feel compelled to respond to some of the things you wrote.

What I find difficult to get a handle on is the "real" reason why the decision was made to definitively reject it. As far as I understand it, the reasons given are 1) the implementation of an RSS feed wouldn't match our current architecture, and 2) we have more popular things to spend our time on.

To be fair, I haven't seen MangaDex's architecture, so it's not fair from my side to comment on it without seeing the full picture. Though it's hard to sympathize with the architectural argument when the API has all the essential parts needed for RSS: /user/follows/manga/feed and /list/{id}/feed
No need for any "complicated" caching mechanisms if there's already an anti-flood system in place. No need for additional measures needed for supporting the polling pattern, as long as the polling adheres to the rate limit. The rate limit already defines what MangaDex considers acceptable traffic. And this is nowhere near as strict for users to experience an adequately usable RSS feed (e.g. my RSS client is set to poll every 10 minutes and that's fine for me).

It still takes development time though. A new endpoint (or otherwise add an optional output parameter in the existing endpoints) would have to be created that outputs XML instead of JSON. And yes, a new non-sequential static value that's linked to the user to function as an "RSS token" would be preferred to bypass the need to stay authenticated in order to access the followed manga feed, but otherwise it can also just be an alternative version of /list/{id}/feed that only works with public lists. That would make the minimum viable solution just a simple change in the output format of what is currently already implemented.

And I fully appreciate the popularity argument. That said, it's not an argument to outright reject the feature. Because it still implies that given enough demand, it could be considered worth spending valuable time and resources into implementing it.
I am fairly surprised that there's not enough support for a quick and easy way to check your feed/get notified about newly released chapters, regardless whether it's through push messages or polling (RSS). Maybe when people read RSS, it doesn't really register for most people, but maybe people really don't care about it. It is what it is.

So that's what leads me to believe there's some reason beyond what you've been saying that gave this feature a permanent rejected stamp. Regardless of the reason(s) (even in the case of the absence of one), it's a choice that MangaDex is entitled to make, so that's something I'll have to accept. The most I can ask is to keep the door open for the future and consider this (comparatively) low hanging fruit in the absence of push notifications (or any other preferred solution) for our manga feed.

 

[kouyo]

I appreciate the technical breakdown, but there’s a fundamental difference between a feature being possible and a feature being sustainable. At MangaDex’s scale, our most expensive resource isn't if it's possible or not, it’s developer focus. MangaDex it's a big site with a small development team.

If a site-wide critical bug pops up, but the only dev available is tied up fixing a parsing error for a niche RSS feed, the entire platform loses. You're right that the data is in the API, and the code isn't "hard" to write, the issue is the technical debt and operational noise. Every "simple" feature is a permanent increase in our attack surface and a potential point of failure during every future migration.

For example, your suggestion of an "RSS token" to bypass authentication is a perfect example of scope creep. To do that securely, we would have to build a new UI for users to generate, revoke, and manage those tokens. That moves us from a "simple format change" into an entirely new security and session management project.

Regarding the "Rejected" status:
This isn't prejudice against RSS, it’s about clarity. Keeping the door "half-open" for every niche request creates a stagnant backlog that weighs us down. By marking it rejected, we’re being honest so that users like you can continue leveraging the API for custom solutions, rather than waiting for a native rollout that isn't coming.

The API is robust precisely so you can build your own bridge. We prioritize a flexible API over specific protocols because it allows the 1% who need specialized tools to build them without slowing down the features the other 99% are asking for.

 

[boursk]

I'm glad we've moved away from arguments about the feature being an "architectural mismatch". That just leaves the argument of whether there is enough support to justify the dev resources to implement and maintain the feature.

If a site-wide critical bug pops up, but the only dev available is tied up fixing a parsing error for a niche RSS feed, the entire platform loses.

Surely I don't have to explain that any high priority production issue would just have the dev commit and push their current changes to their feature branch, checkout the current master, make a bugfix branch and work on the issue immediately. It's kinda crazy to suggest that devs can't quickly switch contexts because they're in the middle of some other feature/lower priority bug. 😓

It's unfortunate that you bring this up:

For example, your suggestion of an "RSS token" to bypass authentication is a perfect example of scope creep. To do that securely, we would have to build a new UI for users to generate, revoke, and manage those tokens. That moves us from a "simple format change" into an entirely new security and session management project.

...when in my post I explicitly reduced the scope of the change down to the absolute bare necessities (i.e. the minimum viable solution). That's the direct opposite of scope creep. I was trying to show that while I know that the feature can be better than just the MVS, I understand that there's already an issue with support for the feature to begin with. So anything beyond the MVS wouldn't be a feasible ask, so I'm more than happy to content with just a different output format that will directly be usable with my RSS client.

I understand wanting to reduce the number of open suggestions by rejecting specific items that haven't garnered enough support over a specific time, but the policy seems to go against the fact that there are pages of suggestions that have been open for a long time with less support/upvotes, yet without a rejected tag. Typically when a rejected tag is added, it seems to be there because there is a specific reason why it can't or shouldn't be added. Lack of support (should) mean(s) it can be added and there is value to it, but there is just not enough demand (yet) to justify investing the resources to develop and maintain the feature.

The API is robust precisely so you can build your own bridge. We prioritize a flexible API over specific protocols because it allows the 1% who need specialized tools to build them without slowing down the features the other 99% are asking for.

I will admit that this does resonate a lot more than most of the other arguments given. But then I also get reminded of the fact that hardly everyone has the means and capabilities to build their own solutions using the API. So they'll either have to content with what's natively supported or take the plunge and hope someone made a script that does exactly what they need and won't break their computer/steal their information.

All in all, I appreciate wanting the software to be as lean as possible in order to reduce waste and maintenance as much as possible. Personally I feel that the given objections are a little too blown out of proportions given the situation, but like I said, it's a choice that MangaDex is entitled to make and for users to accept. As always, I appreciate the friendly and constructive dialog.