Website: Add external Tracker integration [Rejected]

[ThreeFold]

I would really like to see some sort of native AniList integration. However, not making it generic would be a bad idea, so I suggest in here that MD shoudl add third party tracking as a backup. Especially in the case where MangaDex were to go dissappear again.
If this isn't really in the cards, I would love some sort of way to put in my profile outside of the about me page. I think enough people here use some sort of third party tracker to actually make use of it.
Recommended Tracking Sites:

• AniList
• MyAnimeList
• Kitsu

In the meantime MAL-Sync works really well.

 

[Remocracy]

This so much. Maybe it could also sync existing? I have 3k manga and I will not manually go through all of those.

 

[ThreeFold]

This so much. Maybe it could also sync existing? I have 3k manga and I will not manually go through all of those.

I would think that it could be some sort of manual import method, where the user has to resolve discrepancies themselves. \


As a follow on, I am a software engineer with 3 years of full-stack experience and 6 years of professional experience. I'm more than happy to help out if need be.

 

[Remocracy]

I would think that it could be some sort of manual import method, where the user has to resolve discrepancies themselves. \


As a follow on, I am a software engineer with 3 years of full-stack experience and 6 years of professional experience. I'm more than happy to help out if need be.

Maybe there could be a way to choose which source to trust. Like MangaDex could just overwrite the data on the remote end, except if the remote end’s number of chapters read is higher than on MangaDex.

 

[ThreeFold]

Maybe there could be a way to choose which source to trust. Like MangaDex could just overwrite the data on the remote end, except if the remote end’s number of chapters read is higher than on MangaDex.

I've been through the ringer on discrepancy resolution, the best option is to always pick newest data if possible, but manual resolution is and should always be preferred.

 

[Remocracy]

I've been through the ringer on discrepancy resolution, the best option is to always pick newest data if possible, but manual resolution is and should always be preferred.

It might get annoying if I read chapter 90 on an aggregator but then I wanna revisit chapter 57 on MD (I never read it there before)

 

[Tristan__]

I would really like to see some sort of native AniList integration. However, not making it generic would be a bad idea, so I suggest in here that MD shoudl add third party tracking as a backup. Especially in the case where MangaDex were to go dissappear again.
If this isn't really in the cards, I would love some sort of way to put in my profile outside of the about me page. I think enough people here use some sort of third party tracker to actually make use of it.
Recommended Tracking Sites:

• AniList
• MyAnimeList
• Kitsu

In the meantime MAL-Sync works really well.

This sounds good in theory and I agree it'd be a great feature, but doing it'd be horrible to implement, very annoying at best.

• MyAnimeList only got their api and their documentation, considering how long MAL exists, very recently... And it... isn't the best. They also didn't publish their rate limits, which would be very annoying to work with.
• AniList has a better documentation than MAL and have oauth too, but they have horrible rate limits, which is per ip address too and not per oauth token.
• Kitsu has a comprehensive documentation and oauth. But they don't publish their rate limits, which means you'd have the same problems as with MAL.
• MangaUpdates: they got their api very very recently and their rate limit is "make requests until you get banned", which means you have the same problems like with MAL and Kitsu
• Novelupdates: novels?

Not to speak of MangaDex having to store and refresh authentication tokens, which is a no go because MangaDex would have to make their server ip addresses known. A developer already said on Discord that MangaDex has no plans to make requests to 3rd party services.

 

[tristan9]

Once we open OAuth client registration, anyone can develop a service to handle this. We barely have the time to fix our site's data and keep it alive as is, so it's unlikely we'd be able to start integrating with third-parties for fun.

Also these third-parties might not be as enthusiastic at the idea either, due the nature of MangaDex, and our users might think we're selling their data or something, even if it's opt-in.

Overall, it's highly unlikely we would ever implement any of this, even though it'd be sweet in some ways.

 

[Tristan__]

I would think that it could be some sort of manual import method, where the user has to resolve discrepancies themselves.

Sure, that'd maybe work. But that also means users would have to theoretically sync tens of thousands of library entries (the library limit is the sky) with api's that don't have a very friendly rate limits.
It also needs developer time, which isn't something the devs have until at least v5 has feature parity with v3.

 

[ThreeFold]

This sounds good in theory and I agree it'd be a great feature, but doing it'd be horrible to implement, very annoying at best.

• MyAnimeList only got their api and their documentation, considering how long MAL exists, very recently... And it... isn't the best. They also didn't publish their rate limits, which would be very annoying to work with.
• AniList has a better documentation than MAL and have oauth too, but they have horrible rate limits, which is per ip address too and not per oauth token.
• Kitsu has a comprehensive documentation and oauth. But they don't publish their rate limits, which means you'd have the same problems as with MAL.
• MangaUpdates: they got their api very very recently and their rate limit is "make requests until you get banned", which means you have the same problems like with MAL and Kitsu
• Novelupdates: novels?

Not to speak of MangaDex having to store and refresh authentication tokens, which is a no go because MangaDex would have to make their server ip addresses known. A developer already said on Discord that MangaDex has no plans to make requests to 3rd party services.

Not surprising that Rate limiting is so bad. I guess MAL-Sync will still be the go to since it can run on a client connection which would get around most of the annoying rate limits.

 

[ThreeFold]

Once we open OAuth client registration, anyone can develop a service to handle this. We barely have the time to fix our site's data and keep it alive as is, so it's unlikely we'd be able to start integrating with third-parties for fun.

Also these third-parties might not be as enthusiastic at the idea either, due the nature of MangaDex, and our users might think we're selling their data or something, even if it's opt-in.

Overall, it's highly unlikely we would ever implement any of this, even though it'd be sweet in some ways.

This is probably the solution. After registration is open, I can write some script that syncs my updates nightly just in case something was missed by MAL-Sync. Plus, a script to sync stuff from AL to MD.

Since this is a no-go, I think that it's probably really good to add some way to export my reading list in a MAL compatible format so that I could import it into both MAL and AniList with their native solutions. Though I could see this as also problematic due to the nature of having to reach out to grab up to date MAL IDs for manga.

 

[Tristan__]

Though I could see this as also problematic due to the nature of having to reach out to grab up to date MAL IDs for manga.

The MAL id can be added to each title, which means it's possible to just export everything that has the MAL id field set. Same with AL.
MAL and AL ids don't change, so it's impossible to get the wrong id, unless someone added the wrong id.

 

[Remocracy]

Not surprising that Rate limiting is so bad. I guess MAL-Sync will still be the go to since it can run on a client connection which would get around most of the annoying rate limits.

So I guess it's time to pester them for a feature to auto add all MD manga into their DB. My one problem with MAL-Sync is that they'd ignore the "MAL" field and make me go through the search feature.

 

[LunaIsNotABoy]

This sounds good in theory and I agree it'd be a great feature, but doing it'd be horrible to implement, very annoying at best.

• MyAnimeList only got their api and their documentation, considering how long MAL exists, very recently... And it... isn't the best. They also didn't publish their rate limits, which would be very annoying to work with.
• AniList has a better documentation than MAL and have oauth too, but they have horrible rate limits, which is per ip address too and not per oauth token.
• Kitsu has a comprehensive documentation and oauth. But they don't publish their rate limits, which means you'd have the same problems as with MAL.
• MangaUpdates: they got their api very very recently and their rate limit is "make requests until you get banned", which means you have the same problems like with MAL and Kitsu
• Novelupdates: novels?

Not to speak of MangaDex having to store and refresh authentication tokens, which is a no go because MangaDex would have to make their server ip addresses known. A developer already said on Discord that MangaDex has no plans to make requests to 3rd party services.

I just wanted to point out that as someone who is close friends and has worked with Kitsu, the rate limiting should be on their GitHub. I can't remember it off of the top of my head, but I'll edit once I find it out.

 

[Remocracy]

MangaUpdates: they got their api very very recently and their rate limit is "make requests until you get banned", which means you have the same problems like with MAL and Kitsu

From my experience they’ll just give you HTTP 429s if you go too fast, but it’s generally 1 req/2 seconds that stops that.

 

[LunaIsNotABoy]

Once we open OAuth client registration, anyone can develop a service to handle this. We barely have the time to fix our site's data and keep it alive as is, so it's unlikely we'd be able to start integrating with third-parties for fun.

Also these third-parties might not be as enthusiastic at the idea either, due the nature of MangaDex, and our users might think we're selling their data or something, even if it's opt-in.

Overall, it's highly unlikely we would ever implement any of this, even though it'd be sweet in some ways.

Kitsu really wouldn't care from my understanding.

 

[Tristan__]

it’s generally 1 req/2 seconds

which would never be enough for MangaDex

 

[Tristan__]

Kitsu really wouldn't care from my understanding.

They would care if MangaDex hammers them with thousands of requests per second, if we take active user stats into account.

 

[Remocracy]

They would care if MangaDex hammers them with thousands of requests per second, if we take active user stats into account.

which would never be enough for MangaDex

I think that problem would be solved by waiting 5-10 minutes before making a request. If a user reads another chapter the last request can be canceled and the new one can be queued.

 

[Tristan__]

I think that problem would be solved by waiting 5-10 minutes before making a request.

You wouldn't say this if you saw MangaDex's user stats from their plausible analytics and the server requests per second stats
Like Tristan 9 said in his dev blog post, "MangaDex is bigger than you think. No, really.". From what I know, the avg. requests per second rate is multiple times higher now.