[EN] How to export your library to Anime-Planet, AniList, Kitsu, MAL & MangaUpdates

[Bireus]

Source:

https://www.reddit.com/r/manga/comments/1lp7kyl

Github files: https://github.com/Seriousattempts/MangaDex
YouTube video:

It shows that first you get your MangaDex

• Username
• Password
• personal client id
• Secret

You get your personal client id and secret under API Clients in Settings https://mangadex.org/settings by creating them https://api.mangadex.org/docs/02-authentication/personal-clients

Run Export libraries from MangaDex.py

• Paste your MangaDex information at the top
• Choose a location to export the .xlsx files to
• Choose what you want to export. You can export individual files or your entire library

They export as .xlsx files.

From there you either convert them to .xml https://github.com/Seriousattempts/MangaDex/releases/download/V1/Convert.xlsx.to.xml.MAL.py to import to sites like AniList, Anime-Planet, Kitsu or MAL.

Or by Importing them to Mangaupdates https://github.com/Seriousattempts/MangaDex/releases/download/V1/Mangaupdates.MD.List.py.

 

[NightmareOkami]

Source:

https://www.reddit.com/r/manga/comments/1lp7kyl

Github files: https://github.com/Seriousattempts/MangaDex
YouTube video:

It shows that first you get your MangaDex

• Username
• Password
• personal client id
• Secret

You get your personal client id and secret under API Clients in Settings https://mangadex.org/settings by creating them https://api.mangadex.org/docs/02-authentication/personal-clients

Run Export libraries from MangaDex.py

• Paste your MangaDex information at the top
• Choose a location to export the .xlsx files to
• Choose what you want to export. You can export individual files or your entire library

They export as .xlsx files.

From there you either convert them to .xml https://github.com/Seriousattempts/MangaDex/releases/download/V1/Convert.xlsx.to.xml.MAL.py to import to sites like AniList, Anime-Planet, Kitsu or MAL.

Or by Importing them to Mangaupdates https://github.com/Seriousattempts/MangaDex/releases/download/V1/Mangaupdates.MD.List.py.

Ehe not on computer ehe

 

[Darthvader69420]

Yo dude the python file doesn't run. Crashes as soon as I run it. Any solutions?

 

[boursk]

Yo dude the python file doesn't run. Crashes as soon as I run it. Any solutions?

Just checking, because the readme on github mentions the following and the OP does not:

Have python with the following installed:

• pandas
• requests

Do you have those installed? If so, does it show any error messages or anything that could help people help you?

 

[Bireus]

Yo dude the python file doesn't run. Crashes as soon as I run it. Any solutions?

There's also a .exe release out if you have access to Windows

 

[dasdrak]

this script does not get the correct latest read chapter

 

[boursk]

this script does not get the correct latest read chapter

Yeah, from what I can see, the script basically picks up the last volume and last chapter that are available on MangaDex and puts that in their respective fields.

Spoiler: A more technical explanation

The script first calls /manga/status to get all the manga you've marked with a reading status. It will then iterate through each entry and adds all the metadata for each manga by calling /manga/{id}, which contains fields for lastVolume and lastChapter. But as I mentioned before, they have nothing to do with your reading status.

The problem is that a user can arbitrarily read manga chapters here and there and MangaDex will keep track of these individual chapters. These can be retrieved using /manga/read or /manga/{id}/read. If you would just assume that you've read every chapter up till the last chapter you've read on MangaDex, then the script would have to go through every chapter ID in the response for every manga you've marked with a status and check what the largest chapter number is.

The problem is very fixable, but it's just not small enough for me to just go like "just replace this line with this line" and it'll work. I'm not sure if the tool is actively being maintained. The .exe file was added 5 days ago, so maybe it is, in which case you could try to add an issue on github. Unless Bireus is actually the maintainer, but given he linked to reddit as the source, I assumed he isn't...

 

[Darthvader69420]

Just checking, because the readme on github mentions the following and the OP does not:

Do you have those installed? If so, does it show any error messages or anything that could help people help you?

I dont have pandas or request downloaded.

There's also a .exe release out if you have access to Windows

Thanks for the info! There are 3 exe files in the github page. Do you know which one I should install?

 

[boursk]

I dont have pandas or request downloaded.

You can just google python panda/requests to find them.

Thanks for the info! There are 3 exe files in the github page. Do you know which one I should install?

The .exe files are just wrappers for each of the three python files. Use the export first, then either use the MAL or MangaUpdates one depending on what you want to do.

 

[Darthvader69420]

You can just google python panda/requests to find them.


The .exe files are just wrappers for each of the three python files. Use the export first, then either use the MAL or MangaUpdates one depending on what you want to do.

I use anilist so I assume i need to use MAL one?

 

[boursk]

I use anilist so I assume i need to use MAL one?

I don't use anilist, but when reading the github readme, I assume they have a MAL import feature:

2. Choose the external website to export your files to:
MAL: Grab your user_id by exporting your list (https://myanimelist.net/panel.php?go=export), extracting the .gz file and opening the .xml file with notes or another program. Format should look like this:

        <myinfo>
            <user_id>123456</user_id>
            <user_name>YourMALname</user_name>
            <user_export_type>2</user_export_type>
            <user_total_manga>2345</user_total_manga>
            <user_total_reading>678</user_total_reading>
            <user_total_completed>901</user_total_completed>
            <user_total_onhold>234</user_total_onhold>
            <user_total_dropped>567</user_total_dropped>
            <user_total_plantoread>0</user_total_plantoread>
        </myinfo>

Run Convert .xlsx to .xml MAL.py and use that user_id. Choose the proper .xlsx file with the proper status and convert to .xml.

The following sites use .xml to import:

• AniList: https://anilist.co/settings/import
• Anime-Planet: https://www.anime-planet.com/users/import_list.php
• Kitsu: https://kitsu.app/settings/imports
• MAL: https://myanimelist.net/import.php

For Mangaupdates, you run Mangaupdates MD List.py and use your normal login details.

Inb4 you also find out that the script doesn't properly export your read progress from your MangaDex library...

 

[Darthvader69420]

Inb4 you also find out that the script doesn't properly export your read progress from your MangaDex library...

thats fine for me. I just need my stuff out of here before they start nuking stuff.

 

[AnonymousContributor]

the worst part about these other sites is that there's shit not in MAL because it's obscure.

 

[GabrielH]

https://imgur.com/a/auth

Okay already installed python, pandas, and requests. Tried .exe for exporting from mangadex but got auth error. Is this issue from my end?

 

[Bireus]

Okay already installed python, pandas, and requests. Tried .exe for exporting from mangadex but got auth error. Is this issue from my end?

I don't speak dev so the only thing I can assume is either you didn't do the creation of the client & secret key, using a vpn or if you're using 2 factor didn't include your code to the end of your password to think off the top of my head.

 

[boursk]

https://imgur.com/a/auth

Okay already installed python, pandas, and requests. Tried .exe for exporting from mangadex but got auth error. Is this issue from my end?

It might be me, but I can't see the image, so I don't know what the actual error message is you're getting. The description of an "auth" error sounds like there's an issue with the script being able to "login" using the API with the credentials you've given.
I'm assuming you're either getting "✗ Authentication failed: {r.status_code} - {r.text}" or "✗ Authentication error: {str(e)}". It would be useful to see what the full error message is.

Maybe you're saying you tried both, but you can either install python (with the pandas and requests plugins) and run the .py files with python, or only download the .exe files and run those (as the .exe files will have everything you need to run the scripts packaged into it). You shouldn't need to do both. But if you've tried both ways and failed with both, then I get what you're saying.

When looking at the github page, I noticed the maintainer actually did a complete rewrite of the script. The new V2 should now also properly extract the last chapter/volume read values. It's a bit weird these new scripts weren't properly checked into the repository, but are only available in the release pages though...

My recommendations are to first double, triple, quadruple check you've entered the correct username, password, client ID and client secret.
If you're a 100% sure that's correct, try using the older script. If that works, there's a chance there might be a bug in the V2 script.
If all fails, you can either create an issue on github, contact the man on discord or if for some reason you'd rather have 3rd party people who can't really change the code look at it, try to write down whatever the error is you've been given in this thread.

 

[GabrielH]

It might be me, but I can't see the image, so I don't know what the actual error message is you're getting. The description of an "auth" error sounds like there's an issue with the script being able to "login" using the API with the credentials you've given.
I'm assuming you're either getting "✗ Authentication failed: {r.status_code} - {r.text}" or "✗ Authentication error: {str(e)}". It would be useful to see what the full error message is.

Maybe you're saying you tried both, but you can either install python (with the pandas and requests plugins) and run the .py files with python, or only download the .exe files and run those (as the .exe files will have everything you need to run the scripts packaged into it). You shouldn't need to do both. But if you've tried both ways and failed with both, then I get what you're saying.

When looking at the github page, I noticed the maintainer actually did a complete rewrite of the script. The new V2 should now also properly extract the last chapter/volume read values. It's a bit weird these new scripts weren't properly checked into the repository, but are only available in the release pages though...

My recommendations are to first double, triple, quadruple check you've entered the correct username, password, client ID and client secret.
If you're a 100% sure that's correct, try using the older script. If that works, there's a chance there might be a bug in the V2 script.
If all fails, you can either create an issue on github, contact the man on discord or if for some reason you'd rather have 3rd party people who can't really change the code look at it, try to write down whatever the error is you've been given in this thread.

The error is;

uthentication error: HTTPSConnectionPool(host='auth.mangadex.org', port=443): Max retries exceeded with url: /realms/mangadex/protocol/openid-connect/token (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002995C63BE50>, 'Connection to auth.mangadex.org timed out. (connect timeout=None)'))

from my barest understanding, this issue shouldn't be from the program provided right. Also tried older exe file from last week got the same issue.

 

[boursk]

The error is;

uthentication error: HTTPSConnectionPool(host='auth.mangadex.org', port=443): Max retries exceeded with url: /realms/mangadex/protocol/openid-connect/token (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002995C63BE50>, 'Connection to auth.mangadex.org timed out. (connect timeout=None)'))

from my barest understanding, this issue shouldn't be from the program provided right. Also tried older exe file from last week got the same issue.

Ah, connection timeouts (thanks for typing out the error!). Basically, MangaDex isn't responding or isn't responding in a timely manner to the script's request to authenticate with it. This can just happen sporadically, but if it happens consistently, it is an issue. Given you're able to login to the site and post here, you can obviously access the domain auth.mangadex.org. I know I can use the API to authenticate myself. So we can rule out a lot of generic things because of that.

Things you can try:

• Try running the script and if it fails, give it a minute and try again
• If you haven't done so yet, try using the python script instead of the .exe file (I expect them to be the same, but it's better to test than to assume)
• If it still doesn't work, it could be worth trying to authenticate using the API yourself manually and see what kind of response you're getting

Spoiler: How to manually authenticate using the API

First, you're going to need a client. You can use whatever you want, but I prefer to use Postman (when prompted, you don't have to create an account to do this). Then fill in the following and press "Send":

If you get a response like in the picture (where it starts with "access_token"), then you've successfully authenticated with MangaDex using their API. It might be worth noting how long it takes to do the request (shown next to "200 OK"). Anything less than a couple of seconds should be fine.

Under no circumstance should you post or share this successful response unredacted anywhere!

If you keep getting timeouts through postman as well (don't spam the request though! Just try it on different times throughout the day), then there's something wrong with the connection between you and MangaDex.

If you get a different error, then that'll be interesting information to further look into.

If you still get timeouts or get a successful response within a reasonable time when authenticating through the API, you might have to contact MangaDex support in order to troubleshoot further. The script maintainer is essentially using the exact same way to do the authentication with MangaDex as the code example given by MangaDex in their API documentation, so outside of changing the timeout settings (which seem to have been set to "None" from the error message) and retry mechanism (which won't help much if it times out consistently), there's not much the script maintainer can do.

If the request is successful, but takes a really long time, we could try things like explicitly setting the timeout to a large number or find a time of day where the response won't take quite that long, but otherwise you might be out of luck.

Any other error might be interesting to share here or you can also just directly go to MangaDex support.

 

[GabrielH]

If the request is successful, but takes a really long time, we could try things like explicitly setting the timeout to a large number or find a time of day where the response won't take quite that long, but otherwise you might be out of luck.

Any other error might be interesting to share here or you can also just directly go to MangaDex support.

tried the scripts both new and old yet no luck, same auth issue. also tried in my other laptop same issue.

also tried your checking response time through postman, it's showing 6.69 s.

yeah it's connection issue, no problem accessing or checking through post data. I'll try to do it again in different times throughout the day, will be pain manually exporting over 1000 lists otherwise. Thanks for the help!

 

[boursk]

I thought it was surprising how there was a huge update that (among other features) now also fixes the export of the last read chapter/volume fields, but I see that hours after I mentioned it was odd that the V2 scripts weren't checked into the github repo, they did check them them in...I feel seen.

tried the scripts both new and old yet no luck, same auth issue. also tried in my other laptop same issue.

also tried your checking response time through postman, it's showing 6.69 s.

yeah it's connection issue, no problem accessing or checking through post data. I'll try to do it again in different times throughout the day, will be pain manually exporting over 1000 lists otherwise. Thanks for the help!

Anyway, it's good to see that you're able to authenticate through postman. The 6.69s response time is longer than I'd like to see it, but shouldn't be the direct cause of the issue. I'm wondering if the process log in the script showed more information. Would it be possible to copy/paste what's in there?
I'm particularly interested in whether there are non-error lines that might indicate multiple (successful) attempts at connecting with MangaDex. I can imagine that the script might bombard the server with requests and that MangaDex is refusing one somewhere half-way the process and then stops the entire process.

If that's the case, this could easily be fixed by the script maintainer with (longer) sleeps/retries+backoff factor.

I've been kinda working off the assumption that the script is failing at the very first request to MangaDex, since the error is on the URL that tries to authenticate with MangaDex, but the same URL is also used for refreshing the access token. Maybe the URL in the error changes between runs as well? It's kind of a hail Mary at this point, but I figure it would be better to verify these things and it would have a (relatively) simple fix if that was the case.