Dev news: Launching API version 2 (also v1 deprecation notice) | Last updated 2021-01-17

[UnlikelyChopshopNacho]

Is it expected that `/follows` returns a map with follow IDs as keys and not an array?

 

[gondolyr]

@TanmayMudgal619, @sttv: You need to include the chapter hash with the server in this format:

{server}{hash}/{page}

but you don't need to be logged in to view the pages.

For example, https://s2.mangadex.org/data/fcf3e1e5aa3e96d5281aba699be35a60/x1.jpg for your chapter @TanmayMudgal619.

 

[sttv]

@gondolyr It is strange that this is not required for some chapters. Some have the correct address. But thanks.
Ok, it's worked. Thanks

 

[TanmayMudgal619]

@gondolyr
Thanks for your help
How can i use data-saver server for images?
as it's mentioned in api.

 

[gondolyr]

@TanmayMudgal619 Using the

saver={1/true/on/yes}

query parameter will return a

server

value that has the data-saver server for the images. You would use this in the same way as I outlined above.

[ol]
[*] Get the server with the

saver

parameter. For example, https://mangadex.org/api/v2/chapter/152530?saver=1
[*]Using the same format as before, combine the

{server}{hash}/{page_name}

like so: https://s2.mangadex.org/data-saver/fcf3e1e5aa3e96d5281aba699be35a60/x1.jpg. Note that the server is using "/data-saver" instead of "/data" but without knowing if the server will always use that format or URL, I would always get the

server

value from the chapter endpoint.
[/ol]

 

[nonindustrial]

I've been working with v2 of the API and I've discovered an inconsistency related to the structure/datatype on the chapter.groups{} object.

/api/v2/manga/{id}?include=chapters

data.chapters.groups{} is

"groups":[1,2,3]

Where it should probably be structured as

"groups":[{"id":1},{"id":2},{"id":3}]

The reason why is that would then match the endpoint

/api/v2/chapter/{id}

which has data.groups{} as

"groups":[{"id":1,"name":"g1"},"id":2,"name":"g2"},"id":3,"name":"g3"}]

That one little datatype inconsistency prevents you from creating a Chapter{} object that could be used with both endpoints. Everything else appears to match correctly. Unfortunately, fixing it would cause another "Breaking."

 

[TanmayMudgal619]

@gondolyr
Thanks again to help me.😇

 

[Ruhrpottpatriot]

A few questions:
1. Is there a reason why true is defined as 1 in the docs? Because in classic computer science it's false that's defined as 0 and everything else is true, unless you're in the UNIX Shell, then it's 0 being true and 1 being false.
Regardless of that I'd probably constrain the definition a bit further and clearly state that 0 is false/no/off and 1 is true/yes/on. This would also be in line with the most common boolean systems and allow boolean arithmetic.

2. Any plans to change the API auth to something like a token? The current workflow isn't really suited for console based applications. I wanted to build my own analysis tool in python and going via the usual /login webpage in the app to retrieve the cookie values is a royal PITA. With a token you could expose an authentication endpoint where the user sends his username, password and 2FA token (which should be mandatory to access private information anyway) over an encrypted channel and gets back a token which then can be sent for every further request.
Or did I miss something entirely?

@nonindustrial
The current response is fine. Less bytes transmitted over the wire and the intent is clear enough already. What you propose would add unnecessary information.


P.S. For documentation this might be a worthwhile read: https://swagger.io/resources/articles/documenting-apis-with-swagger/

 

[glagan]

Could it be possible to add a meta field in the

followed-updates

endpoint with a

lastPage

attribute, I can know if it's the last page by counting the chapters, but I can't know when it will be the last page.

 

[Teasday]

2020-12-27
- Fixed

isHentai

in a manga's relations always showing up as false
- A manga's alt titles are now trimmed from surrounding whitespace

 

[Teasday]

I suggest pinging me if you want me to notice you, I tend to forget to keep track of things I'm supposed to (like this thread for example).

_{@Ruhrpottpatriot posted:}

1. Is there a reason why true is defined as 1 in the docs?

The reason is PHP's FILTER_VALIDATE_BOOLEAN

2. Any plans to change the API auth to something like a token?

For the proper v5 site's API, yes, we do have one. More importantly, we'll be issuing API keys and all that cool stuff. Not going to bother writing a proper token system for this one, though.

_{@nonindustrial posted:}

I've been working with v2 of the API and I've discovered an inconsistency related to the structure/datatype on the chapter.groups{} object.

I ran into it myself rewriting the reader and I agree I fucked up there, but unserializing it isn't really a huge issue (even if it is a bit ugly) because the manga endpoint still has

data.groups{}

so I'd rather not add a breaking change.

_{@UnlikelyChopshopNacho posted:}

Is it expected that `/follows` returns a map with follow IDs as keys and not an array?

Not really, that's another PHP fuckup that I overlooked. Doesn't really matter though, won't change.

 

[Ruhrpottpatriot]

@Teasday

The reason is PHP's FILTER_VALIDATE_BOOLEAN

Ah, so just a documentation not a technical issue. I'd suggest changing/clarifying that 0 is false and 1 is true. Good docs are important after all.

On that note I have some issue with the API itself:
1. Data is returned in an envelope stating the HTTP-Code, the status name (the "status" field) and then the data. This is entirely unnecessary as the status code names are fixed and the HTTP-Code can be read from the response itself. Overall this adds (unnecessary) things to parse and could be done with entirely. A more suitable response would just include the data or the error message encoded as {"message": "Foo"}. The user then can switch the parsing/deserialization logic based on the HTTP-Code.

2. The "relations" field contains unnecessary data. The "is_hentai" and "name" fields are not needed as the user can query the other information by another manga/{id} call. Yes that adds another call to the server, but any proper webserver can handle this. Should this be too much, problems elsewere are far more likely (e.g. the user querying too much already).

3. The language code in "publication.language" doesn't adhere to ISO639-1 nor ISO639-3, which is bad.

4. The "mainCover" field shouldn't return the URL to the cover but an id to the image. The image itself then can be retrieved from a dedicated endpoint (e.g. v2/manga/covers/{id}). This decouples the cover from the manga (as the URL currently has the manga ID in it) and allows backend server optimizations to only store the image once for the entire site. The same goes for the e.g. v2/manga/{id}/covers) endpoint. Maybe the covers endpoint could include the main cover and the field then could be removed from the main data itself.

5. There are some endpoints missing:
- Get all manga ids. This is more of a convenience endpoint but should be accessible via v2/manga/
- Get demographic id -> name mappings
- Get publication status id -> name mappings
-

6. Sometimes the api returns an incorrect status code querying v2/manga/ returns 400, where it should return 404. Yes, the message indicates that there was no valid manga id provided but v2/manga/ and v2/manga/bad-id are semantically two different api endpoints. Another example can be observed when opening /api/, which returns no code field at all (see point 1) and /api/manga, which redirects to the sites normal HTML 404 page, whereas there should be an api response.

7. Some documentation is missing. E.g. there's no clue where and why the bayesan rating differs from the mean rating, which makes the Bayesan rating far less useful, when in fact it's especially great for low viewed mangas.


That's what I observed when writing a small parser library in Rust. It worked well for the most part, but some parts were wonky and gave me some headaches. Python was slightly better, but only because Python is Python...

For the proper v5 site's API, yes, we do have one. More importantly, we'll be issuing API keys and all that cool stuff. Not going to bother writing a proper token system for this one, though.

First: An API-Key is a token, but that's nitpicking on my side.
More importantly: I would strongly argue against issuing API keys, they're not secure and lack fine grained control. When using API-Keys you need a system to authenticate the user on top of the API key, because again: They are not secure. It's better to use a system like OAuth/OpenIdConnect (which has a multiple libraries and official docs), that handles access control and user identification in one go. The resources by Auth0 and the RFC6749 are good reads on that topic.

Please don't stop short on security just because implementing a token system (which you really don't need to) is too much work. The internet already is a tar-pit sapping away your private data.


But that's enough ranting. Have a nice day!

 

[Teasday]

@Ruhrpottpatriot
"Returns true for "1", "true", "on" and "yes". Returns false otherwise." is verbatim how PHP documents it. If you think it's a documentation issue, take it up with them lol.

1. Status code could be removed, sure. Otherwise, v5 API should follow the jsonapi spec.
2. "Any proper webserver can handle this" is the issue here.
3. I'm very aware, we'll use ISO639 (most likely -3) for v5. In the meantime, we're working with what Holo left us, which is codes that map to flags rather than languages.
6. All /api/ routes outside of /api/v2/* are part of the v1 API. Never mind them.

And yeah, OAuth for v5 is the plan. The whole site is going to be an API driven SPA anyway, so we're taking the auth memes seriously not only for security reasons but also to have better control over what's happening with the site. I should've bothered to be more clear what I was talking about yesterday.

 

[Ruhrpottpatriot]

@Teasday

Otherwise, v5 API should follow the jsonapi spec.

That'd be great. The end user really only needs either the data or an error message. Additional information (status-code, HTTP version, encode, etc.) can (and should be) sent via headers.

All /api/ routes outside of /api/v2/* are part of the v1 API. Never mind them.

That might be, they should still return the proper error codes. A new programmer wouldn't know that they belong to v1 (I didn't know before you told me) and as always you have to factor in human error and gracefully handle malformed URLs. Once I had the issue that I didn't terminate my base URL (mangadex.org/api/) with a trailing slash and the URL lib replaced api with v2, thus resulting in mangedex.org/v2/manga. Now that's an easy spot, but as we all know PICNIC is the word and users are dumb in what they enter.
I'd suggest you add a filter rule for everything except /api/v2/* which returns HTTP 404 with {"message": "Invalid API Endpoint" } as body.

3. I'm very aware, we'll use ISO639 (most likely -3) for v5.

Whether it's -1 or -3 is pretty doesn't really matter, I'm not aware of any lib that doesn't handle both. Nice to know it's on the agenda regardless.

"Any proper webserver can handle this" is the issue here.

That comment was more meant in the direction of: If you're using something like NGINX, Haproxy, etc. you're good to go. The problem is, you're stuck with PHP, which is only half the speed of drogon or actix.🙄 But yeah... PHP the only language where 0 == "true" does in fact equal to true and true == "foobar" == 0 != true 🤮

If you think it's a documentation issue, take it up with them lol.

Naww, it's a documentation issue on your end. I as a programmer don't know that you're using PHP, nor am I interested in it, that's the whole reason for using an API.

And yeah, OAuth for v5 is the plan.

Good, that will cut my parsing library in half. Not that it's long in the first place... Rust takes away all the heavy lifting with speeds comparable to C++. Nice.

The whole site is going to be an API driven SPA anyway

It's been a while since I did serious web development but that might actually the page slower. However, take that with a grain of salt, it's been half a decade.

 

[Isotab]

@Teasday While I was trying to track down another issue I am having I just realized that md@home servers are not being sent in v2. In the v1 api that is automatically done based on your settings, but in v2 I am getting normal s2.mangadex.org/data results no matter what my settings.

I know there is a data saver boolean param but how is this supposed to handle md@home? I might be missing it but I am not seeing anything in the doc and I reread this entire thread and did not see any mention of this.

 

[Teasday]

@Isotab That seems to have been an issue that happened when the "SSL only" option was added, because evidently the old API was updated to handle it but not the v2 one. I blame rdn lol. Will fix soon, thanks for reporting.

 

[TanmayMudgal619]

@Teasday
How can i search manga by name

 

[Isotab]

@TanmayMudgal619 The api does not currently support searching. Seems like that is going to be something that is saved for v5 of the site because it is going to be a huge ordeal to get current search going. So you need to perform a request to the page and parse the html to get the results

 

[aes110]

Thank you for the API
Is there any kind of rate limit in place? (not that i plan on DDOSing the API 👾)

 

[raf555]

@Teasday you said the API is not finished yet, right? I assume you guys will add more Endpoints in future.
I have one request but i don't know whether this request has been requested or not, but i will explain in simple way.

Add one more POST endpoint to follow a manga by id, similar to "mark read" but this is for follow/unfollow a manga.

Thank you!