Bulk Data Project

[curiousCheck]

Collecting a list of known requests that require some initial search processing which could be automated

aka the Things We Want Xnot To Do thread

Spoiler

 

[curiousCheck]

First off

Description: A link to all manga which use icons like below in the descriptions

Preferred Solution: Move the author's links to the author page. The author's page has support for multiple popular links already.
Also fuels more interest for "making the author related links appear more evidently" (suggestions invited, but those would go to the Frontend/UI Suggestions thread

Spoiler: Kafka's Links

<a href=""><img src="

https://imgur.com/dQCXZkU

" alt="twitter"/>Artist's Twitter</a>
<a href=""><img src="

https://imgur.com/oiVINmy

" alt="pixiv"/>Artist's Pixiv</a>
<a href=""><img src="

https://imgur.com/NVVf9Jl

" alt="MelonBook"/>Artist's MelonBook</a>
<a href=""><img src="

https://imgur.com/DByqIm6

" alt="FanBox"/>Artist's FANBOX</a>
<a href=""><img src="

https://imgur.com/5Wohzas

" alt="BOOTH"/>Artist's BOOTH</a>
<a href=""><img src="https://i.imgur.com/H1Q0eHg.png" alt="NicoVideo"/>Artist's Nico</a>
<a href=""><img src="https://i.imgur.com/mLCeebg.png" alt="Skeb"/>Artist's Skeb</a>
<a href=""><img src="https://i.imgur.com/qr4qCHu.png" alt="Fantia"/>Artist's Fantia</a>
<a href=""><img src="https://i.imgur.com/gpw5ezu.png" alt="Tumblr"/>Artist's Tumblr</a>
<a href=""><img src="https://i.imgur.com/TysezOa.png" alt="Youtube"/>Artist's Youtube</a>
<a href=""><img src="https://i.imgur.com/dzEFKZB.png" alt="Personal Website"/>Artist's Website</a>

Spoiler: Other Known Links

https://i.ibb.co/2NM00XW/fanbox-logo.png

If you know of more links that I forgot about, let me know in mangadex discord
Alternatively, Xnot can just check for description that contain Links

 

[curiousCheck]

Copin Comics is no more

https://twitter.com/i/web/status/1580709976426876929

Just checked their website and they do appear to be very dead
Could a dev nuke all the "official" links that linked to them all at once maybe?

 

[Xnot]

migrate tappytoon and mangaupdates links to new schema

add [Redacted] to the end of any [Redacted]

remove (Doujinshi) from all titles maybe?

update alt title thread

 

[curiousCheck]

Official English license links that are from Hachette Book Group
Eg:
http://www.b2c.hachettebookgroup.com/series/Dimension-W/?yen

Hachette is now Yen Press
and most, if not all, of their titles should be available on new yen press website

(Honestly been a while since I saw any of these but I'm sure more still exist)

 

[Xnot]

list of titles without start of publication date

 

[Xnot]

remove all "other" language chapters from existence

 

[BraveDude8]

List of all duplicate links on different entries (i.e. two manga have the same MAL link)

 

[Xnot]

re-verify all inputs against the new frontend regex

 

[curiousCheck]

Clean Amazon/cdjapan/ebookjapan retail links to remove the utm parameters that are used to trace link origins

 

[BraveDude8]

Search for fucked chapter metadata (just fixed a chapter 5,5 from 2018, comma intended)

 

[curiousCheck]

It looks like Official English Links to Toptoon Plus are now broken and replaced with Toptoon Global
Eg: https://toptoonplus.com/comic/100040/ -> https://global.toptoon.com/content/Brave-New-World/102333
Warning: NSFW Links

Would be nice to check that for all toptoon plus links

EDIT (2023/03/08):
Toptoon is now merged into DAYcomics
as per reports, everything shud be available in new platform (including coins) and all links will redirect correctly...

https://www.reddit.com/r/Toptoonplus/comments/10e7qsy

 

[Atifus]

It looks like Official English Links to Toptoon Plus are now broken and replaced with Toptoon Global
Eg: https://toptoonplus.com/comic/100040/ -> https://global.toptoon.com/content/Brave-New-World/102333
Warning: NSFW Links

Would be nice to check that for all toptoon plus links

EDIT (2023/03/08):
Toptoon is now merged into DAYcomics
as per reports, everything shud be available in new platform (including coins) and all links will redirect correctly...

https://www.reddit.com/r/Toptoonplus/comments/10e7qsy

The DAYcomics redirect leads to the main site if you are logged in or the merger announcement if you aren't instead of specific series links on DAYcomics so that would need to be fixed as well.

Eg:
https://www.toptoonplus.com/comic/100306/

or

https://global.toptoon.com/content/Drawing-Romance/100306

to

https://daycomics.com/content/100306

 

[curiousCheck]

After Yen Press launched a revamped website, they have proper series pages now
In addition however, the previous method to show all series using a direct search query link, are all broken

For Example:
https://yenpress.com/series-search/?series-yen539-Adachi-and-Shimamura-Manga
Should be changed to these
https://yenpress.com/series/adachi-and-shimamura-manga

 

[erova]

Niconico sites share the same structure, so manga id in a site is the same on both.

- https://seiga.nicovideo.jp/comic/
to
- https://manga.nicovideo.jp/comic/

Seiga is accessible only with japan ip, meanwhile manga is accessible without vpn, but if you access with japan ip to seiga, it redirects you to manga, as long as you are in /comic/.

you can read chapters whenever you are logged in.

e.g

• https://seiga.nicovideo.jp/comic/35259
• https://manga.nicovideo.jp/comic/35259
• https://seiga.nicovideo.jp/comic/68080
• https://manga.nicovideo.jp/comic/68080