Discussion thread title info cards weights too much and wastes bandwidth. [Dev response]

[mangadexamc]

The current chapter info card for the chapter discussion threads in the forum is pregenerated and weights from 500 to 1000 KBytes, eg:
500 https://og.mangadex.org/og-image/chapter/818f6a39-9463-417d-9f75-6410f758990d
850 https://og.mangadex.org/og-image/chapter/c7d183dd-e39d-4231-a4e4-54754326315a
1050 https://og.mangadex.org/og-image/chapter/083db30b-e5c3-4a0d-b525-48a19b0b42ce

Napkin math for the used disk space:

Title with 10 chapters: from 5MB to 10MB for the title cards
Title with 50 chapters: from 25MB to 50MB
Title with 100 chapters: from 50MB to 100MB

Not a lot per title, but it's for almost all titles with a discussion thread and they would be there forever ever.
So the 100 titles with 100 chapters would eat from 5GB to 10GB of the disk space; 1000 titles = 50GB to 100GB.

Napkin math for the bandwidth:

1 thread with 1MB info card with 1000 views generates up to 1GB of bandwith.
Popular chapters can easily get 2K+ views of the comment threads or more, there are separate threads for each TL/group.

There are 9377 pages in the discussion forum, with 50 threads each, giving 468850 threads total.

With a conservative average estimate that gives:

468850 threads x 100 views x 1MB = up to 49TB of BW spent on these cards.

Options to consider:

a) do nothing
b) remove these info cards: saves both the disk space and BW
c) replace them with a cover (saves the disk space), maybe even with x.512.jpg version and overlay the info with the boring HTML based text: saves the disk space and lowers the BW usage, most users would have the cover pick already in the cache from the Updates page.

The last option demands knowledge of coding and algorithms, though. Or you can just slap <table> there.

 

[tristan9]

The current chapter info card for the chapter discussion threads in the forum is pregenerated

They are lazily generated, actually, and cached for a little bit (I forget the exact amount, but 24h or so), so disk space is not that high.

Napkin math for the bandwidth

All threads don't get regular views, so it's not nearly that high. Also these cards are used for preview images on Discord, Twitter, Reddit, ... And these all cache our response for a little while too.

But you made me curious so over the past 30 days:

• we served about 9.6 million requests for these images
• and it totals about 8.61TB of traffic

These are both really insignificant numbers at our scale, so if you were mainly concerned for us, we will just carry on as-is 😄

 

[mangadexamc]

They are lazily generated

I would be quite surprised if they weren't [but yet you served the full size images instead of previews]

All threads don't get regular views

Of course, that's why I used 100 views/total/avg as a high mark. Some threads have 11k as I saw.

insignificant numbers at our scale

Well, that's ~100TB/year of traffic what could be cut. But yeah, if the main biz is serving high resolution images, loads of hires images...

concerned for us

A bit of that, a bit of watching them slowly loading each time I check the comments...

There are also some quirks with avatars caching, I see my own load again (and slowly) almost every time but I didn't yet found the time to properly diag what is going on. Maybe 7 days is not enough..