How to make every byte count in your PNG. TOPIC compression and squeezing

[doppler]

By nature PNG media files are squeezed out and compressed. For low color count aka: Mangadex media
files are sometimes needlessly large. I will provide some links to Windows based tools. Mac tools exist too.
If you have vivid color PNG's. Unless you need the portability of PNG stick to JPG format and crank up the
level to 9 (lossless). PNG become huge with lot's of color.

A vast majority of Mangadex PNG formatted files are already at the best possible size. Surprisingly there
are many that are just too damn big. Most likely from a poorly written media editor. Because of the way
color tables are written in PNG, you can have lots and lots of colors. But what if all the colors could be
written as grey scale and 256 or less ? Why have 32768 or more colors defined and never use more that
necessary ? Why have color table entries of colors used rarely and only off by 1 bit of a primary color from
another.

If you know about audio and MP3 files. The same was done for Picture media (JPEG Standard). PNG was
an attempt at lossless portable encoding, much like AAC is to MP3. Why burden a PNG with too much
information. If B/w or gray scale is the thing like (Mangadex) let's reduce the burden.

Nobody wants a problematic interface. So here is a easy GUI. https://github.com/tp7/WOptiPNG/releases/tag/0.3
The interface is old so an important updated file is needed. http://optipng.sourceforge.net/

Tweaks: Under setting.

Optimization Level: Default is 2.
You can go to 7 (max). Set it to 5. Slower at 7, but testing progress showed nothing needed going past 5.
If you have the time, or a really fast system use 5. Slow machine stick with default. (not best results)

Threads: Defaults to how many cores you got. Read the next setting to judge this one.
Using max cores and setting "high" priority can destabilize your system. Or make it very unresponsive.
My rule of thumb. Only use 75% of the cores. Unless you have a 2 core system (default to 1).
By leaving some cores uncommitted you have some control to stop or do other things.

Process Priority: Default normal. If you don't know, leave this one where it is.
Even with maxed threads "Normal" will still allow you to get in and do something.
Google search this one before making any change.

Word of warning:
Running max threads and high priority can REALLY heat up things inside the computer.
If your CPU fan is slow, defective, clogged. YOU WILL FRY YOUR CPU. No question about it.
Even check the power supply fan too.

General rule: A dusty PC box is not good. Dust is an insulator. Clean and blow out is good.
BEFORE YOU BLOW OUT THE BOX. PREVENT FANS FROM FREE SPINNING WHILE BLOWING!!!!
The fan may make a cool noise with all the air blowing. But you are damaging the fan and or bearings.
A non working fan is worst than a clogged one.

Have fun.
PS. Don't be surprise that you get a lot of low or 0% non-reductions. Like I said: Mangadex media
is generally reduced well. But there are some JEWELS in there with excess.

Please comment, not sure if the General forum is really read. So below the fold you know.

 

[shouldsleep]

Is this a PSA on how to convert files? Cause I'm pretty sure doing it in PS (commonly used editing software) is less tedious. There are also ways to do it in MS Powerpoint (for the madmen who edit there) but these are done prior to editing. The prior would have an issue of doing it simultaneously since file conversion isn't PS's main use, but if you are editing there then might as well do it if you want.

In a related manner, I don't really care if uploaders (pretty sure it's their decision to compress or not, as per the rules imply) reduce the file size. It's their time that's being used, not mine, no matter how quick it is.


PS. Don't add random tidbits to your post. It detracts the main point
Messy posts = lost readers.

 

[doppler]

Not a PSA. It's something every good webpage designer does before publishing the contents. Using the media program to do the conversion is slow and tedious. Even in batches you must decide on what loss level you want. Then everything get throw down that rabbit hole. Everything I described above will take out the extras not need to view. Every time a photo is taken with a phone lots of extras are applied. Long/Latitude time of day etc... That stuff is not needed and adds to the size of the picture.

In the case of PNG's and Magadex. I couldn't tell the difference of a 32k greyscale and 256 greyscale.

Not everybody in the world is on fast broadband. A large number of remote areas is only covered by 56k satellite.
So if a web page needs 5 seconds to load vs 1 second. Who do you think will get the traffic ? That's the competitive world today.

 

[shouldsleep]

If I am interpreting rule 3.1.1 correctly, it is the uploaders who determine file size so I can't tell what "before publishing content" entails. Don't really know much since I'm not a webpage designer or anything close. Maybe a mod could clarify.

Though looking at it logically I doubt heavily reducing the colour palette as a means for lossy compression could be used in an automated massive scale due to the amount of coloured titles (manhwa and manhua prevalently).

 

[Pika]

I use pnggauntlet to strip unnecessary data in pngs. It takes quite some cpu, but it can strip at least 1MB per chapter depending how big the chapter of course. (There are also times that I stripped 10MB with that thing). It's also lossess in quality too!

 

[doppler]

@pika
I tried pnggauntlet didn't do as well as optipng. Plus it took longer. Another benefit optipng does not change the timestamp of the files. Good for when I rsync elsewhere.

@shouldsleep
It's up to the poster/translators to weigh in the factors with type of container and sizing. Nobody wants to read thru tons of large files.

General:
I have reduced my Mangadex keeping library by a big factor. I don't need to size up my storage. I have also reduced my JPG content as well.
I did not include my results for JPG because it wasn't free. And the outfit selling the package includes Sourceforge content. Not the code, but
a whole program that is shell-called out. Not sure if any royalties are being payed to the authors (primary and derivative works).

Not sure if they wrote any original code at all, but may be a inclusion of multiple (other people) programs. Would love to know the original
JPG module source. I am sure it would be opensource and I would love to make use.

I will divulge to the SourceForge program. It's called PNGQUANT. Even on the website (SourceForge) the author admits OPTIPNG is better.
That's how I found it. With that little bit of info, you may find the pay package I was eluding too.

Edit: My grammar sucks

 

[Pika]

@doppler
From what I can read, optipng doesn't seem to be lossless? The quality is slightly lower.... (albeit minimum)
I'd rather have the png the same quality after throwing it into an optimizer

 

[doppler]

@pika
optipng goes from 2 to 7. 3 is prob where pnggauntlet ends at.
2 just get rid of extras and reduces the color tables. Just doing 2 would benefit.

I go to 5, to rid of the slightly off colors. Easily said by a color 255-255-255 all white
and 255-255-254 slightly off white. Merging 255-255-254 with 255-255-255 is called
loss. So be it. I couldn't see it, the monitor couldn't reproduce it either.

Thats the loss system used by all the encoders. Even with digital video it's the same
but to a much higher degree. There is not enough bandwidth in the world to support
4K HD video. That's why MP4 exists. MPG codec V6 is about to be released. In time
to support 8K res.
edit: grammer tense

 

[BzzBzz]

> From what I can read, optipng doesn't seem to be lossless? The quality is slightly lower.... (albeit minimum)

It's not. Optipng does not reduces quality. Never. With any settings. But pngquant is lossy recompressor for better results.
That's main reaason for optipng reducing size by 0-20% but pngquant reducing size for 50++%.

That author is just moron that compares apples to oranges. It's like comparing png to jpg and say that jpg is better because it's smaller /facepalm.

 

[doppler]

I came across a video that does very good to explain Jpeg compression.
Stick to the end. Very important. Because @ 5:40 why PNG is best for
Mangadex manga and all the rest explains Jpeg is best for high color.

https://youtu.be/Ba89cI9eIg8

 

[DANDAN_THE_DANDAN]

That paragraph per line format is annoying to look at on mobile.

 

[ZipFile]

PNG is kinda dumb, it's optimize-ability highly depends on the source image. PNG is essentially a few heuristics (e.g. N same pixels in a row) + the same algorithm used in zip files. Color pages, pages with tons of shading, noisy/dirty scans, or redrawer/typesetter clumsiness most likely make it impossible to apply lossless optimization at all.

Doing lossy optimizations is a no go, automating lossless one won't give much profit. If you're talking about improving user experience there is two possible options:
[ul]
[*] Recode PNGs to use interlacing. Faster previews, but will hurt file size a bit.
[*] Add JPEG/WebP previews + download button.
[*] Hire optimization nazis to supervise your uploads.
[/ul]
Not much to select from. Making previews considered good enough solution so far.