The Problem

PhotoShop files are too big! And to add to it, you have to save a pre-3.0 compatible image as well if other programs are going to read and display the file.

I was surprised to learn, upon looking at the file format documentation, that PhotoShop files have compression on their pixel data. It just doesn't work. It uses a simple RLE scheme which is utterly useless on continuous-tone images such as photographs. On a couple test images, I found on one that the "compressed" image was 98.8% of the size of raw bytes; on another the composite image was 96.1% the size of the raw data, but furthermore the file was twice that size because the real image data was stored in addition to this pre-3.0 compatible image. The file in question contained an adjustment layer, so the file size more than doubles <sigh>.

The Solution — in abstract

If Adobe would add another compression type to the file format, all would be well. Right now, the various places that store pixel data are flagged with a compression type byte, 0 for raw and 1 for RLE. Why not add a 2, and use a compression scheme that actually works? I nominate the preprocessed-deflated system used in PNG files, because it's the best freely-available lossless image compression around, and already an Internet standard and soon to be an ISO standard.

A sanity check indicates that this is a good idea: on the same file, a PNG file has an image data chunk that's 53.9% the size of raw data, compared to PhotoShop's 96.1%.

It would be nice if the compatibility image were not needed. If other programs could easily read the file without it, especially if the file is just a single data layer plus an adjustment layer and therefore pretty fast to figure out, then we would no longer need this. If Adobe produced a system component to let programs read or import the data without having to write original code to do so, then everyone could use that.

Barring that, we can reduce the cost by reducing redundancy anyway. A simple idea is to store the composite image normally, but all other layers are stored as the difference from the composite. I don't know if something like this would be fruitful, and it depends on the file, too.

The Solution — what we can do ourselves

Even without Adobe improving their file format, we can make a special-purpose compressor that does a far better job on PSD files than the general-purpose tools like Zip.

A refinement of this idea is to make that file a legal image file on its own, so tools to view it will work without updates. Specifically, store the composite image as the main data of a PNG file, and store the rest of the PSD file, compressed using our new scheme, as an extension record in that file.

So far, so good.

Right now I'm doing proof-of-concept development in Perl and using external tools. First job is to be able to read a PSD file, to understand it enough to know where all the pixel data is located. I'm working on a Perl script that reads a PSD file and displays details about what it found.

Next challange will be to decode this RLE data, and re-encode it as a PNG file (or the IDAT guts of a PNG file). There isn't a Perl module for this on CPAN, so I'm exploring easy alternatives.

Please email me with any thoughts or comments on this.


Page content copyright 2000 by John M. Dlugosz. Home:http://www.dlugosz.com, email:mailto:john@dlugosz.com