I recommend reading Winter. It's a good comic. While it doesn't seem to be updated regularly, there's quite a bit of if to be read.

So... Greetings, I thought I'd give you a bit of my mind today. I'm looking at compression. Compression has been a medium size topic in my projects. From the basics of the simplest codecs to the information theory behind it, I've spend quite a few hours on the topic. Here is a bit of data that I've gathered:

---

Name	Type	Method	Lossy?	Min*	Nominal*	Max*	Notes
PNG	Image	LZ77 + Huffman	No	1:1	2:5	10:1	Free**, best on line art
Jpeg	Image	DCT	Yes	2:3	2:1	10:1	Maybe Free**, Best on Photos
Gzip	Data	LZ77 + Huffman	No	1:1	2:1	10:1	Free**
Ogg Vorbis	Music	MDCT + Huffman	Yes	10:1	25:1	40:1	Free**
Mp3	Music	DCT	Yes	10:1	10:1	50:1	Not Free**
Bz2	Data	Arithmetic	No	4:5	5:1	10:1	Maybe Free**
GIF	Image	LZW	Maybe***	1:1	2:5	10:1	Not Free**, muddled with transparency and animation.
AAC	Audio	MDCT + Huffman	Yes	10:1	15:1	20:1	Not Free**, iTunes
MPEG-4	Video	DCT + delta frames	Yes	10:1	100:1	200:1	Free**, very good
Ogg Theora	Video	MDCT + motion	Yes	10:1	100:1	200:1	Free**, alpha

* Min refers to the worst case scenario. Nominal refers to the good case. Best refers to data optimized for that compression.
** Free and Not Free refers to the patents and change depending on who you listen to.
*** Gif only allows 8 bits. Since most images are 32 bits, Gif is lossless on some images and lossy on others.


What interesting things do you see in the above table? Ogg Vorbis and Gzip both use Huffman compression. Audio, Image, and Video codecs use Discrete Cosine Transform (DCT) to compress data, but all are lossy. PNG and Gzip use deflate compression which is a combination of LZ77 and Huffman.

The original intent of this table was to explain the methods currently used for compression are limited by theory and software, rarely by hardware. But compression is overwhelmingly good. Audio compression allows people to keep thousands of high quality music on their computer and on CD. PNG allows me to post huge amounts of vector art on this very site. Jpeg allows me to post low quality (and sometimes high quality photo) versions of pictures

Some things that are missing from this list are the more interesting aspects of the codecs. Gzip can compress text and programs very well. PNG works well on line art, but not photos. Jpeg works well on photos, but not line art. Jpeg has a large range of quality, while PNG only allows lossless. XviD is absolutely amazing video compression because video is very redundant over the time domain. I often wish for a little more compression, but more often I thank math and computer science for the compression we already have. It makes sense that current Audio, Video, and Image compression fits

Compression is an enabling tool. Tools earn their value through use.

I wrote a program that decodes a file to the command line with high verbosity. The output of a 3 MB file is approximately 30 MB. I have to run this program many times and keep the result each time. If I kept many of these large files, my hard drive would be quickly overrun with data. Gzip allows me to compress this data to 300 kB. The command I run is:

oggvorbis5 | gzip > oggvorbis53j.txt.gz
zcat oggvorbis53j.txt.gz | less

This is pretty efficient. I can read the file without taking much hard drive space. I also don't use much memory with this method. Some people don't like the command line, but they also complain about how slow their Visual Studio.Net is on their 3 GHz laptop with 1 GB of RAM. Kate, KDevelop, Linux, and the command line are fast, powerful, and easy.

I hear cries tonight
I hear, I hear

Like all of the best, We've been taken
Like all of the lost, we've been had
Pray God, kingdom come, deliver us,
Amen
All life, is it lost, have they won

Ain't no love in a concrete kingdom,
Ain't much life
Ain't no life in a concrete kingdom,
I hear cries tonight
Ain't no love
Ain't no life
Ain't no right,
What's for my son

When all of the good,
Have been taken
Where all of the lost,
Have gone
Ravaged, then raped, annihilation-amen
All life, is it lost, have they won

Ain't no love in a concrete kingdom
Ain't much light
Ain't no life in a concrete kingdom
I hear the cries tonight

-- Billy Idol, Concrete Kingdom

Permalink