HD For Indies

Some notes on H.264 compression with Tiger:

So now that I've got Tiger and QuickTime 7 installed, but I don't have Final Cut Studio (including next gen versions of Final Cut Pro, Compressor, and DVD Studio Pro yet) I decided to do some preliminary testing with the H.264 codec.

For now, all I have to work with is QuickTime Player Pro 7.0 and Compressor 1.2.1.

H.264 in Tiger without Final Cut Studio: Good News/Bad News

Good News: You CAN specify the H.264 codec in Compressor 1.2.1...

Bad News: ...BUT you do not get as many options as QT Player Pro. This makes sense, since QT Player Pro v7 has been written to be H.264 aware, and an H.264 aware version of Compressor has yet to ship.

For the moment, in Compressor, all you can do is specify a frame rate, key frame rate, and data rate limitation. Which is not bad, but QT Pro gives you deeper/better options. You can:

-generate keyframes on an automatic, user defined, or every frame basis
-set data rate to automatic or numerical value limitation
-optimize for streaming, progressive, or CD/DVD (I'm sure that's hard drive optimized as well) basis.
-and most importantly, you can specify multi-pass encoding.

I started my testing by setting up a Compressor batch, but since I have no way of knowing whether it is doing single or dual pass encoding (that's another conversation as to the difference, suffice it so say dual pass is mo bettah but takes longer). I'm going to stop the Compressor batch shortly and just export files one at a time using QT Player Pro.

I'm impressed with the H.264 output options with QuickTime 7 - some people are bitching about having to pay $30 for a new key after buying a key for QT 6 not so long ago. But there are new technologies involved, and the depth of features in the H.264 export module makes it worth $30....to me, at least.

One good thing to note is that processor utilization is very high - CPU utilization is about 90-95% on average while my dual 2.5 GHz G5 crunches away (prior to noting mdimport's load - more on this below).

It's also clear this process is pretty slow - it's 10% done with an hour and a half estimated remaining.

Oh, and what am I compressing? It's full frame 1920x1080 uncompressed HD source - this is from the DCI StEM (Standard Evaluation Material) footage that came from scanned film saved out as 16 bit TIFF sequences. I converted it to an uncompressed 8 bit HD file using After Effects. This footage was designed be be as clean as possible, with every difficult to compress scenario they could think of thrown in there. Confetti thrown in front of a clear blue sky, shooting through wrought iron fence bars during a panning/tracking shot, vastly different skin color people in frame at the same time, fine details in highlight and shadow in frame simultaneously, etc. So it is, by definition and intent, the ideal footage to test for compression quality.

Interesting note - quitting both Compressor and Batch Monitor does NOT stop the compression process - it's still rolling as a background process, with no foreground applications indicated. This'll bite somebody I'm sure when they think it's not processing anymore...it's necessary to Delete the batch in the Batch Monitor to stop the process. (backtracked note - is this true, or was I just seeing mdimport doing it's thing?)

So I go back to QT Player Pro and select Export as an option from the File Menu. Interestingly enough, the default export is H.264. Nice! Clearly Apple wants to hype this to the moon.

I use the following settings under each bold choice:

Motion

Frame Rate: Custom: 23.976 (I could have used Current, but I want to rule out any funny business)

Key Frames: Automatic - I want to see what it does for this - I'll probably get very few keyframes

Compressor
Quality: is set to High but is greyed out - see Data Rate below for further explanation

Encoding: This is a biggie, I can select Best quality (Multi-pass). Multipass encoding looks at the file twice - once to make a "map" of the data rate needs of the file, and another to compress efficiently according to that map. See below for more on multi-pass encoding.

Data Rate

Data Rate: two choices: Automatic, which enables the Compressor Quality slider, or Restrict To:, which disables the Quality slider but lets you type in a number of your choosing to limit to so many kbits/sec.

Optimized For: allows for optimizing for streaming (more consistent data rate desirable, progressive (spikes not as important) or CD/DVD-ROM (data rate spikes even less important here)

Audio: Also worth noting are changes to the Sound Settings - they've renamed/rephrased some of the settings to be less consumer oriented, more professionally oriented. Good for power users, confusing for noobs. For instance, instead of "Uncompressed" as an audio option, they are calling it "Linear PCM" - technically accurate, but confusing if you don't know what that means. And even get stuck - there's a checkbox for "Little Endian." I kinda know what Big vs. Little is about, but which do I need here? I'll go with the default.

Based on these name changes, I'm wondering if any Compressor 1.2.1 Presets are going to have trouble working under Tiger, since some attribute names have changed? Or if people who upgraded to QT 7 but are still running 10.3.x will have the same problems?


an aside on multi-pass encoding: What's this do for you? Imagine the typical movie trailer - starts with the green "This has been approved for all audiences blah blah blah" screen. Then it cuts to the Paramount animated logo with the stars flying out of the clouds to line up around the mountain. The first (green cue card) scene requires extremely little bandwidth to represent - after you've "done" the type on green, frame by frame there are (in theory) zero changes - it needs very little bandwidth. The next scene, with the clouds, the star shapes, the mountain, etc. - has a LOT of changes frame by frame and needs more data rate. In single pass encoding, the same data rate is used for both scenes. Very inefficient - you didn't need nearly that much for the first scene, and you could've used a lot more for the second scene. This is what multi-pass encoding allows you to do - you can set an OVERALL desired average data rate, and the encoder figures out where you need more bandwidth (busy scenes, lots of motion/action/panning) and where you need less bandwidth (the green cue card is the minimal example, but locked off shots with people standing still are a more realistic movie low data rate example). It does this by "stealing" bandwidth it would've used for the "easy" scenes and giving it to the scenes where there's a lot of motion that needs more bandwidth to represent it well.


Back to my encoding efforts - even after a reboot, there is still a process called "mdimport" using 90% of one CPU running - what is it? Why is it running? I'm tempted to Force Quit it and see what happens. But not yet - a little Googling reveals that this is Spotlight doing it's thing. Hmm...but do I want itto be doing it's thing right now? Especially after copying over a terabyte of files onto a non-boot volume?

started encoding 4:11:35pm on 1920x1080 24p file, 2 min 9 sec long (but Spotlight is still crunching hard in background, stealing cycles from this effort!)

2 min is far longer than necessary to get a compression results test - I just wanted a good long first stab at it to show up. Encoding at 8000 kbits/sec (1MB/sec roughly, same data rate as used on Batman footage, but more pixels in frame, since Batman is wider aspect than this one, fewer pixels tall)

I was thinking that Spotlights mdimport process would give itself a lower priority once it saw that there was a foreground encoding process going on....not true. It maintained 90+% (almost one CPU's worth, full load is 200%) for awhile, then I checked to see QTPlayerHelper is using 113% of CPU, while mdimport is still using 50-60% (about half of one CPU on this dual CPU system).

This will take a while to chug, so I'll update this later when this one gets done. The encoding time will be somewhat meaningless, since Spotlight is stealing a lot of cycles to do it's thing, even though I wish it weren't. Can I turn it off? I can turn off "Movies" as a category in which search results will appear, but does that keep movies from being searched? Dunno.

I'll update this when it gets done compressing the movie...at 5:45pm, after about an hour and a half, it's not quite halfway done...

-mike