HD For Indies

UPDATED AGAIN 11/7/04 -- I'm getting ready to (finally) post the Part II, so I updated this and reposted as prep for Part II

UPDATED Tuesday 3:45pm - a few changes made - all HDV cameras record to 8 bit at this time, and Sony's HDV format cannot be dubbed to D-VHS decks without re-encoding the data. Nor can Sony HDV data be recorded onto Blu-Ray discs.

This article discusses the difference in quality between what various video cameras and camcorders actually shoot versus the quality of what gets recorded to tape. A detailed explanation is given of what happens to the video image after it leaves the camera sensor and before it gets written to tape...and how it gets substantially changed. Concepts of RGB vs YUV, 4:4:4 vs 4:2:2, and interframe vs intraframe compression are explained and discussed with lots of explanatory links.

When I first started getting into digital video back around 1994 or so, before the DV format had been invented and popularized, it was very confusing to talk to vendors and try to get an idea of what kind of quality their video capture cards was capable of. I was just learing the difference between composite, s-video and component interfaces, and hearing vendors say

"Well yes, our video capture card captures at 640x480, and so does our competitor's, but ours has more resolution."

Or

"Well yes, our product and our competitors both capture the same pixel resolution, and even though theirs has a higher data rate, ours is better."

It was very confusing. And, as with all vendor claims, sometimes they were right, sometimes they were wrong, and sometimes, as with many things,

"it just depends."

I see the same thing happening now with the various HD formats.

In a perfect world, shooting digital video would be akin to acquiring successive Photoshop documents many times a second. Poof! Perfect image quality at the size you want. This is not the case.

(For the picky ones of you out there, in a truly perfect world, you'd be shooting successive RAW camera format images to tape, or better yet in the new Adobe Digital Negative (DNG) format. But I obsessively digress.)

In the real world, especially when recording to videotape, this is far from what actually happens.

Most HD video cameras acquire the source image in 4:4:4 RGB, then reduce it to a 4:2:2 YUV image before it is fed to the videotape portion of the camcorder for compression to tape.

If you don't understand 4:4:4 RGB vs 4:2:2 YUV, read this short article to get the gist of it, or this longer one describing 4:2:2, 4:1:1, and 4:2:0 and this longer one talking about uncompressed and lossless video that isn't uncompressed or lossless to really understand.

If you plug a monitor directly into the HD-SDI connector on an HD video camera, it is usually a 10 bit, 4:2:2, full resolution image (1920x1080 or 1280x720) and it looks great - no compression or image quality reduction (in order to reduce datarate/bandwidth) has been applied at this point. There are no blocky chunky compression blocks, the image still has a potential range of 1024 possible levels of subtlety of brightness and color.

However...in order to get the potentially very large amount of video and audio data onto the tape, it has to be compressed. Just the video portion of 10 bit, 1920x1080 full 4:4:4 RGB at 29.97 interlaced fields per second takes up something around 220 MB/sec. Lots of data.

In order to reduce this down to something that can be recorded onto videotape, the following steps are taken in the most common HD tape formats:

1.) The image is broken down from RGB red, green and blue into YUV - luma (roughly speaking, the brightness) and two channels for the color information. Doing this allows the color information to be recorded at a reduced rate as compared to the luma information. Early TV tests determined that using 1/2 has much color data as brightness (black and white) was a reasonable compromise between image quality, human perception thresholds and bandwidth (aka filesize in the digital era).

2.) The image is downsampled (shrunk) in size horizontally, sometimes by as much as half

3.) The bit depth is reduced from 10 bit (1024 gradations of brightness/color) to 8 bit (256 gradations)

4.) A DCT based compression scheme is applied (DCT is short for Discrete Cosine Transformation, the same kind of compression done to JPEG images used on the web)

5.) Additional compression is applied which tries to minimize the datarate/bandwidth by looking for similarities from one frame to the next and essentially saying "That part? Yeah. Looks the same. Do that again. Maybe scootch it over a bit."

This brings the datarate down to something recordable onto videotape, somewhere in the 14 to 23 MB/sec range for the various vendor's solutions.

Let's look at the details of what we get in various video formats. I'll start with some familiar SD (standard defintion) video formats as a place of reference.

DV

DV is a standard definition video format. It records 720x480 pixels in 8 bit, 4:1:1, then applies 5:1 compression on the results for a total of about 3 1/2 MB/sec of data (including audio). This means that DV throws away 3/4 of the color information that it might have kept. This is OK, since the human eye is much more sensitive to black and white detail than it is to color detail. So:
Input Resolution (camera acquires at): 720x480
Output Resolution (to/from tape): 720x480
Frame Rate/s:29.97 interlaced fps for NTSC, 25 interlaced fps for PAL (some cameras can do 24 progressive fps, sorta)
Bit Depth: 8 bit (256 unique levels or gradations of color and brightness)
Color Sampling: 4:1:1 (1/4 of color information retained)
Compression Ratio: 5:1

Camera Costs: roughly $500 to $20,000, sweet spot - $3000-$5000 for indies
Deck Costs: roughly $2000 to $12,000 - sweet spot - $3000-$8000 for indies

As a quick aside, there is also the DVCPRO50 standard from Panasonic, which uses 4:2:2 color sampling but twice the filesize, so a bit over 7 MB/sec. It's better than DV, but cameras/camcorders/decks cost much more than consumer DV gear. Many post production professionals consider it the best bang for the buck for SD (standard definition) video production.
Camera Costs: roughly $15000 to $50,000, sweet spot - I don't know
Deck Costs: $5,000 to $30,000 - sweet spot - I don't know

Digital Betacam, aka Digibeta

Digibeta is better than DV in a few ways. For starters, it's a 10 bit format, meaning it has a possible range of 1024 discrete gradations of color or brightness instead of only 256. Much more subtle detail can be maintained, especially in scenes with subtle color or brightness falloffs, like a sunset, or a sconce light tapering off against a wall. It uses 4:2:2 color sampling instead of 4:1:1, so it's capturing twice as much color data. For those trying to extract a color key from a bluescreen or greenscreen effects shot, the difference is HUGE. It also does a much milder compression - on 2:1. So:

Input Resolution (camera acquires at): 720x486
Output Resolution (to/from tape): 720x486
Frame Rate/s: 29.97 interlaced fps for NTSC, 25 interlaced fps for PAL
Bit Depth: 10 bit (1024 possible gradations of color or brightness)
Color Sampling: 4:2:2 (1/2 of color information retained)
Compression Ratio: 2:1

Camera Costs: $15,000 to $50,000 (cursory look)
Deck Costs: $35,000 to $50,000 (cursory look)

....but Digibeta decks and camcorders are MUCH more expensive than DV, and still quite a bit more than DVCPRO50 gear.

Digibeta: (quote swiped from this page on Ingenious TV

For the technically minded, Digibeta is a Digital format that uses mild 2:1 compression to record the full 4:2:2 component video signal on half inch tape. It has 4 digital audio channels (2 only on most cameras) as well as time code and cue tracks. Camera tapes run for up to 42 minutes and studio tapes run for up to 124 minutes.

The Digital Betacam format encompasses a DCT-compressed component video signal at YUV 4:2:2 sampling with 4 channels of uncompressed PCM-encoded audio at 48khz sampling. A 5th audio track is available for cueing, and a linear timecode track is also used on the tape.


OK, that should give you a decent grounding in how the color sampling and compression stuff works. Let's look at some HD formats now:

HDV

HDV is an interesting format. In theory it sounds great - either 720p (JVC) or 1080i (Sony) acquisition on sub-$4000 cameras, recorded onto standard miniDV tapes, copied over to the computer via FireWire in the native camera codec (a la Varicam footage with AJ-1200A Panasonic deck). But due to less than optimal imaging systems (current camera implementations, surmountable obstacle) and the extremely heavy compression used to record it onto miniDV tapes (inherent in the tape format, insurmountable obstacle), it doesn't work out quite as well as one might hope. But it's the best bang for the buck going in HD.

The creators wanted to have a low cost high definition format that high end consumers could work with. They decided to record in the same format that HD is currently broadcast in, namely an MPEG-2 transport stream. They also decided to record it onto standard miniDV tapes and load it onto computers via FireWire (also called iLink or IEEE 1394a), which is great for convenience and low cost. In theory the MPEG-2 interoperability sounds great, in practice it introduces a number of problems.

For starters, when the HDV format was introduced nobody's mainstream NLE system could edit this stuff. There's a PC solution called Aspect HD that converted it to a wavelet based codec, but then didn't have a (cost effectively) viable way to get it back out to tape. They may have fixed that, I just haven't paid attention since NAB. On the Mac side, there is a slick little application called Lumiere HD which handles the process a bit more elegantly but in a still non-traditional (for NLE operators) kind of way. But it works, and can maintain quality of the footage and get it back out to HDV or D-VHS deck. (D-VHS uses the exact same audio/video data structure as HDV, you just send the signal back over FireWire - it is possible to duplicate HDV tapes to D-VHS with a simple FireWire connection)UPDATE: The Sony won't dub to D-VHS - see other update below for details.).

There are two flavors of HDV now - JVC's in their 720p30 JY-HD10 (and lower end HD1) cameras, and Sony with their recently announced 1080i HDR-FX1.

This one's pretty easy. Everyone agrees the JVC, while first to market, kinda sucks rocks. Poor low light performance, poor color reproduction, lower resolution, poor exposure latitude, and one chip CMOS technology. Kinda icky. It also lists for about $4000 last I checked, which is MORE than the Sony. The camera shoots 1280x720 10 bit 4:2:0, but records it to the HDV format, which compresses it as an MPEG-2 transport stream, meaning it is (at present) awkward at the very least to edit with in most NLE (non-linear editing) systems. But Apple, Adobe, and almost all the other vendors have announced either future support or 3rd party shoehorn solutions to work with it now.

The $3500 Sony due in the US in November, has the same NLE interoperability issues but worse (since it hasn't shipped yet, and existing HDV solutions will have to be modified to work with the new size & framerate), but from what I can tell so far, it looks really nice. It shoots 1920x1080, but interlaced only. (But there will be a PAL version, and thus the possibility of 1080i50 to 1080p25 to 1080p24 exists). There is a pseudo 24p mode, but I bet it's doing some field dropping or interpolating to do it, which is less than optimal. But it is 3 CCDs, has better light sensitivity, tons of controls, and there will be a pro version with XLR inputs and other tweaks in Q1 2005 for about $7000. Some sample footage can be found via a FTP link described here, but because of the heavy compression, I'd be concerned about how scenes looked if there were a lot of fine detail, a lot of motion, a lot of color detail, or a moderate to heavy amount of color correction needed to be applied. This concern applies to the JVC, and basically ANY HDV camera ever made.

The HDV format, by acquiring (when you want maximum data) in a format usually used for distribution (when you want a minimum amount of data), I fear the footage might be "brittle" in that you wouldn't be able to do much to it in post (color correct, composite, etc.) without it "breaking" it and having it degrade heavily with a lot of visible ugly artifacts. Basically, the raw footage might look OK, but as soon as you try to color correct it, it might start looking "a little assy" as a friend of mine would say. Again, I haven't had a chance to play with any source 1080i footage from Sony, this is just a concern I'm airing with the format. The fact that the 720p30 format is using 19 megabits, and the 1080i60 format, with more than twice the amount of pixels, only uses 25 megabits, makes me worry about shots with a lot of motion - are they going to break up into a lot of little compression blocks? Try a fast pan with a DV camera and play it back, or shoot tall grass/waving trees moving in the wind and play that back and you'll know what I mean.

JVC's 720 progressive HDV format in their JY-HD10

Input Resolution (camera acquires at):1280x720 (I may be wrong on this if the CCD array is being upsampled/uprezzed)
Output Resolution (to/from tape):1280x720 (I may be wrong on this if it's recorded to tape differently)
Frame Rate/s: 30 progressive frames per second (no interlaced options, no 24p option)
Bit Depth: 8 bit (256 possible gradations of color or brightness)
Color Sampling: 4:2:0 (the two color channels take turns as to which gets recorded, then that data repeats for a field until it gets updated)
Compression Ratio: 19Mbits/sec = 2.375 MB/sec, that's about 27:1 compression starting from 10 bit 4:2:2, but it is MPEG-2, which is "smarter" and more efficient than the compression methods used in DV, HDCAM, and DVCPRO HD.

Camera Costs: $4000 list, probably closer to $3200 street
Deck Costs: I've yet to see/find prices, but I think I overhead $2000? $4000? See it here.

Sony's 1080 interlaced HDV format in their HDR-FX1

Input Resolution (camera acquires at):the CCD only captures 960x1080, this is upsampled to 1440x1080
Output Resolution (to/from tape):1440x1080 is recorded to tape, on playback it's upsampled to 1920x1080
Frame Rate/s: 29.97 interlaced frames per second, a pseudo 24p is offered (field doubling or interpolating to do so). Eventually a PAL version with 1080i50, which has interesting ramifications for independent filmmakers (better path to 1080p24 than does 1080i60 originated material)
Bit Depth: 8 bit (256 possible gradations of color or brightness)
Color Sampling: 4:2:0 (the two color channels take turns as to which gets recorded, then that data repeats for a field until it gets updated)
Compression Ratio: 25 Mbits/sec = 3.125 MB/sec, that's about 47:1 compression starting from 10 bit 4:2:2, but it is MPEG-2, which is "smarter" and more efficient than the compression methods used in DV, HDCAM, and DVCPRO HD.

Camera Costs: $3500 estimated for the HDR-FX1
Deck Costs: I understand that the JVC deck listed above should work with 1080i, but I can't prove it
Actually a reader informed me that the structure of the Sony HDV 1080i signal is sufficiently different that you cannot dub to a D-VHS deck directly from the Sony. Software fixes, such as the previously mentioned LumiereHD may, in the future, resolve that problem.

So you can see some things start changing with the way the HDV works as opposed to SD cameras. High definition video carries so much more data than SD that the manufacturer's are trying to find ways to shoehorn what could be up to 6 times more data onto the same tape formats. A lot of compression is required to achieve that. All but the highest end HD tape formats also downsample the source image horizontally, reducing the number of pixels it actually records to tape. This Sony camera, for example, also only has a CCD capable of detecting 960 pixels of detail per line, even though the final output needs to be 1920. The 960 pixels are stretched out to 1440, thus inventing some data by interpolating from what they already had. 1440 lines are written to tape in a heavily compressed format. On playback, they are stretched and interpolated even further out to the full 1920 pixels across. While the footage I've seen looks really good, it is not as good as it could be, since they are only picking up 1/2 the detail from the get-go. Sony could viably argue that the CCD is a good match for their lense they use. Let's face it, it's only a $3500 camera, and professional lenses for film can cost tens of thousands of dollars apiece in their effort to capture every little bit of detail. But for $3500, it's a phenomenal bang for the buck from what I can tell so far.

The Sony is the most interesting camera I foresee being readily available between now and NAB next year for independent filmmakers wanting to get into HD on a budget. Even though the CCD array only captures HALF of what the final display resolution is going to be, and then heavily compresses it, the end results are still impressive from the limited amount of footage I've seen so far. A $3500 camera, a $4000 computer setup (computer, NLE & HDV software, a computer monitor), and you can shoot and edit (but not monitor to color correct without spending another $1400 or so minimum). That's a sub-$10,000 setup to shoot and edit your HD movie. Damn. That's cool. If you want me to write a detailed list of the parts involved, email me at mike@hdforindies.com and I'll do a write-up if there's demand.

OK, it's 3:30 AM and I'm beat. So if I got something wrong, email me but be nice. Tomorrow I'll get back on it and get into the "real" HD tape formats. Here's what I'll be covering in ascending resolution and quality:

DVCPRO HD

HDCAM

D-5

HDCAM SR (4:2:2 and optionally 4:4:4)

SRW-1 double rate 4:4:4