HD For Indies

So I'm sitting here on hold with the New Mexico Department of Transportation (curses on them for their just-over-the-border Fast Tax!!!!), and was thinking about Digital Intermediate work, and what it would take to do it.

Traditional 2K film scans are 2048x1536 10 bit log 4:4:4 RGB. Yes, this is 4:3 aspect. Anamorphic squish to get 16:9 output. This actually makes sense beyond the aspect ratio of shooting film, because the human eye is more sensitive to vertical resolution than it is to horizontal resolution. "Um, whatever Mike, get to the hard numbers." Ok, uh, yeah, right....

SO:

2048x1536 pixels in an 8 bit Photoshop document is 9 MB. This is 4:4:4 RGB colorspace. (8 bit film-speak, not web-speak, meaning 24 bit color RGB)

Multiply by 1.25 to take 8 bits per channel up to 10 bits makes it 11.25 MB/frame. Those 10 bits are probably logarhythmic, not linear, so a DPX or Cineon file. Log files store the brighter values more like film does.

Multiply by 24 frames per second = 270 MB/sec

Ouch!

But that's just the raw data rate - multiply by 20% to factor in "safety overhead" in your disk system: 270 x 1.2 = 324 MB/sec (let's round it to 325 to keep life easy)

So 325 MB/sec. Damn. No wonder it's so expensive to do DI work!

So what can do that?

Obviously we're talking about a disk array here, and a lot of disks. So let's talk interfaces first:

FireWire: Nope, it's out - even FireWire 800 taps out at 100 MB/sec in theory, and in reality about 85 MB/sec is the best any Mac is doing...and that Mac isn't a G5. So until/unless newer G5s fix this (the dual 2.5s might, maybe....), FireWire, even with multi-port PCI cards, are out.

ATA-nope, never mind. Thank you for playing.

SCSI - Even Ultra 320 would be tapped at or beyond it's theoretical limit of 320 MB/sec. So you'd need TWO Ultra 320 channels. Two Ultra 160 channels wouldn't suffice. The smart way to work with SCSI is to use SCSI to ATA bridging systems, such as are employed in arrays by Huge & Medea. But they are still pretty costly per gigabyte of storage. But SCSI scales up great, so it's a viable possibility.

SATA - my personal favorite these days, at least at the scale of stuff I'm working on...let's see: at present a 6 drive array is the best that's possible. Even with the fastest big drives out there, about 60 MB/sec is as much transfer speed as you'll get per drive, and that's at the fastest (beginning) part of the disk. About 35 MB/sec at the end of the drive. This would imply you'd need 10 drives to always get the throughput, and that's assuming no aggregate overhead slowdown from hooking that many drives together, and assuming that they can all work full speed the way they get hooked up. My sources tell me that there are some PCI-X 4 port SATA cards coming to Mac. If they work perfectly, and you don't saturate the bus, this solution might maybe work. For a partial solution (not fast enough across entire span of array) you'd need 6 drives. 6 drive arrays are possible now (using 2 internal SATA drives and 2 Seritek cards for 4 more drives), but not compatible with another PCI-X card installed. You're going to need some kind of adaptor to get the images in, so that doesn't work right now. But SATA is a dark horse contender. Although you can only get one drive per SATA connector, I've personally assembled a 6 drive (Raptor 10K 73GB) SATA array that delivered 390 MB/sec using the BlackMagic disk testing utility. But that would only offer 410 GB of space - only enough for 26 minutes of storage. No good. SATA doesn't scale well - the best stuff I see coming down the line would only allow for a maximum of 10 drives. Even with RAID zero (no backup/security if a drive fails), using the biggest drives on the market, that's only 4 TB of storage. But then again, that would (in theory, a big fat caveat here) let you do about 4 1/3 hours of 10 bit 2K files. That's a lot. Not nearly enough for all the files in a feature to be online at all times, but still, that's a lot. For starving indies, this MIGHT MAYBE work. More testing to see if feasible. So how do you view these monstrous files? You need a HIGH resolution monitor, and it needs to be calibrated to your target destination. The new Apple 30" LCD might be an interesting choice for this. Lower res proxies have proven to be viable for color correction, so an FCP workflow with a final render out might be viable.

Fibre channel - Fiber channel allows for a LOT of scalability, is wicked fast, and can have incredibly long cable runs. Especially with the Ultra ATA bridging option, it gives a LOT of performance for the buck. a 3.5 TB setup goes for around $12000 with the goodies you'd want. The Apple card is a dual channel 2 GBit card, which has a maximum theoretical speed of 250 MB/sec per channel, so a maximum theoretical throughput of 500 MB/sec. I don't know how much overhead Fibre Channel has, but it doesn't sound unreasonable to get 325 MB/sec in the real world out of a 500 MB/sec theoretically capable bus. So this sound like a good option.

So let's dance through some storage throughput requirements, just for fun, to see how quickly things add up (some of these figures do NOT include audio):

HDV - (less than DV, which is 3.5 MB/sec) - 1280x720p30 - I think something like 2.5 MB/sec

DVCPRO HD for 720p24 8 bit 4:2:2- 1280x720 (really 960x720 stretched) - 5.4 MB/sec

DVCPRO HD for 720p30 8 bit 4:2:2- 1280x720 (really 960x720 stretched) - 6.75 MB/sec

DVCPRO HD for 1080i60 8 bit 4:2:2- 1920x1080 (really 1280x1080 stretched) - about 14 MB/sec

Uncompressed 720p24 8 bit 4:2:2- 1280x720 - 42.3 MB/sec

Uncompressed 720p30 8 bit 4:2:2- 1280x720 - 52.8 MB/sec

Uncompressed 720p60 8 bit 4:2:2- 1280x720 - 105.5 MB/sec (this one I confirmed from FCP captured data)

Uncompressed 720p24 10 bit 4:2:2- 1280x720 - 52.8 MB/sec

Uncompressed 720p30 10 bit 4:2:2- 1280x720 - 66 MB/sec

Uncompressed 720p60 10 bit 4:2:2- 1280x720 - 132 MB/sec

Uncompressed 720p24 8 bit 4:4:4- 1280x720 -63.4 MB/sec

Uncompressed 720p30 8 bit 4:4:4- 1280x720 - 79.2 MB/sec

Uncompressed 720p60 8 bit 4:4:4- 1280x720 - 158.4 MB/sec

Uncompressed 720p24 10 bit 4:4:4- 1280x720 - 79.2 MB/sec

Uncompressed 720p30 10 bit 4:4:4- 1280x720 - 99 MB/sec

Uncompressed 720p60 10 bit 4:4:4- 1280x720 - 198 MB/sec

Uncompressed 1920x1080p24 8 bit 4:2:2- 95 MB/sec

Uncompressed 1920x1080p30/i60 8 bit 4:2:2- 118.6 MB/sec

Uncompressed 1920x1080p24 10 bit 4:2:2- 118.6 MB/sec

Uncompressed 1920x1080p30/i60 10 bit 4:2:2- 148.3 MB/sec

Uncompressed 1920x1080p24 8 bit 4:4:4- 142.3 MB/sec

Uncompressed 1920x1080p30/i60 8 bit 4:4:4- 177.9 MB/sec

Uncompressed 1920x1080p24 10 bit 4:4:4- 177.9 MB/sec

Uncompressed 1920x1080p30/i60 10 bit 4:4:4- 222.4 MB/sec

Uncompressed 2048x1080 12 bit 4:4:4 (the maximum possible dual link HD-SDI frame size and bit depth, SMPTE 372M) - 227.9 MB/sec

True 2K film scan - 2048x1536 10 bit log, 4:4:4: 270 MB/sec

True 4K film scan - 4096x3072 10 bit log 4:4:4: 1080 MB/sec, aka 1.055 GB/sec (!!!)

What the SMPTE DCI (Digital Cinema Initiative) is recommending for D-Cinema in the future: XYZ CIE color space 4:4:4, gamma 2.6, I _THINK_ they were saying it should be 4K:

So: 4096x3072 12 bit log 4:4:4 XYZ, 24fps - 1728 MB/sec, aka 1.6875 GB/sec

...so you can see how storage throughput gets kinda important. And that SATA or other desktop based storage solutions are clearly, massively, out of their league.

Right now, the fastest connection available that I've heard of (used by the Da Vinci color correction system, and them folks have BUDGET) is a quad link 2 Gigabit Fibre Channel link - theoretically capable of about 1 GB/sec if fully saturated with no overhead losses. So it makes sense that they can only get 10-15 frames per second, because the pipe isn't fat enough.

Even IF you could get two Apple Fibre Channel dual link cards, each of the 4 ports talking to it's own X-RAID bank (each X-RAID has 2 7 drive banks), you could get about 500 to 600 MB/sec out of it from what I heard out of the Apple guys. NOT ENOUGH. OK, what if you chain multiple ones together? With 4 X-RAIDs, if it scaled performance linearly, you could theoretically get there. Congrats, you've just spent about $50K...on STORAGE. But it would all be RAID 50, with a total formatted RAID 50 capacity of about 10 1/3 terabytes.

How much footage would that hold, if it were fast enough?

DCI suggested format D-Cinema footage: about an hour and forty-two minutes. Barely enough to hold the final copy of a feature. Yowza.

Traditional 2K 10 bit DI work: 10 hours, 40 minutes. Now we're talking - we're into the range of feasible storage for a feature.

SMPTE 372M - 2048x1080 12 bit 4:4:4 YUV: 12 hours, 38 minutes - again, suitable for a single feature.

1080p24 10 bit HD 4:4:4: 16 hours, 10 minutes - now we're getting into the realm of what's possible on the desktop, by the way. The new cards from AJA (Kona 2) and BlackMagic Design (DeckLink HD Pro) can do this resolution, bit depth, and color depth. Your storage needs are high - about 180 MB/sec, with safety overhead your array needs to deliver a minimum of about 215 MB/sec at all times to do this res.

Not only is this doable, it's doable with as little as a 4 drive SATA array (but not the full capacity of the array, so limited # of minutes of storage) for a movie short. A 6 drive array still wouldn't deliver the necessary data rate across the full span of the array, but using the 7K400 drives would allow for something like 2 hours and 45 minutes of storage that was fast enough. (6*370 GB formatted capacity = 2.22 TB of RAID 0 storage) Then you'd still have about 450 GB of offline storage - enough for more than 23 hours of 720p24 DVCPRO HD footage. That array would cost you something like $3200 when all the parts (the 4 port SATA card and those 7K400 Hitachi 400GB drives) are readily available - something like 30-90 days from now.

-mike

----------------------
HOW I GOT THESE NUMBERS: The vendor websites I don't trust - I got these numbers the brute, simple way - created an 8 bit Photoshop document the right pixel size. To account for 4:2:2 color space (my PShop doc is 4:4:4), I multiplied by 2/3. 4+2+2=8, 4+4+4=12. 8/12=2/3=0.6666.

To account for 10 bit color, I multiplied my 8 bit numbers by 1.25. 10/8=5/4=1.25.

To account for the difference from 24 to 30fps, I multiplied by 1.25. 30/24=5/4=1.25

To account for the difference between 30p and 60p, I multiplied by 2. Duh.

...and so forth.