HD For Indies

Nope, not a typo - that's five hundred and seventy-five Honest-To-God MegaBYTES per second, running on a G5, for less than $3000 (on top of the G5, of course).

How'd I do it? Read on for details...

I've been researching high speed, large capacity, low cost storage options for over a year in my quest to come up with low cost high quality solutions for independent filmmakers using high definition video. I wanted to find solutions that would allow for high quality, uncompressed HD video, which can require as much as 270 MB/sec (with safety overhead).

(For more on how to do this, read this, this, this, this, and this. Or just read the archives that are organized by month at the top right of this page, there are over 300 articles from this year alone.

I've been looking into SATA (Serial ATA, what the Mac G5's ship with) based solutions for some time now. The stock G5 can handle two drives - not nearly enough. Then along came the Seritek 1S2 card, and it allowed for 2 more with some ungainly cable arrangements. Better, but not great.

(Since then an external port version, the Seritek 1SE2 card has been announced.

Then suddenly out of the blue the other week, one of the better low cost cards on the Windows side suddenly got Mac drivers, and I was all over it. The Highpoint RocketRAID 1820A card comes with 8 SATA connectors, although they are all internal ports. To work around that hindrance, I'm running all the cables out an empty PCI slot cover.

I'm using eight Maxtor Maxline III 300 GB SATA drives, which are the fastest large SATA drives I've found on the market. The Western Digital Raptor drives are faster, but they only hold 74GB of data.

So where are all these drives? I've put them in a couple of MacGuru's Burly Boxes that I've built up. (And I do mean built, it takes an hour and a half to assemble all the parts and load all the drives, and I've done it several times now.)

I also put two more drives (these are Maxtor DiamondMax 10 models, mechanically identical to the Maxline III's) in the two internal bays in the G5 to bring the total of drives up to 10 300 GB drives.

In order to start up the computer, I'm booting from an external FireWire 800 drive (a La Cie model), since none of my RAID formatting software allows the computer to boot from a RAID.

For HD video capture, I'm using a BlackMagic DeckLink HD Pro 4:4:4 card, which comes with a convenient testing utility that measures actual throughput of a given drive or drive system.

(Just to be thorough, the rest of my system that can edit uncompressed 1920x1080 30 bit color 4:4:4 full RGB uses an HDLink with an Apple Cinema Display 23" LCD for HD signal monitoring.)

Using that as a benchmarking utility, and averaging 7 test runs together, I get an average disk read data rate of about 575 MB/sec, and an average Disk write data rate of about 416 MB/sec.

The 1820A card seems to max out at about 345 MB/sec on write speed - even when more drives are added to 1820A, that's as good as it gets. But it can achieve that speed with as few as 6 drives. Adding more drives continues to boost the read speed, but not the write speed.

By adding two more drives to different SATA buses (the built-in ones on the motherboard) allows me to boost the write speed past the 345 MB/sec limit. This is kind of cheating, since I'm using some internal and some external drives for this array, an ungainly setup. But it would also be possible with 3rd party cables to connect external drives to the SATA ports on the motherboard to store the drives outside the G5 case.

OK, so 575 MB/sec is the headline attention grabber...but you do NOT get that performance with all of the disk. As soon as you start filling up the disk, it gets slower. Huh?

In brief: a hard drive is like a record player...not that I've owned one since I was 12. The platters are like the record, the read/write heads are like the needle. And like a record (you've seen them on late night TV, right?) the record starts at the edge when the needle is placed way out at the outermost part. This part is moving very fast under the needle. The maximum amount of hard drive platter is moving under the read write heads every second at the edge. As data is written from the outermost tracks towards the middle, the more full the drive gets, the more it has to work with the inner tracks of the platters, or record in our analogy. And that is the part of the record/platters that moves the slowest under the read/write heads.

If that doesn't make sense, think how fast it was to hang on to the outer edge of the merry go round instead of the middle. Faster=more data throughput.

If that didn't make sense, just trust me on this: hard drives can't read and write data as fast as they get full as they could when they were empty. The transfer rates fall off as you have to work with the innermost tracks.

I tested my 8 Maxline III 300GB drive array by breaking it into 5 partitions of 435ish GB, and a final partition of 50 GB. The reason for that last smaller one was to test performance at the very end of the array where it is the slowest.

Here are the test results from the BlackMagic Disk Speed Test utility:

As one big partition (the whole 2.2TB)- 471/345 read/write

Slice1 (the first 435GB)- 467/345 read/write
Slice2 (the second 435GB)- 451/342 read/write
Slice3 (the third 435GB)- 418/342 read/write
Slice4 (the fourth 435GB)- 293/344 read/write
Slice5 (the fifth 435GB)- 345/354 read/write
Slice-last50GB (the last 50 GB)- 275/290 read/write

So, how's this done for so little?

1 Highpoint RocketRAID 1820A PCI-X card with 8 internal SATA ports: $204 from zipzoomfly.com
10 Maxtor Maxline III 300 GB SATA hard drives - from zipzoomfly.com, $220 each, $2200 from zipzoomfly.com
2 MacGuru's Burly Boxes - $215 each, $430 from macgurus.com
4 Molex to SATA power adaptors (NOT included w/Burly Box) - $6ea, $25 MacGurus.com
buncha zipties: $1 - Fry's or wherever

Total Cost: $2860

If I were really doing this for a production setup, I'd spend a bit more money.

SoftRAID for RAID setup and partitioning - $100

I'd buy drive coolers for all Burly Box drives. Heat is the leading cause of early hard drive death in my experience. 8x$30ea = $240

Since there is really no call for the extreme throughput, unless I really needed the additional capacity, I'd stick with an 8 drive array. It would still give about 2200 usable GB of space, and have the convenience of an all external solution. That would drop the price by 2 drives, so $440 less.

And the biggie for last - all of this has been RAID 0, which means if ANY one drive in the array fails, ALL data is lost short of heroic (and EXPENSIVE) efforts. The only reason why I suggest a RAID 0 is that in the context of a video production job, the most crucial data is the file you create with your NLE (usually Final Cut Pro HD these days) and the audio and video data you capture, plus any graphics you create along the way. As long as you have those assets backed up, you're OK. READ HERE FOR BACKUP STRATEGY DETAILS.

When capturing footage, that's usually done in a large chunk early in the project. After that, it doesn't change much. If you're willing to copy that capture data to backup hard drives (such as FireWire drives like the La Cie d2 or Bigger Disk or Bigger Disk Extreme), you're covered in case of drive failure. You'd lose significant time getting back up (a day, maybe two), but life would go on.

So you'd be looking at $2000 to $2500 worth of backup space if you're at all serious about this.

"Real" setup cost for 8 drive array capable of as much as 470 MB/sec reads and 345 MB/sec writes:

1 Highpoint RocketRAID 1820A: $204
8xMaxtor Maxline III 300GB SATA Drives: 8x$220=$1760
MacGurus Burly Box with coolers and power adaptors: 2x$347=$694
SoftRAID: $99 from softraid.com

Total Cost: $2757

Backup FireWire 800 storage

2 La Cie Bigger Disks Triple Interface (2x$1000) and one 250 GB d2 Triple Interface ($290) =$2290


Worst case scenario, this setup delivers at LEAST 275 MB/sec reads, 290 MB/sec writes all the way to the end of the disk. That's enough for ANY flavor of high defintion video, even 10 bit per channel 1920x1080 60 field per second 4:4:4 RGB, which is only possible on $100,000+ cameras and decks. Even to the 80% mark of the disk, you'd still get at least 345 MB/sec reads and writes. Or, it would make a helluva Photoshop scratch disk. : )

What about X-RAID, or othe RAID 3 or 5 or 30 or 50 solutions, such as those from Huge Systems or Medea? They work, they work well, they are well proven...and they are STILL more expensive than this SATA solution I'm suggesting by a substantial margin, even when you factor in the FireWire backup. Huge is over $4.50/GB, Medea is about $5/GB for systems with similar throughputs by MUCH better features - expandability, scalability, reliability, etc. When it comes to Apple. Huge, or Medea, if you can afford'em, buy'em. If you can't, look into this stuff.

A properly cooled and set up 8x300 disk array formats to about 2220 GB of usable space for about $2750. That's about $1.13/GB. Buy two Bigger Disks and another 250 GB FireWire drive all from La Cie, you've added $2290 to your costs, but are still looking at only $2.27/GB for Mike's Recommended Solution For Indies.(That's me.)

Apple's 1.75TB X-RAID, which has LOTS more flexibility, scalability, reliability, stability, and general goodness, costs $7500. That's over $4.50/GB. Lots of good arguments can be made about how it's worth the money, and I'd agreee with them...for those who can afford it.

For the indie filmmaker, that can be out of reach. Indies typically have more time and greater risk tolerance of down time than a well funded production facility, which is why I'd suggest this SATA with FireWire backup route.

For more info on how to put together a great low cost HD workstation, see here, here, here, here and here. Or keep checking back with this site, HD For Indies as I continue to write about high definition video, digital cinema, and how to get it all done at minimal cost.

If you'd like to keep regular tabs on these issues, check out my Atom and RSS feed links at the top right corner of this page, and use an RSS aggregator such as NetNewsWire or somesuch.

Also of interest: perfect pixel for pixel monitoring of your HD signal for as little as $1400 (for HD 720p work) or $2700 (for HD 1080p or 1080i resolution work) - read up on HDLink

DISCLAIMER: This is my 3rd day or working with the RocketRAID card, I don't really advocate it YET for production usage. I think it shows a lot of promise, but has yet to prove itself in a production environment. For instance, in the middle of duplicating a 5 GB file, I had a kernel panic - the bad kind where a grey scrim lowers, looking like a curtain coming down, and it states in several languages onscreen that it's time to shut down. Eeeeeeeeeeeew. I can't remember the last time I got one of those, so take all this drive stuff with a grain of salt until the drivers and card prove stable and reliable. This is, after all, a 1.0 driver we're talking about...

-mike

Mike Curtis is a digital post production supervisor in Austin, Texas available for consulting on your HD projects. If you have an HD project you are contemplating shooting, talk to him about being a test site for some new equipment under development. Email mike at hdforindies.com