HD For Indies

Graeme Nattress, creator of some plugins for Final Cut Pro, wrote me to tell me of a review he wrote for the Los Angeles Final Cut Pro User Group on the BlackMagic DeckLink HD Pro and HDLink as an HD editing/monitoring solution.

Some of the better stuff from his review:

-has some nice photos that clearly demonstrate the difficulties of installing the DeckLink HD card in slot 4 in a G5

-talks at length about the quality differences between using DeckLink/HDLink combo as opposed to using the Digital Cinema Desktop Preview

-mike

The good geeks over at Accelerate Your Mac have posted a detailed report with photos on the NVidia 6800 Ultra graphics card. Why care? This is the card that can drive the 30" Cinema Display. But in order to do so, it blocks the first PCI-X slot. This review (FINALLY!) has good pictures that shows how much the PCI slot gets blocked. This is important because if you want to edit HD with one of these cards, you're going to have to have at least two cards installed - an HD card and a drive interface card (SCSI, SATA, or fiber channel). If you're using SATA, very few cards presently on the market have external ports - they are usually internal ports and you have to run cables to the outside of the box usually via an empty PCI slot cover. It would be a very tight squeeze to route cables around this NVidia card, since it intrudes into the space that would be used by a PCI card in slot 2. So for the time being, I don't recommend the 6800 Ultra for Mac HD editors working with SATA cards. This may change in over the weeks and months ahead as new hardware alternatives become available.

-mike

UPDATE SATURDAY MORNING on StEM footage actual resolution

FedEx guy was busy this week - I got two key things:

1.) The DCI StEM footage - this is the Digital Cinema Initiative Standard Evaluation Material. It is a little mini-movie they shot on film and carefully scanned at 4K (4096x3072) and carefully color corrected and saved out as 16 bit RGB TIFF files. I bought the 2K (2048x1536) version since that's as much data as I can reasonably work with.

UPDATE SATURDAY MORNING: It's 2K wide (2048), but only 857 or so pixels tall. BUMMER! This is about half the vertical resolution I was expecting. It's 2.4:1 aspect ratio, widest cinema aspect used. So for HD, I've got to letterbox it. So all the anamorphic scaling stuff below is WRONG. I just dropped it into a 1920x1080 comp, let the sides crop off, and let there be black top & bottom. Sniff

My plan is to convert that to various HD formats: 720p, 1080p, and 1080i sizes at 4:2:2 YUV and 4:4:4 RGB color space, and at 8 and 10 bit color depth. Probably about a terabyte of test files by the time I get done. Then I plan to convert it to various codecs (uncompressed, DVCPRO HD, HDV, and maybe DNxHD if I can figure out how) to see how it all holds up. The footage was all shot to specifically test for the things that digital tends to screw up on, so should be very interesting to see the results. It just dawned on me - this stuff might be logarithmic instead of linear, I'll have to deal with that, too. My "punt" position that I'm pretty sure will work:

1.) Batch all 16bit RGB TIFF files to 16 bit PSD (Photoshop) files, because I don't think After Effects can handle 16 bit TIFFs (need to verify)

2.) Import the numbered RGB PSD files into After Effects as a clip

3.) Drop those into a 1920x1080 comp and scale them to fit (they are anamorphic 2048x1536). I'll also create a 1280x720 comp.

4.) If the footage is linear, I'll have to install and figure out eLin (from Red Giant Software) to "de-log" the footage into a linear colorspace. I think that's something eLin can do, I'm not absolutely positive though.

5.) From there, I can render out all my 1920x1080 stuff:
1920x1080 @ 24fps 8 bit 4:2:2 uncompressed
1920x1080 @ 24fps 10 bit 4:2:2 uncompressed
1920x1080 @ 24fps 10 bit 4:4:4 uncompressed
1920x1080 @ 60i 8 bit 4:2:2
1920x1080 @ 60i 10 bit 4:2:2

....and so forth and so on. Those will be my digital masters for codec testing, to see how much each codec shreds the footage from the original.

I'd like to try messing with Cleaner for it's superior scaling algorithm, but I fear it's YUV conversion - I've seen some wierd results when working with DVCPRO HD codec. Also, I don't think it can handle 16 bit anything.

I'd like to use Compression Master, but again I don't think it could use the 16 bit source to compress down to.

I might end up making some kind of interstitial at 10 bit from AE at full size and do the scaling and codec writing from Compression Master or Cleaner, again I'll have to make a short test and fool with it...lots.

Debabelizer has an excellent scaling algorithm as well, but I don't know if it handles 16 bit anything either.

If anyone has any clues on any of this, I'd LOVE to hear from you at mike AT hdforindies DOT com.



2.) I also have sitting patiently on the shelf a SyncRAID XL card that I want to doodle with and see what kind of realworld performance numbers I can obtain from it. I'm thinking that might end up being a card in my server as a secure RAID 5 type setup to backup all this test data I'm generating. I have 5 160 GB drives sitting around, I could get 600 GB of protected RAID out of that. If I don't make all the 1080i variants, I could fit it all on there. Hmm....

-mike

At IBC Baytech announced the CineRAM, a 5 pound device that records uncompressed HD to RAM (LOTS of RAM!). You then transfer the data via Gigabit Ethernet to a computer.

The above is a link to their product page, here is a link to the product PDF front page, here's the back page of the PDF. From their website main page:

CineRAM Features

- Uncompressed, Portable, Solid State, Rugged
- Dual/Single Link HD-SDI (4:4:4 and 4:2:2)
- Supports SMPTE digital  formats used in digital film production
- Network attached imagery file server
- HD-SDI to gigabit Ethernet network bridge
- HD-SDI to DVI monitor interface
- HD-SDI test signal generator


This thing looks very cool and promising. The idea is you download to/through a laptop to a FireWire drive.

-mike

Huge Systems sent out an email today anouncing they have upgraded their Huge Media Vault line of RAID systems to use the Hitachi 7K400 400GB hard drives. This boosts capacity substantially, from 2.5 to 4.0 TB in their 10 drive model. Pricing is now as follows according to their email:

HMV-320R-2000-M (this is the 5 drive, standalone model) $7859 ($3.93/GB)

HMV-320RX-4000-DM (this is the 10 drive, rackmount model) $14902 ($3.73/GB)

But, as with all RAID manufacturer's claims, these prices per gigabyte aren't quite correct. The 400 GB drives actually format to 370 GB. So in a RAID 0 configuration, your cost per GB is more like:

HMV-320R-2000-M - actual formatted RAID 0 capacity 1850 GB, $7859 ($4.25/GB)
HMV-320RX-4000-DM $14,902, actual formatted RAID 0 capacity 3700 GB ($4.03/GB)

But if you're using RAID 3, as you should be for critical data backup, you lose one or two drives' worth of space (depending on whether you're using the 5 or 10 drive model respectively).

So your RAID 3 actual, usable capacity is:

HMV-320R-2000-M - actual formatted RAID 3 capacity 1480 GB, $7859 ($5.31/GB)
HMV-320RX-4000-DM $14,902, actual formatted RAID 3 capacity 2960 GB ($5.03/GB)

I'm not trying to pick on Huge in particular, all RAID manufacturer's claim their capacities and calculate their cost/GB this way. Just trying to point out this is what you actually end up with when you take the wrapper off and get it all set up in your studio setup.

How does this compare? Apple's X-Serve RAID is availabe with 400 GB drives, too, but they use 7 or 14 drives. Apple uses a fiber channel instead of a Ultra SCSI 320 connection, but both interface cards cost about the same, around $500.

Apple X-Serve RAID 0 capacities and cost/GB for RAID 0:

Apple X-Serve RAID 2.8 TB, 7x400 GB drives, actual formatted RAID 0 capacity 2590GB, $8500, so $3.28/GB

Apple X-Serve RAID 5.6 TB, 14x400 GB drives, actual formatted RAID 0 capacity 5180 GB, $13,000, so $2.51/GB


Apple X-Serve RAID 3 capacities and cost/GB for RAID 0:

Apple X-Serve RAID 2.8 TB, 7x400 GB drives, actual formatted RAID 3 capacity 2220 GB, $8500, so $3.83/GB

Apple X-Serve RAID 5.6 TB, 14x400 GB drives, actual formatted RAID 3 capacity 4440 GB, $13,000, so $2.93/GB

Those capacities would be the same for RAID 5, RAID 30, and RAID 50 as well.

I'd recommend adding some other options to the X-Serve RAID, but to keep this as apples to Apples as possible I'm leaving those out for now.

In general, I'd be curious to see how Huge defends their pricing against Apple, especially since the Apple solution can be integrated into an X-SAN Storage Area Network environment for $1000 per node plus a metadata controller server.

With the caveat that I haven't worked hands on with either system, assuming that both systems worked equally well (and that's a large assumption), I'd be hard pressed to not go with the all Apple solution - if there's a problem, there's only one vendor to yell at.

-mike

There's Blu-Ray, there's HD-DVD. Looks like we're heading into another potential VHS vs. Betamax war. Read about it here.

Need to do a shot involving mounting the camera on a car? Either on the hood looking in, or looking out at the road? Here's a comparison of two products specifically made for getting those kinds of shots using smaller, lighter cameras like DV, HDV, and maybe something bigger. StickyPod vs. The Gripper. (Sounds like a wrestling match: "Sunday Sunday Sunday! Be there...")

UPDATE THURSDAY NOON CST: The DOCK has an s-video out port. Confirmed on specs page here. Also, I went by the local Apple Store yesterday to help a friend buy a laptop, and noticed that they had Photo iPods on display. Didn't ask if they were in stock. It does in fact show the artwork picture when playing back a song, but when showing song info the picture is dinky on the side. But it's there.

Been thinking about it some more since yesterday's post on the Photo iPod - Steve Jobs thinks photos are the "next big thing" because everyone has them, they are quick and easy with digital cameras, and there are no rights issues.

It also looks like there might be an s-video connection on the dock....or maybe that's just the multi-plug for audio and video. Dunno. Looked like an S-video plug but with too many holes.

This just means, from a business/manufacturing side, that it CAN work.

That does NOT however, in ANY way, create demand for the products!

As a color screen iPod with more space (on the 60GB model), I think that's great. Supposedly it'll put up the album art for each song it's playing if you have your iTunes set up with'em attached. There are programs to do that now for you, too - just point'em at your iTunes collection and it grabs images off of Amazon (sweet!). Search for them at versiontracker.com, I'm too lazy to do that right now.

Anyway, the $100 extra feature to display photos - personally I don't find it all that compelling. We'll have to wait and see.

-mike

This is completely off topic, but I like iPods, and Apple has some new ones. Plus, for the record, I think I'm the first to have analysis up rather than just the press release stuff. Cuz I'm like that. : ) Anyway, the goods:

iPod Mini: same as before, but now free personalization - get your name engraved on the back for free - looks like 2 lines of info allowed (name & #, name & email, etc.) Still $249, still 4GB, still same 5 colors.

Photo iPod : OK, this is new - 40GB is $499, 60GB is $599. It has a 220x176 pixel, 65,000 color (16 bit color) display. Menus are in color now. iTunes 4.7 (from Apple website): "...lets you import your images directly into iPod Photo from a folder on the hard drive of your Mac or PC." If using Adobe Elements 3.0 or Adobe Album 2.0, can organize your albums and load onto iPod Photo from a PC. Obviously, use iPhoto on Mac. iTunes 4.7 allows for syncing Photos to the unit. I'm guessing that downloading photos will work the same way with the Belkin card reader doohickey that already worked with prior generations of iPods. If not, they'll have a compatible unit soon. It has an A/V port so you can show the pictures on a TV set. Based on viewing the tiny photo on this page, the connections appear to be three RCA connectors - RCA stereo pair for sound and composite video. Eew. I was hoping for an S-video output option at least - composite video is soft and smeary. Maybe that will be an optional cable at some point, but I don't see any such thing on the accessories page today.

U2 iPod - it's 20GB, $350, black with red click wheel. Uh, yeah. Signatures of the band on the back. Comes with a U2 poster. Comes with $50 off coupon for the The Complete U2 - over 400 tracks, including 25 "rare and unreleased songs." I had heard rumor the new album was to be included on the unit, but I don't see mention of that on the page.

"Regular" iPod now in 20GB ($299) and 40GB ($399) sizes. 40GB includes Apple Dock, 20GB doesn't.

Mike's Comments:

iPod Mini a slightly better deal with the free engraving if you buy direct from Apple. Apple continues to try to get all sales to go through them instead of retailers. Good for consumers, bad for retail sellers.

Regular iPod: Is this any different than what came before? Maybe the inclusion of dock with 40GB?

iPod Photo: Interesting...could be a budget presentation machine, would be cool to carry all your presentations in your shirt pocket. Photo sharing? Now you can bore the whole office, not just your roommates with trip photos....my main interest (personally) in this unit is 60 GB capacity - enough to install and run the OS if you need to, and keep all your digital goodies in reach. Make presentations on the road from this tiny thing. I'm sure somebody's going to come out with some middleware to help that possibility. Plus have a zillion songs around, too, as well as all your contacts and your calendar. It's creeping into PDA territory here.

iPod U2 edition: I think Apple and U2 have Jumped The Shark on this one. Even seeing the ad on TV and online, I was kinda creeped out by it, felt like we crossed the commerce/art line too hard here. The iconoclastic, non-individual black outlines in the Apple ads was nice because they weren't anyone specific. Now by sneaking in a greyscale Bono face, it makes it SOMEBODY, and that changes the feel of the ads. Bono with the white iPod signature cord coming off the microphone? It just didn't feel right to me. Plus I think the red on black iPod itself is rather, well, hmm....wasn't black and red cool back around the time the Heavy Metal movie came out? I'm just not down with it. Expect these to be available cheep online or eBay in about 4-6 months.

Just my $0.02.

-mike

The Apple Store is down at the moment, pending an announcement. Probably the new U2 black iPod, perhaps the new 60 GB iPod with photo display, perhaps the new Flash based iPod.

We'll see soon, I'll keep all posted.

New iBooks were released recently, good enough for DV, HDV, and DVCPRO HD editing (but not pixel for pixel HD image display).

-mike

Saw the press release so I thought I'd pass this on - Gorilla is a piece of software to help you plan and produce your independent film. From their website:

With Gorilla, all of your information can be stored in one place: Budgeting, scheduling, crew, cast, profit-sharing, editing notes, and even film festival submissions.

When it's time to create reports, Gorilla can generate over 50 industry-standard reports, from stripboards to breakdown sheets, budget top sheets to variances, your entire production can be run from one program.

Don't know anything else about it, so caveat emptor. But might be useful. The logistics of moviemaking are fearsome, get some help somewhere.

-mike

ATTO has announced a new fiber channel card, the Celerity FC-42XS that has two separate 4 gigabit fiber channel connections for attaching to high speed fiber channel hosted storage, such as Apple's X-Serve RAID.

Each channel can run up to a maximum theoretical throughput of 800 MB/sec. In a Mac G5, this allows for a maximum throughput of 1 gigabyte per second (1 GB/sec), due to the bus limitations of the 133 MHz PCI-X slot.

Mike's Comments: Interesting. I guess you could hook up X-RAIDs to each port, maybe stripe them all together for collosal throughput? Enough throughput for capturing 2K files in realtime, and playing back perhaps 2 streams in realtime, if you had a system capable of displaying that...maybe?

This is complete conjecture, I don't know if it would work: Could you hang a couple of fully populated X-RAIDs off this card in a G5 and display all of a 2K (2048x1536) image on a 30" Cinema Display in realtime, perhaps via Final Cut Pro HD? Dunno, but it would certainly be interesting to try (if I had the $35K in equipment to do it).

-mike

The HDTV Expert page is a nice little page on HDTV issues (the TVs, not production). This article explains some of the hype around manufacturer's quoted claims for contrast numbers for their displays, and is helpful for anyone trying to get a good understanding of display technology, and what makes for a good image. Pay special attention to the part about black values and greyscale representations, especially in the darker range of grays. This matters when you're trying to use a display device to simulate how an image will look on film, or even on another kind of display device (LCD, CRT, projector, film, etc.)

-mike

No, I didn't update EVERY article today, just changed the way the feeds were handled to increase browser compatibility. Now each article linked from an RSS feed shows up on it's own page. This will work with more alternative browsers, as well as make browsing archived articles a little more logical.

I'd recommend setting your RSS and web browsers to open a new window for each link clicked on if you haven't already.

What is RSS? It's a way to quickly and easily see when a site has been updated if the site has an RSS feed.

My Mac RSS feed reader of choice is NetNewsWire.

I use it to quickly skim for new articles from about 40 different websites multiple times every day (how I bring you all these links).

-mike

Interested in optimal display of HD images, and/or the REAL resolution of all the newfangled expensive TVs? Read this.

headline says it all...read the article here.

I'd quibble with some of their details - they didn't catch all the price drops and new models BlackMagic came out with at IBC, and they don't mention the possibility (even the future possibility) of SATA arrays for uncompressed HD work. They also mention multiple striped SATA drives for compressed HD work - simply not true - single SATA is fine for compressed HD.

-mike

THIS is a great article that I read with no small amount of glee - Windows Media 9 is "hitting some snags" as Slashdot put it.

Some salient quotes from the EE Times article:

A number of technical and political issues surrounding VC-1 have reportedly caused growing frustration and constant bickering in the SMPTE engineering community.

The uncertainty has raised questions about the future of Microsoft's Windows Media Video codec. On the assumption that WMV9 was destined to become an industry standard, Microsoft convinced both the Blu-ray Disc Association and the DVD Forum to include it as a mandatory video compression format (along with MPEG-2 and H.264/MPEG-4 AVC) for next-generation high-definition DVD formats. Now, there is speculation that delays or licensing problems for VC-1 could prompt either — or both — of the DVD industry groups to simply delete the Microsoft technology from their specifications.

Microsoft created the impression in the industry that its WMV9 codec had a leg up on H.264/MPEG-4 AVC in quality and licensing terms. But now that the WMV9-based VC-1 has been put to the test in the arduous SMPTE standardization process, VC-1 is "perceived as behind in quality

At the heart of the issues that have slowed SMPTE's standardization process lies the industry's general suspicion of Microsoft. One industry analyst characterized Microsoft's donation of WMV9 to SMPTE as a "calculated maneuver to gain respect for the proprietary technology"

the SMPTE group still needs a reference software decoder and reference bit streams to ensure interoperability among different VC-1 implementations. This work is not yet completed. Nor are there any signs of a reference encoder for VC-1.


The article goes on to say that the VC-1 standard (which is based on the Windows Media 9 codec) might get dropped altogether from the spec for both Blu-Ray and HD-DVD.

As the Slashdot article put it, "Now it turns out that Microsoft cheated and lied: its code is not as good as MPEG 4, the WMV9 reference implementation is not available, and the WMV9 test suite does not exercise all the features. The SMPTE might drop WMV9 after all. Apparently, a highly technical standard body is harder to snowjob than the usual clueless consumers."

HEAVILY updated Saturday afternoon, 4pm CST

They were backordered forever, and now I know why. La Cie had run out of 1TB Bigger Disks with FireWire 400 and FireWire 800 (aka 1394a and 1394b IEEE standards), and said they'd be coming out in mid-September. Then maybe October. Then they quietly rolled out a triple interface version, the La Cie Bigger Disk Triple Interface. This new replacement model has the same FW400 and FW800 connections as before, but now it adds a USB 2.0 interface (and if you were suicidaly paitent, USB 2.0 is backwards compatible with USB 1.1, albeit much much slower).

First Impressions:

The unit harkens back to the days of the full size, full height 5 1/4" hard drive enclosures. By normal desktop drive standards, this sucker is HEAVY. That's because inside this aluminum enclosure there are FOUR hard drives on a commond bus with a hardware RAID 0 controller. I'd gotten some conflicting information at one point as to whether it was a true RAID 0 (data striped across 4 drives) or a JBOD (Just a Bunch of Disks, also known as spanning, where one drive is filled with data, then it rolls over to the next, etc., only giving one drive's worth of performance at all times). Based on the performance I'm seeing, I'm thinking it must be RAID 0 - 70 MB/sec read times are beyond what any single 250GB drive can do.

After 2 1/2 hours of hard formatting, it appears to be slightly under 20% done. So this will probably take about 13 hours (give or take) to fully format. So that takes quite a while. It also makes the fairly small fan run loud. Granted, I'm telling it to constantly write data non-stop for hours at a time, but it's little fan is going full-tilt, to the point that it is much louder than the dual processor G5 it's sitting next to. Louder than anything else in the room electronic (and there's a LOT). So I'd love for them to put a quieter fan on it, preferably larger diameter. The front has a nice long air intake space, but the back has a roughly 1 1/2" meshed square that it releases the heat exhaust from. Too small.

Note: what I'm calling a "hard format" is just using the Write Zeroes option in the Options tab on the Erase screen of Apple Disk Utility under OS X 10.3.x. The reason for doing this is to map out all the bad sectors on the disks that might exist from the factory. This is a good way to make sure the disk is as reliable as possible. I do this with ALL my disks before I use them, even though it takes geological timeframes on large, modern multi-hundred gigabyte disks. It also deletes all data on the disk, so it's best done right when you get it, before you put any data on the drive whatsoever. The La Cie does arrive preformatted for use on Mac, very conveniently. I just like to take this extra step, since I'm going to be using this RAID 0 device for backups. And as with all RAID 0 devices, if one drive in it dies, ALL the data on teh RAID dies.

But while trying to zero write the drive, which looks like it would take about 15 hours, it failed. Repeatedly. Four times on two machines. It looks like Disk Utility doesn't know how to handle the "jump" from one drive to the next. After 7-9 hours, it always fails. Sigh. No zero write for these units.

Next up: performance testing.

FireWire 800 Performance

Interestingly, performance was pretty consistent across the entire capacity of the drive, whether at the beginning or the last 28GB of the drive. Using just the FireWire 800 connection (the fastest) I partitioned it into 5 equal slices, except for the last, which I made a little smaller to allow for an 'end of disk" section of 28GB. All parts of the Bigger Disk had virtually identical throughput that I tested - about 71 MB/sec reads, 51 MB/sec writes. I used BlackMagic Design's Disk Speed Test utility to derive those numbers, running on dual 2.5 GHz G5 running OS X 10.3.4, version 4.6 of the testing utility.

This would imply that this drive (a 4 drive RAID 0, but for simplicity's sake let's just call it a drive) would be perfectly suitable for any standard definition video work, capable of capturing any standard definition video source, be it DV, DVCPRO50, uncompressed, whatever. It might also suffice for dual stream SD editing. Theoretically possible, but haven't tried it yet.

For HD uncompressed work, this drive would be unsuitable, with the possible exception of 8 bit 1280x720 4:2:2 24 frame per second capture. 10 bit is right at the capability limits of the drive with no safety overhead, so you'd be bound to drop some frames, an unacceptable option. But since the most common 720p24 source is the Panasonic Varicam camera, and its tape format (DVCRPRO HD) is only 8 bit, it should work. Plus, for DVCPRO HD, you should be working with a Panasonic AJ-1200A deck with FireWire, anyway.

For compressed HD work, however, you should be fine - any DVCPRO HD via FireWire should be OK (as long as the 1200A deck is on a separate FireWire bus), BlackMagic's PhotoJPEG would be fine, AJA's Q-Rez should be fine for any 4:2:2 work and 24p 4:4:4 work (if they offer 4:4:4 with Q-Rez, not sure).

FireWire 400 Performance

Just for fun I connected via FireWire 400 as well and tested again. This time, I used 5 185GB partitions, and a last one of 10GB.

Read Performance (avg of 3 runs): 35 MB/sec
Write Performance (avg of 3 runs): 28 MB/sec

And again, that performance was across 6 different partitions of the drive, so it's consistent (so really 18 test runs per read and write).

This is clear proof of how much better FireWire 800 is than FireWire 400. This kicks a bunch of stuff out. It should still work for any standard def footage including uncompressed (barely!), and no HD uncompressed is possible with this. HD compressed should still be about the same, but fewer streams possible with lower bandwidth available.

But if all you have is FireWire 400 ports on your G4 or PowerBook or iBook, this is an indication of what's possible (all these tests were done on G5, remember).

USB 2.0 Performance

And just to be thorough I thought I'd try USB 2.0 performance. In part to see what it could do, but also to see if this would make it possible to edit DVCPRO HD footage at 720p or 1080i on my laptop. Since my 12" PowerBook (and other PowerBooks, especially some older ones that still have the minimally required 1 GHz G4 processor) only has a single FireWire 400 port, this creates a problem. When dealing with the Panasonic AJ-1200A deck that handles DVCPRO HD over FireWire, the deck wants to be the only device on a given FireWire bus, otherwise it drops frames and pitches a fit. So how to get around this? Perhaps by using the USB 2 interface on some external drives if you want more capacity than what is possible just using the internal drive in your laptop (and capturing to boot drive is always an "only if desperate/have to" situation, increasing the chance of dropped frames on capture and playback).

Read Performance (avg of 3 runs): 15.5 MB/sec
Write Performance (avg of 3 runs): 15.5 MB/sec

Sad, isn't it? This should suffice for the following video formats: DV, HDV, DVCROPRO HD 720p resolution (but probably not 1080i resolution, which is 14.5 MB/sec, way too close for safety, dropped frames for sure).

If using this setup, be sure to use a High Speed USB rated cable (USB 2.0, not 1.1 or 1.0!) and double check in System Profiler that it says "Up to 480 MB/sec". Just because the bus is theoretically capable of something doesn't mean that you will get performance ANYWHERE near that.

Throughput testing conclusions:

This is a great example of how much difference the drive interface makes. Inside this device are four ATA 250GB hard drives. They can run up to about 50-60 MB/sec each I would imagine (not knowing the exact model used, but that is my experience with modern SATA drives).

On USB 2, only 15.5 MB/sec. On FW400, it increases to 35 MB/sec. On FW800, it increases to 70 MB/sec.

But if I hook up 4 native SATA drives to a good native SATA connection, it's easy to get well in excess of 200, sometimes 240 MB/sec of read performance.

So the interface counts - a lot. And as I mentioned above, just because the interface CAN run that fast doesn't mean it WILL run that fast. Bummer but true.

Noise

The unit is LOUD however, when it's cooling fan fires up - it is MUCH louder than the G5 is most of the time. Besides the fact that I primarily intended this for backups for my own use, this is another reason to use it for backups- I wouldn't want this on/under my desk while I was working, especially if working with audio. I hope La Cie fixes this - it's as loud or louder than the G5 when the G5 is REALLY working or getting hot.

So far, though, it's a fine backup solution, and with deals like the $705 Dell deal I mentioned a few days ago (scroll down), at about $0.75/GB for 935 formatted GB of capacity, it's a great solution for overnight backups of a LOT of data, such as one might use for uncompressed HD work on a RAID 0.

-mike

I'd heard about Mark Van Bavel and his DV Film company a while back, but I finally got a chance to meet him recently and sit down and talk with him, and then we were on a panel together at the Austin Film Festival. His company does transfers from digital source (DV, HD, digital files) to 16 or 35mm film. While not offering the higher end services of companies like eFilm, Technicolor or SwissFX, he also doesn't charge anything close to their rates. For smaller independent projects without extensive post budgets, it's a must-know-about facility.

Whereas I tend to get wired when I talk about all of this digital stuff and to do a lot of future projection of possibilities, Mark remains very calm and speaks from experience about cameras and software, and especially the results that can be obtained, since he's been outputting film from digital source for at least 4 years (that I know of).

He also has an excellent FAQ section on his website, and wrote a book on DV filmout, and offers DV Film and DV Atlantis, two software packages for use in DV=>film conversions. I hope to hit him up for review copies of those at some point and publish commentary on them (the book too).

-mike

Longtime contributor Christopher Barry sent in a bunch of links about HDV and the Sony HDR-FX1 1080i resolution HDV camera:

One of them explained some of the differences between the consumer (HDR-FX1) and pro model (HVR-Z1) models, outlining the features the HVR-Z1 will have that are better than the HDR-FX1:

Some differences of HVE-Z1 versus HDR-FX1:

1  Stereo AND mono microphone on top
2. 2 x XLR Input
3. Black body (instead of charcoal)
4. Separate recessed audio volume controls
5. Time Code preset
6. Camera switchable between 50 Hz /60 Hz
7. Simultaneous use of LCD display and EVF (Viewfinder)
8. EVF switchable between B&W and colour
9. Hyper Gain, zoom display w/numbers, hours meter, Audio Noise Reduction, audio limiter, action safe zone display.
10. DVCAM tapes


NO true 24 or 25 frames per second nor 720 mode.
Sony's point 720 is NO true HD.  Interlace is a must for smooth motion reproduction! And they are wright. Also all that fuzz about 25P. Its only a very few appreciating and using this mode proper!
For US (NTSC)  30P users there might be a difference, but why not buy the better picture in PAL version?

Sony also has an HDV info site with discussion forums.

Here's another review of the HDR-FX1, this time on Hardware Zone.

Just wrote the HD storage options article, and it's been pointed out to me that not everyone wants or needs to edit a feature film. The possibility of industrial films, commericals, or short films still might required uncompressed HD. So on that note, here's some new twists for those needs:

The good news is that if you want to do an uncompressed HDCAM short (or uncompressed Varicam short), it can be done for very little. Consider this:

DP 2.0 GHz G5: $2700 w/2.5 GB of third party RAM
FCP HD: $1000
DeckLink HD: $600
HDLink: $700
HP 23" LCD: $1600
19" monitor: $350
4xBarracuda 7200.7 160GB array: ($100 Seritek 1SE2 (lotsa online places, pray to Google), 4x$113 drives (zipzoomfly.com), 2x$50 PPA, Inc cases (Fry's Electronics), $190 external FW800 drive (La Cie)) - $842

total: $7792

Uncompressed 24p HDCAM time: about one hour 45 minutes
Uncompressed 29.97 interlaced HDCAM time: a bit over an hour, since I don't think the array would be fast enough to reliably capture 1080i60 (another way of saying 29.97 interlaced) all the way to the slowest (end) part of the drives in the array

Now you're all set to capture, edit, and accurately monitor 1080 res HDCAM footage. Spend $100 each on a jog/shuttle controller (Contour Designs Shuttle Pro v2) and an editing keyboard (lots, look around or search this site using bar at top of page) and you're still under $8000 for your uncompressed HD system. Want backup? Now you're under $9000.

Want data protection and only using 24p HDCAM? Use the 5 port SyncRAID XL card and a 5 bay Burly Box (no coolers, on budget!) and another drive, get same array capacity of 594 GB (magic of parity math), drop the FW drive (don't need it now) and the price is $8085. If 24p HDCAM is all you're doing, this is clearly the way to go if you're on a shoestring budget (but gotta render your transitions, no realtime - can you live with that?). Want more capacity? Just use bigger drives. See zipzoomfly.com for good SATA drive pricing. 250's go for about $160, 300 GB for about $210, 400GB for about $400. You'd need 5 of'em.

-mike

Normally I would consider this OT (off topic) for this blog, but this is so cool I decided I wanted to include it. Ampede, maker of that PDF rasterizer for Final Cut Pro, has introduced LayerLink, which lets you import an Illustrator file as a project, with all of its layers intact.

"WTF" is what any decent, respectable filmmaker is saying - "why care?" As a (former) motion graphics artist, this is one of those core functions that sounds like nothing but makes your life miles easier. If you're doing DVD graphics, or title sequences, or anything that would require flying some carefully made type or artwork around, this level of control is crucial. Since it is a native Motion plug-in, not a ported After Effects plug-in, it is full-on, instant, realtime performance.

If you're using Motion more than a day a week, this will probably be a must-buy for you.

It's normally $120 but $99 as an introductory price.

I've talked about Avid's product line in the past on HD For Indies, but for those that want a taste of what the Avid interface is like, download Avid Free DV, their "free as in beer" giveaway version. Think of it as their iMovie - free, consumer/low end, lacking the heavy features. I haven't even looked at it (yet), but I thought it worthy to make note of it here if you're new to this stuff or an FCP user that wants a clue as to what Avid's interface is like.

-mike

La Cie has launched a bunch of new products today, including a new high quality LCD monitor with 10 bit gamma correction (useful for calibrating the display), an Ethernet based NAS (Network Attached Storage) device with 160 to 800 GB of storage that is expandable via FireWire 400, and of interest to me, an external SATA drive that comes with a 2 port PCI card with external data and power connections. Very interesting.

The drives are available in 74GB 10krpm ($449), 160GB ($269), 250GB ($339), and 400GB ($549) sizes and come with one of the dual port cards, and are expected to start shipping in mid-November.

I want to find out more about this set up. According to information on the site, it's a 32 bit, 66MHz card. There is reference to "up to 85 MB/sec" for the drives, and I'm not sure if that is the maximum speed of the drives or the cards. And if you put two cards in slots 2 & 3 in a G5, will the performance capabilities double to 170 MB/sec or not? A very, very interesting question for the things I'm trying to do in studio right now. Might have to order me one or two of these just to play with. That opens the door to a convenient 6 drive SATA array (2 internal, 4 external). With 400GB drives, that would be 2.4TB of RAID 0.

In any case, for the past few months or so I've been seeing a lot of PC deep geek equipment websites advertising various external SATA solutions, so it makes sense for a more mainstream vendor like La Cie to get in on the act.

-mike

With both the SyncRAID XL (see other article, scroll down a bit) and revised X-Serve RAID hitting the market this week, I've decided to take another look at storage options for those wanting to do uncompressed HD editing.

For those wanting to work with compressed HD, rejoice - plain old ATA and SATA drives will suffice just fine (and even FireWire drives under certain circumstances - just don't put FireWire drives on same bus as Panasonic AJ-1200A deck!).

(I've stolen the next paragraph of two from the X-RAID article I just posted, if you've already read it skip on down, the rest is All New!)

Apple X-Serve RAID - now with 1.0, 2.8, 3.5, or 5.6TB of capacity

If you're serious about posting an uncompressed HD feature, this new system (with 3.5 or 5.6TB) is THE low cost, high throughput, data protected (RAID 1+0, 3, 5, etc.) solution. 380 MB/sec read speed is very respectable - supposed to do dual stream 10 bit HD - and that's at RAID 5 if I am reading the press material correctly.

Let's really break down the numbers though. At first blush, it seems like the 5.6TB unit it just over $2/GB. $13,000 divided by 5.6TB is $2.32/GB. Not quite the "just over $2/GB" figure Apple suggests. But how much usable space do you get out of it? All drives lose about 8% of their capacity when you format them, regardless of single drive vs. RAID. So your 14x400 is really 14x370, so usable space is really 5180 GB. And if you're using RAID 5 (because RAID 0 has no data protection), the X-RAID is really two 7 drive RAIDs paired together (each with their own controller), and since each RAID 3 (or 5) loses one disk worth of space for parity, instead of 2*7 drives worth of space, it's more like 2*6 drives worth of space - 4440 GB. So that pushes the price up to $2.92/GB. And if you add the recommended battery powered cache backup ($350) and the required host fiber channel host card ($500), your price is $13,850, or $3.11/GB as a practical, realistic, installed cost.

I'm not dissing the X-RAID here, I think it's a great product in concept (see MacInTouch.com threads for some issues folks have had) and I'd heartily recommend it for all who can afford it. Just pointing out that the end quantity of storage and end costs are not quite as rosy as at first seems...but this is the case with all storage vendors, I'm not meaning to pick out Apple in particular.

(OK, all this below is brand new)

The traditional answer for uncompressed HD used to be a pure SCSI approach, but then vendors like Huge and Medea came out with hybrid systems that allow for a SCSI interface connecting to ATA or SATA drives, bringing costs down. But the costs of a 10 bit HD capable system are still in the $5-$7/GB range, so while they are entirely proven, reliable, plug 'n play solutions, I'm moving past them for now due to cost reasons. If you HAD to have, today, a reliable RAID 3 10 bit HD capable (and who doesn't need that?) lower capacity/overall cost solution than X-RAID, Huge or Medea are the way to go, even though you'd pay more per GB. The X-RAID has good performance and cost/GB, but only in the above $10,000 price points.

Especially in comparison to the RAID 0 with backup option I'd been advocating recently, the new X-Serve RAID stuff starts looking more attractive. Especially since my conversation with Rick at MacGurus - he pointed out that drive failure is NOT the primary cause for concern with RAID 0 (or RAID 10) - that directory corruption was the biggest concern, because according to him none of the Mac based tools can fix that on a RAID 0 or RAID 10. You'd be hosed if that happened, relying on restoring from backup, a timely proposition. My best online RAID 0 with nearline (FW800 based backup) solution is to do a 4x400 drive RAID using a Seritek 1SE2 and 2 internal drives, using the Hitachi 7K400 SATA drives, booting from a FireWire 800 drive. Back up to 1TB La Cie Bigger Disk ($1000, so $1.07/GB of usable formatted capacity)UPDATE: At the moment (don't know how long), they are available for $705 from this Dell link). That would give you 1480 GB of usable online speed capacity for HDCAM footage, but only about 500 to 700 GB (this is a guess) of 10 bit 4:4:4 1080p24 capable throughput. For 1080p24 10 bit 4:4:4, at this point you'd have to use a Highpoint RocketRAID 1820A card, but there are reliability problems that I've yet to resolve with this card so I can't recommend it at this point in time for front line prodution systems. Maybe later after testing and analysis or new drivers, but not yet.

So looking at those numbers:

Highpoint RocketRAID 1820A RAID 0

1820A card with SoftRAID, 2 4 bay Burly Boxes (with coolers) 8x300 array (Maxtor DiamondMax 10 drives, $209/ea from zipzoomfly.com):
RAID 0 usable capacity: 2224 GB
Cost: $2635
Cost/GB: $1.18/GB

RocketRAID 1820A RAID 10 option

RAID 10 usable capacity: 1112 GB
Cost: $2635
Cost/GB: $2.36/GB

...and with RAID 10, according to Rick, you still need to worry about directory damage. I need to research this issue further.

RocketRAID 1820A RAID 0 with FireWire 800 backup:
Adding 2 La Cie Bigger Disks and a d2 Big Disk (500GB)
Storage amount (real, usable, formatted capacity): 2326 GB
Storage cost: $2450 (2x$1000+$450)
Backup Cost/GB: $1.05/GB

SO, RAID 0 PLUS BACKUP COSTS:

Storage capacity (primary, backed up): 2224 GB (backup doesn't add to this total, just backs it up)
Total cost, primary plus backup: $5085
RAID 0 with backup cost/GB: $2.28

Now compare that to Apple's $3.11/GB. Apple's is a constant, unattended protected dataspace, you never have to fool with it, it's just THERE. But Apple's solution is only that price point if you commit to (and need) the whole 5.6TB package. The only advantage I can think of to my solution is that your backup can be unplugged (or even offsite, although that's not practical if doing overnight backups) to prevent lightning or other electrical catastrophe. Nice, but not a huge advantage. Plus, the RocketRAID has not proven itself stable and reliable yet. I'm withholding final judgement until I can finish my own testing.

Other alternative: Seritek 1SE2 with 2 internal, 2 external drives booting from FireWire 800

For this application, we're not looking for maximum low cost capacity. In the previous example, I was using Maxtor DiamondMax 10 drives, because they are fast, large, and have a low cost/GB. For this application, since we only have 4 SATA drives available (last I heard the performance with 2 cards wasn't viable), I'm choosing IBM/Hitachi 7K400 400GB drives, which are available for $390 from zipzoomfly.com (price changes frequently, however). Taking the RAID 0 with FireWire backup approach, here's how it can be done:

(This reminds me - I need to try a slot 2 & 3 combo and see how it performs, I haven't tried that yet in my dual 2.5 GHz G5.)

Seritek 1SE2 card: $100 (gives two external SATA ports)
2 bay Burly Box (with coolers): $190
SoftRAID (allows partitioning): $100
4 Hitachi 7K400 400 GB SATA drives: 4x$390=$1560
1 La Cie d2 160GB hard drive to boot from: $190 (pick your favorite cheaper FW800 drive if you wish, I'm picking a nice/safe/easy choice)

Total cost: $1950
Total usable RAID 0 capacity: 1480 GB
Cost/GB: $1.31

Now adding in backup storage, using a 1TB LaCie Bigger Disk and 500 GB d2 Big Disk Extreme:
Total usable capacity: 1393GB *
Backup Cost: $1000 + $450=$1450
Backup Cost/GB: $1.04/GB

Total Back-upable Storage Capacity: 1393GB
Total Storage Cost (primary plus backup): $3400
Total Storage Cost/GB: $2.44

* You didn't want to fill up your array to the last drop anyway, right? Seriously, HFS+ formatted volumes slow waaaaaaaaay down once over 90-95% full.

I looked into the thought of building your own FireWire 800 drive, but the La Cie is cheaper than what I could find out there. I looked into using $50/ea PPA, Inc. cases instead of 2 bay Burly, but wasn't saving enough to make it worth the substantial quality drop. I looked into loose IDE drives and cheap FireWire 400 cases, but the cost savings was small (again a couple of hundred dollars), but the increased hassle factor was HIGH. Do you REALLY want to manage 5 individual drives for your backups? Me neither. Yeah, you could squeeze a few hundred bucks out of this, but why? More trouble than it's worth. If you're squeezing it that tight, you clearly have a LOT more time than money and don't mind spending it (the time). That's a section of the market that I'm just not interested in serving, although that may be where some folks have to play.

SyncRAID XL for 24p HD-CAM or Varicam based projects

There's a new hardware based RAID card available for Macs, which is great. The bad news is that it is 33MHz PCI, so it will slow down a G5's slot 2 (or 3) if placed in slot 3 (or 2) since those two slots share a bus and slow down to accomodate the slowest card in either slot. So that means a single SyncRAID XL is all that is viable at a time in a G5 with an HD card installed. HD card goes in slot 4, SyncRAID XL in slot 2 or 3. Or swap - SyncRAID in slot 4, HD card in slot 2 (cables need to run out an empty PCI slot cover, slot 3 in this example).

The good news is that it will use what they call RAID XL - it is similar to RAID 3 or 5, but just different enough to not be. It writes all it's parity data (redundant data that can rebuild a lost disk's info in event of failure) to a single drive like a RAID 3, but does tiny chunks of data more like RAID 5. Anyway, the card can do 114 MB/sec reads and writes, and the throughput is limited by the card itself, so faster drives are useless. This means you can go buy 5 drives and get 4 drives worth of storage capacity out of the system. So long as your drives can do at least 30 MB/sec even when full, you'll be fine to maintain that 114 MB/sec throughput the drive is capable of. Pretty much all of the recent 250 GB and larger drives can do this.

It does have internal ports, however, which creates the same problems I've been addressing in my ongoing coverage of the Highpoint RocketRAID 1820A card. Search for "1820A+cabl" using the search bar at the top of this site for more info on that topic.

SO:

For those doing 24p HDCAM projects, or Varicam (720p at up to 60fps), this card is fast enough. To be specific: if you're doing 8 bit, 4:2:2, 1920x1080 at 23.976, 24, or 25 fps, this card should just squeak by as fast enough for capturing and single layer work. If you shot at a video friendly 29.97 interlaced frames per second, you're hosed - this is not the solution for you. That data rate is about 118 MB/sec, and needs some safety margin above that. This card only does 114 MB/sec in RAID XL configuration, and that's IT.

If you're doing Varicam work (1280x720p) and capturing via HD-SDI, 720p at 60fps (the max rate) isabout 105 MB/sec, which is getting dangerously close to the limits of the card. But more likely you're working at 720p24 or 720p30, so the data rates are 42 or 52 MB/sec (approx), which is quite comfortably handled by this. I also just realized that if you're working with 720p24 stuff, a pair of SATA drives on a Seritek 1S2 or 1SE2, or a La Cie Bigger Disk, would suffice for single stream editing.

If you are doing a 24p HDCAM project, get one of these cards, a 5 bay Burly Box (or other compatible enclosure), 5 drives, and you are off to the races. Let's look at a couple of possible configs. The benefit of this system is that you DON'T need backup the way you did with the RAID 0 based systems described above. As it has been described to me, you can yank out a drive in the middle of a playback operation, and the system will choke for a few instants then keep playing back just fine. Way way cool. The downside to this card is that there is NO, ZERO, NONE, tech support available from the manufacturer. They have Mac drivers and that's all the effort they have or are going to put into it anytime soon. That's why I'd buy it from a knowledgeable vendor like MacGurus, Rick knows what he's doing over there.

here's a few of different ways you could set it up:

1480 GB with Hitachi 7K400 400 GB drives

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 400 GB Hitachi 7K400 drives: 5x390
(no SoftRAID needed)

Total usable RAID XL capacity: 1480 GB (same as the Seritek 1SE2 example above - one drive is dedicated to parity data)
Total cost: $2520 w/out coolers, $2665 with coolers
Total cost/GB: $1.70/GB without coolers, $1.80 with (might as well!)

=================================================

Using Maxtor DiamondMax10 300 GB drives (better cost/GB, smaller, faster)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Maxtor DiamondMax10 300 GB drives: 5x$209
(no SoftRAID needed)

Total usable RAID XL capacity: 1112 GB
Total cost: $1796 w/out coolers, $1940 with coolers
Total cost/GB: $1.61/GB without coolers, $1.75 with (might as well!)

=============================================

Using Hitachi 7K250 250 GB drives (better cost/GB, smaller size, though, same performance as 7K400 models)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Hitachi 7K250 250 GB drives: 5x$148
(no SoftRAID needed)

Total usable RAID XL capacity: 928 GB
Total cost: $1552 w/out coolers, $1702 with coolers
Total cost/GB: $1.67/GB without coolers, $1.83 with (might as well!)

==============================================

To get accountant geeky, you'll note that the increased cost/GB efficiency of the drives is largely offset by the fixed costs of the housings and SyncRAID card. So your cost/GB holds to a pretty narrow band - about $1.75/GB ($1.61 to $1.83 is the range).

In the limited context of 24 or 25p HDCAM (8 bit 4:2:2 1920x1080), this is an excellent low cost solution. Keep in mind, however, that this system will NOT play back dual stream HD as I understand it - so you'll have to render all your uncompressed transitions or PIP stuff.

They are supposedly working on faster cards, I'll report what I know when I know something.

It might, MAYBE be possible to install two of these (in slots 2 & 3) and NOT stripe them together, just run two separate RAIDs if you got clever and dremeled out the PCI slot cover on the card to dig an escape tunnel for the SATA ports. Did I mention all the ports are internal, like the Highpoint RocketRAID 1820A? I should have. That way, you could double the storage capacity of the system.

Random other thought: Even from a FireWire 800 based La Cie 1TB Bigger Disk, it'd take about 4 hours to read 1TB from that drive...and whether a SATA RAID could do an asynchronous write (write while reading from FireWire drive) I don't know - I'll have to generate a bunch of test data and test actual transfer times. So the RAID XL is a very valid option if you can live with it's limits.

In summary:

If you can afford it, get the fully populated X-Serve RAID. Fast, scalable, high availability, protected, warrantied.

If you are on a budget editing, consider a cheapie Seritek 1SE2 based array on another PCI-X G5 for a capture station then copy the footage over. Nah, that REALLY sucks, because then you're stuck with 50 MB/sec Ethernet transfers (fastest gigabit Ethernet Mac to Mac transfer I've done got 50 MB/sec of real world performance, stopwatching it).

OK. So if you can't afford $14K for the X-RAID setup, what to do?

If Highpoint's RocketRAID can be shown to work reliably and stably (it hasn't yet), RAID 0 with FireWire backup is atractive and scales up to 2960 GB of usable RAID 0. Backing up to three 1TB La Cie drives (plus one 250GB) would let you work with a total of 8 2/3 hours of 24p HDCAM footage, or 5 hours of 24p HDCAM SR footage. But if it had irrepairable directory damage when full, you'd be looking at about 16-25 hours of data copying time to get it all back up and running as it was since your last backup. Ick.

OK, what else? If you're working offline/online for feature work, or on shorter projects, I've been running a Seritek based array for months without a hitch (but not for much uncompressed work). Depending on the flavor of HD you're working with, you'd have 1 to 4 hours of uncompressed HD storage. Search (using search function at top of page) for "partitioning" to find articles on this site dealing with partition strategies for particular flavors of HD on smaller arrays. Back it up with FireWire and you might get taken down a day or two if catastrophe hits, but not taken out of the game IF you've stayed on top of your backups.

NOTE ON BACKUPS: And with footage, you only have to back up your captured footage once on your vulnerable array. I configure my FCP to save a backup every 10 to 30 minutes. I have Toast set up to back up that directory every night during projects. Photoshop docs, After Effects files, artwork, titles, etc. all do NOT get saved on disk arrays and get regular backups elsewhere. This keeps your backup regime light - once you've captured all your footage and backed it up and are just editing, you don't need to back it up again - it's not changing until you capture more footage. If I have to re-render my transitions, I don't sweat it. That's TIME, not WORK. Set it rendering and walk away. Time is valuable, to be sure, but my decisions would all be safely backed up, I could go do other things elsewhere while the machine regenerated rendered footage.

If you're doing 24p HDCAM or Varicam footage (DVCPRO HD tape), assuming it works as advertised (mine's on the way), the SyncRAID sounds great - low cost, protection in case of drive failure (immune to the kind of directory troubles RAID 0 has as I understand it at this point), and you MIGHT be able to put two of them in there for more storage if you get clever with a dremel tool on your cable routing (detach and drill out the PCI slot covers to allow cables to pass through). You can't effectively stripe them together for faster storage, but you MIGHT be able to set up two separate RAIDs to double your storage capacity.

If you can't afford backups, you can run without - I wouldn't, though. If the deck is inhouse (a rare luxury) you could batch recapture - but that takes work AND time. If you have to rent it, then it's time, work, AND money to recapture. How often do drives fail? In my 5 computer household, where each machine has 2-5 drives, a drive dies about once a year or two. With a bunch of drives (up to 8) in an array, my gut says be prepared for a failure about once a year or so. But that's a total guess. Run your own risk cost analysis - what does a backup system and plan cost me in terms of time, money, and complexity? What would it cost me in terms of time, work, and money if I didn't have backups and had to recapture all the footage? In a client environment, I can't imagine that every being acceptable. In an inhouse environment, it would really, really suck and be survivable the way an appendectomy is survivable - painful, time consuming, expensive, and leaves a bad taste and an ugly scar.

Hmm, on that cheery note, I'm off for a run.

-mike

So I've recently noted the existence of some stuff, and as my brain tends to do, it mashes them all together. Consider the following evidence:

SATA drives of up to 400GB capacity are available for under $1/GB
the new single processor 1.8 GHz G5 is $1500
the SyncRAID XL 5 port SATA PCI card is $300
5 bay Burly Boxes (SATA enclosures) are $270 standard, $420 with drive coolers for each drive
Apple's PCI-X (but PCI compatible) 2GB fiber channel card is $500
Apple's 10 client OS X Server software is $500

There is a need to keep data backed up securely and safely for editing and/or for workgroup access.

So consider this:

Scenario 1: You either need to backup a ton of data and keep it online and accessible over gigabit ethernet (about 50 MB/sec top transfer speed Mac to Mac)

OR

Scenario 2: You need a machine to be accessible via X-SAN on a fiber channel network, and push file around at over 100 MB/sec. This would be nearline for high data rate uncompressed HD, but online for some (1080p24 4:2:2 8 bit, all 8 bit 720p work) HD and all SD work.

In either situation, you are, of course, budget constrained, so fast/cheap/good is the order of the day.

I don't know if what I'm thinking of will all work together as planned for some of the following reasons:

-I don't know if multiple SyncRAID XL cards will "play nice" in the same computer
-I don't know how slow Apple's 2GBit fiber would run if forced to operate in 33Mhz PCI mode
-I don't know if X-SAN can access drives that are NOT fiber channel based on a machine, but that does have a fiber channel card installed in the machine

Assuming everybody plays nice (and that's always a stretch), here's some interesting possibilities:

For scenario 1 (Ethernet server):

Buy the following items:

single processor 1.8 GHz G5, with 10 user OS X Server software (unlimited users is $500 more), drop the modem, downgrade to Combo drive:

$1869

Buy 2GB of third party RAM (from crucial.com or somesuch): $200

Buy 3 SyncRAID XL 5 port SATA cards ($300/ea from macgurus.com), remove the PCI slot covers, drill them out to allow 5 SATA cables to pass through. Tape over any sharp edges to avoid cable chafing.

Buy 3 5 port Burly Boxes from MacGurus for $270 apiece. Add drive coolers for $450 more if you wish (will be safer, cooler, live longer, but run LOUDER). $810 without coolers, $1260 without.

Take a deep breath and buy 15 400 GB Hitachi 7K400 drives, presently $390/ea at zipzoomfly.com. If you're smart, you'll buy an extra Just In Case for that rainy day.

Buy (or just borrow to set up) any monitor capable of 1024x768. Run it headless or not, it'll work either way, your choice.

Monitor Cost: $0 (headless)

TOTAL COST:$9329 if my math is right

If add drive coolers and upgrade to dual 2.0 GHz G5, is about $10,300 for a nice workgroup server with a LOT of storage on it.

Install your modified cards, mount all the drives in their respective Burly Box chassis, set up the Server, connect all 15 SATA cables to the drives, set up the 3 disk arrays. You now have an OS X server with three 1480 GB protected RAIDs, for a total of 4440 GB of quite fast storage. Each is capable of about 115 MB/sec throughput. It's RAID XL, which is something like a cross between RAIDs 3 & 5 - dedicated parity drive, tiny divided pieces of data. In any case, you can literally yank a drive out in the middle of an operation, and the Mac will pause briefly and resume...AT IT'S PRIOR READ SPEED. Nice. Oh, and of course you didn't lose any data.

If something isn't playing nice, perhaps bumping up to a dual 2.0 GHz Mac for $500 more is worth it.

Scenario Two: high speed, online, X-SAN node

BIG FAT DISCLAIMER: This may totally not work, as SAN is all about consolidated storage in one place. But if a client node can read/write, and client can write to client...it would be cool. I have a gut feeling it probably won't, but it's a fun idea.

Do the same thing, but upgrade do a DP 2.0 GHz G5, drop out 5 drives, one Burly Box, one SyncRAID XL card, and order a fiber channel card when you order that Mac. It'd be something like this:

Buy the following items:

Dual processor 2.0 GHz G5, with 10 user OS X Server software (unlimited users is $500 more), drop the modem, downgrade to Combo drive, add the fiber channel card:

$2369

Buy 2GB of third party RAM (from crucial.com or somesuch): $200

Buy 2 SyncRAID XL 5 port SATA cards ($300/ea from macgurus.com), remove the PCI slot covers, drill them out to allow 5 SATA cables to pass through. Tape over any sharp edges to avoid cable chafing.

Buy 2 5 port Burly Boxes from MacGurus for $270 apiece. Add drive coolers for $450 more if you wish (will be safer, cooler, live longer, but run LOUDER). $810 without coolers, $1260 without.

Take a deep breath and buy 10 400 GB Hitachi 7K400 drives, presently $390/ea at zipzoomfly.com. If you're smart, you'll buy an extra Just In Case for that rainy day.

Buy (or just borrow to set up) any monitor capable of 1024x768. Run it headless or not, it'll work either way, your choice.

Monitor Cost: $0 (headless)

X-SAN software: $1000/seat (so then is server software necessary? Probably, but dunno, need to read up on X-SAN...)

TOTAL COST: $9109

Hook it all up, and IF it all plays nice together (large if), you have a nice fast node on your X-SAN, perfect for stashing assets you want available at high speed. I don't know how far you could push it, but if the SATA storage is accessible over the X-SAN, you should be able to get at least 1 if not 2 or maybe 3 uncompressed SD streams working over that fiber pipe. If for some reason the PCI factor screws up the deal, bump the machine up to a dual 2.0 GHz G5 (cost is now $9109) and put the PCI-X card in slot 4 where it can run full speed and attach to your pre-existing X-SAN setup. Assuming IDE drives can play on X-SAN (SyncRAID volumes appear as a single IDE drive to the Mac, the beauty of hardware RAID), you should be able to read/write at over 100 MB/sec over the X-SAN setup.

X-SAN requires a fair bit of infrastructure - a fiber switch (costly!), $1000/seat for anybody who wants access to the SAN, etc.

Maybe. Or maybe more likely not. I'll have to read, we'll have to see. But option one seems like a pretty sure bet, and bumping up to a dual 2.0 GHz box is not a big deal.

Formac has some new LCD monitors, including the Gallery Xtreme 19", that could be useful for monitoring HD via HDLink, one2one, or AJA's box (forget the name). (Use the search bar at top of this page if you don't know what those are.)

Here's the specs for the 19" model which could be good for 720p (Varicam footage) monitoring:

1280x1024 resolution
DVI input
280 nits brightness
700:1 contrast ratio
15 ms pixel response

The 19" model has 1280x1024 resolution, enough for 1280x720 resolution footage. It sells on their website for $599.

Another nice feature that they offer that I wish others would (Hello, Apple!) is a Zero Dead Pixels Guarantee for $99. Good idea! I just hope they aren't disingenuous to suggest that STUCK pixels (one or more of the red, green, or blue elements in a pixel is stuck off or on) isn't the same as a DEAD (always black) pixel, and therefore not covered. If I'm paying extra, I want it PERFECT.

-mike

Over on the Creative Cow's forums there is an excellent discussion on going out to film from Varicam originated material. This K. Han guy seems to have a good grasp on the nitty gritty involved in getting maximum quality out of Final Cut Pro, and what it's shortcomings are. Some of his points:

FCP render engine is YUV ONLY, not RGB...it has a great YUV environment, which is great for video, but converting YUV to RGB and back to YUV is BAD and introduces errors and quality loss. If you're destined for film (or digital projection that isn't video based), this is NOT GOOD.

Algolith CAS (Content Adaptive Scaling) can't run well in FCP, since FCP plugin environment is 8 bit only, and YUV in and out at that (Mike Note: Final Cut Pro's scaling algorithm SUCKS, I'd never use but for roughs, never final anything!)

Magic Bullet for editors - beware! It uses After Effects API (Application Programming Interface), so it's 8 bit only, not FXscript that would allow for greater color depth.

so there's no high quality scaling capability available native to FCP...a problem

biggest challenge in DIY DI is monitoring - large cost! RGB linear to RGB log previewing is complex and costly, can be done on desktop hardware, but costs $30-$40K

To get best results, do your editing but do NO effects inside of FCP, then carry that timeline to your RGB based post processing application, such as Shake, combustion, After Effects, Digital Fusion, etc. to do all of your color correction and transitions there (Mike note: Automatic Duck makes a line of products specifically to do this.)

If you want to read his specific two posts, here's links:

Post 1

Post 2

-mike

BlackMagic Design has released new drivers, version 4.6 (scroll down to find link on that page) that now supports downconversion of DVCPRO HD footage to standard def in realtime. From their site:

These new drivers add support for DVCPRO HD down conversion in all standard and high definition DeckLink cards. You can now playback DVCPRO HD material, with RT Extreme HD effects in Final Cut Pro HD, and then output down converted standard definition video in real time.

HDTV Down Conversion is now enabled via the Mac OS X™ Blackmagic DeckLink System Preferences, allowing this feature to be enabled in video applications such as Blackmagic Deck Control.

HD video outputs are now enabled via down converter on all standard definition DeckLink cards. When a standard definition DeckLink card is used, all playback of HD files will automatically be down converted and outputted via standard definition video outputs.
This new Mac OS X™ software update is for all shipping Blackmagic SD and HD DeckLink models, including Kona SD.


Mike's Commentary: This offers some new workflow possibilities, regardless of whether you're working in compressed DVCPRO HD or uncompressed HD:

-if you're on a tight budget (and who isn't), you can use a standard definition monitor with component inputs to monitor your HD footage if you have the DeckLink HD Pro card (single or dual link model) since it has analog monitoring capabilities.

-since they added realtime capture to DVCPRO HD last week in version 4.5, this means you can capture your HD footage to DVCPRO HD and use that as an offline codec if the frame size and frame rate matches something DVCPRO HD can do. (DVCPRO HD cannot do 1920x1080 progressive frame rates, so 1080p24 is out unfortunately).

-if you have HDCAM 1080i60 footage, you can use DVCPRO HD as either an offline or online codec - the ability to work with HDCAM footage and NOT have to have a disk array is a very nice option. 1080i60 DVCPRO HD footage is about 50 GB/hr, so a $400 400GB second SATA drive in your G5 could hold nearly 7 1/2 hours of footage.

-mike

Updated Wednesday early morning with new analysis and pricing

I've finally gotten back to messing around with the Highpoint RocketRAID 1820A cards I've got after a busy weekend at SXSW.

A few conclusions:

-the fastest RAID 0 configuration I could come up with used 6 drives on an 1820A card in slot 2, 6 drives on an 1820A card in slot 4, 2 internal drives, and on an empty array managed 592 MB/sec reads, 526 MB/sec writes (average of 5 runs). That setup yields 3892 GB of usable, formatted RAID 0 space.
-this 6+6+2 config could be built for under $4000 - 3 4 Bay Burly Boxes (3x$230 each from MacGurus), 2 Highpoint RocketRAID 1820A cards and 14 Maxtor DiamondMax 10 300 GB SATA hard drives ($204 for card, $209 per drive, both from ZipZoomFly, SoftRAID formatting software from SoftRAID, and some time with a screwdriver and dremel tool (pliers and a blowtorch optional).
-6 drives on each of 2 1820A cards are faster than 8 drives on each of 2 1820A cards
-adding a couple of internal drives (attached to "normal" connectors in G5) helps, but not as much as I'd hope
-slot 4 definitely yields faster results than slot 2 for RAID applications....but both are very fast (412 vs 470 MB/sec reads...writes are card limited to about 345-350 MB/sec)
-I, as well as others, have had drives not show up upon reboot (happened to me when I set up 16 drives in one array and rebooted). Shutting down, spinning down all drives, turning on drives a minute or two before computer helps sometimes not always
-so I still have hesitancy to suggest anyone use this as a primary working machine
-if you're just doing 8 bit 24p HDCAM, the SyncRAID XL from MacGurus with a 5 bay Burly Box is worth considering. If you're doing 10 bit or 1080i60 it definitively isn't fast enough. I have one on order, report coming soon.
-for optimal speed, put 1820A in slot 4
-for optimal expansion options, put 1820A is slot 2 or 3 and HD card in slot 4 of G5, which allows other RAID cards to be insalled (if you have a place to run the cabling)
-since the 1820A is having drive dropout problems, I'd recommend NOT buying the SATAport from MacGurus for use with the 1820A at this time, until it can be determined whether it's a hardware, software, or whatware problem with drives not mounting.
-almost broke 600 MB/sec on one test run (597.3MB/sec read). Mac SATA throughput scales way, WAY up there, FAR beyond SCSI capabilities with a single card. This is also more than the theoretical maximum of 2 gigabit fiber channel connections.
-while I haven't specifically tested this yet, I see no reason why you couldn't have 2 separate 8 drive arrays running in a Mac G5, each running either RAID 0 or RAID 10 (RAID 10 is limited to no more than 2TB per card right now, is a driver problem that should be surmountable). If that driver issue is resolved, that would allow on EACH card up to 2960 GB of usable RAID 0 space using 400 GB drives, or 1480 usable TB of RAID 10 (mirrored disk array, fully redundant set of disks). This would allow the Mac to have up to 5920 GB of usable space for uncompressed HD editing, or 2960 GB of of RAID 10 for uncompressed HD editing. Have I posted RAID 10 results? Somebody bug me if I haven't yet.

Testing setup: all drives tested in array were Maxtor 300GB, either DiamondMax 10 or Maxline III, either way they generate virtually identical performance statistics. The dual 2.5 GHz G5 was running OS X 10.3.4 with 512MB of RAM (until replacement RAM arrives, I got a bad stick or two), formatted using SoftRAID 3.0.3, performance was tested using BlackMagic Design's Disk Speed Test application. My reasoning for using that as a benchmarking tool is because it supposedly simulates capturing digital video to disk, and as that is my primary interest that's a good metric for me. The system was booted from a La Cie Bigger Disk (1TB) on FireWire 800 since some of these tests used both interal SATA drive slots as part of the RAID testing.

-mike

Can I write a drier, dorkier, longer headline? I'm not sure, but clearly some part of my brain is working on it. I'm not sure who else is selling it, but I found this at Mac Gurus recently - the new SyncRAID XL card. I described it in another article I wrote and may not have published yet, so here's what I said there:

There's a new hardware based RAID card available for Macs, which is great. The bad news is that it is 33MHz PCI, so it will slow down a G5's slot 2 (or 3) if placed in slot 3 (or 2) since those two slots share a bus and slow down to accomodate the slowest card in either slot. So that means a single SyncRAID XL is all that is viable at a time in a G5 with an HD card installed. HD card goes in slot 4, SyncRAID XL in slot 2 or 3. Or swap - SyncRAID in slot 4, HD card in slot 2 (cables need to run out an empty PCI slot cover, slot 3 in this example).

The good news is that it will use what they call RAID XL - it is similar to RAID 3 or 5, but just different enough to not be. It writes all it's parity data (redundant data that can rebuild a lost disk's info in event of failure) to a single drive like a RAID 3, but does tiny chunks of data more like RAID 5. Anyway, the card can do 114 MB/sec reads and writes, and the throughput is limited by the card itself, so faster drives are useless. This means you can go buy 5 drives and get 4 drives worth of storage capacity out of the system. So long as your drives can do at least 30 MB/sec even when full, you'll be fine to maintain that 114 MB/sec throughput the drive is capable of. Pretty much all of the recent 250 GB and larger drives can do this.

It might, MAYBE be possible to install two of these (in slots 2 & 3) and NOT stripe them together, just run two separate RAIDs if you got clever and dremeled out the PCI slot cover on the card to dig an escape tunnel for the SATA ports. Did I mention all the ports are internal, like the Highpoint RocketRAID 1820A? I should have. That way, you could double the storage capacity of the system.

SO:

For those doing 24p HDCAM projects, this card is fast enough. For doing compressed HD work, with DVCPRO HD or PhotoJPEG, it'd be completely fine. Also, for any framerate 720p 8 bit work it'll be fine, and for some 720p 10 bit work as well. To be specific: if you're doing 8 bit, 4:2:2, 1920x1080 at 23.976, 24, or 25 fps, this card should just squeak by as fast enough for capturing and single layer work. If you shot at a video friendly 29.97 interlaced frames per second, you're hosed - this is not the solution for you. That data rate is about 118 MB/sec, and needs some safety margin above that. This card only does 114 MB/sec in RAID XL configuration, and that's IT.

If you are doing a 24p HDCAM project, get one of these cards, a 5 bay Burly Box (or other compatible enclosure), 5 drives, and you are off to the races. Let's look at a couple of possible configs. The benefit of this system is that you DON'T need backup the way you did with the RAID 0 based systems described above. As it has been described to me, you can yank out a drive in the middle of a playback operation, and the system will choke for a few instants then keep playing back just fine. Way way cool. The downside to this card is that there is NO, ZERO, NONE, tech support available from the manufacturer. They have Mac drivers and that's all the effort they have or are going to put into it anytime soon. That's why I'd buy it from a knowledgeable vendor like MacGurus, Rick knows what he's doing over there.

here's a few of different ways you could set it up:

1480 GB with Hitachi 7K400 400 GB drives

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 400 GB Hitachi 7K400 drives: 5x390
(no SoftRAID needed)

Total usable RAID XL capacity: 1480 GB (same as the Seritek 1SE2 example above - one drive is dedicated to parity data)
Total cost: $2520 w/out coolers, $2665 with coolers
Total cost/GB: $1.70/GB without coolers, $1.80 with (might as well!)

=================================================

Using Maxtor DiamondMax10 300 GB drives (better cost/GB, smaller, faster)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Maxtor DiamondMax10 300 GB drives: 5x$209
(no SoftRAID needed)

Total usable RAID XL capacity: 1112 GB
Total cost: $1796 w/out coolers, $1940 with coolers
Total cost/GB: $1.61/GB without coolers, $1.75 with (might as well!)

=============================================

Using Hitachi 7K250 250 GB drives (better cost/GB, smaller size, though, same performance as 7K400 models)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Hitachi 7K250 250 GB drives: 5x$148
(no SoftRAID needed)

Total usable RAID XL capacity: 928 GB
Total cost: $1552 w/out coolers, $1702 with coolers
Total cost/GB: $1.67/GB without coolers, $1.83 with (might as well!)

==============================================

To get accountant geeky, you'll note that the increased cost/GB efficiency of the smaller capacity drives is largely offset by the fixed costs of the housings and SyncRAID card. So your cost/GB holds to a pretty narrow band - about $1.75/GB ($1.61 to $1.83 is the range).

In the limited context of 24 or 25p HDCAM (8 bit 4:2:2 1920x1080), this is an excellent low cost solution. Keep in mind, however, that this system will NOT play back dual stream HD as I understand it - so you'll have to render all your uncompressed transitions or PIP stuff.

They are supposedly working on faster cards, I'll report what I know when I know something.


-mike

DealMac has this link to Dell's site for the La Cie Bigger Disk Extreme 1TB Triple Interface hard drive for $705, about $300 less than I just paid...each....for two....dammit.

I don't know how long the deal is good for, so if you know you need it, snag it. At this price it is the most convenient best deal going for bulk backup of RAID 0 or single drives.

I've been in a hurry buying stuff lately, and I can see where it's cost me several hundred dollars already. On impulse I bought three "cute" uninterruptible power supply (UPS) devices, and I should have ordered an ugly but serviceable refurb from UPS Refurb. I could have gotten more capable units for about 25% less, and saved $120. Take your time, be sure of what you want, don't order anything after midnight - your judgement is shot!

Plus, some 400 GB hard drives that I'd ordered (and thought I'd cancelled) two months ago showed up on my doorstep. Not only did I not want or need them, I've paid $40/drive more than I can get them for now. Drat! I have to pay for shipping them back, so I've now paid overnight shipping AND return shipping for...nothing.

So take your time, shop carefully, and really consider if you Have To Have something as soon as it is available. I should heed my own advice and not buy anything until it's been readily available for a couple of months - what am, 12 years old at Christmas? Grumble grumble....ego grumble grumble....first on block grumble grumble....

-mike

Apple has re-introduce the single processor 1.8 GHz PowerMac G5 at a new, lower price point - $1500.

System specs are as follows:

1.8GHz PowerPC G5
600MHz frontside bus

512K L2 cache
256MB DDR400 SDRAM
Expandable to 4GB SDRAM
80GB Serial ATA
8x SuperDrive
Three PCI Slots
NVIDIA GeForce FX 5200 Ultra
64MB DDR video memory
56K internal modem

Mike's Commentary: So what does this mean for HD editors?

This model is clearly their business class desktop for those who don't want an iMac. But it is NOT a pro caliber workstation. The bus speed has been dropped to 600 MHz from 900 MHz, and even more critically, this machine has NO PCI-X slots. This means no high speed SCSI or SATA cards, and more importantly, no low cost HD capture cards. Yes, technically, you could go get yourself a Pinnacle Cinewave card and slap it in here, but they are so expensive as compared to the other more recent options you'd be shooting yourself in the foot about 4 different ways.

So I firmly do not recommend this machine for any serious stage of HD post production. It's too slow and lacks the critical slot for HD related peripherals and devices.

The only scenario I could see it being useful in is if you're working with Varicam footage via FireWire on a Panasonic AJ-1200A station, this could be a low cost capture/edit station...but I have a sneaking suspicion that if you have a dual processor 1.42 GHz G4 sitting around, it'll be faster than this G5. Since they don't sell G4s anymore, if you were working with the 1200A and had it available on an ongoing basis to monitor (FireWire DVCPRO HD to deck, HD-SDI or analog to monitor) it could work...but more slowly than it's more costly brethren. And lacking that deck you wouldn't have a way on either station to cost effectively monitor your HD footage "for real" (Digital Cinema Preview doesn't count as I've already written about, use the search bar at the top of this screen and enter "Digital Cinema P