HD For Indies

Adam Wilt's discussion of DVCPRO HD codec (and some other codecs)

Among other things, he talks about the rumor that 1080i50 will have a higher horizontal resolution than the 1080i60 model - that DVCPRO HD 1080i50 might be 1440x1080, instead of 1080i60's 1280x1080. So even better for the indie moviemaker crowd since it would capture more detail. The next question is whether it will truly be PAL (which stands for Phase Alternating Line) and skip every other field for color data. In other words, is it going to be 4:2:2 like it's NTSC cousin or 4:2:0?

Sony (totally unbiased source, right?) has this article on film vs HDCAM resolution

Steve Mullen's excellently detailed write-up on the Sony HDR-FX1

In short, this is the best in-depth article on the camera I've seen so far. Nice job Steve!

Some interesting bits:

-he talks about how the sensors on the CCD array are double width, to give the 1920x1080 output from the 960x1080 CCD array

-he talks about how the green pixel sensors are offset, halfway between the red and blue pixels, thus increasing effective resolution...sometimes...depending on color and stuff....to give an effective resolution of 1440 (1.5x the ostensible 960 pixel resolution)

-he talks about how the CCD chip has micro lenses to focus the light coming towards the chip directly and only on the portion that measures light - this is way clever, and helps the camera's light sensitivity

-he talks about how the FX1 uses an elementary stream, not a transport stream of MPEG-2, and thus won't record directly onto D-VHS tape. Bummer.

-it's 4:2:0

-he talks about the Pro model WILL have XLR connectors, and he has photos to prove it

-he discusses the CinemaFrame 30 mode, and how it's decent quality

-he discusses CinemaFrame24 mode, and how it drops every 5th frame, so movement will have an uneven flow, a stutter to it. Ick.

I bought a dual 2.5 GHz G5 today almost on a lark - they had them in stock (only 2) at my local Apple Store (Barton Creek Square Mall in Austin, Texas), so I jumped at the chance and got me one.

I also received my 2nd (working) Highpoint RocketRAID 1820A card in the mail and laced up a horrible mess of cables.

I also happen to have 16 300GB drives sitting around the house. Bwahahahahaaaaaa....

If you haven't read my previous report (let's call it Frankentosh 1.0), scroll down to the article below to read it.

I tested with BlackMagic card in slot 4 (finally got a Mac it'll fit in Previously I couldn't get it to fit in my Dual 2.0 GHz G5 for fear of breaking something on the motherboard)

I put the 2 1820A cards in slots 2 & 3, and just lay the G5 on it's side and draped the cables over the top end of the computer, putting the clear airflow panel on the supine Mac and closing it as much as it would (not much). Drill/punch/cut time if I wanted to seal up the Mac.

The upshot -

Not a whole lot faster. My nomenclature:

X+Y+Z

where X=number of drives connected to the first 1820A connector
Y=# of drives on 2nd 1820a
Z= # of drives on internal SATA connectors (the "normal" ones)

I tried 7+7+2, got 481MB/sec reads, 424 MB/sec read/write
I tried 6+6+2, got 495.3/430.5 read/write (448/433 on last part of array)
6+6 was 439/380 read/write, 390/380 at the tail of the array
I tried 7+2 just to see if that did any better than 8+2, but was in slot 2 or 3, and got 500/425 read/write
In slot 4, 8+2 got 575/415 read/write

Lessons learned:

The card doesn't scale performance in a way that would be substantially useful for HD usage with an HD card installed.

With two cards installed, the RAIDman software that comes with the 1820A won't create a RAID bigger than 2TB. Dumb but true (they're working on it)

RAIDman can't span a RAID across two cards.

SoftRAID and Apple Disk Utility, while they cannot create RAID 10 or RAID 5 volumes, can create RAID 0 volumes that span cards (and can include internal SATA drives in the "standard" 2 internal SATA slots).

"Only" 430 MB/sec write speed seems to be the cap...but it holds it all the way to the end of the array, so clearly it's a card limitation, not a drive limitation. I was a bit bummed - with single SATA cards, my experience has been that drives achieve about 90% of the speed in an array (per drive) that they would by themselves. So In theory, this 16 drive array might have achieved 960 MB/sec.

I got read speeds a bit over half of that, but write speeds well below half of that.

This is all still RAID 0, so back up everything regularly to a stack of drives or else your are playing Russian Roulette with a LOT of bullets in the chambers. I'd expect an array of this size to fail every 3-12 months, just at a guess. Remember, 16 drives in an array makes the array 16 times more likely than a single drive to fail if you're only using RAID 0.

If you did suffer a crash, even with 50 MB/sec transfers, you'd be looking at over 25 hours to restore (after fixing) 4.37 TB in a fully populated RAID 0. This was for stunt purposes - nothing in HD res is going to nee that kind of throughput. If you needed this kind of capacity, break it into smaller logically sized/throughput capable units. But it is nice to know you could read & write 2K film scans (2048x1436 10 bit) in real time. Hmm, how much footage is that? About 4 hours and 42 minutes. Not bad!

Nope, not a typo - that's five hundred and seventy-five Honest-To-God MegaBYTES per second, running on a G5, for less than $3000 (on top of the G5, of course).

How'd I do it? Read on for details...

I've been researching high speed, large capacity, low cost storage options for over a year in my quest to come up with low cost high quality solutions for independent filmmakers using high definition video. I wanted to find solutions that would allow for high quality, uncompressed HD video, which can require as much as 270 MB/sec (with safety overhead).

(For more on how to do this, read this, this, this, this, and this. Or just read the archives that are organized by month at the top right of this page, there are over 300 articles from this year alone.

I've been looking into SATA (Serial ATA, what the Mac G5's ship with) based solutions for some time now. The stock G5 can handle two drives - not nearly enough. Then along came the Seritek 1S2 card, and it allowed for 2 more with some ungainly cable arrangements. Better, but not great.

(Since then an external port version, the Seritek 1SE2 card has been announced.

Then suddenly out of the blue the other week, one of the better low cost cards on the Windows side suddenly got Mac drivers, and I was all over it. The Highpoint RocketRAID 1820A card comes with 8 SATA connectors, although they are all internal ports. To work around that hindrance, I'm running all the cables out an empty PCI slot cover.

I'm using eight Maxtor Maxline III 300 GB SATA drives, which are the fastest large SATA drives I've found on the market. The Western Digital Raptor drives are faster, but they only hold 74GB of data.

So where are all these drives? I've put them in a couple of MacGuru's Burly Boxes that I've built up. (And I do mean built, it takes an hour and a half to assemble all the parts and load all the drives, and I've done it several times now.)

I also put two more drives (these are Maxtor DiamondMax 10 models, mechanically identical to the Maxline III's) in the two internal bays in the G5 to bring the total of drives up to 10 300 GB drives.

In order to start up the computer, I'm booting from an external FireWire 800 drive (a La Cie model), since none of my RAID formatting software allows the computer to boot from a RAID.

For HD video capture, I'm using a BlackMagic DeckLink HD Pro 4:4:4 card, which comes with a convenient testing utility that measures actual throughput of a given drive or drive system.

(Just to be thorough, the rest of my system that can edit uncompressed 1920x1080 30 bit color 4:4:4 full RGB uses an HDLink with an Apple Cinema Display 23" LCD for HD signal monitoring.)

Using that as a benchmarking utility, and averaging 7 test runs together, I get an average disk read data rate of about 575 MB/sec, and an average Disk write data rate of about 416 MB/sec.

The 1820A card seems to max out at about 345 MB/sec on write speed - even when more drives are added to 1820A, that's as good as it gets. But it can achieve that speed with as few as 6 drives. Adding more drives continues to boost the read speed, but not the write speed.

By adding two more drives to different SATA buses (the built-in ones on the motherboard) allows me to boost the write speed past the 345 MB/sec limit. This is kind of cheating, since I'm using some internal and some external drives for this array, an ungainly setup. But it would also be possible with 3rd party cables to connect external drives to the SATA ports on the motherboard to store the drives outside the G5 case.

OK, so 575 MB/sec is the headline attention grabber...but you do NOT get that performance with all of the disk. As soon as you start filling up the disk, it gets slower. Huh?

In brief: a hard drive is like a record player...not that I've owned one since I was 12. The platters are like the record, the read/write heads are like the needle. And like a record (you've seen them on late night TV, right?) the record starts at the edge when the needle is placed way out at the outermost part. This part is moving very fast under the needle. The maximum amount of hard drive platter is moving under the read write heads every second at the edge. As data is written from the outermost tracks towards the middle, the more full the drive gets, the more it has to work with the inner tracks of the platters, or record in our analogy. And that is the part of the record/platters that moves the slowest under the read/write heads.

If that doesn't make sense, think how fast it was to hang on to the outer edge of the merry go round instead of the middle. Faster=more data throughput.

If that didn't make sense, just trust me on this: hard drives can't read and write data as fast as they get full as they could when they were empty. The transfer rates fall off as you have to work with the innermost tracks.

I tested my 8 Maxline III 300GB drive array by breaking it into 5 partitions of 435ish GB, and a final partition of 50 GB. The reason for that last smaller one was to test performance at the very end of the array where it is the slowest.

Here are the test results from the BlackMagic Disk Speed Test utility:

As one big partition (the whole 2.2TB)- 471/345 read/write

Slice1 (the first 435GB)- 467/345 read/write
Slice2 (the second 435GB)- 451/342 read/write
Slice3 (the third 435GB)- 418/342 read/write
Slice4 (the fourth 435GB)- 293/344 read/write
Slice5 (the fifth 435GB)- 345/354 read/write
Slice-last50GB (the last 50 GB)- 275/290 read/write

So, how's this done for so little?

1 Highpoint RocketRAID 1820A PCI-X card with 8 internal SATA ports: $204 from zipzoomfly.com
10 Maxtor Maxline III 300 GB SATA hard drives - from zipzoomfly.com, $220 each, $2200 from zipzoomfly.com
2 MacGuru's Burly Boxes - $215 each, $430 from macgurus.com
4 Molex to SATA power adaptors (NOT included w/Burly Box) - $6ea, $25 MacGurus.com
buncha zipties: $1 - Fry's or wherever

Total Cost: $2860

If I were really doing this for a production setup, I'd spend a bit more money.

SoftRAID for RAID setup and partitioning - $100

I'd buy drive coolers for all Burly Box drives. Heat is the leading cause of early hard drive death in my experience. 8x$30ea = $240

Since there is really no call for the extreme throughput, unless I really needed the additional capacity, I'd stick with an 8 drive array. It would still give about 2200 usable GB of space, and have the convenience of an all external solution. That would drop the price by 2 drives, so $440 less.

And the biggie for last - all of this has been RAID 0, which means if ANY one drive in the array fails, ALL data is lost short of heroic (and EXPENSIVE) efforts. The only reason why I suggest a RAID 0 is that in the context of a video production job, the most crucial data is the file you create with your NLE (usually Final Cut Pro HD these days) and the audio and video data you capture, plus any graphics you create along the way. As long as you have those assets backed up, you're OK. READ HERE FOR BACKUP STRATEGY DETAILS.

When capturing footage, that's usually done in a large chunk early in the project. After that, it doesn't change much. If you're willing to copy that capture data to backup hard drives (such as FireWire drives like the La Cie d2 or Bigger Disk or Bigger Disk Extreme), you're covered in case of drive failure. You'd lose significant time getting back up (a day, maybe two), but life would go on.

So you'd be looking at $2000 to $2500 worth of backup space if you're at all serious about this.

"Real" setup cost for 8 drive array capable of as much as 470 MB/sec reads and 345 MB/sec writes:

1 Highpoint RocketRAID 1820A: $204
8xMaxtor Maxline III 300GB SATA Drives: 8x$220=$1760
MacGurus Burly Box with coolers and power adaptors: 2x$347=$694
SoftRAID: $99 from softraid.com

Total Cost: $2757

Backup FireWire 800 storage

2 La Cie Bigger Disks Triple Interface (2x$1000) and one 250 GB d2 Triple Interface ($290) =$2290


Worst case scenario, this setup delivers at LEAST 275 MB/sec reads, 290 MB/sec writes all the way to the end of the disk. That's enough for ANY flavor of high defintion video, even 10 bit per channel 1920x1080 60 field per second 4:4:4 RGB, which is only possible on $100,000+ cameras and decks. Even to the 80% mark of the disk, you'd still get at least 345 MB/sec reads and writes. Or, it would make a helluva Photoshop scratch disk. : )

What about X-RAID, or othe RAID 3 or 5 or 30 or 50 solutions, such as those from Huge Systems or Medea? They work, they work well, they are well proven...and they are STILL more expensive than this SATA solution I'm suggesting by a substantial margin, even when you factor in the FireWire backup. Huge is over $4.50/GB, Medea is about $5/GB for systems with similar throughputs by MUCH better features - expandability, scalability, reliability, etc. When it comes to Apple. Huge, or Medea, if you can afford'em, buy'em. If you can't, look into this stuff.

A properly cooled and set up 8x300 disk array formats to about 2220 GB of usable space for about $2750. That's about $1.13/GB. Buy two Bigger Disks and another 250 GB FireWire drive all from La Cie, you've added $2290 to your costs, but are still looking at only $2.27/GB for Mike's Recommended Solution For Indies.(That's me.)

Apple's 1.75TB X-RAID, which has LOTS more flexibility, scalability, reliability, stability, and general goodness, costs $7500. That's over $4.50/GB. Lots of good arguments can be made about how it's worth the money, and I'd agreee with them...for those who can afford it.

For the indie filmmaker, that can be out of reach. Indies typically have more time and greater risk tolerance of down time than a well funded production facility, which is why I'd suggest this SATA with FireWire backup route.

For more info on how to put together a great low cost HD workstation, see here, here, here, here and here. Or keep checking back with this site, HD For Indies as I continue to write about high definition video, digital cinema, and how to get it all done at minimal cost.

If you'd like to keep regular tabs on these issues, check out my Atom and RSS feed links at the top right corner of this page, and use an RSS aggregator such as NetNewsWire or somesuch.

Also of interest: perfect pixel for pixel monitoring of your HD signal for as little as $1400 (for HD 720p work) or $2700 (for HD 1080p or 1080i resolution work) - read up on HDLink

DISCLAIMER: This is my 3rd day or working with the RocketRAID card, I don't really advocate it YET for production usage. I think it shows a lot of promise, but has yet to prove itself in a production environment. For instance, in the middle of duplicating a 5 GB file, I had a kernel panic - the bad kind where a grey scrim lowers, looking like a curtain coming down, and it states in several languages onscreen that it's time to shut down. Eeeeeeeeeeeew. I can't remember the last time I got one of those, so take all this drive stuff with a grain of salt until the drivers and card prove stable and reliable. This is, after all, a 1.0 driver we're talking about...

-mike

Mike Curtis is a digital post production supervisor in Austin, Texas available for consulting on your HD projects. If you have an HD project you are contemplating shooting, talk to him about being a test site for some new equipment under development. Email mike at hdforindies.com

So today I called my local Apple Store to ask them about something, and it turns out they had Dual 2.5 GHz G5s in stock. Wow. Apple's online store estimates 3-5 weeks for those to ship, and their online quotes are "optimistic" at best in my experience, ESPECIALLY when dealing with recently released products.

So before the phone had clattered to the floor, I was whipping down the highway to go get me one.

Once I got there, I talked to their business sales guy, and he knocked several points off of the selling price since I bought AppleCare and ProCare for the unit. You could argue that extended warranties aren't worth it, but for this unit, for the purposes I'm going to use it for, I felt it was well worth the investment.

Also, for those looking to save some money on your editing setup, if you buy it with your G5 purchase, you can purchase Final Cut Express for $99, which is upgradeable to Final Cut Pro for $699, which saves you $200. Yes it takes longer, but it's $200.

I'll be hooking up a stupendous array (16 drives) and seeing how fast I can Make It Go just for fun in the next few days.

-mike

While researching for another article (to be posted) I came across this on the Sony site - a 1920x1080 resolution high defintion video camera, "block" style - meaning it's meant for industrial mounting, not for field shooting. It has no handle, it has no microphone or mike jacks, it lacks almost all controls you'd expect to find on a "real" videocamera.

And yet....the idea is intriguing. It has three 1/2" 1.5 megapixel CCD sensors, and it comes with a lense. I'm curious if a ravingly desperate indie might be able to make use of this somehow to shoot an HD feature/short/commercial by strapping a handle onto it (duct tape, anyone?) recording to an HD deck of some flavor. This unit does support 24psf, so you can shoot a film-like 24 progressive frames per second. It also has an HD-SDI output, so that's cool, that's the industry standard high definition connector to use. Not really production viable, but it's a beguiling idea.

Or it might be the ideal "lowest possible cost high quality" camera for certain tasks like strapping onto a helmet during some Extreme Sporting Activity with a wire running to a battery powered SRW-1 deck (with EXCELLENT production insurance). Again not really, but a fun idea.

Anyway, it'd be fun to have one of these to doodle around with and see what the image quality was, how much control can be exerted over the camera, what the exposure latitude is, etc. etc.

It would be a good "lab mule" to play with, since it does have an HD-SDI output.


Anybody played around with one of these, in whatever capacity? Drop me a line at mike@hdforindies.com if you have, I'd love to hear whatever you can tell me.

-mike

It hasn't shipped yet, nor has a price been determined, but Flip4Mac is saying they'll be able to encode Windows Media 9 stuff from a Mac based application. This is a nice piece of the puzzle, since the two competing formats for high definition DVDs will include a Windows Media 9 based encoding scheme as a requirement of the playback devices. Plus, as much as I don't admire Microsoft for their business practices, they've made a really, really nice and impressive codec with WM9.

From their website:

-Import or Export components for Windows Media Video 7, 8 and 9,
-Windows Media Audio Standard, Professional and Lossless

Standard version

-single-pass video encoding
-constant (CBR) and variable (VBR) bit rates
-up to 48 kHz audio sampling rates

Pro HD version adds

-two-pass video encoding
-up to HD video resolutions
-5.1 channel audio (WMA 9 Professional)
-up to 96 kHz audio sampling rates (WMA 9 Professional)

The Pro version will obviously be the one of interest to HD afficionados.

Note also the ability to import Windows Media 7, 8, or 9 assets - this has long been a thorn in my side when trying to work with client provided video files. Windows Media has often been targeted as a delivery codec, not a working codec. Most QuickTime codecs can be edited and worked with in the standard video manipulation applications. This hasn't always been the case with Windows Media files, even on Windows computers. But I haven't had to deal with Windows Media files in a year or two, so I'm out of touch.

Note this doesn't promise compatibility with the upcoming HD-DVD and Blu-Ray specs - just WM9 compatibility. I'd imagine they might either upgrade the software to compatibility with those specifications, or offer another product supporting that capability. Either way, there exists the possibility of it costing more. So if you want to author for high defintion DVDs (which are over a year away from shipping, anyway), this isn't a guaranteed solution yet.

Then again, this software isn't even shipping yet. Click the link to sign up for more info if you wish, or keep reading this site.

-mike

After not being able to find pricing information online about the new Medea VideoRAID XTRM I contacted Medea, and they got back to me. I talked to them about this as yet unreleased Fiber Channel interfaced, SATA drive product that offers RAID 3 data protection.

Here's some details:

-the VideoRAID XTRM connects to a Mac or PC via a 1GB Fiber Channel connection

-Fibre Channel 2GB interface - their card or Apple's? I think there's a choice involved

-15 drives in device, configured in 3 separate RAID 3's of 5 drives each

-Each block of 5 looks like 1 logical drive unit to controller

-Single controller, 3TB of usable space, redundant power supplies, RAID 3 redundancies, redundant fans

-controller is NOT redundant (only one controller)

-2GB interface, 3 groups of 5 drives. At this time, you cannot stripe the 3 groups together into one bigger, faster unit. That limits the sustainable transfer speed to about 125 MB/sec. Thus this device is good for HDCAM 1080p24 (8 bit 4:2:2) footage, but is cutting it closer than I'd like to see for 1080i60 footage or 10 bit 1080p24 footage (120ish MB/sec, too close to 125 for safety's sake).

-expect a 3TB device to be roughly $10K

-it's using SATA drives internally

Mike's Comments:

The Good News: It's 3 individual RAID 3 devices - this means that in each group of 5 drives, one drive is dedicated to storing parity data for the other 4 drives. This means that if ANY one drive in the group of 5 fails, your data can be retrieved via a rebuild process. If two drives in the group fail, "is not your day." The price performance will be pretty good - if the device has 3 TB or RAID 3 space, that means 15 250GB drives, therefore 4 drives times 250 GB/drive times 3 groups of drives is 3TB. That's about $3.33/GB. That's very good for RAID 3 storage - better than my other price performance leader, the fully populated Apple X-RAID at about $3.50/GB. As this is a Fiber Channel device, lots more of these can be hooked up via a Fiber Channel hub.

The Bad News: The 3 groups of 5 drives can't be striped together to work faster than 125 MB/sec (1 gigabit/sec) due to the limitations of the current controller. If you're doing 8 bit HDCAM 1080p24 footage (8 bit 4:2:2 1920x1080 at 23.98, 24, or 25 progressive frames per second) or PAL timed 1080i50 (50 fields per second, same size, still 4:2:2) this is a very nice fit for a single stream solution.

However, if you want to work with 10 bit 4:2:2 1080p24, or NTSC timed (59.94 fields per second), or any 4:4:4 footage, this product is NOT acceptable. I'd imagine Medea is quite aware of this and probably working on a solution to fix this.

So if you want to do any 1080 res footage in 10 bit, or 1080i60 (really 59.94), or any 4:4:4 1080 res footage, I'd say DON'T buy this product at this time until it is modified to be acceptable for those applications.

As a general rule, I try NOT to buy equipment on the future promise it'll do something. I buy it based on what it does TODAY. All too often future hoped for features either never come about or are integrated into a new replacement product that you can't upgrade your current product to match.

-mike

Robert over at BareFeats has once again posted a very useful article, this time doing a head to head comparison between a dual G5s, Dual Xeon, Dual Opteron, and a Pentium 4. They tested Photoshop, After Effects, Bryce, and Cinebench (a 3d rendering benchmarking application).

Mike's Comments: Results: The G5 did suprisingly well, winning many of the tests and blowing the doors off of the competition in After Effects by a wide margin. Then again, it all depends on exactly what you test for - your mileage may vary depending on the kind of work you do. Prepress Photoshop needs are different from an artist's Photoshop needs, and an After Effects motion graphics animator may need completely different portions of code used than a compositor/visual effects artist. Mileage varies.

Now that I'm getting a high speed (hopefully) stable testing platform established for HD work, I'll be testing and documenting a variety of workflows and solutions, and stopwatching them all. I'll post results as I get them.

-mike

I've been asked this more than a couple of times, so I'll answer it here. A reader wrote in and asked:

Why would you use the HD link instead of just the graphics card output and
then FCP HD video monitoring with the HD Cinema display?

What are the advantages?


I get asked this one a lot -

1.) HDLink allows for a true interlaced display
2.) HDLink shows a true video signal, not an onscreen representation
3.) Fullscreen mode in FCP can choke on 1080 res footage, even on dual 2.0 GHz G5
4.) The Biggie - if using DVCPRO HD, what you see on screen is a preview approximation of the HD signal - you aren't seeing all the detail of the signal. Try messing with Low/Medium/High quality preview and you'll see what I mean.
5.) High Quality won't play back full speed often
6.) Even if it plays back fullscreen in high quality, there are playback speed inconsistencies in FCP fullscreen mode that don't occur with HDLink
7.) HDLink can be calibrated in a more controlled fashion.
8.) And oh yeah - HDLink maps your video pixel for pixel to the display instead of stretching it up or down. While you can set FCP to show 1:1, it still isn't as good a representation.
9.) and oh yeah! The way colors look on uncalibrated computer monitor (even 23" LCD) vs. the gamma corrections (and calibration possible) with the HDLink are SIGNIFICANTLY different - would throw your color work waaaaaaaaaaay off.
etc. etc.

I break it down like this - the FCP way is a preview of your footage. The HDLink way is a much more accurate representation of your footage.

In terms of budget, I'd break it down like this:

1.) If you can afford it, work with both a SMPTE C studio grade calibrated HD monitor (for color accuracy) and an HDLink with Cinema Display (for seeing fine detail)

2.) If you can't afford that, work primarily with HDLink with Cinema Display, then rent a SMPTE C studio HD calibrated monitor at the end of your post cycle to do your critical color correction work.

3.) If you can't afford that, forgo the monitor rental and the the best you can with HDLink.

4.) If you can't afford that, use Digital Cinema Desktop Preview (or whatever they call it) and rent a monitor at the end of your post cycle.

5.) If you can't afford that, work with Digital Cinema Desktop Preview and suffer the consequences. (Your color corrections will be waaaaaaaay off!)

-mike

Robert at BareFeats has posted a very nice and thorough article on 1820A RAID setups on a Mac G5

Robert at barefeats.com and I have been discussing the 1820A card, he emailed me this nice summary of the pros & cons:

FEATURES or advantages

You have room to grow with 8 ports
It captures 10 bit uncompressed HDTV with 4 or more drives in a RAID 0 set.
The included RAIDMan software supports RAID 0, 1, 5, 10 and JBOD.
It's compatible with both SSC and non-SSC drives.
The kit includes 8 SATA internal cables.
It runs on G4 and G5 Power Macs under OS X 10.2 or newer.
It's an affordable solution ($204 at ZipZoomFly)

"NON-FEATURES" or deficiencies

None of the 8 ports were external.
You can't boot your Mac from any drive connected to the RocketRAID.
There's no hot-swap support. You must Shutdown your Mac to connect/disconnect.
It does not support Deep Sleep
It "hits the wall" at 340MB/s WRITE speed, with 5 or more drives in RAID 0 set.
Their RAIDMan utility runs slower than Apple's RAID software and SoftRAID.
It lacks support for G3 Power Macs with PCI slots
There's no support for Macs running OS 8, 9 or 10.1
The plastic connectors break easily. (This can be an issue especially if heavily shielded external use data cables are routed out an empty PCI slot to external SATA boxes.)
Cables provided do not have proper shielding for external use.

Mike adds:

Cons: it is a 1.0 driver - I've already had a kernel panic
I'm using SoftRAID to partition and format (Disk Utility will format but not partition an array) since it gets better performance than the included RAIDman software
Have to use RAID 0 for HD work, and that creates data vulnerability
RAID 10 works, but is constrained to about 180 MB/sec write speed
RAID 5 - I haven't been able to get it to work at all, perhaps due to all the various drivers I've installed over time. They say it works and others have used it successfully without event
RAIDman software - funky/clunky to set up, and is slower than Apple Disk Utility or SoftRAID
Two card installs don't run much faster than 1 card installs - clearly the card is the bottleneck, not the drives
Two card installs are limited to 2TB per card at the moment due to technical reasons

MacWorld has a review of Discreet Combustion 3, a sub-$1000 compositing and visual effects program.

As a 10 year After Effects veteran (back from when it was CoSA), I've sat down to learn combustion not once but twice....and walked away from it as "gosh that's hard" after an hour or two. I really do need to learn it...someday.

It is very powerful, has great keyers, grain management, floating point support, particles, etc., but it is highly non-Maclike.

The good news is that if you master it, you now grok the basis of all the higher end Discreet products and can hand off projects to those higher end products as well with seamless integration.

I'd break it down like this: If you are looking to do design heavy motion graphics, use After Effects.

If you are looking to do basic effects work, either After Effects or Combustion will work.

If you want to get fancy/serious/high end with your effects work, Combustion can do things After Effects can't.

-mike

According to this MacWorld article, Sony is developing a 200GB version of it's Blu-Ray disc technology.

It uses 8 layers to store up to 200GB, but Sony made no formal announcement that it would become a product, at this time it is merely a product demonstation.

Sony also announced that standard (not 200GB) Blu-Ray disc playback will be incorporated into the PlayStation 3. This should help push adoption.

Mike says: In case I didn't mention it, about three weeks ago the Blu-Ray group added MPEG-4 AVC (Advanced Video Codec) and VC-1 (the Microsoft Windows Media 9 based codec) to their high definition DVD specification to match the codec (compressor/decompressor) choices that the competing HD-DVD specification offered. These two competing formats are vying to be the standard that Hollywood studios might pick to provide high definition versions of movies on disc.

Sony also purchased MGM Studios (the movie company. The Big One. Yeah, I know!) recently, so this gives them the leverage to release movies on the Blu-Ray format.

But this creates a possible schism - Sony/MGM discs might be released on Blu-Ray, but other studios may go with HD-DVD, with it's "good enough" capacity and much lower manufacturing costs. Disc manufacturing facilities can be retooled to produce HD-DVD and regular DVDs with a very brief (coffee break duration) setup. And in business, cheap wins. Always.

Hey! They did it! Let me look at my watch - FIVE MONTHS after FCP 4.5 ships. This is why I say don't bother with Digital Voodoo - they are slooooooow in development, overpriced, and in general just out of contention these days.

I sincerely and forcefully do NOT recommend their products.

If you have one of their products, you have my sympathy, and can read the full article here.

From Apple's site:

If Final Cut Pro HD unexpectedly quits on you after using Software Update, a few short steps should fix it up.

Warning: This document describes how to enter commands in the Terminal application. Users unfamiliar with Terminal and UNIX-style environments should proceed with caution. The entry of incorrect commands may result in data loss and/or unusable system software.


Save any files or work you may have open, because you'll need to restart the computer after these steps.

Open Terminal (found at /Applications/Utilities/).

Type
sudo update_prebinding -root / -force


Press Return.

Enter your password when prompted.

Restart your computer.


Final Cut Pro HD should now open normally.

This document will be updated when more information becomes available.

UPDATED TUESDAY - SEE BOTTOM
ProMax, a longtime desktop video systems integrator has a new product of interest to those wanting to put together their own uncompressed HD editing system. The ProMax 1.8TB External Array SATAMAXe-HD for G5 is a $3295 1.8 terabyte RAID 0 that comes with (I believe) a Highpoint RocketRAID 1820A card and two external 4 bay hard drive enclosures.

That's $1.83/GB, including the host card. Nice!

From their website:

The SATAMAXe-HD is our most economical high definition storage solution yet. It brings RAID 0 performance with 3 hours of 10 bit uncompressed High Definition video for under thirty-five hundred dollars. This 1.8 TB of usable storage allows you to fill the drives up almost completely and still get performance to keep outputting even as the client keeps adding changes right up to air time.



I think what the have here is 8 250 GB drives, with an actual formatted, usable capacity of 1.8TB. I think. I'm not sure.

The card they have pictured looks similar to the two 1820A cards I have at my studio at the moment - similar but not the same, since the SATA port layout is different on that card as compared to mine.

I'm also curious how they are solving the various cabling issues -

Are they running SATA cables out of any empty PCI slot cover?

If so, how are they anchoring the cables?

How are they managing the cables inside the case without making a huge mess?

Also, what formatting software are they using? Is SoftRAID included?

I'm working on the answers to this, when you see on your RSS or Atom feed that this has been updated, it'll be because I have more answers.

In comparison, the 8x300GB Burly Box solution I'm working on building will hold 2.2 usable terabytes, and costs $2704, for a cost per gigabyte of $1.21. However expect to spend 3-4 hours to install drives, run cabling, and physically set it up. As opposed to and hour or more with the ProMax solution to just install the card and connect SATA cables. Plus theirs has a warranty and tech support department, the Burly solution does not - you'd have to talk to each component manufacturer's support folks, and they could easily all finger point at the others. With ProMax, you could call up and say "It don't work." and it's their job to figure it out.

I'd expect to see 2.2TB and 3.0TB versions of this array (with 300 and 400GB drives) in the future. The 2.2TB will cost I'll bet $300-$500 more, but the 3.0TB will be much more expensive, since the drives are so much more. It'd cost at least $2000 more. Not very cost effective...but big.

-mike

UPDATED TUESDAY AFTERNOON

Talked to Jerry Miles over at ProMax, he gave me the scoop on the setup:

-How fast is it? With an EMPTY DISK - 418 MB/sec reads, 330-340 MB/sec WRITES (and that was with 4 Hitachi and 4 Western Digital drives, what he had sitting around at the time)

-it is using the Highpoint RocketRAID 1820a card

-they are routing their cables out the back of the card, they've punched some holes in it for routing (be sure to anchor those somehow!!! A yanked cable would be disastrous!)

-they did some serious stress testing on the array to try worst case scenarios - capturing footage until the array was full, rendering transitions, forcing the drives to read from opposite ends of the disks in rapid succession (think about playing notes from song 1 and song 15 on a CD in rapid intercut succession)

-using Hitachi 7200 drives, probably 7K250 model, definitely 250GB

-they've had good history and success with IBM/Hitachi drives, they will stick with them

-using Apple's Disk Utility to format, NOT the RAIDman software that comes with the Mac driver package

-the 4 port Firmtek card that is coming out has problems with drives with SSC (the Hitachi 7K250 and 7K400 models), this card does not

-THE CARD DOESN'T SUPPORT DEEP SLEEP

Meanwhile, I'm building up my own 8 drive SATA RAID, and it's giving me trouble to the point of having to check each component (cards, cables, drives, chassis) one by one to find out where the problem/s are. Makes the idea of "here's a check, gimme!" much more appealing.

-mike, Tuesday afternoon

No news posts this weekend, even though I have a post-IBC backlog of about 20 things I want to cover -- between building up 3 different 4 bay SATA drive chassis for testing to begin this week (insert evil "bwahahahahaaaaa.....") and the Austin City Limits Festival, I'm beat (and burnt! 6 hours in Texas sun...).

I was a bit peeved to discover my Burly Box from MacGurus (4 bay SATA hard drive chassis, too tired to link, see prior articles) didn't actually come with SATA power connectors, only Molex. No clue on the web page about that. While many SATA drives have dual power (both Molex and SATA power plugs), not all do, especially the newer ones, like the DiamondMax 10 and Maxline III 300GB drives from Maxtor. The good news is they do have these adaptors and I've ordered them, but it'll probably be sometime between Monday and Wednesday before I receive them.

And when they say drive KIT, they aren't kidding - even after having done it before, I think it'll take me an hour to an hour and a half to get a 4 drive unit fully built up and cabled with drive coolers for each drive.

The Burly Box needs to have the Centronics plates removed (2 screws) and replaced with one compatible with SATA port mounts (2 screws). Then a SATA port cable needs to be anchored (2 screws). Each drive to be mounted in a drive cooler or drive carrier needs 4 screws (whichever you choose, I'd advocate the coolers - heat is the #1 cause of drive death). The drive coolers need their own power connection, the drives need power and data connections...you get the idea. The one nice thing about this arrangement is since it isn't SCSI, there are no SCSI IDs to set and no termination to worry about. Thank goodness. But this comes at the hassle of one cable per drive.

I also managed to get my new Highpoint RocketRAID 1820A card (8 port PCI-X Serial ATA card) installed in my dual 2.0GHz G5 - it's an octopus, literally. Cable management and routing took a while to figure out, and it's a big gob of zipties and labelled cables. I'll take some pictures and post a link on how to get this monster set up properly (well, at least as properly as I got it). It's a bear, since the cables all have to go out the one empty PCI slot cover, AND need to not make contact with DeckLink HD Pro 4:4:4 card sitting in slot 2. The cables also need to be anchored, so that they won't come loose from the ports on the PCI card in general, and won't yank/break anything inside the chassis should the cable get yanked or tripped on. My solution of the moment - a great ugly gob of zipties going through the mesh holes in the back of the G5, and then zip tie anchoring pairs of cables to the prior pair of cables.

If you want to try to get two 1820A cards and an HD capture card in a system, you've got to start cutting/punching holes in the case. Definitely save that one for the advanced class.

So I realize I can't start testing tomorrow - I'm stuck waiting on the power adaptor cables - grrrr.....might have to go to Fry's and see if I can find a temp solution.

-mike

Just saw Sky Captain and the World of Tomorrow this afternoon at a matinee showing (yes, I have no "real" job where I go to an office and sit in a cube. Thank f***iing god...). If you're any kind of geek like I am, you know that this is the first largescale movie with top tier talent made 99% by shooting actors against bluescreen. A typewriter, just the fuselage of a plane, and a few other minimal props were used throughout the movie. It was also shot entirely on HD video (8 bit HDCAM I believe) and almost all the digital effects work and animation was done on desktoop Macs & PCs. It was also edited on Final Cut Pro, which I do so dearly love. So how well did it work? It was good, but not. Read on...

Here's what I just wrote to some film geek friends of mine:

Wow - great art direction and look, sucky sucky acting, bizarro lift-o-matic script, and inconsistent effects work.

Can the acting and pacing be any flatter? Don't think so.

There are glimpses of fun stuff in there, but there's no chemistry between the leads. Jude Law's character so cries out for the pluck, easygoing confidence and attitude of Indiana Jones or Han Solo...and so doesn't have it. I liked the last line of the film, I can see what Conran was TRYING to do, but it just didn't have heart in the end. First time director working with big name talent, and the talent doesn't know what they're doing on that big empty set.

This is the kind of job that requires, you know - ACTING! Pretending to be somewhere, doing something, feel something. And this ain't it.

The world they've created is fantastic - the director certainly deserved kudos for his vision of this fantastic universe. From a world creation standpoint, he gets an A+. The opening sequence of the Hindenburg III coming in to dock with the Empire State Building is beautiful The look they've created digitally is just beautiful, with a lot of highlight haloing and muted, desaturated colors. The concept artists had a blast with the robots, the virtual sets (no construction budget - previsualization leads directly into set production!), the costumes (the only consistently real props) and the art direction are lovely to behold.

The acting and direction, however, especially during the first hour or so, is totally flat. It's like they were figuring it out during shooting and finally got a clue about 2/3 of the way through.

The acting seems flat and dissafected - quite possibly because they were standing against bluescreen and had NOTHING to react to. Or was this in part a bit of the 30s & 40s serial feel he was trying to create? If so, I think inappropriate for a modern audience. Whatever the reason, the action in the first part of the film feels flat and affected. Perhaps it was the inexperience of the director and the team, or the technical constraints of the full bluescreen shoot, but the action and drama feels not very active and not very dramatic. When Polly Perkins (played by Gwynneth Paltrow) runs from the giant robots, it feels totally lame - she's not running very fast, she doesn't look or act very afraid. When she and Sky Captain (Jude Law) are flying down NYC streets in a P40 fighter plane and turning upside down and hanging a left at 20 feet, they don't look very concerned or intense at all. It totally doesn't feel right. Maybe I'm not getting it and that's the intended feel of the old serial films, but it comes across lame in my book.

This may be an ongoing problem with virtual set movies. Perhaps if this doesn't do well at the box office Hollywood will say "Gee, that didn't work. We're not doing that again." I think this would be a mistake. My gut says this film will not do great at the box office (hope it does, though). It appeals to a niche audience that appreciates this truly Old School look at leitmotif, not the broadest general moviegoing audience. I think critics and film buffs will like the look, but general audiences may say "Feh." and see Alien vs Predator of somesuch. Oh, the horror...

(BTW, what do you call the predator/alien at the end? Predalien? Where's the fanboy art on THAT? And what do you call the babies? Pralines?)

Reel me back in, I ramble all too readily. ADD? Just a little bit. See?

OK - I think it would be a mistake to write off this filmmaking methodolgy. Because of the minimal to non-existent sets, there are technical challenges to making complicated shots. Locked off shots are much easier. You have to have a bluescreen set big enough to capture all the action. Actors have nothing to react to or act against or act with. But come on, folks, this is really the first one. (The first VALID one. Able Edwards doesn't count). Give this new style a chance. Kerry Conran was the writer director, and all this came from a very small, indie production group. And it shows.

While I laud them for what they pulled off on a small budget (I'm betting Jude Law, Gwynneth Paltrow, Angelina Jolie and Giovanni Ribisi's fees were a huge chunk of the overall budget), it still looks small budget.

The effects shots are spotty - some are quite good, others feel somewhere north of a SciFi Original, but south of a top tier feature. I've noticed this trend in too many movies to mention the last couple of years - big Hollywood pictures are so rushed to get out they farm shots all over the place, and not everyone delivers the same quality (nor should they - farm out shots to the folks talented enough to get them done, but don't overpay - why pay ILM rates for just wire removal?).

Overall, I'd say the movie suffers from being bogged down by it's twin "tech" words - technology and technique. Too many locked off shots, not enough flair and vision in the cinematography (although there's tons of good conceptual stuff). Too many locked off shots, too much under/bad acting when it should really be roaring.

Perhaps the next film to get made more in this style will have a more experienced director at the helm, someone better with actors.

Plus, of couse, the plot has a ton of issues. But I'll leave that alone for now. There's no "make them act better" menu in an editing program, nor a "make the plot make sense" button.

OK, it's late, I'm off to bed.

The folks over at CamcorderInfo have this very long, very detailed review of the HDR-FX1 camera. Some highlights:


-it DOES have an audio line in, it just isn't XLR-but you can use an adaptor from Beachtek.com (or similar) to resolve that problem.
-the CineFrame 24p mode uses a 2:3:2:3 cadence, and should work pretty well for indie filmmaker types. YES!
-lense is fixed. You better like it
-it DOES have component analog video output. Monitoring solved!
-LOTS of physical controls, Sony's moved away from menu driven stuff for this camera
-3 ways of controlling the manual zoom
-250,000 pixel viewfinder (shows lots of detail, better for critical HD focusing. But use a monitor if shooting a "movie")
-low light performance: Sony says minimum lux for the HDR-FX1 is 3, and "falls between their DCR-VX1000 (with a minimum lux rating of 1) and their DCR-HC1000 (with a minimum lux rating of 5)"
-2 channels of audio. That's it.
-fully native 16:9: "native 16:9 CCDs, viewfinder, and LCD screen"
-Reviewer likes the HDR-FX1 better than the JVC JY-HD10 HDV camera or the Panasonic AG-DVX100 or the Canon XL-2. -optional shoulder mount/brace around $400

Mike says: This is the camera to get for indie digital moviemaking. Period.

Read the full review, tons of great details and photos, this is just a quick skim.

So I continue to doodle with low cost HD solutions. There are some new products, new configurations, and some new analysis in this posting.

If you wanted to put together a system capable of letting you produce a moderately sized uncompressed HD work, including high quality monitoring, it could be done for as little as what follows.

Sufficent for a SHORT work (final TRT an hour or more? Depends on how much rendering done!), using offline/online codecs:

Dual 2.0 GHz G5: $2500
more RAM: $200 (3rd party, of course! Crucial.com or whatever)
HDLink for HD previewing: $700
Apple 23" LCD: $2000 (ouch, expensive in this package)
-OR- get the HP 23" LCD for $1600 or so...not sure this is gonna work, but interesting
DeckLink HD: $600
two cheapo PPA, Inc. external SATA cases: $50/ea, $100 (Fry's electronics)
4 Maxtor Maxline III 300GB drives: 4x$230=$920
1 FireWire 800 drive to boot from - $200
1 decent 19" monitor: $240 (such as Sonic G90f, can do up to 1920x1440 res)
Final Cut Pro HD: $1000 ($400 education price, isn't it?)
SoftRAID formatting software (yeah, you DO need it): $100
La Cie 500GB FireWire drive: $500 (for offline codec storage & work)

TOTAL PRICE: $8660

This is the rock bottom, basement floor kind of a system I'd even consider using.

If it were me, I'd add the following:

another 19" CRT: $240
Matias tactile pro keyboard: $100
ShuttlePro v2 jog/shuttle: $100
yet more RAM: $200 or more
use a RocketRAID 1820A instead: $100 more
drop the cheap cases, get a 4 bay Burly Box from MacGurus as well as a 2 bay model - add $235
buy two more drives - $460
...and you stll have 2 unused slots on that 8 port SATA card for more drives.

Now you have a nice 3 monitor system, with 1.8TB of RAID 0 space (back that data up!)

Total price of improved system: just over $10,095

Oh, excuse me, gotta use marketable numbers. Drop the keyboard:

Now it's $9,995

This is the thiiiiiiiiiiin solution for a number of reasons - You might get away with this....or you could face disaster if a drive ate a bearing during your production. You'd have to recapture ALL your footage at the very least.

Further tweaks - you could cost effectively bump up to a second 4 port Burly Box for only $100 more, I'd want to install the drive coolers to ensure drive longevity (add $30/drive mechanism), and I'd want backup for all my data - cheapest way would be 500GB La Cie FireWire 800 drives - you'd need 4 or 5 of them depending on array capacity. 5x$500 each is $2500 more.

(For the Ultimate Big Kahuna Client Impressive recommendation, see here.)

It's a total roll of the dice - you never know when something might go wrong. I've been running a 4 drive Barracuda 7200.7 array for over 4 months without any trouble. (Seagate 160GB drives, the model Apple shipped standard in Rev A dual 2.0 GHz G5s) But a drive could fail whenever, unexpectedly, and I'd lose all the data on it. But I'm covered - the crucial client project footage is backed up, and only some test render stuff is stored on the array not backed up. If a drive shagged, I'd be irritated and dissapointed, but not freaking out.

...such as my friend the editor did the other week, when on deadline for a major cable network project, the drive with ALL the audio decided to give up the ghost. It took them about 30-60 hours of labor to go back and recapture from DV all that stuff. And since somebody in the workflow had decided not to bother with timecode on some converted Digibeta footage, they had to hand match a ton of stuff. UUUUUUUUUUURRRRRRRGH. Pain...and that was only a single FireWire drive, not part of any multi-terabyte array.


Another note: as I chug through my ever expanding Excel sheet with all this data, I'm realizing that the bigger Hitachi 7K400 400GB drives are incredibly cost ineffective - they literally cost TWICE as much as the Maxtor Maxline III drives, but only provide 1/3 more space. Plus they also are slower, especially towards the end of the drive where all drives get slower. This can limit the kinds of footage you can safely work with to the maximum capacity of your disk array (especially in 4 drive configurations).

If you're trying to configure a large, fast array, the ONLY reason to choose the 7K400 drives over the Maxtors is if you need more than 2.4TB of space on that 8 port card. The Hitachis are just very cost ineffective. If I needed 3.6TB, I'd strongly consider the possibility of building 2 6 drive arrays (3 4 bay enclosures) attached to two separate 1820A cards. But then I'd have to literally cut a hole in the back of my G5 to get the cables out since their wouldn't be an empty PCI slot to sneak the cables out of.

Feh. My warranty expires in the next couple of weeks on my dual 2.0 GHz G5, anyway....

; )

-mike

Firmtek has finally wised up and released a version of their two port SATA card with two external SATA ports, so it is no longer necessary to thread the SATA cables from the internal (inside the G5 chassis) ports through an open PCI chassis in order to plug in external drives. For those wanting to assemble a 4 drive array on the cheap, this makes life much simpler and easier.

Other features:
-can hot swap drives
-no driver, just plug in and go (card and drives)
-can do RAID 0 or 1
-support for large drives

Available Q4 2004, $99 suggested price. The prior model with internal ports (Seritek 1S2) sells for about $65 online.

This is good news for those who want to do short form, uncompressed HD work.

This card, external casing for two drives (such as from PPA, Inc., Granite Digital, or MacGurus), two internal drives, SoftRAID 3.x for formatting, and an external FireWire drive to boot from is the lowest cost solution I'm aware of that allows for uncompressed HD capture.

With drives such as the Maxtor Maxline III 300GB, a 4 drive array could capture 1080i60 8 bit 4:2:2 uncompressed HD footage (such as HDCAM) for most of the capacity of the array (drives get slower as they fill and start writing data to the slower inner tracks of the platters).

Such an array could also handle 10 bit 4:2:2 1080p24 footage until the array was almost full.

For HDCAM SR RGB 4:4:4 10 bit footage, some amount of footage could be captured and played back, but probably only the first 1/3 or less of the array could handle it - thereafter you'd start dropping frames.

But you get the idea - cheap speed.

-mike

The Highpoint RocketRAID 1820A SATA RAID, that has been popular on the Windows side of the fence, finally got Mac drivers last week.

In my initial write-up, I discussed the possibility of setting up large RAID 5 arrays for HD usage.

Well, not so.

In short, RAID 5 won't work for HD capture but can work for playback, and while it is possible to put two of these cards in a G5 in slots 2 & 3 with an HD card in slot 4, RAID capacity would be limited to 4TB total. "Whaaaaaaaa?" as John Stewart would say. Read on for details.

Because the 1820A relies on the host computer to do all the hard math of RAID 5 implementation, write performance pretty much blows as the G5 struggles to generate parity data on the fly. What's that mean? That means I'm back to my old standby recommendation of Go With RAID 0, But Buy A Bunch Of FireWire 800 Backup So You Don't Lose All Your Data If One Drive Fails.

In the Clever But Desperate category, one could have two arrays - one for capture, one for playback/edit. But then both RAIDs would have to be no more than 2TB each. Bummer. Whaaaaa, again - due to obscure technical limitations, if you have two 1820A cards installed, they both have to be no larger than 2TB each for it to work. Don't ask why, I don't know, seems crazy to me too. If you know why, please let me know. However, with a single card installed you can exceed 2TB no problem.

The read performance should be fine (I'll find out soon enough), but because the G5 has to do all the heavy math of the RAID 5 stuff when writing, and BareFeats is reporting that a 4 drive array only generated 50 MB/sec writes. Whenever you had to render something, it would take longer...or would it?

This from a BareFeats.com article

We ran the G5 using a single, empty Hitachi 7K250 SATA drive (55MB/s) as the FCP scratch volume. Then we used an empty four drive SATA RAID 0 array (220MB/s) as the FCP scratch volume. No change in render speeds. You would have to be using a really, really slow drive for FCP scratch to see render speed affected.

But since the CPU would be doing double duty (RAID 5 parity data generation and rendering of the effects) it would certainly take longer than usual. How much longer? I'll know in a week or so.

Tomorrow I'm going to build a 2.4TB array and start messing with it and testing it. I am returning my demo loaner DeckLink HD Pro card and HDLink, because I liked them so much I bought my own (after the price drop - whew!).

I'll post test results and notes as I go.

-mike

Reader Maarten Somers is attending IBC and wrote in with this report:

- Matrox HD solution: is not coming as a turnkey solution. They just sell
the video card and breakout box. The card remains the same for SD or HD,
just changing the breakout box will give you HD.

Prices: the SD version is about 5000 euros, the HD is about 10000 euros (yep pretty
expensive) and -in the contrary what you said in your blog- it's WITHOUT the
pc-system!

- I've been mailing with Jeff Kreines and the Kinetta Camera IS coming.
You can expect production units around beginning 2005. They will probably
even have 72 frames/seconds.

- The Blackmagic booth at IBC was completely run over! It was incredible
crowded... I think it says enough about their success.

- Nucoda grading solution was incredible. Those guys have a great working
product out there that can definitely be used for feature films. It runs on
a regular PC with Windows XP and is rock solid! The price tag however is
about 150.000€! Really expensive, but I have no knowledge what a lustre
would cost.


Mike says:

So eek - the Matrox HD stuff sounds very powerful, but very expensive. I'd want to see a side by side feature analysis to see if it cost justifies. For broadcast markets this may well be the thing, but for indies it seems like feature overkill. How many spinning chrome 3D logos do YOU intend to use in your sensitive tale of a boy coming of age?

In a prior email, I had stated that the Kinetta camera was vaporware and not something to plan on using. I think Jeff Kreines has a great idea, and I'm just waiting to see if it can actually come to market. I have high hopes for it. But it isn't shipping yet, so I don't consider it a viable option at this time. Hopefully he'll have shipping, field tested product by next NAB.

BlackMagic continues to rock the house with their new products and pricing. See breakdown and analysis of my discussion on their new stuff. Their 4:2:2 DeckLink HD Pro should work very well for the needs of the vast majority of the market.

Nucoda - yeah, I hear it's really great and an amazing new low price point for 2K film color correction work. Only $150,000!!!!

.....eek.

-mike

RaveHD, maker of Linux based, codec agnostic DDR, has updated their product with a new user interface.

Unlike other hard disk drive based digital disk recorders, RaveHD is based on open standards, standardized hardware, and Linux.

Read about it here.

UPDATE FRIDAY

I hadn't noticed that they'd dropped the price for the software only version from $8000 to $5000.

The system is really geared towards VFX workflows, not editing workflows. But the more I learn about it, the more I like it (in theory).

This is from a press release I received today:

RaveHD, used by some of the top studios in the world is the first Linux based disk drive recorder to open it's source code to it's clients. Priced at $4,995.00 this software package is geared towards studios who require high throughput of SD, HD and Dual Link formats. Using a standard file system and a frame base concept RaveHD is able to communicate easily with other software packages and to be easily integrated into the pipeline.

Input from Studios such as Tippett has produced a software package that is designed for visual effects and the flexibility required in doing them.

The demonstration will focus on the database functionality of RaveHD, capture, VTR, playback and the new quicktime to frame/frame to Quicktime conversion tool. We will also cover in detail how RaveHD can be used in the workflow. This as a hands-on demonstration and you should leave with a clear understanding of this new product and how it is changing the way studios think about HD.

Don't miss this opportunity to see RaveHD in action and talk with lead programmer Jason Howard. Seating is limited so please RSVP to ramona@spectsoft.com for location details.

Updated Tuesday morning - see bottom

See this prior posting about the HDR-FX1 PAL version some sample footage from it.

I'll regurgitate what I wrote earlier:

I don't know when then plan to ship it, but if I interpret their stated "worldwide by end of year" I'd read that as, well, by the end of the year. So if you shot 1080i50, then did field blending (with Cleaner, After Effects, Magic Bullet, Compression Master, or SOMETHING), you'd have 1080p25. That's cake to conform to 24 fps for theatrical release. It's not perfect, but it beats the hell out of the other $3500 options, such as DV PAL (720x576). Even if you assume that field blending halves the vertical resolution, it's still a true source of 960x540 - worst case scenario that's still 20% more resolution. And in truth it's a lot more, but I'm not sure how to to the math off the top of my head. I know it's at least that much better. If you just dumped a field, it'd be 20% better. But since you're smart field averaging, it's better than that.


And at about the same price, that rocks the house. For now, it looks like sound might be an issue with the HDR-FX1 until the pro version with XLR audio inputs comes out. In the meantime, if the FX-1 lacks even minijack inputs, you could (ugh!) record separate audio and sync it in post (suck suck suck - thought we were past that!). Hopefully you'll have some international projects so you can shoot MOS (mit out sound - without sound - no audio).

OK, now THIS is huge for indie filmmakers. Process would be this:

Buy a PAL HDR-FX1

Shoot your footage

Get your footage onto your Mac with Lumiere HD

Somewhere in there, process the footage to do the smartest interpolation your software will do to convert 50 interlaced fields per second into 25 progressive frames per second. Here's the list of potential tools to use off the top of my head: Cleaner, After Effects (and plugins for After Effects: Algolith, Re:Vision's ReTimer or whatever they call it, Magic Bullet Suite HD, etc.), Compression Master, Compressor, DeBabelizer, Shake, or other application that will do an intelligent field merge/blend/average/interpolate/mash. Or if that proves onerous, just post process your final movie at the end with one of the above tools.

Edit away with that new footage. If you need to, use the "Recompress" function in Media Manager in Final Cut Pro HD to create an effective offline codec'd version of your footage for space savings and realtime functionality. DVCPRO HD 1080i50 willl be available on the "next major version" of FCP HD, I'm guessing in the first half of next year (either MacWorld in January, or more likely at NAB in April).

When done editing your 1080p25 project (even though the computer may be acting as if it's 1080i50), do the complicated conform to 24p.

How complicated is it? In Final Cut Pro HD, go to the "Tools" menu and select "Conform 25 to 24"

Tough, ain't it?

;p

(shoot me now, I used a smilie in my blog!)

Now that you have a 1080p24 master, it's cake to make a world master of this thing. Use your 1080i50 project (you saved that separately, right?) for PAL markets, then convert your 1080p24 to 1080i60 using a classic telecine style conversion. This is easy with Final Cut and/or a variety of other tools. You can also make a DVD master from this 1080p24 master, but I wouldn't do the straight export from Final Cut Pro timeline (it does a sucky quality downconversion). I'd render it out to separate file and use a tool with better downsampling (Cleaner, Debabelizer, Compresion Master, After Effects etc.) to kick out a 720x480 anamorphic master, then compress MPEG-2 from that.

I predict that in much the same way that PAL DV has been the smart choice for a few years for those seeking to create a theatrical master, PAL HDV will be the new smart, low cost option, but with much better resolution.

Now put that in your indie pipe and smoke it.

-mikey, smug "il putard" of the digi-post world

UPDATE TUESDAY MORNING: I don't recall seeing anywhere that the camera had analog HD monitoring outputs. If it doesn't, how on earth can you accurately focus? The LCD viewfinder? No way! You're stuck going back to "measure to the lense" techniques, using the markings on the ring. And measure to exactly where? Another good question. Focus pulling? Hello?

UPDATE TUESDAY EVENINGAccording to somebody, they claim somewhere on avsforum.com that there was an updated Sony PR bit about analog component outputs....this would help the monitoring issue...although HD-SDI would be better, this'll work.

Scroll to the bottom of the first post, and read the instructions. I felt it would irresponsible to just post a direct link.

NOTE: This footage was from a prototype of the PAL version of the HDR-FX1 camera - so clearly one will be in the works! I don't know when then plan to ship it, but if I interpret their stated "worldwide by end of year" I'd read that as, well, by the end of the year. So if you shot 1080i50, then did field blending (with Cleaner, After Effects, Magic Bullet, Compression Master, or SOMETHING), you'd have 1080p25. That's cake to conform to 24 fps for theatrical release. It's not perfect, but it beats the hell out of the other $3500 options, such as DV PAL (720x576). Even if you assume that field blending halves the vertical resolution, it's still a true source of 960x540 - worst case scenario that's still 20% more resolution. And in truth it's a lot more, but I'm not sure how to to the math off the top of my head. If you just dumped a field, it'd be 20% better. But since you're smart field averaging, it's better than that.

They sampled it down to about 5 megabits/sec (660 kilobytes/sec) Windows Media 9 at 1280x720, but it gives you a starter idea of what the camera can do. This came from some folks at Sony, so it's Official Demo Product, so take that with a grain of salt. I was going to say something cynical about how this might have been color corrected on a Smoke or Flame, and then I read this:

"No color correction was used- footage is straight off the camera."

...and I am all the more impressed.

The shots, especially the white flower in daylight (with no blowout to white) show off the light sensitivity and dynamic range of the camera - and it's impressive.

But even with this downsampled 720 res footage, there is a softness to the focus. I had just earlier today messed with some HDCAM footage that had been captured uncompressed straight from tape that had been shot on a Sony F900 (project I did last summer), and the focal clarity was just stunning. This camera, on the other hand, uses lower quality lenses, and it's CCD only has 960 pixels horizontally (1/2 the final display resolution).

Nevertheless, it beats the pants off any DV camera costing 2-3 times as much you'll use in terms of detail captured to tape - it captures 960x1080 pixels, rather than 720x480. That's 3 1/3 times as many pixels. Especially when the "pro" version ships next year at the $7K price point, this camera will be CHOICE for low budget indie filmmakers. FINALLY, there will be SOME middle ground between the $4000 DV cameras and the $65,000 Varicam, or the 6 figure price tag of a fully configured Sony F900 or F950 camera. Granted, the higher end cameras make prettier images...but at 20 times or more the price, are they 20 times better? Ah, no.

Granted, I'd shoot on a Viper Filmstream if I could...but I doubt I'll be able to afford that in the next year. This other camera I could just go buy if I felt like it and start shootin' stuff. Use Lumiere HD to get it into Final Cut Pro HD. In fact, I may just go ahead and decide to do that...

-mike, feeling whimsical about credit card debt today

So I'm doodling around with some workflow ideas, and it hits me -

Media Manager in Final Cut Pro HD has a "Recompress To" function built into it. You can use it to convert from one codec to another.

A couple of quick tests revealed:

Converting 1920x1080 8 bit 4:2:2 BlackMagic 2Vuy codec (their 4:2:2 codec) footage (originally shot on HDCAM, captured uncompressed) to DVCPRO HD 720p24 can be done...but the downconversion math is sloppy - various scaling artifacts remain. Could be done for offlining purposes if you had to, though...and it does not require the presence of a second deck to dub it..

Converting 1080i60 BlackMagic codec footage (again, shot HDCAM, captured uncompressed 1080i60 4:2:2 8 bit) to the DVCPRO HD 1080i60 codec WORKS. Conversion of a 20 second clip took roughly 2 minutes on a Rev A Dual 2.0GHz G5. So that's about 6:1 render time. One hour of source footage would take six hours to convert, and so forth. But it could be done in an unattended fashion overnight. The footage looks quite good, and the only thing I've noticed so far on the single clip is that the conversion makes the footage LIGHTER as played back out via HD-SDI. I'm monitoring the actual HD signal via HD-SDI, so it's not a QuickTime "computer screen only" thing. But the shift is mild and probably correctable with a realtime color correction adjustment of some sort. Certainly not enough to get really upset about, but enough to want to color correct for if I cared much about final quality output. I'll see if it's clipping any values too at some point.

Interestingly, the two formats look identical on computer screen, it's only on HD monitor do they look different. Perhaps something in the BlackMagic card? I wonder if I converted back to BlackMagic codec if it would go back to looking the way it was before....yet another thing to test later.

So this offers a very interesting workflow - since the BlackMagic cards cannot at this time (as the Kona2 claims it can/will) convert on the fly during capture from uncompressed HD-SDI to DVCPRO HD written to hard drive during capture, you could always capture to uncompressed, flop all clips on a timeline, and then Media Manager Recompress to DVCPRO HD, and all your logging data would be intact.

So workflow could be something like this:

-Log and capture all day your HDCAM 1080i60 footage (or until your array is nearly full) on a SATA RAID, cost $1000 - $3000 depending on capacity and casing options.

-Then at night (or over lunch) use Media Manager to Recompress to DVCPRO HD 1080i60.

-When you return and check that it worked correctly, you can toss the uncompressed footage or copy offline and shelve it for relinking later if you want to finish uncompressed, not rent a deck again, and you have tons of drive space sitting around.

-Edit your way merrily along with the relatively low bandwith (14MB/sec) DVCPRO HD footage. You now have realtime color correction, effects, etc., and can monitor out of a BlackMagic DeckLink HD family card (haven't tried a Kona2 yet for this).

-Lay it off to DVCPRO HD via FireWire to an AJ-HD1200A deck if you wish, or to a Sony HDCAM deck via the HD-SDI on your DeckLink HD family card.

This way, if you had, say, 40 hours of footage, it could be stored on 1.8TB of storage.

Devil's advocate position: since the RAID 5 only writes about 50MB/sec with the Highpoint RocketRAID 1820A card, but reads around 200 MB/sec in some configurations, you could use it as "safe" storage space.

Or consider a FireWire 800 hardware RAID 5, such as the eRAID5 from FireWire Direct. That unit even supports clustering, so that two machines physically close together can share access to the same array via FireWire 800, such as a log/capture station and an editing station. Or just use Gigabit ethernet to connect them.

Anyway, yet another way to save money.

The only downside to this is that it ONLY works with size and framerates supported by DVCPRO HD...and 1080p24 (1920x1080, 24 progressive frames per second, what you'd want for theatrical release film stuff) is not one of them.

Camcorder Info has a report from IBC and it included this snippet:

"Sony was showing a professional version of the HDR-FX1. The model does not have a name yet, and basically looked like a darker black version of the HDR-FX1 announced earlier this week. The only discernable, major difference is the inclusion of two XLR audio inputs, and a second manual audio dial control wheel. Sony would not officially say if the camcorder has 24P. However, we got a sense that it would not include this feature. With no official price available, a range of around 6,000 euros was suggested. Sony was also showing an HDV deck, though they did not have a model number or information on its availability"

This seems to confirm that the standard model does not have XLR inputs as I already thought. I had a reader write in to say an Apple guy he knew thought that at least were mini (1/8th inch, like headphone jacks) line inputs on the HDR-FX1, but I haven't heard any solid confirmation beyond this thirdhand "think so" level of evidence....and the press releases have made no mention of it in the feature list. Harrumph. But it's still a pretty groovy camera, and the pro model would be a lovely little digi-indie on the cheap solution even without true 24p (and the pro model I'd think would also include the CineFrame feature) if used with Magic Bullet Suite HD, for instance. If films have been made from DV's 29.97 interlaced up to film (poorly, low commercial success unless your last name is Von Trier), why not use higher res DV for an improved product (even if destined for SD (standard definition) delivery?

-mike

I've received some emails, and in comments I see around the web, about the technical density of this blog. Yeah, I don't slow down to explain concepts very often. Sorry. I need an FAQ, but don't have time to stop and write one. I need a wiki for this stuff too. Someday for both.

But in the meantime, others have done some nice work in this regard - I direct you to the top right side of this site to the section titled Links. These are the best, most information laden, technically accurate sites I've found. Each has a wealth of useful information for those seeking to shoot and post HD, and several have deep, rich veins of FAQ knowledge to be mined. I HIGHLY encourage anyone serious about HD stuff to thoroughly dig around on these sites.

If you can smugly answer that you already have, look again - based on a lot of requests for camera information, I've added a link to CamcorderInfo.com.

And more important than that, if you know of a good site for good deepgeek info on HD matters, please email me at mike@hdforindies.com and let me know! This site is all about sharing the wealth, so please help me do so.

I want to say thanks to the half dozen or so readers that have sent in links that led to postings on the blog in the last week or so - very much appreciated, and all are welcome to do so.

-mike

At IBC on Friday, an Apple honcho "announced that the next major release of Apple’s Final Cut Pro will support Panasonic’s new ADX4000 1080i 50FPS camera and P2 tapeless DVCPRO-based solid-state memory products."

I can't find any reference to this camera on the Panasonic site or via Google search. No idea on price, quality, or shipping date. So please take all that follows as one big slice of conjecture.

1080i50 could be hornswaggled into a 1080p24 without too much trouble. Funny, not two hours ago I was suggesting to Frank Reynolds (I'll post further updates on that conversation soon) that Apple will eventually support 1080i50, which someone could make a 25p mode for, which could work with minor modifications to work for authoring 1080p24 footage, even if only using it as an offline codec (maybe even online) with realtime effects AND a relatively small datarate (11.2 MB/sec). Even shooting 1080i50, just using smart field interpolation would yield good results as compared to Varicam. Varicam only records 960 pixels horizontally and 720 vertically, for a total of 691,200 pixels. Presuming the ADX 40000 is recording to DVCPRO HD (a pretty safe assumption), it will record to tape 1280x1080 pixels, for a total of 1,382,400 pixels. But those 1080 pixels will be field averaged or interpolated. Worst case scenario, you're getting only 540 vertical lines of resolution (it's really better than that, but we'll start there). So that would be, um, 691,200. Woops, back where we started, but worse - the human eye is more sensitive to vertical resolution than it is to horizontal resolution, so the distribution of these pixels (1280x540) isn't as optimized for viewing as Varicam (960x720). Interesting.

So it's all up to Panasonic to make a 25p mode with this camera. Will they do it? I dunno. Will 1080i50 processed in software to 1080p25, then time altered to 1080p24, look any better overall than 720p24 from Varicam? I think maybe a little better, but it's a LOT more hassle and work.

If no 25p mode, careful analysis required to see which camera (Varicam or ADX 4000, assuming the glass/optics/CCDs are quality comparable to Varicam, that's a LARGE assumption since I have NO further info on this camera) looks better.

Just doing a little calculator math, 1080i50 should be a touch over 11 MB/sec as DVCPRO HD codec. Presently, 720p24 is 5.7 MB/sec. So SHOULD end up with better result, even with field blending.

If the camera DOES have a TRUE 25p mode (NOT just dropping a field out and duplicating the other, but actually recording 25 progressive frames per second), then we've got a hit - a camera that shoots a good image, and a post workflow that's very fast and inexpensive-you could in theory cut a feature on a PowerBook and some FireWire drives. Yeehaw!

I hope Panasonic did it "right" as I see it. But it would be perfectly understandable from a price and technology perspective if the camera had no progressive capabilities whatsoever. Understandable, but irritating.

-mike

Automatic Duck has used IBC to roll out new versions of it's timeline integration tools. "This powerful new solution allows digital media content to be imported into Adobe Premiere Pro AAF, OMF and XML files, enabling Premiere Pro users to finish projects offlined on Final Cut Pro or Avid editing systems." Also of note, at IBC they will "showcase its next generation timeline integration engine software for users of Adobe After Effects. Automatic Duck Pro Import AE 3.0 is the only timeline integration solution in the industry able to import media files from Apple Motion, Apple’s newest real-time motion graphics design solution. Additionally, Pro Import AE 3.0 enables import of XML files from Final Cut Pro."

OK, two products here, both interesting for their own reasons. I'll start with Pro Import AE 3.0. Earlier versions of this tool have been around for some time. Automatic Duck specializes in tools to let editing applications (first Avid, then Final Cut Pro, now Premiere Pro) get the editing timeline, with all the tweaks (edits, heads, tails, PIP, FX, etc.) into an effects program, like combustion or After Effects. So this new version of Pro AE Import will let you move your whole timeline, not just a rendered movie, from Final Cut into After Effects. This is GREAT if you have a lot of subtle color work to do, or a lot of effects, or basically anything where you still want SOME degree of editing control but to be able to harness the power of After Effects on your footage. For the kinds of intense motion graphics and compositing heavy work I've been doing for the last 8+ years, this is an INCREDIBLE time saver and boon to workflow.

The problem is that After Effects "blows chunks like a rocket launcher" as an editor - ALL previews have to be laboriously rendered to RAM to be viewed, even for just straight footage - NOTHING "just plays back" in After Eff