HD For Indies

Adam Wilt's discussion of DVCPRO HD codec (and some other codecs)

Among other things, he talks about the rumor that 1080i50 will have a higher horizontal resolution than the 1080i60 model - that DVCPRO HD 1080i50 might be 1440x1080, instead of 1080i60's 1280x1080. So even better for the indie moviemaker crowd since it would capture more detail. The next question is whether it will truly be PAL (which stands for Phase Alternating Line) and skip every other field for color data. In other words, is it going to be 4:2:2 like it's NTSC cousin or 4:2:0?

Sony (totally unbiased source, right?) has this article on film vs HDCAM resolution

Steve Mullen's excellently detailed write-up on the Sony HDR-FX1

In short, this is the best in-depth article on the camera I've seen so far. Nice job Steve!

Some interesting bits:

-he talks about how the sensors on the CCD array are double width, to give the 1920x1080 output from the 960x1080 CCD array

-he talks about how the green pixel sensors are offset, halfway between the red and blue pixels, thus increasing effective resolution...sometimes...depending on color and stuff....to give an effective resolution of 1440 (1.5x the ostensible 960 pixel resolution)

-he talks about how the CCD chip has micro lenses to focus the light coming towards the chip directly and only on the portion that measures light - this is way clever, and helps the camera's light sensitivity

-he talks about how the FX1 uses an elementary stream, not a transport stream of MPEG-2, and thus won't record directly onto D-VHS tape. Bummer.

-it's 4:2:0

-he talks about the Pro model WILL have XLR connectors, and he has photos to prove it

-he discusses the CinemaFrame 30 mode, and how it's decent quality

-he discusses CinemaFrame24 mode, and how it drops every 5th frame, so movement will have an uneven flow, a stutter to it. Ick.

I bought a dual 2.5 GHz G5 today almost on a lark - they had them in stock (only 2) at my local Apple Store (Barton Creek Square Mall in Austin, Texas), so I jumped at the chance and got me one.

I also received my 2nd (working) Highpoint RocketRAID 1820A card in the mail and laced up a horrible mess of cables.

I also happen to have 16 300GB drives sitting around the house. Bwahahahahaaaaaa....

If you haven't read my previous report (let's call it Frankentosh 1.0), scroll down to the article below to read it.

I tested with BlackMagic card in slot 4 (finally got a Mac it'll fit in Previously I couldn't get it to fit in my Dual 2.0 GHz G5 for fear of breaking something on the motherboard)

I put the 2 1820A cards in slots 2 & 3, and just lay the G5 on it's side and draped the cables over the top end of the computer, putting the clear airflow panel on the supine Mac and closing it as much as it would (not much). Drill/punch/cut time if I wanted to seal up the Mac.

The upshot -

Not a whole lot faster. My nomenclature:

X+Y+Z

where X=number of drives connected to the first 1820A connector
Y=# of drives on 2nd 1820a
Z= # of drives on internal SATA connectors (the "normal" ones)

I tried 7+7+2, got 481MB/sec reads, 424 MB/sec read/write
I tried 6+6+2, got 495.3/430.5 read/write (448/433 on last part of array)
6+6 was 439/380 read/write, 390/380 at the tail of the array
I tried 7+2 just to see if that did any better than 8+2, but was in slot 2 or 3, and got 500/425 read/write
In slot 4, 8+2 got 575/415 read/write

Lessons learned:

The card doesn't scale performance in a way that would be substantially useful for HD usage with an HD card installed.

With two cards installed, the RAIDman software that comes with the 1820A won't create a RAID bigger than 2TB. Dumb but true (they're working on it)

RAIDman can't span a RAID across two cards.

SoftRAID and Apple Disk Utility, while they cannot create RAID 10 or RAID 5 volumes, can create RAID 0 volumes that span cards (and can include internal SATA drives in the "standard" 2 internal SATA slots).

"Only" 430 MB/sec write speed seems to be the cap...but it holds it all the way to the end of the array, so clearly it's a card limitation, not a drive limitation. I was a bit bummed - with single SATA cards, my experience has been that drives achieve about 90% of the speed in an array (per drive) that they would by themselves. So In theory, this 16 drive array might have achieved 960 MB/sec.

I got read speeds a bit over half of that, but write speeds well below half of that.

This is all still RAID 0, so back up everything regularly to a stack of drives or else your are playing Russian Roulette with a LOT of bullets in the chambers. I'd expect an array of this size to fail every 3-12 months, just at a guess. Remember, 16 drives in an array makes the array 16 times more likely than a single drive to fail if you're only using RAID 0.

If you did suffer a crash, even with 50 MB/sec transfers, you'd be looking at over 25 hours to restore (after fixing) 4.37 TB in a fully populated RAID 0. This was for stunt purposes - nothing in HD res is going to nee that kind of throughput. If you needed this kind of capacity, break it into smaller logically sized/throughput capable units. But it is nice to know you could read & write 2K film scans (2048x1436 10 bit) in real time. Hmm, how much footage is that? About 4 hours and 42 minutes. Not bad!

Nope, not a typo - that's five hundred and seventy-five Honest-To-God MegaBYTES per second, running on a G5, for less than $3000 (on top of the G5, of course).

How'd I do it? Read on for details...

I've been researching high speed, large capacity, low cost storage options for over a year in my quest to come up with low cost high quality solutions for independent filmmakers using high definition video. I wanted to find solutions that would allow for high quality, uncompressed HD video, which can require as much as 270 MB/sec (with safety overhead).

(For more on how to do this, read this, this, this, this, and this. Or just read the archives that are organized by month at the top right of this page, there are over 300 articles from this year alone.

I've been looking into SATA (Serial ATA, what the Mac G5's ship with) based solutions for some time now. The stock G5 can handle two drives - not nearly enough. Then along came the Seritek 1S2 card, and it allowed for 2 more with some ungainly cable arrangements. Better, but not great.

(Since then an external port version, the Seritek 1SE2 card has been announced.

Then suddenly out of the blue the other week, one of the better low cost cards on the Windows side suddenly got Mac drivers, and I was all over it. The Highpoint RocketRAID 1820A card comes with 8 SATA connectors, although they are all internal ports. To work around that hindrance, I'm running all the cables out an empty PCI slot cover.

I'm using eight Maxtor Maxline III 300 GB SATA drives, which are the fastest large SATA drives I've found on the market. The Western Digital Raptor drives are faster, but they only hold 74GB of data.

So where are all these drives? I've put them in a couple of MacGuru's Burly Boxes that I've built up. (And I do mean built, it takes an hour and a half to assemble all the parts and load all the drives, and I've done it several times now.)

I also put two more drives (these are Maxtor DiamondMax 10 models, mechanically identical to the Maxline III's) in the two internal bays in the G5 to bring the total of drives up to 10 300 GB drives.

In order to start up the computer, I'm booting from an external FireWire 800 drive (a La Cie model), since none of my RAID formatting software allows the computer to boot from a RAID.

For HD video capture, I'm using a BlackMagic DeckLink HD Pro 4:4:4 card, which comes with a convenient testing utility that measures actual throughput of a given drive or drive system.

(Just to be thorough, the rest of my system that can edit uncompressed 1920x1080 30 bit color 4:4:4 full RGB uses an HDLink with an Apple Cinema Display 23" LCD for HD signal monitoring.)

Using that as a benchmarking utility, and averaging 7 test runs together, I get an average disk read data rate of about 575 MB/sec, and an average Disk write data rate of about 416 MB/sec.

The 1820A card seems to max out at about 345 MB/sec on write speed - even when more drives are added to 1820A, that's as good as it gets. But it can achieve that speed with as few as 6 drives. Adding more drives continues to boost the read speed, but not the write speed.

By adding two more drives to different SATA buses (the built-in ones on the motherboard) allows me to boost the write speed past the 345 MB/sec limit. This is kind of cheating, since I'm using some internal and some external drives for this array, an ungainly setup. But it would also be possible with 3rd party cables to connect external drives to the SATA ports on the motherboard to store the drives outside the G5 case.

OK, so 575 MB/sec is the headline attention grabber...but you do NOT get that performance with all of the disk. As soon as you start filling up the disk, it gets slower. Huh?

In brief: a hard drive is like a record player...not that I've owned one since I was 12. The platters are like the record, the read/write heads are like the needle. And like a record (you've seen them on late night TV, right?) the record starts at the edge when the needle is placed way out at the outermost part. This part is moving very fast under the needle. The maximum amount of hard drive platter is moving under the read write heads every second at the edge. As data is written from the outermost tracks towards the middle, the more full the drive gets, the more it has to work with the inner tracks of the platters, or record in our analogy. And that is the part of the record/platters that moves the slowest under the read/write heads.

If that doesn't make sense, think how fast it was to hang on to the outer edge of the merry go round instead of the middle. Faster=more data throughput.

If that didn't make sense, just trust me on this: hard drives can't read and write data as fast as they get full as they could when they were empty. The transfer rates fall off as you have to work with the innermost tracks.

I tested my 8 Maxline III 300GB drive array by breaking it into 5 partitions of 435ish GB, and a final partition of 50 GB. The reason for that last smaller one was to test performance at the very end of the array where it is the slowest.

Here are the test results from the BlackMagic Disk Speed Test utility:

As one big partition (the whole 2.2TB)- 471/345 read/write

Slice1 (the first 435GB)- 467/345 read/write
Slice2 (the second 435GB)- 451/342 read/write
Slice3 (the third 435GB)- 418/342 read/write
Slice4 (the fourth 435GB)- 293/344 read/write
Slice5 (the fifth 435GB)- 345/354 read/write
Slice-last50GB (the last 50 GB)- 275/290 read/write

So, how's this done for so little?

1 Highpoint RocketRAID 1820A PCI-X card with 8 internal SATA ports: $204 from zipzoomfly.com
10 Maxtor Maxline III 300 GB SATA hard drives - from zipzoomfly.com, $220 each, $2200 from zipzoomfly.com
2 MacGuru's Burly Boxes - $215 each, $430 from macgurus.com
4 Molex to SATA power adaptors (NOT included w/Burly Box) - $6ea, $25 MacGurus.com
buncha zipties: $1 - Fry's or wherever

Total Cost: $2860

If I were really doing this for a production setup, I'd spend a bit more money.

SoftRAID for RAID setup and partitioning - $100

I'd buy drive coolers for all Burly Box drives. Heat is the leading cause of early hard drive death in my experience. 8x$30ea = $240

Since there is really no call for the extreme throughput, unless I really needed the additional capacity, I'd stick with an 8 drive array. It would still give about 2200 usable GB of space, and have the convenience of an all external solution. That would drop the price by 2 drives, so $440 less.

And the biggie for last - all of this has been RAID 0, which means if ANY one drive in the array fails, ALL data is lost short of heroic (and EXPENSIVE) efforts. The only reason why I suggest a RAID 0 is that in the context of a video production job, the most crucial data is the file you create with your NLE (usually Final Cut Pro HD these days) and the audio and video data you capture, plus any graphics you create along the way. As long as you have those assets backed up, you're OK. READ HERE FOR BACKUP STRATEGY DETAILS.

When capturing footage, that's usually done in a large chunk early in the project. After that, it doesn't change much. If you're willing to copy that capture data to backup hard drives (such as FireWire drives like the La Cie d2 or Bigger Disk or Bigger Disk Extreme), you're covered in case of drive failure. You'd lose significant time getting back up (a day, maybe two), but life would go on.

So you'd be looking at $2000 to $2500 worth of backup space if you're at all serious about this.

"Real" setup cost for 8 drive array capable of as much as 470 MB/sec reads and 345 MB/sec writes:

1 Highpoint RocketRAID 1820A: $204
8xMaxtor Maxline III 300GB SATA Drives: 8x$220=$1760
MacGurus Burly Box with coolers and power adaptors: 2x$347=$694
SoftRAID: $99 from softraid.com

Total Cost: $2757

Backup FireWire 800 storage

2 La Cie Bigger Disks Triple Interface (2x$1000) and one 250 GB d2 Triple Interface ($290) =$2290


Worst case scenario, this setup delivers at LEAST 275 MB/sec reads, 290 MB/sec writes all the way to the end of the disk. That's enough for ANY flavor of high defintion video, even 10 bit per channel 1920x1080 60 field per second 4:4:4 RGB, which is only possible on $100,000+ cameras and decks. Even to the 80% mark of the disk, you'd still get at least 345 MB/sec reads and writes. Or, it would make a helluva Photoshop scratch disk. : )

What about X-RAID, or othe RAID 3 or 5 or 30 or 50 solutions, such as those from Huge Systems or Medea? They work, they work well, they are well proven...and they are STILL more expensive than this SATA solution I'm suggesting by a substantial margin, even when you factor in the FireWire backup. Huge is over $4.50/GB, Medea is about $5/GB for systems with similar throughputs by MUCH better features - expandability, scalability, reliability, etc. When it comes to Apple. Huge, or Medea, if you can afford'em, buy'em. If you can't, look into this stuff.

A properly cooled and set up 8x300 disk array formats to about 2220 GB of usable space for about $2750. That's about $1.13/GB. Buy two Bigger Disks and another 250 GB FireWire drive all from La Cie, you've added $2290 to your costs, but are still looking at only $2.27/GB for Mike's Recommended Solution For Indies.(That's me.)

Apple's 1.75TB X-RAID, which has LOTS more flexibility, scalability, reliability, stability, and general goodness, costs $7500. That's over $4.50/GB. Lots of good arguments can be made about how it's worth the money, and I'd agreee with them...for those who can afford it.

For the indie filmmaker, that can be out of reach. Indies typically have more time and greater risk tolerance of down time than a well funded production facility, which is why I'd suggest this SATA with FireWire backup route.

For more info on how to put together a great low cost HD workstation, see here, here, here, here and here. Or keep checking back with this site, HD For Indies as I continue to write about high definition video, digital cinema, and how to get it all done at minimal cost.

If you'd like to keep regular tabs on these issues, check out my Atom and RSS feed links at the top right corner of this page, and use an RSS aggregator such as NetNewsWire or somesuch.

Also of interest: perfect pixel for pixel monitoring of your HD signal for as little as $1400 (for HD 720p work) or $2700 (for HD 1080p or 1080i resolution work) - read up on HDLink

DISCLAIMER: This is my 3rd day or working with the RocketRAID card, I don't really advocate it YET for production usage. I think it shows a lot of promise, but has yet to prove itself in a production environment. For instance, in the middle of duplicating a 5 GB file, I had a kernel panic - the bad kind where a grey scrim lowers, looking like a curtain coming down, and it states in several languages onscreen that it's time to shut down. Eeeeeeeeeeeew. I can't remember the last time I got one of those, so take all this drive stuff with a grain of salt until the drivers and card prove stable and reliable. This is, after all, a 1.0 driver we're talking about...

-mike

Mike Curtis is a digital post production supervisor in Austin, Texas available for consulting on your HD projects. If you have an HD project you are contemplating shooting, talk to him about being a test site for some new equipment under development. Email mike at hdforindies.com

So today I called my local Apple Store to ask them about something, and it turns out they had Dual 2.5 GHz G5s in stock. Wow. Apple's online store estimates 3-5 weeks for those to ship, and their online quotes are "optimistic" at best in my experience, ESPECIALLY when dealing with recently released products.

So before the phone had clattered to the floor, I was whipping down the highway to go get me one.

Once I got there, I talked to their business sales guy, and he knocked several points off of the selling price since I bought AppleCare and ProCare for the unit. You could argue that extended warranties aren't worth it, but for this unit, for the purposes I'm going to use it for, I felt it was well worth the investment.

Also, for those looking to save some money on your editing setup, if you buy it with your G5 purchase, you can purchase Final Cut Express for $99, which is upgradeable to Final Cut Pro for $699, which saves you $200. Yes it takes longer, but it's $200.

I'll be hooking up a stupendous array (16 drives) and seeing how fast I can Make It Go just for fun in the next few days.

-mike

While researching for another article (to be posted) I came across this on the Sony site - a 1920x1080 resolution high defintion video camera, "block" style - meaning it's meant for industrial mounting, not for field shooting. It has no handle, it has no microphone or mike jacks, it lacks almost all controls you'd expect to find on a "real" videocamera.

And yet....the idea is intriguing. It has three 1/2" 1.5 megapixel CCD sensors, and it comes with a lense. I'm curious if a ravingly desperate indie might be able to make use of this somehow to shoot an HD feature/short/commercial by strapping a handle onto it (duct tape, anyone?) recording to an HD deck of some flavor. This unit does support 24psf, so you can shoot a film-like 24 progressive frames per second. It also has an HD-SDI output, so that's cool, that's the industry standard high definition connector to use. Not really production viable, but it's a beguiling idea.

Or it might be the ideal "lowest possible cost high quality" camera for certain tasks like strapping onto a helmet during some Extreme Sporting Activity with a wire running to a battery powered SRW-1 deck (with EXCELLENT production insurance). Again not really, but a fun idea.

Anyway, it'd be fun to have one of these to doodle around with and see what the image quality was, how much control can be exerted over the camera, what the exposure latitude is, etc. etc.

It would be a good "lab mule" to play with, since it does have an HD-SDI output.


Anybody played around with one of these, in whatever capacity? Drop me a line at mike@hdforindies.com if you have, I'd love to hear whatever you can tell me.

-mike

It hasn't shipped yet, nor has a price been determined, but Flip4Mac is saying they'll be able to encode Windows Media 9 stuff from a Mac based application. This is a nice piece of the puzzle, since the two competing formats for high definition DVDs will include a Windows Media 9 based encoding scheme as a requirement of the playback devices. Plus, as much as I don't admire Microsoft for their business practices, they've made a really, really nice and impressive codec with WM9.

From their website:

-Import or Export components for Windows Media Video 7, 8 and 9,
-Windows Media Audio Standard, Professional and Lossless

Standard version

-single-pass video encoding
-constant (CBR) and variable (VBR) bit rates
-up to 48 kHz audio sampling rates

Pro HD version adds

-two-pass video encoding
-up to HD video resolutions
-5.1 channel audio (WMA 9 Professional)
-up to 96 kHz audio sampling rates (WMA 9 Professional)

The Pro version will obviously be the one of interest to HD afficionados.

Note also the ability to import Windows Media 7, 8, or 9 assets - this has long been a thorn in my side when trying to work with client provided video files. Windows Media has often been targeted as a delivery codec, not a working codec. Most QuickTime codecs can be edited and worked with in the standard video manipulation applications. This hasn't always been the case with Windows Media files, even on Windows computers. But I haven't had to deal with Windows Media files in a year or two, so I'm out of touch.

Note this doesn't promise compatibility with the upcoming HD-DVD and Blu-Ray specs - just WM9 compatibility. I'd imagine they might either upgrade the software to compatibility with those specifications, or offer another product supporting that capability. Either way, there exists the possibility of it costing more. So if you want to author for high defintion DVDs (which are over a year away from shipping, anyway), this isn't a guaranteed solution yet.

Then again, this software isn't even shipping yet. Click the link to sign up for more info if you wish, or keep reading this site.

-mike

After not being able to find pricing information online about the new Medea VideoRAID XTRM I contacted Medea, and they got back to me. I talked to them about this as yet unreleased Fiber Channel interfaced, SATA drive product that offers RAID 3 data protection.

Here's some details:

-the VideoRAID XTRM connects to a Mac or PC via a 1GB Fiber Channel connection

-Fibre Channel 2GB interface - their card or Apple's? I think there's a choice involved

-15 drives in device, configured in 3 separate RAID 3's of 5 drives each

-Each block of 5 looks like 1 logical drive unit to controller

-Single controller, 3TB of usable space, redundant power supplies, RAID 3 redundancies, redundant fans

-controller is NOT redundant (only one controller)

-2GB interface, 3 groups of 5 drives. At this time, you cannot stripe the 3 groups together into one bigger, faster unit. That limits the sustainable transfer speed to about 125 MB/sec. Thus this device is good for HDCAM 1080p24 (8 bit 4:2:2) footage, but is cutting it closer than I'd like to see for 1080i60 footage or 10 bit 1080p24 footage (120ish MB/sec, too close to 125 for safety's sake).

-expect a 3TB device to be roughly $10K

-it's using SATA drives internally

Mike's Comments:

The Good News: It's 3 individual RAID 3 devices - this means that in each group of 5 drives, one drive is dedicated to storing parity data for the other 4 drives. This means that if ANY one drive in the group of 5 fails, your data can be retrieved via a rebuild process. If two drives in the group fail, "is not your day." The price performance will be pretty good - if the device has 3 TB or RAID 3 space, that means 15 250GB drives, therefore 4 drives times 250 GB/drive times 3 groups of drives is 3TB. That's about $3.33/GB. That's very good for RAID 3 storage - better than my other price performance leader, the fully populated Apple X-RAID at about $3.50/GB. As this is a Fiber Channel device, lots more of these can be hooked up via a Fiber Channel hub.

The Bad News: The 3 groups of 5 drives can't be striped together to work faster than 125 MB/sec (1 gigabit/sec) due to the limitations of the current controller. If you're doing 8 bit HDCAM 1080p24 footage (8 bit 4:2:2 1920x1080 at 23.98, 24, or 25 progressive frames per second) or PAL timed 1080i50 (50 fields per second, same size, still 4:2:2) this is a very nice fit for a single stream solution.

However, if you want to work with 10 bit 4:2:2 1080p24, or NTSC timed (59.94 fields per second), or any 4:4:4 footage, this product is NOT acceptable. I'd imagine Medea is quite aware of this and probably working on a solution to fix this.

So if you want to do any 1080 res footage in 10 bit, or 1080i60 (really 59.94), or any 4:4:4 1080 res footage, I'd say DON'T buy this product at this time until it is modified to be acceptable for those applications.

As a general rule, I try NOT to buy equipment on the future promise it'll do something. I buy it based on what it does TODAY. All too often future hoped for features either never come about or are integrated into a new replacement product that you can't upgrade your current product to match.

-mike

Robert over at BareFeats has once again posted a very useful article, this time doing a head to head comparison between a dual G5s, Dual Xeon, Dual Opteron, and a Pentium 4. They tested Photoshop, After Effects, Bryce, and Cinebench (a 3d rendering benchmarking application).

Mike's Comments: Results: The G5 did suprisingly well, winning many of the tests and blowing the doors off of the competition in After Effects by a wide margin. Then again, it all depends on exactly what you test for - your mileage may vary depending on the kind of work you do. Prepress Photoshop needs are different from an artist's Photoshop needs, and an After Effects motion graphics animator may need completely different portions of code used than a compositor/visual effects artist. Mileage varies.

Now that I'm getting a high speed (hopefully) stable testing platform established for HD work, I'll be testing and documenting a variety of workflows and solutions, and stopwatching them all. I'll post results as I get them.

-mike

I've been asked this more than a couple of times, so I'll answer it here. A reader wrote in and asked:

Why would you use the HD link instead of just the graphics card output and
then FCP HD video monitoring with the HD Cinema display?

What are the advantages?


I get asked this one a lot -

1.) HDLink allows for a true interlaced display
2.) HDLink shows a true video signal, not an onscreen representation
3.) Fullscreen mode in FCP can choke on 1080 res footage, even on dual 2.0 GHz G5
4.) The Biggie - if using DVCPRO HD, what you see on screen is a preview approximation of the HD signal - you aren't seeing all the detail of the signal. Try messing with Low/Medium/High quality preview and you'll see what I mean.
5.) High Quality won't play back full speed often
6.) Even if it plays back fullscreen in high quality, there are playback speed inconsistencies in FCP fullscreen mode that don't occur with HDLink
7.) HDLink can be calibrated in a more controlled fashion.
8.) And oh yeah - HDLink maps your video pixel for pixel to the display instead of stretching it up or down. While you can set FCP to show 1:1, it still isn't as good a representation.
9.) and oh yeah! The way colors look on uncalibrated computer monitor (even 23" LCD) vs. the gamma corrections (and calibration possible) with the HDLink are SIGNIFICANTLY different - would throw your color work waaaaaaaaaaay off.
etc. etc.

I break it down like this - the FCP way is a preview of your footage. The HDLink way is a much more accurate representation of your footage.

In terms of budget, I'd break it down like this:

1.) If you can afford it, work with both a SMPTE C studio grade calibrated HD monitor (for color accuracy) and an HDLink with Cinema Display (for seeing fine detail)

2.) If you can't afford that, work primarily with HDLink with Cinema Display, then rent a SMPTE C studio HD calibrated monitor at the end of your post cycle to do your critical color correction work.

3.) If you can't afford that, forgo the monitor rental and the the best you can with HDLink.

4.) If you can't afford that, use Digital Cinema Desktop Preview (or whatever they call it) and rent a monitor at the end of your post cycle.

5.) If you can't afford that, work with Digital Cinema Desktop Preview and suffer the consequences. (Your color corrections will be waaaaaaaay off!)

-mike

Robert at BareFeats has posted a very nice and thorough article on 1820A RAID setups on a Mac G5

Robert at barefeats.com and I have been discussing the 1820A card, he emailed me this nice summary of the pros & cons:

FEATURES or advantages

You have room to grow with 8 ports
It captures 10 bit uncompressed HDTV with 4 or more drives in a RAID 0 set.
The included RAIDMan software supports RAID 0, 1, 5, 10 and JBOD.
It's compatible with both SSC and non-SSC drives.
The kit includes 8 SATA internal cables.
It runs on G4 and G5 Power Macs under OS X 10.2 or newer.
It's an affordable solution ($204 at ZipZoomFly)

"NON-FEATURES" or deficiencies

None of the 8 ports were external.
You can't boot your Mac from any drive connected to the RocketRAID.
There's no hot-swap support. You must Shutdown your Mac to connect/disconnect.
It does not support Deep Sleep
It "hits the wall" at 340MB/s WRITE speed, with 5 or more drives in RAID 0 set.
Their RAIDMan utility runs slower than Apple's RAID software and SoftRAID.
It lacks support for G3 Power Macs with PCI slots
There's no support for Macs running OS 8, 9 or 10.1
The plastic connectors break easily. (This can be an issue especially if heavily shielded external use data cables are routed out an empty PCI slot to external SATA boxes.)
Cables provided do not have proper shielding for external use.

Mike adds:

Cons: it is a 1.0 driver - I've already had a kernel panic
I'm using SoftRAID to partition and format (Disk Utility will format but not partition an array) since it gets better performance than the included RAIDman software
Have to use RAID 0 for HD work, and that creates data vulnerability
RAID 10 works, but is constrained to about 180 MB/sec write speed
RAID 5 - I haven't been able to get it to work at all, perhaps due to all the various drivers I've installed over time. They say it works and others have used it successfully without event
RAIDman software - funky/clunky to set up, and is slower than Apple Disk Utility or SoftRAID
Two card installs don't run much faster than 1 card installs - clearly the card is the bottleneck, not the drives
Two card installs are limited to 2TB per card at the moment due to technical reasons

MacWorld has a review of Discreet Combustion 3, a sub-$1000 compositing and visual effects program.

As a 10 year After Effects veteran (back from when it was CoSA), I've sat down to learn combustion not once but twice....and walked away from it as "gosh that's hard" after an hour or two. I really do need to learn it...someday.

It is very powerful, has great keyers, grain management, floating point support, particles, etc., but it is highly non-Maclike.

The good news is that if you master it, you now grok the basis of all the higher end Discreet products and can hand off projects to those higher end products as well with seamless integration.

I'd break it down like this: If you are looking to do design heavy motion graphics, use After Effects.

If you are looking to do basic effects work, either After Effects or Combustion will work.

If you want to get fancy/serious/high end with your effects work, Combustion can do things After Effects can't.

-mike

According to this MacWorld article, Sony is developing a 200GB version of it's Blu-Ray disc technology.

It uses 8 layers to store up to 200GB, but Sony made no formal announcement that it would become a product, at this time it is merely a product demonstation.

Sony also announced that standard (not 200GB) Blu-Ray disc playback will be incorporated into the PlayStation 3. This should help push adoption.

Mike says: In case I didn't mention it, about three weeks ago the Blu-Ray group added MPEG-4 AVC (Advanced Video Codec) and VC-1 (the Microsoft Windows Media 9 based codec) to their high definition DVD specification to match the codec (compressor/decompressor) choices that the competing HD-DVD specification offered. These two competing formats are vying to be the standard that Hollywood studios might pick to provide high definition versions of movies on disc.

Sony also purchased MGM Studios (the movie company. The Big One. Yeah, I know!) recently, so this gives them the leverage to release movies on the Blu-Ray format.

But this creates a possible schism - Sony/MGM discs might be released on Blu-Ray, but other studios may go with HD-DVD, with it's "good enough" capacity and much lower manufacturing costs. Disc manufacturing facilities can be retooled to produce HD-DVD and regular DVDs with a very brief (coffee break duration) setup. And in business, cheap wins. Always.

Hey! They did it! Let me look at my watch - FIVE MONTHS after FCP 4.5 ships. This is why I say don't bother with Digital Voodoo - they are slooooooow in development, overpriced, and in general just out of contention these days.

I sincerely and forcefully do NOT recommend their products.

If you have one of their products, you have my sympathy, and can read the full article here.

From Apple's site:

If Final Cut Pro HD unexpectedly quits on you after using Software Update, a few short steps should fix it up.

Warning: This document describes how to enter commands in the Terminal application. Users unfamiliar with Terminal and UNIX-style environments should proceed with caution. The entry of incorrect commands may result in data loss and/or unusable system software.


Save any files or work you may have open, because you'll need to restart the computer after these steps.

Open Terminal (found at /Applications/Utilities/).

Type
sudo update_prebinding -root / -force


Press Return.

Enter your password when prompted.

Restart your computer.


Final Cut Pro HD should now open normally.

This document will be updated when more information becomes available.

UPDATED TUESDAY - SEE BOTTOM
ProMax, a longtime desktop video systems integrator has a new product of interest to those wanting to put together their own uncompressed HD editing system. The ProMax 1.8TB External Array SATAMAXe-HD for G5 is a $3295 1.8 terabyte RAID 0 that comes with (I believe) a Highpoint RocketRAID 1820A card and two external 4 bay hard drive enclosures.

That's $1.83/GB, including the host card. Nice!

From their website:

The SATAMAXe-HD is our most economical high definition storage solution yet. It brings RAID 0 performance with 3 hours of 10 bit uncompressed High Definition video for under thirty-five hundred dollars. This 1.8 TB of usable storage allows you to fill the drives up almost completely and still get performance to keep outputting even as the client keeps adding changes right up to air time.



I think what the have here is 8 250 GB drives, with an actual formatted, usable capacity of 1.8TB. I think. I'm not sure.

The card they have pictured looks similar to the two 1820A cards I have at my studio at the moment - similar but not the same, since the SATA port layout is different on that card as compared to mine.

I'm also curious how they are solving the various cabling issues -

Are they running SATA cables out of any empty PCI slot cover?

If so, how are they anchoring the cables?

How are they managing the cables inside the case without making a huge mess?

Also, what formatting software are they using? Is SoftRAID included?

I'm working on the answers to this, when you see on your RSS or Atom feed that this has been updated, it'll be because I have more answers.

In comparison, the 8x300GB Burly Box solution I'm working on building will hold 2.2 usable terabytes, and costs $2704, for a cost per gigabyte of $1.21. However expect to spend 3-4 hours to install drives, run cabling, and physically set it up. As opposed to and hour or more with the ProMax solution to just install the card and connect SATA cables. Plus theirs has a warranty and tech support department, the Burly solution does not - you'd have to talk to each component manufacturer's support folks, and they could easily all finger point at the others. With ProMax, you could call up and say "It don't work." and it's their job to figure it out.

I'd expect to see 2.2TB and 3.0TB versions of this array (with 300 and 400GB drives) in the future. The 2.2TB will cost I'll bet $300-$500 more, but the 3.0TB will be much more expensive, since the drives are so much more. It'd cost at least $2000 more. Not very cost effective...but big.

-mike

UPDATED TUESDAY AFTERNOON

Talked to Jerry Miles over at ProMax, he gave me the scoop on the setup:

-How fast is it? With an EMPTY DISK - 418 MB/sec reads, 330-340 MB/sec WRITES (and that was with 4 Hitachi and 4 Western Digital drives, what he had sitting around at the time)

-it is using the Highpoint RocketRAID 1820a card

-they are routing their cables out the back of the card, they've punched some holes in it for routing (be sure to anchor those somehow!!! A yanked cable would be disastrous!)

-they did some serious stress testing on the array to try worst case scenarios - capturing footage until the array was full, rendering transitions, forcing the drives to read from opposite ends of the disks in rapid succession (think about playing notes from song 1 and song 15 on a CD in rapid intercut succession)

-using Hitachi 7200 drives, probably 7K250 model, definitely 250GB

-they've had good history and success with IBM/Hitachi drives, they will stick with them

-using Apple's Disk Utility to format, NOT the RAIDman software that comes with the Mac driver package

-the 4 port Firmtek card that is coming out has problems with drives with SSC (the Hitachi 7K250 and 7K400 models), this card does not

-THE CARD DOESN'T SUPPORT DEEP SLEEP

Meanwhile, I'm building up my own 8 drive SATA RAID, and it's giving me trouble to the point of having to check each component (cards, cables, drives, chassis) one by one to find out where the problem/s are. Makes the idea of "here's a check, gimme!" much more appealing.

-mike, Tuesday afternoon

No news posts this weekend, even though I have a post-IBC backlog of about 20 things I want to cover -- between building up 3 different 4 bay SATA drive chassis for testing to begin this week (insert evil "bwahahahahaaaaa.....") and the Austin City Limits Festival, I'm beat (and burnt! 6 hours in Texas sun...).

I was a bit peeved to discover my Burly Box from MacGurus (4 bay SATA hard drive chassis, too tired to link, see prior articles) didn't actually come with SATA power connectors, only Molex. No clue on the web page about that. While many SATA drives have dual power (both Molex and SATA power plugs), not all do, especially the newer ones, like the DiamondMax 10 and Maxline III 300GB drives from Maxtor. The good news is they do have these adaptors and I've ordered them, but it'll probably be sometime between Monday and Wednesday before I receive them.

And when they say drive KIT, they aren't kidding - even after having done it before, I think it'll take me an hour to an hour and a half to get a 4 drive unit fully built up and cabled with drive coolers for each drive.

The Burly Box needs to have the Centronics plates removed (2 screws) and replaced with one compatible with SATA port mounts (2 screws). Then a SATA port cable needs to be anchored (2 screws). Each drive to be mounted in a drive cooler or drive carrier needs 4 screws (whichever you choose, I'd advocate the coolers - heat is the #1 cause of drive death). The drive coolers need their own power connection, the drives need power and data connections...you get the idea. The one nice thing about this arrangement is since it isn't SCSI, there are no SCSI IDs to set and no termination to worry about. Thank goodness. But this comes at the hassle of one cable per drive.

I also managed to get my new Highpoint RocketRAID 1820A card (8 port PCI-X Serial ATA card) installed in my dual 2.0GHz G5 - it's an octopus, literally. Cable management and routing took a while to figure out, and it's a big gob of zipties and labelled cables. I'll take some pictures and post a link on how to get this monster set up properly (well, at least as properly as I got it). It's a bear, since the cables all have to go out the one empty PCI slot cover, AND need to not make contact with DeckLink HD Pro 4:4:4 card sitting in slot 2. The cables also need to be anchored, so that they won't come loose from the ports on the PCI card in general, and won't yank/break anything inside the chassis should the cable get yanked or tripped on. My solution of the moment - a great ugly gob of zipties going through the mesh holes in the back of the G5, and then zip tie anchoring pairs of cables to the prior pair of cables.

If you want to try to get two 1820A cards and an HD capture card in a system, you've got to start cutting/punching holes in the case. Definitely save that one for the advanced class.

So I realize I can't start testing tomorrow - I'm stuck waiting on the power adaptor cables - grrrr.....might have to go to Fry's and see if I can find a temp solution.

-mike

Just saw Sky Captain and the World of Tomorrow this afternoon at a matinee showing (yes, I have no "real" job where I go to an office and sit in a cube. Thank f***iing god...). If you're any kind of geek like I am, you know that this is the first largescale movie with top tier talent made 99% by shooting actors against bluescreen. A typewriter, just the fuselage of a plane, and a few other minimal props were used throughout the movie. It was also shot entirely on HD video (8 bit HDCAM I believe) and almost all the digital effects work and animation was done on desktoop Macs & PCs. It was also edited on Final Cut Pro, which I do so dearly love. So how well did it work? It was good, but not. Read on...

Here's what I just wrote to some film geek friends of mine:

Wow - great art direction and look, sucky sucky acting, bizarro lift-o-matic script, and inconsistent effects work.

Can the acting and pacing be any flatter? Don't think so.

There are glimpses of fun stuff in there, but there's no chemistry between the leads. Jude Law's character so cries out for the pluck, easygoing confidence and attitude of Indiana Jones or Han Solo...and so doesn't have it. I liked the last line of the film, I can see what Conran was TRYING to do, but it just didn't have heart in the end. First time director working with big name talent, and the talent doesn't know what they're doing on that big empty set.

This is the kind of job that requires, you know - ACTING! Pretending to be somewhere, doing something, feel something. And this ain't it.

The world they've created is fantastic - the director certainly deserved kudos for his vision of this fantastic universe. From a world creation standpoint, he gets an A+. The opening sequence of the Hindenburg III coming in to dock with the Empire State Building is beautiful The look they've created digitally is just beautiful, with a lot of highlight haloing and muted, desaturated colors. The concept artists had a blast with the robots, the virtual sets (no construction budget - previsualization leads directly into set production!), the costumes (the only consistently real props) and the art direction are lovely to behold.

The acting and direction, however, especially during the first hour or so, is totally flat. It's like they were figuring it out during shooting and finally got a clue about 2/3 of the way through.

The acting seems flat and dissafected - quite possibly because they were standing against bluescreen and had NOTHING to react to. Or was this in part a bit of the 30s & 40s serial feel he was trying to create? If so, I think inappropriate for a modern audience. Whatever the reason, the action in the first part of the film feels flat and affected. Perhaps it was the inexperience of the director and the team, or the technical constraints of the full bluescreen shoot, but the action and drama feels not very active and not very dramatic. When Polly Perkins (played by Gwynneth Paltrow) runs from the giant robots, it feels totally lame - she's not running very fast, she doesn't look or act very afraid. When she and Sky Captain (Jude Law) are flying down NYC streets in a P40 fighter plane and turning upside down and hanging a left at 20 feet, they don't look very concerned or intense at all. It totally doesn't feel right. Maybe I'm not getting it and that's the intended feel of the old serial films, but it comes across lame in my book.

This may be an ongoing problem with virtual set movies. Perhaps if this doesn't do well at the box office Hollywood will say "Gee, that didn't work. We're not doing that again." I think this would be a mistake. My gut says this film will not do great at the box office (hope it does, though). It appeals to a niche audience that appreciates this truly Old School look at leitmotif, not the broadest general moviegoing audience. I think critics and film buffs will like the look, but general audiences may say "Feh." and see Alien vs Predator of somesuch. Oh, the horror...

(BTW, what do you call the predator/alien at the end? Predalien? Where's the fanboy art on THAT? And what do you call the babies? Pralines?)

Reel me back in, I ramble all too readily. ADD? Just a little bit. See?

OK - I think it would be a mistake to write off this filmmaking methodolgy. Because of the minimal to non-existent sets, there are technical challenges to making complicated shots. Locked off shots are much easier. You have to have a bluescreen set big enough to capture all the action. Actors have nothing to react to or act against or act with. But come on, folks, this is really the first one. (The first VALID one. Able Edwards doesn't count). Give this new style a chance. Kerry Conran was the writer director, and all this came from a very small, indie production group. And it shows.

While I laud them for what they pulled off on a small budget (I'm betting Jude Law, Gwynneth Paltrow, Angelina Jolie and Giovanni Ribisi's fees were a huge chunk of the overall budget), it still looks small budget.

The effects shots are spotty - some are quite good, others feel somewhere north of a SciFi Original, but south of a top tier feature. I've noticed this trend in too many movies to mention the last couple of years - big Hollywood pictures are so rushed to get out they farm shots all over the place, and not everyone delivers the same quality (nor should they - farm out shots to the folks talented enough to get them done, but don't overpay - why pay ILM rates for just wire removal?).

Overall, I'd say the movie suffers from being bogged down by it's twin "tech" words - technology and technique. Too many locked off shots, not enough flair and vision in the cinematography (although there's tons of good conceptual stuff). Too many locked off shots, too much under/bad acting when it should really be roaring.

Perhaps the next film to get made more in this style will have a more experienced director at the helm, someone better with actors.

Plus, of couse, the plot has a ton of issues. But I'll leave that alone for now. There's no "make them act better" menu in an editing program, nor a "make the plot make sense" button.

OK, it's late, I'm off to bed.

The folks over at CamcorderInfo have this very long, very detailed review of the HDR-FX1 camera. Some highlights:


-it DOES have an audio line in, it just isn't XLR-but you can use an adaptor from Beachtek.com (or similar) to resolve that problem.
-the CineFrame 24p mode uses a 2:3:2:3 cadence, and should work pretty well for indie filmmaker types. YES!
-lense is fixed. You better like it
-it DOES have component analog video output. Monitoring solved!
-LOTS of physical controls, Sony's moved away from menu driven stuff for this camera
-3 ways of controlling the manual zoom
-250,000 pixel viewfinder (shows lots of detail, better for critical HD focusing. But use a monitor if shooting a "movie")
-low light performance: Sony says minimum lux for the HDR-FX1 is 3, and "falls between their DCR-VX1000 (with a minimum lux rating of 1) and their DCR-HC1000 (with a minimum lux rating of 5)"
-2 channels of audio. That's it.
-fully native 16:9: "native 16:9 CCDs, viewfinder, and LCD screen"
-Reviewer likes the HDR-FX1 better than the JVC JY-HD10 HDV camera or the Panasonic AG-DVX100 or the Canon XL-2. -optional shoulder mount/brace around $400

Mike says: This is the camera to get for indie digital moviemaking. Period.

Read the full review, tons of great details and photos, this is just a quick skim.

So I continue to doodle with low cost HD solutions. There are some new products, new configurations, and some new analysis in this posting.

If you wanted to put together a system capable of letting you produce a moderately sized uncompressed HD work, including high quality monitoring, it could be done for as little as what follows.

Sufficent for a SHORT work (final TRT an hour or more? Depends on how much rendering done!), using offline/online codecs:

Dual 2.0 GHz G5: $2500
more RAM: $200 (3rd party, of course! Crucial.com or whatever)
HDLink for HD previewing: $700
Apple 23" LCD: $2000 (ouch, expensive in this package)
-OR- get the HP 23" LCD for $1600 or so...not sure this is gonna work, but interesting
DeckLink HD: $600
two cheapo PPA, Inc. external SATA cases: $50/ea, $100 (Fry's electronics)
4 Maxtor Maxline III 300GB drives: 4x$230=$920
1 FireWire 800 drive to boot from - $200
1 decent 19" monitor: $240 (such as Sonic G90f, can do up to 1920x1440 res)
Final Cut Pro HD: $1000 ($400 education price, isn't it?)
SoftRAID formatting software (yeah, you DO need it): $100
La Cie 500GB FireWire drive: $500 (for offline codec storage & work)

TOTAL PRICE: $8660

This is the rock bottom, basement floor kind of a system I'd even consider using.

If it were me, I'd add the following:

another 19" CRT: $240
Matias tactile pro keyboard: $100
ShuttlePro v2 jog/shuttle: $100
yet more RAM: $200 or more
use a RocketRAID 1820A instead: $100 more
drop the cheap cases, get a 4 bay Burly Box from MacGurus as well as a 2 bay model - add $235
buy two more drives - $460
...and you stll have 2 unused slots on that 8 port SATA card for more drives.

Now you have a nice 3 monitor system, with 1.8TB of RAID 0 space (back that data up!)

Total price of improved system: just over $10,095

Oh, excuse me, gotta use marketable numbers. Drop the keyboard:

Now it's $9,995

This is the thiiiiiiiiiiin solution for a number of reasons - You might get away with this....or you could face disaster if a drive ate a bearing during your production. You'd have to recapture ALL your footage at the very least.

Further tweaks - you could cost effectively bump up to a second 4 port Burly Box for only $100 more, I'd want to install the drive coolers to ensure drive longevity (add $30/drive mechanism), and I'd want backup for all my data - cheapest way would be 500GB La Cie FireWire 800 drives - you'd need 4 or 5 of them depending on array capacity. 5x$500 each is $2500 more.

(For the Ultimate Big Kahuna Client Impressive recommendation, see here.)

It's a total roll of the dice - you never know when something might go wrong. I've been running a 4 drive Barracuda 7200.7 array for over 4 months without any trouble. (Seagate 160GB drives, the model Apple shipped standard in Rev A dual 2.0 GHz G5s) But a drive could fail whenever, unexpectedly, and I'd lose all the data on it. But I'm covered - the crucial client project footage is backed up, and only some test render stuff is stored on the array not backed up. If a drive shagged, I'd be irritated and dissapointed, but not freaking out.

...such as my friend the editor did the other week, when on deadline for a major cable network project, the drive with ALL the audio decided to give up the ghost. It took them about 30-60 hours of labor to go back and recapture from DV all that stuff. And since somebody in the workflow had decided not to bother with timecode on some converted Digibeta footage, they had to hand match a ton of stuff. UUUUUUUUUUURRRRRRRGH. Pain...and that was only a single FireWire drive, not part of any multi-terabyte array.


Another note: as I chug through my ever expanding Excel sheet with all this data, I'm realizing that the bigger Hitachi 7K400 400GB drives are incredibly cost ineffective - they literally cost TWICE as much as the Maxtor Maxline III drives, but only provide 1/3 more space. Plus they also are slower, especially towards the end of the drive where all drives get slower. This can limit the kinds of footage you can safely work with to the maximum capacity of your disk array (especially in 4 drive configurations).

If you're trying to configure a large, fast array, the ONLY reason to choose the 7K400 drives over the Maxtors is if you need more than 2.4TB of space on that 8 port card. The Hitachis are just very cost ineffective. If I needed 3.6TB, I'd strongly consider the possibility of building 2 6 drive arrays (3 4 bay enclosures) attached to two separate 1820A cards. But then I'd have to literally cut a hole in the back of my G5 to get the cables out since their wouldn't be an empty PCI slot to sneak the cables out of.

Feh. My warranty expires in the next couple of weeks on my dual 2.0 GHz G5, anyway....

; )

-mike

Firmtek has finally wised up and released a version of their two port SATA card with two external SATA ports, so it is no longer necessary to thread the SATA cables from the internal (inside the G5 chassis) ports through an open PCI chassis in order to plug in external drives. For those wanting to assemble a 4 drive array on the cheap, this makes life much simpler and easier.

Other features:
-can hot swap drives
-no driver, just plug in and go (card and drives)
-can do RAID 0 or 1
-support for large drives

Available Q4 2004, $99 suggested price. The prior model with internal ports (Seritek 1S2) sells for about $65 online.

This is good news for those who want to do short form, uncompressed HD work.

This card, external casing for two drives (such as from PPA, Inc., Granite Digital, or MacGurus), two internal drives, SoftRAID 3.x for formatting, and an external FireWire drive to boot from is the lowest cost solution I'm aware of that allows for uncompressed HD capture.

With drives such as the Maxtor Maxline III 300GB, a 4 drive array could capture 1080i60 8 bit 4:2:2 uncompressed HD footage (such as HDCAM) for most of the capacity of the array (drives get slower as they fill and start writing data to the slower inner tracks of the platters).

Such an array could also handle 10 bit 4:2:2 1080p24 footage until the array was almost full.

For HDCAM SR RGB 4:4:4 10 bit footage, some amount of footage could be captured and played back, but probably only the first 1/3 or less of the array could handle it - thereafter you'd start dropping frames.

But you get the idea - cheap speed.

-mike

The Highpoint RocketRAID 1820A SATA RAID, that has been popular on the Windows side of the fence, finally got Mac drivers last week.

In my initial write-up, I discussed the possibility of setting up large RAID 5 arrays for HD usage.

Well, not so.

In short, RAID 5 won't work for HD capture but can work for playback, and while it is possible to put two of these cards in a G5 in slots 2 & 3 with an HD card in slot 4, RAID capacity would be limited to 4TB total. "Whaaaaaaaa?" as John Stewart would say. Read on for details.

Because the 1820A relies on the host computer to do all the hard math of RAID 5 implementation, write performance pretty much blows as the G5 struggles to generate parity data on the fly. What's that mean? That means I'm back to my old standby recommendation of Go With RAID 0, But Buy A Bunch Of FireWire 800 Backup So You Don't Lose All Your Data If One Drive Fails.

In the Clever But Desperate category, one could have two arrays - one for capture, one for playback/edit. But then both RAIDs would have to be no more than 2TB each. Bummer. Whaaaaa, again - due to obscure technical limitations, if you have two 1820A cards installed, they both have to be no larger than 2TB each for it to work. Don't ask why, I don't know, seems crazy to me too. If you know why, please let me know. However, with a single card installed you can exceed 2TB no problem.

The read performance should be fine (I'll find out soon enough), but because the G5 has to do all the heavy math of the RAID 5 stuff when writing, and BareFeats is reporting that a 4 drive array only generated 50 MB/sec writes. Whenever you had to render something, it would take longer...or would it?

This from a BareFeats.com article

We ran the G5 using a single, empty Hitachi 7K250 SATA drive (55MB/s) as the FCP scratch volume. Then we used an empty four drive SATA RAID 0 array (220MB/s) as the FCP scratch volume. No change in render speeds. You would have to be using a really, really slow drive for FCP scratch to see render speed affected.

But since the CPU would be doing double duty (RAID 5 parity data generation and rendering of the effects) it would certainly take longer than usual. How much longer? I'll know in a week or so.

Tomorrow I'm going to build a 2.4TB array and start messing with it and testing it. I am returning my demo loaner DeckLink HD Pro card and HDLink, because I liked them so much I bought my own (after the price drop - whew!).

I'll post test results and notes as I go.

-mike

Reader Maarten Somers is attending IBC and wrote in with this report:

- Matrox HD solution: is not coming as a turnkey solution. They just sell
the video card and breakout box. The card remains the same for SD or HD,
just changing the breakout box will give you HD.

Prices: the SD version is about 5000 euros, the HD is about 10000 euros (yep pretty
expensive) and -in the contrary what you said in your blog- it's WITHOUT the
pc-system!

- I've been mailing with Jeff Kreines and the Kinetta Camera IS coming.
You can expect production units around beginning 2005. They will probably
even have 72 frames/seconds.

- The Blackmagic booth at IBC was completely run over! It was incredible
crowded... I think it says enough about their success.

- Nucoda grading solution was incredible. Those guys have a great working
product out there that can definitely be used for feature films. It runs on
a regular PC with Windows XP and is rock solid! The price tag however is
about 150.000€! Really expensive, but I have no knowledge what a lustre
would cost.


Mike says:

So eek - the Matrox HD stuff sounds very powerful, but very expensive. I'd want to see a side by side feature analysis to see if it cost justifies. For broadcast markets this may well be the thing, but for indies it seems like feature overkill. How many spinning chrome 3D logos do YOU intend to use in your sensitive tale of a boy coming of age?

In a prior email, I had stated that the Kinetta camera was vaporware and not something to plan on using. I think Jeff Kreines has a great idea, and I'm just waiting to see if it can actually come to market. I have high hopes for it. But it isn't shipping yet, so I don't consider it a viable option at this time. Hopefully he'll have shipping, field tested product by next NAB.

BlackMagic continues to rock the house with their new products and pricing. See breakdown and analysis of my discussion on their new stuff. Their 4:2:2 DeckLink HD Pro should work very well for the needs of the vast majority of the market.

Nucoda - yeah, I hear it's really great and an amazing new low price point for 2K film color correction work. Only $150,000!!!!

.....eek.

-mike

RaveHD, maker of Linux based, codec agnostic DDR, has updated their product with a new user interface.

Unlike other hard disk drive based digital disk recorders, RaveHD is based on open standards, standardized hardware, and Linux.

Read about it here.

UPDATE FRIDAY

I hadn't noticed that they'd dropped the price for the software only version from $8000 to $5000.

The system is really geared towards VFX workflows, not editing workflows. But the more I learn about it, the more I like it (in theory).

This is from a press release I received today:

RaveHD, used by some of the top studios in the world is the first Linux based disk drive recorder to open it's source code to it's clients. Priced at $4,995.00 this software package is geared towards studios who require high throughput of SD, HD and Dual Link formats. Using a standard file system and a frame base concept RaveHD is able to communicate easily with other software packages and to be easily integrated into the pipeline.

Input from Studios such as Tippett has produced a software package that is designed for visual effects and the flexibility required in doing them.

The demonstration will focus on the database functionality of RaveHD, capture, VTR, playback and the new quicktime to frame/frame to Quicktime conversion tool. We will also cover in detail how RaveHD can be used in the workflow. This as a hands-on demonstration and you should leave with a clear understanding of this new product and how it is changing the way studios think about HD.

Don't miss this opportunity to see RaveHD in action and talk with lead programmer Jason Howard. Seating is limited so please RSVP to ramona@spectsoft.com for location details.

Updated Tuesday morning - see bottom

See this prior posting about the HDR-FX1 PAL version some sample footage from it.

I'll regurgitate what I wrote earlier:

I don't know when then plan to ship it, but if I interpret their stated "worldwide by end of year" I'd read that as, well, by the end of the year. So if you shot 1080i50, then did field blending (with Cleaner, After Effects, Magic Bullet, Compression Master, or SOMETHING), you'd have 1080p25. That's cake to conform to 24 fps for theatrical release. It's not perfect, but it beats the hell out of the other $3500 options, such as DV PAL (720x576). Even if you assume that field blending halves the vertical resolution, it's still a true source of 960x540 - worst case scenario that's still 20% more resolution. And in truth it's a lot more, but I'm not sure how to to the math off the top of my head. I know it's at least that much better. If you just dumped a field, it'd be 20% better. But since you're smart field averaging, it's better than that.


And at about the same price, that rocks the house. For now, it looks like sound might be an issue with the HDR-FX1 until the pro version with XLR audio inputs comes out. In the meantime, if the FX-1 lacks even minijack inputs, you could (ugh!) record separate audio and sync it in post (suck suck suck - thought we were past that!). Hopefully you'll have some international projects so you can shoot MOS (mit out sound - without sound - no audio).

OK, now THIS is huge for indie filmmakers. Process would be this:

Buy a PAL HDR-FX1

Shoot your footage

Get your footage onto your Mac with Lumiere HD

Somewhere in there, process the footage to do the smartest interpolation your software will do to convert 50 interlaced fields per second into 25 progressive frames per second. Here's the list of potential tools to use off the top of my head: Cleaner, After Effects (and plugins for After Effects: Algolith, Re:Vision's ReTimer or whatever they call it, Magic Bullet Suite HD, etc.), Compression Master, Compressor, DeBabelizer, Shake, or other application that will do an intelligent field merge/blend/average/interpolate/mash. Or if that proves onerous, just post process your final movie at the end with one of the above tools.

Edit away with that new footage. If you need to, use the "Recompress" function in Media Manager in Final Cut Pro HD to create an effective offline codec'd version of your footage for space savings and realtime functionality. DVCPRO HD 1080i50 willl be available on the "next major version" of FCP HD, I'm guessing in the first half of next year (either MacWorld in January, or more likely at NAB in April).

When done editing your 1080p25 project (even though the computer may be acting as if it's 1080i50), do the complicated conform to 24p.

How complicated is it? In Final Cut Pro HD, go to the "Tools" menu and select "Conform 25 to 24"

Tough, ain't it?

;p

(shoot me now, I used a smilie in my blog!)

Now that you have a 1080p24 master, it's cake to make a world master of this thing. Use your 1080i50 project (you saved that separately, right?) for PAL markets, then convert your 1080p24 to 1080i60 using a classic telecine style conversion. This is easy with Final Cut and/or a variety of other tools. You can also make a DVD master from this 1080p24 master, but I wouldn't do the straight export from Final Cut Pro timeline (it does a sucky quality downconversion). I'd render it out to separate file and use a tool with better downsampling (Cleaner, Debabelizer, Compresion Master, After Effects etc.) to kick out a 720x480 anamorphic master, then compress MPEG-2 from that.

I predict that in much the same way that PAL DV has been the smart choice for a few years for those seeking to create a theatrical master, PAL HDV will be the new smart, low cost option, but with much better resolution.

Now put that in your indie pipe and smoke it.

-mikey, smug "il putard" of the digi-post world

UPDATE TUESDAY MORNING: I don't recall seeing anywhere that the camera had analog HD monitoring outputs. If it doesn't, how on earth can you accurately focus? The LCD viewfinder? No way! You're stuck going back to "measure to the lense" techniques, using the markings on the ring. And measure to exactly where? Another good question. Focus pulling? Hello?

UPDATE TUESDAY EVENINGAccording to somebody, they claim somewhere on avsforum.com that there was an updated Sony PR bit about analog component outputs....this would help the monitoring issue...although HD-SDI would be better, this'll work.

Scroll to the bottom of the first post, and read the instructions. I felt it would irresponsible to just post a direct link.

NOTE: This footage was from a prototype of the PAL version of the HDR-FX1 camera - so clearly one will be in the works! I don't know when then plan to ship it, but if I interpret their stated "worldwide by end of year" I'd read that as, well, by the end of the year. So if you shot 1080i50, then did field blending (with Cleaner, After Effects, Magic Bullet, Compression Master, or SOMETHING), you'd have 1080p25. That's cake to conform to 24 fps for theatrical release. It's not perfect, but it beats the hell out of the other $3500 options, such as DV PAL (720x576). Even if you assume that field blending halves the vertical resolution, it's still a true source of 960x540 - worst case scenario that's still 20% more resolution. And in truth it's a lot more, but I'm not sure how to to the math off the top of my head. If you just dumped a field, it'd be 20% better. But since you're smart field averaging, it's better than that.

They sampled it down to about 5 megabits/sec (660 kilobytes/sec) Windows Media 9 at 1280x720, but it gives you a starter idea of what the camera can do. This came from some folks at Sony, so it's Official Demo Product, so take that with a grain of salt. I was going to say something cynical about how this might have been color corrected on a Smoke or Flame, and then I read this:

"No color correction was used- footage is straight off the camera."

...and I am all the more impressed.

The shots, especially the white flower in daylight (with no blowout to white) show off the light sensitivity and dynamic range of the camera - and it's impressive.

But even with this downsampled 720 res footage, there is a softness to the focus. I had just earlier today messed with some HDCAM footage that had been captured uncompressed straight from tape that had been shot on a Sony F900 (project I did last summer), and the focal clarity was just stunning. This camera, on the other hand, uses lower quality lenses, and it's CCD only has 960 pixels horizontally (1/2 the final display resolution).

Nevertheless, it beats the pants off any DV camera costing 2-3 times as much you'll use in terms of detail captured to tape - it captures 960x1080 pixels, rather than 720x480. That's 3 1/3 times as many pixels. Especially when the "pro" version ships next year at the $7K price point, this camera will be CHOICE for low budget indie filmmakers. FINALLY, there will be SOME middle ground between the $4000 DV cameras and the $65,000 Varicam, or the 6 figure price tag of a fully configured Sony F900 or F950 camera. Granted, the higher end cameras make prettier images...but at 20 times or more the price, are they 20 times better? Ah, no.

Granted, I'd shoot on a Viper Filmstream if I could...but I doubt I'll be able to afford that in the next year. This other camera I could just go buy if I felt like it and start shootin' stuff. Use Lumiere HD to get it into Final Cut Pro HD. In fact, I may just go ahead and decide to do that...

-mike, feeling whimsical about credit card debt today

So I'm doodling around with some workflow ideas, and it hits me -

Media Manager in Final Cut Pro HD has a "Recompress To" function built into it. You can use it to convert from one codec to another.

A couple of quick tests revealed:

Converting 1920x1080 8 bit 4:2:2 BlackMagic 2Vuy codec (their 4:2:2 codec) footage (originally shot on HDCAM, captured uncompressed) to DVCPRO HD 720p24 can be done...but the downconversion math is sloppy - various scaling artifacts remain. Could be done for offlining purposes if you had to, though...and it does not require the presence of a second deck to dub it..

Converting 1080i60 BlackMagic codec footage (again, shot HDCAM, captured uncompressed 1080i60 4:2:2 8 bit) to the DVCPRO HD 1080i60 codec WORKS. Conversion of a 20 second clip took roughly 2 minutes on a Rev A Dual 2.0GHz G5. So that's about 6:1 render time. One hour of source footage would take six hours to convert, and so forth. But it could be done in an unattended fashion overnight. The footage looks quite good, and the only thing I've noticed so far on the single clip is that the conversion makes the footage LIGHTER as played back out via HD-SDI. I'm monitoring the actual HD signal via HD-SDI, so it's not a QuickTime "computer screen only" thing. But the shift is mild and probably correctable with a realtime color correction adjustment of some sort. Certainly not enough to get really upset about, but enough to want to color correct for if I cared much about final quality output. I'll see if it's clipping any values too at some point.

Interestingly, the two formats look identical on computer screen, it's only on HD monitor do they look different. Perhaps something in the BlackMagic card? I wonder if I converted back to BlackMagic codec if it would go back to looking the way it was before....yet another thing to test later.

So this offers a very interesting workflow - since the BlackMagic cards cannot at this time (as the Kona2 claims it can/will) convert on the fly during capture from uncompressed HD-SDI to DVCPRO HD written to hard drive during capture, you could always capture to uncompressed, flop all clips on a timeline, and then Media Manager Recompress to DVCPRO HD, and all your logging data would be intact.

So workflow could be something like this:

-Log and capture all day your HDCAM 1080i60 footage (or until your array is nearly full) on a SATA RAID, cost $1000 - $3000 depending on capacity and casing options.

-Then at night (or over lunch) use Media Manager to Recompress to DVCPRO HD 1080i60.

-When you return and check that it worked correctly, you can toss the uncompressed footage or copy offline and shelve it for relinking later if you want to finish uncompressed, not rent a deck again, and you have tons of drive space sitting around.

-Edit your way merrily along with the relatively low bandwith (14MB/sec) DVCPRO HD footage. You now have realtime color correction, effects, etc., and can monitor out of a BlackMagic DeckLink HD family card (haven't tried a Kona2 yet for this).

-Lay it off to DVCPRO HD via FireWire to an AJ-HD1200A deck if you wish, or to a Sony HDCAM deck via the HD-SDI on your DeckLink HD family card.

This way, if you had, say, 40 hours of footage, it could be stored on 1.8TB of storage.

Devil's advocate position: since the RAID 5 only writes about 50MB/sec with the Highpoint RocketRAID 1820A card, but reads around 200 MB/sec in some configurations, you could use it as "safe" storage space.

Or consider a FireWire 800 hardware RAID 5, such as the eRAID5 from FireWire Direct. That unit even supports clustering, so that two machines physically close together can share access to the same array via FireWire 800, such as a log/capture station and an editing station. Or just use Gigabit ethernet to connect them.

Anyway, yet another way to save money.

The only downside to this is that it ONLY works with size and framerates supported by DVCPRO HD...and 1080p24 (1920x1080, 24 progressive frames per second, what you'd want for theatrical release film stuff) is not one of them.

Camcorder Info has a report from IBC and it included this snippet:

"Sony was showing a professional version of the HDR-FX1. The model does not have a name yet, and basically looked like a darker black version of the HDR-FX1 announced earlier this week. The only discernable, major difference is the inclusion of two XLR audio inputs, and a second manual audio dial control wheel. Sony would not officially say if the camcorder has 24P. However, we got a sense that it would not include this feature. With no official price available, a range of around 6,000 euros was suggested. Sony was also showing an HDV deck, though they did not have a model number or information on its availability"

This seems to confirm that the standard model does not have XLR inputs as I already thought. I had a reader write in to say an Apple guy he knew thought that at least were mini (1/8th inch, like headphone jacks) line inputs on the HDR-FX1, but I haven't heard any solid confirmation beyond this thirdhand "think so" level of evidence....and the press releases have made no mention of it in the feature list. Harrumph. But it's still a pretty groovy camera, and the pro model would be a lovely little digi-indie on the cheap solution even without true 24p (and the pro model I'd think would also include the CineFrame feature) if used with Magic Bullet Suite HD, for instance. If films have been made from DV's 29.97 interlaced up to film (poorly, low commercial success unless your last name is Von Trier), why not use higher res DV for an improved product (even if destined for SD (standard definition) delivery?

-mike

I've received some emails, and in comments I see around the web, about the technical density of this blog. Yeah, I don't slow down to explain concepts very often. Sorry. I need an FAQ, but don't have time to stop and write one. I need a wiki for this stuff too. Someday for both.

But in the meantime, others have done some nice work in this regard - I direct you to the top right side of this site to the section titled Links. These are the best, most information laden, technically accurate sites I've found. Each has a wealth of useful information for those seeking to shoot and post HD, and several have deep, rich veins of FAQ knowledge to be mined. I HIGHLY encourage anyone serious about HD stuff to thoroughly dig around on these sites.

If you can smugly answer that you already have, look again - based on a lot of requests for camera information, I've added a link to CamcorderInfo.com.

And more important than that, if you know of a good site for good deepgeek info on HD matters, please email me at mike@hdforindies.com and let me know! This site is all about sharing the wealth, so please help me do so.

I want to say thanks to the half dozen or so readers that have sent in links that led to postings on the blog in the last week or so - very much appreciated, and all are welcome to do so.

-mike

At IBC on Friday, an Apple honcho "announced that the next major release of Apple’s Final Cut Pro will support Panasonic’s new ADX4000 1080i 50FPS camera and P2 tapeless DVCPRO-based solid-state memory products."

I can't find any reference to this camera on the Panasonic site or via Google search. No idea on price, quality, or shipping date. So please take all that follows as one big slice of conjecture.

1080i50 could be hornswaggled into a 1080p24 without too much trouble. Funny, not two hours ago I was suggesting to Frank Reynolds (I'll post further updates on that conversation soon) that Apple will eventually support 1080i50, which someone could make a 25p mode for, which could work with minor modifications to work for authoring 1080p24 footage, even if only using it as an offline codec (maybe even online) with realtime effects AND a relatively small datarate (11.2 MB/sec). Even shooting 1080i50, just using smart field interpolation would yield good results as compared to Varicam. Varicam only records 960 pixels horizontally and 720 vertically, for a total of 691,200 pixels. Presuming the ADX 40000 is recording to DVCPRO HD (a pretty safe assumption), it will record to tape 1280x1080 pixels, for a total of 1,382,400 pixels. But those 1080 pixels will be field averaged or interpolated. Worst case scenario, you're getting only 540 vertical lines of resolution (it's really better than that, but we'll start there). So that would be, um, 691,200. Woops, back where we started, but worse - the human eye is more sensitive to vertical resolution than it is to horizontal resolution, so the distribution of these pixels (1280x540) isn't as optimized for viewing as Varicam (960x720). Interesting.

So it's all up to Panasonic to make a 25p mode with this camera. Will they do it? I dunno. Will 1080i50 processed in software to 1080p25, then time altered to 1080p24, look any better overall than 720p24 from Varicam? I think maybe a little better, but it's a LOT more hassle and work.

If no 25p mode, careful analysis required to see which camera (Varicam or ADX 4000, assuming the glass/optics/CCDs are quality comparable to Varicam, that's a LARGE assumption since I have NO further info on this camera) looks better.

Just doing a little calculator math, 1080i50 should be a touch over 11 MB/sec as DVCPRO HD codec. Presently, 720p24 is 5.7 MB/sec. So SHOULD end up with better result, even with field blending.

If the camera DOES have a TRUE 25p mode (NOT just dropping a field out and duplicating the other, but actually recording 25 progressive frames per second), then we've got a hit - a camera that shoots a good image, and a post workflow that's very fast and inexpensive-you could in theory cut a feature on a PowerBook and some FireWire drives. Yeehaw!

I hope Panasonic did it "right" as I see it. But it would be perfectly understandable from a price and technology perspective if the camera had no progressive capabilities whatsoever. Understandable, but irritating.

-mike

Automatic Duck has used IBC to roll out new versions of it's timeline integration tools. "This powerful new solution allows digital media content to be imported into Adobe Premiere Pro AAF, OMF and XML files, enabling Premiere Pro users to finish projects offlined on Final Cut Pro or Avid editing systems." Also of note, at IBC they will "showcase its next generation timeline integration engine software for users of Adobe After Effects. Automatic Duck Pro Import AE 3.0 is the only timeline integration solution in the industry able to import media files from Apple Motion, Apple’s newest real-time motion graphics design solution. Additionally, Pro Import AE 3.0 enables import of XML files from Final Cut Pro."

OK, two products here, both interesting for their own reasons. I'll start with Pro Import AE 3.0. Earlier versions of this tool have been around for some time. Automatic Duck specializes in tools to let editing applications (first Avid, then Final Cut Pro, now Premiere Pro) get the editing timeline, with all the tweaks (edits, heads, tails, PIP, FX, etc.) into an effects program, like combustion or After Effects. So this new version of Pro AE Import will let you move your whole timeline, not just a rendered movie, from Final Cut into After Effects. This is GREAT if you have a lot of subtle color work to do, or a lot of effects, or basically anything where you still want SOME degree of editing control but to be able to harness the power of After Effects on your footage. For the kinds of intense motion graphics and compositing heavy work I've been doing for the last 8+ years, this is an INCREDIBLE time saver and boon to workflow.

The problem is that After Effects "blows chunks like a rocket launcher" as an editor - ALL previews have to be laboriously rendered to RAM to be viewed, even for just straight footage - NOTHING "just plays back" in After Effects. So you have to be pretty much DONE with your edit, except for minor minor minor tweaks, like trimming a few frames on the transition, when you move to After Effects, or it's a big, huge, slow pain.

My first 1080p24 HD project, done summer 2003, was shot HDCAM, edited offline, then the creative director (technically competent enough to be dangerous) insisted on NOT buying Pro AE Import and manually recreating the edit in After Effects. Big drag - he had to by hand and by eye, match the in and out point on each shot from Final Cut to After Effects, manually importing each clip and lining it up. Ugh. He thought we'd only have to do it once and be done with it. But of course, the client wanted to tweak the edit later, so we had to try to edit in After Effects again...and again...and again. HUGE slowdown pain in the butt.

So what to do about this?

Automatic Duck PPro Import lets you move a timeline from Final Cut Pro into Premiere Pro. I'm a bit of an uneducated Mac snob when it comes to Premiere Pro - old Premiere frankly sucked for professional usage, and the Pro version I hear is much much better, but I just don't know much about it. I need to learn more about it. But one of the things it does that I'm jealous of is share the rendering engine of After Effects, which rocks. You can actually select a clip in Premiere Pro, click copy, go to After Effects, click paste, and do your After Effects stuff to it. Let's say you duplicate the layer, mask the copy, set up keyframed effects on the copy, all kinds of stuff. When you're done setting up all your keyframes and whatnot, highlight all the layers you've created and used, click copy, go back to Premiere Pro, and then paste it back into the timeline at the correct place. You can then render it right there in the Premiere Pro timeline, so no render/import/manually place and align process as we used to do. You can still edit the keyframes you've set WITHIN THE PREMIERE TIMELINE. How killer is that? Keeps it all tight and integrated.

So my back of the head daydream for desktop Digital Intermediate (or Digital Mastering) workflow so far goes something like this:

Shoot/acquire your footage however.

If not at first, eventually capture it to an FCP rig at anything up to 10 bit 4:4:4 1920x1080. Offline at a variety of lightweight, full resolution codecs.

Do your editing.

Batch capture to final quality if necessary.

When done editing, either get the timeline into AE for final tweaky/post/effects work (but you're married to AE for output from then on!), or better yet

when done editing, port the timeline to Premiere Pro. Obviously, a lot of careful research has to be done to see what features will carry over correctly. Color correction? Masks? Split screens? Animated variables? All need to be confirmed that they not only port but function exactly as intended in the Premiere Pro timeline.

NOW the post guys can copy/paste clips as needed into After Effects for their work, checking it in the context of the timeline. Work can be done at 16 bits per pixel (I think, need to verify) and the high end rendering output capabilities of After Effects/Premiere Pro timeline can be utilized.

So why not Premiere Pro from the start? I dunno, I don't trust it because I don't know enough about it yet, and it may not have all the features I need. Plus, then you need a loaded-for-bear PC to run this stuff on, including storage and whatnot.

It'll be interesting to learn more about this stuff.

This is just my musings to date, not anything I'd recommend for sure to a client at this point. And again, every project has it's own needs.

Porting FCP to Premiere Pro could be tougher than just knowing what is and isn't supported correctly (and that's a MAJOR challenge!). It also involves making sure codecs work cross platform and aren't going to get messed with (color/gamma/luma shifts, etc.) crossing platforms. Some codecs are platform specific - DVCPRO HD codecs are NOT available, nor likely to be anytime soon, on the PC. So reprocessing to other codecs is a pain as well.

But portability, flexibility, and choices are very nice things to have in a post production workflow.

-mike

Apple has a page up on their UK site listing a bunch of announcements of interest for Final Cut Pro HD users. Among them (but not all inclusive):

An Apple guy "announced that the next major release of Apple’s Final Cut Pro will support Panasonic’s new ADX4000 1080i 50FPS camera and P2 tapeless DVCPRO-based solid-state memory products."

I'll have a separate post on what this means shortly.

Automatic Duck has new versions of it's timeline integration tools. "This powerful new solution allows digital media content to be imported into Adobe Premiere Pro AAF, OMF and XML files, enabling Premiere Pro users to finish projects offlined on Final Cut Pro or Avid editing systems." Also of note, at IBC they will "showcase its next generation timeline integration engine software for users of Adobe After Effects. Automatic Duck Pro Import AE 3.0 is the only timeline integration solution in the industry able to import media files from Apple Motion, Apple’s newest real-time motion graphics design solution. Additionally, Pro Import AE 3.0 enables import of XML files from Final Cut Pro."

Again, more about it in a separate post coming up.

AVC Alliance news "The H.264/AVC codec was ratified by the DVD Forum in June 2004 as the H.264 Advanced Video Codec to be included in the next generation High Definition (HD) DVD format. H.264/AVC is based on open standards and will ship in Apple’s QuickTime software in the first half of next year."

BlackMagic Design with all their new cards. I've already reported on this.

Logic Keyboards is showing a variety of color codec keyboards for applications including choices for Final Cut Pro HD.

Red Giant Softwareis showing a variety of cool things - Magic Bullet Suite, Magic Bullet Suite for Editors, and eLin (which I just wrote about here). All of which are cool and have uses for HD. I've posted on these in the past. Magic Bullet Suite (for HD, only for After Effects at this point unfortunately) could be quite useful in conjunction with the Sony HDR-FX1 mentioned below to shoot 1080i60 and convert it to 1080p24 for theatrical distribution (or for a more film-like look).

Sony used IBC to debut their $3500ish 1080i prosumer camcorder, the HDR-FX1, which I've written a couple of articles about in the last week, here and here.

OK, I'm breaking my rules here. This isn't really HD, but it is 24p. Camcorder Info has this article comparing the XL-2 to the DVX100A Lots of details, check it out if you're into the 24p standard definition action.

Both cameras can shoot 16:9 24p to DV tapes using a telecine like transfer to get 24 progressive frames per second (film frame rate) onto 29.97 interlaced frames per second (video frame rate) video tape.

-mike

So I got a letter from a reader in Belgium trying to piece together how to do a pure digital workflow.

Basically, he was thinking of shooting with the Kinetta camera (doesn't exist yet), capturing 10 bit 4:4:4 straight to hard disk, never touching tape, and storing on SATA RAID with something like the Burly Box I mentioned in my big "how to" post the other day. A 15:1 ratio for a 2 hour feature would yield the need for 18 terabytes just to hold the source captured footage.

Some feedback on that theory, not to beat up the guy but to get down to brass tacks of reality:

1.) Kinetta camera is total vaporware at the moment. F-950 or Viper Filmstream are the viable options at the moment.

2.) SATA RAID only scales, at present, to a max of 3.2 TB of HD usable space. Even then, it is RAID 0, so if any one of the eight 400 GB drives fails, you're data is toast. So back it all up to FireWire drives. In increasing order of size but decreasing storage cost efficiency, I'd recommend the La Cie line: the Big Disk (500GB, $500), Bigger Disk (1TB, $1200), Bigger Disk Extreme (1.6TB, $2200). But once you've shot/captured all your footage, it only has to be backed up once.

3.) If you really want to keep this kind of quantity of data online at the same time (20TB and up), I'd recommend a bunch of X-RAID units set up as RAID 30 (maybe 50?), striped in way to provide sufficient throughput. Truthfully, with this amount of footage you'd need something in excess of 25TB I'd bet by the time you're done, but let's start there. 25TB worth of X-RAID RAID 30 or 50 would cost something like $100,000. So bag that right off the bat if you're an indie.

What would it cost to store, even literally offline (as in sitting unplugged on the shelf, NOT reduced quality)? Normally 10 bit 4:4:4 1080p24 footage is 625 GB/hr. 30 hours is about 19 GB. Eek. That'd take about 20 or more La Cie Bigger Disks at $1200 apiece. Cost: $24,000. OK, that's better but not great. But then woops, it is vulnerable if that's your only copy. To back it up, another set of disks, another $24K. Would it be better to use the Medea G-RAID ($700 500 GB FireWire800 2 drive RAID) in RAID 1 to store data?

So, what are the options? If you wanted to insist of pure digital acquisition straight to drive, it can be done. (pros: highest possible quality, already logged & captured, no timecode backmatch issues. cons: huge storage costs, always tethered to a big road case of drives on set that requires an electrical plug (car chase, anyone?)

As for field hard disk capture, I'm working on something, I'll have more to say later.

The alternative: HDCAM SR 4:4:4 double rate on Sony SRW-1 deck. Price is N/A on the site, so I don't know if they are available to mere (well funded) mortals yet. Cost/hour: $200-$225 for media (tape)

So why even consider the DFR (digital field recorder) route? Because the DFR will cost a LOT less to rent per day as compared to Sony most likely. There are other benefits as well. More later.

Right now, untiil hard drive prices get cut in half or more, tape is cheaper, but still has it's hassles and issues.

OK, back to the reader stuff:

4.) If you shoot with Viper, it's logarithmic not linear footage data. This means it doesn't look right without adjust the way the colors look. HDLink previewing through an Apple 23HD can do this, however, by loading a custom CLUT.

5.) If you do go the full digital route, asset management, capture/storage/offline/online codecs and workflows are CRUCIAL to maintaining quality and getting things done in a reasonable timeframe and reasonable cost.

6.) Post workflow is funky and careful - for instance, full 4:4:4 is possible on the desktop with FCP, but you get no realtime effects for most tasks (all tasks? Gotta check), so LOTS of time will be spent watching render progress bars, and LOTS of space will be required. It might even be viable/recommendable to do things like process 10 bit 4:4:4 footage down to 8 bit 4:2:2 footage just to get realtime effects, then Media Manager relink at the end and do the Big Pfhat Final Render.

7.) Gratuitous self plug: And if you are, get yourself a knowledgable post production supervisor. Ask a LOT of questions, get them involved VERY EARLY in such stages as choosing a camera, documenting workflows, doing tests, etc. I, ahem, am available for such things. I say this with caution, since the point of this blog is to disseminate information and try to avoid the BS middlemen and disinformation that is out there in the market. So while I am putting myself out on the commercial market, I'm trying to keep very clear and clean about putting good information out there in terms of what I think and what I have found to work in my own personal experience.

-mike

Frank Reynolds is a friend of mine, an indie editor who spends time in both Austin (my town) and NYC.

We've been discussing the pros and cons of digital editing for several years, ever since we were both on a digital editing panel at the SXSW Film Festival in Austin back around 2000 or so.

Our conversation recently was interesting enough I thought to warrant reposting on the blog, where my theory meets his practice. Here it is in chronological order. I frequently use a row of ---------- to separate quotes from his emails from my responses to his email. I'll use a row of =============== to separate emails between us.

==========================================
I kicked it off with:

In a message dated 9/9/2004 10:43:49 PM Eastern Standard Time, miked writes:

check my site - I just posted the how to make killer HD station (and update for sub-$10K HDCAM editing station
update, based on all currently shipping stuff. Read it and tell me what
you think.

-------------
Frank replied

It seems like a fine update, though you know about this stuff far more than I do...

The thing that caught my eye is that at best it can hold only 8.5 hours of 1080 24p.  That is nowhere near enough for a feature.  (If you can't hold at least 25 hours, I don't want to even consider it, and if it can't hold 40 hours, I'd probably consider it, but not use it.)  It's one of the reasons I never bothered to call back that job that wanted an editor to own his own 1080 24p system...it just seemed unrealistic. They actually said in the listing:  "We don't want to down-convert."  I mean, I'd be surprised if even Rodriguez and Lucas didn't use down conversions.  (I know Lucas was using Beta SP downconversions at the time of Episode II, but HD-wise, that's an eternity ago.  My ILM contact left, so I don't know what was done for Episode III.  Do you kow how Rodriguez works?)

On a system that can only hold 8.5 hours, I'd have to constantly recapture chunks of the movie to work on different sections...not only is that a pain in the ass (shades of Avid editing circa 1996, when hard drives were 4 gigs, and you worked at, like, 50:1 compression), but that meant I'd have to have a 1080 HD deck constantly at my disposal.  Big bucks.

I don't think editing features at 1080 full-res is there yet...hell, we only got easy feature editing at 720 full res three months ago...

==========================================

Mike replied

You said 8.5 hours wasn't enough - that's at final res. You do your offline edit using PhotoJPEG compressed 1920x1080 footage, which looks fair to great depending on the quality setting. 3.2TB will hold something like 200 hours of totally watchable PhotoJPEG compressed HD. You rent the deck and capture all your footage to offline codec. When you are comfortable with all your selects, go back, rent the deck again, and batch capture your used selects (FCP helps you with this).

Didn't you work this way with standard def projects on Avid ever?

==========================================

Frank said

Sure, and it was a major pain in the ass....especially during preview time.  We'd cut the movie at, like, 10:1 res, but they'd want us to re-batch-capture at a higher compression for screenings (2:1 or 3:1), then after the screenings, they'd want to make all these changes, which were usually extensive enough to have to go *back* to 10:1 to make them all, then go *back* to 2:1 for another screening, etc. etc.

I ain'ts going through that again if I can help it...

And I don't do "selects" anymore....I learned from experience: I want access to all of my dailies all of the time...take 2 may be 99% crap, except for that brilliant little reaction...

Just to give you an example: I ran a test on IN THE BEDROOM after we finished editing it...I kept every full take that we used even a moment of...maybe just a quick reaction, maybe an audio line reading, etc...but I deleted all the rest from the drives.  I still had half the dailies online.

besides, I'm sure within 18 months this'll all be obsolete anyway, and we'll be able to edit at 1080 final res all the time...that's what was so great about DV and FCP when it first came out: no more of these pixellated, single-field Avid offline resolutions....

==========================================

-----------
Sure, and it was a major pain in the ass....especially during preview time.  We'd cut the movie at, like, 10:1 res, but they'd want us to re-batch-capture at a higher compression for screenings (2:1 or 3:1),
-----------

If doing 24p work, using the PhotoJPEG codec is plenty good enough - if you capture at say, 75% quality to start with, the data rate is ballpark of 15MB/sec (varies depending on footage visual complexity) which is the same rate as the 1080i30 DVCPRO HD format. Curious to see what it would look like? Export a still from Photoshop and do a plain (not optimized) JPEG at whatever percentage and that'll give you a good idea of quality an filesize.

---------------
then after the screenings, they'd want to make all these changes, which were usually extensive enough to have to go *back* to 10:1 to make them all, then go *back* to 2:1 for another screening, etc. etc.
---------------

Capture at "good enough to look at" quality and edit with that. It'll be low enough datarate to capture lots and keep on single SATA or even FireWire drives.

I took a sample clip - the collapsing iceberg from the Apple NAB DVCPRO HD sample disc.

PhotoJPEG @ 25% sample: 2.6MB/sec (good enough to edit, wouldn't screen with it)
PhotoJPEG @ 50% sample: 8 MB/sec (mediocre for screening)
PhotoJPEG @ 75% sample: 15 MB/sec (looks quite good, screenable)
 
----------
I ain'ts going through that again if I can help it... 

And I don't do "selects" anymore....I learned from experience: I want access to all of my dailies all of the time...take 2 may be 99% crap, except for that brilliant little reaction...
-------------------

If you don't like that plan, consider downconverting to 720p24 DVCPRO HD - entirely screenable, broadcast quality format, and you get realtime effects, color correction, etc. (which you don't get with PhotoJPEG compressed footage). DVCPRO HD 720p24 is 5.7MB/sec.
 
----------------------------
Just to give you an example: I ran a test on IN THE BEDROOM after we finished editing it...I kept every full take that we used even a moment of...maybe just a quick reaction, maybe an audio line reading, etc...but I deleted all the rest from the drives.  I still had half the dailies online.
----------------------------

OK - so if you want 40 hours of footage, in DVCPRO HD 720p24 it's 800 GB of footage...that's a La Cie Bigger Disk for $1200, and you still have a bunch of room left over.

Assuming 15 MB/sec (for 1080i DVCPRO HD or 1080p24 PhotoJPEG@75%), you'd need 2160 GB - so two Bigger Disk Extremes (1.6TB each, $2200) - cost is $4400. Or a stack of La Cie d2 drives (500GB each, $500 each) for $2500.
 
-------------
besides, I'm sure within 18 months this'll all be obsolete anyway, and we'll be able to edit at 1080 final res all the time...that's what was so great about DV and FCP when it first came out: no more of these pixellated, single-field Avid offline resolutions....
--------------

The real fix for this is native codec editing - it would require working with the encoded format of HDCAM or HDCAM SR. Unlikely that Sony will play nice with Apple that way, since Apple and Panasonic are in bed the way they are, and Sony a.) makes their own PCs, and b.) makes it's own editing system (XPRI).

I think it'll take more than a year or two for uncompressed HD to be easy to store 40 hours on. Even HDCAM's quality - 4:2:2 8 bit - takes a bit under 350 GB/hr. So 40 hours is 14 terabytes - beyond what SATA can do on a Mac. Have to go Fibre Channel or SCSI for that.

And if you bump up to 10 bit 4:4:4, then it's 625 GB/hr, so 40 hours is 25TB. If we extrapolate current trends, we'll get there, but it'll take a while. I think it'll be, well, not too terribly long - 3-5 years before that is desktop viable I'd guess. We've gone from 250 to 400 GB within a year. I'd expect terabyte desktop drives to be announced within 2 years, 3 at the most. Rotational velocity and areal densities have kept increasing for years at a pretty good clip.

Some kind of PCI-Express card with a 2 GB Fibre Channel link, connected to a bridged interface to a bunch of Serial ATA (or other more advanced, fast, but inexpensive bus) will happen.

Say, this is really good stuff - do you mind if I post this letter on the site?

-mike

==========================================

Frank replied

In a message dated 9/12/2004 1:04:22 AM Eastern Standard Time, mcurtis@3beam.com writes:

Capture at "good enough to look at" quality and edit with that.

------------------

Yeah, but remember, we do our test screenings in *movie theaters.*  The filmmakers are going to want the best quality image possible, not "good enough to look at," to project 15 feet high.  When we cut films in standard-def, we projected the 2:1 Beta SP because that's the best we could do at the time.  (We couldn't afford a film workprint matchback.)  If the Avid we had could have done uncompressed, we would have done that.  I doubt the director and producers would accept a 75% 1080 hi-def picture to project 15 feet high if 100% is possible on the machine, even if that means more work and a pain in the ass for me....

To become the "grumpy old man editor" for a minute:  that is currently one of the few major drawbacks to digital editing in low-budget feature films that are shot on film.  When we cut on film, if we wanted to see it projected in a movie theater, just to see how it would play big, all we had to do was take the cut workprint to a screening room and thread it up.  (Film workprint is 1:1 uncompressed. ;-) ) Now we either have to do a workprint matchback from the digital cut list, which is expensive, or project the video, which doesn't always tell us the full story.  We did a lot of projecting of the video cut of IN THE BEDROOM because the director demanded it, and it helped us a lot, but even then, it's still not projecting the film.  In fact, after the film was finished and premiered at Sundance, we went back and made a change based on the fact that we now finally watched the *film* in a theater. (There's a scene where Tom Wilkinson is driving and looks at a license plate in the car in front of him.  In the video projected cut, the license plate was blurry in the wide shot, so we cut to a close-up of it.  But on the film, the wide shot was crystal clear, which made the close-up unnecessary, so we went back and cut the close-up out.)

Sure, posting the letter is fine with me.

Frank

 =============================================
Mike said

In a message dated 9/12/2004 1:04:22 AM Eastern Standard Time, mcurtis@3beam.com writes:

Capture at "good enough to look at" quality and edit with that.


Yeah, but remember, we do our test screenings in *movie theaters.*  The filmmakers are going to want the best quality image possible, not "good enough to look at," to project 15 feet high.  When we cut films in standard-def, we projected the 2:1 Beta SP because that's the best we could do at the time.  (We couldn't afford a film workprint matchback.)  If the Avid we had could have done uncompressed, we would have done that.  I doubt the director and producers would accept a 75% 1080 hi-def picture to project 15 feet high if 100% is possible on the machine, even if that means more work and a pain in the ass for me....
-----------------------------------------------------------

OK then, a few choices:

1.) Show them 75% JPEG and they'll be happy. Beats the crap out of ANY SD projection. Attached are some stills, they came from DVCPRO HD (so they've been compressed TWICE).

2.) Come up with the budget to have 25 TB of storage. Woops - tough, ain't it?

3.) Capture your DAILIES at max quality, and after screening them downsample using the "Recompress To" feature in Media Manager to managable sizes. Only have to capture once then.

As for work in progress, well fuck, man - it's work in progress. They'll have to deal. Truthfully, you won't be able to afford a full 1920x1080 projector - you'll be working with one that ACCEPTS a 1080 signal, but downsamples it to 720 or so for display (read up on it, full res projectors are super expensive. Check the blog for the FIRST rear projection TV that does 1080 res - ships next year, $10,000).
 ----------------------------------------------------

To become the "grumpy old man editor" for a minute:  that is currently one of the few major drawbacks to digital editing in low-budget feature films that are shot on film.  When we cut on film, if we wanted to see it projected in a movie theater, just to see how it would play big, all we had to do was take the cut workprint to a screening room and thread it up.  (Film workprint is 1:1 uncompressed. ;-) ) Now we either have to do a workprint matchback from the digital cut list, which is expensive, or project the video, which doesn't always tell us the full story.  We did a lot of projecting of the video cut of IN THE BEDROOM because the director demanded it, and it helped us a lot, but even then, it's still not projecting the film.  In fact, after the film was finished and premiered at Sundance, we went back and made a change based on the fact that we now finally watched the *film* in a theater. (There's a scene where Tom Wilkinson is driving and looks at a license plate in the car in front of him.  In the video projected cut, the license plate was blurry in the wide shot, so we cut to a close-up of it.  But on the film, the wide shot was crystal clear, which made the close-up unnecessary, so we went back and cut the close-up out.)

 ---------------------------------------------------

I think'd you'd be pleasantly surprised with the quality of what you'd get with HD of any flavor.

 Attached below are source, PhotoJPEG at 25, 50, and 75%
--
Open them up in Photoshop one at a time - group opening makes it say it can't open it. Use the Open Dialog if you have to.


NOTE TO READERS: I'm about to port servers so I didn't post these images. If you want to see the images, email me at mike@hdforindies.com and I'll send them to you.

-mike

From "Rog" a reader who's at IBC:

regarding that so-called cineframe-mode in the sony-hdv camera, he told me that it's some sort of a "magic bullet" within the camera. it records interlaced but then combines (interpolates) the just recorded fields..... make of that as you will... unfortunately he couldn't tell me if you loose some manual adjustment capability when shooting in that mode (like the cheapo-panasonic does when shooting 24p)..... furthermore the pro-version (which they had on display, looks & feels nice) is only different to the consumer-version insofar as it can record dvcam and got the xlr-input. no real progressive mode.


Mike's comments: Two ways they could be going on this, neither sounds like it is going to be great quality. Almost certainly either some field doubling or field averaging going on. Too bad. Will this look better than the DV res panasonic 24p? I'm thinking/hoping so, but reserving judgement until I see'em.

Other info gleaned:
-pro camera shown
-pro camera has XLR inputs
-pro camera also records DVCAM (as opposed to DV and HDV only)

-mike

Storage Review has this great lengthy article All About All Things RAID. It's long, it's detailed, it's geeky, but your prize for slogging through the whole thing is that you will Deep Grok RAID.

Here's the super shortly version:

RAID 0-data striped across all drives for speed. ZERO safety, less safe than a single drive. But wicked fast. 100% storage efficiency - X amount of drives, X drives' worth of space.

RAID 1-all the data gets written twice - once to each drive (2 drives total). It's safe but inefficient - of 2 drives, only 1 drive's worth of storage space is available.

RAID 2-no man's land - never used anymore.

RAID 3-striping with dedicated parity on one drive - data is striped across multiple drives, but one drive is dedicated to parity data, which allows for any one drive in the array to fail without risking any data. It's pretty efficient - it gets "total minus one" amount of storage. 3 drive minimum. With 3 drives, 2 store data and 1 stores parity info. In an 6 drive array, 1 drive is parity and 5 store data. So more drives, more efficiency. This RAID level is usually what is recommended for video data.

RAID 4-data is striped in a slightly different way to RAID 3 (think of them as brothers).

RAID 5- striping with distributed parity - all drives store data and parity information in a way such that any one drive can fail, and no information is lost. This setup is better for lots of little reads, but we need a low number of contiguous reads and writes. So not as ideal for our purposes.

RAID 10, 30, 50, 53, etc. - combinations of the above with various benefits. For pretty big arrays, 30 would be appropriate for our purposes. Essentially a number of RAID 3s striped up as bigger RAID 0.

For a tight summary page of the pros & cons of each level, see this link.

Read and enjoy!

-mike

Too tired to make all these individual posts, so here they are in sequence:

Article on Sony's new HDR-FX1 HDV camera

Adobe announces forthcoming plugin for HDV support in Premiere Pro

Pinnacle Liquid Edition 6 Will support multistream HD editing for under $500

Canopus parterns with 1 Beyond for HD PRO RT workstations

Media 100 844/X HD - have to add something about this one - HAHAHAHAHAHHA!!!! Be sure to recommend to the competition across town - I soooooooo don't have confidence in this product. It should be capable of a lot of simultaneous HD compositing stuff...if it works, if it ships, if it gets good support. These guys have been on financial life support, and their interoperability is looooooooow. I think these guys are off the back of the pack, and can't fight their way back into the peleton. I'd rather buy a CineWave than this...and I think CineWave is waaaaaaay too expensive compared to the competition.

OK, I'm done ragging on hapless victims. I better stop before I get run over by karma.

-mike, flogger of the helpless

Matrox announced Axio, their loaded-for-bear, realtime HD and SD system that should ship in the first quarter of next year.

From this Creative Cow article, here's the quoted specs:

-Guaranteed full quality, full frame rate, full resolution playback at up to 1080i at 29.97 fps
-At least two layers of uncompressed 10-bit HD video plus two layers of graphics in real time, with effects
-At least 4 layers of uncompressed 10-bit SD video plus 6 layers of graphics in real time, with effects
-Realtime primary and secondary color correction with shot-to-shot color matching
-Realtime 3D DVEs, chroma/luma keying, speed changes, blur/glow/soft focus, and much more
-Uncompressed 8- or 10-bit HD and SD editing
-Compressed HD editing (offline and online finishing-quality MPEG-2 I-frame)
-DV, DVCPRO, DV50, and MPEG-2 I-frame SD editing
-24-fps editing in HD and SD with pull down and reverse pull down
-DV-1394, SD SDI, and HD SDI input and output
-RGB and YPbPr analog component output
-4-in/8-out AES/EBU balanced audio, 8-in/8-out SDI embedded audio

Quoted price of $11,495, I think that includes computer & storage, but they seemed a bit vague on the point. I need to look into it more. From the Creative Cow article: "The Matrox Axio software and hardware components are integrated into an approved workstation-class PCI-X computer, coupled with a robust storage subsystem, and sold through a network of Matrox authorized resellers."

Price competitive (roughly) with a Final Cut Pro HD setup, but better realtime capabilities I'd guess - Matrox has a great track record of providing huge realtime capabilities. I'll need to see what they are suggesting as storage. I'd also like to see the quality of those effects.

More on this as I learn about it. If I were at IBC, I'd be all over these guys. But my own curiosity is a bit tough to use as justification for flight & hotel to Amsterdam for only a convention.

BTW, I'll be speaking at the Austin Film Festival come October, for anyone who happens to be in Austin at the time. I'll post details later.

-mike

Sony has announced their first true 1920x1080 rear projection TV. It's a 70 incher, retail cost $10,000, shipping early next year (so you can't buy me one for Christmas....oh darn).

Read the press release here.

Thanks to Christopher Barry for sending in the link.

-mike

Sony has issues this press release which includes further details on the actual functionality of their HDV camera implementation.

The most interesting thing I found in there was the actual pixel dimensions of the CCD. I've seen stats claiming 1.2 megapixels and 1.07 megapixels. They're both sorta right, here's the scoop:

Each of the 3 CCDs has a resolution of 1012x1111. I have no idea how they arrived at that number.

The total "valid" pixels as they call it is 972x1100. From there the 960x1080 is derived. So there you go - While you view the signal as 1920x1080, it started as exactly half that many. Then it is upsampled to 1440x1080 for the MPEG-2 encoding. Then it is upsampled on playback to 1920x1080. So this format is resolution constrained.

But that's nothing new - Sony's HDCAM starts at 1440x1080 and gets upsampled to 1920x1080.

Panasonic's DVCPRO HD tape format only records 1280x1080 pixels and upsamples on playback.

Panasonic's Varicam shoots a 10 bit, 1280x720 image. But when it records to tape, it downsamples and records 960x720 at 8 bit.

That's exactly (and only) twice the resolution of the standard definition Digibeta format. While the 1280x720 broadcast standard is 2 1/2 times SD (standard definition video) standard, as DVCPRO HD records it, it's only twice the resolution.

Sony's HDCAM only records 75% of the pixels of a standard 1080 signal, and also heavily compresses them (as all the HD formats do).

Sony's HDCAM SR records all the pixels in a standard 1080 signal, and only lightly compresses them. But the SRW-5000 is a $100,000 deck with the dual link HD-SDI 4:4:4 RGB board installed.

I need to look up Panasonic's D-5 format and check it out some more, but I get the sense that HDCAM SR is technically superior.

-mike

Digit Magazine is reporting from the IBC conference in Amsterdam that Apple as announced future support for some additional video formats. Sony's MPEG-2 IMX (used in Sony's MSW900 and PDW530) and Panasonic's P2 format. P2 is used on solid state cards in cameras and is designed and optimized for ENG applications (electronic news gathering).

Read the full article here for more details. Both of these formats are aimed mostly at ENG applications, so indie filmmaker types won't see much benefit from them. Plus, they are both standard definition.

Apple has also previously announced future support for HDV, the MPEG-2 based format used in consumer/prosumer low cost HD cameras.

I've updated my earlier post on recommended HD equipment with a variety of new details, including:

-revised DeckLink pricing
-revised HDLink pricing
-revised system configs
-links to products mentioned
-storage options and pricing for various sized SATA arrays
-additional info on how much of each type of HD footage can be stored on a 3.2TB array
-and other myriad details and tweaks

For those wanting to have a low cost HD capture station (such as for log & capture), that can be put together for as little as $6200 with a terabyte of HD capable storage.

For those wanting a killer HD post production station, with an Apple 23 & 20 monitors, 23" LCD HD monitoring solution, pro level audio monitoring, 3.2TB of storage, 4:4:4 RGB capabilities, , separate jog/shuttle control, editing keyboard, DVD authoring, motion graphics package, 2.5GB RAM, that can be had for around $18,000.

All pretty cool.

The rate of change, the rate at which product prices are dropping, is moving much much faster than it did for standard definiton video. Already, compressed HD editing stations are costing what DV edit stations cost just a few years ago, and I think uncompressed HD stations are already where SD stations were 5 or more years ago in terms of costs...but with even greater features.

I started to write that within a year it'll be possible to have a really nice HD editing station for under $10K with lots of storage and monitoring included.

Hmm, how far off is that?

Woops, it's already here - you can put together a system capable of cutting an HDCAM feature now for under $10,000. OK, granted, a short feature, say 80-90 minutes, not Lord of the Rings. A G5, 1.5GB RAM, FCP HD, jog/shuttle, a decent 19" CRT, a decent 17" CRT, HDLink & Apple 23" LCD for HD monitoring, a 1.2TB (1.0 TB usable for HD throughputs), and the new DeckLink HD Pro Single Link could be squeezed in for just under $10,000. If you were careful, you could cut a short feature on that.

-mike

BlackMagic Design today announced a slew of new products, and cut prices heavily on every HD product they sell. Rawk!

Here's the short version of their new HD card lineup:

DeckLink HD - $595 (same as before)
DeckLink HD Plus - $995 (now includes genlock & AES/SPDIF in/out with world clock)
DeckLink HD Pro Single Link - $1495 (just like prior HD Pro but only single link HD-SDI, 4:2:2 only no 4:4:4)
DeckLink HD Pro Dual Link - $1995 (it is the existing HD Pro card, just renamed & repriced)
HDLink - repriced to $695, same device as before

Other new products:

Workgroup Video Hub:

-SDI routing switcher with RS-422 remote control routing
-12 SDI ins, 24 SDI outs
-enough for 12 DeckLink cards or broadcast deck inputs and outputs and control
-extra 12 for monitoring
-allows interconnection of DeckLink, Avid, Discreet, etc. via HD-SDI
-extra 2 I/O ports for realtime processor they're working on for HD downconversion etc.
-3RU tall, 1 inch deep
-$4995 retail
-can connect 2 of'em for 4:4:4 Dual Link operation (nice!)

DeckLink Multibridge

-it's an uber-breakout box
-all in one converter and tethered breakout box for edit/design systems
-handles 4:4:4
-dual link HD-SDI and SD-SDI, converts to and from analog YUV
-4 channels XLR audio in and out (balanced)
-8 channels AES/EBU digital audio via DB-25
-composite NTSC/PAL output
-RCA stereo pair audio output
-14 bit D/A 4:4:4 conversion for video
-does broadcast quality HD downconversion (soon)
-yes it does birectional conversion
-$1995 for dual link
-soon a $1495 SD single link version
-available soon (not yet)

All of this will be on the show floor of IBC.

I've had a hard time getting through to tech support lately, I'd imagine this is why.


Grant Petty sent out an email this morning from IBC, with greater detail and context on all the new products. Here it is in full:
------------------------------------------------------------
Hi,

I am writing this from the show floor at IBC 2004 in Amsterdam, and wanted to send you an updated on the new products we are announcing here at the show today.

We have an incredible roll out of 2 new models of DeckLink HD cards, 2 new models of DeckLink Multibridge, an all in one analog converter that also makes a perfect break out box, as well as a Workgroup Videohub, a routing switcher.

Introducing Workgroup Videohub

We have really sold a lot of DeckLink cards since Blackmagic Design has been running, however once most people get more than two or three systems, and then get a deck or two, the cabling becomes a complete nightmare.

Workgroup Videohub solves this problem. It's a true professional SDI routing switcher with RS-422 remote control routing built in. It has 12 SDI inputs, and 24 SDI outputs with the 12 control ports. This is enough connections for 12 DeckLink cards or broadcast deck inputs and outputs, and control.

The extra 12 outputs are for dedicated for independent monitoring, but you could use them for analog converts for dubbing decks etc. Independent monitoring is really nice because you can quickly see any workstation or deck in a facility.

Because Workgroup Videohub is all SDI that means any equipment such as DeckLink, Avid and Discreet systems can also be connected together. For anyone that's worked in a large post production would remember know how nice it is to instantly switch video in a facility with a router. Monitoring is nice as you can just look at any deck or workstation in the facility with a click of a button. Now with Workgroup Videohub, that's affordable for everyone.

What's really nice is Workgroup Videohub includes an extra 2 inputs and outputs for an internal real time video processor which we will be releasing software for various video processing tasks. The most obvious is HD down conversion, and mixing etc. For years I have hoped to have a professional routing switcher that does both HD and SD with down conversion built in, and it's exciting to see it finally released.

At only 3 rack units high and less that 1 inch thick you can reverse mount it in the equipment rack as a patch panel or connectors to the rear as a traditional routing switcher. Because it's so thin, you can even mount it in the rear of a rack, so you save valuable front rack space for decks and other equipment.

Most people would never be able to afford a routing switcher before, and so this is a very significant product for the industry. It's not for everyone, as some people will need more inputs and outputs, but for most workgroups, it's ideal.

Retail cost is only US$4,995, which is totally unheard of for a HD-SDI only router. But Workgroup Video also includes remote control routing as well as standard definition support. You can also connect 2 routers for 4:4:4 Dual Link operation.

Workgroup Videohub is available now, however we are still ramping up production, so I assume it in reality will become more available over the few weeks.

Introducing DeckLink Multibridge

A lot of people have wanted us to produce a break out box in the past, however I have always resisted, as I hated all the break out boxes with the multi pin tether where the cable is stiff, and the break out box was more annoying that useful.

The other solution was to use external converters, however the problem with brick style module converters is it's an expensive way to convert anything to analog when you add up the pile of converters you need. Also, on currently available converters, quality is often limited to 4:2:2.

With the change over to HD, the need for an elegant break out box solution and the release of the Sony 4:4:4 broadcast decks, we knew it was time to solve all these problems with the one single product. That's DeckLink Multibridge.

DeckLink Multibridge is an all in one converter, but it's also a SDI tethered break out box for editing and design systems.

DeckLink Multibridge features Dual Link 4:4:4 HD-SDI and standard definition SDI connections and converts to and from analog component YUV video. YUV video switches to both HD and standard definition automatically, and you can also connect RGB analog.

Also included is 4 channels of balanced audio in, and 4 channels out on standard XLR connectors, as well as 8 channels of AES/EBU digital audio in and out on an industry standard multi channel DB-25 connector. There is also an independent composite NTSC/PAL output and low level stereo audio output for audio monitoring using consumer HiFi equipment.

The analog output is the same incredible 14 bit 4:4:4 conversion that DeckLink HD Pro uses, and it's incredible quality. The D-A converters are extremely high speed, and it uses active filtering not passive, so the quality on fine HD image detail is just incredible. Even the composite NTSC/PAL is 14 bit at 5x over-sampled, so chroma burst is accurately converted eliminating the slight chroma phase errors seen on other converters.

DeckLink Multibridge even includes a very high speed real time processing engine, which we are working on software updates for. This high speed processing engine will be shown at IBC running a completely redesigned hardware based HD down converter. This down converter is full broadcast quality, and has high quality filters for excellent down conversion quality that's suitable for mastering.

DeckLink Multibridge has enough features to combine many separate converters into the one product, and it's lower cost that even a simple 4:2:2 HD-SDI to analog converter without audio from other companies. DeckLink Multibridge features bi-directional analog conversion, audio conversion, and is high definition and standard definition, even up to dual link 4:4:4.

I think if you wanted to build the same functionality that you get with DeckLink Multibridge using separate converters and SDI audio embedders, you would need more than 6, and it would cost thousands of dollars more. You would not get dual rate HD and standard definition support, because hardly any converters support this feature, and none do dual link HD 4:4:4 as far as I know.

All these features are just amazing value at US$1,995 retail for the Dual Link HD/SD version, and we will also be releasing a lower cost standard definition 4:2:2 only model called DeckLink Multibridge SD at US$1,495.

DeckLink Multibridge is going through the final testing phases, and we expect it to released in about 4 weeks, but it might be sooner. I will keep you updated on that.

New DeckLink HD Products

When we were developing DeckLink Multibridge we quickly realized that it's such as great bundle with DeckLink HD, but often you really need genlock when connecting to bigger systems that require it. So we have made changes to the DeckLink HD line to accommodate these needs.

New DeckLink HD Plus

DeckLink HD Plus is very similar to DeckLink HD however it's single link SD/HD-SDI, so only has a single SDI output. However it includes genlock/HD Tri Sync input, as well as AES/SPDIF in and out, with word-clock out.

If your looking for a HD card for larger systems that use house sync, and are going to be using converters such as Multibridge and even connecting to SDI routers such as Workgroup Videohub, then this is a perfect product.

DeckLink HD Plus is priced at US$995, and it's available now.

DeckLink HD

DeckLink HD is a great card that's been doing really well for us, and it's perfect when your working in systems that don't need genlock and can lock the deck to the card itself. It's now has a new price of US$595, so at this price it should allow everyone to get into HD easily.

New DeckLink HD Pro Single Link

Because not everyone is building a large system with converters and routers, it's important we have more options for cards with analog video monitoring built in. These cards are perfect for freelancers and people running the single system.

We are also close to a new software update for HD Pro cards that include hardware based HD down conversion that will allow simultaneous HD and SD outputs from DeckLink HD Pro dual link models, and we also plan on adding composite NTSC/PAL output as well as DVC HD acceleration into that software update. We will be showing this update at IBC, and it should ship sometime in November.

Also at IBC is a new DeckLink HD Pro single link model, and this is priced at US$1,495. This models is identical to the current DeckLink HD Pro, however only includes a single SDI input and output connection. Not everyone needs dual link SDI, but do need the high quality analog monitoring, and also HD support. So this will be a good option.

DeckLink HD Pro Dual Link models have been re-priced to US$1,995 to allow a lower cost dual link option.

Both single link and dual link versions of DeckLink HD Pro are available now from dealers.

HDLink

Since we started shipping HDLink it's shipped hundreds worldwide, and has been a great product for us. To make it more affordable, we are now reducing the price to US$695.

This will allow HD monitoring to be much more affordable for users, and we think it will also feed back into our other products once an affordable monitoring solution is available. This is exciting for us, as we know the high cost of HD monitoring has been keeping a lot of people from upgrading to HD, and this should help change that.

IBC 2004

We will have 3 DeckLink Multibridge's and 1 DeckLink Multibridge SD running on the demo stations on our IBC booth all connected together into a Workgroup Videohub, and the DeckLink Multibridge's will be connected for monitoring.

If your going to the show, please drop by our both so we can show you these new products. We can show you how all these products work, and how they connect together for a complete dual rate SDI post production workflow.

We have been secretly running these products in our Singapore based post house for weeks, and they are working great!

I will keep you updated!


Regards,

Grant Petty Blackmagic Design

X-Factor, the add-on to After Effects 6.5, now supports Render Queue network rendering to allow idle machines on the network to help render frames from an After Effects composition. This new version will ship in October.

Red Giant's Film Fix is from the same guys that created Magic Bullet - the crew at The Orphanage. These guys rock. This new plugin for Windows After Effects and Digital Fusion allows for repair of tears, exposure jitter, dust, scratches, etc. It's $2K with an intro price of $1500.

Read the rest of the article for the details. If you're working on a doc with old footage, this could be quite useful. But I'll bet it gets pretty techie, and is exactly the kind of massively time intensive project that indies shouldn't get sucked into themselves. Farm it out.

-mike

So I've been researching this HD stuff for more than a year now, after a career in digital media production stretching back to around 1990. As I started looking into this HD stuff, there were three areas that were deal killers in the beginning. The cost of the HD cards, the cost of HD monitoring solutions, and the cost of HD capable storage. Now I have it all solved, and can recommend a high end, large capacity, very client presentable solution for about $17,500.

UPDATE FRIDAY MORNING: BlackMagic cut their pricing across the line this morning at IBC. So the $17,500 system would be more like $16,400. Read my post on new BlackMagic products/pricing for full details.

If you want to skip the backstory/preamble, scroll down to What Mike Recommends

The HD capture cards have come down markedly in price -from $10,000 a couple of years ago, the BlackMagic DeckLink HD cards are now $595 (updated Friday morning), and the new BlackMagic HD Pro cards with built in analog monitoring, dual HD-SDI input/output, and realtime downconversion are only $2000 (updated Friday morning). The Kona2 from AJA has virtually identical specs at the same price as well (it's still $2500 as far as I can tell as of Friday morning).

The monitoring solution was a bit tougher - studio HD monitors start around $5000, then the HD-SDI card to accept an HD signal is another $3500 or so. An outboard HD-SDI to HD analog converter box is $1700 to $3000, depending on desired features (downconversion, etc.). But there is now a low cost solution for monitoring as well. The BlackMagic HDLink is a $700 (updated price as of Friday) device that allows an HD-SDI high definition video signal to be viewed on a 1920x1200 pixel resolution computer LCD panel, such as Apple's Cinema Display 23. The black levels are a bit high due to the lack of contrast in current LCDs, but the image is absolutely tack sharp - every pixel of the source signal is mapped to a pixel on the display, perfectly one to one. eCinema has a similar line of products - they were the first to market with this concept of LCD monitoring - but their product line starts at over $3000. Pricey.

Storage was an even tougher nut to crack - SCSI and Fibre Channel are the traditionally recommended solutions, but they are ungodly expensive, even when bridged to lower cost ATA or SATA drives. FireWire, especially FireWire 800, seemed promising, but even with multiple FireWire 800 buses the G5s are crippled - they can't achieve the data rates necessary for uncompressed HD work. Serial ATA, or SATA, drive technology offers a fast bus for fast drives, but the stock G5 can only handle 2 drives. A 2 port PCI card came along that allowed for 2 more (the Seritek 1S2 card) and this barely allowed for HD throughputs, but only with RAID 0. So if one drive in the array failed, all data was lost. So a redundant backup system, such as FireWire 800, was necessary (or at least fervently recommended - I had a friend have to recapture 250 clips on deadline last week after a drive failed at the Wrong Time).

That changed today. The best SATA card on the PC side now has Mac drivers - the Highpoint RocketRAID 1820A card has 8 SATA ports AND allows for RAID 5, so that if any one drive in the array fails, all the data can be recovered easily. And it's only $200. This allows for up to 8 drives to be used in external chassis (from the likes of MacGurus.com) to allow up to 3.2 terabytes of storage in RAID 0 configuration, or up to 2.5TB in RAID 5 configuration.

Putting all this information together, this is what I'm recommending at the moment for those that want to do uncompressed 1080 resolution work on a single station. The $17,500 price reflects the biggest array, the nicest monitors, etc. It's the super capable, client impressing system. It can be done for less.

What Mike Recommends

Apple G5 DP 2.0 GHz (2.5's aren't available/stable yet): $2500
(if you aren't a heavy Motion user, stick w/GeForce 5200 or drop in a Radeon 9600 Pro. I'm a bit worried about cable routing with the 9800 Pro XT or NVidia 6800 Ultra cards)
bump up the RAM to at least 2.5GB: $400
Apple Video Production Suite (Includes Final Cut Pro HD 4.5, Motion 1.0, DVD Studio Pro 3)- $1300
BlackMagic DeckLink HD Pro Dual Link - $2000 (updated pricing Friday)
Highpoint 8 port RocketRAID 1820A SATA card - $200 (download Mac drivers from web)
HDLink - $700 (updated pricing Friday)
Apple Cinema Display 23HD - $2000 (for monitoring HD signals via HDLink)
MacGurus Burly Box case with coolers (or other acceptable substitute - I like the per-drive cooling) - $330 per 4 drive enclosure
hard drives - 4, 6, or 8 drives - my favorites at the moment, at each size, are:

250GB: Western Digital WD2500JD (the recent ones, not older ones, much better now) - $167 at zipzoomfly
300GB: Maxtor Maxline III 300GB drives - fastest large SATA on the market - $230 at zipzoomfly
400GB: Hitachi 7K400 400GB drives - largest SATA on the market, and still pretty fast - $456 at zipzoomfly

Arrange those to taste. Remember, only 4 drives per Burly Box, so you'll probably need 2. Also, with a 4 drive array RAID 0, you can't capture some flavors of 1080 all the way to the end of the array. Research carefully. If you want to do RAID 0 for maximum speed, I recommend backing it all up to FireWire800 drives, such as the La Cie Bigger Disk or Bigger Disk Extreme. The Bigger Disk is a better deal per GB ($1200, 1 TB) vs. the Bigger Disk Extreme (1.6TB, $2200).

Main Computer monitor - either another Apple 23 LCD ($2000) or a La Cie Electron Blue IV (about $700) - $700 or $2000
Secondary computer monitor (for bins, pallettes, etc.) - $250 to $400 for CRT, or $700 (Apple Studio 17) to $1300 (Cinema 20) for Apple LCDs, your choice.

I'd also recommend an improved keyboard and mouse over ships with the Mac.

Some options:

Logic Keyboards offers a variety of replacement keyboards with Final Cut Pro HD keys that are color coded and labelled for the Final Cut Pro commands.

EZ Keyboards offers a model with a jog shuttle wheel included for $190. Without, they offer a MacAlly based one for $140.

Personally, I'm a bit of a typing snob - I type about 90 or 100 words a minute when I'm on full-tilt-caffeinated-boil mode ("So THAT'S how he blogs so damn much!), so I'm a big fan of the clean, white, Matias Tactile Pro Keyboard. It lacks color coded or labelled FCP keys, it has no jog shuttle, it's $100. So why? Awesome positive clicky feel. If you're a hard core typist, you'll love it. It's the closest thing I've ever typed on a computer to an IBM Selectric II. (Yeah, I'm that old. Deal.) If you want, you can always stick on the little labels that came with FCP, even through they will peel off in a few weeks. Learn fast. But to go with that keyboard, I use a Countour Designs Shuttle Pro v2 for contextual jog shuttle control in FCP. In comes with presets that alter button functions depending on whether your in log/capture, timeline, etc. Way way smart/slick/cool, it's $100. They offer a smaller ShuttleXPress as well. It's $50, sometimes $40 online.

For a mouse, my present favorite is the Logitech MX 300 a simple 4 button model that fits my hand well. Mice are very personal, so pick your own. $30 online.

And that ought to hold you. Loaded up G5 2.0GHz, 2.5GB RAM, 9600 video card, full tilt, 3.2 TB RAID, Apple 23" & 20" LCDs for computer, HDLink & Apple 23 LCD for HD monitoring, DeckLink HD Pro card, Matias keyboard, Shuttle Pro, Logitech Mouse, Production Suite, all the trimmings will set you back a bit under $17,500 by the time you get all the cables and whatnot figured out. Add some powered studio reference monitors, maybe the Alesis M1 Active or somesuch for about $400. That still keeps you under $18K.

Some other configurations I threw together based on the above stuff (updated pricing as of Friday):

low cost "headless" offline capture-500B FW800 5,150.00
low cost headless online capture-no storage 4,650.00
low cost online capture w/monitoring-no storage 8,650.00
low cost 720 Uncmprsd Edit-no storage 6,480.00
high end 720 uncmprsd edit-no storage 10,630.00
low cost 1080 edit-no storage 8,280.00
high end 1080 edit-no storage 12,430.00

Storage options:

4x300 array - 1.2TB -$1165 (2 internal, 2 boxed, must boot from FW drive)
4x400 array - 1.6TB -$2045 (2 internal, 2 boxed, must boot from FW drive)
6x300 array - 1.8TB - $1,914.00 (2 external multi-drive enclosures)
6x400 array - 2.4TB - $3,234.00 (2 external multi-drive enclosures)
8x300 array - 2.4TB - $2,704.00 (2 external multi-drive enclosures)
8x400 array - 3.2TB - $4,512.00 (2 external multi-drive enclosures)

If you're interested in the details of any of these configs (it's all based on an Excel sheet I've built) you can email me at mike@hdforindies.com


What makes all this impressive is the fact that 18 months ago, the solutions were vastly more expensive. HD Capture card? Cinewave RT with HD option, about $14K. 23 Inch LCD monitors? $3500...each. 24 inch HD monitor? $30K. 3.2TB of storage? Easily $15-20K. Total HD edit station cost? Around $75,000. A year before that, an Avid capable of HD realtime cross dissolves was in the $150-$200K range. My, how things change fast. That's a more than fourfold drop in price in a year and a half.

And it just makes me cringe to know that a lot of VARs still recommend these kinds of last generation solutions. In part because it is what they know works...and also because it allows them to make the kinds of margins they are used to. I had a VAR tell me in March of this year that he intentionally steered customers away from AJA/BlackMagic cards, implying they were lower quality and/or unreliable, simply because he couldn't make the margins he did on CineWave. UGH.

With this kind of (newer) system, you can edit and work with any flavor of HD you wish. If you want to work in fully unfettered 1080p 4:4:4 RGB from data off of a HDCAM SR deck at any framerate you wish, it's not a problem.

Let's see what that will handle with various formats. Since I'm not sure what the RAID 5 formatted capacity will be, I'm going to assume RAID 0 for the moment. You could spend $4000 more dollars and back up all the data to FireWire 800 if you were worried about drive failure. Then just keep a spare drive sitting around.

3.2TB will format to 2960 GB of usable RAID 0 space (on anybody's array, not just this SATA solution). That would hold:

24p formats:

-something around 200 hours of PhotoJPEG compressed 1920x1080 for offline editing
-144 hours of 720p24 DVCPRO HD footage from Panasonic AJ-HD1200A deck
-8.86 hours of HDCAM (8 bit, 4:2:2 1920x1080 24p)
-7 hours of 4:2:2 HDCAM SR YUV (10 bit, 4:2:2 1920x1080 24p)
-4.73 hours of 4:4:4 HDCAM SR RGB (10 bit, 4:2:2 1920x1080 24p)

29.97 interlaced formats:

-something in excess of 150 hours of PhotoJPEG compressed 1920x1080 footage for offline editing
-59 hours of 1080i DVCPRO HD captured from a Panasonic AJ-HD1200A deck
-7 hours of 1080i HDCAM (8 bit 4:2:2 YUV 1920x1080 29.97i)
-5.67 hours of 1080i HDCAM SR 4:2:2 YUV (10 bit 4:2:2 1920x1080 29.97i)
-3.78 hours of 1080i HDCAM SR 4:4:4 RGB (10 bit 4:4:4 1920x1080 29.97i)

So if you work with an offline/online codec solution, this will allow for plenty of room to cut a feature. That should be sufficient for anybody's project.

Questions, comments, want to throw the bullshit flag down on the field? Please do so at mike@hdforindies.com

Also, if you'd like to discuss solutions to your particular workflow or project needs, I am available for that kind of work as well. Again, email me at mike@hdforindies.com

-mike

UPDATE SUNDAY - see bottom for update

As reported first at Barefeats.com, the HighPoint RocketRAID 1820A 8 port PCI-X SATA card now has Mac drivers.

"Um, geek the f out, Mike, but so what?"

This is HUGE for those wanting to do their own HD stuff on a Mac G5. In the past, I've been suggesting to folks that they do a 4 drive SATA array and use FireWire 800 for fully redundant backup. 4 drives was all that was realistically possible with the current technology, a Seritek 1S2 for 2 external drives and two internal SATA drives, while booting off of a FireWire 800 drive.

Now the game has changed - there is an EIGHT port card available, AND it supports RAID 5. This means you can hook up to 8 drives to the beast, and get PLENTY of throughput, AND still have data protection - if one drive goes bad, then it's OK - the distributed parity magic of RAID 5 will save your ass and your data. You might have to suffer downtime while it rebuilds the data, but you have a constant level of protection from drive failure. If you erase a file by mistake, it's still your fault though.

Robert over at Barefeats.com reported that he was over at Promax's test facility and they got 240MB/sec reads and 255MB/sec writes off of only 4 drives in a RAID 0 configuration.

When used with MacGurus Burly Box and 8 Maxtor Maxline III 300GB drives ($230 at zipzoomfly yields a 2.4 TB RAID 0 array, or a 1.92GB RAID 5. Need more? How about 8 Hitachi 7K400 400GB drives in the same cases, for a 3.2TB RAID 0 or 2.56TB RAID 5.

OK, so what's this cost, and why would I want it? These two arrays represent the maximum amount of storage possible with a SATA array for HD utilization. You can't add more of these, and there's a LOT of cables running around to do this.

COSTS:

2.4TB RAID 0: $2750

3.2TB RAID 0: $4600

That's $1.24/GB for 2.4TB RAID 0, something less for RAID 5 (not sure of the capacity off the top of my head)

For the 400 GB based array, that's $1.54/GB (RAID 0 3.2TB)

Those prices include the SATA card, which is $204 from zipzoomfly, by the way.

How does that compare?

If you went to a Value Added Reseller last year, they'd tell you the smart play was to buy from Huge or Medea:

Huge:

MediaVault U320-RX, 2.5TB, dual channel SCSI rack mountable, scalable: $9950 +$450 for SCSI card

Cost per GB of RAID 0: $4.16/GB RAID 0

Medea:

The 10 drive, 2.0TB, 200 MB/sec capable VideoRAID RTRX is $9800 and requires a SCSI card, about $450 extra.

Cost per GB of RAID 0: $5.13/GB

Are the Huge, Medea, and Apple systems more proven, field tested reliable, and scalable in ways SATA RAID is not? Definitely. But they cost way, way more.

Now Apple is in the game with their X-Serve RAID. let's see how it stacks up at similar capacities:

Apple X-Serve RAID, 1.75TB with 512MB cache and Fibre Channel card: $8300

Cost per GB of RAID 0: $4.74

If you bump it up to the 3.5TB array, it gets more cost competitive - 3.5TB costs $11,800.

Cost per GB of RAID 0 (3.5TB): $3.37

This is a simple comparison. For RAID 5 capacities, merely increase costs by 12-25%. But the relative ratios remain the same. This simple price comparison also doesn't factor in scalability, expandability, features, controllability, etc. Just a simple cost per unit of storage.

But any way you slice it, this SATA array solution is 1/3 or 1/4 of the price of the other options, and still offers RAID 5 (at an unknown performance level, I'll know within the week, however) and gives enough storage to cut a feature (assuming an offline/online codec workflow).

So see the next post for how to configure all this...

UPDATE SUNDAY

After doing some research and reading over the weekend, I've learned some more about RAID levels and this card in particular.

RAID 5 stores "N-1" amount of data, where N is the number of drives in the array. 1 drive's worth of space is consumed by parity data that is distributed across all the drives. So in an 5 drive array, only 4 drives worth of space is available. In an 8 drive array, only 7 drives worth of space is available.

While this card is capable of RAID 5, it is software RAID 5 not hardware RAID 5 (as I had assumed, but you know what they say about the word "assume"). The difference is this - in a hardware RAID, all the hard work is done by dedicated hardware on the RAID controller or in the RAID chassis. In a software RAID, all the hard work of RAID functionality is done on the computer, and this saps CPU cycles available for other things (like FCP tasks, for instance).

So while the 1820A is capable of RAID 5, it is not going to be advisable for HD utilization. So RAID 0 will be the way to go with the 1820A I predict, and FireWire backup will be the way to go. RAID 0 offers no protection whatsoever for your data. If any one drive fails, all data is lost (short of truly heroic efforts).

So I'll be able to test this week to see what the performance is truly like - I got confirmation on shipment of my 1820A card on Saturday. I'll be testing 4, 6, 8, and possibly 10 drive configurations, and with RAID 0, 1, 5, and 10.

-mike

Robert over at BareFeats has posted some interesting stuff of late. For one, we finally found out whether the new G5 dual processor 2.5GHz machines fixed the FireWire 800 performance problem. It didn't. Read the details here (you'll have to scroll down a bit).

Secondly, he's posted some test results of Apple's Motion graphics application on various machines with various configurations. As expected, a fast CPU makes a difference, but a fast GPU (graphics card) makes an even bigger difference. A lot of RAM helps as well. He has some nice detailed commentary on all the nitty gritty.

As always, he's a very valuable resource in my research efforts.

-mike

Kano Technologies has introduced a Blu-Ray based storage device. Blu-Ray is the 23GB optical storage format, a successor to DVD based storage media. It looks like a CD or DVD, but using a finer laser (blue not red), can inscribe itty bittier bits onto the media, holding 23.3 GB per one sided disc. Kano's external SCSI device has a suggested price of $3000. A quickie Google search reveals that media costs about $30 a disc.

Mike's Comments:Somebody asked me about the upcoming Blu-Ray discs for storage or backup recently. Just for fun, I thought I'd post this to think about backups. I own an AIT+ tape backup system. It is SCSI based, I paid about $2000 for the drive in 1999, and media was about $2/GB the last time I bought a 5 pack of tapes. It dawned on me a couple of years ago that external FireWire hard drives weren't just faster than tape, they were also cheaper. If you shop carefully, you can find ATA hard drives for around 50 to 75 cents a GB. An external FireWire case can be had for $100 or less. Get a big drive and a cheap case, you're looking around $1.25 or less per gigabyte. And a FW drive can hold up to 400GB presently, and backs up around 35 MB/sec or more, and is online rather than offline storage.

So I've given up on tape backups.

But what about Blu-Ray? Well, maybe for DV or HDV projects. But for HD, let's chug a little math. Even HDCAM footage, at 8 bit 4:2:2 1080p24 is around 350 GB/hr. So a Blu-Ray disc would hold less than four minutes of footage.

Never mind.

And at a top speed of 11 MB/sec, it would take quite a while to back up.

This brings to mind the old adage:

Q: "What's the only thing that can fight off it's own weight in wild dogs?"

A: "Wild dogs."

For HD applications, the only thing that can quickly and cheaply back up hard drives is hard drives these days. RAID 3 or RAID 5 yields sufficient protection for most users, barring a total catastrophe (such as huge power surge, lightning hit, building burns down, or intern/editing assistant wipes the drives by accident.)

RAID 3 or RAID 5 at this time, however, is still quite expensive. The convenience factor shouldn't be overlooked - with RAID 3 or 5, you ALWAYS have an up-to-the-second backup of your data with no downtime. Copying a terabyte at 35MB/sec takes about 8 1/2 hours.

But I've found that the lowest cost solution is to just get RAID 0 (if required for your throughput needs) and then back it up with redundant drives, FireWire being the lowest cost scalable solution I've found so far. This also allows for the posibility of keeping those redundant drives offsite, away from possible fire (or interns).

So I feel pretty confident that the Sony will be better than the JVC JY-HD10. Here's why:
-1080i rather than 720p
-60 vs 30 fps (neither is 24p, but 60i can be shoehorned better than 30p can)
-3 chip CCD rather than one chip CMOS
-good low light sensitivity (purportedly)

Here's what's good about this camera for indie filmmakers:
-best bang for the buck in terms of pixels per dollar. 1080i for under $4k. Excellent!
-decent shooting controls
-good low light sensitivity
-purportedly more film like gamma
-a pseudo 24p mode (need to research more)
-lots of physical controls on camera
-nice/flexible (literally) LCD screen

Here's why this camera is NOT good for indie filmmakers:
-1080i60, not 1080p24. Way not film standard frame rate. But 60i can be crunched into 24p more readily than 30p can. Depending on this camera's light sensitivity (which won't be great, since 960x1080 has to come of 1/3 inch chips), could be decent documentary camera. UPDATE - need to find more on this pseudo-24p mode...
-MPEG-2 25Mbit data stream that it shoots is VERY heavily compressed - they are acquiring what others distribute in - not much latitude for color correction etc....plus it only looks mediocre at best. Don't do a fast pan with this stuff
-MPEG-2 transport stream is awkward to edit (must use Lumiere or equivalent to get it into FCP HD).That'll change with some future version of FCP HD, they've pledged native HDV support.
-8 bit 4:2:0 color space - not as good as the 4:2:2 of DVCPRO HD or HDCAM SR, or the 3:1:1 much less compressed HDCAM. Obviously the 4:4:4 of HDCAM SR RGB is preferable.
-from what I can tell, no separate audio inputs for better mikes than the built in. So no body mikes, etc. Dammit.
-but they mention a CineFrame mode or somesuch, which maybe records 24p via a telecine like process. If true, then CinemaTools in Final Cut Pro HD could potentially extract it back to a source 24p

Other good news - one article mentioned that Sony plans on having a pro version of the camera sometime in Q1 2005 around $7000. That would probably add audio inputs, and other such pro features.


COMPARISON OF SONY'S HDV TO PANASONIC'S DVCPRO HD

HDV 720p:
720p30 ONLY
1280x720 pixels
8 bits/channel, 4:2:0 color space
19Mbit (2.375 MB/sec)
audio: MPEG-1 Layer 2 - compressed? Research!
how many channels of audio?

DVCPRO HD 720p
720p24, p30, p60, or anything from 4 to 60 fps
960x720 true resolution
5.7 MB/sec (24p), 7.125 MB/sec for 30p
8 bit 4:2:2 colorspace
uncompressed audio (right?) how man channels of audio? 4?)

HDV 1080i:
1080i60
960x1080 source res (1440x1080 is the HDV spec, they are scaling to that)
8 bit 4:2:0 colorspace
25MBit (3.125 MB/sec)
MPEG-1 Layer 2 audio (compressed in some way, right?) how many channels?

DVCPRO HD
1080i60
1280x1080 true resolution
14MB/sec
8 bit 4:2:2 colorspace
how many channels of what kind of audio?

Sony has (finally) taken the wraps off their HDV camera, and it's about time. I was just thinking the other day that the HDV format might be in trouble since there was only one vendor (JVC) that was making two sucky cameras available for the pseudo-prosumer market.

Here are some links I used while compositing this information:

http://www.macworld.com/news/2004/09/07/sony/index.php/?lsrc=mcrss-0904

http://biz.yahoo.com/prnews/040907/latu053_1.html

http://www.camcorderinfo.com/content/Sony-Announces-High-Definition1080i-HDR-FX1-First-3-CCD-HDV-Camcorder.htm


Here's the specs:

-name is HDR-FX1
-on sale in Japan Oct. 15th, worldwide by end of year
-can't store video on Blu-ray (yet? -mike)
-price in Japan is about $3700 in US dollars
-US release in November hasn't been nailed down
-international release by Jan 1, 2005
-has 3 recording modes:
HDV mode has 1440x1080 at widescreen 16:9 (I'm guessing this is the same stretch factor that HDCAM uses)
DV mode, also at 16:9, 720x480 (so anamorphic)
DV 4:3 mode, also 720x480
-all modes are 60i (59.97 I'd imagine)
-all modes 25MBit/sec
-standard HDV: MPEG-2 video and MPEG-1 audio
-uses miniDV

0.33 inch, 1.12 megapixel CCD (run the math - is that 1440x1080?) 1.07 megapixels of effective resolution
-Karl Zeiss lense
-12x optical zoom
-color monitor is 250K, 3.5" LCD
-weights 2.0 KG w/o batteries

3 batteries available:
2.1 kg=65 min recording (any mode)
2.2 kg=130 for HDV, 240 for DV
2.3 kg model (NP-F970) a max of 205 min HDV and 215 min DV

batteries sold separately, aka EXTRA (at least on the Japanese version)

Sony expecting to sell to semiprofessional and high end consumers

prices expected to drop as other manufacturers release their HDV models

Canon & Sharp are also in HDV camp, but no products announced as yet

HDV has two resolutions: 720p and 1080i. No 1080p

transferring HDV to Blu-ray has been hampered by DRM and fears of piracy (both Sony & Hollywood). So might never, ever happen.

FROM THE PRESS RELEASE:

-Sony's new three-chip, one-megapixel Super HAD(TM) CCDs
-on-chip micro-lens on top of the CCD sensor that increases the light focusing rate for focusing on the fly
-on-chip micro-lens on top of the CCD sensor that increases the light focusing rate for focusing on the fly

More notes from other article
-the camcorder has a shooting range from 32.5mm to 390mm
-12x optical zoom
-non-perpetual zoom ring that allows for professional-like control
-option to switch between the zoom ring, the two zoom levers as well as a variable zoom control on the handle for greater shooting flexibility.
-expanded focus and peaking functions
- In the expanded focus mode, the camera's LCD image is magnified up to four times its original size without any loss of resolution (this sounds good)
-The peaking function emphasizes the outline of objects creating clear contrast and clarity in a scene (this does not - filtering the image before compression will introduce artifacts that will be undesirable for certain post operations)
-The 3.5" SwivelScreen(TM) hybrid LCD offers 250,000 pixels -- the highest resolution of any consumer camcorder LCD
-external audio level switch.
-BUT it seems there is no external line in for better microphones from what I've read so far - BUMMER!!!
- The Cinematone Gamma(TM) and Cineframe(TM) functions enable high quality picture processing to create video with the warmth, softness and richness similar to a big screen movie.
-Smooth, seamless, shot transitions are achieved using the Shot Transition(TM) function. With settings to control focus, zoom, iris, gain, shutter and white balance, focus can gradually be shifted from the front of the screen to a deeper subject, or vice versa, enabling an effortless transition in depth of field. (SOUNDS COOL)
-For creativity and control, the HDR-FX1 camcorder allows users to define their own default settings through the Picture Profile(TM) function. This function offers six different profiles that can be customized and taken advantage of, depending on the scene. Scenes may include a setting for filming sunsets, another for filming people, and another for recording in black and white. And for ultimate control, the iris, gain, white balance, shutter speed and focus can also be adjusted manually. (can they be saved to memory stick?)
-FireWire on camera for HDV transfer (of course)
-Sony is introducing their own HDV videotape - I'm cautiously dubious as to it's advantage. What's cheaper, their $18 HDV (purportedly niced up DV tapes) or miniDVCAM?
-------------------

The camcorderinfo.com article had some conflicting information, claiming the CCDs had a 960x1080 resolution. That makes the math work for an approximately 1 megapixel image acquisition. If it were 1440x1080, that would be about 1.5 megapixels, and they'd certainly claim that. The HDV spec calls for 1440x1080, which is non-square pixels. Sony is just interpolating 960 up to 1440, the same way 1440 is interpolated up to 1920 in their higher end cameras.

Should street around $3300 to $3500

"Based on our lab testing, we believe that the FX1 low light performance will be between the DCR-HC1000 and the DCR-VX2100." If the HDR-FX1 does deliver low light performance between the DCR-HC1000 and DSR-PD170, it will be some of the best on the market, as those camcorders are among the highest rated for low light performance on CamcorderInfo.com

While the camcorder does include a "faked" 24 frames progressive scan and 30 frames progressive scan mode called CineFrame 24 and 30, it does not offer true progressive scan filming

Sony officials stress that the lack of a 24P option in the HDR-FX1 is not symbolic of the company's greater direction.

The HDR-FX1 includes a Cinematone Gamma feature which, according to Sony, creates a professional-looking film-like feel for the video by improving skin tone capabilites and better 3-D depth of field.

Although not much information is available, it appears that Sony Broadcast is planning to introduce a higher-end version of the HDR-FX1. According to Sony consumer, that camcorder will be available in the 1st quarter of 2005 for under $7,000

So I got my Burly Box from MacGurus this week, but I haven't built it up yet. The world is full of complications. After lengthy research, I've learned that:

-the new 4 port card coming to market doesn't like the current Hitachi 7K400 drives currently on the market. There is a feature called SSC which the 7K400s have enabled, that the Intel chip on the 4 port card doesn't like. Harrumph.

-Hitachi will have non-SSC 7K400 drives to market...in about 8 weeks (end of October). Drat.

-the 5 port card that is even further from market will work with the current 7K400 drives (likely to be this).

So I'm dithering on what to do - whether to wait, whether to buy some lesser drives (like the faster Maxtor Maxline III drives, but they are 300 not 400 GB). Decisions decisions....

I like the idea of a hot swap rackmount solution, and have found one - the SATAview from Granite Digital. It's pricey, though, for the number of drives I want. They also make a hotswap SATA desktop chassis as well, the 4 drive system is $578, plus $30 for each additional hot swap drive sled/bracket/whatever.

-so I'm still dithering on whether to keep the Burly Box (and it's fine, nothing wrong with it for it's purposes). It may not fit my now expanded purposes for it.

Thoughts, comments, suggestions? Send'em in to mike@hdforindies.com

-mike