HD For Indies

Graeme Nattress, creator of some plugins for Final Cut Pro, wrote me to tell me of a review he wrote for the Los Angeles Final Cut Pro User Group on the BlackMagic DeckLink HD Pro and HDLink as an HD editing/monitoring solution.

Some of the better stuff from his review:

-has some nice photos that clearly demonstrate the difficulties of installing the DeckLink HD card in slot 4 in a G5

-talks at length about the quality differences between using DeckLink/HDLink combo as opposed to using the Digital Cinema Desktop Preview

-mike

The good geeks over at Accelerate Your Mac have posted a detailed report with photos on the NVidia 6800 Ultra graphics card. Why care? This is the card that can drive the 30" Cinema Display. But in order to do so, it blocks the first PCI-X slot. This review (FINALLY!) has good pictures that shows how much the PCI slot gets blocked. This is important because if you want to edit HD with one of these cards, you're going to have to have at least two cards installed - an HD card and a drive interface card (SCSI, SATA, or fiber channel). If you're using SATA, very few cards presently on the market have external ports - they are usually internal ports and you have to run cables to the outside of the box usually via an empty PCI slot cover. It would be a very tight squeeze to route cables around this NVidia card, since it intrudes into the space that would be used by a PCI card in slot 2. So for the time being, I don't recommend the 6800 Ultra for Mac HD editors working with SATA cards. This may change in over the weeks and months ahead as new hardware alternatives become available.

-mike

UPDATE SATURDAY MORNING on StEM footage actual resolution

FedEx guy was busy this week - I got two key things:

1.) The DCI StEM footage - this is the Digital Cinema Initiative Standard Evaluation Material. It is a little mini-movie they shot on film and carefully scanned at 4K (4096x3072) and carefully color corrected and saved out as 16 bit RGB TIFF files. I bought the 2K (2048x1536) version since that's as much data as I can reasonably work with.

UPDATE SATURDAY MORNING: It's 2K wide (2048), but only 857 or so pixels tall. BUMMER! This is about half the vertical resolution I was expecting. It's 2.4:1 aspect ratio, widest cinema aspect used. So for HD, I've got to letterbox it. So all the anamorphic scaling stuff below is WRONG. I just dropped it into a 1920x1080 comp, let the sides crop off, and let there be black top & bottom. Sniff

My plan is to convert that to various HD formats: 720p, 1080p, and 1080i sizes at 4:2:2 YUV and 4:4:4 RGB color space, and at 8 and 10 bit color depth. Probably about a terabyte of test files by the time I get done. Then I plan to convert it to various codecs (uncompressed, DVCPRO HD, HDV, and maybe DNxHD if I can figure out how) to see how it all holds up. The footage was all shot to specifically test for the things that digital tends to screw up on, so should be very interesting to see the results. It just dawned on me - this stuff might be logarithmic instead of linear, I'll have to deal with that, too. My "punt" position that I'm pretty sure will work:

1.) Batch all 16bit RGB TIFF files to 16 bit PSD (Photoshop) files, because I don't think After Effects can handle 16 bit TIFFs (need to verify)

2.) Import the numbered RGB PSD files into After Effects as a clip

3.) Drop those into a 1920x1080 comp and scale them to fit (they are anamorphic 2048x1536). I'll also create a 1280x720 comp.

4.) If the footage is linear, I'll have to install and figure out eLin (from Red Giant Software) to "de-log" the footage into a linear colorspace. I think that's something eLin can do, I'm not absolutely positive though.

5.) From there, I can render out all my 1920x1080 stuff:
1920x1080 @ 24fps 8 bit 4:2:2 uncompressed
1920x1080 @ 24fps 10 bit 4:2:2 uncompressed
1920x1080 @ 24fps 10 bit 4:4:4 uncompressed
1920x1080 @ 60i 8 bit 4:2:2
1920x1080 @ 60i 10 bit 4:2:2

....and so forth and so on. Those will be my digital masters for codec testing, to see how much each codec shreds the footage from the original.

I'd like to try messing with Cleaner for it's superior scaling algorithm, but I fear it's YUV conversion - I've seen some wierd results when working with DVCPRO HD codec. Also, I don't think it can handle 16 bit anything.

I'd like to use Compression Master, but again I don't think it could use the 16 bit source to compress down to.

I might end up making some kind of interstitial at 10 bit from AE at full size and do the scaling and codec writing from Compression Master or Cleaner, again I'll have to make a short test and fool with it...lots.

Debabelizer has an excellent scaling algorithm as well, but I don't know if it handles 16 bit anything either.

If anyone has any clues on any of this, I'd LOVE to hear from you at mike AT hdforindies DOT com.



2.) I also have sitting patiently on the shelf a SyncRAID XL card that I want to doodle with and see what kind of realworld performance numbers I can obtain from it. I'm thinking that might end up being a card in my server as a secure RAID 5 type setup to backup all this test data I'm generating. I have 5 160 GB drives sitting around, I could get 600 GB of protected RAID out of that. If I don't make all the 1080i variants, I could fit it all on there. Hmm....

-mike

At IBC Baytech announced the CineRAM, a 5 pound device that records uncompressed HD to RAM (LOTS of RAM!). You then transfer the data via Gigabit Ethernet to a computer.

The above is a link to their product page, here is a link to the product PDF front page, here's the back page of the PDF. From their website main page:

CineRAM Features

- Uncompressed, Portable, Solid State, Rugged
- Dual/Single Link HD-SDI (4:4:4 and 4:2:2)
- Supports SMPTE digital  formats used in digital film production
- Network attached imagery file server
- HD-SDI to gigabit Ethernet network bridge
- HD-SDI to DVI monitor interface
- HD-SDI test signal generator


This thing looks very cool and promising. The idea is you download to/through a laptop to a FireWire drive.

-mike

Huge Systems sent out an email today anouncing they have upgraded their Huge Media Vault line of RAID systems to use the Hitachi 7K400 400GB hard drives. This boosts capacity substantially, from 2.5 to 4.0 TB in their 10 drive model. Pricing is now as follows according to their email:

HMV-320R-2000-M (this is the 5 drive, standalone model) $7859 ($3.93/GB)

HMV-320RX-4000-DM (this is the 10 drive, rackmount model) $14902 ($3.73/GB)

But, as with all RAID manufacturer's claims, these prices per gigabyte aren't quite correct. The 400 GB drives actually format to 370 GB. So in a RAID 0 configuration, your cost per GB is more like:

HMV-320R-2000-M - actual formatted RAID 0 capacity 1850 GB, $7859 ($4.25/GB)
HMV-320RX-4000-DM $14,902, actual formatted RAID 0 capacity 3700 GB ($4.03/GB)

But if you're using RAID 3, as you should be for critical data backup, you lose one or two drives' worth of space (depending on whether you're using the 5 or 10 drive model respectively).

So your RAID 3 actual, usable capacity is:

HMV-320R-2000-M - actual formatted RAID 3 capacity 1480 GB, $7859 ($5.31/GB)
HMV-320RX-4000-DM $14,902, actual formatted RAID 3 capacity 2960 GB ($5.03/GB)

I'm not trying to pick on Huge in particular, all RAID manufacturer's claim their capacities and calculate their cost/GB this way. Just trying to point out this is what you actually end up with when you take the wrapper off and get it all set up in your studio setup.

How does this compare? Apple's X-Serve RAID is availabe with 400 GB drives, too, but they use 7 or 14 drives. Apple uses a fiber channel instead of a Ultra SCSI 320 connection, but both interface cards cost about the same, around $500.

Apple X-Serve RAID 0 capacities and cost/GB for RAID 0:

Apple X-Serve RAID 2.8 TB, 7x400 GB drives, actual formatted RAID 0 capacity 2590GB, $8500, so $3.28/GB

Apple X-Serve RAID 5.6 TB, 14x400 GB drives, actual formatted RAID 0 capacity 5180 GB, $13,000, so $2.51/GB


Apple X-Serve RAID 3 capacities and cost/GB for RAID 0:

Apple X-Serve RAID 2.8 TB, 7x400 GB drives, actual formatted RAID 3 capacity 2220 GB, $8500, so $3.83/GB

Apple X-Serve RAID 5.6 TB, 14x400 GB drives, actual formatted RAID 3 capacity 4440 GB, $13,000, so $2.93/GB

Those capacities would be the same for RAID 5, RAID 30, and RAID 50 as well.

I'd recommend adding some other options to the X-Serve RAID, but to keep this as apples to Apples as possible I'm leaving those out for now.

In general, I'd be curious to see how Huge defends their pricing against Apple, especially since the Apple solution can be integrated into an X-SAN Storage Area Network environment for $1000 per node plus a metadata controller server.

With the caveat that I haven't worked hands on with either system, assuming that both systems worked equally well (and that's a large assumption), I'd be hard pressed to not go with the all Apple solution - if there's a problem, there's only one vendor to yell at.

-mike

There's Blu-Ray, there's HD-DVD. Looks like we're heading into another potential VHS vs. Betamax war. Read about it here.

Need to do a shot involving mounting the camera on a car? Either on the hood looking in, or looking out at the road? Here's a comparison of two products specifically made for getting those kinds of shots using smaller, lighter cameras like DV, HDV, and maybe something bigger. StickyPod vs. The Gripper. (Sounds like a wrestling match: "Sunday Sunday Sunday! Be there...")

UPDATE THURSDAY NOON CST: The DOCK has an s-video out port. Confirmed on specs page here. Also, I went by the local Apple Store yesterday to help a friend buy a laptop, and noticed that they had Photo iPods on display. Didn't ask if they were in stock. It does in fact show the artwork picture when playing back a song, but when showing song info the picture is dinky on the side. But it's there.

Been thinking about it some more since yesterday's post on the Photo iPod - Steve Jobs thinks photos are the "next big thing" because everyone has them, they are quick and easy with digital cameras, and there are no rights issues.

It also looks like there might be an s-video connection on the dock....or maybe that's just the multi-plug for audio and video. Dunno. Looked like an S-video plug but with too many holes.

This just means, from a business/manufacturing side, that it CAN work.

That does NOT however, in ANY way, create demand for the products!

As a color screen iPod with more space (on the 60GB model), I think that's great. Supposedly it'll put up the album art for each song it's playing if you have your iTunes set up with'em attached. There are programs to do that now for you, too - just point'em at your iTunes collection and it grabs images off of Amazon (sweet!). Search for them at versiontracker.com, I'm too lazy to do that right now.

Anyway, the $100 extra feature to display photos - personally I don't find it all that compelling. We'll have to wait and see.

-mike

This is completely off topic, but I like iPods, and Apple has some new ones. Plus, for the record, I think I'm the first to have analysis up rather than just the press release stuff. Cuz I'm like that. : ) Anyway, the goods:

iPod Mini: same as before, but now free personalization - get your name engraved on the back for free - looks like 2 lines of info allowed (name & #, name & email, etc.) Still $249, still 4GB, still same 5 colors.

Photo iPod : OK, this is new - 40GB is $499, 60GB is $599. It has a 220x176 pixel, 65,000 color (16 bit color) display. Menus are in color now. iTunes 4.7 (from Apple website): "...lets you import your images directly into iPod Photo from a folder on the hard drive of your Mac or PC." If using Adobe Elements 3.0 or Adobe Album 2.0, can organize your albums and load onto iPod Photo from a PC. Obviously, use iPhoto on Mac. iTunes 4.7 allows for syncing Photos to the unit. I'm guessing that downloading photos will work the same way with the Belkin card reader doohickey that already worked with prior generations of iPods. If not, they'll have a compatible unit soon. It has an A/V port so you can show the pictures on a TV set. Based on viewing the tiny photo on this page, the connections appear to be three RCA connectors - RCA stereo pair for sound and composite video. Eew. I was hoping for an S-video output option at least - composite video is soft and smeary. Maybe that will be an optional cable at some point, but I don't see any such thing on the accessories page today.

U2 iPod - it's 20GB, $350, black with red click wheel. Uh, yeah. Signatures of the band on the back. Comes with a U2 poster. Comes with $50 off coupon for the The Complete U2 - over 400 tracks, including 25 "rare and unreleased songs." I had heard rumor the new album was to be included on the unit, but I don't see mention of that on the page.

"Regular" iPod now in 20GB ($299) and 40GB ($399) sizes. 40GB includes Apple Dock, 20GB doesn't.

Mike's Comments:

iPod Mini a slightly better deal with the free engraving if you buy direct from Apple. Apple continues to try to get all sales to go through them instead of retailers. Good for consumers, bad for retail sellers.

Regular iPod: Is this any different than what came before? Maybe the inclusion of dock with 40GB?

iPod Photo: Interesting...could be a budget presentation machine, would be cool to carry all your presentations in your shirt pocket. Photo sharing? Now you can bore the whole office, not just your roommates with trip photos....my main interest (personally) in this unit is 60 GB capacity - enough to install and run the OS if you need to, and keep all your digital goodies in reach. Make presentations on the road from this tiny thing. I'm sure somebody's going to come out with some middleware to help that possibility. Plus have a zillion songs around, too, as well as all your contacts and your calendar. It's creeping into PDA territory here.

iPod U2 edition: I think Apple and U2 have Jumped The Shark on this one. Even seeing the ad on TV and online, I was kinda creeped out by it, felt like we crossed the commerce/art line too hard here. The iconoclastic, non-individual black outlines in the Apple ads was nice because they weren't anyone specific. Now by sneaking in a greyscale Bono face, it makes it SOMEBODY, and that changes the feel of the ads. Bono with the white iPod signature cord coming off the microphone? It just didn't feel right to me. Plus I think the red on black iPod itself is rather, well, hmm....wasn't black and red cool back around the time the Heavy Metal movie came out? I'm just not down with it. Expect these to be available cheep online or eBay in about 4-6 months.

Just my $0.02.

-mike

The Apple Store is down at the moment, pending an announcement. Probably the new U2 black iPod, perhaps the new 60 GB iPod with photo display, perhaps the new Flash based iPod.

We'll see soon, I'll keep all posted.

New iBooks were released recently, good enough for DV, HDV, and DVCPRO HD editing (but not pixel for pixel HD image display).

-mike

Saw the press release so I thought I'd pass this on - Gorilla is a piece of software to help you plan and produce your independent film. From their website:

With Gorilla, all of your information can be stored in one place: Budgeting, scheduling, crew, cast, profit-sharing, editing notes, and even film festival submissions.

When it's time to create reports, Gorilla can generate over 50 industry-standard reports, from stripboards to breakdown sheets, budget top sheets to variances, your entire production can be run from one program.

Don't know anything else about it, so caveat emptor. But might be useful. The logistics of moviemaking are fearsome, get some help somewhere.

-mike

ATTO has announced a new fiber channel card, the Celerity FC-42XS that has two separate 4 gigabit fiber channel connections for attaching to high speed fiber channel hosted storage, such as Apple's X-Serve RAID.

Each channel can run up to a maximum theoretical throughput of 800 MB/sec. In a Mac G5, this allows for a maximum throughput of 1 gigabyte per second (1 GB/sec), due to the bus limitations of the 133 MHz PCI-X slot.

Mike's Comments: Interesting. I guess you could hook up X-RAIDs to each port, maybe stripe them all together for collosal throughput? Enough throughput for capturing 2K files in realtime, and playing back perhaps 2 streams in realtime, if you had a system capable of displaying that...maybe?

This is complete conjecture, I don't know if it would work: Could you hang a couple of fully populated X-RAIDs off this card in a G5 and display all of a 2K (2048x1536) image on a 30" Cinema Display in realtime, perhaps via Final Cut Pro HD? Dunno, but it would certainly be interesting to try (if I had the $35K in equipment to do it).

-mike

The HDTV Expert page is a nice little page on HDTV issues (the TVs, not production). This article explains some of the hype around manufacturer's quoted claims for contrast numbers for their displays, and is helpful for anyone trying to get a good understanding of display technology, and what makes for a good image. Pay special attention to the part about black values and greyscale representations, especially in the darker range of grays. This matters when you're trying to use a display device to simulate how an image will look on film, or even on another kind of display device (LCD, CRT, projector, film, etc.)

-mike

No, I didn't update EVERY article today, just changed the way the feeds were handled to increase browser compatibility. Now each article linked from an RSS feed shows up on it's own page. This will work with more alternative browsers, as well as make browsing archived articles a little more logical.

I'd recommend setting your RSS and web browsers to open a new window for each link clicked on if you haven't already.

What is RSS? It's a way to quickly and easily see when a site has been updated if the site has an RSS feed.

My Mac RSS feed reader of choice is NetNewsWire.

I use it to quickly skim for new articles from about 40 different websites multiple times every day (how I bring you all these links).

-mike

Interested in optimal display of HD images, and/or the REAL resolution of all the newfangled expensive TVs? Read this.

headline says it all...read the article here.

I'd quibble with some of their details - they didn't catch all the price drops and new models BlackMagic came out with at IBC, and they don't mention the possibility (even the future possibility) of SATA arrays for uncompressed HD work. They also mention multiple striped SATA drives for compressed HD work - simply not true - single SATA is fine for compressed HD.

-mike

THIS is a great article that I read with no small amount of glee - Windows Media 9 is "hitting some snags" as Slashdot put it.

Some salient quotes from the EE Times article:

A number of technical and political issues surrounding VC-1 have reportedly caused growing frustration and constant bickering in the SMPTE engineering community.

The uncertainty has raised questions about the future of Microsoft's Windows Media Video codec. On the assumption that WMV9 was destined to become an industry standard, Microsoft convinced both the Blu-ray Disc Association and the DVD Forum to include it as a mandatory video compression format (along with MPEG-2 and H.264/MPEG-4 AVC) for next-generation high-definition DVD formats. Now, there is speculation that delays or licensing problems for VC-1 could prompt either — or both — of the DVD industry groups to simply delete the Microsoft technology from their specifications.

Microsoft created the impression in the industry that its WMV9 codec had a leg up on H.264/MPEG-4 AVC in quality and licensing terms. But now that the WMV9-based VC-1 has been put to the test in the arduous SMPTE standardization process, VC-1 is "perceived as behind in quality

At the heart of the issues that have slowed SMPTE's standardization process lies the industry's general suspicion of Microsoft. One industry analyst characterized Microsoft's donation of WMV9 to SMPTE as a "calculated maneuver to gain respect for the proprietary technology"

the SMPTE group still needs a reference software decoder and reference bit streams to ensure interoperability among different VC-1 implementations. This work is not yet completed. Nor are there any signs of a reference encoder for VC-1.


The article goes on to say that the VC-1 standard (which is based on the Windows Media 9 codec) might get dropped altogether from the spec for both Blu-Ray and HD-DVD.

As the Slashdot article put it, "Now it turns out that Microsoft cheated and lied: its code is not as good as MPEG 4, the WMV9 reference implementation is not available, and the WMV9 test suite does not exercise all the features. The SMPTE might drop WMV9 after all. Apparently, a highly technical standard body is harder to snowjob than the usual clueless consumers."

HEAVILY updated Saturday afternoon, 4pm CST

They were backordered forever, and now I know why. La Cie had run out of 1TB Bigger Disks with FireWire 400 and FireWire 800 (aka 1394a and 1394b IEEE standards), and said they'd be coming out in mid-September. Then maybe October. Then they quietly rolled out a triple interface version, the La Cie Bigger Disk Triple Interface. This new replacement model has the same FW400 and FW800 connections as before, but now it adds a USB 2.0 interface (and if you were suicidaly paitent, USB 2.0 is backwards compatible with USB 1.1, albeit much much slower).

First Impressions:

The unit harkens back to the days of the full size, full height 5 1/4" hard drive enclosures. By normal desktop drive standards, this sucker is HEAVY. That's because inside this aluminum enclosure there are FOUR hard drives on a commond bus with a hardware RAID 0 controller. I'd gotten some conflicting information at one point as to whether it was a true RAID 0 (data striped across 4 drives) or a JBOD (Just a Bunch of Disks, also known as spanning, where one drive is filled with data, then it rolls over to the next, etc., only giving one drive's worth of performance at all times). Based on the performance I'm seeing, I'm thinking it must be RAID 0 - 70 MB/sec read times are beyond what any single 250GB drive can do.

After 2 1/2 hours of hard formatting, it appears to be slightly under 20% done. So this will probably take about 13 hours (give or take) to fully format. So that takes quite a while. It also makes the fairly small fan run loud. Granted, I'm telling it to constantly write data non-stop for hours at a time, but it's little fan is going full-tilt, to the point that it is much louder than the dual processor G5 it's sitting next to. Louder than anything else in the room electronic (and there's a LOT). So I'd love for them to put a quieter fan on it, preferably larger diameter. The front has a nice long air intake space, but the back has a roughly 1 1/2" meshed square that it releases the heat exhaust from. Too small.

Note: what I'm calling a "hard format" is just using the Write Zeroes option in the Options tab on the Erase screen of Apple Disk Utility under OS X 10.3.x. The reason for doing this is to map out all the bad sectors on the disks that might exist from the factory. This is a good way to make sure the disk is as reliable as possible. I do this with ALL my disks before I use them, even though it takes geological timeframes on large, modern multi-hundred gigabyte disks. It also deletes all data on the disk, so it's best done right when you get it, before you put any data on the drive whatsoever. The La Cie does arrive preformatted for use on Mac, very conveniently. I just like to take this extra step, since I'm going to be using this RAID 0 device for backups. And as with all RAID 0 devices, if one drive in it dies, ALL the data on teh RAID dies.

But while trying to zero write the drive, which looks like it would take about 15 hours, it failed. Repeatedly. Four times on two machines. It looks like Disk Utility doesn't know how to handle the "jump" from one drive to the next. After 7-9 hours, it always fails. Sigh. No zero write for these units.

Next up: performance testing.

FireWire 800 Performance

Interestingly, performance was pretty consistent across the entire capacity of the drive, whether at the beginning or the last 28GB of the drive. Using just the FireWire 800 connection (the fastest) I partitioned it into 5 equal slices, except for the last, which I made a little smaller to allow for an 'end of disk" section of 28GB. All parts of the Bigger Disk had virtually identical throughput that I tested - about 71 MB/sec reads, 51 MB/sec writes. I used BlackMagic Design's Disk Speed Test utility to derive those numbers, running on dual 2.5 GHz G5 running OS X 10.3.4, version 4.6 of the testing utility.

This would imply that this drive (a 4 drive RAID 0, but for simplicity's sake let's just call it a drive) would be perfectly suitable for any standard definition video work, capable of capturing any standard definition video source, be it DV, DVCPRO50, uncompressed, whatever. It might also suffice for dual stream SD editing. Theoretically possible, but haven't tried it yet.

For HD uncompressed work, this drive would be unsuitable, with the possible exception of 8 bit 1280x720 4:2:2 24 frame per second capture. 10 bit is right at the capability limits of the drive with no safety overhead, so you'd be bound to drop some frames, an unacceptable option. But since the most common 720p24 source is the Panasonic Varicam camera, and its tape format (DVCRPRO HD) is only 8 bit, it should work. Plus, for DVCPRO HD, you should be working with a Panasonic AJ-1200A deck with FireWire, anyway.

For compressed HD work, however, you should be fine - any DVCPRO HD via FireWire should be OK (as long as the 1200A deck is on a separate FireWire bus), BlackMagic's PhotoJPEG would be fine, AJA's Q-Rez should be fine for any 4:2:2 work and 24p 4:4:4 work (if they offer 4:4:4 with Q-Rez, not sure).

FireWire 400 Performance

Just for fun I connected via FireWire 400 as well and tested again. This time, I used 5 185GB partitions, and a last one of 10GB.

Read Performance (avg of 3 runs): 35 MB/sec
Write Performance (avg of 3 runs): 28 MB/sec

And again, that performance was across 6 different partitions of the drive, so it's consistent (so really 18 test runs per read and write).

This is clear proof of how much better FireWire 800 is than FireWire 400. This kicks a bunch of stuff out. It should still work for any standard def footage including uncompressed (barely!), and no HD uncompressed is possible with this. HD compressed should still be about the same, but fewer streams possible with lower bandwidth available.

But if all you have is FireWire 400 ports on your G4 or PowerBook or iBook, this is an indication of what's possible (all these tests were done on G5, remember).

USB 2.0 Performance

And just to be thorough I thought I'd try USB 2.0 performance. In part to see what it could do, but also to see if this would make it possible to edit DVCPRO HD footage at 720p or 1080i on my laptop. Since my 12" PowerBook (and other PowerBooks, especially some older ones that still have the minimally required 1 GHz G4 processor) only has a single FireWire 400 port, this creates a problem. When dealing with the Panasonic AJ-1200A deck that handles DVCPRO HD over FireWire, the deck wants to be the only device on a given FireWire bus, otherwise it drops frames and pitches a fit. So how to get around this? Perhaps by using the USB 2 interface on some external drives if you want more capacity than what is possible just using the internal drive in your laptop (and capturing to boot drive is always an "only if desperate/have to" situation, increasing the chance of dropped frames on capture and playback).

Read Performance (avg of 3 runs): 15.5 MB/sec
Write Performance (avg of 3 runs): 15.5 MB/sec

Sad, isn't it? This should suffice for the following video formats: DV, HDV, DVCROPRO HD 720p resolution (but probably not 1080i resolution, which is 14.5 MB/sec, way too close for safety, dropped frames for sure).

If using this setup, be sure to use a High Speed USB rated cable (USB 2.0, not 1.1 or 1.0!) and double check in System Profiler that it says "Up to 480 MB/sec". Just because the bus is theoretically capable of something doesn't mean that you will get performance ANYWHERE near that.

Throughput testing conclusions:

This is a great example of how much difference the drive interface makes. Inside this device are four ATA 250GB hard drives. They can run up to about 50-60 MB/sec each I would imagine (not knowing the exact model used, but that is my experience with modern SATA drives).

On USB 2, only 15.5 MB/sec. On FW400, it increases to 35 MB/sec. On FW800, it increases to 70 MB/sec.

But if I hook up 4 native SATA drives to a good native SATA connection, it's easy to get well in excess of 200, sometimes 240 MB/sec of read performance.

So the interface counts - a lot. And as I mentioned above, just because the interface CAN run that fast doesn't mean it WILL run that fast. Bummer but true.

Noise

The unit is LOUD however, when it's cooling fan fires up - it is MUCH louder than the G5 is most of the time. Besides the fact that I primarily intended this for backups for my own use, this is another reason to use it for backups- I wouldn't want this on/under my desk while I was working, especially if working with audio. I hope La Cie fixes this - it's as loud or louder than the G5 when the G5 is REALLY working or getting hot.

So far, though, it's a fine backup solution, and with deals like the $705 Dell deal I mentioned a few days ago (scroll down), at about $0.75/GB for 935 formatted GB of capacity, it's a great solution for overnight backups of a LOT of data, such as one might use for uncompressed HD work on a RAID 0.

-mike

I'd heard about Mark Van Bavel and his DV Film company a while back, but I finally got a chance to meet him recently and sit down and talk with him, and then we were on a panel together at the Austin Film Festival. His company does transfers from digital source (DV, HD, digital files) to 16 or 35mm film. While not offering the higher end services of companies like eFilm, Technicolor or SwissFX, he also doesn't charge anything close to their rates. For smaller independent projects without extensive post budgets, it's a must-know-about facility.

Whereas I tend to get wired when I talk about all of this digital stuff and to do a lot of future projection of possibilities, Mark remains very calm and speaks from experience about cameras and software, and especially the results that can be obtained, since he's been outputting film from digital source for at least 4 years (that I know of).

He also has an excellent FAQ section on his website, and wrote a book on DV filmout, and offers DV Film and DV Atlantis, two software packages for use in DV=>film conversions. I hope to hit him up for review copies of those at some point and publish commentary on them (the book too).

-mike

Longtime contributor Christopher Barry sent in a bunch of links about HDV and the Sony HDR-FX1 1080i resolution HDV camera:

One of them explained some of the differences between the consumer (HDR-FX1) and pro model (HVR-Z1) models, outlining the features the HVR-Z1 will have that are better than the HDR-FX1:

Some differences of HVE-Z1 versus HDR-FX1:

1  Stereo AND mono microphone on top
2. 2 x XLR Input
3. Black body (instead of charcoal)
4. Separate recessed audio volume controls
5. Time Code preset
6. Camera switchable between 50 Hz /60 Hz
7. Simultaneous use of LCD display and EVF (Viewfinder)
8. EVF switchable between B&W and colour
9. Hyper Gain, zoom display w/numbers, hours meter, Audio Noise Reduction, audio limiter, action safe zone display.
10. DVCAM tapes


NO true 24 or 25 frames per second nor 720 mode.
Sony's point 720 is NO true HD.  Interlace is a must for smooth motion reproduction! And they are wright. Also all that fuzz about 25P. Its only a very few appreciating and using this mode proper!
For US (NTSC)  30P users there might be a difference, but why not buy the better picture in PAL version?

Sony also has an HDV info site with discussion forums.

Here's another review of the HDR-FX1, this time on Hardware Zone.

Just wrote the HD storage options article, and it's been pointed out to me that not everyone wants or needs to edit a feature film. The possibility of industrial films, commericals, or short films still might required uncompressed HD. So on that note, here's some new twists for those needs:

The good news is that if you want to do an uncompressed HDCAM short (or uncompressed Varicam short), it can be done for very little. Consider this:

DP 2.0 GHz G5: $2700 w/2.5 GB of third party RAM
FCP HD: $1000
DeckLink HD: $600
HDLink: $700
HP 23" LCD: $1600
19" monitor: $350
4xBarracuda 7200.7 160GB array: ($100 Seritek 1SE2 (lotsa online places, pray to Google), 4x$113 drives (zipzoomfly.com), 2x$50 PPA, Inc cases (Fry's Electronics), $190 external FW800 drive (La Cie)) - $842

total: $7792

Uncompressed 24p HDCAM time: about one hour 45 minutes
Uncompressed 29.97 interlaced HDCAM time: a bit over an hour, since I don't think the array would be fast enough to reliably capture 1080i60 (another way of saying 29.97 interlaced) all the way to the slowest (end) part of the drives in the array

Now you're all set to capture, edit, and accurately monitor 1080 res HDCAM footage. Spend $100 each on a jog/shuttle controller (Contour Designs Shuttle Pro v2) and an editing keyboard (lots, look around or search this site using bar at top of page) and you're still under $8000 for your uncompressed HD system. Want backup? Now you're under $9000.

Want data protection and only using 24p HDCAM? Use the 5 port SyncRAID XL card and a 5 bay Burly Box (no coolers, on budget!) and another drive, get same array capacity of 594 GB (magic of parity math), drop the FW drive (don't need it now) and the price is $8085. If 24p HDCAM is all you're doing, this is clearly the way to go if you're on a shoestring budget (but gotta render your transitions, no realtime - can you live with that?). Want more capacity? Just use bigger drives. See zipzoomfly.com for good SATA drive pricing. 250's go for about $160, 300 GB for about $210, 400GB for about $400. You'd need 5 of'em.

-mike

Normally I would consider this OT (off topic) for this blog, but this is so cool I decided I wanted to include it. Ampede, maker of that PDF rasterizer for Final Cut Pro, has introduced LayerLink, which lets you import an Illustrator file as a project, with all of its layers intact.

"WTF" is what any decent, respectable filmmaker is saying - "why care?" As a (former) motion graphics artist, this is one of those core functions that sounds like nothing but makes your life miles easier. If you're doing DVD graphics, or title sequences, or anything that would require flying some carefully made type or artwork around, this level of control is crucial. Since it is a native Motion plug-in, not a ported After Effects plug-in, it is full-on, instant, realtime performance.

If you're using Motion more than a day a week, this will probably be a must-buy for you.

It's normally $120 but $99 as an introductory price.

I've talked about Avid's product line in the past on HD For Indies, but for those that want a taste of what the Avid interface is like, download Avid Free DV, their "free as in beer" giveaway version. Think of it as their iMovie - free, consumer/low end, lacking the heavy features. I haven't even looked at it (yet), but I thought it worthy to make note of it here if you're new to this stuff or an FCP user that wants a clue as to what Avid's interface is like.

-mike

La Cie has launched a bunch of new products today, including a new high quality LCD monitor with 10 bit gamma correction (useful for calibrating the display), an Ethernet based NAS (Network Attached Storage) device with 160 to 800 GB of storage that is expandable via FireWire 400, and of interest to me, an external SATA drive that comes with a 2 port PCI card with external data and power connections. Very interesting.

The drives are available in 74GB 10krpm ($449), 160GB ($269), 250GB ($339), and 400GB ($549) sizes and come with one of the dual port cards, and are expected to start shipping in mid-November.

I want to find out more about this set up. According to information on the site, it's a 32 bit, 66MHz card. There is reference to "up to 85 MB/sec" for the drives, and I'm not sure if that is the maximum speed of the drives or the cards. And if you put two cards in slots 2 & 3 in a G5, will the performance capabilities double to 170 MB/sec or not? A very, very interesting question for the things I'm trying to do in studio right now. Might have to order me one or two of these just to play with. That opens the door to a convenient 6 drive SATA array (2 internal, 4 external). With 400GB drives, that would be 2.4TB of RAID 0.

In any case, for the past few months or so I've been seeing a lot of PC deep geek equipment websites advertising various external SATA solutions, so it makes sense for a more mainstream vendor like La Cie to get in on the act.

-mike

With both the SyncRAID XL (see other article, scroll down a bit) and revised X-Serve RAID hitting the market this week, I've decided to take another look at storage options for those wanting to do uncompressed HD editing.

For those wanting to work with compressed HD, rejoice - plain old ATA and SATA drives will suffice just fine (and even FireWire drives under certain circumstances - just don't put FireWire drives on same bus as Panasonic AJ-1200A deck!).

(I've stolen the next paragraph of two from the X-RAID article I just posted, if you've already read it skip on down, the rest is All New!)

Apple X-Serve RAID - now with 1.0, 2.8, 3.5, or 5.6TB of capacity

If you're serious about posting an uncompressed HD feature, this new system (with 3.5 or 5.6TB) is THE low cost, high throughput, data protected (RAID 1+0, 3, 5, etc.) solution. 380 MB/sec read speed is very respectable - supposed to do dual stream 10 bit HD - and that's at RAID 5 if I am reading the press material correctly.

Let's really break down the numbers though. At first blush, it seems like the 5.6TB unit it just over $2/GB. $13,000 divided by 5.6TB is $2.32/GB. Not quite the "just over $2/GB" figure Apple suggests. But how much usable space do you get out of it? All drives lose about 8% of their capacity when you format them, regardless of single drive vs. RAID. So your 14x400 is really 14x370, so usable space is really 5180 GB. And if you're using RAID 5 (because RAID 0 has no data protection), the X-RAID is really two 7 drive RAIDs paired together (each with their own controller), and since each RAID 3 (or 5) loses one disk worth of space for parity, instead of 2*7 drives worth of space, it's more like 2*6 drives worth of space - 4440 GB. So that pushes the price up to $2.92/GB. And if you add the recommended battery powered cache backup ($350) and the required host fiber channel host card ($500), your price is $13,850, or $3.11/GB as a practical, realistic, installed cost.

I'm not dissing the X-RAID here, I think it's a great product in concept (see MacInTouch.com threads for some issues folks have had) and I'd heartily recommend it for all who can afford it. Just pointing out that the end quantity of storage and end costs are not quite as rosy as at first seems...but this is the case with all storage vendors, I'm not meaning to pick out Apple in particular.

(OK, all this below is brand new)

The traditional answer for uncompressed HD used to be a pure SCSI approach, but then vendors like Huge and Medea came out with hybrid systems that allow for a SCSI interface connecting to ATA or SATA drives, bringing costs down. But the costs of a 10 bit HD capable system are still in the $5-$7/GB range, so while they are entirely proven, reliable, plug 'n play solutions, I'm moving past them for now due to cost reasons. If you HAD to have, today, a reliable RAID 3 10 bit HD capable (and who doesn't need that?) lower capacity/overall cost solution than X-RAID, Huge or Medea are the way to go, even though you'd pay more per GB. The X-RAID has good performance and cost/GB, but only in the above $10,000 price points.

Especially in comparison to the RAID 0 with backup option I'd been advocating recently, the new X-Serve RAID stuff starts looking more attractive. Especially since my conversation with Rick at MacGurus - he pointed out that drive failure is NOT the primary cause for concern with RAID 0 (or RAID 10) - that directory corruption was the biggest concern, because according to him none of the Mac based tools can fix that on a RAID 0 or RAID 10. You'd be hosed if that happened, relying on restoring from backup, a timely proposition. My best online RAID 0 with nearline (FW800 based backup) solution is to do a 4x400 drive RAID using a Seritek 1SE2 and 2 internal drives, using the Hitachi 7K400 SATA drives, booting from a FireWire 800 drive. Back up to 1TB La Cie Bigger Disk ($1000, so $1.07/GB of usable formatted capacity)UPDATE: At the moment (don't know how long), they are available for $705 from this Dell link). That would give you 1480 GB of usable online speed capacity for HDCAM footage, but only about 500 to 700 GB (this is a guess) of 10 bit 4:4:4 1080p24 capable throughput. For 1080p24 10 bit 4:4:4, at this point you'd have to use a Highpoint RocketRAID 1820A card, but there are reliability problems that I've yet to resolve with this card so I can't recommend it at this point in time for front line prodution systems. Maybe later after testing and analysis or new drivers, but not yet.

So looking at those numbers:

Highpoint RocketRAID 1820A RAID 0

1820A card with SoftRAID, 2 4 bay Burly Boxes (with coolers) 8x300 array (Maxtor DiamondMax 10 drives, $209/ea from zipzoomfly.com):
RAID 0 usable capacity: 2224 GB
Cost: $2635
Cost/GB: $1.18/GB

RocketRAID 1820A RAID 10 option

RAID 10 usable capacity: 1112 GB
Cost: $2635
Cost/GB: $2.36/GB

...and with RAID 10, according to Rick, you still need to worry about directory damage. I need to research this issue further.

RocketRAID 1820A RAID 0 with FireWire 800 backup:
Adding 2 La Cie Bigger Disks and a d2 Big Disk (500GB)
Storage amount (real, usable, formatted capacity): 2326 GB
Storage cost: $2450 (2x$1000+$450)
Backup Cost/GB: $1.05/GB

SO, RAID 0 PLUS BACKUP COSTS:

Storage capacity (primary, backed up): 2224 GB (backup doesn't add to this total, just backs it up)
Total cost, primary plus backup: $5085
RAID 0 with backup cost/GB: $2.28

Now compare that to Apple's $3.11/GB. Apple's is a constant, unattended protected dataspace, you never have to fool with it, it's just THERE. But Apple's solution is only that price point if you commit to (and need) the whole 5.6TB package. The only advantage I can think of to my solution is that your backup can be unplugged (or even offsite, although that's not practical if doing overnight backups) to prevent lightning or other electrical catastrophe. Nice, but not a huge advantage. Plus, the RocketRAID has not proven itself stable and reliable yet. I'm withholding final judgement until I can finish my own testing.

Other alternative: Seritek 1SE2 with 2 internal, 2 external drives booting from FireWire 800

For this application, we're not looking for maximum low cost capacity. In the previous example, I was using Maxtor DiamondMax 10 drives, because they are fast, large, and have a low cost/GB. For this application, since we only have 4 SATA drives available (last I heard the performance with 2 cards wasn't viable), I'm choosing IBM/Hitachi 7K400 400GB drives, which are available for $390 from zipzoomfly.com (price changes frequently, however). Taking the RAID 0 with FireWire backup approach, here's how it can be done:

(This reminds me - I need to try a slot 2 & 3 combo and see how it performs, I haven't tried that yet in my dual 2.5 GHz G5.)

Seritek 1SE2 card: $100 (gives two external SATA ports)
2 bay Burly Box (with coolers): $190
SoftRAID (allows partitioning): $100
4 Hitachi 7K400 400 GB SATA drives: 4x$390=$1560
1 La Cie d2 160GB hard drive to boot from: $190 (pick your favorite cheaper FW800 drive if you wish, I'm picking a nice/safe/easy choice)

Total cost: $1950
Total usable RAID 0 capacity: 1480 GB
Cost/GB: $1.31

Now adding in backup storage, using a 1TB LaCie Bigger Disk and 500 GB d2 Big Disk Extreme:
Total usable capacity: 1393GB *
Backup Cost: $1000 + $450=$1450
Backup Cost/GB: $1.04/GB

Total Back-upable Storage Capacity: 1393GB
Total Storage Cost (primary plus backup): $3400
Total Storage Cost/GB: $2.44

* You didn't want to fill up your array to the last drop anyway, right? Seriously, HFS+ formatted volumes slow waaaaaaaaay down once over 90-95% full.

I looked into the thought of building your own FireWire 800 drive, but the La Cie is cheaper than what I could find out there. I looked into using $50/ea PPA, Inc. cases instead of 2 bay Burly, but wasn't saving enough to make it worth the substantial quality drop. I looked into loose IDE drives and cheap FireWire 400 cases, but the cost savings was small (again a couple of hundred dollars), but the increased hassle factor was HIGH. Do you REALLY want to manage 5 individual drives for your backups? Me neither. Yeah, you could squeeze a few hundred bucks out of this, but why? More trouble than it's worth. If you're squeezing it that tight, you clearly have a LOT more time than money and don't mind spending it (the time). That's a section of the market that I'm just not interested in serving, although that may be where some folks have to play.

SyncRAID XL for 24p HD-CAM or Varicam based projects

There's a new hardware based RAID card available for Macs, which is great. The bad news is that it is 33MHz PCI, so it will slow down a G5's slot 2 (or 3) if placed in slot 3 (or 2) since those two slots share a bus and slow down to accomodate the slowest card in either slot. So that means a single SyncRAID XL is all that is viable at a time in a G5 with an HD card installed. HD card goes in slot 4, SyncRAID XL in slot 2 or 3. Or swap - SyncRAID in slot 4, HD card in slot 2 (cables need to run out an empty PCI slot cover, slot 3 in this example).

The good news is that it will use what they call RAID XL - it is similar to RAID 3 or 5, but just different enough to not be. It writes all it's parity data (redundant data that can rebuild a lost disk's info in event of failure) to a single drive like a RAID 3, but does tiny chunks of data more like RAID 5. Anyway, the card can do 114 MB/sec reads and writes, and the throughput is limited by the card itself, so faster drives are useless. This means you can go buy 5 drives and get 4 drives worth of storage capacity out of the system. So long as your drives can do at least 30 MB/sec even when full, you'll be fine to maintain that 114 MB/sec throughput the drive is capable of. Pretty much all of the recent 250 GB and larger drives can do this.

It does have internal ports, however, which creates the same problems I've been addressing in my ongoing coverage of the Highpoint RocketRAID 1820A card. Search for "1820A+cabl" using the search bar at the top of this site for more info on that topic.

SO:

For those doing 24p HDCAM projects, or Varicam (720p at up to 60fps), this card is fast enough. To be specific: if you're doing 8 bit, 4:2:2, 1920x1080 at 23.976, 24, or 25 fps, this card should just squeak by as fast enough for capturing and single layer work. If you shot at a video friendly 29.97 interlaced frames per second, you're hosed - this is not the solution for you. That data rate is about 118 MB/sec, and needs some safety margin above that. This card only does 114 MB/sec in RAID XL configuration, and that's IT.

If you're doing Varicam work (1280x720p) and capturing via HD-SDI, 720p at 60fps (the max rate) isabout 105 MB/sec, which is getting dangerously close to the limits of the card. But more likely you're working at 720p24 or 720p30, so the data rates are 42 or 52 MB/sec (approx), which is quite comfortably handled by this. I also just realized that if you're working with 720p24 stuff, a pair of SATA drives on a Seritek 1S2 or 1SE2, or a La Cie Bigger Disk, would suffice for single stream editing.

If you are doing a 24p HDCAM project, get one of these cards, a 5 bay Burly Box (or other compatible enclosure), 5 drives, and you are off to the races. Let's look at a couple of possible configs. The benefit of this system is that you DON'T need backup the way you did with the RAID 0 based systems described above. As it has been described to me, you can yank out a drive in the middle of a playback operation, and the system will choke for a few instants then keep playing back just fine. Way way cool. The downside to this card is that there is NO, ZERO, NONE, tech support available from the manufacturer. They have Mac drivers and that's all the effort they have or are going to put into it anytime soon. That's why I'd buy it from a knowledgeable vendor like MacGurus, Rick knows what he's doing over there.

here's a few of different ways you could set it up:

1480 GB with Hitachi 7K400 400 GB drives

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 400 GB Hitachi 7K400 drives: 5x390
(no SoftRAID needed)

Total usable RAID XL capacity: 1480 GB (same as the Seritek 1SE2 example above - one drive is dedicated to parity data)
Total cost: $2520 w/out coolers, $2665 with coolers
Total cost/GB: $1.70/GB without coolers, $1.80 with (might as well!)

=================================================

Using Maxtor DiamondMax10 300 GB drives (better cost/GB, smaller, faster)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Maxtor DiamondMax10 300 GB drives: 5x$209
(no SoftRAID needed)

Total usable RAID XL capacity: 1112 GB
Total cost: $1796 w/out coolers, $1940 with coolers
Total cost/GB: $1.61/GB without coolers, $1.75 with (might as well!)

=============================================

Using Hitachi 7K250 250 GB drives (better cost/GB, smaller size, though, same performance as 7K400 models)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Hitachi 7K250 250 GB drives: 5x$148
(no SoftRAID needed)

Total usable RAID XL capacity: 928 GB
Total cost: $1552 w/out coolers, $1702 with coolers
Total cost/GB: $1.67/GB without coolers, $1.83 with (might as well!)

==============================================

To get accountant geeky, you'll note that the increased cost/GB efficiency of the drives is largely offset by the fixed costs of the housings and SyncRAID card. So your cost/GB holds to a pretty narrow band - about $1.75/GB ($1.61 to $1.83 is the range).

In the limited context of 24 or 25p HDCAM (8 bit 4:2:2 1920x1080), this is an excellent low cost solution. Keep in mind, however, that this system will NOT play back dual stream HD as I understand it - so you'll have to render all your uncompressed transitions or PIP stuff.

They are supposedly working on faster cards, I'll report what I know when I know something.

It might, MAYBE be possible to install two of these (in slots 2 & 3) and NOT stripe them together, just run two separate RAIDs if you got clever and dremeled out the PCI slot cover on the card to dig an escape tunnel for the SATA ports. Did I mention all the ports are internal, like the Highpoint RocketRAID 1820A? I should have. That way, you could double the storage capacity of the system.

Random other thought: Even from a FireWire 800 based La Cie 1TB Bigger Disk, it'd take about 4 hours to read 1TB from that drive...and whether a SATA RAID could do an asynchronous write (write while reading from FireWire drive) I don't know - I'll have to generate a bunch of test data and test actual transfer times. So the RAID XL is a very valid option if you can live with it's limits.

In summary:

If you can afford it, get the fully populated X-Serve RAID. Fast, scalable, high availability, protected, warrantied.

If you are on a budget editing, consider a cheapie Seritek 1SE2 based array on another PCI-X G5 for a capture station then copy the footage over. Nah, that REALLY sucks, because then you're stuck with 50 MB/sec Ethernet transfers (fastest gigabit Ethernet Mac to Mac transfer I've done got 50 MB/sec of real world performance, stopwatching it).

OK. So if you can't afford $14K for the X-RAID setup, what to do?

If Highpoint's RocketRAID can be shown to work reliably and stably (it hasn't yet), RAID 0 with FireWire backup is atractive and scales up to 2960 GB of usable RAID 0. Backing up to three 1TB La Cie drives (plus one 250GB) would let you work with a total of 8 2/3 hours of 24p HDCAM footage, or 5 hours of 24p HDCAM SR footage. But if it had irrepairable directory damage when full, you'd be looking at about 16-25 hours of data copying time to get it all back up and running as it was since your last backup. Ick.

OK, what else? If you're working offline/online for feature work, or on shorter projects, I've been running a Seritek based array for months without a hitch (but not for much uncompressed work). Depending on the flavor of HD you're working with, you'd have 1 to 4 hours of uncompressed HD storage. Search (using search function at top of page) for "partitioning" to find articles on this site dealing with partition strategies for particular flavors of HD on smaller arrays. Back it up with FireWire and you might get taken down a day or two if catastrophe hits, but not taken out of the game IF you've stayed on top of your backups.

NOTE ON BACKUPS: And with footage, you only have to back up your captured footage once on your vulnerable array. I configure my FCP to save a backup every 10 to 30 minutes. I have Toast set up to back up that directory every night during projects. Photoshop docs, After Effects files, artwork, titles, etc. all do NOT get saved on disk arrays and get regular backups elsewhere. This keeps your backup regime light - once you've captured all your footage and backed it up and are just editing, you don't need to back it up again - it's not changing until you capture more footage. If I have to re-render my transitions, I don't sweat it. That's TIME, not WORK. Set it rendering and walk away. Time is valuable, to be sure, but my decisions would all be safely backed up, I could go do other things elsewhere while the machine regenerated rendered footage.

If you're doing 24p HDCAM or Varicam footage (DVCPRO HD tape), assuming it works as advertised (mine's on the way), the SyncRAID sounds great - low cost, protection in case of drive failure (immune to the kind of directory troubles RAID 0 has as I understand it at this point), and you MIGHT be able to put two of them in there for more storage if you get clever with a dremel tool on your cable routing (detach and drill out the PCI slot covers to allow cables to pass through). You can't effectively stripe them together for faster storage, but you MIGHT be able to set up two separate RAIDs to double your storage capacity.

If you can't afford backups, you can run without - I wouldn't, though. If the deck is inhouse (a rare luxury) you could batch recapture - but that takes work AND time. If you have to rent it, then it's time, work, AND money to recapture. How often do drives fail? In my 5 computer household, where each machine has 2-5 drives, a drive dies about once a year or two. With a bunch of drives (up to 8) in an array, my gut says be prepared for a failure about once a year or so. But that's a total guess. Run your own risk cost analysis - what does a backup system and plan cost me in terms of time, money, and complexity? What would it cost me in terms of time, work, and money if I didn't have backups and had to recapture all the footage? In a client environment, I can't imagine that every being acceptable. In an inhouse environment, it would really, really suck and be survivable the way an appendectomy is survivable - painful, time consuming, expensive, and leaves a bad taste and an ugly scar.

Hmm, on that cheery note, I'm off for a run.

-mike

So I've recently noted the existence of some stuff, and as my brain tends to do, it mashes them all together. Consider the following evidence:

SATA drives of up to 400GB capacity are available for under $1/GB
the new single processor 1.8 GHz G5 is $1500
the SyncRAID XL 5 port SATA PCI card is $300
5 bay Burly Boxes (SATA enclosures) are $270 standard, $420 with drive coolers for each drive
Apple's PCI-X (but PCI compatible) 2GB fiber channel card is $500
Apple's 10 client OS X Server software is $500

There is a need to keep data backed up securely and safely for editing and/or for workgroup access.

So consider this:

Scenario 1: You either need to backup a ton of data and keep it online and accessible over gigabit ethernet (about 50 MB/sec top transfer speed Mac to Mac)

OR

Scenario 2: You need a machine to be accessible via X-SAN on a fiber channel network, and push file around at over 100 MB/sec. This would be nearline for high data rate uncompressed HD, but online for some (1080p24 4:2:2 8 bit, all 8 bit 720p work) HD and all SD work.

In either situation, you are, of course, budget constrained, so fast/cheap/good is the order of the day.

I don't know if what I'm thinking of will all work together as planned for some of the following reasons:

-I don't know if multiple SyncRAID XL cards will "play nice" in the same computer
-I don't know how slow Apple's 2GBit fiber would run if forced to operate in 33Mhz PCI mode
-I don't know if X-SAN can access drives that are NOT fiber channel based on a machine, but that does have a fiber channel card installed in the machine

Assuming everybody plays nice (and that's always a stretch), here's some interesting possibilities:

For scenario 1 (Ethernet server):

Buy the following items:

single processor 1.8 GHz G5, with 10 user OS X Server software (unlimited users is $500 more), drop the modem, downgrade to Combo drive:

$1869

Buy 2GB of third party RAM (from crucial.com or somesuch): $200

Buy 3 SyncRAID XL 5 port SATA cards ($300/ea from macgurus.com), remove the PCI slot covers, drill them out to allow 5 SATA cables to pass through. Tape over any sharp edges to avoid cable chafing.

Buy 3 5 port Burly Boxes from MacGurus for $270 apiece. Add drive coolers for $450 more if you wish (will be safer, cooler, live longer, but run LOUDER). $810 without coolers, $1260 without.

Take a deep breath and buy 15 400 GB Hitachi 7K400 drives, presently $390/ea at zipzoomfly.com. If you're smart, you'll buy an extra Just In Case for that rainy day.

Buy (or just borrow to set up) any monitor capable of 1024x768. Run it headless or not, it'll work either way, your choice.

Monitor Cost: $0 (headless)

TOTAL COST:$9329 if my math is right

If add drive coolers and upgrade to dual 2.0 GHz G5, is about $10,300 for a nice workgroup server with a LOT of storage on it.

Install your modified cards, mount all the drives in their respective Burly Box chassis, set up the Server, connect all 15 SATA cables to the drives, set up the 3 disk arrays. You now have an OS X server with three 1480 GB protected RAIDs, for a total of 4440 GB of quite fast storage. Each is capable of about 115 MB/sec throughput. It's RAID XL, which is something like a cross between RAIDs 3 & 5 - dedicated parity drive, tiny divided pieces of data. In any case, you can literally yank a drive out in the middle of an operation, and the Mac will pause briefly and resume...AT IT'S PRIOR READ SPEED. Nice. Oh, and of course you didn't lose any data.

If something isn't playing nice, perhaps bumping up to a dual 2.0 GHz Mac for $500 more is worth it.

Scenario Two: high speed, online, X-SAN node

BIG FAT DISCLAIMER: This may totally not work, as SAN is all about consolidated storage in one place. But if a client node can read/write, and client can write to client...it would be cool. I have a gut feeling it probably won't, but it's a fun idea.

Do the same thing, but upgrade do a DP 2.0 GHz G5, drop out 5 drives, one Burly Box, one SyncRAID XL card, and order a fiber channel card when you order that Mac. It'd be something like this:

Buy the following items:

Dual processor 2.0 GHz G5, with 10 user OS X Server software (unlimited users is $500 more), drop the modem, downgrade to Combo drive, add the fiber channel card:

$2369

Buy 2GB of third party RAM (from crucial.com or somesuch): $200

Buy 2 SyncRAID XL 5 port SATA cards ($300/ea from macgurus.com), remove the PCI slot covers, drill them out to allow 5 SATA cables to pass through. Tape over any sharp edges to avoid cable chafing.

Buy 2 5 port Burly Boxes from MacGurus for $270 apiece. Add drive coolers for $450 more if you wish (will be safer, cooler, live longer, but run LOUDER). $810 without coolers, $1260 without.

Take a deep breath and buy 10 400 GB Hitachi 7K400 drives, presently $390/ea at zipzoomfly.com. If you're smart, you'll buy an extra Just In Case for that rainy day.

Buy (or just borrow to set up) any monitor capable of 1024x768. Run it headless or not, it'll work either way, your choice.

Monitor Cost: $0 (headless)

X-SAN software: $1000/seat (so then is server software necessary? Probably, but dunno, need to read up on X-SAN...)

TOTAL COST: $9109

Hook it all up, and IF it all plays nice together (large if), you have a nice fast node on your X-SAN, perfect for stashing assets you want available at high speed. I don't know how far you could push it, but if the SATA storage is accessible over the X-SAN, you should be able to get at least 1 if not 2 or maybe 3 uncompressed SD streams working over that fiber pipe. If for some reason the PCI factor screws up the deal, bump the machine up to a dual 2.0 GHz G5 (cost is now $9109) and put the PCI-X card in slot 4 where it can run full speed and attach to your pre-existing X-SAN setup. Assuming IDE drives can play on X-SAN (SyncRAID volumes appear as a single IDE drive to the Mac, the beauty of hardware RAID), you should be able to read/write at over 100 MB/sec over the X-SAN setup.

X-SAN requires a fair bit of infrastructure - a fiber switch (costly!), $1000/seat for anybody who wants access to the SAN, etc.

Maybe. Or maybe more likely not. I'll have to read, we'll have to see. But option one seems like a pretty sure bet, and bumping up to a dual 2.0 GHz box is not a big deal.

Formac has some new LCD monitors, including the Gallery Xtreme 19", that could be useful for monitoring HD via HDLink, one2one, or AJA's box (forget the name). (Use the search bar at top of this page if you don't know what those are.)

Here's the specs for the 19" model which could be good for 720p (Varicam footage) monitoring:

1280x1024 resolution
DVI input
280 nits brightness
700:1 contrast ratio
15 ms pixel response

The 19" model has 1280x1024 resolution, enough for 1280x720 resolution footage. It sells on their website for $599.

Another nice feature that they offer that I wish others would (Hello, Apple!) is a Zero Dead Pixels Guarantee for $99. Good idea! I just hope they aren't disingenuous to suggest that STUCK pixels (one or more of the red, green, or blue elements in a pixel is stuck off or on) isn't the same as a DEAD (always black) pixel, and therefore not covered. If I'm paying extra, I want it PERFECT.

-mike

Over on the Creative Cow's forums there is an excellent discussion on going out to film from Varicam originated material. This K. Han guy seems to have a good grasp on the nitty gritty involved in getting maximum quality out of Final Cut Pro, and what it's shortcomings are. Some of his points:

FCP render engine is YUV ONLY, not RGB...it has a great YUV environment, which is great for video, but converting YUV to RGB and back to YUV is BAD and introduces errors and quality loss. If you're destined for film (or digital projection that isn't video based), this is NOT GOOD.

Algolith CAS (Content Adaptive Scaling) can't run well in FCP, since FCP plugin environment is 8 bit only, and YUV in and out at that (Mike Note: Final Cut Pro's scaling algorithm SUCKS, I'd never use but for roughs, never final anything!)

Magic Bullet for editors - beware! It uses After Effects API (Application Programming Interface), so it's 8 bit only, not FXscript that would allow for greater color depth.

so there's no high quality scaling capability available native to FCP...a problem

biggest challenge in DIY DI is monitoring - large cost! RGB linear to RGB log previewing is complex and costly, can be done on desktop hardware, but costs $30-$40K

To get best results, do your editing but do NO effects inside of FCP, then carry that timeline to your RGB based post processing application, such as Shake, combustion, After Effects, Digital Fusion, etc. to do all of your color correction and transitions there (Mike note: Automatic Duck makes a line of products specifically to do this.)

If you want to read his specific two posts, here's links:

Post 1

Post 2

-mike

BlackMagic Design has released new drivers, version 4.6 (scroll down to find link on that page) that now supports downconversion of DVCPRO HD footage to standard def in realtime. From their site:

These new drivers add support for DVCPRO HD down conversion in all standard and high definition DeckLink cards. You can now playback DVCPRO HD material, with RT Extreme HD effects in Final Cut Pro HD, and then output down converted standard definition video in real time.

HDTV Down Conversion is now enabled via the Mac OS X™ Blackmagic DeckLink System Preferences, allowing this feature to be enabled in video applications such as Blackmagic Deck Control.

HD video outputs are now enabled via down converter on all standard definition DeckLink cards. When a standard definition DeckLink card is used, all playback of HD files will automatically be down converted and outputted via standard definition video outputs.
This new Mac OS X™ software update is for all shipping Blackmagic SD and HD DeckLink models, including Kona SD.


Mike's Commentary: This offers some new workflow possibilities, regardless of whether you're working in compressed DVCPRO HD or uncompressed HD:

-if you're on a tight budget (and who isn't), you can use a standard definition monitor with component inputs to monitor your HD footage if you have the DeckLink HD Pro card (single or dual link model) since it has analog monitoring capabilities.

-since they added realtime capture to DVCPRO HD last week in version 4.5, this means you can capture your HD footage to DVCPRO HD and use that as an offline codec if the frame size and frame rate matches something DVCPRO HD can do. (DVCPRO HD cannot do 1920x1080 progressive frame rates, so 1080p24 is out unfortunately).

-if you have HDCAM 1080i60 footage, you can use DVCPRO HD as either an offline or online codec - the ability to work with HDCAM footage and NOT have to have a disk array is a very nice option. 1080i60 DVCPRO HD footage is about 50 GB/hr, so a $400 400GB second SATA drive in your G5 could hold nearly 7 1/2 hours of footage.

-mike

Updated Wednesday early morning with new analysis and pricing

I've finally gotten back to messing around with the Highpoint RocketRAID 1820A cards I've got after a busy weekend at SXSW.

A few conclusions:

-the fastest RAID 0 configuration I could come up with used 6 drives on an 1820A card in slot 2, 6 drives on an 1820A card in slot 4, 2 internal drives, and on an empty array managed 592 MB/sec reads, 526 MB/sec writes (average of 5 runs). That setup yields 3892 GB of usable, formatted RAID 0 space.
-this 6+6+2 config could be built for under $4000 - 3 4 Bay Burly Boxes (3x$230 each from MacGurus), 2 Highpoint RocketRAID 1820A cards and 14 Maxtor DiamondMax 10 300 GB SATA hard drives ($204 for card, $209 per drive, both from ZipZoomFly, SoftRAID formatting software from SoftRAID, and some time with a screwdriver and dremel tool (pliers and a blowtorch optional).
-6 drives on each of 2 1820A cards are faster than 8 drives on each of 2 1820A cards
-adding a couple of internal drives (attached to "normal" connectors in G5) helps, but not as much as I'd hope
-slot 4 definitely yields faster results than slot 2 for RAID applications....but both are very fast (412 vs 470 MB/sec reads...writes are card limited to about 345-350 MB/sec)
-I, as well as others, have had drives not show up upon reboot (happened to me when I set up 16 drives in one array and rebooted). Shutting down, spinning down all drives, turning on drives a minute or two before computer helps sometimes not always
-so I still have hesitancy to suggest anyone use this as a primary working machine
-if you're just doing 8 bit 24p HDCAM, the SyncRAID XL from MacGurus with a 5 bay Burly Box is worth considering. If you're doing 10 bit or 1080i60 it definitively isn't fast enough. I have one on order, report coming soon.
-for optimal speed, put 1820A in slot 4
-for optimal expansion options, put 1820A is slot 2 or 3 and HD card in slot 4 of G5, which allows other RAID cards to be insalled (if you have a place to run the cabling)
-since the 1820A is having drive dropout problems, I'd recommend NOT buying the SATAport from MacGurus for use with the 1820A at this time, until it can be determined whether it's a hardware, software, or whatware problem with drives not mounting.
-almost broke 600 MB/sec on one test run (597.3MB/sec read). Mac SATA throughput scales way, WAY up there, FAR beyond SCSI capabilities with a single card. This is also more than the theoretical maximum of 2 gigabit fiber channel connections.
-while I haven't specifically tested this yet, I see no reason why you couldn't have 2 separate 8 drive arrays running in a Mac G5, each running either RAID 0 or RAID 10 (RAID 10 is limited to no more than 2TB per card right now, is a driver problem that should be surmountable). If that driver issue is resolved, that would allow on EACH card up to 2960 GB of usable RAID 0 space using 400 GB drives, or 1480 usable TB of RAID 10 (mirrored disk array, fully redundant set of disks). This would allow the Mac to have up to 5920 GB of usable space for uncompressed HD editing, or 2960 GB of of RAID 10 for uncompressed HD editing. Have I posted RAID 10 results? Somebody bug me if I haven't yet.

Testing setup: all drives tested in array were Maxtor 300GB, either DiamondMax 10 or Maxline III, either way they generate virtually identical performance statistics. The dual 2.5 GHz G5 was running OS X 10.3.4 with 512MB of RAM (until replacement RAM arrives, I got a bad stick or two), formatted using SoftRAID 3.0.3, performance was tested using BlackMagic Design's Disk Speed Test application. My reasoning for using that as a benchmarking tool is because it supposedly simulates capturing digital video to disk, and as that is my primary interest that's a good metric for me. The system was booted from a La Cie Bigger Disk (1TB) on FireWire 800 since some of these tests used both interal SATA drive slots as part of the RAID testing.

-mike

Can I write a drier, dorkier, longer headline? I'm not sure, but clearly some part of my brain is working on it. I'm not sure who else is selling it, but I found this at Mac Gurus recently - the new SyncRAID XL card. I described it in another article I wrote and may not have published yet, so here's what I said there:

There's a new hardware based RAID card available for Macs, which is great. The bad news is that it is 33MHz PCI, so it will slow down a G5's slot 2 (or 3) if placed in slot 3 (or 2) since those two slots share a bus and slow down to accomodate the slowest card in either slot. So that means a single SyncRAID XL is all that is viable at a time in a G5 with an HD card installed. HD card goes in slot 4, SyncRAID XL in slot 2 or 3. Or swap - SyncRAID in slot 4, HD card in slot 2 (cables need to run out an empty PCI slot cover, slot 3 in this example).

The good news is that it will use what they call RAID XL - it is similar to RAID 3 or 5, but just different enough to not be. It writes all it's parity data (redundant data that can rebuild a lost disk's info in event of failure) to a single drive like a RAID 3, but does tiny chunks of data more like RAID 5. Anyway, the card can do 114 MB/sec reads and writes, and the throughput is limited by the card itself, so faster drives are useless. This means you can go buy 5 drives and get 4 drives worth of storage capacity out of the system. So long as your drives can do at least 30 MB/sec even when full, you'll be fine to maintain that 114 MB/sec throughput the drive is capable of. Pretty much all of the recent 250 GB and larger drives can do this.

It might, MAYBE be possible to install two of these (in slots 2 & 3) and NOT stripe them together, just run two separate RAIDs if you got clever and dremeled out the PCI slot cover on the card to dig an escape tunnel for the SATA ports. Did I mention all the ports are internal, like the Highpoint RocketRAID 1820A? I should have. That way, you could double the storage capacity of the system.

SO:

For those doing 24p HDCAM projects, this card is fast enough. For doing compressed HD work, with DVCPRO HD or PhotoJPEG, it'd be completely fine. Also, for any framerate 720p 8 bit work it'll be fine, and for some 720p 10 bit work as well. To be specific: if you're doing 8 bit, 4:2:2, 1920x1080 at 23.976, 24, or 25 fps, this card should just squeak by as fast enough for capturing and single layer work. If you shot at a video friendly 29.97 interlaced frames per second, you're hosed - this is not the solution for you. That data rate is about 118 MB/sec, and needs some safety margin above that. This card only does 114 MB/sec in RAID XL configuration, and that's IT.

If you are doing a 24p HDCAM project, get one of these cards, a 5 bay Burly Box (or other compatible enclosure), 5 drives, and you are off to the races. Let's look at a couple of possible configs. The benefit of this system is that you DON'T need backup the way you did with the RAID 0 based systems described above. As it has been described to me, you can yank out a drive in the middle of a playback operation, and the system will choke for a few instants then keep playing back just fine. Way way cool. The downside to this card is that there is NO, ZERO, NONE, tech support available from the manufacturer. They have Mac drivers and that's all the effort they have or are going to put into it anytime soon. That's why I'd buy it from a knowledgeable vendor like MacGurus, Rick knows what he's doing over there.

here's a few of different ways you could set it up:

1480 GB with Hitachi 7K400 400 GB drives

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 400 GB Hitachi 7K400 drives: 5x390
(no SoftRAID needed)

Total usable RAID XL capacity: 1480 GB (same as the Seritek 1SE2 example above - one drive is dedicated to parity data)
Total cost: $2520 w/out coolers, $2665 with coolers
Total cost/GB: $1.70/GB without coolers, $1.80 with (might as well!)

=================================================

Using Maxtor DiamondMax10 300 GB drives (better cost/GB, smaller, faster)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Maxtor DiamondMax10 300 GB drives: 5x$209
(no SoftRAID needed)

Total usable RAID XL capacity: 1112 GB
Total cost: $1796 w/out coolers, $1940 with coolers
Total cost/GB: $1.61/GB without coolers, $1.75 with (might as well!)

=============================================

Using Hitachi 7K250 250 GB drives (better cost/GB, smaller size, though, same performance as 7K400 models)

SyncRAID XL card - 33MHz PCI, 5 port version - $300 from MacGurus.com
5 Bay Burly Box from MacGurus - $270 (add $29/ea drive for coolers, I'd recommend'em) - $420
5 Hitachi 7K250 250 GB drives: 5x$148
(no SoftRAID needed)

Total usable RAID XL capacity: 928 GB
Total cost: $1552 w/out coolers, $1702 with coolers
Total cost/GB: $1.67/GB without coolers, $1.83 with (might as well!)

==============================================

To get accountant geeky, you'll note that the increased cost/GB efficiency of the smaller capacity drives is largely offset by the fixed costs of the housings and SyncRAID card. So your cost/GB holds to a pretty narrow band - about $1.75/GB ($1.61 to $1.83 is the range).

In the limited context of 24 or 25p HDCAM (8 bit 4:2:2 1920x1080), this is an excellent low cost solution. Keep in mind, however, that this system will NOT play back dual stream HD as I understand it - so you'll have to render all your uncompressed transitions or PIP stuff.

They are supposedly working on faster cards, I'll report what I know when I know something.


-mike

DealMac has this link to Dell's site for the La Cie Bigger Disk Extreme 1TB Triple Interface hard drive for $705, about $300 less than I just paid...each....for two....dammit.

I don't know how long the deal is good for, so if you know you need it, snag it. At this price it is the most convenient best deal going for bulk backup of RAID 0 or single drives.

I've been in a hurry buying stuff lately, and I can see where it's cost me several hundred dollars already. On impulse I bought three "cute" uninterruptible power supply (UPS) devices, and I should have ordered an ugly but serviceable refurb from UPS Refurb. I could have gotten more capable units for about 25% less, and saved $120. Take your time, be sure of what you want, don't order anything after midnight - your judgement is shot!

Plus, some 400 GB hard drives that I'd ordered (and thought I'd cancelled) two months ago showed up on my doorstep. Not only did I not want or need them, I've paid $40/drive more than I can get them for now. Drat! I have to pay for shipping them back, so I've now paid overnight shipping AND return shipping for...nothing.

So take your time, shop carefully, and really consider if you Have To Have something as soon as it is available. I should heed my own advice and not buy anything until it's been readily available for a couple of months - what am, 12 years old at Christmas? Grumble grumble....ego grumble grumble....first on block grumble grumble....

-mike

Apple has re-introduce the single processor 1.8 GHz PowerMac G5 at a new, lower price point - $1500.

System specs are as follows:

1.8GHz PowerPC G5
600MHz frontside bus

512K L2 cache
256MB DDR400 SDRAM
Expandable to 4GB SDRAM
80GB Serial ATA
8x SuperDrive
Three PCI Slots
NVIDIA GeForce FX 5200 Ultra
64MB DDR video memory
56K internal modem

Mike's Commentary: So what does this mean for HD editors?

This model is clearly their business class desktop for those who don't want an iMac. But it is NOT a pro caliber workstation. The bus speed has been dropped to 600 MHz from 900 MHz, and even more critically, this machine has NO PCI-X slots. This means no high speed SCSI or SATA cards, and more importantly, no low cost HD capture cards. Yes, technically, you could go get yourself a Pinnacle Cinewave card and slap it in here, but they are so expensive as compared to the other more recent options you'd be shooting yourself in the foot about 4 different ways.

So I firmly do not recommend this machine for any serious stage of HD post production. It's too slow and lacks the critical slot for HD related peripherals and devices.

The only scenario I could see it being useful in is if you're working with Varicam footage via FireWire on a Panasonic AJ-1200A station, this could be a low cost capture/edit station...but I have a sneaking suspicion that if you have a dual processor 1.42 GHz G4 sitting around, it'll be faster than this G5. Since they don't sell G4s anymore, if you were working with the 1200A and had it available on an ongoing basis to monitor (FireWire DVCPRO HD to deck, HD-SDI or analog to monitor) it could work...but more slowly than it's more costly brethren. And lacking that deck you wouldn't have a way on either station to cost effectively monitor your HD footage "for real" (Digital Cinema Preview doesn't count as I've already written about, use the search bar at the top of this screen and enter "Digital Cinema Preview" for more articles covering this issue).

For an extra $500, you get 1000 vs 600 MHz bus and PCI-X slots and a faster processor and he brings his brother along to move things twice as fast...you get the idea. The dual 2.0 GHz G5 is the baseline machine for Final Cut Pro HD uncompressed HD work, period.

Don't buy the single 1.8 GHz model for HD work thinking you'll be saving money, either now or later.

But for other tasks in the studio, it might be viable. A Photoshop box, an Illustrator box, sure. A DV editing box for offline work...maybe, but I'd rather have the dual 2.0. An After Effects box, NO, get the dual 2.0. OK, enough said...

-mike

Apple today unveiled some new options on their X-Serve RAID, aka X-RAID (even though that's not it's real name, that's what folks call it). By using the latest 400 GB drives from IBM/Hitachi, maximum capacity has jumped from 3.5 to 5.6 terabytes - a whopping 58% increase - while pricing only increased a modest $2100 (only 15 or 20% increase), so 5.6TB is $13,000.

They've also obtained certification from Cisco and SUSE Linux and optimized the system to work with its Xsan Storage Area Network file system.

Apple is claiming a sustainable 380 MB/sec read speed and claims it is suitable for 10 bit dual stream editing. I'm guessing that's 10 bit 4:2:2, at least at 24 fps progressive if not 29.97 interlaced. 10 bit 4:4:4? Have to wait and see. It's not like they're sending one out to me for testing anytime soon.

They are also offering some other models, some of which weren't offered before: a 4x250 model (1TB capacity), a 7x400 model (2.8TB capacity), and of course the 14x400 model (5.6TB capacity).

Mike's Commentary: If you're serious about posting an uncompressed HD feature, this new system is THE low cost, high throughput, data protected (RAID 1+0, 3, 5, etc.) solution. 380 MB/sec read speed is very respectable - and that's at RAID 5 if I am reading the press material correctly.

Let's really break down the numbers though. At first blush, it seems like the 5.6TB unit it just over $2/GB. $13,000 divided by 5.6TB is $2.32/GB. Not quite the "just over $2/GB" figure Apple suggests. But how much usable space do you get out of it? All drives lose about 8% of their capacity when you format them, regardless of single drive vs. RAID. So your 14x400 is really 14x370, so usable space is really 5180 GB. And if you're using RAID 5 (because RAID 0 has no data protection), the X-RAID is really two 7 drive RAIDs paired together (each with their own controller), and since each RAID 3 (or 5) loses one disk worth of space for distributed parity, instead of 2*7 drives worth of space, it's more like 2*6 drives worth of space - 4440 GB. So that pushes the price up to $2.92/GB. And if you add the recommended battery powered cache backup ($350) and the required host fiber channel host card ($500), your price is $13,850, or $3.11/GB as a practical, realistic, installed cost.

I'm not dissing the X-RAID here, I think it's a great product in concept (see MacInTouch.com threads for some issues folks have had) and I'd heartily recommend it for all who can afford it. Just pointing out that the end quantity of storage and end costs are not quite as rosy as at first seems...but this is the case with all storage vendors.

When you're done running that math, what does that equate to for digital movie makers? If you're using the BlackMagic codec, you'll end up with about this much storage for each of these video types:

HDCAM 1080 res 24p - 8 bit 4:2:2: about 13 hours
HDCAM 1080 res 60i - 8 bit 4:2:2: about 10 1/2 hours
10 bit 4:2:2 1920x1080 24p: about 10 1/2 hours
10 bit 4:4:4 1920x1080 24p: about 7 1/2 hours

Now, 380 MB/sec read performance will play any of these back just fine, but I don't have confirmation on the exact guaranteed RAID 5 write performance, I'm ASSUMING it would capture any of these (that last requires about 200 MB/sec of throughput, including safety margin), but I haven't had it verified yet.

I'd feel very comfortable editing a feature with that, assuming I had an offline/online all HD workflow, or worked all online with a short feature with a low shooting (or at least captured shot) ratio.

The 7x400 model should be able to play anything short of 10 bit 4:4:4 1080i50 or 1080i60, but in terms of capturing, I suspect that it will fall short, and be capable of around 110 to 120 MB/sec write performance instead of the 190ish MB/sec I'm guessing it will do (no hard numbers to support that, pure conjecture). That would mean you could capture 24p HDCAM material, and any Varicam (DVCPRO HD) material via uncompressed HD-SDI, but not 8 bit 1080i60, not 10 bit 1080p24, not 4:4:4 1080 anything (there are no 4:4:4 1280x720 capable cameras out there that can't do 1920x1080 as well - and why shoot lower res if you can help it?).

-mike

David Santiago, CG supervisor for Sky Captain and the World of Tomorrow, and Gary Walker of TexFX

World of Tomorrow: Macs for 2D compositing (After Effects), Windows render composite farm (Maya), 240 processor 3D Linux render farm, dual 2.8 Xeons

Photoshop for paint

storyboard in advance for EXACTLY what you'll need to have to figure out most effective way of getting stuff done. Get a creative director or concept artist to help figure it out.

Use VFX when it's impossible to shoot with a camera, too expensive, too dangerous to shoot with real camera

most VFX shops went out of business because they do rob Peter to pay Paul, do A for cheap for hope of B and B never happens. Deferment is dangerous.

making squibs: "don't do it, it's illegal, in post 9/11 environment, anything to deal with explosive requires federal & state certification and local permits, etc." Lots of stuff the practical VFX guy does would be federal felony to do without permits etc. Even then, it's all about skill

practical vs. digital squibs: probably go practical unless if fails the 3 questions above test.

Coolest biggest thing you're proud of it:

Gary in What Lies Beneath - spent 7 weeks on it - filling in virtual chunks of set - he did a camera shot down the hall goes through the floor and you're looking underneath. Very challenging

David - hard to pick one, he liked the Black Hawk Down bullet hits and building damage that was done digitally since he gets so many comments about it.

he did a shot to go from live to CG to live in one shot

David works on a lot of proprietary stuff.

A lot of new studios are starting up, producers & investors will be dissapointed because they are going to blow it. Paint/canvas/brush doesn't make you an artist.

On Sky Cap, sub-contractors were hired to do stuff who weren't qualified. If they'd hired experienced folks up front, it would have cost half as much.

"Don't buy by price, buy by value." Same as with everything.

The practical effects guy then took everyone outside and did the exact same demostration/show he does for junior high students.

No, I'm not kidding.

-mike

Notes on editing from Sandra Adair (School of Rock, Before Sunset, Dazed & Confused, etc.)

she likes having a completely dissociated relationship to the material, so that she comes into it with no bias

editor's cut: finished a week after principal photography (her goal, not always), it's her chance to have her version/interpretation of the movie, to put her stamp on the film

HOW DO YOU FEEL ABOUT DIRECTOR/EDITORS IN GENERAL - TALK ABOUT THAT SEPARATION OF CHURCH/STATE, DIRECTOR/EDITOR

doesn't read the script too much, wants her fresh approach to what she's looking at

watch dailies twice every day, by self and with director (if can), then watch before cutting - has seen everything 2-3 times, have a feel and gist of what it's supposed to be and what they're after

Sandra as de facto writer, changing the story: she excised some shots that didn't add anything to the film, "completely useless", took out another completely redundant scenes.

Can't have same characters in location A & B, need something inbetween to add a sense of time.

She isn't a director and it's not her film - has to be in line with what director/studio want to make. Within that have tremendous freedom (with Rick, anyway). Everyone is willing to work with anyone.

On The Ringer, the assistant guy said an environment was created where everyone can contribute.

Scripts - script should be as tight as possible to save money. Shooting something you aren't sure is going to go in, never mind - get the script RIGHT, and ONLY shoot what you NEED to have.

Convenient extra shots - spend a 1/2 day or day as backup, B-roll, intro, transition shots, etc.

The economy of a well written script - allows characters, plot, etc. to come through once rendered as a film.

Have enough coverage to have choices

A minute a page (if formattted to industry standard) is pretty standard


get in and out of a scene with a moving camera, keeps it alive

it needs to BE cinematic, not something you'd just read as a page, render things in a visual way that uses the mood, the lighting, the framing, etc. that controls how the film is received.

watch out for not having reaction shots in scenes. People will be concerned with covering the words in a scene, and having reaction shots can make or break a scene (especially a comedy - the audience laughs at reaction, not the joke's punchline).

Set up a world where these characters can exist. Give context, establishing shots, etc. Establish their surroundings to inform about the people.

recommendations on editing

In The Blink of An Eye
Conversation

are two books on editing by Walter Murch (plus he uses Final Cut Pro, he cut Cold Mountain on FCP)

editing is an intuitive craft - be affected by what came before and what came after, but be intuitively aware of focus, performance, framing, etc.

find the most key heightened moment of a scene - a look, a reaction, a line delivery, and glom onto that. And lock that in, and build from there.

She does her primary cut as above, then sets it aside and starts again and re-does it from a different perspective (still using that initial piece).

She gets in the zone and doesn't allow any interruptions - no calls or visits, has headphones on.

when she's done, she feels she's explored everything worthwhile to explore. All the viable choices that were rational to try, she's tried'em. Then she puts it aside. The next day she looks at her 2, 3, 5 versions and take notes (like this one, this starts OK). She might make a combo cut gluing together the best of all pieces.

When she comes out of those sessions, she doesn't know the exact steps that she did, it just happened. Clearly, she gets into a fugue state and just rolls with it, not getting pulled out of that fugue state.

Take up dance or playing a musical instrument - there's something about the sense of timing from doing those activities that has to be learned, can't be taught, and has to get used to it.

Take up mediation or conflict resolution - quality of communication with director is key - rather than talk about logistics of solving a problem, back up and look at the core issue being addressed. As an editor, you're sitting between writers, directors, the studio, etc. - find a way to communicate with them all and make them feel validated.

Editing docs - be very, very organized, and be extremely clear about the kind of movie you want to make. What's plot/subplot, who are the chacters, how going to put together in a way that most supports your intent.

Sandra suggests making an outline.

Make a fast cut following that outline and see where that leads you.

Get interview transcribed if you can, and cut it on paper as a script as a way to begin.

Find the footage that's most exciting to you and build from that. If you know what really works, anchor that and build around it.

Think about your structure - what's beginning/middle/end? The structure informs everything about how you're going to cut.

If you change your structure after doing some cutting, you'll have to recut.

More on reaction shots - doesn't have to just the other person's physical reaction, can be other things going on in the scene.

When watching dailies, if there's a gap in the scene, calls script supervisor or assistant director to fill them in on "I need a so and so"

She'll keep a wish list of shots that she'll need, as she knows what they are she'll feed them to the production people, sometimes she gets it and sometimes she doesn't. If it's key, she'll jump on the phone and make sure she gets it.

Music: she doesn't use much music in first cut. The music is always changing and doesn't get locked down until very very end of process. Music is really really important.

important to sustain objectivity, especially working with writer/directors, tough for them to separate from what they wrote and what they thought they directed to what they actually got. Three films - written/shot/edited.

Can be a wrestling match with directors - if it didn't get shot the way they intended, or it didn't work, and that there's a better/different way to make it work. If you're tied to how it was on the set, it can be very difficult to have an open mind about it.

Screened A Scanner Darkly to crew - the crew was talking about how hot it was that day - they were attached to the experience they had shooting it - the crew didn't have useful editorial feedback

you have to keep in mind how hard people have worked on their project, and in the end it is their film, and if it sucks they get blasted and if it works they get adulation. They can have NOTHING to do with how it was edited, they're going to get all the feedback on it as "their" movie. Sandra wants to make the best film for that director - the loyalty is to the director. Producers and studios come and go, the most direct line to a quality film is to stay with the director.

It's a collaboration between her way and the other person's willingness to listen and compromise. Sometimes there's no compromising and it all works.

She'll tease a director (Sexless) who shot some scenes with so much footage (6 hours for a 2 minute scene) and it was excruciating to watch that much film for that short a scene. She'd tease the director about having him cut that 2 minute scene. She gives him the 6 hour monster, soon they are saying "cut it together as best you can, I have to go now"

see some software company with out-takes to have a bunch of parts to put together a scene

Sandra made some copies of dailies from a movie she cut, made VHS copies, and copied the logs, and Avid screens and stuff, taught an editing workshop for a week, used that scene. It was a simple scene with a lot of coverage.

one person in class (bank robbery scene) cut it from a comedic standpoint - and made it FUNNY! Surprised her - the power of editing is amazing

on a Scanner Darkly were some surveillance scenes (character spying on roommates) that were shot with up to 11 cameras. It was a TON to edit. One day had 23 hours of dailies!

What's the usual amount of footage? Shooting on film, 2 hours a day of footage. Sometimes as little as an hour. Two hours number is using two cameras.

On digital films, might be 4-5 hours a day

On a big feature get fewer hours. But 1-3 hours per day per camera is a ballpark.

Sandra charges a flat weekly rate, 5 days a week, 6th day is time and a half, double rate for 7th day.

Sandra puts in some sound effects and stuff herself. On bigger stuff have a music supervisor and music editor. She does a lot of music work herself cuz wants to. On big projects, picture editor does picture, music editor is cutting and finding music, sound FX guys doing their thing, color correction is way way way down the road at the end.

It's precarious to do favors for people, because producers will call and ask what you did the last 2-3 projects. Helps to have someone pulling for you to get your in...as always.

If you work your way up to cut Hollwood features, get an agent.

What was Sandra most proud of, and which was hardest - most proud of Dazed & Confused, because was her first big feature as a solo editor, first film with Linklater. It was fun and challenging. First cut was 3 hours long and very personal to Rick. threw herself into it and enjoyed it. Most difficult film was a doc that she cut that director was not clear about what he wanted, and was mostly in Spanish (about a Mexican culture situation). She felt hopeless not understanding language and culture, and the director wasn't around. Was in over her head. Very difficult and uncomfortable.

Won't work on Final Cut unless she has to. For features, having to preview conform and all that stuff previews Final Cut.

Things can do in the Avid in one keystroke take 2 or 3 in Final Cut.

(Mike thought: I need to think hard and carefully about where the "correct" budget point to aim for when contemplating a Final Cut Pro based solution. At what price point in budget do you turn over to Avid? Why not use Avid DV XPress for offline to hand up to bigger system? Total Aside: talking to a post guy, he didn't want the OS X "cruft" in their file system - their Linux isn't real and like other Linux, etc.)

I'm speaking in the morning at the Austin Film Festival (Driskill Crystal Room 1), and wanted to give a quick outlay of some baseline editing setups for indies. This represent the minimal, baseline, just-enough approach.

Any terms you find that are not defined or linked in this article, use the Blogger search tool at the top of this page to find more information about them in this blog. If this is your first time to HD For Indies, welcome. I've ben writing this since March of this year, and now have about 320 articles, and something like 150,000 words in here. There's a lot of stuff. At the top of this page are some handy tools: a search bar (for content within the blog), links to archives of articles from past months, and a list of useful links on HD related topics.

For a variety of reasons you can find described in other articles, I find Final Cut Pro HD to be a good editing solution, therefore these solutions are based on Macintosh based editing systems. Since AJA and Aurora are still not shipping their latest HD cards, and I've had success to date with BlackMagic's products, I'm recommending BlackMagic's HD stuff.

HDV edit station:

Something better than a 1 GHz G4. Preferably a dual G5 ($2500-$3000). At least 1GB of RAM ($120), preferably 2 ($200) or more. Final Cut Pro HD ($1000) or Production Suite ($1300), at least a 19 inch computer monitor ($350-$700), preferably a second monitor, at least 17" ($250-$400). LumiereHD ($200). Can use DVCPRO HD codec for offline and online codec. DVCPRO HD 1080i60 is about 50 GB/hour. Start by adding ATA (for G4) or SATA (for G5) internal drives for storage, if you need more, consider FireWire 800. If you want to do uncompressed color correction and effects, add a disk array as described below.

If you want to monitor your HD video, either a DeckLink HD ($600, no analog outputs) or DeckLink HD Pro 4:2:2 ($1500, has analog HD component hookups). Either rent an HD monitor during final color correction ($100-$500/day, depending on size & resolution), or buy an HDLink ($700) and a 23" LCD monitor (HP is $1600, Apple is $2000). Otherwise you're going to be hopelessly lost trying to color correct your video on a computer screen.

Compressed HD with DVCPRO HD

Conveniently, the exact same thing as above, but without Lumiere HD. Beware trying to capture DVCPRO HD via FireWire with the deck and FireWire drives on the same bus. BAD. Keep the deck connected to a FireWire 400 port, your hard drives on FireWire 800. I don't recall reading that this setup works OK absolutley positively. SATA storage would definitely work, even external SATA storage. Seritek 1SE2 card with the MacGurus external SATA port card gives you external SATA ports, buy external SATA drives from Granite Digital, Cooldrive.com, or MacGurus.

Uncompressed HD for HDCAM

Start with what's above, add the new SyncRAID XL 5 port SATA card ($250). Add 5 250, 300, or 400 GB drives ($160, $230, or $400 each from zipzoomfly.com) and configure as RAID XL, for 4 drives worth of capacity, and will give you just enough throughput for 8 bit, 1920x1080, 24fps, 4:2:2 color - this is what you'd get off of a Sony F900 in 24p mode. The SyncRAID will not, not, NOT handle 1080i60. These will ONLY work for 24p projects in 8 bit. You'll need external cases, such as the 5 bay Burly Box from MacGurus. (somewhere around $380 with drive coolers, around $250 without)

If you want to do 4:4:4, or 10 bit, or 1080i60, you need something faster. For now, that's a RAID 10 or RAID 0 with a Highpoint RocketRAID 1820a 8 port SATA card ($200). This will change as new hardware becomes available between now and January.

For HD monitoring, again you're looking at the HDLink with LCD monitor. Until I can put the HDLink/Apple 23" LCD combo side by side with a nice Sony BVM-24 HD calibrated monitor, I'd suggest using the HDLink for "pretty close" calibration, but be ready to rent a good calibrated HD monitor for your critical color correction work.

These are just single user, single station solutions. If you'd like some assitance in setting up multi-station editing setups, contact me.

Some camera solutions for HD, moving up in price/quality:

starting at the bottom rung of what might be considered HD and working up in rough order:

JVC JY-HD10 camera, approx $4000 (may drop soon with Sony's entry into HDV): This is a 720p HDV camera. It shoots 1280x720, 30 progressive frames per second at 8 bit 4:2:0 HDV format, a highly compressed MPEG-2 transport stream based format recorded onto mini-DV cameras. I've written about the format extensively, just use the Blogger Search bar at the top of this screen to search for HDV. The data rate is about 2 1/2 MB/sec - even less data than DV. It's heavily compressed, which creates problems in post when trying to color correct or extract a color key. There is no 24p mode.

Sony HDR-FX1 ($3500), due in November, and the approximately $7000 version (with XLR connections and other goodies) due by January or so. It's a 1080i camera, recording 1440x1080 to tape at 3.125 MB/sec. It has a pseudo-24p mode that will have some timing artifacts most likely that will be less than ideal. This is the most interesting camera under $10,000 by far, perhaps the most interesting camera under $20,000. It is a three chip 960x1080 resolution camera. The green chips are offset halfway between the blue and red sensors, helping it get a higher effective resolution.

Both of the above cameras record to miniDV tapes, even though it isn't DV on the tapes. It's HDV, which Final Cut Pro HD doesn't support yet. Therefore you need something to help you work with this footage, I recommend LumiereHD.

Panasonic Varicam (approx $65,000) - 1280x720 progressive framerate camera that records standard modes of 23.976, 24, 25, 30, 59.94, and 60 fps. It will also record any framerate from 4 to 60 fps in one frame even increments. While at first glance one might think the HDR-FX1 would look better, this in not true once you see the quality of the footage through the better chips, the better lenses, etc. It records at a datarate of up to 14 MB/sec for 720p60. 720p24 is only 5.7 MB/sec. Final Cut Pro HD supports this format for realtime effects, transitions, and color corrections. This is a GREAT intermediate solution. Pretty good quality, extremely low system requirements (1 GHz G4 or better, 1GB RAM, 20-50 GB/hr storage, 6-14 MB/sec throughput required).

Sony F900 (uses HDCAM in it's camcorder). It's an 8 bit, 4:2:2, 1920x1080 output (even though the tape format stores it as 1440x1080, so you lose some resolution, and it's also compressed with JPEG like compression). The camera is capable of 10 bit 4:2:2 1920x1080 at 23.976, 24, 25, 29.97, and 30 progressive frames per second, or 50 or 59.94 fields per second. Very versatile.

(Sony HDCAM SR (recommended for Sony F900, F950, Thomson Viper, and similar high end cameras) tape format can capture all 1920x1080 pixels output by the Sony F900 and F950 cameras, as well as the Thomson Viper 1920x1080 10 bit 4:2:2 and 4:4:4 output.)

Sony F950 is an updated version of the F900, with no built in tape recorder, but it can do dual link HD-SDI, which allows for true RGB, 4:4:4 output.

Beyond this level, I'm not sure which camera is truly "better" in terms of image quality, so I'm leaving it open.

Thomson makes the Viper camera, which can do either 4:2:2 linear, 4:2:2 log, or 4:4:4 log (I think I've got that right, don't hold me to it on the 4:2:2 log). Shooting log allows for another stop or two of exposure if I recall my conversation with Geoff correctly.

Panavison has their new Genesis camera, rental only, it's built from ground up as a film shooter's camera that outputs HD. Read the blog entry (do a search) for more info. Nice, nice camera from what I've read so far.

The Dalsa Origin can shoot 4K resolution (4096x3072) at 10 bit (or better? gotta check notes, sorry, is late on Friday now). It can shoot 4K and downsamle to 2K (possibly HD 1920x1080 as well). But I here it's a pain to shoot with, not shooter friendly, has to capture to a big heavy hard drive array.

Kinetta is working on camera that should be very cool.

ARRI is working on their version of an HD (and higher) resolution digital camera.

So that's a quicky rundown. Hope to see you at the panel, or at least hope you were there and got something useful out of it.

-mike

I discovered last night that my RSS and Atom feeds haven't updated themselves since Tuesday. It's fixed now.

Sorry...

So I'm sitting in Dobie Mall waiting for the next movie to start, and they have free wireless, so there you go.

Cool things from the first day of Austin Film Festival:

Met and talked to David Santiago for 10-15 minutes who was CG supervisor on Sky Captain and the World of Tomorrow. He's busy doing consulting work designing production studio server and workflows for some large, large, large productions. Had a very intense and deeeeeep geek conversation with him involving octuple 2GB fiber channel connections, edge servers, metadata controllers, stuff like that. Definitely expanded my mind and made me realize how many levels of sophistication there are and how many levels there are between where I am and where he is.

Ran into Turk Pipkin, whom I know but doesn't really know me, and he introduced me to John Lee Hancock, director of The Alamo. I had a fun conversation with him about shooting digital and digital post production. He said he hadn't shot anything digital or HD but was open to it if it were the right project, and that he LOVED working DI (Digital Intermediate process) on The Alamo at eFilm, a high end bad ass facility that does that kind of stuff. I mentioned talking to David Santiago and he said he'd seen Jon Avnet (producer of Sky Captain) at eFilm as they bumped into each other in the halls and that he'd seen a few reels of it and been impressed, but hadn't had time to see the entire film yet. His wife saw it and liked it. Mentioned my short idea and he chuckled and seemed to like it, so that's always nice. Any directors out there want to work with me on an HD project this winter?

Ran into John Pierson of indie moviemaking fame, had a brief chat with him, was flattered he remembered me from meeting at Oslo. He's wrangling a film project at the moment and I think teaching at UT. Way, way cool to have him living/working in Austin, more support for the whole Austin As Indie Filmmaking Cool Place Brigade.

Sat in on a Burnt Orange Production panel, it's very cool. Here's a link to a pdf about what they're all about. Burnt Orange is affilliated with the University of Texas at Austin, and is their serious attempt to attract filmmaking to Austin and produce some films in Austin. A film called "Dot" I think is their first film (I walked in late) and was shot on HD. I talked with Tom Schatz who runs the Film Institute (which is affiliated w/UT & Burnt Orange) about HD workflows. Sandra Adair, who usually edits for Richard Linklater, was going to edit Dot, but she's going to be doing the remake of Bad News Bears instead. Good for her, too bad for Dot.

Other gossip: Miramax is officially about to be dead, if the dissolution papers aren't signed yet, they will be RSN (Real Soon Now).

Met Maya Perez who's running at least the panel side of Austin Film Festival as well as other stuff and talked to her about their summer film school stuff for kids, and the possibility of helping them shape their curriculum and possibly guest lecturing. It would be good for my soul to do some volunteer work, I think I need that (thanks Mary!).

Met Barbara Holden, who is trying to get a theater/gaming/entertainment/film conference going for next summer (is that right? Might have something wrong in there). I like the idea of confluence, I especially see a burgeoning relationship between the gaming folks and the movie folks. There used to be a clear schism between "game res" and "film res" animators and modellers, game industry folk were perceived as the "junior varisty" of animation by the film crowd (Hey film guys! Try making YOUR characters do all their stuff....in real time....on an X-Box! Not so easy....) but that gap is dwindling, especially as you look at the cinematic portions of high end games these days. Anyway, that thing next summer should be very interesting.

Digging through my swag bag, there's a DVD with some scenes from Saw, which shows at the Festival this weekend. Between the 30 seconds of footage I just watched and the trailers, it looks not scary, just unpleasant to watch. Hmmm. Not sure if I'll go see it. Had a conversation with Annika (actress) today about Cabin Fever, and how that wasn't scary, just unpleasant and suffery to watch. I think Saw might be the same thing.

I haven't had a chance to really go through the film lineup, but I'm exicted to see Steam Boy (latest anime from Otomo of Akira fame) on Sunday. Geek ahoy!



-mike

OK, real quick: two software updates of note that caught my eye.

First, BlackMagic has released version 4.5 of the drivers for the DeckLink HD product line. The biggest news is that it now supports DVCPRO HD, including (if I'm reading the release notes right, I haven't installed it yet) the abilit to capture directly from HD-SDI to DVCPRO HD. This is BIG, since that means you can use older, non-FireWire decks decks to capture not only Varicam footage, but also HDCAM, HDCAM SR, or whatever other HD-SDI source material directly to the efficient DVCPRO HD codec, which not only makes small files (5.7 to 14 MB/sec) but is supported for realtime effects in Final Cut Pro HD, so your transitions, color corrections, etc. are all accelerated in real time (up to a limit, of course). I haven't hooked it up to see if it will directly capture to 1280x720 (Varicam sized footage) at 24 frames per second, but if so that's COOL.

That would also allow use of DVCPRO HD codec for offline codec use with HDCAM, HDCAM SR, and other footage for later recapture at full uncompressed res. SWEET!

The workflow would be something like this: shoot your footage with camcorder (or camera & recorder), rent a deck to capture EVERYTHING to offline codec (log as you go or afterwards by defining subclips), do your edit and rough color correction, once you have a fairly tight edit, use Media Manager to recapture just the clips you need at full res. There's a trick with Media Manager to get subclips back to be just basic clips so entire master clip doesn't have to be grabbed. OK, off to late night Magnolia goodness with my buddy Patrick who just got married (he works at Wide Load Games, cool stuff, checkitout).

Oh, and StoryBoard Artist 4.0 was released. I haven't used this program in awhile, but it's a good quick means for non-drawing type folks (like me!) to be able to rough out a storyboard that the rest of the production team can understand. Comes with a big library of simply drawn people, places, and things. Pretty cool.

-mike

Just a reminder: I'll be speaking at the Austin Film Festival about Digital Filmmaking this Saturday, October 16th at 9am in the Citadel 1 Room in the Driskill Hotel.

Other stuff: Blog entries will abate to very little I'd imagine over the next week as I attend the film festival. Sorry.

I'm also in the middle of gutting and resetting up my studio, a task I began in August but was interrupted by the need to post and HD short film. Never finished the task, need to get back on it. Three rooms of my house are unusable because of all the stuff piled up, and the dogs are getting snooty about (they're big and can push stuff over by "accident").

Many articles in draft form that I need to finish up, the backpile is getting seriously tall. There's a bunch of interesting new SATA gear coming onto the market, including a hardware RAID card for Mac (finally!).

I've been doing a lot of testing on various hardware I have sitting around the house, and will be posting results once I get them. But that'll be in 10-14 days.

I do realize the blogging has slowed to a trickle of late, but fret not, I'll be back to my mad pace before too much longer. Lots of time sitting in studio waiting for test runs to complete, so there will be time to write.

Updated Tuesday 10/12/04 (Happy Birthday Kimbo!) with link to other 1820A troubleshooting article

I'm going to try to publicly respond to more reader mail when I think it's relevant. Here's a couple of letters and the responses I gave. I've edited down the sender's letter to the salient points, so I'm not quoting them.

Letter 1 (paraphrased): How to do low cost HD Post with a DP G4?

I have an old Russian 35mm film camera that I want to mess around with and shoot some stuff to telecine for eventual usage on whatever high def DVD format comes along in a year or two. bonolabs.com will telecine this material to DVCPRO HD on hard drive and mail the drive to me to transfer the data from and mail their drive back. This is an expensive hobby, not a big production. I have a dual 1GHz G4. What hard drives should I buy, and how should I go about this? Keep it cheap!

Mike Sez:

For starters, if you're working in DVCPRO HD and they're sending you that data, you can get by.

If you want to work uncompressed, you're screwed with G4, gotta get G5.

If you're wanting to work 24p, you're stuck at 1280x720 res with DVCPRO HD format, which is actually only 960x720 internally.

1.5GB RAM is a floor to work from - FCP HD requires 1GB RAM for HD work.

In my email I said this, but it's not the best way to go: "Buy a Seritek 1S2 or 1SE2 card, an external SATA case such as from Granite Digital or MacGurus, and a couple of 300 GB Maxtor Maxline III or DiamondMax 10 drives. Don't format as array, leave as 2 separate volumes."

Instead, do this:

If you don't have a second internal ATA drive, get the ATA version of the Maxtor DiamondMax 10 300GB drive, or the ATA version of the Hitachi 7K250 drive for internal storage FIRST and see if that meets your needs, before buying external storage.

DVCPRO HD 720p24 is only 5.7 MB/sec, so pretty much ANY drive will do, EXCEPT for a FireWire drive on the same bus that the AJ-HD1200A Panasonic DVCPRO HD deck will be plugged into. The 1200A DEFINITELY wants to be the only kid on the block (so to speak) when it comes time to lay back to tape. Also, last I heard, when FCP HD lays back to 1200A for a 24p project, the flags weren't being set right for the deck to understand that the source was 24p with a 2:3:2:3 cadence introduced. Instead the deck saw it as a 60p project. This is a problem for whomever wants to do stuff with the tape. There was an update that came out over the summer for working with FCP HD and DVCPRO HD, I can't recall if it fixed that specific problem...think so, not sure. Make sure you get that update before working with deck anyway.

In any case, DVCPRO HD 720p24 is about 20GB/hr, so a 300GB drive (which formats to about 280) will give you nearly 14 hours of storage.

If you're just plinking around with short material, external SATA may be utterly unnecessary. FireWire 800 would be fine for external storage as well if you have that available, and it's more portably usable than SATA (not everyone has external SATA plugs!). Granite Digital, FireWire Direct, La Cie, lots of sources for that stuff.

You have no viable way of monitoring HD with a G4 short of renting a Panasonic AJ-HD1200A deck, connecting via FireWire to an HD monitor. Daily rental cost for the pair is about $700-$1200 (and up), depending on HD monitor used. Be careful about 24p support on HD monitors. Only rent this at the end of your project when you're ready for color correction and layback to tape. Last I heard there were issues going back to DVCPRO HD with 720p.

If you shoot 30fps might be able to cheat and use the 1080i DVCPRO HD codec. Early tests looked weak in vertical resolution doing this due to progressive vs interlaced issues, but I haven't messed with it in months since I learned a lot more.

If had a G5, could go DeckLink HD ($600) to HDLink ($700) to an appropriately sized DVI-D input LCD monitor (for 720 or 1080 work - $500-$2000).

Seriously consider getting a G5 rather than upgrading your G4.

Run the costs of deck/monitor rental vs. upgrading and owning your own monitoring solution, let me know what you find out.

EVERYTHING happening in serious HD is on G5, G4 has lots of problems handling the efforts required for HD throughputs (in regards to HD-SDI and uncompressed throughputs, anyway)

-mike

Letter 2: "My 1820a card is sucking rocks. What gives?

A reader wrote in (I paraphrase here):

I'm having trouble with my Highpoint RocketRAID 1820A card that I set up using 4 drives and a MacGuru's Burly Box like you said. Drives keep not mounting, and thus the RAID can't mount. What gives!


Mike Sez

yeah, I've had similar problems and have heard of others having the same problems.

I don't have anything new to tell you, I've spent the last week writing and working on some other stuff.

I think it is one of four things:

1.) flaky 1.0 drivers - hopefully will be fixed over time
2.) loose connections/cheap cables - do you have skinny internal cables that came with 1820A, or thicker externals that came from MacGurus?
3.) You've got a bad card - I had one card that consistently sucked, got it replaced and is all better now.
4.) The card just suxors, and we're all screwed

I am thinking/hoping that it's #1.

This card has been on the PC side since June and I hear a lot of people recommending it. I have been lazy and haven't trolled around on any user boards for user experiences on the PC side with this thing, if anyone wants to spend some time and Google this one and tell me what they found, I'd love to hear it.

I also had trouble when I first tried to just "plug everything in" including a 4 drive coolers (one per drive) in the Burly Box. It was a big huge cabling mess. I pulled everything out, did NOT install the drive coolers, labelled everything and taped down extraneous cables so I could see exactly what was going on and be SURE nothing had pulled loose (yes I am anal).

Through this process I deduced that I had a bad card and got it replaced.

You can also try this suggestion in the article directly above this one (just scroll up) as a troubleshooting methodology.

Since then all has been OK. I have yet to install drive coolers, that's next on my list.

-mike

UPDATED 10/12/04 with eCinema EPD 121 pricing (it's $3300), and further analysis/commentary at end

With the rollout of the AJA HDP and eCinema Systems EPD121 (augmenting their first-to-market EPD100), there are now FOUR devices on the market that allow you to monitor an HD video signal on a 1920x1200 pixel LCD panel, such as the Apple Cinema Display 23, the Sony SDM-P232WB, or the HP f2304. Which one's best? That I can't answer. Which one's best for you? That I can take a stab at.

For starters, read this excellent article to understand the benefit ALL of these devices bring to HD monitoring. Sharper imagery, lower cost, lower weight, lower heat dissipation. I love it. The contrast thing? Well, LCDs will get better over time, the blacks are pretty elevated (aka bright) right now.

There are now three entrants in the HD-SDI to DVI-D (with RCA stereo pair) converter box arena. I'm tabling the $6300 EPD100, it's so expensive compared to the others I'm leaving it on the bench for now as non-viable for the budget constrained HD editor. But it's there, and it works. I'm learning more about it's viability and I'll have more to say on it later.

BlackMagic Design's now $700 HDLink - I have one and pretty much like it so far, with some caveats. The preview is beautiful to behold, as all these devices do it scales down 720p and NTSC footage to maintain a one to one pixel ratio between the signal and the display (except for the option to double the size of NTSC & PAL to better fill the screen). My only complaints so far are some occasional audio distortions and pops/glitches (pops on my first unit, occasional distortions on my second unit). The unit also runs quite warm, as it uses it's metal case as a heat dissipator. Apparently it has all kinds of untapped capabilities under the hood, so I'm waiting to see what they can make it do. The calibration software, however, is awkward in the extreme, and while technically it will let you calibrate to a target display, it needs a lot of work for a production environment. See my recent FAQ answer and my earlier report for more details. In short, however, this device will take an SDI signal, an HD-SDI signal, or a dual link 4:4:4 HD-SDI (RGB or YUV) and convert it to DVI-D and analog RCA stereo pair. It does introduce a delay, however, it was very disconcerting to watch video via HDLink and listen to audio through the digital coax coming out of the DeckLink HD Pro card (instead of the analog jacks on the HDLink as I should have to maintain time synchronicity).

eCinema Systems has the $6300 EPD100 and the $3300 EPD121 (pdf link). eCinema Systems was first to market with a product of this type, but a large $6300 device is tough to market against a small $700 device...until you look closer. The eCinema box appears to have better controls and more features, but at nearly 10 times the price it's tough to justify. The EPD121 looks to be their lower end entry, it's basically the same thing but lacks front panel controls (all modifications have to be done via computer connected via USB interface. It's USB 2).

AJA at IBC announced their HDP product, which at first glance seems to be virtually identical to the HDLink but for the color of the case. This is not the case. AJA has a long track record of making broadcast quality converters, so when they say it's good, it's good. The main difference between this unit and the HDLink is that instead of one (or two for dual link) HD-SDI inputs and single or dual link output, the HDP has one input and TWO HD-SDI outputs besides the DVI-D connector on the opposite side of the unit. It also has a sync in port, useful in a facility type installation, especially since this thing not only has a passthrough for HD-SDI it also serves as a duplication amplifier. So you get three outputs from this device - two HD-SDI and one DVI-D (plus audio). Unlike the one2one or HDLink products, however, this unit does NOT have a USB input for

So which one is right for me?

If you need graticules, and need them today, the one2one box with Option 1 (graticules) is the way to go. Pricey but efffective.

If you need 4:4:4 capabilities, the HDLink is the only way to go as far as I can tell. Working in 4:4:4 Viper Filmstream Log? You can load a CLUT (color look up table) to make your footage look "normal" with HDLink. HDLink, like the eCinema products, also allows for calibration via USB 2 connection.

If you need HD-SDI passthrough, or two downstream HD-SDI taps, or house sync, AJA's HDP is the way to go.

If you need basic 4:2:2 HD-SDI conversion to DVI-D and RCA stereo pair, assuming my $700 price point is accurate, the factors to weigh are these: do you need the house sync and downstream HD-SDI outputs (HDP), or do you need calibration and possible future features via USB update (HDLink)?

Updated Commentary 10/12/04: But the other factor is that BlackMagic is a small company trying to do a lot. They've done a lot of impressive work, but I'm slightly concerned over their ability to keep on top of the tech support and software end of things. The HDLink apparently has added capabilities buried in it that haven't been actualized yet, and it's calibration software I find currently woefully inadequte.

eCinema, whilst costing much more, is the product I'm hearing recommended from Bay Tech and Band Pro. Whether that is strictly based on their more dedicated-pro-level feature set, or is also influencing their recommendation because it costs more and offers them better margin, I can't say definitively since I haven't played with a 121 yet.

-mike

MacWorld is reporting that JVC is joining the BDA (Blu-Ray Disk Association), a week after Twentieth Century Fox did the same.

Mike's Comments: In the ongoing struggle between HD-DVD and Blu-Ray to be The Chosen One for the next generation of DVDs that will play back HDTV resolution movies and entertainment, Blu-Ray seems to be picking up some steam lately.

A schism, a la the Betamax vs. VHS format wars of the 80s, is still entirely possible - last I heard, rumor was that Sony was going with Blu-Ray Disc (a natural since they're helping develop the format) in their Playstation 3, whilst Microsoft was being courted to house an HD-DVD in the upcoming X-Box 2. In this metaphor, I'd say that Blu-Ray is the Betamax (technically superior but more expensive) and the HD-DVD is VHS (technically inferior (15 vs. 23GB capacity) but much less costly. The same production lines that can make HD-DVDs can retool in short order, say lunchtime or overnight, to make regular DVDs.

-mike

The Digital Cinema Initiative has been working on system requirements and specifications for digital cinema formats and distribution...and are still working on it. The DCI is made up of seven major studios (Disney, Fox, MGM, Paramount, Sony Pictures Entertainment, Universal and Warner Bros. Studios).

After not having a press release since last November, on September 8th they announced they'd release the finalized spec on September 30th.

I inquired last week after the deadline had passed, and was told something to the effect of "they're all in an all day meeting."

So clearly they are still working on it.

During the summer they announced that the format would be based on JPEG 2000.

-mike

If you're using an Highpoint RocketRAID 1820A card in your Mac, here's one more thing to try if you're either:

1.) Not having volumes mount at startup, or

2.) having drives suddenly dismount unexpectedly in the middle of working with it.

If you're having problem 1, try turning on your drives about a minute or so before the Mac, so all drives have a chance to get fully spun up and ready before powering up the Mac.

If you're problem 2, or the above fix isn't helping problem 1, consider this:

Totally circumvent the "middleware" plugs in the cabling arrangement between your 1820A card an the drives.

Background: It's quite possible to have a total of 3 SATA data cables involved per drive: 1 from the 1820A (or Seritek, or whatever) card to a header that presents a female port to the outside of the computer, a second straightforward SATA cable (should be thicker external grade) that goes to the external hard drive enclosure, and a third that goes from the female connector that faces the outside world to the drive itself.

If you're having dismounts in the middle of operation, maybe, perhaps, it's due to signal loss. Try this:

Have a single cable go DIRECTLY from the port on the card to the drive, circumventing all headers. Yes, this is cumbersome and means your external drives are permanently mated to the G5 (or whatever computer), but it might be a valid way to test if signal degradation is the source of your problems. I wouldn't want to run this way permanently, not even if I had the external cases and Mac on a rolling platform together, but it might be a good troubleshooting methodology.

Then again it might not make any difference, either because your cables are bad or that isn't the source of the problem in the first place.

-mike

So I now have several readers writing in with their own RocketRAID experiences, and some folks are having persistent trouble, and some folks aren't.

Some things that MIGHT make a difference in reliability of RocketRAID (just from a "does it mount on desktop?" level of functionality):

-a small sampling seems to indicate that formatting with SoftRAID 3.0.3 (haven't tried 3.1 yet) works better than Apple Disk Utility works better than the included RAIDman. I can't swear by that, would need more data, but this seems to be the case so far.

-It DEFINITELY seems to help to start up the Burly Box (or whatever enclosure you're using) a minute or more before the Mac, and let all the drives get up, rolling, and comfy before starting the Mac. Since the 1820A can't hotswap (or hot mount, perhaps), this is going to be a good habit going forward.

-some folks are having persistent troubles, some folks are having none. Bad cards can be out there (I've had one), so don't rule out the possibility of needing to replace the card.

-mike

OK, I knew I saw this thing at NAB in the Apple booth and then never heard of it again. This is their new, lower cost follow up to the $8000 EPD100, the first device that could convert an HD-SDI signal into a DVI-D signal for monitoring on an Apple 23" Cinema Display or similar device.

It will do graticules of varying sizes, and has lots of control over them. It'll do 16:9, 2:35:1, 4:3 within 16:9 or 2.35:1, crosshairs of varying thickness and color, and graticule lines can be different colors.

I don't know price, at NAB they mentioned $3000 and up depending on options (graticules was one option). But that was months ago before HDLink cut it's price from $1300 to $700 and AJA came out with their HDP.

See the next article comparing it to the other HD-SDI to DVI-D boxes.

-mike

If you are already editing HD as I've recommended with a BlackMagic DeckLink HD family card and formatted your SATA RAID with SoftRAID, here's good news: new drivers with new features for both of these products:

BlackMagic Design has released a new master driver for all of their DeckLink HD products. This new version, 4.4.3, adds support for the new models (the HD Plus, the Pro 4:2:2, etc.) but also adds a new broadcast quality HD=>SD downconversion capability to all DeckLink HD cards. Highly recommended for all users

SoftRAID has released version 3.1, which is now bootable. Huh? In the past, if you formatted a RAID (aka disk array) with SoftRAID, you couldn't start up your Mac from that volume. Now you can.

-mike

Engadget has this article talking about how 20th Century Fox has expressed an interest in the Blu-Ray format and joined the Blu-Ray Disk Association (BDA). This doesn't mean they've committed to putting their movies out on Blu-Ray, just that they are joining the forum to have access and influence on the spec.

Mike's Commentary: DVDs are great, but but they can't do high definition TV. There are two competing formats for the next generation of DVDs that will be able to handle HDTV.

In one corner is HD-DVD, which is based on a not-too-distant technology from current DVDs. The same production line can turn out standard DVDs, swap out the master mould, and then start stamping out HD-DVDs. It's cheaper than Blu-Ray, but most of the backers are technology companies (as I understand it, correct me if I'm wrong). It also holds less - around 15 GB per disc. The format allows for Windows Media 9 based encoding, MPEG-2, and MPEG-4 AVC (Advanced Video Codec) known as H.264 AVC.

Their competitor is the Blu-Ray Disc. As the name implies, it uses a blue laser (shorter wavelength for a more concentrated, smaller beam). This allows the Blu-Ray disc to hold about 23GB per disc. At one point MPEG-2 was going to be the only compression choice, but they wised up and added the other two formats as well. The major impediment I see for this format is the cost - the discs are expected to be significantly more expensive to produce than HD-DVDs, and studios are definitely going to care about that. But Sony is one of the major backers of the format. And they happen to own a movie studio or two, so with the addition of 20th Century Fox, there are now multiple studios expressing interest in the Blu-Ray standard. Whether they can convince other studios to back Blu-Ray over HD-DVD remains to be seen.

The last thing anyone wants is for there to be two competing formats for hi def DVDs. Consumers would almost certainly balk, and the adoption rate of hi def DVDs would certain suffer substantially.

-mike

Toshiba in a couple of years will be shipping TVs based on SED (surface-conduction electron-emitter display...yeah that). Brighter, better contrast, less smearing than LCD, but as bright as CRTs. Sounds very promising, but as with all tech, "What's it cost, when's it shipping?"

Read the article here.

If you find a good article relating to the kinds of HD stuff I'm reporting on here, please send it in! If it's been a day or two and I haven't posted about it, I've probably missed it, so please fill me in. It's easy! Just email me at

mike at hdforindies.com

If you want, I'll be happy to use your name as submitter.

The same thing goes for good websites, too if I haven't mentioned them before. I have a Google bar at the top of the blog that will let you search all the contents of my blog, not just the (ridiculously long) front page.
-mike

Creative Cow has this article about how the Star Trek show has moved to an all HD acquisition and editing workflow for their fourth season that starts shortly.

Insert ubernerd Star Trek/HD hybrid humor here, such as

"We've beamed him aboard, but he's been, uhhh....

a.) "...compressed"
b.) "...JPEG'd...it looks painful"
c.) "...color space reduced"
d.) "...downrezzed" (or should that be downsampled?)
e.) "...dithered to dog food." (Hey, this is the early days of beaming, maybe they could only do 8 bit dither rather than 24 bit matter...)
f.) "....ummm, I'm not sure what happened, but he's definitely missing some bits"
or
g.) "....interlaced...and damn, it ain't pretty." (He was a pretty progressive guy before...)

:D

And yes, I pretty much posted this just so I could geek out on my nurd (that's one notch beyond nerd) humor.

Have a better one? SEND IT IN, HOSS!

Send to:

mike at hdforindies.com

And let your geek flag fly high.

In our case, that might as well be a propeller beanie.

-mike

The Hollywood Reporter has this article talking about Walter Murch (who edited Cold Mountain with Final Cut Pro) coming back to Final Cut Pro again, this time to edit in a workgroup environment in HD. From the sound of the article, it looks like they'll be offlining in DVCPRO HD 720p24 and using that for smaller screenings. They'll be using the X-SAN beta software running on X-Serves with X-RAIDs attached. (Maybe X-Men should be cut on OS X with X-RAID, X-Serve, X-SAN?). Like the Love Boat theme says, "Exciting and new."

(Come aboard, we're expecting you....")

I think this is sooooooooo the way things are going to be in the future.

The article also mentions Murch's upcoming book on editing with FCP, Behind the Seen. Pre-order yours today.

Which reminds me, I need to get my Amazon Associates stuff set up...I soon plan to have a big honkin' list of purchasable stuff from all my research. Prices will be same as usual, but let's just say you'd be doing your part to keep HD For Indies moving along.

-mike

DTS announced a new software based DTS audio software encoding package that runs on Mac or PC. Pricing hasn't been set yet, but for those serious about producing high quality multi-channel audio from a desktop based solution, this is great news to have a DTS option.

It runs on fairly modest hardware (G4 based Macs) and has 3 different settings for audio bandwidth.

There is also a network version that can use up to 99 Macs or Windows boxes to distribute encoding jobs.

Mike's Comments: Why should you care? This is yet another piece of the equation to produce high quality content from a desktop based system without having to turn to high end, high cost proprietary solutions.

Don't know how/why I missed this, but AJA at IBC introduced their own HD-SDI to DVI-D and RCA stereo pair adaptor. This is their version of the eCinema box or HDLink box. It looks to be the same size as the HDLink, but with 2 HD-SDI outputs and genlock input. Nice.

At first glance, the difference seems to be that the HDLink can handle 4:4:4 (which it appears that the HDP doesn't), and that the HDP offers not one but TWO HD-SDI passthrough outputs, whereas the HDLink offers none. It also appears that AJA HDP unit does NOT have USB input, so there is no means of loading or adjusting the color lookup tables involved.

So if you're doing straightforward HD production and want HD-SDI passthrough, the AJA HDP might be your unit.

If you're doing digital film work, involving either 4:4:4 or log data (or both), the HDLink might be a better pick.

Same price for either (supposedly), just pick your need.

A friend told me that the price was the same as the new HDLink pricing ($700), and this product in fact is what prompted Blackmagic Design to drop their pricing on the HDLink.

If anybody has better info than this, or commentary, please drop me a line. There's a thread discussing it on Creative Cow.

I'm been too busy to read it all...

-mike

I'll be speaking on the Digital Filmmaking Panel at the Austin Film Festival two weekends from now, on Saturday, October 16 from 9am to 10:15am.

I don't know what the other panelists' focus will be, but I'll be talking about the tools and techniques to obtain maximum quality results at minimal expense in a minimal amount of time. Stuff you want to hear about, right?

I'll also have a significant announcement to make relevant to anyone looking to shoot a movie on an HD camera.

I'll be speaking in the Driskill Hotel in the Citadel 1 room at 9am as part of the Austin Film Festival Conference.

Hey, if you can't toot your own horn on your own blog, where can you?

-mike

Posts may get a little thin this week, very busy prepping for some upcoming projects.

Things I'm working on for the site:

-the long delayed Part II of the "Videocamera vs Videotape Quality: What's Missing?" series. I'll cover the DVCPRO HD, HDCAM, HDCAM SR, and D-5 formats.

-an HDLabs Report on the new La Cie Bigger Disk Triple Interface. This is a 4 drive, 1 TB FireWire800 device from LaCie. I'm not sure whether it's JBOD or RAID 0. JBOD would be slower but more recoverable in the event of a drive failure, RAID 0 would be faster. I've been having some difficulty with the unit so far, tried to Zero Write it (an Apple Disk Utility option), and it failed twice at about the 40% mark, some software thing having lost communication with another software thing. That was on a G5 on FireWire 800. Now I'm trying it again on my old G4/867 (single processor) using old fashioned FireWire 400. It will take approximately forever. I'll finish the report once I have it up and running.

-more Very Cool Ideas for post production workflow stuff that I've been working on for about 3 months, that I haven't discussed on the site before (yes, I have goodies I haven't shared yet).

-mike

Panasonic is giving away a sampler DVD that includes DVD res MPEG files, and of more interest to Mac folks with Final Cut Pro 4.x installed, a sample DVCPRO HD based Final Cut Pro HD file with the media included.

The footage is all Varicam shot 1280x720 footage (which means it was downsampled to 960x720 before being written to tape).

So fill out the web form and get it in the mail, and in a week or two you'll have your own footage to play with.

Apple was handing out their own sample footage disk at NAB, I don't know if they are available anymore.

If you're on a PC, much of this footage will not be accessible - most of the footage is in a QuickTime DVCPRO HD codec, which exists only on Macs only with Final Cut Pro HD installed.

But there's still other stuff of interest, including sample stills, etc.

Regardless of your platform, "go git it."

-mike

UPDATED MONDAY (see bottom)

Christopher Barry, an early adopter of HD For Indies wrote in from Australia with this interesting bit of kit - the same HD viewfinder that is used on the Panavision Genesis camera. It has 1280x720 pixel resolution, an excellent overexposure detection tool, and lots of other nifty features.

These are links to an Australian vendor that is making them available, I'm sure they're findable in the US, I'm just too lazy to go seeking out links for them.

-mike

> Update Monday 10/3/04: Here's a direct link to the Accuscene site, and here's an interview with French DP who used it.

Got this from Russel Branch, Sales Manager of Accuscene:

this Viewfinder lists at £11,100, equivalent to just under $20,000 [US].

Whilst this may seem expensive, the Exposure Mode does mean that the need for a waveform monitor is removed, whilst the clarity of the unit is roughly equivalent to a 20 inch broadcast monitor, meaning that for owner/operator type shoots you also don't need a reference monitor. Since it runs from the camera batteries you also don't need a generator, allowing you to travel light and set up/tear down in a fraction of the normal time.

End Update

Excellent hands-on, detailed article talking to Dion Beebe, director of photography on Collateral.

He talks about the drastically reduced lighting needs, the need to knock highlights back (even including set dressing & paint choices), and how the F900 did a better job NOT blowing out highlights as compared to the more expensive Thomson Viper Filmstream.

If you're a shooter, or planning an HD shoot, this is a great article on Things To Know.

-mike

OK, I'm cheating on quality a bit here from my usual pristine standards, but it's a fun idea:

The new Sony HDR-FX1 camera (I've written about it here, here, here, here, here, here, and here) kicks out a 1920x1080 interlaced signal at 59.94 fields per second. Sony has announced they'll have a PAL version around the end of the year, hence 1080i50. It's really only 960x1080 from the CCDs, and we're going to process it to an effective 540 to 1080 vertical pixel resolution by the time we're done. But from a $3500 camera, that's a helluva thing.

Using LumiereHD, you can work with HDV footage in Final Cut Pro HD using proxy footage and switching back for a final render. Read about the non-traditional workflow on their site. LumiereHD does not yet support 1080i of any support, but Frederic promises me they will.

I don't expect to be able to get 1080i50 HDV footage into FCP HD until sometime next year. At NAB (National Association of Broadcasters tradeshow in April), I expect Apple to support 1080i50 (the PAL frame rate for 1080 res) in the next version of FCP HD, likely to be called version 5.0. (It might not ship until summer, though - I have no hard data, only supposition).

But once that occurs, it should be possible to own the camera and the post production system to run it on to shoot, edit, and produce a final DVD for under $10,000. Camera options and accessories, lighting equipment, additional sound equipment such as microphones, crew, cast, are all extra. I'm focusing on the part I know about. Final Cut Pro HD's audio tools are known to blow from a serious professional's standpoint, but you can still do a lot of stuff with them.

Sony HDR-FX1 -$3500

Mac G5Dual 2.0 GHz - $2500
extra 1GB of RAM - $200
Production Suite - $1300
DeckLinkHD - $600
HDLink - $700
CRT monitor - $500
3 300GB FireWire 800 drives - 3x$375
1 cheapie SATA RAID - $840
1 FW drive to boot from- $200
1 23" 1920x1200 DVI-D LCD monitor - $1600 to $2000 (HP L2335 is $1600ish, Apple's is $2000)
======================================
$9565

That leaves room for more backup storage, editing keyboard, jog shuttle, a cheapie 2nd monitor for bins, whatever it is you want to do.

So the workflow might go like this:

Shooting ratio: go nuts: 10:1

final TRT (total running time): 90 minute feature

Shoot your HDV 1080i footage, it's 3.125 MB/sec = 11.25 GB/hr, call it 11 to keep it easy. Actually, that's 1080i60, so 1080i50 is probably more like 9 MB/sec.

DVCPRO HD 1080i50 = 12 MB/sec = 43.5GB/hr, call it 45 GB/hr


As a practical matter, I suspect that DVCPRO HD 1080i50 and native HDV support will probably happen at the same time, NAB next year. MAYBE one will be before the other in January, but I doubt it. NAB is the place to launch new stuff like this. And version 5.0 is probably what's next. So NAB, both of'em (or maybe only one, but I doubt it).

900 minutes = 15 hours,

15 hours times (9 GB/hr for HDV, plus 45 GB/hr for DVCPRO HD 1080i50 working codec) = 15 x 54 = 810 GB worst case scenario that you have EVERYTHING and want it all online, all the time, in both formats.

But the good thing is that this is all low data rate footage - no need for expensive disk arrays for this stuff. Buy three 300GB FireWire drives (about $375 apiece or less) and you've got it all the space you'll need for raw storage. Maybe buy one more to back up all your source HDV footage so you won't have to recapture if a drive craters on you. (Don't forget to Zero Write those drives using the Options button in Disk Utility!)

Buy another 300 GB FireWire 800 drive for render & scratch space - $370

Edit, edit, edit at 1080i50, using DVCPRO HD as a working codec.

Now I get tricky: Using the Media Manager (possibly in conjunction with LumiereHD), relink the source demuxed HDV audio and video (this is LumiereHD stuff, read here)ONLY FOR YOUR FINAL EDIT TIMELINE FOOTAGE.

NOW use Media Manager to Recompress To a high quality interstitial working codec - maybe BlackMagic's free lossless codec, or a more space saving but non-realtime lossless codec, or a very lightly compressed interlaced choice like M-JPEG-A at 90+% quality.

NOW convert the 1080i50 footage to 1080p25 footage using the deinterlacing options in your favorite program (geeky de-interlacing details) from the following list: Discreet Cleaner, Adobe After Effects, Discreet Combustion, Apple Compressor (comes with FCP HD), Compression Master. Some of these have their problems, I'm still picking my favorite for best quality AND ease of bulk conversion. When outputting, keep it as 1080i50, but since it's been de-interlaced, it's really 1080p25. Use the best quality codec you can that will play back in real time on SOMEBODY'S system, not necessarily your own.

Put the NEWLY created 1080 "p25" footage in a DIFFERENT folder, being SURE to use a batching system that keeps the names the same as the source files. Otherwise you're hosed, having to manually match clip by clip. Nobody wants that.

Obvioulsy, you're going to need some more space to do this. Preferably, it's to your low cost SATA array. Read all about how to do it by using the Google search bar at the top of this page to search within my archives for the word "SATA", I've written 10 or 12 articles on SATA RAID, including some DIY kits as well as pre-built systems.

Now use Media Manager again (see how handy it is?) to relink to this new, 1080p25 footage.

Caveat - I haven't tried this, you might well get into trouble trying to convert a 1080i50 timeline to a 1080p25 one. So back up and convert ALL your footage before you start editing. Woops, that makes for huge footage. OK, convert to a working codec. Woops, creates linking/timing problems. OK, I don't have this all figured out yet. But by the time the software gets updated and the cameras ship, I bet I will.

INSERT MAGICAL FIXIT HERE, TBD.

When done, convert 25 to 24 fps. How hard is that? A pulldown menu in Final Cut Pro HD. Seriously, it's just one step. Cool.

Master to whatever format you want, using your SATA RAID to a rented deck, or sending the file to a post house that can use the codec you send them on a FireWire drive.

What about a DVD you say?

Final Cut Pro HD's built-in downconversion from HD to SD blows - I'd kick out a digital master (one big QuickTime movie) and then use one of the earlier mentioned tools (After Effects, etc.) to shrink it down to standard definition anamorphic, and use THAT file with Compressor to make your MPEG-2 files for DVD Studio Pro 3 that you got in the Production Suite.

Make your menus and stuff with Motion. Wheedle some graphic elements from a friend with Illustrator and/or Photoshop.

This is a rough pass at the idea, so don't take it as gospel. Fortunately, NONE of the software (or the camera, for that matter) exists yet, so nobody can bitch at me about "I bought it and it didn't work!"

If you see holes (and I know of at least one large gaping one) in my methodology, PLEASE email me and let me know your wisdom on how to work through this either at all or better than I am so far - my email is mike at hdforindies dot com. (I'm spelling that out to attempt to hold back the deluge of spam I've been getting (the cost of running a blog), you can figure it out I'm sure.)

-mike, off to get a well deserved BEER!

Digital Producer has this review of the Sony XPRI HD editor. To my knowledge, this is the only NLE capable of editing in native HDCAM format. This is great in the same way that Final Cut Pro HD supports native DVCPRO HD editing - very low bandwidth, as-shot-on-camera lossless quality. I'm curious, though, whether the system captures via HD-SDI and recompresses to HDCAM codec, and if so does that introduce loss? The article does state that this $85,000 system that comes with 500GB (ooooh, two whole drives!) cannot edit 4:4:4 HDCAM SR footage at this time.

However, the system is EXTREMELY powerful, comes with multiple physical (as in actual hardware) controls for editing and color correction, and can do things like 3 effects on each of 2 layers and play back in realtime. FCP HD can't do that by a long shot.

For ultra-time constrained productions, this could be a very nice system. It's low storage requirements (native HDCAM codec runs around 50 GB/hr).

For folks looking to edit an HD movie, however, it's both too much and too little. No support for 4:4:4, and it didn't mention uncompressed footage, it might be an all native HDCAM codec/IMX machine. I'm not sure.

The theory sounds nice to edit with - native HDCAM, superb realtime effects, excellent controls...but $85,000? I can set up an entire post production suite for that!

This isn't a film editor's box, especially an indie film editor's box. I keep coming back to Final Cut Pro HD as the best solution in that regard for quality, interoperability, market acceptance, power (for the money), scalability, and low cost.

Anybody disagree? Please email me why you think so - mike at hdforindies dot com.

-mike

Here is the Adobe link announcing the format launch.
Here is a nice MacWorld summary & analysis article about the new format and what it means.

Mike's Commentary & Analysis:

Still image manipulation is always running ahead of video manipulation for a variety of reasons, market share and ease of hardware access are two entertwined reasons (still image manipulation hardware is cheaper than video manipulation hardware).

Adobe has released a new file format (and Photoshop plug-in to read/write it) that is non-proprietary, inviting others to use it both to read and write the file format. It's called DNG, for Digital NeGative. If you are at all familiar with the RAW camera file format, DNG is a standard for cameras to write a standardized RAW format - most RAW file formats have to support each camera type, one at a time. DNG is a format for all vendors to subscribe to, and all of the cameras already supported by the Photoshop Raw Camera Plug-In can now be converted to DNG.

So why care about it over on the video side? Because DNG is all about snagging the raw output of the CCDs and letting the user manipulate it as they please, to control certain assumptions about white point, gamma, sharpening, and other image factors.

My 3D buddies have been using RAW format to generate HDRI (High Dynamic Range Imagery) to to environmental lighting and reflections in their 3D applications with stunning results for a few years now. HDRI imaging is the new standard on how to light your scenes for photorealistic integrated 3D/live composites.

I think, in time, folks are going to want to be able to capture the raw CCD output from their video cameras. Certainly the highest end professionals will want to be able to, and visual effects artists will want to be able to. I think it'll be years before the possibility, and might involve producing cameras with a special "bypass" mode that allows for capture directly from the CCD array straight to a hard drive. But wait a minute, isn't that what the Dalsa Origin is already doing, just not to this particular format?

I think it'll be very interesting to see where all this goes.

In the meantime, I'd highly advocate that Adobe come out with an After Effects (and Premiere Pro) compatible File Format Plug-In so that those applications can read in DNG formats as well. Still image photographers aren't the only ones that will want to play with this.

I've already emailed with 3 or 4 stop motion artists who are shooting digital still photography for their high definition (be it HDTV or theatrical destination) work, and I've advocated using Photoshop, the RAW file format, and annoyingly laborioius scripting via the Actions functionality to batch process to color correct shot by shot.

Let these folks just bring in a sequence into After Effects and tweak it there.

It would be WONDERFULLY powerful.

And that's what Adobe's all about, right?

-mike

Since I got busy and did a poor job of covering what's new at IBC, here's a nice couple of summary articles via Creative Cow

Part 1

Part 2

I especially like his closing comments about Final Cut Pro HD:

Final Cut is now a serious tool that broadcasters see as a replacement for their now aging standalone systems like Avid and others. Apple has to crack the Unity market with Xsan though. Expect more third parties to be offering FCP as a high-end editing solution.