HD For Indies

CORRECTION 6/30/04: Reader Brad Gallo wrote in to say:

"Mike, just a correction to your post regarding the new
monitors and 6800 card. In the keynote address, Job's
specifically indicated that each dvi connector was
actually a dual dvi port. so one 30" monitor per port
is standard with the 6800. As Jobs put it, " dual"
"dual DVI" 6800 card. So the 6800 card WILL run two
30" monitors for those who have the cash/budget."

So DISREGARD all the "need a second card for second monitor" stuff below.

-------end update-----

So here's a quick rundown on the new Apple LCD monitors announced today as expected:

20 inch model - 1680x1050 - big enough for 720p footage by a long shot. $1299

23 inch model - 1920x1200 - similar to a reskinned older 23", $1999. Great resolution to get pixel for pixel previews of 1080p or 1080i work.

30 inch model - 2560x1600 resolution, $3299. Requires special video card (see below).

All 3 of these monitors have dual USB 2.0 interaces and dual FireWire 400 interfaces. Awesome.

Sneaky deal: if you're planning on a 720p project, run don't walk and get the 17" model that supports 1280x1024 pixels for only $699. It's the older style, but hey! It works great...ADC connector as well.

Links:

Apple's page (with specs link) on the new monitors

QTVR of one of the new monitors (I think the 23", can't be sure)

Mike's Comments:

Sorry I've been offline so much, I'm having ISP troubles from the home studio. Plus, oh yeah, amazingly sweet/cute/nice/lovely 6 foot tall marathon running cutie new girlfriend. Researching online for new HD cards? Umm...no. Researching online to hike the Grand Canyon. But I digress.

: )

The new monitors - great, awesome, fantastic, gorgeous. Yes, I am breathing deep of the Apple smoke. The 20" is more than adequate for 720p work, (consider the older style 17" for a bargain 720p previewing solution with the HDLink).

Kinda as I suspected, these do NOT have HD inputs, be they analog or digital. Perhaps some future product. But these monitors are still of use for editors to work with timelines or to preview content using One2One or HDLink

The new 23": Perfect for 1080 res previewing with HDLink or One2One. And yes, you CAN drive this from a (recent) PowerBook. My dinky little 12" PowerBook (new 1.33 GHz model) will drive this 1920x1200 pixel monitor, in full 24 bit color, in ADDITION to the built in display. And of course, the new 20" is drivable from the PowerBooks as well.

The new 30": 2560x1600 pixel resolution. Wow. Which is enough for true 2K (2048x1536) film work. Will BlackMagic make a compatible HDLink for it? Good question. But $3299 - ouch, more $$$ than I thought. And in order to drive that, there is only one card to do it, the new $600 ($450 as an upgrade on a new dual 2.5) NVIDIA GeForce 6800 Ultra DDL Card, which requires BOTH of it's DVI interfaces to drive this one monitor. This is going to be a transitional technology - I'll bet in a year or two they'll have a single connector of some sort to drive it. NOT AVAILABLE UNTIL "August". And since it takes both monitor outputs to drive the one display, if you wanted a second display for bins, palettes, previews, etc., you'd have to use a slower PCI slot...which would leave only one slot available in the machine. So not even room for a SCSI card if you want to include an HD capture card. But then again, isn't 4 million pixels on one monitor enough for most folks? With my 24" CRT and a 15" LCD, I only use about 3.4 million pixels at best (usually just over 3 million at 1920x1200 and 1024x768).

So it'll be October I bet until dual 2.5 GHz G5s and the 30" are readily available at retail stores on the shelves I bet. If you are counting on a dual 2.5 & a 30" for a project, don't count on laying hands on it until mid October at the earliest to be sure. Apple has a history of announcing one ship date and the reality of mortal civilians being able to lay hands on it can be something else ENTIRELY.



Other tidbit from the Apple site:


"Special note on the NVIDIA GeForce 6800 Ultra: due to size of this advanced graphics card, the adjacent PCI or PCI-X slot will be blocked and cannot be used. This reduces the number of available PCI or PCI-X slots from three to two."

The card is so big it crowds at LEAST one slot, and might interefere with another (maybe, totally my conjecture).

So you'd be limited to a 4 SATA drive array with the presently availabe SATA cards.

If you had a 4 port PCI-X SATA card, you could do a 6 drive array.

So as great as the 30" monitor is, and as much as I'd like one (not ordering yet!), a dual 2.5 G5 with the NVIDIA card (to drive the 30" display) and the 30" display is $6748 before you add, um, ANYTHING else.

Other Apple goodies:


The next OS upgrade will be tiger and ship in "mid 2005" (September?). Some features:

-H.264 codec for next-gen HD DVDs - good that Apple is working on this.

-Safari browser - RSS support in a big/deep way. Good.

-Core Video- THIS IS HUGE: GPU will do a lot of video work on top of/besides the CPU. THIS IS HUGE for post production/editing processing. Especially since it will be OS level, not just application level. Expect Apple to have good support for Motion, FCP HD, and DVD Studio Pro somewhere between Tiger's ship date (middle-ish 2005) and early 2006 (either MacWorld San Francisco in January or NAB in April).

-mike

Got a press release from AJA, here it is in it's entirety:

###
EMBARGOED UNTIL MONDAY, JUNE 28, 2004, 12:00 NOON PT

AJA Video Systems' Io
Receives Final Cut Pro HD Qualification
WWDC 2004, San Francisco, - June 28, 2004 - AJA Video Systems Inc., a leading
manufacturer of standard and high definition video conversion and interface
solutions, announced that its Io FireWire® Interface for editing on the
Power Mac G5 and G4 has been qualified by Apple for use with its award-winning
Final Cut Pro HD editing software. Qualification is achieved through rigorous
testing and analysis to ensure that devices are compatible with current
Apple hardware and software.

“We are proud that Io is one of the first plug-in solutions for editing
on the Mac to be qualified for Final Cut Pro HD," said John Abt, president
of AJA Video Systems. “Io was developed to ensure that it is the best possible
product in its class. We are pleased that, once again, AJA desktop video
products meet Apple's high standards of quality and reliability.”

“Final Cut Pro HD delivers on the promise of HD, enabling professional
editors to capture, edit and output broadcast-quality HD video over a single
FireWire cable,” said Ron Okamoto, Apple's vice president of Worldwide
Developer Relations. “Io provides powerful new opportunities for the post-production
professional, and we are pleased to have it available for Final Cut Pro
HD users.”

About Io
Io provides simple, elegant plug-in solutions for professionals working
with Final Cut Pro on a Power Mac G4 or G5. With a single FireWire connection,
Io connects 10 bit uncompressed video and 24 bit multi-channel audio, analog
or digital, to the Power Mac. Io products are the simple solution for building
a Mac-based video finishing system.

About AJA Video Systems, Inc.
Since 1993, AJA Video has been a leading manufacturer of SDI and HD-SDI
video interface and conversion solutions, bringing high-quality, cost-effective
digital video products to the professional broadcast and post production
markets. AJA offers Io (the “ultimate” capture box), SDI and HD-SDI QuickTime
PCI capture cards, miniature stand-alone converters, and a complete line
of rack mount interface and conversion cards and frames. With headquarters
located in Grass Valley, California, AJA Video designs and manufactures
all of its products in the USA, with an extensive sales channel of dealers
and systems integrators around the world. For further information, please
see our website at www.aja.com.
###

Updated Thursday, July 1st 9:15am CST - see bottom of article for reader comments/feedback

I was cruising over in the 2-pop FCP HD forums and somebody presently in school is learning Avid XPress, but as a G5 DP 2.0 at home and wanted to know which was "better" - Avid or Final Cut Pro. Egads. A bit of a Ford vs. Chevy issue. Or more accurately, maybe Acura vs. Mercedes. Here's what I had to say:

---------------

I would say it depends on your goals and your market. If you want to be able to do your OWN projects, the scalability and DIY aspects of FCP HD are excellent. If you want to get work in the professional market...it depends. Smaller jobs are increasingly FCP jobs. Bigger more serious projects are often Avid jobs. If you're just starting out, are you going to have access to those projects? Probably not.

Learning both would be best, obviously. The beauty of the Avid lineup is that the tools are pretty much the same across the product line, except that the higher end setups have more of them. Some features aren't present on XPress, but many are. So learning Xpress lets you grow into the bigger stuff, or at least give you a path.

Some markets vary - Dallas apparently is a big Avid market, Austin is more of an FCP market.

True story: my editing buddy that used to own his own facility in the Bay Area ran into unfortunate timing when he tried to expand into Austin during the dotcom boom/bust cycle. So he had to sell his stake in the company and take a job as the inhouse Avid editor at an agency. All Avid gear, what he'd been working on for years, he totally knows his s**t, no problem.

But that agency folded earlier this year. Now, as an editor with 20 years experience but only knowing Avid, he's having problems finding freelance jobs (the kind of stuff you're likely to pick up as a starter) because everybody wants Final Cut Pro, and maybe even After Effects knowledge to go with it. So he missed out on a lot of projects and badly wants to learn FCP now.

My gut says that FCP is going to dominate the market for not-super-highend stuff. It's too good, too cheap, too scalable, too affordable and accessible. Avid makes some very, very good products. Side by side, the Avid does more. You can quibble about the UI of FCP is better and more fluid, and I'd agree with you. But in terms of power & features, Avid kicks ass.

The dilemna is that Avid costs a bazillion more dollars. For roughly comparable systems, you can get into an FCP rig for a fraction of the cost once you move into uncompressed SD and HD (compressed or not) systems.

Is Avid more powerful? Yes. Is it 3 to 5 times better, in line with the cost difference? I would argue no. If I can get into an FCP system with 80% of the features for 1/2, 1/3, 1/4 of the price, that just makes too much business sense.

The QUALITY is the same in the argument I'm making here - nobody's saying (that I've heard) that Avid 4:2:2 uncompressed looks better than FCP's. Or that the quality of the results of the color corrector looks better in Avid. I hear arguments that the tools are better, but not the results.

So from a straight business decision perspective, it's hard to cost justify Avid for the next few years.

Avid is getting much more aggressive about bringing their costs down, and the DNxHD stuff looks promising, both in terms of power, quality, and price.

And the better tools let you work faster, which saves money. The crucial question is this: does the increase in productivity offset the increased dollar outlay? It depends on the cost of your editor & facilities, and the amount of money the project brings in. In high end facilities doing top notch commercial work, where the speed of turning over a project is crucial in a multi-hundred dollar an hour billing situation, Avid can cost justify and be the smart choice handily. In smaller, less costly situations, especially for independent film editing, it has a harder time justifying it's costs.


But at the moment, I think the momentum is with FCP and will be in the future. Apple DOESN'T CARE about how much FCP costs, they just want to sell boxes. That's their market. Avid ONLY sells editing stuff, and depends on that for their future and profit. Apple has an edge there.

There are strong legacy arguments to be made that Avid is perceived as superior in the market, especially among established film & advertising clients; and that the best/established editors work on Avid. But for MOST of the jobs being done out there (corporate & commercial), FCP makes too much sense.

Ultimately, the market is a BUSINESS. If an Avid can't justify it's higher cost with it's superior features and ultimately faster workflow, it won't be selected. Final Cut Pro based solutions, with lower costs, greater scalability, arguably better user interface, and still arguably quite good and sufficiently competent feature set, has a lot going for it.

But I also endorse the other guys' argument. It's the cutter, not the scissors (shooter vs. arrows). Learn your tradecraft and it will be easily applicable to whatever valid tools are out there.

-mike

UPDATE THURSDAY JULY 1ST 9:15 AM CST

Reader Jason Rodriguez wrote in:

Hey Mike,

Just read your post on the AVID vs. FCP thing. Very good, but I
thought I might add something.

You mention that AVID is much more powerful. Having used everything
from Xpress to Symphony, and having an FCP rig, I must say that
ironically when it comes to image quality, FCP has AVID beat hands
down, unless you're talking about the DS/Nitris. AVID still can't
keep up with FCP when coupled with an uncompressed 10-bit card such as
Blackmagic, Cinewave, or AJA. In addition when it comes to HD, FCP
totally smothers AVID, unless you put on some of the VERY expensive
add-ons to the adrenaline to get you uncompressed HD. But now that
Blackmagic is supporting dual-link 12-bit RGB in FCP, again, FCP has
pulled ahead in it's capability for image quality. AVID has no
capability of 32-bit floating-point processing, it's DVE is absolute
junk (unless again you're using DS), frankly the only thing it's
really good at IMHO is group editing (via Unity), and media
management. Also AVID has a nice edge over FCP with Animatte and it's
color corrector, although the latter can be fixed with Color Finesse.
Another concession I'll make to AVID is that it can mix resolutions on
the timeline, something that you need a Cinewave card for in FCP.

But frankly FCP is much more flexible when it comes to codecs,
importing and exporting files (AVID's a painful import/conversion
process), rules when it comes to image quality with it's 10-bit 4:2:2
codecs (except of course the DS/Nitris again, but that's not even
original AVID code, that's Softimage), and is catching up in the media
management department.

I know it's easy to think that AVID's much more powerful compared to
FCP because of cost, but frankly the two are pretty evenly matched.
For instance how about real-time streams? You get eight uncompressed
streams in FCP, while the fastest AVID's are getting you six-seven,
and even then it's not the "highest quality" effects, to get those you
often must render. FCP's are true full resolution when playing off
the timeline when in 8 and 10-bit mode.

At NAB this past year I was traveling around with a group from work
that were trying to configure an edit system, either with FCP/Cinewave
or AJA I/O, and the AVID. Their big selling point wasn't the capabilities
of the system, it was Unity. They were pushing that hard. And I can
see why, because frankly that's the only thing that I really feel they
have left as and edge-group workflow/editing/media management.

These are just my opinions, and I do feel that you presented a
nice argument for both platforms on your website, and like you said,
it's good to know both and be familiar with both platforms if you want
to be a working editor-and I've been following that advice myself,
again I've used everything from Xpress to Symphony, and of course
FCP-they all have their strengths and weaknesses, and I think you
addressed the issue well.

Thanks again,

Jason Rodriguez
Virginia Beach, VA

This is a press release from RaveHD:

FOR IMMEDIATE RELEASE
SpectSoft announces release of RaveHD software-and-boardset for Linux

Oakdale, Ca. -- June 21, 2004 -- SpectSoft, LLC today announced that the popular RaveHD system used in major motion pictures like "Starship Troopers 2", is now available in a software-and-videocard only version. This joins the complete turnkey systems which SpectSoft has offered in the past and has installed in facilities like the famed Tippett Studios. "Allowing customers total flexability was the ultimate goal," commented Ramona Howard, Marketing & Sales Director for SpectSoft. "The RaveHD software/card combo sells for under $8,000 ($US) and we provide a list of certified components on the SpectSoft website, we can't get much more flexable than that."

Using AJA's new I/O card which supports SD/HD & DL on the same board, RaveHD supports a whole gammut of video formats that has made this combination a very serious contender in the DDR market. End-users are now able to build up DDR-based systems and/or install the software on existing systems to reduce the cost. Previous DDR-based systems have cost $100,000 ($US) or more. SpectSoft is breaking new ground by releasing a full-featured software package that rivals these pricier systems as well as providing the client access to the source code.

RaveHD is a second generation, open source, Linux based, hybrid digital disk drive recorder and video editor geared toward the high-end film and graphics industries. It utilizes open standards and protocols to achieve a product that is elegant, easy to integrate and totally customizable. It features a data agnostic core that allows it to work with today's uncompressed standard and high definition video as well as tomorrow's super-high datarate video.

With features like frame oriented, standard file system storage and centralized database asset tracking, RaveHD is the first DDR to really focus on the problems faced in working with film. Other features like RP188 (embedded timecode), Varicam support, realtime color correction, programmable cadence (2:3, 3:2:3, 2:3:3:2, etc), and RS422 machine control are standard as well. For more information on RaveHD, visit: http://www.spectsoft.com

About SpectSoft: SpectSoft, LLC, was founded in 1997 to specialize in Linux-based software for use in the film, media, and broadcast industries. Since its inception, SpectSoft has worked on technical projects with some of the industry's biggest and brightest companies like Dreamworks Animation (Glendale, CA) and Tippett Studio (Berkeley, CA) -- helping them push the envelope and further the state-of-the-art in the film pipeline process. Throughout the years, SpectSoft has been able to maintain a relatively small footprint, the family values and ownership that make it so unique.

For more information on SpectSoft, visit: http://www.spectsoft.com

Media Contact: Ramona Howard ramona@spectsoft.com Phone: +1.209.847.7812



Mike's Comments: For those doing effects laden high res projects, this Linux based DDR does all kinds of very power and useful things in real time. It would be interesting to map out how this would/could interoperate either with an FCP editing system (any benefit?), or with effects footage in progress. Devil's advocate position: while this thing can be controlled like a deck, exactly what new does it do for you that an NLE doesn't? Have to read up on it.

first off: Apple released some component updates for FCP HD users working with the Panasonic 1200A deck (the FireWire DVCPRO HD deck). Get it here.

from the readme:

"...Final Cut Pro QuickTime DVCPRO HD Component Updater. All users of Panasonic AJ-HD1200A VTRs should use this updater to update the firmware of their AJ-HD1200A VTRs. This updater only updates QuickTime components used by Final Cut Pro HD. It does not update the Final Cut Pro HD (version 4.5) application."

------

Techseekers.net has posted some pictures supposedly of one of the new 2.5 GHz G5 systems with watercooling.

Unlike some previous pictures/rumor stuff, this seems to have the same motherboard as the current (and older) G5s. If this is true, that lessens the likelihood that the FireWire 800 and USB 2.0 write speed issues have been fixed. This, of course, is my conjecture based on spy pics of potentially non-shipping product. So who knows for sure (just some Apple engineers, and they ain't talkin'.)

---------------

AppleInsider is reporting that Apple is gearing up to ship 30" LCD monitors.

Highlights:

-new aluminum look to match G5
-single pedestal for physical support
-2 FireWire, 2 USB ports
-20" model - approx 1600x1024 res, $999
-23" model - 1920x1200 still, but between $1499 and $1799
-30" model - 2560x1600, $2999
-mention was made of a NEW DVI connector - woops, might need a specific graphics card or adaptor for cards....bummer.

Mike's Comments: this is in line with rumors from the other week about size/res/pricing info, but no mention was made of HD signal input capabilities as had been rumored in the past. Too bad. Maybe in next year's models. Or maybe as a pricey ($500? $1000?) add-on to the units. That's total, whip-out-of-my-ass conjecture, though.

Would make sense for Steve to announce at WWDC next Monday.

------

Apple also announced iPod integration with many recent BMW and Mini models. This has nothing whatsoever to do with HD, it's just really bad ass. I bought an Acura RSX Type S two summers ago because there were no Mini dealerships in Texas at the time. I continue to pine for a Mini, this just makes it worse. I keep staring at my steering wheel and plotting ways to rig the iPod remote to it but still be able to spin the wheel more than 2 turns lock-to-lock in tight cornering situations...mmm....tight cornering situations...


---------

Reader Christopher Barry might as well be a contributing author to this site...as soon as I get the Forums section up (ETA: unknown) I want to post the entirety of our 150+ and growing email exchange on the exact and proper way to do Varicam/Panasonic 1200A/G5/Final Cut Pro HD workflow.

some links of interest from our recent conversations:

FCP HD 3 way color correction

the cool scopes and hyper accurate color correction of Synthetic Aperture's Color Finesse

audio out of SDI back to DVCProHD 1700 deck is fuct...

breakout box from Kona2 - what's 2X Loop BNC do?

Magic Bullet for Editors - only in standard def for FCP HD...dammit.

After Effects version of Magic Bullet, which will do HD no problem

specs on Panasonic Frame Rate Converter (the hardware one) that can do 720p==>1080p conversion

some camera charts

and pdf for camera chart pricing

more camera cards & charts

DP dude talks about Varicam and Viper cameras

Sebsky Tools for moving from Avid <==>FCP HD back and forth (bins etc.)

yet another data rate chart...everybody seems to have their own numbers...

pre-release review of Kona2 card

review of 9800 XL card, which might block PCI-X slot 2 (bad for dual Seritek SATA card install)

BlackMagic announces specs on DeckLink HD Pro card

AJA video converter (1 of 3)

AJA video converter (2 of 3)

AJA video converter (3 of 3)

quality assesment of HDCAM vs. other digital storage devices with generational loss analyzed

Panasonic D-5 HD mastering system ($100K list plus options)

Panasonic's web page on 1200A deck (the coolio FireWire DVCPRO HD one)

OK, that's enough for now. That ought to keep you busy for a few days...

-mike

...and thanks again to Christopher for all his hard work and research.

Notes from Day 3 at Martini Shot

mark clips for capture in 8 bit - works so long as no timecode break

so what happens if you capture DVCPRO HD, but then drop into a comp in AE and cook out Blackmagic or other uncompressed? Will there be a color difference?

NEED TO TEST THAT!

DROPPED FRAMES DURING ATTEMPT! because I pressed keys during capture? Dunno. Worked second time.

checking capture settings...

-----------

logging clips and capturing one a time - working OK

capturing close to a take break - possible time code break? Yeah, hoses it - rewinds for preroll too far - so still have to be very careful about timecode breaks and capturing too close to a scene change

Panasonic has better color reproduction than Sony (is argued) - CineGamma

Capture presets with BlackMagic:

there are ONLY two presets for 720 format with BlackMagic:

720 59.94 Hz 8 bit
720 60 Hz 8 bit

further exploration - these are the PRESETS - you can go in and make your own. I was under the impression that the presets were it, but you can set up 720p 10 bit and PhotoJPEG as well- it's just that BlackMagic doesn't install premade presets for those.

Varicam camera does NOT have deck control...Blayne didn't know off the top of his head if others have deck control either...Sony F900 doesn't for sure

...so the idea of capturing off the camera after the shoot has it's STRONG limitations. You can capture from tape on camera to hard drive - but without deck control for sure. Timecode can apparently be an issue as well. So it's not a very efficient solution. And if it isn't repeatable, you're DEFINITELY in trouble if you're capturing offline quality footage such as DVCPRO HD, PhotoJPEG for BlackMagic, or QRez for AJA.

if shooting greenscreen, could capture direct to disk 10 bit 1280x720 - good monitoring solution, too, if using BlackMagic w/eCinema or HDLink. Again, for 10 bit only capturing live to disk during shooting, not from tape later.

they have had issues getting timecode when trying to do camera to deck dubs - no timecode on the HD-SDI-a configuration issue?


shooting clean in field and fix in post is the gwoing/prevailing trend - this is how it's going to be done quite often from now on. Old skool film DPs don't like it, makes them more of a technician, less of an artist...unless they are involved in post, which they aren't always.

Blayne says - a lot of film DPs are getting pissed about shooting clean and fixing in post - have to get used to the fact that "it's a computer with a lense on it"

but the little remote box -ECU (External Control Unit) AJ-EC3 gives you access to a lot of the controls within the camera that are otherwise only accessible from the on camera menus. So there are a lot of color corrections things you can do within the camera, just by twiddling knobs on this box while on set and watching the monitor to see what happens. BUT if you overdo it or do it wrong, you've baked those adjustments into the footage. So while it is theoretically quicker to do this color adjustment on set, if incorrectly done you're in trouble since can't be undone. For instance, once overexposed to white, there's no bringing it back...whereas if you shoot flat/full exposure range in camera and fix in post, any adjustment made in post can be re-adjusted while still in post.

Another tradeoff - if doing it on set, you're burning valuable on set time to do this. Time on set is VERY valuable and expensive. Time in post is less expensive and (if on your own system) not nearly so time critical.

INCLUDE A PICTURE OF the little ECU (external control unit)

It lets you dial in a lot of your color stuff, gives you direct access to the menu stuff but gives you a physical control for a lot of the knobs without having to go through itty bitty menus.

Is a lot like the difference between iPod & iTunes - both do generally the same things, but is a helluva lot easier with the mouse & larger screen in iTunes than it is with the tiny jog dial and a few buttons (and no keyboard) in iPod.


Other bits and details:

cooldrive.com for SATA enclosures

Fuji makes tape for Panasonic for DVCPRO HD - price is $75/46 minute tape (more expensive than HD CAM stuff!)

Maxell makes 32 minute tapes, about $1 a minute

Media Toolbox in Austin- they sell media, all kinds of video tape - they carry some kinds of video tape and can get it next day

Panasonic issued this press release on the availability fo the Goodman Guide on the SDX900 DVCPRO camera.

BareFeats.com is reporting that the new G5 2.0 GHz machines STILL have the same constrained FireWire 800 performance. From his report:

" In my quick test with a striped pair of LaCie Big Disk Extremes on the built-in FW800 port, the sustained write speed is still stuck at 54MB/s while the same pair connected to a G4 PowerBook gave a sustained WRITE speed of 74MB/s. (A PowerBook beats a G5? Huh?)"

See his full report for discussion of new vs. old G5 speed (same), test results on the 5200 graphics card ("wimpy"), 5200 graphics card performance stats, and buyer recommendations (pay $50 for the 9600 card).

While the new machines have a new internal designator (PowerMac 7,3, older 2.0 GHz machines were 7,2).

Keep in mind, this is ONLY results for the 2.0 GHz machines. While I'd expect the new dual 1.8 machines to have similar problems (and they aren't appropriate for HD use anyway), the new 2.5 GHz machines due in "July" (my quotes to indicate lack of confidence in that shipping date) may or may not have these FireWire 800 problems, as they appear to have a substantially different motherboard (so the possibility exists but isn't confirmed).

-mike

Adobe has posted a 30 day demo that is mostly feature complete (no MPEG-2, no additional support apps) here.

Why care? After Effects and combustion are the two best (relatively) low cost applications for doing special effects work. After Effects is a better tool for motion graphics, combustion is arguably better for effects work. But in my opinion, After Effects is a great all around workhorse solution.

FireWire Depot rolled out their eRAID system not too long ago, and now they've added what they're calling clustering - the ability for two computers to be attached to the same RAID - sorta like a mini-SAN.

Mike's Comments:

This could potentially be useful in multi-editor situations....whether the performance is sufficient for both to edit at the same time remains to be seen.

Other research thoughts

I've been meaning to do some tests to see how well my old G4/733 running Server 10.3.4 could serve source video files to two Macs (a G4/867 single processor and a first gen G5 dual 2.0). The server just has some basic ATA drives in it. Will it support DV footage? How many streams? Will it support DVCPRO HD? At which data rate? I have two matching 250 GB drives - will they need to be striped to serve files viably? Too many tests, too many ideas, not enough time...

-mike

In case I hadn't made it clear, I've been hanging out at Martini Shot Productions, an Austin TX based facility that edits HD and shoots with and rents out their Varicam.

Here are my notes from today:

There are 3 or 4 cartridge sizes of DVCPRO.

The large tapes (and the extra large tapes in the 1700 new deck)

So any of the large size tapes can be used to record DVCPRO HD.

The large sized tapes hold 46 minutes 720p footage, regardless of framerate you shoot - because it is ALWAYS writing 60 fps to deck

There is a larger format tape (extra large) that holds 2 hours of footage

Trying to capture 720p24 - not happening

rendering to DVCPRO HD takes about 3:1 realtime, either exporting from QT Player or in FCP timeline

have to use the FRC to capture anything but 60p using BlackMagic card as it stands now without new drivers

the codecs (uncompressed BlackMagic 8 bit 2Vuy and DVCPRO HD, both at 720p60) look the SAME on HD Monitor (other than the squarsh factor), but look different on computer screen

BUT in After Effects, DVCPRO HD shows up as 1280x720

so I was wrong about trouble in post!!! Only in FCP HD does it show up as 960x720. Elsewhere, it shows up as 1280x720. AND when you add an uncompressed 1280x720 codec piece of media to a 960x720 DVCPRO HD timeline, it PROPERLY handles aspect ratio. THIS IS GOOD!

AFTER LUNCH:

do some AE render tests with both sources (uncompressed and DVCPROHD) and both outputs (SAME TWO) and import into FCP and drop into 2 timelines: uncompressed and DVCPRO HD

If they all look the same, we're golden

ANSWER: QT Player, uncompresed source, & Compressor all look the same - only Cleaner darkens the footage.

Cleaner is quite noticably darker on computer screen, but still looks about the same on HD screen.

PLAY WITH 720P DIFFERENCER FILE TO CHECK!

I did - Cleaner is the only one that darkens. So don't use Cleaner to codec flip!

dropping frames!!! dropped several in a row on rendered footage from AFter Effects. Why? Perhaps because I was using dual outputs in the Render Queue - After Effects was calculating what the frames looked like, then writing the next new frame to one BlackMagic 8 bit 2Vuy codec, one to DVCPRO HD codec. But then the copied over After Effects renders are dropping, too

don't know why. Footage that was captured from hard drive worked fine, files rendered from After Effects to BlackMagic 8 bit 2Vuy codec - dropped frames big time! Why?
---------

installed cinewave card, went to pinnaclesys.com to download drivers etc.

installer requires serial # and registration - weird

installing a bit funky

Card ran HOT - didn't get over 50 degrees Celsius in his G4, in Slot 2 in G5 (scant millimeters away from 9800 Pro card with heat fan & heat sink) it ran 53 shortly after installation, was up to 55 when we were having trouble and just shut it down and bailed.

We had deck control working, we had the presets loaded the same way he had it working in his G4 under 10.2.8, but we had all kinds of weird frame dropping problems - it captured a few seconds then lost it.

More to play with later. I'm going to do some further testing, hopefully with 1080i footage, later.

-mike

I've written a bit about this before, but here's their official page on their Swift Data 200 internal 5 drive expansion kit.

While waiting for tape captures to run, I was talking to Craig and Martini Shot (Austin, TX based shoot/post facility, they rent Varicam) about various workflows enabled by all these new toys. I took some notes while we talked, some of this might be new to some of you out there. Here they are in raw "as I typed 'em" form:

-----------------------------------

DVCPRO HD WORKFLOWS:

option 1: capture uncompressed, work uncompressed, render uncompressed

Pros: always high quality, guaranteed no quality loss, most boards support RT effects on uncompressed, render & layback uncompressed

cons: need tons of high data rate, high cost storage

option 2: capture via FireWire, edit DVCPRO HD compressed codec, render DVCPRO HD, lay back to tape DVCPRO HD

pros: low data rate, no RAID needed, fine for cuts only (functionally lossless)

cons: if any color correction, effects, layering etc., you're rerendering back to a compressed codec, generational loss

option 3: capture FireWire, edit DVCPRO HD compressed codec, essentially work knowing are getting some loss....THEN change your sequence settings to uncompressed, re-render...lay back to tape either uncompressed, or create a DVCPRO HD version from your uncompressed...does this actually help you skip any generational loss?

could re-render after capture via FireWire, so working losslessly

option 3a: immediately flip to uncompressed & re-render your timeline, work from there

option 3b: work in RT mode with DVCPRO HD, then flip to uncompressed at end to re-render digital master

Maxell 30 min tapes $35ish, Panasonic 44min $75

other note - when codec flipping, Cleaner makes it darker - Compressor does not

Option 3 pros & cons:

Pros: optimal quality, minimal drive space

cons: when is the ideal time to flip the timeline over? At the beginning, IF you have enough drive space? Or at the end, so you can work as fast as possible? Guess it depends on when you want uncompresssed workflow, and when you're willing to wait to render. I need to do some tests to see how much rendering is required AFTER you flip a sourced as DVCPRO HD timeline to uncompressed. Does it have to re-render after each change, or can it RT after rendering it once?


---------------------------------

Yeah, I oughta know the answer to that last one already, but my knowledge is lumpy. I know a lot about After Effects and hard drives, less about details of FCP timeline manipulation. I wouldn't say I'm an idiot savant (some of you might, I'm sure...), more of a dilletante savant. I know enough to be dangerous and have bad/crazy ideas, only some of which might work.

-mike

UPDATE 6/15/04 10:30 PM: I was wrong about difficulty in post - DVCPRO HD 720p footage shows up as 1280x720 in QT Player and After Effects. ONLY in Final Cut Pro HD does it show up as 960x720. And even in there, if I bring in a 1280x720 piece of media from elsewhere, it PROPERLY handles aspect ratio. THIS IS GOOD.

UPDATE 11:45PM TUESDAY 6/15/04: Based on an email conversation, let me clarify - the ONLY way to EVER get true 10 bit detail footage off of a Varicam is to capture to hard drive straight of the HD-SDI tap LIVE AS YOU SHOOT IT. After on tape, too bad, foverer 8 bit. You might be able to CAPTURE it to a 10 bit file, but the detail level is forever 8 bit. In other words, even if you set up your capture system to capture 10 bit, your file would still be functionally equivalent (pixel for pixel match when decompressed) to the DVCPRO HD codec, which is compressed, and 8 bit.

Today I was over at Martini Shot (thanks Blayne & Craig!) with my G5 and 600 GB, $750 SATA RAID testing capture performance with their DVCPRO HD deck. Results? For uncompressed, excellent, for DVCPRO HD, not so great. Read on for more...

UNCOMPRESSED RESULTS: FANTASTIC

So here's the setup: My dual G5 2.0 running OS X 10.3.4. I have a Seritek 1S2 card in it, and I have 2 internal and 2 external Barracuda 160 GB drives (these are the same model the G5 ships with). The two externals are in PPA Inc. external SATA cases (good enough for lab work, not so great for long term production gear). Also have a FireWire 400 drive to boot from, and just to be safe I have a FireWire 800 drive I'm capturing audio to. I have a DeckLink HD card mounted in the slot 2, the Seritek in slot 4 (not ideal, but I couldn't get the HD card in Slot 4 after a LOT of trying). Running Final Cut Pro HD, and connected to the smaller Panasonic DVCPRO HD field deck. Not the new FireWire 1200A model, the older one. They have a hardware frame rate converter and a Varicam as well. More on those later.

As I've previously discussed, hard drives generally write from the outer edge of the drive towards the hub. As the drive gets full (or you write to a partition towards the end of the drive), the drive performance (or array performance) drops. So the crucial test isn't what happens when the drive is empty, it's what happens with the drive is almost full that is the biggest stress test.

This little $750 array performed almost flawlessly - after getting FCP configured (about 10 minutes, including a funky deck control setup due to lack of a male-to-male 9 pin RS422 cable), we were able to capture 1280x720 4:2:2 8 bit video from the DVCPRO HD deck via the HD-SDI connection. 105.4 MB/sec for the video. This little array is good for 115 MB/sec read & write down to the last 10 GB of the array, so that gives us a little bit of headroom for this data rate.

In short, it worked fine. Messing around with it for a couple of hours, one time I dropped a frame during playback, but that was after a lot of single frame jumping around and then hitting play. I'm willing to cut it a bit of slack in that regard.

I captured a 60GB+ file at one go, which I think spanned multiple 60GB partitions, and it didn't drop a single frame.

I'll do a long (20+ minute) capture at some point just to make sure it behaves.

So for 720p work from DVCPRO HD, this little array is good for about 1 1/2 hours of uncompressed 720p60 footage, or over 3 1/2 hours of 720p24 footage (if we can make it capture 24p footage directly without using the FRC.....more testing tomorrow on that). DVCPRO HD footage is always 4:2:2 8 bit - the tape format isn't 10 bit, ever. You can get 10 bit off the HD-SDI if you record to HDCAM, but not to DVCPRO HD.

One thing I hadn't realized - I had been reading that the DVCPRO HD format will handle 1080i footage but not 1080p footage. The Varicam ONLY shoots 720p - it doesn't shoot ANY 1080 res footage whatsoever. If you shoot with a Sony HD camera (or other 1080 res source) and record to DVCPRO HD is the only way to get 1080i footage into DVCPRO HD format.

Apple's DVCPRO HD Frame Rate Converter

I captured some uncompressed footage that had been shot at 24p on Varicam (referred to as 24 over 60 - Varicam footage always wants to be treated as 59.94 based, regardless of your shooting framerate). Tried to use the new FRC - under Tools I selected DVCPRO HD Frame Rate Converter. It said no can do - this isn't DVCPRO HD footage. OK, that makes sense. I went back and converted by footage to DVCPRO HD. Cleaner darkened the footage substantially, Compressor did not. But neither file maintained the flags to extract the correct 24p from this 60p footage. Quite likely the flags weren't in the 60p uncompressed footage to begin with. Drat. So no way to use the FRC if you don't have the FireWire deck as far as I can tell so far. Might be a way, I don't know yet.

The DVCPRO HD Codec

Others have probably figured this out already, but for me this was new. I knew that the DVCPRO HD codec only used 960 pixels horizontally and stretched back out on playback via the deck to 1280 horizontal pixels.

What I DIDN'T know is that the DVCPRO HD codec REALLY wants to work in exactly 960 pixels wide. I captured some footage uncompressed, then used Cleaner, QuickTime Player, and Compressor to convert it to DVCPRO HD codec. Against my wishes and intent, it shrank it to 960 pixels wide.UPDATE 6/15/04: THIS IS WRONG. SEE POST AT TOP AND DAY 2 POSTING Cleaner also made the footage substantially darker. Compressor didn't.

This really sets up a Devi's Triangle for posting with 720p footage. If I want to have an offline codec, I can use DVCPRO HD and get realtime effects, but suffer a funky size/aspect (960x720 instead of 1280x720).

If I want to to realtime color correction uncompressed, I can do it with 8 bit footage but not with 10 bit footage. This is moot with the DVCPRO HD codec, since it is always 8 bit not 10 bit. But in 1920x1080 footage from HDCAM, I can't get realtime color correction and effects in 10 bit. The efficient path through post will be difficult to plot, and I imagine might vary depending on how much post work you want to do to your piece, and how much hassle you're willing to put up with, and where the balance in your creative process is - are you going to spend more time editing or more time adding effects/titles/etc.? Whichever is more might tip your workflow in that direction.

Lots more research to be done.

The DeckLink HD displayed the footage, but with pillars (black bars on left and right) during stills, but freaked out when I tried to play back moving video (rather than parking a cursor on the timeline to display one frame still) it freaked out and showed untimed garbage.

Shooting with Varicam for best results

And if you want best results off your Varicam - honest to God 1280 pixels wide (not the subsampled 960 from DVCPRO HD tape format), 10 bits per pixel (DVCPRO HD is 8 bit), and not compressed (DVCPRO HD is only 14 MB/sec for 1080i60, about 5.4 MB/sec for 720p24) then you need to capture straight off the HD-SDI tap on the camera to hard drives LIVE AS YOU SHOOT. After you've shot it and captured to tape, it is forever 8 bit. PERIOD, the end, no getting true 10 bit footage, ever.

But if you do that, you've got a really nice file. We'll be trying that soon, maybe next week sometime. That would require 142 MB/sec. With 20% safety margin, that's 170 MB/sec. That means my little array could capture probably no more than approximately 300 MB of footage before the speed dropped too low. That's about 35 minutes of uncompressed 1280x720 4:2:2 10 bit footage. Shooting a greenscreen commercial or short industrial? That'd be totally adequate for on set capture.

OK, enough for now - got other work to do.

MacCentral has this article about Fujitsu 100 GB notebook drives, and discusses how other manufacturers are coming out with their 100 GB notebook drives as well.

Why should you care? More space on internal laptop drives for mobile editing.

So here's what I'm up to this week:

Sunday night (tonight): hopefully finish up my formal book proposal and submit it

Monday: Lug my G5, FireWire drives, and SATA array over to a place that has their own Varicam and Panasonic DVCPRO HD deck (not the new 1200A unfortunately, the older HD-SDI only model). Get it hooked up, do some capture and playback tests on legitimate HD gear.

Tuesday or Wednesday: Drive up to Dallas and hang with some guys from HDNet to see what they're all about.

Other day Tuesday or Wednesday - day two of testing with Varicam, G5, SATA RAID, and DVCPRO HD deck.

Thursday or Friday: receive a copy of Lumiere to mess with, start writing up results and info from the week

Friday: Do some writeups (hopefully) on what learned with G5 and my dinky little 4 drive SATA RAID stuff, wrap that into Part 2 of my SATA RAID report, including some serious useful space with backup at cost comparisons with SCSI/ATA bridged solutions like Huge & Medea.

Saturday & Sunday - write write write, play with Lumiere, write write write some more

Dat's da plan, I'll have to see how the week actually goes.

So starting sometime between Thursday and Saturday I should have a LOT of interesting things to say about SATA RAIDs (costs and real world performance testing), Lumiere, HDV, etc.

So a lot of good, meaty, real world info next week from this week's beefy research.

If you have a specific question about working with Varicam/DVCPRO HD/HDV/SATA RAID, now is the time to let me know at mike@hdforindies.com

-mike

So I've been writing about lowering the costs of entry in HD, focusing pretty strictly on the post side - cheaper formats and cameras (HDV format, JVC camera), cheaper decks (Panasonic 1200A), cheaper storage (SATA RAID), cheaper monitoring (eCinema or HDLink to 23HD Cinema Display), cheaper editing solutions (G5 and FCP instead of Avids).

But cameras and post gear are only a part of the equation of making an HD movie or short. Some of the folks I've been talking to have had the following problems:

1.) Availability of HD gear - in Austin, Texas, there are only two Sony HD cameras for rent - one from Gear, one from Ian Ellis. Both are 900 models, not 950 models. If more, somebody tell me. Bexel is in Dallas, next closest market, 3 hour drive away.

2.) Availiability of skilled DPs - on more than one project team I've spoken to, the DP didn't have a lot of self confidence in shooting with HD, and wanted a LOT of support - a large grip truck (requiring it's own insurance and a class C driver) and an engineer. Suddenly, their "let's just rent camera/grip/gaffer/lighting stuff and everybody volunteers their time" shoot ballooned upwards in hard costs. Killed the project. Or at least killed HD on the project. When people don't understand something well enough to be comfortable they get scared of it. When clients get scared, they throw money at the part they don't understand hoping it will fix it. Maybe it will, maybe it won't; maybe it'll be what's needed, maybe it's drastic overkill. Often they don't know, they are just trying to buy their way out of insecurity. I'm not saying the DP was right and needed all this stuff, it's just that I've seen it happen with other technological challenges when shooting.

3.) Budget scope creep, as exemplified by the bigger truck, requiring it's own insurance, and a hired truck driver. All unanticipated costs.

4.) Insurance - another local shoot was hindered by the requirement of the city (the property's owner where they were going to shoot) to have at least half a million dollars insurance. Their shoot involved three people sitting, standing, or talking. No cranes, no car chases, no action shots, usually locked off or dolly shots. The city put up apparently ridiculous hurdles to shooting. Small productions get no luv from city administrations unless they have budget and are bringing money into the equation. How bad was it? For a one day shoot, renting a Sony 900 package was going to be $1300. Insurance was $1100. If they got rained out, they'd have to buy insurance AGAIN.

Devil's advocate position on that one (and don't sue me, this is just a thought): What's to stop you from claiming a budget for all your donated time? They WOULD have charged X, wouldn't they? Why can't you factor that into budget? Again, if you get in trouble, not my problem, just an idea.

5.) The permits and hassles involved with getting shooting permits, even for out-of-the-way locations. You'd be amazed how tough it can be. One advantage of the small HDV type cameras is that they don't look "real." You can get away with a lot of "Oh, we're just making a home movie for fun." with those, so long as your crew looks like friends and not, you know, CREW.

Again, don't cry to me (feel free to cry to your momma) if you get ticketed/fined/whatever. Not my fault, not my problem, you did it to yourself. Scoot. Leemie alone, leemie outta it.

OK, just my thoughts...not my actions

-I Am Jack's Pending Lawsuit

I've been talking to a few folks trying to put together budgets for HD projects.

Frequently they find they can't quite cost justify owning their own FCP HD setup instead of doing a traditional online session, even though they would have more creative control, more time to put it together at their own pace, and offer the possibility of editing in HD all the way through their production process.

If it is impractical to buy, look in to renting. I have no hard data on this, but one post facility I've talked to said they were putting in a FCP HD editing suite, with a dual G5, HD-SDI capture card, X-RAID storage, and planned on renting it as a 4 wall solution - the room is yours for six weeks for $6000. You get to capture, edit, whatever, with whomever you wish to bring in. B, if you want me to point you out by name, let me know.

Others facilities may have similar offerings this year.

Other options if there isn't a suitable rental setup, and you can't justify buying your own rig:

1.) See if the editor will buy the system at project end at a discount from it's now used value (the discount is his fee)

2.) IF he's not interested in it, no problem - sell it on the open market. eBay. Whatever.

Just to pick numbers out of a hat, let's say you're looking at purchasing a $15000 setup (this would be a pretty heavy setup if done my way).

How much would an offline/online setup, where you offline at home in SD and online at a post facility in HD cost overall? Let's pretend $10K for the sake of argument. I have no idea how feasible that is, and of course every project is different. I'm just picking numbers out of a hat.

Do you think you could sell this now used equipment at the end of the project for $8-$10K?

If you sold it for $8K, and it cost you $2K more than the offline/online arrangement, do you think you got more than $2K worth of value and experience out of it?

Your call.

Keep in mind, if you're really cash strapped (and who isn't?) put the whole system on the credit card, make minimal payments, then sell it when the final deliverables are done. Now pay off your credit card bill, and pony up the cost difference between new and old with money from your post budget. You've essentially rented your choice of gear now.

Anyone think this is viable? Think it's not? Wanna throw the bullshit flag down on the field for a 10 yard penalty?

Let me know - mike@hdforindies.com

After Heuris dropped their prices the other week, they also posted this page, which compared the features of the two products, and made some comments about the Lumiere products that cast doubt on the validity, efficacy, and quality of the product and the company.

Lumiere just put this page up on their site that addresses all the concerns raised and states their position on their product offerings and future plans.

It also has a nice summary of what it is, what it does, and the problems it solves.

-mike

Lloyd Kaufmann, of Toxic Avenger fame (or infamy), wrote a book called "Make Your Own Damn Movie!" with a lot of very practical tips for indie/guerilla filmmakers, such as keep a copy of the script with no sex or foul language for city permit people, neighborhood groups, etc. to see.

Much of it boils down to legally questionable tactics, amounting to one of my favorite quotes (where'd it come from?):

"It is a far, far better thing to beg forgiveness on Monday, than to ask permission on Friday."

Always liked that one.

Anyway, a fun (if legally questionable) read on how to get your indie film made, start to finish.

-mike

Neat article about pros & cons of digital vs. film. Robert Rodriguez (the digital advocate) and Daniel Pearl (the film advocate) talk about why they like and don't like different aspects of film and digital.

Here's the link.

Very interesting and practical discussion.

-mike

Talked to a Canon rep at the HD Expo over the weekend in Dallas, he said that existing Canon XL-1S cameras in the warehouses and in stock were IT, no more could be factory ordered. Here's why he says:

-Canon is discontinuing the XL-1S
-Canon is replacing it with a new camera
-new camera will shoot DV and HDV
-probably 3 chip

Now, since they are discontinuing rather than supplementing to their product line, this to me says that the camera should be in the same price range as the current XL-1S, which streets for about $3000 these days after rebates and whatnot.

To me, this says that it will HAVE to be around the same price point. If they were going to have it cost more than $4000, they would keep the XL-1S at the $3000 (or lower) price point, so there would be valid market differentiation between the $3000 and $4500 (or $5000) camera. But I betcha it comes in around $3500, give or take a few hundred dollars.

So a 3 chip, sub-$4000 camera, with replaceable lenses? HELL yeah! Indies, start your pre-roll....

...and with Lumiere and DVCPRO HD as a working codec, and using uncompressed for 720p30 final output, you've got a helluva low cost, semi-decent quality setup...

-I Am Jack's Scarcely Contained Glee

While emailing with reader Christopher Barry today, I learned two things:

1.) There is a little hardware box to control frame rate ramping for the Panasonic Varicam. Follow this link for contact info, follow this link for the PDF file with the specs.

2.) After all my Tyler Durden references, Christopher suggested that any reader who wishes anonymity shall be known as...

"No one in Project HD4NDs has a name. Except in post production. In post production, his name is Robert Paulsen. His name is Robert Paulsen...[everybody now!] his name is Robert Paulsen...his name is Robert Paulsen...his name...

: )

-I am Jack's Smirking Revenge

MacCentral has this article on HD-DVD. The article discusses the two competing formats, support for Windows Media 9 as a requirement, and other interesting stuff.

Why am I posting a link to this? Because this is what you'll be watching your HD movies on in a few years.

-mike

UPDATE THURSDAY 12:30PM CST Bill Weisman from 24p Entertainment has posted this more detailed article.

Hollywood Reporter has this article on the new Panavision Genesis digital cinema camera. The article has a picture of the unit, too, with the Sony deck attached on top.

Highlights:

-same weight (with capture deck) as film camera with 400 foot load.
-shoots 4:4:4 10 bit log
-uses standard Panavision lenses
-35mm sized imaging chip (hooray!) which allows for "normal" 35mm style depth of field and dynamic range
-dockable Sony SRW-1 HDCAM SR deck can mount on it
-has 4:4:4 dual link HD-SDI outputs as well as 4:2:2 HD-SDI monitoring output
-up to 50 fps
-available in November

Mike's Comments: Clearly it's shooting HD, whether is is shooting SMPTE 372M (which is 2048x1080 pixels) is unknown. It is shooting 10 bit log, which is good, that's a Cineon file (or at least compatible with). No price or rental rate given, but this is a legitimate digital cinema camera, rather than a modified electronic news gathering camera. That is VERY good.

How does it compare to the SMPTE DCI (Digital Cinema Initiative) recommendations?

It's better than most of what is on the market, but only has about a 1/4 of the resolution of the DCI spec (which is in progress), and is 2 bits per channel shy of DCI's recommendation...which means the 1/4 of the # of steps in color compared to what the DCI spec recommends.

OK, we're off the Mac thing, moving on, moving on.

Letting it go, letting it go....

InfoWorld has an article with Apple's statements on 3.0 GHz G5s and G5 laptops.

Tom Boger, Apple's Director of Power Mac Product Marketing said "not to expect a G5 anytime soon in a PowerBook -- certainly not before the end of the year."

As for the promised 3.0 GHz G5s, "no we are not getting to 3GHz anytime soon."

Mike's Comments: I'll read that as not in this calendar year. An announcement next January? Believable. Shipping by March? Probable. Gee, that's almost a year behind schedule. Surprising? Not really, given the G4 roadmap. Me grouchy and surly about it? Not a surprise.

Want toys.

Want want WANT!

OK, I need a beer.

-disgruntledmikey

Think Secret has a report on the supposedly upcoming new Apple LCD monitors.

Purported lineup:

20" model at 1650x1080 (could do pixel for pixel 720p footage)
23" model at 1920x1200 (could do pixel for pixel 1080p footage)
30" model at 2560x1600 (could do pixel for pixel 2K, SMPTE 372M, lots of stuff! : )

Sleek aluminum framing to complement current Power Mac lineup.

DVI, not ADC, in part because the biggie needs 150 watts of power to drive that humongous screen. So PC users are invited to the party, no ADC-DVI adaptor required.

I'm gonna have to get me one of those.

Rumor is one or more of these might be HD ready as well - wouldn't THAT be fantastic?

Mike's Comments: All the rumor boards have been buzzing about a 30" monitor for well over a year now. Since Apple shot their abortive CPU upgrade wad already with the pre-announcement of a 2.5 GHz machine not until "July" (see post below), this could be one of their big news items at WWDC in a couple of weeks, although Think Secret things they are launching them sooner than that.

Pricing said to be lower than current stuff.

-the 20" is presently $1300, I could see that dropping as low as $999
-the 23" down to $1299 to $1499 and
-the 30"? Probably in the $2000 to $2500 range, no way more than $2999.

I think what they describe is inevitable (right, Mr. Anderson?), it's just a matter of when the components are cheap enough to do it at a given price point.

This is EXACTLY why the $500 rebate on the Cinema23HD ends on the 26th...two days before the WWDC starts.

Updated 11pm Wednesday: See the post directly above this one about new LCD screens. See why I say these things?

OK, this is just me pointing out a deal at the moment, NOT advocating it as the best solution. For those of you on a tight budget, consider this:

Apple's (now) OLDER G5s, the Dual 2.0 and Dual 1.8 models, should be available for less if you beat up the dealers enough. The new models are better and cost less, so except for the "Yeah well this is what I have NOW in stock" factor, you might make some progress. I wouldn't pay more than $2100 for an older 1.8 dual, no more than $2250 for a older dual 2.0. Try for less. But that's just me. Consider that the new Dual 2.0 G5 with better SuperDrive is $2500 (are the graphics cards better? More/less RAM? Have to check).

But Apple is STILL doing their Cinema 23HD $500 off thing - so buy either an older dual 1.8, an older dual 2.0, or a new dual 2.0, and then save $500 on the 23HD. THIS DEAL IS GOOD ONLY THROUGH JUNE 26th which is right before WWDC, when new LCD panels that might be better/cheaper are expected. Caveat Emptor.

DeckLink HD cards are $1000 through the end of the month from BlackMagic Design. This is the same card they've been shipping for awhile, it works nicely, it's only limitations that I know of are these: it is only single link HD-SDI, it tops out at 10 bit not 12 bit, and it has no analog monitoring, and HD-SDI to analog component adaptors are around $2000. BUT...if you are going to monitor with a $1300 HDLink and a 23HD, it's a helluva deal. But keep in mind, new stuff like the DeckLink HD Pro and whatever else might happen.

If your machine comes with a Barracuda 160 GB SATA drive, you can buy 3 more for $360 from zipzoomfly.com (shipping incl).

La Cie's standard price for an Electron IV Blue giant CRT computer monitor is $700. Helluva screen for timelines, bins, etc.

So let's put all that together with FCP HD:

(older) Dual 1.8 GHz G5: $2100
Cinema Display 23HD: $1500 (after mail-in rebate, only until June 26)
DeckLink HD card: $1000 (only until June 30th)
SATA array - Seritek card, SoftRAID, 2 PPA Inc. cases, 3 more drives, FW drive to boot from: $850ish (rounding here a bit)
2x512 MB RAM from crucial.com: $200
Final Cut Pro HD: $1000

This would give you a $6650 editing station, ready to plug into a deck and go with over an hour of storage for uncompressed HD (a few caveats in there, but generally speaking)

Woops, you can't do any monitoring with that beyond the full screen mode in FCP, and I wouldn't trust that at ALL for color work.

So to monitor:

Add $1300 for an HDLink to run your HD-SDI to the 23HD, brings the price to $7950.

But hey, you need a computer monitor so use that Electron Blue IV from La Cie for $700, brings us to $8650.

So still, that's well below $10K for an HD editing station.

Bump it up to a new 2.0 GHz, you're still around $9K.

Want a second monitor for bins etc.? Add about $250.

Not bad!

Want a bigger array? Add another Seritek and 2 more drives for another $300ish for a total of just under 900 GB formatted capacity.

Want more? Then use $192 Hitachi 7K250 (250GB) drives instead, or the soon to be released $420 7K400 (400 GB) drives.

You see where I'm going with this.

Disclaimer: it's quite likely there will be cool new developments at WWDC that make this not wise, or that the DeckLink HD card may not do some things you want it to (maybe DVCPRO HD realtime conversion), stuff like that. This monitoring solution is unproven for doing color critical work, they don't have their calibration utility done yet last I heard. BUT, I just wanted to point out that deals are out there.

Updated noon CST - see bottom for HD editing recommendations

Today Apple announced new G5 computers. Liquid cooled, even! Here's the press release.

Here's the skinny:

$1999 Power Mac G5

Dual 1.8 GHz 64-bit PowerPC G5;
256MB 400 MHz 128-bit DDR SDRAM (4GB maximum);
80GB Serial ATA 7200 rpm hard drive;
NVIDIA GeForceFX 5200 Ultra with 64MB DDR SDRAM;
3 PCI slots (64-bit, 33 MHz)

this is similar to the prior dual 1.8, EXCEPT that this model no longer has ANY PCI-X slots

$2499 Power Mac G5

Dual 2.0 GHz 64-bit PowerPC G5;
512MB 400 MHz 128-bit DDR SDRAM (8GB maximum);
160GB Serial ATA 7200 rpm hard drive;
NVIDIA GeForceFX 5200 Ultra with 64MB DDR SDRAM;
3 PCI-X slots (one 64-bit 133 MHz, two 64-bit 100 MHz)

This sounds pretty much functionally identical to the current Dual 2.0 units...but $500 less.

$2999 Power Mac G5 (due in July)

Dual 2.5 GHz 64-bit PowerPC G5;
512MB 400 MHz 128-bit DDR SDRAM (8GB maximum);
160GB Serial ATA 7200 rpm hard drive;
ATI RADEON 9600 XT with 128MB DDR SDRAM;
3 PCI-X slots (one 64-bit 133 MHz, two 64-bit 100 MHz)

This, obviously, is the interesting one....if Apple runs to form, a few units will eek out in towards the end of the month, and volume availability will be sometime in the August-early October timeframe. That's what happened last year...

It's dissapointing that Steve Jobs' twice promoted "3.0 GHz by next summer" isn't happening. If they ship the 2.5 in volume by September 21st (official end of summer) they'll have missed their speed increase goals by half - instead of a 50% speed increase in a year, it'll be a 25% speed increase in a year. Dissapointing, but better than what we've got now.

I note that Apple rolled these out prior to WWDC, with little fanfare. I take this to mean that they wanted to downplay the "missed goal" PR aspect.

The Good News

Apple is STILL running their $500 off 23HD deal - so you can STILL get a 23HD and a 2.0 GHz G5, only now you're paying just $4000 for it instead of $4500 like last week, or $5000 as of prior to March. But I'm betting there will be new displays announced at WWDC, so caveat emptor...

These machines are (supposedly) shipping immediately at these prices. Good news. And hey, there is a new Apple Store opening in Austin (where I live) THIS WEEKEND so I'll go out there and see the new toys. Man, that's gonna be a madhouse - at one point, the University of Texas at Austin was the single largest Mac install site in the world.

The "Not Feeling So Fresh" News

....but that may be pointless...since the dual 1.8 is a minor modification of the current 1.6 chassis, and the dual 2.0 seems to be exactly what we've already had, with the useful addition of an 8X SuperDrive and $500 less (ok that's cool).

ONLY the dual 2.5, not available until "July" uses the liquid cooling technology, and only that model APPEARS to have a different motherboard.

So I'm not expecting to see any changes in in the present FireWire 800 or SATA setup This is dissapointing on a couple of fronts - when the dual 2.5 ships, it MIGHT fix the USB 2.0 (runs attached drives FAR below wire speed), FireWire 800 (G4s can write FireWire 800 faster than G5s by a factor of TWO), and NONE of these units increase the internal capacity of the G5s to allow for more internal SATA drives. So that huge long write-up I did yesterday on SATA arrays still holds true for ALL of these new machines.

Some come August sometime, when I can lay hands on one of the new dual 2.5s, I'll check it out then.

My guess is that IF the dual 2.5 has a different motherboard, then the possibility of revised USB 2.0 and FireWire 800 can be addressed for all future machines. And IF it is a new motherboard, that offers the possibility that next January Apple might have a new enclosure available for that new motherboard at MacWorld Expo, and that MIGHT allow for more internal drives. OK, end of speculation. But that is Apple's usual rev/mod policy - announce in summer, ship in summer/fall, no possible tweaks until announce in January, ship in Feb/March (or just announce & ship in March, which was the original plan for these G5s).

But pardon my cynical tone - this is good news - faster machines at lower prices, faster SuperDrives, and a fastest Mac yet available come the fall.

Mike's Recommendations for HD Editors

1.) If you need a Mac NOW, get the NEW G5 Dual 2.0 GHz. It costs less, has a faster SuperDrive, and should work great (assuming no new problems, which is unlikely). On a super tight budget? Get the OLD Dual 2.0 GHz, but I wouldn't pay more than $2200 for it.

2.) DO NOT GET THE NEW DUAL 1.8 if you want to edit HD. Here's why: unlike the OLD Dual 1.8, this new unit does not have ANY PCI-X slots, so adding a Kona2, Aurora Pipe HD, or DeckLink HD or HD Pro is out of the question.

3.) If you're starving (budgetarily speaking), the OLD Dual 1.8 has PCI-X slots, and can work as an HD station. It's certainly not my top pick, and I'd want to double check the specs and hardware requirements on the new HD capture cards, but these should be available for around $2K or less. Get the dual 2.0 if you can, though. I would NOT pay more than $2K for one of these. But that and a $1000 DeckLink HD card (that price good through end of month), and the 23HD rebate, you've got a G5, a 23HD Cinema Display, and an HD capture card for $4500. Add $1000ish for a small SATA array and $1000 Final Cut Pro HD, you're ready to plug into a deck and edit for only $6500. Wow. That's enough room to cut a 44 minute TV episode (OK, barely, using offline/online).

OH YEAH. APPLE ALSO ROLLED OUT A NEW 20" iMac WITH 1.25 GHZ G4 (NOT G5) AND SUPERDRIVE

Hello all, and welcome to the first HD For Indies' HD Labs Report.

Update Thursday June 10th 7pm CST Turns out some of my HD throughput numbers were wrong - I pulled them from the BlackMagic Design website, and found out they were optimistically low. I'm putting the article back up, but keep in mind I'm going to be adjusting these numbers. 10 bit 1080i29.97 is more like 160 MB/sec, not 136.72 MB/sec as I said (based on BlackMagic's website). All my other numbers are likely to be off by a similar percentage. I'll tweak all the relevant data later, and expect a report early next week on the exact and precise usefulness of SATA storage for HD, including backup costs, usable space for a given format, etc. Huge has gotten back in touch with me in a prompt and friendly manner, I'm hoping to hear back from Medea before I publish my next report.

End Update

This article is going to give an overview of SATA RAID solutions for G5s, and discuss how to build low cost ($1 to $2 per gigabyte), high speed (up to 400 MB/sec), high capacity (this minute up to 1.5 TB, up to 2.4 TB in a few weeks, hopefully up to 7.2 terabytes later this summer) using off the shelf components and a little bit of cleverness.

I've been meaning to write this article for awhile, and with the impending release of the new G5s with their modified cases that might be able to handle additional SATA drives, I thought it was high time I got this out there.

Background: What is SATA?

SATA is short for Serial ATA. ATA has been the low cost standard for consumer hard drives for several years now. Serial ATA is an evolutionary change in the format that allows for higher performance, simpler cabling, and changed scalability.

Regular ATA has evolved through a variety of flavors, most recently from ATA-6 to ATA-100, which referred to the maximum # of megabytes per second the bus could transfer. Serial ATA allows for up to 150 MB/sec across a single SATA bus.

The cabling has changed also - if you ever had to put a new drive in your ATA based computer, you might have had to adjust jumper settings (itty bitty little bits of plastic that you had to add or remove with tweezer like precision). More recently, ATA interfaces had Cable Select which made it easier. Either way, you had a thick, awkward ribbon cable to deal with. Serial ATA is different - it is a serial, not parallel connection, so fewer wires are required. It is a small (less than one inch) wide connector that easily presses on or pulls out (mixed benefit to that, more on it later).

The other significant change to the way it works as compared to regular ATA (also now dubbed P-ATA, for parallel ATA) is that unlike PATA, which allowed for a convenient 2 drives per cable, SATA is strictly one drive per cable. This means you have to have a SATA cable for every drive, you can't daisy chain one off an existing drive.

The Current G5s and How SATA Works There

Why do we care? Because Apple switched over to SATA drives with the introduction of the first G5 models last year.


The current G5s (and this may change very soon, maybe even this week, but certainly by WWDC on June 28th) have room for only 2 SATA drives in their standard configuration. They ship with one drive, and you can upgrade (overpriced!) when you order online to include a second identical drive. By using Apple's Disk Utility, you can stripe these two together (after backing up all your data, striping wipes out the contents of any drive in the array) to form what appears to be a single volume to the computer, but is actually two drives both working full speed to make one twice as big, (nearly) twice as fast volume. And if using Apple's Disk Utility, you can boot from it. Two drives, however, is not fast enough to do uncompressed HD work...although it is plenty fast enough to do standard definition work, all the way to the tail of the drive.

What is this "tail of drive" you speak of, white man?

Hard drives are kind of like record players - there is a disk and a head, akin to the record and the needle. When the record is playing from the edge, the needle is covering a lot of ground per second - it's linear speed is pretty fast. When it gets close to the hub, the record is still spinning at the same speed, but the linear speed of the record under the needle is a lot slower. Hard drive speed comes in part from how much record moves under the needle per second - so at the "end" of the drive (closer to the spindle hub), data transfer rates are much lower than at the beginning (edge) of the drive. So when you first stripe up a 2 drive Barracuda array, using the drive the G5 came with and a second one you bought online, you'd be ecsctatic (if you're a geek like me) and relish the nearly 100 MB/sec write speeds. However, as the disk array fills up with data, when the drive was just about full you'd only be getting about 55 or 60 MB/sec. If this were below some crucial threshold you needed, you'd be in trouble. If you're working with "full on" HD footage, 1920x1080 at about 60 fields per second, 10 bit uncompressed, that is just over 135 MB/sec...well short of what this two drive array is capable of. Even 1280x720 8 bit 60 frame per second video is about 110 megabytes per second...still beyond the 2 drive array limitations we have here.

So how to do better than that?

There are some multi-hundred dollar options to get additional drives in the case, such as the Wiebetech G5 Jam, and another that allows for a sturdier mounting of 3 drives, but both of these solutions merely mount the drives and don't give you any additional SATA connections to plug the drives into (I'm cheating a bit here, they do offer solutions, read on). I'm not a big proponent of these solutions, since they are expensive for what they do (in my humble opinion) and increase the thermal load on your G5, making the fans run higher and louder more often, as well as offering very limited scalability - at best 3 more drives. And I recall that one of them specifically said NOT to use 10,000 rpm drives with their solution. So how do you get more SATA connectors to move the data from the drives to the computer?

SATA PCI and PCI-X cards

For many months, a company called FirmTek has been shipping the Seritek 1S2 card. It is low cost (under $70), plug-n-play simple, PCI card, and it has two SATA connectors on it. Sounds great, right? Well...sorta.

The connectors are internal - that is, the SATA connections are on the end of the card inside the computer, not on the backplate, which would let you plug something in without opening the G5 case.

This is fine if you're using one of the (expensive) internal solutions, but what if you want to have the cables outside the case? Huh? Why would you do that? (Hang on, I'll answer in a minute). For a one card solution, if you have an empty PCI slot, you can simply run the cables out the empty slot and there you have it. Somebody bug me about it and I'll post pictures of mine.

For a two card solution, it gets trickier. If you're doing this for HD, you already have an HD card. Due to the G5s tricky PCI-X implementation, it gets funky. Your G5 has 1 AGP slot (slot 1) which is for your AGP graphics card only. Slots 2 & 3 share a 100 MHz PCI-X bus. If you put a PCI card in either slot 2 or 3, it slows BOTH slots down to PCI speed. If you have 2 PCI-X cards in slots 2 & 3, you are OK so long as you don't saturate (too much data) the bus. Slot 4 is it's own high speed 133 MHz bus. So that is where your PCI-X capture card (Aurora Pipe HD, Decklink HD, AJA Kona2) card has to go.

So all that technobabble means all your slots are full, and there now is NO empty PCI slot cover to remove and run the cables through if you want 4 new SATA connections with the Seritek 1S2 card. So what to do? If you're feeling brave and handy with a dremel tool, you unscrew the PCI slot cover from the Seritek card and drill/dremel yourself some holes to run your 2 cables through. Be sure and not leave any sharp edges to chafe the cables, and preferably line the holes with tape or somesuch.

Not comfortable with that solution? Then wait a bit. I am talking to two vendors who are introducing cards with 4 or 8 ports on them into the Mac market. Both are PCI-X, so what-card-goes-where is made a bit simpler. One card will have external ports, the other hasn't decided yet.

So now you have more SATA connectors outside the box. So what? I now have raw SATA drives...where do they go? One solution is the PPA, Inc. external SATA cases you can pick up at Fry's for $50 each. These are cheap cases that run hot with rickety little fans. I'm using them now, but I'm not a big fan of them, since they lack a, um, big fan.

Granite Digital offers solutions like this or this that could work as well.

Addonics also offers things like this that are close but not total solutions (no power supply!). Dig around on the web and find other SATA case solutions too. They are out there.

So now you have 2 or 4 more SATA drives hooked up. You also still have the drive your G5 came with if you bought it stock - a 160 GB Barracuda drive. You can buy a second one of these aftermarket for about $120 from vendors like zipzoomfly.com (formerly googlegear).

I've had some difficulty using Apple's Disk Utility to format SOME kinds of SATA arrays. Sometimes the read speed is half of the write speed, sometimes I can't even set up the array. Last time I tried to set up a 4 drive Barracuda 160 array (using the drive that came with my G5 and 3 more just like it) I couldn't successfully set up the array. That may have been due to other things I had done, I don't know - you might be able to get it to work.

But another issue with Apple's Disk Utility is that while it will allow you to partition a SINGLE drive, it will NOT allow you to partition a disk array.

Crap. So what now?

SoftRAID.

SoftRAID is a $100 third party utility that will let you set up and partition drives on SCSI, SATA, ATA, or FireWire. You also have some options in how you set up the drive, such as file system (HFS+, UFS, or Case Sensitive HFS+), and block sizing options. There is a specific preset to use to optimize the array's block size to work with larger chunks of data at a time. Bad for database efficiency, good for huge gobs of digital video files.

By using SoftRAID, you can format and partition your array.

I have used SoftRAID this spring to set up SCSI arrays with an ATTO UL4S card, as well as a variety of SATA arrays using Seritek 1S2 cards.

The only hassle with SoftRAID, and it's a biggie, is that you can't boot up a computer from a SoftRAID array

Bummer.

So instead, I've booted those machines from FireWire 400 or FireWire 800 drives.

I've seen some issues, however, with doing this. Sometimes, ESPECIALLY with SCSI arrays (may be contingent on the ATTO card, dunno), the FireWire bootup drive isn't recognized, and the computer just sits there and blinks dumbly at you with it's question mark drive icon. With SATA arrays, it happens every once in a while.

To fix it, just power everything down and try again and it'll work. For mission critical, middle-of-huge-rock-concert applications, This Is Bad. For your home studio, it's an annoyance, for sure, but it isn't lethal.

So finally, you can now have your 4 or 6 drive SATA array. The 2 drives inside the G5, plus the 4 more identical ones in external cases from Addonics, Granite Digital, PPA Inc., or whomever.

HEY BOY! HOW FAST DO SHE RUN!

OK, finally I'm getting into the geeky bits.

Highlights:

I set up a 6 drive array using Western Digital 74 GB Raptor 10K drives. These drives are SMALL - only 74 GB each, but spin at 10,000 rpms. Single drives are good for about 72 mb/sec at the head (beginning) and a bit over 50 MB/sec at the tail (end). Striped into a 6 drive array of around 415 GB, and using BlackMagic's Disk Speed Test (ships with DeckLink cards, won't run without one present), I was able to get a measured 390 to 400 MB/sec read speeds on an otherwise empty disk array. That would be enough in THEORY to do TWO streams of 1920x1080, 10 bit per channel (30 bit color), 24 frame per second, 4:4:4 Dual Link HD-SDI video. That would also be enough to do a 2K Digital Intermediate process on scanned film and play it back in realtime, 4:4:4 uncompressed RGB for a movie.

But that isn't a realistic test - this only measured performance at the beginning of the drive, where it's fastest. What about at the end, when your disk array is almost full?

Again using the same utility, I measured the last of 10 partitions, so this was measuring around the 375 GB mark of a 415 GB array - and it was still doing about 310 MB/sec for reads and writes.

So how solid are these numbers? I ran the test 5-7 times for each drive or partition, and tossed the high and low numbers to get rid of anomalies. I also emailed my tech connection at BlackMagic Design, and here's what he said:

"The test measures bandwidth while writing large chunks of data, emulating video data transfer. The values you get are real 'sustained' values, and certainly not burst values."

So I take it it's basically a solid testing methodology, since it takes about half a minute to gather it's data, but a bit of fudge factor/safety room would be appropriate.

OK, so for this particular drive setup, we now know how fast it'll go at the beginning and end. What about in the middle? How fast does it slow down? I diced the array into 10 equal partitions (41.5 GB), and here's what I got (read/write averages):

Slice 1: 396/385
Slice 2: 383/387
Slice 3: 380/385
Slice 4: 380/384
Slice 5: 385/384
Slice 6: 382/385
Slice 7: 358/370
Slice 8: 350/351
Slice 9: 334/340
Slice 10: 310/314

So you can see it holds a very high percentage of it's maximum speed for a long time.

I noted that once striped into an array, the drives don't hit their maximum speed possible when in a single drive configuration. In my research, I found that a six drive SATA array still performed at about 90% of 6 times the performance of a single drive...in other words, the array was about 90% speed efficient. Not bad for overhead, promises this technology could scale pretty well.

I also tried the other options SoftRAID allows, and found that disabling Journalling offers a very slight speed improvement, on the order of a few percent...I wouldn't recommend it - tiny performance increase for a large decrease in safety/stability of your system. I also tried the other block size allocations, and as one would guess, digital video worked best. But I tried'em.

But this isn't the most cost effective solution, unless you needed EXTREME performance for a small amount of footage.

With the above 6 Raptor setup, it would cost, at minimum, about $1700 to get the drives, external cases, and SoftRAID. Plus an external FireWire drive to boot from.

And it only would give you a a bit over an hour's worth of 1080p24 10 bit footage. Not even an hour's worth of 1080i footage. But you could fill the RAID and not worry about the array getting too slow towards the end.

So let's find something a little more cost effective, shall we?

Let's start simple.

Let's say you have a G5 with a 160 GB standard drive.

Let's go buy 3 more of those, and SoftRAID, and a Seritek 1S2 card, and 2 PPA,Inc. cases. Total cost outlay: $360 for the drives from zipzoomfly.com (I just checked) including free second day shipping. $100 for SoftRAID, purchaseable and downloadable online. $100 for 2 external SATA cases. $200 for a FireWire boot drive that you can use Carbon Copy Cloner to replicate your present boot drive over onto.

So that's roughly $830. And you have a 596GB array. I'm saving this text file to this exact same setup right now. This array does 200 MB/sec reads and 220 MB/sec writes at the head, and 115 MB/sec reads and writes at the tail.

So this would be fast enough for 1920x1080i, 4:2:2, 10 bit, 29.97 interlaced footage at the head of the disk, since that only requires about 135 MB/sec. And it would hold a total of about an hour and ten minutes of that kind of footage...except that it isn't fast enough across the entire span of the array. So how do you deal with this?

Partitioning. Here's the slice test results for this 4 drive array, 60 GB partitions except the last, read/write averages:

Slice 1: 209/222
Slice 2: 197/213
Slice 3: 200/205
Slice 4: 189/200
Slice 5: 185/192
Slice 6: 176/185
Slice 7: 182/178
Slice 8: 161/162
Slice 9: 146/148
Slice 10: 131/133
Slice 11: (last 10 GB): 115/115

If we factor in a 10% safety factor, that still implies that our inexpensive little sub-$1000 array can safely capture & play back 1080i29.97 10bit until roughly a bit over halfway into the array. Hmm, that's not great...what else can we do?

here's what's possible with this drive setup for various footage:

720p24 8 bit: the entire array, presuming you're capturing 24 not 60fps directly
720p60 8 bit: all the way through slice 8, so roughly 90% of the array can do this
720p60 10 bit: just one partition less, so roughly 80% of the array
1080p24 8 bit: the array could handle it until almost (90-95%) full
1080p24 10 bit: the array could handle it until about 90% full
1080i29.97 10 bit: slice 7 passed, so about 60% of the disk

Obviously there are other size/framerate/bit depth options, but this article is getting long enough as is.

So how do you deal with these limitations?

If you're working on a single project (such as a feature) that you've purchased this getup for, you will know your format and bit depth and frame rate, so you can partition appropriately. What to do with the rest of the space? Use it for non-online throughput critical stuff - this is a great place to capture offline res footage, be it DV, DVCPRO HD, whatever.

If you are a facility and don't know what's walking in the door next, you could either partition the array into key slices (works for everything, then the next partition is "everything but 1080i29.97", then "only 720p work", etc.) and use your Final Cut Pro preferences to use (or not use) the appropriate scratch disks when capturing, that way you could trust the "how much time/space do I have left" indications in Final Cut Pro when capturing.

"OK, Mike" you say, "What if I want some MORE space?"

Then you have options:

1.) On top of the above described setup, get 2 more Barracuda 160 GB drives, another Seritek 1S2 card, and 2 more drive cases, pushing your capacity up to about 890 GB of space. but that isn't the most cost effective approach...

2.) Buy some IBM/Hitachi 7K250 drives. Why am I being so specific? Because these are the fastest drives presently available on the market at a reasonable size and a reasonable price. Zipzoomfly.com has'em in stock, with free 2nd day shipping, for $192 as of Tuesday 6/7/04. They are good for about 60 MB/sec at the head, and 30 MB/sec at the tail. A good rule of thumb seems to be that most drives hold about 80% of their speed until about the 70% mark, then they drop to about 1/2 speed by the tail. Using that as a guideline, you could create a 1.5 TB array for around $1600 for the drives, cheapie PPA, Inc cases, SoftRAID, and two Seritek cards. At this point, seriously consider other, more durable/reliable/stable casing options besides PPA, Inc. I use'em for lab work, bu wouldn't be thrilled in a production environment with those long term. Note that this is about $1/GB - Huge & Medea cost around $5/GB at this price point...but offer more (more defined later).

You'd have a system that could do ANY 4:2:2 HD signal for the ENTIRETY of the array.

3.) Also consider the upcoming (shipping in days/weeks, not months) IBM/Hitachi 7K400 drives - expected to cost around $400, but with the same performance specs, and 400 GB capacity. With 6 drives, 4 PPA Inc. cases, 2 Seritek 1S2 cards, a $200 FireWire drive, and SoftRAID, you're spending a hair over $3000 for a 2.4 TB array. Price that against ANYBODY and be happy. You're paying about a buck and quarter per gigabyte for this solution.

So what's the downside to all this?

For starters, it's messy.

SATA cables don't stay in very well on the PPA Inc. cases, to the point that I use tape to make sure I don't bump them and disconnect them under my desk by my knees. Plus, you're hand tooling on some metal parts if you want a 6 drive array (for the moment, not for long). Not usually recommended.

There's a lot of cabling, involved, too - 4 cables coming out the back of your Mac is inconvenient.

It's ungainly - a total of 5 external cases now, including your FireWire drive that you HAVE to boot off of if you're partitioning or want better speed (substantially!) than Apple's Disk Utility offers. (Plus, you never want to capture to the drive you boot off of, anyway). Another FireWire drive for audio capture wouldn't be a bad idea, also.

And the biggest drawback - it's insecure. This is RAID Level Zero. If ANY one drive in the array goes bad, poof! There goes all your data. Solutions from Medea, Huge, Apple, etc. are usually RAID 3 or RAID 5, both of which give you some security in the event of a drive failure. If one drive fails, NONE of your data is lost, you just might have some downtime while the array rebuilds it's data.

So what to do about this?

FireWire backup

If you don't have to have moment by moment, perfectly syncronized backups, buy FireWire drives to back up onto.

The only time you're creating tremendous amounts of unique, difficult to replace data is two phases of production: when you capture your footage (usually at the beginning of your process), when you render effects (but this is all re-createable in event of drive failure), and when you capture your final high res footage during online. Even then, at all three phases, NONE of this data isn't replaceable, it's just INCONVENIENT to replace it - the biggest factor is whether you do or don't have a deck sitting there handy, and how long would it take you to redigitize. (I've written at length about these issues previously on the blog).

So long as all your UNIQUE data lives on another drive (like your boot drive) that includes your Final Cut Pro project, your bins, your Photoshop artwork, your After Effects files, etc., you're OK in the event of array failure. Backup all these files I just mentioned to CD (or DVD) if you have to every night, and you're out a day's work at worst.

I'd highly recommend something like the La Cie Big Disk to back up your media data onto - one terabyte, single volume. Set it copying overnight, and walk away. There's plenty of inexpensive syncronization software out there too.

What are the odds you're going to capture MORE than one terabyte in a day? Slim. It takes about 8 1/2 hours to back up 1 TB at an uninterrupted 35 MB/sec, a very reasonable expectation of a FireWire 800 drive. So any given day's work won't take more than overnight to back up.

So maybe you need two of these drives, maybe three for your 2.4 TB array. No big deal - they are $1200 for a 1TB system, and can be daisy chained together.

So total cost of a storage system that will do 6 hours of uncompressed 1080p24 10bit footage, and have separate volume backups of it, and give you an additional 600 GB of offline storage (perfect for DV, DVCPRO HD, PhotoJPEG, or other working codec): $6000 for a total of 5.4 TB of data.

OK, that's enough for today.

This summer, I'm expecting those 4 port and 8 port SATA cards to ship, which should allow for a total of 10 or even 18 total SATA drives if it all works out. That would allow for up to 7.2 terabytes of online storage. If you need more than that, you're probably doing something wrong.

Comments, questions, rants, desire to throw down the bullshit flag on the field? Email me at mike@hdforindies.

End of HD Labs Report # 1.

-mike

OK, in a hurry here working on the SATA RAID write-up -

Broadcast Newsroom.com has an article on Thomson's new Super SloMo HD camera (the first of it's kind), an HD switcher, and support for Final Cut Pro in their NewsEdit product line (this last was announced at NAB)

Digital Producer.com's first look at AJA Kona2 card This is an exciting card, does uncompressed 12-bit 4:4:4 RGB HD/SD capture and output for Final Cut Pro. This is one of the two Next Big Thing cards on the market this summer. (Other is DeckLink HD Pro).

Broadcast Newsroom.com also has this article (part 3A of ongoing) on display technologies - CRT, LCD, Plasma, & DLP. Haven't read the series yet, but definitely EXACTLY the kind of issue to be paying attention to, for studio usage, for client presentation, and for at home viewing purposes.

-mike

So I've been telling myself since I started this thing that I wanted to publish more than just tradeshow reports, refurbished news with commentary, and my own editorials on the future comings and goings of HD for the independent (i.e. budget constrained) producing crowd.

With that in mind, I am pleased to announce HD For Indies' HD Labs Reports, which will be an ongoing series of articles of hard data on actual production gear, documenting my explorations into new frontiers of high performance, low cost, production viable equipment, configurations, workflows, etc.

I'll be kicking off this new endeavor with an article on Serial ATA arrays, discussing their viability for surprisingly high performance, low cost HD uncompressed online storage solutions.

I'll be posting that article up later today.

Future topics will be HDV editing solutions on the Mac, tests and reviews of the new LCD based HD monitoring solutions, non-standard but viable workflows for offlining HD, etc.

Boxx Technologies (here in Austin) has announced their HD [pro] RT system, which can do up to 3 streams of uncompressed 1080p24 HD and color correct them in realtime. WOW. 8 and 10-bit uncompressed 4:2:2 over HD-SDI, HDV i/o over FireWire, 1080i, 1080p, 1080PsF, 720p, HDV.

It uses Cineform HD to replace Premiere Pro's processing engine, it can do 6 channels of AES/EBU audio, is based on Premiere Pro, comes with Encore DVD, and can be configured with up to 2.4 terabytes of RAID 5 storage (rackmount of deskside). It uses SATA RAID storage, which I'll have more to say about shortly.

Starts at $20K, reasonably configured with 1.4 TB array etc. is just over $21K.

In terms of raw processing bang for the buck, this looks like the best thing I see out there.

In terms of editing workflow, integration with over finishing stations, etc., it's decent if not downright solid.

Order before the end of the month and get a $3000 discount.

Not bad...

-mike

So I've been doing a lot of thinking about HD over the past year, and two of the biggest cost barriers to entry have been storage and monitoring. You can look at the cost of the computer, the capture card solution, the editing software, and get excited about the low cost possibilities, but once you start factoring in the cost of storage and monitoring the smile fades.

However, that's changing. As I'll soon publish some of my test results on Serial ATA (SATA) arrays, you'll see that high speed, low cost storage can be achieved.

Today news.com has an article about how Intel is offering manufacturers low cost components and designs to assemble high speed, low cost storage arrays. This will enable so called "white box" manufacturers to do for storage what white box makers have done for workstations and servers - generic, low cost, high performance units.

Intel is investing in SATA and Serial Attached SCSI infrastructures, looking to generate single chip solutions rather than chipsets that have to be integrated. Lower cost, ease of integration will result.

Traditionally, HD arrays used to always be SCSI of Fiber Channel. Nowadays, clever vendors such as Huge, Medea, & Apple offer arrays that still have SCSI or FC connections to the computer, but within the storage devices themselves, ATA or Serial ATA drives are used. SATA drives can have vaguely similar performance characteristics as compared to SCSI, but at a much lower price point. Need X amount of MB/sec? Instead of 2 or 3 expensive SCSI disks, 4 cheaper SATA disks will do it handily...and still cost less.

A good case in point: I was assisting a client configure a G5 based product that needed high throughput. The Seagate Cheetah 15K3 74GB drives cost $550. The Western Digital 74 GB Raptor 10K drives cost $220. The Raptor isn't quite as fast as the Cheetah, and doesn't have the ultra-low seek time of the SCSI based device (not as crucial for HD utilization), but does offer something like 80+ % of the raw throughput....at less than half the price.

Now that Intel is looking to commoditize some of the components used in RAID storage, it should bring prices down over the next year or so.

System integrators like Huge and Medea have to charge a premium for their wares. They have a small user base (how many people need ultra-high speed storage? I mean, come on!) and have to amortize their research and development, advertising and marketing, distribution etc. costs. When and if the white box manufacturers can provide storage systems of sufficient throughput and stability and security, this will pose an economic threat to the existing players. Whether these white box manufacturers will produce equipment suitable for HD's specific needs will be the more interesting question over the next year or two.

In any case, expect the costs of HD capable storage to continue to fall over the next year or more at an even faster pace than it has in the past.

At least, that's my opinion, I might be wrong.

-mike

Update Friday, 6/11/04, 2pm CST The link comparing Heuris product to Lumiere's has some information that is somewhere between uncharitable to technically not quite right. Lumiere has a new page addressing the concerns Heuris raised here.

Heuris has revved their HDV product lineup - from their website:

"Now you can have all the tools you need to integrate your Apple workstation with your JVC HDV camera and DVHS player for UNDER $500!

HEURIS' Indie HD Toolkit includes these tools:

POST NAB SHOW SPECIAL, XtractorHDV NOW $99!

XtractorHDV - If all you want is the ability to edit the content in Final Cut Pro, this is the only tool you need!  An import tool used to transfer HD files from the JVC JY-HD10U or GR-HD1 camera to the desktop via firewire and then edit within Final Cut Pro. $199 (Post NAB show special $99) Click here to order

XtoHD - A player utility used to play the HD file back through a DVHS deck through an IEEE-1394 connection.  This tool can also be used to record footage to DVHS tape after it has been encoded using XportHD or MPEG Power Professional DTVHD.  $99 Click here to order

XportHD -easy-to-use, 720p, 30fps, HD MPEG 2 encoding plug-in for QuickTime.  This means it can be accessed from Final Cut Pro! (Will ship Q2 2004)"

You can read the entire website posting here.

UPDATE TUESDAY 6/8/04: At the bottom of the page is a link to a page comparing the Heuris and Lumiere products, but it has some misleading, if not technically inaccurate, information. I'll be posting a head-to-head at some point in the near future.

Sorenson has announced their new compression line for compressing video to a variety of different formats, including MPEG-4, H.264 (includng an HD variant), and various HD encoding options for Windows Media 9.

It'll also do Real Media, QuickTime, Windows Media, MPEG-1, 2, & 4, etc. etc. etc. The whole shebang will be $400, or you can buy bits and pieces for $120-$200 each.

Ships in third quarter (kind of a long lead time announcement, doncha think?)

-mike

High res. How high res? 7680x4320 pixels. 32 million pixels. 20 audio channels. 60 frames per second. 450 inch screen.

Obviously, still in early development.

Read about it here.

UPDATE 10AM Tuesday: AppleInsider is reporting that "G5 backorders have gone one step further...Apple has actually stopped resller orders on the models" and that those attempting to place orders are being put on a waiting list and told to expect more info "later this week."

OK, that sounds like we're getting realllllllly close.

To paraphrase Ice Man in Top Gun, "They must be close...my propeller beanie is spinning."

Also, as I suspected, the exterior case is apparently the same, there may be room for more internal drives (great for SATA arrays), the motherboard appears to be new/different, which offers hope that the FireWire 800 throughput problems might be resolved.

Either way, good news for storage for uncompressed HD.
-----------------------------------------------------

All the rumor boards are starting to buzz about new macs coming soon.

See these links:

AppleInsider's coverage
SpyMac's coverage
Cnet's coverage

Spymac reports all G5 orders online have changed from "Same business day" to "7-10 days" indicating stocks are running low.

In short, new Mac announcements expected by WWDC (in late June). The unknowns: how fast will they be, will the substantial problems with FireWire (speed & reliability) be resolved, and most importantly, will Apple lame out again and fail to ship these new machines within a reasonable timeframe (in the, you know, geological/tectonic sense) from the announcement?

I'm not holding my breath....but dammit, I will be buying one.....must....resist...Steve's....Reality...Distortion...Field....

-mike

if you look at the top right of this website, there are several links under the word, uh, Links.

I've been coming across some good websites with good information. I've added HD 24, Cinematography.net, Bare Feats, and Bill Weisman's 24p Entertainment.

There is a wealth of useful information on these sites.

While attending the HD Expo in Dallas this weekend, I saw and obtained a copy of the Apple DVCPRO HD Frame Rate Converter. The demo guys said that another demo guy had downloaded it off the Apple site, but I couldn't find it on the Apple site.

The ReadMe document is dated April 15, 2004.

From the PDF file:

"The DVCPRO HD Frame Rate Convertor is a versatile plug-in specifically designed for converting 720p60 DVCPRO HD variable frame rate media. This plug0in makes no assumptions about your intended media frame rate, so any combination of frame rate conversion and duplicate frame removal is possible."

I'll be playing with it this week.

BlackMagic Design has a sale on their DeckLink HD card for only $1000 in June.

Mike's Comments: Good deal, but when a vendor does this kind of price drop, it is usually in anticipation of shipping something better, or that prior product not doing something the new stuff will.

But if this card does everything you need it to, it's the best price ever on this stuff...

Here are my raw notes from Bill Weisman (of 24p Entertainment) on HD production and post production. Dude knows his stuff, big time. He's a consultant for hire.

Bill Weisman, 24p Entertainment

Sony F900 - version 3 software upgrade gives you another stop and a half of exposure - IMPORTANT

Sony shoots 23.98, 29.97, 30, 59.94, 60 - interlaced or progressive

Panasonic - 4 to 60 fps

Viper has no tape deck built into it, is camera not camcorder, is a camera head only

Sony HDCAM or HDCAM SR or D-5, or DVCPRO HD deck when working with Viper

What's the difference?

Audio - Sony HDCAM deck only has 4 channels of audio - it CANNOT do 5.1 for finishing

HDCAM SR has 12 channels of audio

Panasonic has 8 channels of audio

just know what deck you're going to use ahead of time and figure it out

Question: Can you cross record, from Sony camera to Panasonic deck? Can you do vice versa? In general yes, but be careful

Panasonic caveat - you can't capture over/undercrank on panasonic from deck,

It's usually OK to do a camera test to fool with it for a half hour on their camera in their shop

If using Viper - you can record straight to hard drives....which is another whole set of problems.

What framerates were used?

Don't shoot at 24fps exactly because of audio issues - if recording audio, DAT or others record at 29.97. If you shoot 24fps, you'll have timiing/editing issues.

NEVER shoot at 24fps (use 23.976) unless you are doing it for a specific reason.

How do you know what 24fps looks like? Do a test at the rental house,they'll let you do it. Use a rental company at a resource.

So you're shooting 23.98: for artistic reasons I want to shoot a different framerate on the show: put in a NEW tape - don't mix framerates on the same tape!!! Hoses you in post - your post house doesn't pay attention that close, not their job to watch that close.

LABEL THE TAPE if you're going to shoot different frame rates on the show.

on the tape, it says 50 and 40 - if shooting 24fps, is 50 (equivalent of 5000 feet of film) - if shooting 29.97, only 40 minutes.

The 90 minute Sony tapes DO NOT fit in HDCAM decks!

Label your tapes prominently about the framerate and progressive/interlaced, label what you were shooting, put the date on it.

when done shooting, push in the record tab IMMEDIATELY so that it is protected.

how do you mix the framerate for editing purposes? Need a good post production supervisor.

When downconverting, downconvert to a format that has SMPTE timecode - therefore miniDV is out, since it doesn't have SMPTE timecode.

Downconvert to a digital format.

digitizing footage is a 1-2 day deal. Friday to Monday morning counts as one day usually.

how does framerate affect audio? It doesn't if you do it right. Record your audio on your camera and you'll never have problem.

there is no audio at 29.97. DATs are 29.97. If shooting at non-29.97.

Clock-it box or Lock It Box that makes sure your timecode matches between DAT and tape

shoot 4:3 or 16:9 - can do for both - shoot 16:9 but protect for 4:3 can be wise - so you CAN use the 4:3 and it'll work, but so will 16:9

HAving a camera engineer can help you out - get it in camera rather than post - shooting Barbara Walters who has a lot of facial lines, can shoot at 24p and soften the look by menu selections.

Can save settings on a memory stick and call'em up with these scene files - Sony & Panasonic both do it.

Titling and credits etc. - did 4:3 titles so they wouldn't be cut off. run credits in the 4:3, in the 16:9 overflow, put in "more stuff" for extra stuff, out-takes, behind the scenes, ec.

JUST KNOW WHAT THE FINAL PRODUCT IS FOR

Pan & Scan can be done in post, but EXPENSIVE

free run timecode - timecode keeps running at all times, even when battery is off. Why do this? Any time shooting multi-camera, gonna lock the timecodes together from multi-camera so will have'em all in sync together.

record run - tape 1 hour 1, timecode stops/starts with deck. Can't jam-sync cameras that way, creates a big pain in post.

Lay down bars and tone at the head of every tape to make sure camera is set up at zero db. 30 seconds of bars, 5 seconds black. You still want to do this just to give yourself 5 seconds on pre-roll if only for capture purposes...

always use a hi-def digital lense when working HD. Fuji, Canon & Zeiss now make HD prime lenses if you want'em.

Last Star Wars was shot with zooms, not primes. Can be done.

He likes Fuji lenses, personal preference.

P+S Teknik 35mm adaptor lets you use film prime lenses with a Sony camera. Gives you true 35mm depth of field, and it takes away a stop and a half.

Can get wireless zoom and focus adaptors for HD cameras, you have marking rings just like the other stuff, and these are accurate and reliable. So focus pulling doesn't have to be right on top of the camera.

With tripods, fluid heads - they are geared for a certain weight range - (18-55 pounds for this model). An XL-1 won't get the right smooth resistance with too light of a camera. Match your head to camera. An HD rig can add up to 40-60 pounds...it might be more weight for smooth results.

You want the best quality movement for your head. have the right tool for the right job.

Matte box? Not required unless aesthetically desired.

Can stack up filters 4-5 deep, just like a film camera.

Sony & Panasonic can record 2 discrete channels of audio. Sony adaptor allows 4 discrete channels of audio. More discrete channels is good. DAT is only 2 channels of audio. May need 2 DATs, or a DA-88 that does up to 8 channels of audio.

Audio on camera or DAT or DA-88? If you anticipate 7 + audio channnels, consider renting an HD deck and recording that natively onto the deck. Whenever possible, record audio to the camera. DAT backup was nice to have, too, but on tape in sync is awfully nice to have.

when thinking of technology, think down the line to make sure it'll work in post.."So we saved $20 on set, but know we gotta rent $1000 piece of gear to work with it in post? SHIT!"

That's why pre-production

what size color monitors do I use on set? Don't use plasma on set. Bigger the monitor, the better the detail you can see. Always color. Viewfinder is B&W on camera. 20" color monitor is just about right. Monitors: when a DP on a feature, the DP has to be treated as king - don't speak unless spoken to! It's disrespectful since he's the artist, and you're wasting his time to make him explain. Have one monitor solely for DP, and have a PA keep EVERYONE away (no director, ANYBODY). This monitor is for DP and his lead gaffer. If a problem, take it up with gaffer.

If a plasma, it's so big it can be seen by everyone...bad...

HDLink or eCinema LCD setups? It isn't SMPTE C phosphors, and the LCD doesn't have all the adjustable knobs. So not recommended for color critical work. So Apple monitors are great if that's all you can afford, but if you can do the CRT based SMPTE C phosphor solutions.

Astro 9" LCD monitor that mounts on top of the camera to preview what you're doing. Camera operator can watch this 6 or 9 inch monitor. Costs more, more convenient, if in budget, is OK. Real operators can deal with viewfinder for the day.

****************************
comment - shooting flat for post fix, or get it in camera? Pros/cons, risk/benefits? - depends on your DP skill, post production skill, operator skill, and total budget....depends on overall creative/budget decision


posit: If you overexpose, the detail is gone. If shoot flat and color correct later. Film lets you capture that detail back. In film if you underexpose, you're screwed. In HD if you overexpose, you're screwed. Just know what the camera can do and not do and work it out.
**************************

always use the same video tape stock - try to rent the same exact cameras in a multi-camera setup, so everything matches.

can you jam sync different cameras? Can be done, risky, dangerous, if done wrong you're screwed, NOT recommended unless a seasoned professional.

SDX900 camera - 2/3" chip, 22 pounds, 24 fps NTSC camera

1/4", 1/3", 1/2", 2/3" chips - smaller the chip the suckier. Bigger chip, more chips, better

POST PRODUCTION: what for offline/online?

Does your editor own and know well his own system?

If you have to buy or rent, that's a different story. If offlining on Avid Express, he wants to online most likely with a higher end Avid system. All the material comes over OK. Did rough color correction on Avid DV Express Pro, he walks it over to his Avid DS/HD, and it all opens up OK in DS with color correction etc.

Budgets, graphics, compositing, etc....so many variables to include such that there is no one magic way.

QUESTIONS TO ASK ABOUT DOWNCONVERSIONS:

downconverts to DVCAM for offlining. Had 4 48k audio channels. Downconverted to DV CAM, but it only does 32k for 4 channels. Woops....introduces problems. Either use digital beta for offline or figure it out to get it in there.

downconvert to DVCAM 16:9 blackmatte, 24p timecode on bottom left, 29.97 timecode bottom right outside of picture area.

If want to submit to a festival, just cover the timecode with a chunk of black and you're done.

If DVD downconverts, specify DVD-R or DVD+R

5.1 surround mix - careful with decks

when finishing, gonna do a 5.1 mix, also do a LTRTME (left total, right total, Music, Effects). Everything will be there for HDCAM mix for a digital presentation.

budget out your post production properly - so many HD jobs get screwed up front, spend their time up front (and money) so they've backloaded their budget - doomed a bunch of problems that have to be fixed in post, so post budget goes into de-fucking the production mistakes instead of making it look good. We call these jobs turd polishing, and its very frustrating. So instead of post making your film look GREAT, post budget gets blown just making it not look FUCKED. Instead of "Wow, that's great!" it's "Hey, that looks OK..."

If shooting other formats with HD to intercut some DV or 35mm or whatever - upconvert to HD at 23.98, it's your HD master, downconvert and work from there.

HD Switchers? Snell & Wilcox makes a HD switcher, it's all new, they are starting to get out there. There's only 1 1080p24 truck in existence. Damn. HDNet has 1080i60 trucks.

Would you recommend editing natively with FCP HD if shot on Varicam?

Downside to the $25K Panasonic deck: lacks some functions, but Bill's guess is that the carriage and heads are not the same quality as the $65K deck, it might be more likely to eat master tapes.

hi def post production workflows

HD cameras go from $4000 to $150,000.

lots of format choices.

Post houses see a little of everything.

Don Stokes, post production guy

24p hidef is the buzzword of the moment.

Lot of talk, not a lot of understanding of the production process

understand what your final delivery process is going to be.

encouraging folks to acquire in HD, consider editing and finishing in SD.

No one realy wants to pay the ticket for what HD requires on infrastructure, lots of beating up on post folks for better costs.

has samples on HD D-5

shot on HDCAM, but HDCAM can't handle it in standard resolution. Shoulda shot on HDCAM SR.

Because it is a compressed format, they made tradeoffs. The codec has some colors that are not reproducable (well). Every scene with Barney had venetian blind effect, some was grey, some was purple, back and forth. Weird!

That color really doesn't show up. Sony came back with a firmware fix that didn't really work. Delivered master on HDCAM...and the artifacting was re-introduced because HDCAM had the same compression artifact.

the solution was to deliver on HD D-5.

Client downconverted to SD, and NEVER delivered over the air. Barney is not HD-able. Odd.

Varicam hasn't had any issues.

Problems they are seeing - if do produce in 24p and film isn't final destination, then it's great for acquisition, make sure can keep in sync in 24p. HAving down downconvert with 30i, matchback for EDL to 24p has sync problems.

Be aware not all monitors will support flicker free 24 fps footage.

Make sure monitor or projection system supports 24p.

Post Asylum is where Don Stokes works.

They've had no power for a week.

showed their reel - they have some nice work

Ron Sussman - edited in SF at Good Pictures after LA stuff - creative editor - works on Avid DS

been doing a job a month in high def. Uphill battle educating clients about the advantages. Convincing them of cost advantages is tough.

Did a Nissan project from 35mm, easy to make a downconvert from a 2K master to whatever.

99.9% of their stuff is shot 35mm. Edit it like normally would, decide at that point to go to hidef. Make those decisions at film-HD time as to framing/aspect/etc.

Compatibility issues with different decks is an issue. Mastered to all possible formats and framerates. Getting a deck that would play back and layoff from A to B was a big hurdle. There is no standard in hidef as their is in NTSC, article in NYTimes next week, working on ultrahidef - working a 10,000 pixel res in Japan.

ABC is 720p so is Fox, 1080i for CBS

educating clients as to exactly what it is, and what it's benefits are

they have a 34" 16:9 HD monitor (says HD on it) in suite. Edited SD on that set, client thought was HD...

more detail, more color depth in HD as compared to SD. Car stuff - rain drops on a car badge is gorgeous...in SD is no big deal.

couple of spots shot HDCAM 30p, cut them to be 24seconds long, sped up the audio, then finished it, re-introduced pulldown to introduce 24p-30i look.

Subaru stuff done at 4K from 35mm to keep that stuff as pristine as possible so that could use it for anything...size of storage for it all was a real issue.

Tezro - nice and fast, "it rocks"

Martin Spirit - Video Post & Transfer - producer (BBC, Travel Channel, Discovery Channel), Brushfire Films is his company as a sub-company of VPT&T. Edits on Disreet Inferno and Fire.

Acquisition is the most important thing - have the source in HD even if you don't finish with it now.

He's doing a lot of long form work, do a 24p to 29.97 interlaced, EDL list for longform, if over 1000 events, CMX only goes up to 999 events, so chop it up.

Take your 29.97 EDL into their Discreet, the machine decides what to do, cut on the "A" frame of the 5 frame 3:2 stuff so that it will know which to do.

They also do film-out. They shoot at 23.98 and they help the transfer. AlphaCiel uses an ArriLaser to go out to film in Seattle.

They've used just about every format, but finding that could take 40year old 8mm and transferring to HD and getting good results.

DPs can do a lot to the F900 in the field, BUT, once it's done it's done, there is no going back. They suggest shooting it flat so can choose to fix it later, it isn't baked in.

light for high def WELL - had some lattitude on film that you do NOT have here. So stuff that was within spec on film is NOT OK on HD, since there is much less "acceptable fiddle room" is better

12TB of storage for 3 Avids - and they keep running out of room.

As resolution gets higher, need yet more storage.

3 HD jobs in house at the moment, need lots of drive space. Facilities have to be able to have the storage.

delivery of the offline EDL is the problem. Prep for what you HAVE to deliver is an issue. Explaining the process to producers on how to get 23.98 to 30i and back is an issue. Shooting true 24p is a problem, you can't downconvert perfectly to 29.97, you'll lose frames. So 23.98 is the best way to shoot.

Acquisition in film vs. HD: All about budget.

Effects done in camera: there's no going back if it isn't perfect.

There is a bias from distributor's end originated in film as opposed to hidef, that bias is changing but slowly.

They play with footage carefully before they show it to clients - checking stuff in advance to make sure the final product is something the post facility is happy with.

long form - offline FCP or Avid ExpressDV - considering having a linear suite strictly for conform at VP&T.

again, clients are going to start asking for hidef, the AGENCIES are NOT pushing it. Clients are saying this because their stuff looks like shit next to the NFL stuff. Clients pushing it, agencies not. Agencies aren't forward thinking, they don't WANT shelf life - they do the spot and move on, and Hey! Next year, we gotta do another spot, maybe the client wants HD now (or not).

HDCAM SR deck - 12 channels of audio - how do you online it with digital mixers with 8 channels out?

use that 12 channels - put your 5.1 on the first 6, then for the other 6 channels, that's 3 foreign language stereo pairs....

client education once again - stereo mix is done and client listens at home on surround and bitches about it....

Here are my raw notes from the SWAFT Convention - it's raw, just as I typed it. Apologies since much of it isn't organized at all.

HD Super Session Panel

Peter Caranicas - moderator, editor of Below The Line

Randall Dark - hidef TV from early days - President of HD Vision Studios

if you want to see HD, gottta have HD camera, editing system, HD display system

gear was super high 5-10 years ago

has a facility in Hollywood - HW wouldn't embrace a tech that didn't duplicate the look of film - not until Thorpe, Lucas, etc. that had the feel and look of film, they wouldn't adopt

need HD for aftermarket and stock shots


Richard Cheski - company specializes in extreme adventure sports - they are working on converting the film library to hidef

shooting 16mm for size and weight in the extreme circumstances

----------

Larry Thorpe - from Sony, now at Canon - 22yrs at Sony - most of it on hidef quest - in last 18 months "all hell has broken loose" after worrying that it wasn't going to happen

hi def imaging is his thing

Canon Broadcast gave him a good new opportunity -

Q: bigger capture chips? Sony have a lock on 3 chip CCD for 1080p24

Karl Mesenbach - head of advertising for HDnet - for 2 1/2 yrs trying to get advertisers interested, no responses unless you're in 20-50 million houses. On Monday, SportsCenter will be HD,

NOBODY is finishing in HD 24p - everybody is downconverting. Go from nobody to everybody in short order

HDNet is 1080i


15-20 hours of original stuff every week

QUESTION: HOW LONG UNTIL COMMERCIALS ARE POSTED HD?

toughest nut by far is the comercial market - agencies are the gatekeepers, clients are sorely undereducated on HD, most stuff is shot on 35mm film for top end spots

Took 10 years to convince to get Sony to shoot their OWN spots in hidef for Sony's own commercials, took a senior exec mandate

when the flexibility, both on set and in post, is understood, then it'll happen

when clients see their upconverted spots back to back with native HD spots, THAT will drive it

business issue - all the broadcasters are saying "Why bother?" They are balking at broadcasting in HD, even if they are

they don't get any extra revenue (broadcasters don't) since it doesn't bring a premium in advertising

color went through the same thing

there WAS an increase in advertising in 66 when color went big

prowess of hidef TV for programming, but especially for commercials, is NOT undertood. A lot of experimenting has to take place (DPs, directors, etc.)

30" set is NOT a hidef TV

flat screens are no longer so intrusuve

a BIG screen is a new palette to show off a car, food, etc.

with increased size & res, can back away from object of interest, don't HAVE to zoom in as we do now to get the resolution

when clients see new imagery behind the products, then there is a new ballgame in the advertising market

Fox will be 720p football- Giants and Beras will be football games in NYC & Chicago - when clients see

this year is the kickoff year for things to start changing dramatically

60 hrs of hidef primetime today, 100 hours a day on cables, 14-20 hours a day on HDNet and directTV, etc.

1 million HDTVs sold in january, expecting 6.8M sold this year

past the 1% mark of penetration - once you are there, the "hockey stick kicks in" and it moves on quickly

Director wants film, he gets film. But directors have to be convinced.

Every time they bid on jobs, it's the director & DP feel comfortable with film...so that's what they do, since they are the decision makers.

Ability to et more clients and interest and juice is if you stake your claim as an HD Guy. As a biz decision, advertisers will start losing eyeballs if NOT shot in hidef

hidef costs less than film, more than digital video

QUESTION/COMMENT - so many of the post options are perceived as "do it the traditional way"

Richards Group account planners - Chick-Fil-A is using 7 year old footage in their spots

directors are complaining about lenses, framerate

Larry says in answer to that: biggest obstacle was fear factor for directors.

made equipment free to directors, gave a challenge - 4 minute piece - gave them support - maybe Sony could overcome the fear factor

very successful

3 year project - 8 stories to the theme of dreams the frist time, last year was "Joy" asa theme, this year 10 pieces to the theme of magic

In march in New York, 1000 creatives showed up to check it out

in many cases directors and DPs aren't aware of what's out there for lenses etc.

frame rates - 90-300 fps is beyond the capabilities

60fps cameras were shown at NAB

This year highest res, variable frame rate, with recorders, 1000 fps recorders

session this afternoon on post - attend!

fast paced frenetic editing is BAD for children - it is done to make up for little NTSC screen. Commercials stimulate with fast cuts. With HD, wide angle, hidef, frenetic editing won't be needed, scenes can last a little longer

with 16mm going to hidef, you have to nail your exposure - you used to have a stop or two of lattitude, no more - gotta nail it

HDNet Films - shooting HD res films! talk to them about that 2929 production - 10-30M dollar films, still doing 35mm

Lucia - HD acquisitions

even non-HD channels are buying HD material, because it looks better

WealthTV is looking for content

if you want to make good margins, have to bring a sponsor in with it.

WealthTV deal of Randall - WealthTV owns the domestic TV rights, HE owns the stock footage, non-broadcast rights, AND overseas rights

put the deal together cleverly to maximize your ROI

MISSED OPPORTUNITY FOR HD CONTENT ACQUISITION - LOW COST DIY POST

in the last 40 years, no activity like HD trucks - 8 to 10Million apiece INVEST IN THOSE COMPANIES!! HUGE SURGE IN THOSE BUSINESSES

P+S Teknik PRO35 is in 2 Dallas post houses

24p or 60i?If doing non-fiction, shoot 60i. Don't shoot natural history at 24p. Shoot fiction at 24p - gives that sense of poetic distancing, "It happened yesterday" "If you're shooting natural history at 24p you're a hypocrite."

Hollywood wasn't willing to try HD until they had 24p cameras

almost all pilots are shot in HD. Because it is faster & more cost effective.

If you have content, transfer to PAL+ for overseas markets

HDNet is not interested in 720p, not interested in film transfered to HD. HDNet is NOT interested in legacy footage

CBS etc. won't take anything not 35mm or 720 or 1080 res HD

DON'T shoot 16mm for ANYTHING.

"don't spend more than $2K for an HDTV" is the advice from Karl Meisenbach.

Larry Thorpe is saying in 5 years, 50% of the nation will have hidef

HD will move much faster than color, Larry Thorpe thinks

In digital cinema, Larry says, digital theaters - 4Kx2K, next week at InfoComm a 4K projector will be unveiled.

$50K projectors are coming to theaters, theater owners will begin to listen.

In 5 years, digital cinema will start taking a big bite

"What will it take for Hollywood to jump on that bandwagon?" Digital Cinema Initiative - recommending standards all the way through. They know a lot more now than they did 2 or 3 years ago.

Lower distribution costs are crucial.

Larry Thorpe looking to new communal viewing experiences.

--------------------------------------------------------------------

Designing a home studio panel

Roman Flute - DVD & graphics

Shareen Wornson - Avid editor, Express DV Pro

Carolyn Macartney - professor of film at SMU, shooting a doc in Africa on aPDF150

Heather Courtney - Austin based - new film in post is being done w/FCP, was shooting in Mexico on MiniDV.

nobody is working with HD from home.

So much for that.
--------------------------------------------------

----------------------------------

Making the change to high def tv

------------------------

55" HDTV is average size sold these days

rear projection is under $1500 now

"say you're from HD Expo" 7 locations in Dallas, any one item is 10% off, good through the end of the month. Cables and surge protectors

line conditioner/surge protectors -

Bill Weisman, Producer/24P consultant

BUG BILL ABOUT 1080P24 PSF Progressive Segmented Frames

loss of pixels on HDTV - manufacturer's spec is 1% is OK - 1 million pixels on screen, 1000 pixels

vendors break it up in 9 quadrants. If a "stuck" red pixel, can't be fixed. When you buy the TV, SEE THE UNIT, TEST FOR DEAD PIXELS.

Non-fiction for movies - are strictly 1080i, 1080p in the futre. Will NOT take anything 720p.

they wanted to be really sharp and to pop, so they foster 1080i

AVS Forum - AVSFORUM for buying TVs, etc. - big section on TVs and stuff

HDNet is available on Charter, Time Warner, Adelphia, etc., not on Comcast, not on Cox, hope to close those this year.

by 2006, all television transmissions should be digital transmissions - not necessarily hidef, but DTV. Rabbit ears don't work from 2006 on.

hd will be more of a standard by 2008-2010

DRS systems are dirt and scratch removal system

97% of all sets will show you 480p, 720p, and 1080i.

Very few sets show 1080p

Mitsubishi 82" is about $20K does a TRUE 1080p signal

Faroudja chips - native rate scalers (NRS), in last few years are in TVs, Sony, Mitsubishi,

WHEN TALKING 1080P AND ONLY THE MITSUBISHI DOES IT, ARE WE TALKING 24P, OR _ANY_ PROGRSSIVE? FOR THOSE THAT DON'T "TRULY" DO IT, HOW ARE THEY FAKING IT?

1080p60 cameras are going to come in time, so that will happen too.

DVD specs: Windows Media 9 on red laser DVD, vs. Sony Blu-Ray 23GB disks using MPEG-2 or MPEG-4 or whatever

hidef DVD players- DON'T BUY YET! Standards are not yet defined - who wants to trust Microsoft?

HD-DVD - "won't see at stores for at least 18 months"

HDNet doing 2 live sporting events per weekend - soccer on Saturdays, NASCAR or CART racing

HDNet owns 2 trucks, options on a 3rd.

ONCE AGAIN, INVEST IN COMPANIES THAT MAKE THESE!!!

interactive broadcasting on HDNet? no plans for that as yet

top recommendations for Plasma, CRT, and LCD:

Plasma - Pioneer Elite is the biggie - deep cell technology, 50", still the deepest blacks, brightest whites, cost around $10-12K range (speakers or no, etc.) pure monitor goes for about $7K, but is monitor only

LCD - flat panel LCD - LG is hitting the market, Mitsubishi makes a great one; very rare to see them over 42"

LCD rear projection - Sony is backordered, but they are way good, standard and XBR lines are good, XBR $5400 to $6400 for 62"+, $3000 to $4400

rear projection - biased, best for money is Mitsubishi, they guarantee the set will be compatible in the future even if they have to upgrade it, ATSC tuners included, for the money is a great deal

CRT based - Sony.

Hidef ready vs. hidef capable - depending on who's making the set, if it has an onboard HD tuner for cable or over air, are called integrated sets. Unless it says HD integrated, it probably doesn't have a tuner in it. In most cases you won't need it, SO LONG AS YOU AREN'T DOING OVER THE AIR STUFF.

Decide whether you want to watch over the air stuff or not. If yes over the air, ATSC tuner needed. If cable, don't need it.

90% of sitcoms are shot HD, CSI and those kinds of primetime shows are shot 35mm, transferred to HD. Slow process for 1 hour dramas to acquire in HD, but are posting in HD for sure.

male, 35-55 demographic for HDNet, looking for shows that fit

HDNet is like a blend of ESPN, Spike, and McNeil/Lehrer : )

-------------------------------------------------------------------------------

Next up: HD post production workflows

I'll post anything of interest from there.

Other news: I've started a relationship with a local company that has their own Varicam, so I'll be able to do some hands-on testing in the near future. I'm working on establishing a relationship with someone local with a Sony F900 so I'll be able to test with that, too.

I'm helping some folks shoot/post an HDCAM short on a few weeks as well as an R&D project, and noodling around with the idea of shooting my own short, just to run my own baby through the process, "eat my own dogfood" to borrow a metaphor from Steve Jobs.

-mike

While digging around in indiewire, I stumbled across this link from an article last September about a prototype 4K res HDTV in Japan. They had to make a custom camera, record to 16 HDTV recorders. Some people got nausea because the footage (shot from a moving car) was so realistic.

It has 4000 lines of VERTICAL resolution, giving it total of about 16 times the resolution of HDTV.

This resolution is in excess of what the SMPTE Digital Cinema Initiative was recommending.

Anyway, read all about it.

Updated 6/6/04 (see bottom)

This is of niche interest, but worthy of posting on since I've now talked to 4 people that were planning on making stop motion short movies and were planning on either shooting 8mm film or digital video cameras.

My immediate response:

"Why on EARTH would you do it that way?"

...because that's how they know to do it.

My answer for all 4 was the same - at least consider the possibility of NOT using an 8, 16, or 35mm motion film camera, NOT using a DV or Digibeta camera - use a digital still camera!

You'd need to have manual exposure control and shut off all the auto-everything so you could maintain consistency from frame to frame.

Stop motion is incredibly time consuming - why NOT use a digital still camera? The color gamut is larger, the exposure latitude is greater, the exposure timing is open ended, the resolution is HUGELY higher for a pittance of the cost - want to shoot 2K, uncompressed, RAW format? OK, no problem.

What about software, you say? Just hit up versiontracker.com (on a roll here, no links!) and do a search for "stop motion". For Mac OS X, I got 5 applications, at least 3 were shareware. iStopMotion was the one I'd heard most recently of, so I recommended that twice in the last 24 hours.

So here's what you'd do: Shoot digital stills in RAW format. Either capture straight to the hard drive or to the camera card. Straight to hard drive is best. Be sure your camera supports RAW mode. Google it if you need to. It basically is the raw CCD data, and there are interesting things you can do with that data in Photoshop, there's a plugin to mess with that stuff. It used to be an expensive addition to Photoshop, I think it MIGHT MAYBE be included in Photoshop CS. Dunno. But RAW mode is to digital photography what the incredible exposure lattitude of film gives you in the lab/darkroom in the analog realm. And, it's not even compressed - no JPEG artifacts whatsoever.

But I have thousands of frames, you say, I don't want to have to open up each one in Photoshop!

Fine, I say - Photoshop has a quite conveniently deep batch processing and scripting mode. Worst case, you have to run a batch for each take, since each scene would have it's own particular needs for exposure & color correction.

Think of it this way - if you want to shoot digitally in 2K res, 4:4:4 color space, utterly uncompressed, you're talking about a multi hundred thousand dollar digital cinema motion camera, and a storage system costing many tens of thousands of dollars.

Or, you could, you know, just use a $1000 Digital Rebel or somesuch and get 90+ % of the image quality.

And still shoot snapshots on the weekend, and carry it around in a purse or backpack. Try that with a Dalsa. : )

So you shoot numbered RAW files. (JPEGs work, just not as well)

Batch process'em in Photoshop to color correct, by scene, to 8 or 16 bit TIFF files.

These are, of course, VERY high resolution, at least 2K on a side (that's just a 4 megapixel camera approximately)

Use Cleaner, Compressor, Compression Master or somesuch to batch convert that into a QuickTime movie - use the Microcosm codec if 16 bits per channel (48 bit color)....or leave as numbered TIFF files, that would probably work fine, too.

Use Cleaner or somesuch again to batch that into a lower res, proxy file for editing. Make sure you keep the same file/folder naming and filing structure as the source material.

Take these proxy files to Final Cut Pro and do an edit. Just worrying about editorial decisions at this point.

If you want/need to, get it over into After Effects via Automatic Duck products for careful work, like very high quality pans and zooms, special effects, or floating point color correction via Color Finesse (which is included in the now shipping version 6.5, by the way).

If not, use the Offline trick in Media Manager to swap out for the high res stuff - then change your setup prefs, and render out (slooooow) your high quality master. Not all effects in FCP HD will process even at 10 bits per channel, so beware - you could be processing your results down to 8 bits per channel quality, but still taking up 16 bits per channel amounts of space (twice as much!). Therefore, After Effects, with support for 16 bits per channel, is mo betta for final cookout.

After Effects has a built-in process to handle proxies - you can work with proxies, then render with the full res stuff - even in 16 bits per channel color. After Effects can also handle custom aspect ratios, such as if you wish to process traditional 2K (2048x1536) resolution files at 1.85:1 aspect (or whatever).

Render out your finals from After Effects when you're done messing with it there - congratulations, if you want to go to 35mm film, you have high resolution, 4:4:4 color, 2K (or higher) 16 bit files. That is full on Hollywood quality, assuming your camera and cinematography made imagery sufficient to make the effort worthwhile. So long as you properly cropped and accounted for aspect ration, you're all set. After Effects can do that for you, too.

And all this can be done with simple, low cost, FireWire, SATA, or ATA drives. No mega-expensive array required. If you have one, great. But not fundamentally required.

If you're that serious, just send this enormous QuickTime (or more likely #'d TIFF files, that's what SwissFX and the like wanted the last time I talked to them) to the film-out vendor and you're golden (insert lengthy discussion on color calibration, white point, gamma, etc. here).

Or render a file out to an appropriate codec for the intended tape format you want, throw on whatever media (FireWire drive, DVD-R, series of DVD-Rs), mail it to a post house and have them import that into their NLE and drop it to the tape format of your choice. If you put it in a native codec for their NLE, then you save even more money. See? You don't even have to have your own deck. Easy as pie.

The advantage of the above described workflow is that image quality is kept INCREDIBLY high throughout the entire process, allowing for gorgeous imagery and very fine, smooth looking color correction. And from this Digital Master, you can now sample down to whatever you want. Just for fun, let's say you made 2048x1080 (a SMPTE 372M standard, still can be sent over HD-SDI at at least 12 bits per channel linear not log) frames after cropping in After Effects from your source high res files. This would let you master to 35mm film as well as Hollywood does presently, but also make an HD master (1920x1080 after a little cropping) in D-5, HDCAM, HDV, or other HD format; a file for DVD authoring, and a standard definition video master (that would be incredibly clean and pristine). How's that sound?

Of course, there are other simpler, less complex workflows too - shoot JPEGs, keep'em in labelled folders by scene and take, batch'em to QT movies in the working codec of your choice, etc., kick out to DV/HDV/DVCPRO HD/HDCAM/film whatever.

Assuming your color correction was calibrated, but that's another lengthy article discussing tests etc.

Anybody want me to flesh this out? I'd be willing to, but only if there is some (any) demand for it.

mike@hdforindies.com

UPDATE 6/6/04: Reader Jason Rodriguez wrote in (thanks!) to say this:

"I think the main problem that you'll encounter is soft lenses with the digital still cameras. Most broadcast lenses are very, very good, whereas many DSLR lenses aren't. You'll basically have to invest in Canon "L" series or something like that (preferably the prime lenses) to get good results, because the other lenses will not live up to a blow-up on screen-they're just too soft."

Good, LONG article on Digital Media Net by their editor Charlie White on HD For The Masses.

Check it out, a good (but long) read.