HD For Indies

WiebeTech has announced the Quad BayDock 800, a four-drive, hot-swappable FireWire 800/USB 2.0 storage enclosure... [MacMinute]

Upcoming product announcements for NAB:

Leitch-VelocityHD is a new HD capable NLE, similar user interface to VelocityQ

AJAwill be showing IO LD and LA, with digital (SDI) and analog breakouts respectively for 10 bit uncompressed video. They'll also have a 2nd generation HD D/A converter, the HD10C2

Quantel will be showing a whole bunch of digital intermediate tools. Expensive, not likely for indies.

The Foundry will be showing Furnace for Avid/DS and After Effects. Sounds intriguing - the blurb mentioned motion estimation, which is frequently used in retiming, and automated dirt removal (useful for DI, not for HD).

Red Giant, the distributor for The Orphanage's wares (among others) will be showing Primatte Keyer. It's a good, solid keyer, and affordable for the indie crowd. Also, Magic Bullet for editors - this sounds good, as it has been an After Effects only tool until now. Also Misfire, a plugin for film type effects - "grain, splotches & scratches."

Profound Effects has created some excellent After Effects plugins in the past, and will show some new ones. Digital Anarchy and GenArts will have some of their own to show as well.

Boxx Technologies based in Austin, Texas, will have two new workstations, allowing for 8 to 12 internal drives. Boxx makes some very powerful and interesting HD wares, and are worth checking into.

Sony HDCAM SR is being pushed for high end post work as a tape and raw file format. XDCAM uses blue laser 23GB optical disks and is being pushed for a shooting format.

From the March 2004 Post Magazine -

Aurora is rolling out a new HD card at NAB called ReaktorHD with these features:
-one HD/SDI input
-two HD-SDI outputs
-RS-422 deck control
-composite, s-video, and component output

This is big for a couple of reasons - you can monitor HD footage on on an HD TV instead of an HD monitor, and expensive HD-SDI to analog SD converters aren't needed.

This sounds promising, I'll be checking it out and posting on it at NAB next month.

Headline says it. It's Journalled, RAID Level 5, supports AppleTalk, CIFS, NFS. 180+ MB/sec.RAID controller is 2 GB/sec Fibre Channel. Gigabit Ethernet connection.

Check it out.

Sounds really cool. Must be ungodly expensive.

Was talking to some folks tonight about a possible project to work on. It was shot with the Panasonic 24p camera that shoots 1280x720 pixel, 24 progressive frames per second. They had downconversion dubs made to DVCAM tapes. I don't know if they burned timecode onto those, or letterboxed, or handled the telecine & pulldown process yet. I know they had 6 tapes, with a total of about 3 1/2 hours of footage for a final total running time (TRT) of about 12 minutes.

Let's assume for the moment, to keep it simple, that is was telecined to 30i from 24p. How can we post this in an environment without decks, without HD monitoring?

Let's look at some possible workflows to post this to maximum quality for minimal cost.

Fortunately, he's already done downconverts to DVCAM, which answers one of the first questions/challenges: whether to do the offline in HD res or downconverted to something easy, like DV.

It's already DVCAM, so the question is moot for this project, but an interesting one to ask for other projects. Is it cheaper, or is there a turnover point, to downconvert all your footage to DVCAM (720x480@30i), rather than rent a deck and capture all your footage at offline (PhotoJPEG circa 50% quality in Final Cut Pro) but at final target resolution (1280x720@24p in this case)?

I don't know the cost issues yet.

IF YOU DOWNCONVERT TO DVCAM:

PROS: If you downconvert to DVCAM, you're paying on a tape by tape basis. The more tapes, the higher the cost. Once DVCAM, it has very light/low system and storage requirements. DV is about 13 GB/hr of storage with audio. Can edit on low end systems, including laptops. Easy to run tests out to DV or DVD, and preview those on easy, standard, consumer gear.

CONS: It isn't the same resolution as your final. It is interlaced 30fps footage instead of 24 progressive frames per second. That means there is some variation in exactly where your edit might lie. Colors may shift during conversion, the color space of DV isn't as good as HD so some values may be clipped, throwing off color correction decisions that you make.

IF YOU GO HDCAM LORES - If rent a deck (fixed cost per day) and capture to an offline compression (say, PhotoJPEG, somewhere around 30-60% quality)

PROS: You are now working with the same pixel dimensions and frame rate as your final copy will be. Colors will be accurate, but images compressed somewhat. In my casual testing, PhotoJPEG 50% looks about 80-90% as good as uncompressed footage to the naked eye. Editing points could be counted on to stay exactly the same, down to the frame. High quality previews can be exported as high quality proxies for effects work, knowing full quality can be subbed out later. Data rate is reasonable - PhotoJPEG 50% is usually within 2.5 to 5 MB/sec - ballpark of DV. That's roughly 10-20GB per hour of footage. Could make a D-VHS HD offline rough with some (or a lot of) export work.

CONS: You have to have a system with an HD capture card, and right now that requires a G4 or G5. Keying or color correction would exacerbate the quality difference between this compressed and your final uncompressed footage. Previewing is more difficult, you'd need some kind of HD monitoring throughout post production if you wanted to see even vaguely accurate colors (on computer screen just flat out doesn't count). Making a copy to view on standard definition systems, such as VHS, DVD, or any NTSC display system, is more complex than with an NTSC offline version.

Pros and cons either way. There different costs involved with either path. Depending on what equipment you had, specifically monitoring equipment and/or HD capture card, can tilt it one direction or the other.

Either way - if you edit 720p24 or 480i30, when you're done with your offline, you need to do an online edit.

You'd need to rent a deck at this point, and presuming that your timecodes all matched up nice and neat, recapture just the final selects in your edit with appropriate heads and tails. This'll take some time, and if you've gone the DVCAM route involves some careful footsteps with CinemaTools perhaps.

Batch capture all your final selects that are in your final edit. Decisions need to be made. If you have a DeckLinkHD card (as I do), you can either do real time color correction in 8 bit per channel color (256 values per color channel), or do non-real time color correction on 10 bit/channel color (1024 values per color channel). Speed or quality? Your call.

My tests on a dual 2.0GHz G5 indicate that 1080p24 footage takes about 6 times longer than real time to render a color correction using the 3 way color correction filter in Final Cut Pro. 720p24 footage should go faster, probably about 3 times real time. Not so bad.

Do your color corrections in 8 or 10bit color. When done, tell to Render All if you haven't been rendering as you go. For a 12 minute short, will probably take 35-40 minutes to render all those color corrections and transitions (your cross dissolves etc.).

Now is the time that you need a color accurate HD monitor. Rent one. If it doesn't have an SDI input, you'll need an HD-SDI to HD component adaptor. I don't know rental costs, but they cost $1800-$3000 to buy new. How long does it take to color correct all the scenes in a 12 minute short? I don't know, but I'd think at least a day of monitor rental.

If it isn't the same day that you rented the HDCAM deck, go back and rent it again now and lay down your master to tape. Make multiple copies while you're at it. Hopefully you scheduled it such that you have the HD deck & monitor together for as little time as is necessary.

Congratulations - you now have a high quality uncompressed digital master, you did uncompressed color correction at up to 10 bits per channel (very nice quality).

If you want your own digital master for future downconversion to whatever media of choice (say, uncompressed SD for a broadcast Digibeta master, or for a DVD), just Export a digital master from Final Cut Pro to your external FireWire drive. Also keep your FCP project file on that drive or burn it to a CD. Congrats, now all you'd need to make further changes is an editing system with deck and that data on CD.

More thoughts on this process later. If this project moves ahead, I'll be documenting the steps in detail, step by step, with screengrabs etc., as I go.

This could be the next "It" camera - if the rumors are true and the hopes are right - 25 fps, 1080i, 3 CCD, Sony.

This sounds good - the article hints that it could be the kind of thing that would be perfect for the low cost indie crowd. There is also some commentary about/from JVC, saying they'll have a follow up to the GR1 (the suckier of the two, um, sucky HDV cameras out now), but that new product will not be shown or announced at NAB next month in Vegas. All in all some very good info. I'm curious to see if it all pans out - 3 chip CCD sounds great, implied PAL sounds great (25 fps converts to 24 fps with a 4% speed change), 1080 interlaced resolution sounds good but not great (progressive would be better). Remains to be seen if it's all true. I'm surprised - my understanding of the HDV spec was that 1280x720 pixel, 30 progressive frames per second was the high end of the specification. Time will tell.

Check it out!

Talked briefly with Curtis Clayton, the director of Rick (with Bill Pullman) about shooting on HD. More to follow later. While attending Super Size Me, turns out Richard Day, the writer/director of Straight Jacket was sitting right behind me, and we talked for a few minutes about shooting and posting with HD. Interestingly, he said that shooting with HD was quick and easy - it was post that created difficulty. After color correcting on HD with D-5, they had another round of color timing issues to address during the transfer to film - why correct it twice? I didn't clarify whether that was a DI process or a film only timing issue he was talking about, I need to follow up.

One director I spoke to said that he liked that HD let him do long and/or multiple takes. When you shout cut, everybody stands down and gets out of shooting mode. With HD, let it roll, and everybody stays at that level of attention. Don't have to call for a take, make sure everybody's ready, and/or shout "Settle!" a couple of times.

One director discussed irritation with some HD to film transfer houses. We were talking tape formats and the group consensus wasn't sure but thought that D-5 was uncompressed, but we're sure HDCAM is. He said it was frustrating to do D-5 color correction, then the HD to film transfer facility wanted to transfer from HDCAM - which is compressed, so you lose the benefit of all that subtle nice uncompressed HD D-5 stuff. The transfer house should work from D-5. For that HDCAM transfer, I wonder if their system is 8 bit or 10 bit. I would hope to god 10 bit.

It was very interesting to hear a director say that while they (meaning the people making the film) don't understand all the nitty gritty of this digital stuff, quite often the post house and the transfer house DON'T UNDERSTAND IT ALL EITHER. I don't know if that would be a problem with the reps people talk to or the techs behind the scenes (I hope not the latter), but does feed my suspicions that EVERYBODY is still trying to get a handle on how to do all this right. I was very impressed with Christian Zak who was on my panel from Technicolor...but I also don't know what their rates are.

Richard Day mentioned that digital projection was problematic - when they projected digitally at Sundance 5 times, each time it was like watching a different movie. Apparently, as he put it, the HD projectors require very regular calibration. They SHOULD be calibrating to the color bars on each movie's tape. But they calibrate once in the morning to the first tape and leave it there the rest of the day. This results in everyone's movie being projected with the colors from the first movie's calibration settings. NOT good. He alluded that it was sort of like backwards film process. With film, the process starts out hard and awkward but gets easier and more certain as you movie through post production. By the time it gets to the projector in a theater, about the only thing you are worried about is focus and brightness. With digital projection, maybe it's a little red, and the gamma might be off as well, for instance. There are more things to go wrong, more variables beyond your control, when projecting digitally.

It was very encouraging to sit in the balcony at the Paramount, lean over to stare at the huge Barco projector, and look at the inputs on that unit and see one BNC labelled "HD-SDI" Excellent! This means High Definition Serial Digital Interface. HDCAM uses that interface. In theory, you could even haul your G5 and storage up there and plug in and play back your movie as an edit in progress. OK, totally unlikely, but a fun idea. More fun would be to take the Barco home and project it, oh, I don't know, on the side of the neighbor's house. I have a friend that does a downsampled version of that in San Francisco called Potrero Walk-In Movies. Rock on, BJ.

I also saw Blackballed: The Bobby Dukes Story, which was scathingly funny, great fun and proof positive that it's the story, writing, and acting that counts (not the film stock you used). I spoke with Chris Lechler from werkaround productions who worked on the film. They shot on the Panasonic 24p camera that is only DV resolution, but they did some excellent color work on it at a high quality post production facility. It fascinates me that people will shoot movies on DV (super low cost) and then go into an Inferno session (ultra high cost) with it. But for these guys it made a lot of sense - they ran 2 or 3 cameras at a time, and were running and and sliding in the woods with them. Had to have something small, light, inexpensive, that could handle a bit of beating. I really hope they can go somewhere with this, it was very very funny. Another advantage of shooting digital: I think he said he shot something like 55 hours of footage for this film. Definitely room for some extras on the DVD.

It was interesting to notice how much raw energy was captured on DV in both Blackballed and Super Size Me.

I continue to see the sense of picking the right camera for the job - sometimes it's film, sometimes it's DV, sometimes it's HD. The camera has to do what the scene calls for. You can't just say "I'm going to shoot on HD, cuz I think that would be cool" and expect to have it all work out smoothly. Curtis (speaking of Rick) said that it worked out quite well that the film was mostly interiors and nighttime exteriors for shooting on HD.

That's it - I MUST buy a voice recorder and keep it with me at all times, I'm missing all kinds of good info because I'm not taking notes in these chance encounters. Pardon me, I'm blogging here.

I think it is very interesting that there are films showing here that were shot on HD and are being projected digitally, as well as films shot on HD and being projected on film.

I very much want to talk to both camps and see if anyone is closing deals without a film print, and what the shape of that deal was. On deep background, of course.

OK, to bed.

Just saw Able Edwards at a screening at the Dobie theater as a SXSW Film event. The entire movie was made by shooting actors against a small greenscreen (using about a 6 foot wide area) shot with a Canon XL-1 (link is to present XL-1s, a slightly updated model) DV camera. Sort of a Walt Disney/Howard Hughes/Citizen Cane black and white retro affair. I think it's great that one guy essentially assembled the whole movie from greenscreen shots using combustion from DV footage with a total budget of around $30,000....

But what a mess! Bad acting, bad "I'm old, see?" makeup, bad compositing. I asked some questions afterwards and learned about their use of the XL-1 and combustion. He said he'd been putting the pieces together for about a year and a half. If only they'd had a Digibeta (or even BetaSP) to work from. HDV didn't exist yet, but certainly would have been the way to go had they done it since last year under those budget constraints.. I should be kinder, hindsight is always better than my vision....and it's more movie than I've ever made. But eek.

Lesson to be learned: check your workflow, make sure you can get good results. But sometimes, your budget is your budget.

Sorry been offline, been seeing lots of movies, lots of panels, talking to lots of folks. I'm working on an article about how HD formats are still compressed, how to work with truly uncompressed footage, etc. I'm trying to get full specs on all the formats in question to be thorough. Had a very interesting conversation with the rep about the Varicam HD camera. Why no link? It's 1:33:11am, I am tired.

More to be posted soon.

This is just some notes at random I thought were worth mentioning from other panelists, as well as some of my thoughts based on what they said:

-ALWAYS do separate source audio for both raw quality and quality control reasons. Sync sound.

-audio tri-level sync issues can be a real problem when working with HD. When I know exactly what that is, I'll say something about it (did I mention I don't do audio, other than "YO! Craig, where is it?")

-Panavision guy talked about their HDCAM stuff rented for something like $10,000-$12,000 per week for a full on package including the camera, 2 lenses, SDI adaptor, etc.

-I asked the Panavision rep if it were possible to use the camera to capture from, the same way a lot of indie DV folks both shoot and capture footage into their computer from the one device (the camera). He said there is an available SDI tap for the camera. I pressed and asked about deck control. At first he said you'd have to manually do it and press play on deck and capture on the computer, and then we talked about RS control and he said he'd have to double check and get back to me. IF THIS WORKS, then it's a potential major budget point - if you are truly a starving artist production, could have editor (or assitant editor - hey, that's what they're there for) capture at night after the shoot wraps for the day. Capture all your offline footage and edit away for your digital dailies. Yes, you'd still need to have that camera or a deck available later for the final high res capture later in production. Unless you digitized everything and copied to low cost storage like FireWire drives.....but nah, probably cheaper to just rent the deck later. Depends, though on deck rental vs. storage cost. As an exercise in geekitude, I should Excel-sheet this one up and know the crossover point, wherever it lies...

Christian from Technicolor talked about tape to tape color correction as one of those hidden costs of HD production. What about purely in machine color correction? It would be waaaaaaaaaaay slower, not real-time. A quick test on a 1920x1080p24 10-bit per channel test file using the 3-way color corrector in Final Cut Pro shows it took just under 6X realtime to render color correction. So even just spot checking, occassionally rendering, and leaving all the bulk rendering for an overnight/lunchtime burn, that would still take, conservatively speaking, at least twice if not three times as long to do as compared with a realtime session (not the 30-50% I hypothesized earlier), PLUS rendering time. But it's a possible way to save money...if you don't shoot yourself in the foot doing it wrong. Christian and Jenn (the cinematographer) were both very clear that HD intended for film needs to be color corrected and prepared in a very specific way, otherwise the transfer will blow.

Theory: is it possible/viable to hire a colorist to come work on your system? Or you bring your media to them if on a compatible system? Are there any hired gun colorists that will bring their monitor to your setup? Am I smoking crack to think that this MIGHT be a business model in the future for colorists to freelance after hours when not at their day job running the quarter million dollar realtime film color correction systems? This must sound nuts to any film production person...but I've seen TREMENDOUS change in the publishing arena over the last 15 years. Remember typesetters? People that just set type for a living? That's a job that doesn't exist anymore. Setting type isn't done as well as they did it, but it's 80-90% as good, and you can do it yourself. Will this happen for some HD projects? We'll see. Back to panel notes:

Digital dailies were also mentioned as a significant non-obvious cost in the HD production workflow. There are boxes that can attach to the back of the camera that will make a DV dub as you record your HDCAM footage. There's your digital daily right there. Just rent one of those. Again, no need to rent a deck or go to a post house.

Kristin Petrovich from HD Expo (which looks to be a good resource for HD info, bookmark that one) mentioned 4 key areas to be sure of before starting an HD project:

Deliverables- know where your movie is destined - is it going to film? To video? Video first, maybe theatrical film later? Be very clear about your intent for your project, so that you can shoot as you need to. 24p is the most flexible deliverable - it can be made into film or video easily - but is the most awkward to work with from an editing perspective.

Bring your editor to Pre-Production Meetings - again, most of the panelists nodded vigorously at this one in agreement - get an editor and get them involved EARLY if you are planning an HD project. HD is not just one format, it's many. DV is simple it's either NTSC (720x480@29.97fps interlaced) or PAL (720x576@25 fps interlaced). That's it. HD is a set of standards, ranging in size from 720x480 up to 1920x1080 at a variety of frame rates, either progressive or interlaced, depending - 23.976, 24, 25, 29.97, 30, 59.94. See? It gets complicated.

As an aside, check this out to see how complicated the standards are just to look at this footage. This is a list for just consumers to figure out. Imagine how bad it is in the professional spectrum!

OK, back on target - just get your editor involved early to help figure out all the gotchas. This dovetails into the earlier comments about HD having the potential to save money, but that potential can be lost if mistakes are made.

Roles & Control - On a proper HD shoot, it's quite possible to have your DP, who's been shooting for decades, to be working with a DIT - a Digital Imaging Tech. Be clear about who's in charge and who's just advising who. DITs know the technology, but be clear on exactly who has authority to declare a take acceptable or not. You don't want on set quibbling.

What was the 4th thing? Sorry, didn't write it down or something.

"HD is not film. If you can afford it, just shoot film" was advocated by several panelists.

I pitched in "HD is not film...but that's OK." It is a different medium with different capabilities.

Keep in mind HD can't do a lot of things we take for granted with film. Because the image sensor is so small, the image is recorded onto a tiny area. That means any inconsistencies or dirt or smudges on the lense are magnified on the HD sensor. So better lenses are required to get similar results.

Also, because of the small sensor, HD doesn't have the same depth of field that film does. This creates all kinds of issues, like having to get waaaaaaaaaaaay back with the camera to get separation between your actors in the background to achieve depth of field. What if you have to shoot in tight spaces, like indoors, or bathrooms? Wooooops. Back way up, or build sets (oops, there goes money). The adaptor that is great that lets you use 35mm Arri compatible lenses is also adds about 6 inches in length to the camera, making it even more cumbersome.

HD cameras, especially with the 35mm adaptor, are heavy and cumbersome. Hire a DP who works out. No joke, that was suggested, and only half kiddingly.

HD cameras are tethered, by both power and monitoring needs. Sometimes that tether might just be a few feet long. Rather limiting!

Don't forget, HD requires power, unlike a conventional film camera.

Want to shoot slow motion? There is the Varicam, but it only shoots 60 frames per second and then samples out of that time base, grabbing the frames it "wants" to achieve the desired frame rate. So the slow mo isn't as smoothly progressive, especially if you want to change the framerate during a shot. Things that are cake (just turn the knob) get way more complicated with HD.

Film has about 11 or 12 stops of latitude for light imaging. HD only has about 5. That means it is very important to get the image you want in camera - you do NOT have the same amount of push/pull/fudge room in the "lab" as you do with film. There is no lab - only digital color correction.

Avoid patterns - wardrobe test! Black and white stripes are the worst!

Do careful motion tests - if there are things like a picket fence in a pan, or in a travelling shot (shooting through a car for a conversation, for instance), you will get some wierd artifacting.

Certain pans produce undesirable stair stepping artifacting that film would not. Even a landing plane, ostensibly a slow event, can/has introduced some undesirable artifacting.

I was wanting to ask but didn't get the chance when they mentioned these things - were what they considered "bad" about some aspects of how HD looks truly BAD, or simply DIFFERENT from how film would react? Are they judging the image on it's own merits, or judging it against how film would look as a golden standard? Obviously, big motion moire patterns when shooting stripes or patterns would be distracting and bad, but other things....hmmm....

The depth of field issue is usually problematic with HD, but the P+S Technik 35mm Pro adaptor ameliorates those issues somewhat.

One of the big areas where HD doesn't look like film, and flat out doesn't look as good, is how it handles highlightts. Aiming the camera at a light, or at a car in daylight, or having people walk by a window indoors with strong daylight coming in can reveal (in a bad, eek-that's-icky kind of way) the video roots of HD. One way to address this issue is to stack ND (Neutral Density) filters up on the lense. Altman put as many as 6 ND filters on top of each other to shoot The Company, his Neve Campbell movie about ballet. He had to shoot into the stage lights, and while ostensibly crazy to stack that many on at one time, it worked. Hey - 220, 221 - whatever it takes.

Looking for film festivals that will take digital formats instead of film? Check out withoutabox.com. Kristin advocated it highly as a one stop shop for a variety of indie filmmaker needs once the film was finished and in the can, I mean on the tape.

OK, that's the end of my notes. Off to the frog party...

Asked and confirmed in two panels - I asked Cristian Zak from Technicolor's Digital Intermediate group whether they were seeing films mastered on HD and sold in that format, then again in Sex, Lies & Videotape: The Business of Being Independent. Christian said yes, and I got a nice clear answer from one of the panelists in Sex/Lies about whether they were seeing deals cut when the filmmaker only had an HD version, not a film version. He said yes they were doing those kinds of deals, but it was on a case by case basis. I asked "...so it's a viable business methodology?" and he said "Yes, definitely." But it's on a case by case basis, and exactly how it is paid for - out of an advance, or the distributor pays for it outright, etc. varies on the deal. Good to know. There was also some discussion in that panel about DV and HD mastering, and the sentiment that DV wasn't perceived as "good enough" to show theatrically was echoed, but that ultimately it was the quality of the image that mattered. HD can be acceptable or not, it just depends on whether the end product looks good onscreen. It can look good onscreen if properly done. End of argument. It's doable. Of course, doing it well...but hey, that's 300 other postings to cover...

The panel also had some good discussion about the value of a movie once in the can - what you spent to develop it has NOTHING to do with it's perceived value in the marketplace. (witness such films as Battlefield Earth). "If you spent $5 million to make a movie, and the market only says it's worth $100,000, that's it." Keep it in mind. But it was also said that if you go into movie making looking at the back end, trying to do it for profit, you'll lose money. It's done for love, not profit. This is the semantics of moviemaking, not my usual topic for this blog. But an interesting point to keep in mind, nonetheless.

OK....Guerro's!

Panel went well today, but with such a large panel (7? 8?), it was difficult to cover much ground. A lot of good information was disseminated, however. My favorite new bit of info: P+S Technik makes their Pro 35 Digital Adaptor - from their website:

"allows broadcast videographers and digital filmmakers to attach any Arri PL mounted prime 35mm film lens to their high definition (HD) or standard definition (SD) 2/3” video camera to obtain the three-dimensional quality of a 35mm film camera on videotape"

Way, way cool. Check it out.

In general, everyone on the panel also urged the importance of dilligent planning - know where you are going to go. If you are really, really pushing for a theatrical distribution, shoot 24p. If you know you are ONLY going to video, any of the 30 based formats is acceptable. If unsure, 24p can go anywhere, but interlaced footage has it's limits when trying to go back to 24.

Whatever your goals, just plan EVERYTHING through in advance. Have your editor have long, technical discussions with the post production facility about EXACTLY how the workflow needs to commence BEFORE commit to a camera, or facility, or workflow - how files & data & EDLs will get handed around. Someone on the panel talked about editing their project in Final Cut Pro and then walking into an Avid based suite and not being have the two systems play nicely. Automatic Duck might have been able to help (although to her credit I don't know if the useful product even existed when she had that problem).

While it is possible to save money working with HD, those potential savings are highly, HIGHLY contingent on you doing things right. It is very, very easy to assume something will work that won't, and it'll cost you an arm and a leg to fix it later.

I have further notes, and I'll enter them later, but or now I'm off to Guerro's and then to the frogdesign party (I used to work there).

Hitachi today announced a 400GB drive, the Deskstar 7K400. This would be useful for uncompressed HD arrays. 7200 rpm (means it's fast), with throughput from 60 MB/sec (at the beginning of the drive) down to 30 (at the tail). This means that even at the end of the disk, a 6 disk array could still do single layer 1080i 29.97 in 10 bit.This opens the possibility to have a 4 terabyte array connected to a Mac G5 (soon, when 4 port PCI-X SATA cards ship). That's TEN HOURS of uncompressed, 10 bit per channel, 1920x1080, 24 frame per second footage.

Check it out.

And check out the specs page if you're my kind of geek. Or just wait and I'll do some drive writeups to say what this means.

OK, blatant self promotion time - I'm going to be on a panel at SXSW Film Conference in Austin, Texas tomorrow at 11AM in Room 12AB of the Convention Center. Check it out.

I'm working on a production workflow methodology for HDV with Final Cut Pro. There are a couple of software solutions for dealing with the fact that FCP doesn't work natively with HDV footage whatsoever.


The Mac can't open/play/read HDV footage without some help.

The catch is to convert it to something you can use, edit it, and if you want to go back to HDV you have to recompress it with 3rd party software. Details coming.

There are some interesting possibilities to be explored with shooting full dynamic range footage - means you shoot to get details in both shadows and highlights as much as possible. This means your footage looks murky and low contrast as you shoot it. This is OK, you're going to fool with black & white points and gamma in post. The guys at The Orphanage are way up on this, and have a package called Magic Bullet for dealing with a hole variety of DV and HD conversion issues. Very much worth checking out.

One path would be to stay within Final Cut Pro, another would be exporting and working in After Effects Production Bundle. I know the AE way will work, and be very high quality. The problem is that it is a HUGE extra pain, with MAJOR time spent shuffling files, and requires a bunch more storage and file management, and the possibilities of getting confused and lost are large. Would DEFINITELY want to utilize Automatic Duck's Automatic Compostion Import, which is a tool to port your FCP timeline into After Effects. They also make products to pull Avid timelines into FCP.

The smooth, graceful way to do this? Working on it. Major geekboy detail mode.

Another challenge is how to deal with the fact that it's 30 frames per second if you want to go to film, which is 24.

There are at least 3 software options that I'm aware of that can do pixel level, frame to frame morphing to try to smoothly interpolate from one frame rate (30 for us) to another (the desired/sacred/purportedly holy 24fps).

They are:

Realviz Retimer HD is a plugin for AE/FCP. Retimer Pro is available for Mac.

ReelSmart's Twixtor is an AE plugin. Works in FCP now. I've doodled casually with it before, looks like it takes some tweaking. But that was a couple of major versions ago.

The third is a recently announced plugin for Shake (Apple's high end compositing application) and also for After Effects. Shake costs $5000 all by itself. Oops. Anyway, the plugin is called called Algolith. I may be getting this wrong, but it sounds like to do the kind of conversion I'm interested, I'd need the Shake plugin. Pricey. But Shake is an awesome advanced compositing application (used heavily on Lord of the Rings among others). But it's also fearsomely complex to walk up to (I've dabbled a bit, it's tough to learn).

UPDATE: Saturday, 3/13/04 5:45PM:

...unfortunately this 30==>24 theory doesn't work. Had a conversation with the guy from Technicolor and he said they've tried everything and it just doesn't work sufficiently well. A neat idea, but not technically feasible at this time, even with motion vector analysis based software. So this was just a failed theory. For now.



What will the quality be like? How much trouble is it to set up on a take by take basis? Can you just throw the whole movie at it and let it chug (seriously doubt it). How much hand tweaking, and trial and error, is required to get this to work worth a damn? How skilled does the operator have to be? Once it is all set up, how many days/weeks does it take to render? I'm figuring this is going to be VERRRRRY SLOW to render, even on a fast machine like a dual G5 or a dual Xeon or dual Opteron. Overall, will this be time/cost effective? I don't know. These software packages & the required plug-ins cost anywhere from $2K to $8K. I'm thinking the $1500 standalone package will probably work the best...but it's a one trick pony. The other packages with plugins offer a lot of other functions and capabilities.

BUT my theory is that once it is set up and running, if it takes a week or two to finish this process (assuming it does it right and doesn't have to be redone multiple times), THIS IS OK for an indie filmmaker. This of course is predicatd on lots of testing to figure out how to use it correctly. Unless it's the last 2 weeks before Sundance submissions deadlines or somesuch, I'm still hypothesizing this will be acceptable in the indie world. This is whole retiming thing for indies is a stretch - I take for granted that I am a VERY technical boy, very used to using complex software, doing things like putting together motion composite visual effects with a few hundred individual pieces of graphics and video and animation all into one shot.

When I first started coming up with all this HD stuff and was talking to Frank Reynolds (editor of In The Bedroom) whom I met when moderating a previous SXSW panel, he said a lot of people poke away at editing their films over months, as time allowed. As an indie, it's not like you're trying to time your movie against the next Spiderman release. Finish your film, then sell it. So perhaps, maybe, this might be a valid approach.

Don't shoot me yet, I plan on cleaning up this post and adding names, links, prices, etc. tonight.

Just wanted to get this done in the meantime.

Hey all, just to be clear -

All content on this site is copyright 2004-2007 Mike Curtis and 3beam, Inc. dba HD For Indies, with the obvious exception of externally linked material. Is it clear I wrote it? It's mine. Is is just an RSS feed link? Not mine.

No part of this site (caveat as above) may be reproduced in part or in whole without explicit written permission from Mike Curtis.

Period.

However, if you'd like to use content from here, please feel free to contact me, I'll usually grant permission or be open to discussing syndication rights - I instituted this policy after some blatant content lifting.

-the management

This is a quick rundown of how I helped a client do a project with HD footage...without ever having any HD post gear whatsoever inhouse. NO HD deck, no HD capture card, no HD monitor, nothin'. BUT we had the HD footage and were able to edit, composite, and do effects with the uncompressed, full resolution footage.

I'm changing some details to protect the guilty & foolish (myself included) and altering facts to make us look smarter. In other words, this is altered to what we SHOULD have done.

During preproduction, the client realized shooting HD would only add about $1600 to the day rate. Our daily expenses were around $30K so this was, incrementally speaking, cheap. Excited to play with new tech they committed.

Our client deliverable was a DVD. It was decided before I was brought in to shoot on HD and sample it down for a 1K master. I posted & produced effects and edits at 1024x768 resolution. This would conveniently fill a standard projector when we made a PhotoJPEG Quicktime movie at the end of the project.

During the shoot, they had a live tap to a DV deck, with a converter box to do realtime downconversion from HD to SD (standard definition, NTSC) as well as handle the 24 progressive frames per second from the HDCAM to the 59.94 interlaced fields per second of DV. This ended up being fairly problematic due to DV timecode breaks. And as is often the case, the problem was between the chair and the deck- the person hired to hit record & pause didn't pay sufficient attention. Grr. Grr. Grr. Calm...calm...calm...Timecode breaks are bad enough in a proper SMPTE timecode environment, but DV timecode, um, sucks. We should have simply shot the HD and gotten a local telecine and dub if possible. I'm saying telecine but what I really mean is downsampling the frame size, letterboxing it on black (no true 16:9 signal, too much of a pain to work with), and handling the 24p to 29.97i issues. Ideally, 24p timecode should have been in the lower left, and SMPTE 29.97 dropframe timecode should have been in the lower right.

In any case, we now had our (faultily timecoded) DV tapes in our hot little hands, and we immediately shipped the HDCAM tapes and a hard drive of the huge honkin' variety up to a vendor in California.

We did our offline edit in Final Cut Pro with the DV footage, then emailed an EDL and FTP'd a small QuickTime movie to the vendor in California. From our EDL, and checking against our little 320x240@30fps movie we'd sent, they digitized the HD footage and visually matched and confirmed it was the same shots we asked for. In reality, they weren't and they didn't, and we were stuck in a terrible bind. We didn't have some shots from our EDL, we had random substitute shots instead, some shots started late or ended too early. Bad bad bad editing juju. Much "but that's ALL WE'VE GOT!!!!" editing ensued with "interesting" continuity ramifications. So make sure the QuickTime is good enough for them to tell what's going on, and to pester them to be sure it all matches. Good timecode on your tapes HELPS A LOT.

OK, back to magical perfect faerie land:

The vendor sent back tapes & hard drives and it all worked perfectly- we took the HD shots and did all our After Effects compositing work with those, rendered out our modified clips, put those back into Final Cut even though our systems couldn't possibly play them back in real time. That's OK, we knew what it needed to be and were just swapping out shots.

We rendered out a 1024x768 master movie.

From that, we compressed a 1024x768 PhotoJPEG movie for computer projection by client, and also made a DVD version for kiosks and sales presentations.

HDCAM footage downsampled in AE, FCP, or Cleaner to NTSC resolution looks WONDERFUL - it has MUCH better color reproduction than traditional video and the detail is AMAZING compared to any other SD video you've ever shot or seen. $150,000 cameras shooting 1920x1080 and shrinking down to 1024 do a GREAT job.

So, short version (offlining a film works similarly):

-shoot HD (or film)

-promptly get DV dubs (or telecine if from film), making sure you have letterboxed footage, NOT true 16:9 amamorphic. Do YOU havea 16:9 monitor? Didn't think so. Me either. Have them put 24p timecode in lower left, traditional SMPTE 29.97i timecode in lower right, keeping both timecodes OUT of the image area but in the title/action safe area.

-send HDCAM to vendor with HDCAM deck with your big hard drive.

-do your offline edit - the key thing here is that THIS is when you're making all your editorial decisions. Get it perfect. Get it right. Get it DONE from an editorial perspective. All your narrative arc, character development, continuity issues, whatever...get it figured out NOW at this stage. Be FINISHED.

-send EDL and a QuickTime movie (so they can visually confirm & conform) to the vendor

-vendor digitizes only the clips you need. Be sure to specify generous handles at beginning and end, 5 or more seconds per clip.

-Vendor sends drive back to you.

-you copy HD footage to local drives

-if you cleverly handle FCP just right, can convince it to link to the HD footage based on it's timecode...you'd be smarter than we were. But we didn't have FCP 4 with it's included 24 frame goodies.

-do your effects work, color correction, compositing, whatever you're going to do.

-render that out to a master file if you're going back to HD and send it on hard drive back to vendor

-vendor puts it out to tape

-in meantime, downsample it for video, projection, DVD, etc. usages.

-congratulations, you can say you've worked with a DI process

-we've long called this DM - Digital Master

Note: you could work with film this way if you had to for some reason, like you had a lot of effects, or wanted to color grade everything to create a look.

Geeky visual effects post production notes:
One advantage of shooting HD for SD distribution is that we had a LOT of extra resolution to work with. In the end I had no greenscreen keying to do (storyboard changed), but it would have been ideal to key at HD res and sample down the footage with alpha channel for SD compositing (super clean edges for composites). It also let us do a lot of pan and scan stuff - we had tons more pixels than we needed, we could scale up or down, pan & scan, etc. and always have more resolution than we needed.

In another instance, some footage was shoot loosey goosey handycam style, wobbling all over the place. After it was shot we realized it should have been a locked off shot. With SD footage with would have been disastrous. With HD, I was able to stabilize it in After Effects and crop down to just the portion I needed for our SD (standard definition, NTSC) final output without ever seeing the edge of the video source.

We also had a lot of effects work to do on this project - it involved cell phones that had been designed but not built yet. All we had were some Photoshop mockups of the user interface for the phone, and some hand built utterly non-functional phones (Carved out of soap? Who knows...) I had to take the Photoshop user interface mockups, animate them in After Effects to respond to when the filmed user pretended to press a button, and track and stabilize the phone UI onto the moving phone in their hand. The involes scaling and distorting those Photoshop UI comps on a frame by frame basis to match the movements and position of the phone screen. Fortunately, there are tools to help with this. Unfortunately, they aren't perfect. But by getting close on the HD footage, by the time it was scaled down to SD resolution, it looked flawless. Getting it that good had we shot on SD would have been MUCH harder.

So for shorter pieces, or graphics/effects/compositing heavy pieces, shooting HD for SD is worth considering as a cost saving methodology. FX are expensive.

-mike

Oh yeah - I'm Mike Curtis, this is my blog.

You can email me at mike@hdforindies.com

My plan is to make this a participatory workgroup - to eventually have FAQs, forums, etc. The point is to build a group of knowledgeable individuals to share info with. 2-pop's HD section and forums are a graveyard; nothing seems to happen there. Hopefully this will fare better.

But the blog is a start until I get enough content out to organize it into articles.

For now, if you are involved in HD production, or are investigating it for your own use, and want to learn & share info, please email me (no spam, please! Like that'll slow'em down...)

I will be posting detailed, geeky reports on my research in SATA arrays, realtime performance, FCP rendering times on realistic test projects, post production workflow strategies, various hardware/workflow combos for various production scenarios and goals, links to HD relevant articles, etc.

-mike

In my last post I explained some of the cost challenges involved in HD post production. (Actual shooting production will be another posting)

Storage for a lot less that still does everything:

Storage has been the biggest "gotcha" cost item in HD systems (although monitoring is a close second). The traditional answer has been SCSI drives. Ultra320 SCSI dual cards are about $450 for the Mac, the biggest fastest drives hold only 74GB and cost over $550 each. Painful math - $550 for about 40 minutes of footage? Ouch. Plus cabling and casing.

When I started researching this in November of 2003, the recommended inexpensive hard drive solutions were from Medea, Huge and Apple (their X-RAID) were the best deal for the dollar, but the lowest cost systems that could shovel data fast enough still cost about $10 per gigabyte of storage. Oh, and how much storage did I need? Just to hold the final uncompressed version (not including all the render space, heads & tails, unused footage, offline footage, graphics, etc.) of a 2 hour movie would take about 800 gigabytes of data. Terabyte (1000 gigabyte) storage systems were around $6000...for just the RAID (Redundant Array of Inexpensive Disks), not including the interface card...or the computer. So you're starting to see the issue here, right? Two terabyte systems cost about $10,000 from Medea. Apple does substantially better - a 3.5 TB (terabyte) system costs aout $12000 ready to roll with interface card as of January of 2004. This is a more practical amount of storage to finish a film.

Hard drives are very much like a record player (if you're old enough to remember these. I never owned one, but my parents did). But you've seen DJs, right? : )

Hard drives start at the outer edge, like a record player. When they read (or write) data, they start at the outermost edge and procede inwards towards the middle. The edge of the disk (or record) is moving faster than the middle of the disk. Remember sitting on the outer edge vs. the middle of the merry go round? Yeah. That. Data can be read (or written) faster at the outer edge of the disk than it can towards the middle (towards the spindle). This means that sitting down and testing an empty drive will give results indicative of just the outer edge - the fastest part. As the drive fills up, it reads and writes data...slower. By the time you get to the innermost, last tracks, it is common for the data rate to only be about 1/2 of what it was at the outer tracks. So when measuring hard drive performance, it is important to know not just how fast the drive is at the beginning, but also how fast at the end....and how quickly it drops it's datarate as it heads towards the end.

I'm partial to Apple's Final Cut Pro as an editing solution, thus I'm partial to Apple G5s to edit on.
Apple ships their G5 computers with the capacity for 2 internal Serial ATA drives. Serial ATA, or SATA, is a good interface for drives, capable of carrying 150 megabytes per second of data over that SATA data bus. The bad news is that only one drive can be attached per bus. Two drives of even the fastest sort (Western Digital Raptor 10,000 rpm, 74 GB, in case you were wondering) are capable of about 70ish MB/sec at the beginning of the disk, falling to the low 50s by the end. That's not enough with the overhead involved in running two separate streams of data (that means the read/write head, the needle in our record metaphor, having to skip back and forth while playing two songs each entailing 110 MB/sec).

How can you get more drives in there? One solution is the G5 Jam from Wiebetech. They sell the bare kit for $500, which includes a PCI card that has 2 internal SATA connectors to attach to their custom mounting plate that fits two more drives in a different location in the G5 case. But those Raptor drives cost around $250 each. So this gives a system capable of about 220 MB/sec to very nearly the full capacity of the array for about $1500.

Promax in LA has been advertising for months a 1 Terabyte, SATA based array for Mac G5s. Don't know if it is available yet, but they want $2000 for it. Seems a bit high to me. But hey, I'm cheap. They make some very solid solutions and are well respected in the business, so this isn't a dig at them.

I found a way to do it for even less. A company called Firmtek makes the PCI based SATA card, called the Seritek 1S2 for about $70. A company called PPA, Inc. makes external SATA enclosures that Fry's Electronics sells for $50 each. The Seritek card's SATA outputs are on the inside of the computer. If the SATA cables from the PPA kit are run out an empty PCI slot cover, that's an easy way to make a 4 drive SATA array. SoftRAID makes some OS X software to format and partition the array into the portions that are and are not capable of the necessary data rate. I'll have a detailed report on this later.

Very soon, there will be SATA cards with 4 external ports available for Mac G5 computers (can't say yet, but know it's coming). This makes it possible to have 6 drive arrays with G5s. And the cards will be PCI-X, which means they can sit next to an HD capture card (more on why in later report). So this opens the door to having 10 drive arrays. My favorite fast SATA drive is the Hitachi 7K250 - a 7200 rpm 250 GB disk capable of about 63 MB/sec at the beginning and around 25 MB/sec at the end. This would yield PLENTY of throughput to capture & edit dual stream all the way to end of a 2.5 TB array, at a total cost of under $4000...about $1.50/GB. Not bad. Apple's X-RAID maxed out at 3.5 TB (instead of 2.5) costs about $12000 ready to roll, or about $3.42/GB

There are several disadvantages to this setup - it is NOT pretty, a big flimsy cabling mess, it's only RAID 0 not RAID 3 or 5 (more on that later), I wouldn't want to move it after setting it up....but it's (somewhat) affordable...compared to the options.

What else can be done to bring down the cost?

Core Hardware - the Computer

As I said, I'm partial to Apple's Macintosh computers. Right now, the dual 2.0 GHz G5 is very fast and reasonably priced for the power it provides. They start around $3000 but include a dual head video card (can drive 2 separate computer monitors) and can read and burn both CD and DVD disks. Handy for burning dailies to DVD. With the free included DVD software, iDVD. Bonus! You'll need some more RAM, but not huge amounts - $200-$250 will be plenty.

If you're really going to max out the SATA storage, you'll want a FireWire drive to boot from. FireWire 800 drives are faster and only cost $50-$100 more. $400 will get you a fast 250 GB FireWire 800 drive to boot from, run applications, and store certain non-throughput critical files. More on that later.

But this is the basic box that will do much (but not all) of what Avid systems (and other vendors' hardware) will do that cost many, many times more than this.

A couple of large, high resolution computer monitors that aren't 16:9 (or 16:10) aspect ratio can be had for $700 each, such as the very nice La Cie Electron22blue IV. 2048x1536 resolution that is supported by the optional ATI Radeon 9800 Pro card on the G5. The stock card may support that resolution, I haven't checked yet.

Core Hardware - HD Capture Cards

This is the core of your hardware system - the gadget that can take the feed from your HD-SDI spigot on the deck and turn it into the pretty pictures on your screen. Right now, my favorite best bang for the buck candidate is the DeckLinkHD card. A single PCI-X card capable of real time color correction and transitions in 1080 resolution footage. Unfortunately at this time, only in 8 bit per channel footage, not 10 bit, and not the very efficient PhotoJPEG codec used to get entirely watchable, editable, HDTV viewable footage at approximately 1/10th of the data rate of uncompressed footage. Perhaps in the future. But it also does everything from NTSC and PAL up to 1080i29.97 through it's configurable SDI taps. And does realtime HD to NTSC or PAL conversion (within limits). And other useful tricks.

Monitoring: How NOT to spend $10,000 on a pretty TV

instead of an incredibly pricey multiformat studio monitor (which would be great, but we can't afford it), consider getting an HD-SDI to component coverter box (around $2000) from Cobalt or AJA and plugging that into an HDTV monitor. Which is just an HDTV without a built in tuner, which costs extra and we won't use. There are $1500 CRT based models, there are $2000 LCD based models, find something good. I'll be looking into best-bang-for-buck HDTVs for our usage in a later installment that is viable for studio usage, i.e. not huge, consistent color, calibratable, no built in tuner to save costs, etc. I still wouldn't trust this setup for final color decisions, but hey, it's something to work with and work from. And it's not affecting the quality of my footage, just the clarity of my view of it.

Editing Applications: which one is right for me?

There are three viable editing applications for serious work, all from companies that start with the letter "A" -

Apple's Final Cut Pro - my personal favorite, highly scalable, LOTS of high end users and it's where the industry heat and attention is and has been focused for a few years - immensely powerful and scalable, only $1000. Such a deal. Gripe if you want about it's shortcomings, but come on, it's only a grand. Mac only.

This is a good solution for those who want to do all of the post production themselves - FCP scales well, handling everything from 1GB/hr offline media up to uncompressed 10 bit per channel 1920x1080 interlaced HD footage. Many feature films have been cut with this, recently Cold Mountain. My favorite, I'll explain why later.

Also, it has a nice balance of new and old - new enough to be based on a modern set of user interface paradigms, but old enough to have matured through many versions (4.1.1 is the latest) and have many (but not all) of the bugs worked out, and a lot of depth in its feature set (although many things could be improved, like sync sound editing, media handling, backups, etc.)

Avid's XPress line - Avid invented digital nonlinear editing and has ruled the roost for a long time. After losing market share on the low end, they came out with this product line that maintains excellent file & effects compatibility with their higher end offerings. While not capable of HD editing, makes for a very viable offline solution that you can trust will hand off extremely smoothly to a high end offline solution, down to the same interfaces for color correction etc. It's essentially a cut down version of their higher end applications, so you can trust file compatibility better than any of the other solutions discussed here. It's the most flexible in terms of platforms - it will run on Macs or PCs. Also, bundled with their innovative Mojo hardware acceleration product it is a very powerful system. One of the good things is that the user interface is just like that of the other, higher end Avid stuff, just without all the features. One of the bad things is that the user interface is just like the other Avid products - it was started a LOOOOOOOOONG time ago and grew organically - which is to say they've bolted all kinds of stuff onto and into the UI they already had, and it is a bear to walk up to cold. STEEP learning curve if you are walking up to it having never edited before.

Adobe's Premiere Pro - after being considered an also-ran for many years by the professional editing community. Adobe got serious and did a total re-write and came out with Premiere Pro. It is vastly improved - but is it good enough to choose over the other products? Remains to be seen. But it is lowest cost, serious editing application that will run HD cards like DecklinkHD. PC only. Only $700, very affordable. Also, Adobe offers some very cost effective bundles that include Premiere Pro, After Effects, Encore DVD, and Audition. Their Pro Bundle throws in Photoshop and upgrades to After Effects Professional. It has a pretty friendly user interface that will be comfortable and familiar to folks who've worked with other Adobe products - and it integrates beautifully with Photoshop, After Effects, and Encore (their DVD authoring software). If you want to product something graphics or visual effects heavy and go straight to DVD, this is a VERY good choice (if you're happy with Premiere as an editor).

There are other choices, but these are the three most serious contenders.

So all in all, my presently recommended setup would break down something like this:

Monitoring: 4000
Mac G5: 3000
FW800:400
RAM: 200
2 big monitors: $1400
HD card: 2000
storage: 4000
FCP : 1000
======================
$16,000
(no sales tax if bought in Texas, either! We have a production exemption)

So that's about $16000 for a soup-to-nuts uncompressed, realtime capable, 10 bit HD 1080p24 system with over 6 hours of uncompressed storage. With this system, you can do 10 bit color correction (non-real time, have to render), do visual effects with additional software, etc. etc. And that's not bad at all.

Don't want uncompressed, don't want HD monitoring? Can cut many thousands out of this.

This, as I see it, does the most but cost the least. It could be faster (the Cinewave cards are worthy of exploration for that), but the quality will be excellent.

And once again, as always, I plan on breaking each of these topics out into individual articles in their own rights, with a heavily biased perspective of how applicable/useful stuff is from a high function, low budget perspective.

Go rent a deck for several days at key points during production and you are in business. But more on how to produce on the cheap in later installments....

In short, it's this:

Indies don't have money, but they do have time.

If you shoot HD and have your editor go to some post facility, you are burning a HUGE hourly rate as you sit in there and try to make decisions as fast as possible while $100 bills go poof every time you take a coffee break or have try to tweak an edit a different way.

Is that how you really want to work?

What might you REALLY want to do editing your film?

How about hang out in a comfy environment of your choosing of creation?

How about take as long as you feel comfortable editing your film?

How about having complete creative control to mess with different pacings, different editorial decisions, etc.?

How about having the freedom to do some George Lucas style cutting, where you do split screen edits, speed up and slow down certain actions, etc.?

How about if you want to do a complete, full color grading of the ENTIRE film? And you want to do it all on an uncompressed, 10 bit per channel version of your film? Huh, what's that mean? What if you want to do some cutting edge color treatment of your whole film to give it that intense look that you just saw in some top dollar release?

What if you want to tweak color, shot by shot, for the whole film, but be able to change your mind and go back and fix it after you come up with a better way later? Like you always do?

What if you just want complete control and want to take as long as and your editor see fit?

Then do it this way, do it my way. If you can scratch up 15 to 20 grand, you can OWN your own editing system. Or sell it when you're done and treat it like a rental system expense.

Independent filmmakers don't have some things but do have others.

One of the things they don't have, pretty much as a rule, is big huge heaping gobs of cash tothrow around on production and post production. If they were, they pretty much wouldn't be independent anymore. But I digress.

But what they DO tend to have is a bit more TIME. Unless there is a looming film festival deadline, from what I've seen independent filmmakers have some time. Their editors tend to work on a project basis rather than an hourly basis...or at least a weekly/monthly rate not a day rate.

The working difference between systems with full realtime capability and systems of equal image quality that have to render their effects, color corrections, and transitions much slower than real time is chunky - a 3 second dissolve just plays back on a realtime system, it might take 10-15 seconds to render on the lesser system. Color correction can eat a lot of time. But your editor probably has it. It will slow him down by a factor of, say, 30-50% per day, probable worst case (I'm researching those stats now). But having him operate in your studio (or his), without that under-the-gun pressure, allows for better decision making...and a better product.

The kinds of solutions I'm suggesting are NOT as fast as the top flight systems costing 10 to 20 times more. BUT what they do offer is EXTREMELY high quality, and the luxury of your own time to get it done as you see fit.

A lot of folks are editing offline at home and then going in for online HD sessions with realtime everything. Great!

Work with either FCP or Avid DV Express for that workflow if you wish.

If you want to do everything yourself, be sure to have a guru on tap (there's an email link on this page somewhere, right?), but once trained and prepped you can do an awful lot yourself or with your knowledgeable friends if the groundwork is carefully laid.

I've spelled out in another posting how to get a high end, kick ass, uncompressed, 1080p24 10bit system. If you don't want to do everything yourself, there are all kinds of ways to scale back to save money.

Don't want to do your own effects, final quality color correction or be able to integrate effects yourself? You don't need uncompressed. Pat yourself on the back and save a few thousand dollars.

Don't want to have to see your work in full HD? Save many many more thousands of dollars and just work with DV dubs of your HD footage.

...and so forth and so on.

You've come this far doing everything yourself. Keep working with smart people you trust who know their craft, but work smart on the least amount of equipment to do what you want while maintaining the quality you need.

There are some very interesting tradeoffs to be made in terms of letting things take a little longer to get done but having it save a bunch of money. Check your options, and carefully analyze what you want to get done on your film.

-mike

For the sake of argument here, let's say you're someone already familiar with DV editing. DV is great - the quality is pretty darn good, it's relatively inexpensive, it's a pretty mature technology such that there are a lot of options for cameras, editing, etc. You can viably edit on your laptop if you have to, and your desktop computer can thoroughly rock for around $5K including a nice NTSC preview monitor. Now let's say you want to move up to HD for possible feature work. How much more does it take?

THE COMPUTER ITSELF:

While even laptop computers can now comfortably edit DV footage, the file sizes and quantity of data to be transferred for HD is fearsomely large. Avid sells their systems that are HD capable for small home prices. In the last year or two, the very fastest desktop computers available could just barely handle it.

THE COMPUTER MONITOR TO SEE IT ON

DV is only 720x480 pixels. Pretty much any monitor is sufficent to see that. Want to work with HD at full size? Be ready to spend over $1000. Want a 16:9 or so aspect ratio computer monitor screen? Whether CRT or LCD, be ready to spend $2000 to $3000 for a screen capable of at least 1920x1080 pixels.

STORAGE REQUIREMENTS

Well, for starters, the files are stupendous in size. DV is a compressed file format - some detail information is lost for the sake of keeping the data rate low. DV is about 3 1/2 megabytes per second, something well within the capabilities of even the lowliest hard drives sold today. That works out to around 12 gigabytes per hour of footage. What is called uncompressed video runs about around 21 to 23 megabytes per second, or about 76 to 83 gigabytes per hour of footage.

At present, HDV doesn't edit natively. And if you want to project an image on the size of screen used in theaters, it behooves you to work with uncompressed footage. What is the preferred format for movie bound HD? 1080p24 10bit. Huh? What's that? That's 1920x1080 pixels, 24 frames per second, 10 bits per channel (red,green, blue) of color. Uncompressed 1080p24 10bit runs about 110 MB/sec, or just under 400 gigabytes per hour. Want to edit two streams uncompressed for a real time transition in your editor? Your drive system needs to be capable of two separate data transfers of 110 MB/sec, so 220 MB/sec, with dual simultaneous data transfers as well. (More on what that means later).

Damn.

And regular hard drives can't do this. Regular drives these days can do 30-40 MB/sec if they are pretty fast. As a matter of fact, not even 4 regular hard drives can do this. If you look at the recommended specs, they are suggesting high speed SCSI drives. Or at least systems with high speed (Ultra320 SCSI) systems that interface over SCSI even if they are using ATA (same kind in your computer probably, but theirs are faster) drives. OK, more on this later. Moving on:

INTERFACE:

With DV, you can jus plug a FireWire (or iLink, or IEEE 1394a, all the same thing) cable between your camera and computer's built in port (or $50 add in card). Whee, lookit go. That's all there is to it. But with HD, it's not so easy. Even the "low" end cards were $5000 to $10000 until very recently (in 2003). Avid STILL charges over $100K for their realtime capable HD editing systems (if my info is still up to date). And the cards usually input and output HD SDI, which stands for High Definition Serial Digital Interface. This is what HDCAM decks spit out and read in. It's a great format - digital, high definition, crisp, clean easy. And expensive.

DECKS:

HDCAM decks have cost about $80K and up. Maybe they are down to $60K now, I haven't looked in a while, but that's still pretty pricey.

MONITORING:

A good/decent NTSC preview monitor can be had for about $1000 and up. If you're on the cheap, an awful lot of indie productions have been done by running plain TVs off of the camera's s-video output, or off of a hardware codec, like the Sony DVMC-DA2.

Want a multiformat studio monitor from Sony that will take an HD-SDI signal? Those start around $5500 and go up from there. Oh, and the HD-SDI board that goes in the monitor so it can accept the HD SDI signal? It's another $3500 or so.

So this has been, to say the least, rather disheartening for the typical independent producer. All this hype about gorgeous digital high resolution imagery you can edit on your computer is unreachable. Impractical. Hasn't mattered for 99% of the folks out there.

However....

...stay tuned for the next exciting edition....

With HD formats, you do get the advantage of digital as compared to film.

It's already zeroes and ones ready to go into your computer, so there is no expensive telecine process. And no film developing. And no waiting for dailies. And no scanning in for effects work, compositing the effects, then paying for a film out to preview it....and see if you have todo it over again. No sweating that the film scanning and filmout will somehow not match your other footage. Want to digitally color correct the whole thing like Oh Brother Where Art Thou and others since then? No problem, you just need lots of disk storage and time but it can be done. There's a lot of talk about DI these days (digital intermediary). This is where an entire film is scanned into the computer, conformed, color corrected, everything made perfect. That is the golden master from which the film print is struck, the HD master is made, the DVD master is made, the broadcast master is made, etc. The idea is that you make the film digital, and go back to film (and everywhere else) from that one source. For HD work since it was already digital, I think the term Digital Master is more appropriate. It isn't an intermediate format for anything - it's The Source from which all versions flow. This can be a huge time and money saver, as often the same stuff gets done to different formatted versions of the film.

This one is pretty simple, although we could geek out at length. Don't worry, I will later on.

HDCAM allows for a variety of shooting formats in terms of pixel dimensions and frame rates, but the one independent filmmakers are going to care about is known as 1080p24. This is a format that shoots 1920x1080 pixels (DV is 720x480 for NTSC, 720x576 for PAL) at 24 frames per second - quite intentionally the same rate as is used in movies shown on film. It has real, honest to God (and SMPTE) time code. It's uncompressed - that is, after the image is captured in 4:2:2 color space (OK, another lectuer to follow on what that means) it isn't JPEG'd or MPEG'd or anything else to make the data rate smaller..and throw out detail. Gorgeous imagery, a relatively easy path to a film transfer, 30 bit color for nice gradations of color tone and flexibility in color correction...why not use it? Oh yeah, it's really expensive from an indie's budgetary point of view. But if you've got the money, this is clearly way to go.

HDV also has some choices, but the one we care about most is 720p30, which involves 1280x720 pixels at 30 frames per second. 720p is much less resolution than 1080p, but it is still 3 times better than video. And it is progressive full frames, instead of interlaced fields (this is another topic I'll post or link about soon). BUT it is 30 frames per second...which is problematic.

HDCAM cameras are expensive - prices start somewhere around a REALLY nice Mercedes and get into the starter home price range. So you're probably renting HDCAM gear, not buying it.

HDV is dirt cheap - around $4K. But the cameras aren't as nice, the images aren't as nice by a long shot, the HDV format is heavily compressed, which is problematic for effects work and heavy color correction, and it shoots the wrong frame rate. Why use it? Oh yeah, you an afford it.

Low end indie films will shoot HDV and make do. Documentary filmmakers will shoot with it and be happy. Low end indie movies going direct to broadcast (SD or HD) or DVD (and eventually high definition DVD if the Blu-Ray guys will get their butts in gear) will probably shoot with it if they can carefully post it for optimal results.

I think that with a lot of careful work in shooting style & post production, it may be possible to shoot on HDV (maybe not with the HD10U) and end up with usable 24 fps material. Maybe.

HDV is this new low cost shooting format. If Robert Rodriguez were 23, I think he'd be shooting HDV if he had to make El Mariachi now as a poor college student.

Presently the only camera viably using this format is the JVC JY-HD10U. It shoots several formats, but what really floats my boat is the 1280x720 pixel, 30 progressive frames per second mode. This is THREE TIMES the pixel resolution of DV. How much detail is that? Look at the output of your DV camera. Turn it on it's side. Put two more just like it next to it. THAT is the resolution of of the 720p format in HD, EXACTLY. AND, it has a better color space than NTSC DV video. Reds can be redder, greens greener, etc. And it records to a standard, inexpensive DV tape, but it records a custom data format, HDV that is already an MPEG-2 stream, high quality and low data rate (thus doesn't require huge, fast, expensive drive arrays. More on those later). And it feeds your comptuer over a standard FireWIre/IEEE 1394/iLInk interface that you're already used to with DV. And it uses 30 bit color instead of 24 bit color...this means 1024 levels each of red, green, and blue as opposed to 256 in 24 bit systems. This means better, more subtle changes in light can be recorded with this format...and that compared to a 24 bit signal, more aggressive color correction can be done to it before it starts to artifact and look "bent" or "broken" from overly harsh color correction.

Sounds great, right? Well......sorta.

It's a one chip camera. What do you expect for 4 grand? And the glass is less than awesome. As the resolution gets higher, quality lenses start to really, REALLY matter. Oh, and it shoots 30 frames per second, not the film standard of 24. This 30 vs. 24 matter might be addressable through a clever algorithm of dropping certain frames...or more likely through a careful pixel morphing technique as used by certain high end post production software. (A report on this later).

But I think for documentary work and low end features, it'll be great. With some careful shooting and post production work, I think it can be somewhere between decent and nice.

It's also a SMALL one chip camera. Not the camea, the chip. The single chip they use is the same size as the 3 chip used in a Canon XL-1 DV camera...but from that same surface area of sensor it has to pick up 3 times as much detail...so that means 1/3 as much light falls on each pixel sensor....so that means it isn't very sensitive to light...so that means it doesn't shoot in available light as well as some DV cameras that cost about the same amount of money or less. So that means you are back to carefully lighting your sets as you would for a film shoot...which takes longer, which costs more, and takes more lights, which cost more, and probably requires more crew, which costs more.

But that new data format, HDV...it's an MPEG-2 stream but at a funny size compared to the MPEG-2 we're used to dealing with in DVDs etc. It comes with a little PC based editing app, but I wouldn't edit anything more than a 5 minute home movie with that....and only with big sharp pointy things, um, pointing at me. There is a piece of software called Aspect HD that will allow you to edit HD on Premiere Pro on a WinXP system...but it converts the HDV to a wavelet based codec...and you can't preview on an HD monitor (as far as I know from my demo in November)...and to go back to HD requires additional software. Aspect HD was $1200 last time I checked. What about Final Cut Pro, the indies favorite? There are some lumpy, inelegant solutions available that I'm playing with. I'll have a VERY detailed report on that, but I'm still working on that.

The problem is that while a some editing applications can handle MPEG-2, none (as of December, my last serious investigation into the matter, but more to follow) can natively edit this 1280x720p30 HDV format. You have to convert it to some other codec and work with that. For PCs, the best available option is Aspect HD that I'm aware of. For Macs, there is a $500 application that will let you convert the HDV native format into uncompressed video. Eh....not a convenient option.

It also doesn't have "real" timecode the way A/V professionals refer to it. DV has some similar issues of the lack of true timecode, hopefully workarounds will be found for HDV as well.

So the issues are these:

1.) Single chip camera - not great color resolution, poor light sensitivity. The JVC rep hinted that low cost, non CCD technology based imaging sensors were where they were headed (did he say LCOS? I thought that was a display tech? Check notes!). Sony has the only HD 3 CCD solution, and isn't going to sell it cheap to either camera purchasers or other OEM manufacturers. They make some beautiful gear, but is costs a heap.

2.) Lack of professional controls on the camera - this is a problem with this particular camera, not the HDV format itself. Just like DV, it's a format, not a camera. Better glass, better controls can make for great looking image.

3.) Non-standard data format - I expect this to be resolved within 2004. NAB is coming up in a few weeks, hopefully there will be some good news there. But presently the format is a pain to edit with. Might be worth it for early adopters to start familiarizing ourselves with it, though, and get used to what HDV can and can't do.

4.) Light sensitivity - with that small single chip, color separation isn't great, and light sensitivity is not as good as a lot of DV cameras out on the market. For those that ask whether they should get an XL-1S or a HD10U, it depends - what's more important, color fidelity and exposure lattitude, or super crisp sharp detail? Marketing reasons (with the attendant havoc wreaking it would inflict on existing pricing structures) are the reasons keeping somebody from producing an HD equivalent of a modern 3 CCD XL-1 or VX1000 for $10,000 to $20,000 as I see it. Those two cameras were the first two to really deliver on the high quality, low cost promise of DV. We need an equivalent for HDV. And while the HD10U is a big step in that direction....we aren't there yet. You can't reliably shoot available light indoors well with the HDV from what I've seen and heard. I've seen some good indoor concert footage, but I've not seen any great outdoor footage. But hey, this was 4 months after the camera was released. Skilled operators with experience, send me some samples!

5.) Time sampling/frame rate: This camera shoots 30 frames per second. So no fields, which is good for film applications, but it isn't 24 fps, which is what gets projected in theaters, so that's bad.Hmm. There may be post production solutions for this. But are they time and money reasonable? Research will tell. But I'm on it.

6.) Timecode - DV has some similar (but not same) issues and was dealt with. Hopefully, when the editing applications integrate the HDV codec they'll handle this as well. Hopefully. Somebody told me that since HDV is a MPEG-2 stream it can't carry timecode data. I don't know whether that precludes reading teh data off the deck over LANC, or if that kind of data can be sent over FireWire as well, or what. But that's just a blue sky guess on my part. No idea of the actual technical details....yet.

But, the advantages can be this: 3 times the resolution of DV with a better color space, 30 bit vs. 24 bit color, a highly compressed data stream to work with that doesn't require massive, expensive, high capacity arrays.

I'll have more to say on this camera and this format later. This was my stream of conciousness rant at 11:30 at night with one eye open.

When the DV format came out, there was a huge wave of excitement in the independent filmmaking community. Instead of having to rent an expensive BetaSP camera, indies could just buy an XL-1 or VX1000 and shoot their productions at their own schedule. Instant feedback, the ability to use available light, inexpensive DV editing stations all made for a very affordable and expedient process.

BUT....

when it came time to sell it as a movie, there were two large stumbling blocks:

1.) THE QUALITY

DV doesn't look as good as film. One can argue the semantics of the viability of a new medium to death, but from both the technical and casual observer's standpoint, DV doesn't do well in an auditorium. There can be field issues (we'll cover it later), the colors are smeary and vague, and the biggest sin is the general lack of detail and sharpness. I suspect there has been a hesitancy on the part of many of the people that buy films for distribution that something shot on video just doesn't have the "movie" feel to it, that it'll feel cheap when viewed on the big screen. I'm aware of only a smattering of films that have made it to the big screeen that were shot in large part with DV. Some documentaries have gotten away with it because the subject matter was so compelling, and shooting docs on film is horrifically expensive. Personally, I've only seen one film that I can recall that was a normal, narrative film that wasn't "experimental" or a documentary, or a portion of another film (28 Days Layer, Timecode). It was Pieces of April, a lovely little film shot on DV because that was the only way the writer/director could get it done. But films like that are the exceptions, not the usual business case. I believe shooting on DV creates a barrier to the distributability of a movie.

2.) THE COST OF FILM TRANSFER

After saving all this money producing on DV, then there is the relatively staggering cost of transferring DV to film. $30K to $60K are the numbers the last time I checked (and I'll be checking again soon). The top notch places like SwissFX produce some of the nicest work...but cost the most. In order for the independent filmmaker to show their work in a festival, they had to pop for the cost of a film print, which right there could keep their movie from moving forward towards the possibility of it being seen and possibly picked up by a distributor.

SO HOW DOES HD HELP THIS?

1.) THE QUALITY

Well shot HD, such as on the high end HDCAM cameras, is gorgeous and detailed, with rich colors and tack sharp detail. YES, it looks different from film. But when converted to film and projected in a theater, it looks "good." People don't question the quality, the feel, the look, they are just watching a movie. The quality is there (if it's done right), and I think that as more movies are shot on HD they'll be able to not face the quality question - it looks good, the audiences will feel OK to pay $8 each to see it, end of argument.

2.) THE COST OF VIEWING (instead of projecting film at a festival)

Transferring from HD is a technically simpler process than transferring from DV. I haven't checked prices recently (don't worry, I will soon), but before too long it should be a more commoditizable process. It's less technically daunting, so hopefully more facilities will start offering it as service and drive the price down further.

Plus, if you had to, you can show it to someone on an HDTV on a D-VHS deck. These decks were $2000 when they came out a few years ago, but I was out at Fry's Electronics a week or two ago and they told me they were expecting to have $300 to $400 record/playback decks available in the next few weeks. Converting HD to HDV requires some third party software, but the ability to show your work in high resolution for $1000 worth of reuseable hardware and software is a very, very nice thing.

But more important than that in the short term, more and more film festivals are projecting digitally, allowing for these movies to be shown WITHOUT having to be transferred to film first, AND they already are at a high resolution so they look good when projected.

I'm going to be very interested to see if any independent filmmakers can pull off a deal where they only have an HD version of the film when the deal is signed, and the distribution company either advances the cost of the film transfer out of the moviemaker's compensation, or the distribution company picks up the cost of the film transfer outright as part of the deal (plus then there is never any question about who owns the film print as the deal moves forward).

But I think HD based films will be viable for wide distribution of mainstream films and audiences, if the footage is prepared correctly, won't notice or care. And thus hopefully the people who buy your movie won't care either.

I think HD is going to be the Next Big Thing for independent filmmakers. In the past a few have been able to produce films with HD...but it has been well funded experimenters. In the mainstream, George Lucas and Austin's own Robert Rodriguez have produced popular, successful, mainstream films using HD technology. But they were already successful and chose HD for reasons that aren't always applicable for the independent filmmaker. True, instant feedback and already digitized footage are a boon to anybody, but both of these filmmakers could afford the high end gear costing hundreds of thousands of dollars. I'm interested in exploring what what can be done in the thousands of dollars scale. Just a couple of years ago Avid's systems to do uncompressed HD cost a couple of hundred thousand dollars...I'm looking to replicate most of that capability for about 1/20th of that cost now.

HD editing systems can now be had for under $10K...including hours worth of uncompressed real time storage. I've been doing my own research into this and will be publishing my results here.

Cameras have been mightily expensive in the past - how does $50K to $150K grab you - without lenses? New formats, such as HDV, show tremendous promise for the future. I think that HDV will do for HD for the DV format did for traditional NTSC & PAL video. Presently the only cameras offering this format aren't using it to it's full potential, but I think that will change over the next year or so. JVC makes a $4000 camera that shoots 1280x720 progressive frames...for under $4000.

The bulk of my inquiries will break down into three broad areas: cameras & shooting (my area of least expertise); editing station hardware/software; and digital post production workflow (my specialty). I'll be doing in depth reports and analysis on how to streamline processes, minimize costs and maximize quality in the areas of editing systems, storage for editing systems, monitoring options, and especially workflow methodology.

My recent research has led me to find new ways to dramatically cut costs, open previously shut doors in the film industry, and allow creative options for the independent filmmaker that were not financially feasible in the past.

So who the hell am I? My name is Mike Curtis, I've been pushing pixels and watching progress bars for about 15 to 20 years. I've been focusing on digital post production and effects work for the past 10 years or so, doing work for companies such as Ford, Compaq, HP, IBM, etc. etc. All of this work has been done on Macs and PCs - I came at this from the lower end of the hardware spectrum, and have made a career out of staying 2 to 3 years ahead of what my clients could do inhouse.

So keep reading as I post new content, and I'll be digging into all this stuff.

Enjoy, and feel free to comment, suggest, or throw the BS flag on the field if you think I'm full of it.

Pinnacle Systems Inc. announced on Tuesday that CinéWave 4.5 is now available. It's the only standard definition (SD) and high definition (HD) solution full qualified by Apple for use with Final Cut Pro 4, and the latest version features new real-time HD capabilities optimized for use on the Power Mac G5, full support for Apple RT Extreme, desktop mirroring for use with multiple programs such as Apple's Shake, and more. Pinnacle has also added a 66MHz bus interface on its PCI-X compatible TARGA Ciné Engine video capture card, resulting in faster transfers. [MacCentral]

Welcome all! This is my blog to share my latest thoughts, research, rants, etc. on utilizing HD for independent filmmaking.