.comment-link {margin-left:.6em;}

High Definition Video for Independent Filmmakers
A How To Guide for Digital Filmmakers
Welcome all! This is my blog to share my latest research,
thoughts, etc. on utilizing HD for independent filmmaking.

YES, I am available for consulting
Contact me at mike@hdforindies.com

All content copyright 2004-2007 Mike Curtis.

Tuesday, March 21, 2006

Gotchas & fixes for working with HD in mixed Mac/PC workflow 

Notes on trouble working with HD in mixed Mac/PC environment

...so I get a call from a client of mine who I've consulted with on working with Varicam footage in the past. They want my advice on how to work with some HD footage.

They've shot some 24p Varicam footage (that's the DVCPRO HD based $60,000ish HD camcorder) and they need some help with workflow. I think they have Final Cut Pro HD on a G5, but they don't know how to use it. Hey, they want me to do it for them, I'll all in favor of that.

I assume they're going to cut it in Final Cut Pro.

And then, because the Spidey Sense tingles, I ask

"Whatcha gonna do with it?"

"Oh, we're going to do some After Effects with it." client says.

"On what platform?" I ask.

"On PC." they say.

Ding ding ding!

Turns out they have Premiere Pro 1.5 on a generic PC with no HD-SDI board, and they have 720p24 Varicam footage.

Suddenly, a simple FireWire capture gets a lot more complicated.

What they want me to do is capture HD on a Mac and deliver it to them for usage on a WinXP box for use in After Effects, and maintain highest possible quality.

Long story short, what I end up doing is capturing over HD-SDI into my BlackMagic Multibridge Extreme. As an interesting side note, I'd left my 9 pin deck control cable at somebody else's place I'd lent it to (wait, that isn't the interesting part, hang on), so I didn't have the cable handy.

What I ended up doing was changing the setup in the Log and Capture window to use HD-SDI for audio and video capture, but FireWire for deck control. Worked like a charm - not a problem.

Nothing is ever as easy as you think it'd be. After setting up the presets for 24p capture, I captured the whole 30 minutes worth of footage as one file (as per client request), but it kept coming out as 60p. After a bunch of testing including capturing snippets of video from various portions of the tape, I finally figured out that they'd recorded bars at the head of the tape at 60fps, but shot all their content at 24fps. Since my capture started in the section with bars, my settings were ignored and the footage captured at 60fps, since it started as 60fps on the tape (it is a bad, Bad, BAD thing to mix frame rates in a given capture, and in general it is Bad Working Practice to mix frame rates even on the same tape). After figuring it out, I started my capture after the bars and it worked fine.

Then on the second tape, there were a number of timecode breaks towards the end of the tape. Let me be clear - Time code breaks are the bane of an editor's existence. They are not the potholes but the flat tires on the highway of post. Everything has to come to a stop and you have to fix each one. (or perhaps the STDs, the Severe Tire Damage thingies that keep you from driving Out the In in a parking garage is a more apt metaphor - they CAUSE the flat tires). After it failed to capture as one big file a couple of times in a row, I had to go in and find out how many and where the time code breaks were and work around each and every one of them (or are time code breaks like land mines? I just don't like'em.) So it took longer to find and work around the time code breaks than it did to capture the other 25 minutes of footage (which worked fine and unattended up until that point).

After capturing the footage to the Apple 8 bit Uncompressed 4:2:2 codec (no point in using the 10 bit codec since the tape format is only 8 bit), I used Compressor to convert it to a good cross platform codec.

I used to use the BlackMagic codec for cross platform work, but it introduces a very noticeable shift in the luma (brightness) of the footage, so this time I tried the Sheer codec from Bitjazz. It is mathematically lossless QuickTime codec (exact match of the source material) that is usually around 40% smaller than the Apple or Blackmagic codec. Yeah, the files are smaller. Yeah, the video looks and is exactly the same. No, I don't know how it works, I just trust that it does.

(I tried to capture directly to Sheer, but ended up with half sized files (640x360). This is a known issue with BMD cards and Sheer, and Andreas Wittenstein, the developer of Sheer, is working on it.)

There is a free reader version of the codec downloadable for QuickTime (and an AVI version is under development) from the site at the link above for both Mac and Windows, thus it's cross platform.

But because they needed to read it on a PC, I used Cleaner to flatten the file (flattening gets rid of the resource forks - used to be a Mac QT file wouldn't read on a PC unless it was flattened. I haven't had to move footage cross platform in a while, so I went ahead and flattened it to be sure).

OK, so now it was time to get the file over to them. They'd sent me an MS-DOS formatted FireWire drive.

Again, I'm not sure if this is still the case, but it used to be that there was a 2GB file size limitation for MS-DOS volumes. At around 30 GB, obviously this wasn't going to work for the uncompressed Sheer files. So I reformatted the drive to be Mac OS Extended (Journalled) and told the client to use MacDrive, a PC utility that lets you mount, view, read and write files on a Mac formatted drive attached to a WinXP box. There's even a free time limited demo.

The client delivered the drive they had with a USB cable, even though the drive has FireWire 400 ports as well. Copying about 60GB of data over is time consuming, and USB 2.0 is about half or less of the speed of FireWire 400. So I used one of my own cables to speed up the process.

Got a call from the client just a little while ago - all is working well.

In the end, I spent several hours troubleshooting and doing this job. If all had gone smoothly, it should've taken about an hour and a half.

Lessons learned/observed:

1.) It's never going to be as easy as you think.
2.) Yes you CAN use FireWire for deck control for the Panasonic 1200A deck while capturing audio/video over HD-SDI
3.) Sheer works as a cross platform codec
4.) Never assume the client has done it right - EVERY job ALWAYS has a bunch of stuff go wrong. The only question is how many things go wrong on the average job - is it 3, 5, 9? (yes this is a repeat of #1, but it bears repeating)
5.) Never assume a post workflow is going to work as expected unless you personally have done it before, or someone you have good reason to TRUST has done EXACTLY the same thing before with the exact same software and gear and it has worked for them.
6.) Always ask a zillion questions when you have to integrate with somebody else's post process. Get incredibly specific and make no assumptions - you have Software X? What version? (people have amazingly out of date software all too often...I'm still on Photoshop CS #1, not CS 2). You have a BMD/AJA card? Which one? What drivers? (Lots of PC stuff has lagged getting 720p24 support - 720p60 might be supported initially, but 720p24 often came later). So forth and so on. You might have 9 steps in your post process (or more), so be sure that there are no breaks in your chain. It only takes on weak link to fail, or cause a major workaround that eats a lot of time and budget.
7.) ...thus I try to ALWAYS charge by the hour rather than flat rate.

: )

-mike
Comments:
Hey Mike... Is the Blackmagic codec luma shift a known issue? Do you have a link to more info about it? I've been rendering my finals for a feature I'm doing to the BM 10bit codec and your post gave me the willies! Glad that your back!
 
Yo, Mike! The bars from a Varicam ALWAYS record at 60p and no flagged frames. You cannot capture bars at 24p. Them's the breaks. Also to mdotstrange, unless the blackmagic codec is RGB, you will have the luma shifts when working with that codec in FCP and then going to AE and back. It is well known (don't know about documented). If you do not want luma shifts and you know you are going to be working in AE after capturing in FCP, capture to an RGB codec and don't forget to turn on RGB rendering in FCP. I know that AJA provides a 10 bit RGB codec, not sure about BM Decklink. I wish there was a fix for this, and I guess the easiest would be for Apple to implement a 'capturable' native RGB codec, but YUV is supposedly 'more efficient'. I wish Apple would let people develop a hardware accelerated card with a codec akin to Media100 (dare I say it). It would make this process so much easier. Also, if you render to an animation out of AE, you won't see the shift, but your colors will still be kinda funky as they will have been translated from YUV to RGB.

--Jeremy
 
the day will come when we will see no time code breaks...

it is near...

=)
 
Thanks Jeremy! My material was never captured as they are mixed stopmotion/cg animation clips comped and rendered out of AE as 10bit Blackmagic .mov's... So I'll watch out for the YUV/RGB color funkiness, when I do my grading I'm gonna use the Decklink-->HDlink--->23" Cinema Display to display my AE timeline as I tweak the colors and levels...and pray that it'll all be ok.
 
I've seen this luma shift thing as a slight picture darkening even when doing conversions. Like converting from Uncompressed HD Quicktime (YCbCr not RGB) to DVCProHD Quicktime.

Is clearly visible when you open both Quicktimes side by side.
 
thanks for posting the step by step of the consultation... people benefit from understanding the business side as well as the technical. Glad you're back and great work on the site overall - Chris Carota
 
Hi Mike
You know you can just give the 8 bit Apple compressed files to your client because it is binary compatible with BMD codec - and didnt introduce luma shift (PP 2.0)

I use it for garding in FT

-Kaspar
 
Kaspar - the question would be whether it was interchangeable with QT on the PC side - but that is an excellent idea...using with FTHD is one thing, using with Windows After Effects is another....but that's a good idea.

-mike
 
PP 2.0 meaning Primere Pro 2.0 on a PC the BMD codec can open the apple codec just fine.

-Kaspar
 
aah right now I rember that latest AE crashed on 8bit BMD codec be it apple or straight BMD

-Kaspar
 
One more thing before I stop trashing your blog ;)

when using BMD capture utilty then I seem the get full rez with sheer - I will go get and HDV deck to make sure this works.

I have live capture planned on fryday (from sony F750) and need to test this myself as my sonnet card will not arrive by that time I will have to make due with only two disk stripe (raptor 150) and it is not enugh for 10 bit capture UC - but sheer seems to be handeling it

-Kaspar
 
Hey guys, why not just capture the stuff into the Mac DVCPRO HD, and then batch export out into the PC compatible codec of choice?

Seems like a lot of hoopes and expense for something that seems pretty easy. I'd like to hear what I'm missing.

I suppose for overcranked stuff, you'd have to download the panasonic plugin and render those out separately as there's no batch function.
 
Hi Mike,

Thanks loads for taking the time to walk us through all this hard-earned knowledge

One small thing jumped out at me though:

"USB 2.0 is about half or less of the speed of FireWire 400"

USB2 is rated at 480Mbps and FireWire 400 is 400Mbps. In my experience, USB2.0 provides very fast file transfers. On what are you basing your statement that USB2 is half as fast as FW?
 
Regarding the timecode breaks: Did you have your preferences set to: Make New Clip on timecode break? I've found this to work perfectly for me, and have checked the auto created clips for tc accuracy--it is perfect. It is slow, but FCP will jog back find the break and create a safe handle for recapture around the break, then recapture and continue on. You'll find auto created clips based on your logged name in you capture bin. I've had tapes with over twenty tc breaks. FCP handled them perfectly. What a huge time - and grief - saver!
 
Tony - yes I did have set to make new clip, it made several and seemed to get lost at some point. Perhaps I should have just waited it out.

Jack Kelly - USB 2.0 on MACS is slow - Apple's fault probably, but that's the reality I was dealing with at the moment. MacBook Pros are faster than PowerBooks and G5s, check barefeats.com for a story on that.

Anonymous about capture FireWire and export - that's basically what I did, expect captured full raster instead of via FireWire (1280 not 960 wide) so I wouldn't have to fool with fixing aspect during export.

Kaspar - that is all verrrrrrry interesting, drop me an email and let's chat - I think we both have knowledge the other wants on this stuff.

Also, as to Apple codec - so you're saying Apple codec was readable as BMD codec on a PC with After Effects 6.5 but not 7? Is that the case?

-mike
 
I had some issues capturing Varicam Footage (all shot 24p) via firewire with a 1200a ...I got, not TC break errors, but Cadence Break errors. Switching to HD-SDI and 9 pin control on a Kona 2 solved this. I let the Kona transcode to DVCPro HD.

I wonder how robust the FW device control is on the 1200a...
 
Mike, regarding your statement "I wouldn't have to fool with fixing aspect during export." Is there a quaility hit that it takes? I have done batch exports of DVCPro HD 720, to the Animation coded and told Final Cut/Quicktime to resize to 1920x1080. It looked great, and the aspect ratio looked correct. Am I missing it by a few pixels?
 
It's interesting how this seems entirely opposite to the experiences of London VFX artists.

I can receive, edit, and produce, for Online (broadcast) standard production - video from cam to PC without issue, as soon as the QT format becomes involved Apples desire for exclusivity makes the entire process twice as difficult... even using fully uncompressed formats Mac's insist on shoving in code that isn't necessary, and cannot open even uncompressed AVI - let alone internet-standard codecs... it's the equivalent of not being able to open a JPG.

Between apples price monopoly and the old myths of Windows crashes... Macs are fast becoming a thing of the past in Soho Post houses (London).

As an ironic side note, of the 3 systems (2 PC one MAC) involved in the production, the Mac crashed 6 times compared to the PC's combined at 2...

Basically the platform war is over.
 
Opinions and experiences vary depending on a number of factors.

I think the format war is over in terms of pick what works best for what you need, and in many instances either can work fine.

I run 5 Macs non-stop and very rarely have a hard crash/lockup, and even that is running a lot of alpha/beta/experimental code.

-mike
 
Post a Comment


Links to this post:

Create a Link

This page is powered by Blogger. Isn't yours?

Listed on BlogShares