HD For Indies

UPDATE - this article has been added to, heavily, towards the bottom, a three times already today talking about further post considerations - live capture issues, capture from tape issues, 24p on 60i issues in post, etc.)

Hey all -

so here's what's up (this is just logistics, no real data of serious interest - skip down for learnable stuff)

-we shot Friday through Sunday, reviewed Sat/Sun
-tore down on Monday, captured some more footage for Adam Wilt to take with, he took off
-packed up Monday and didn't drive off in the U-Haul until 8:30pm
-dropped by Neil Halloran's to return his gear that he had graciously loaned (and he himself helped out on Saturday)
-neighbors of his had dropped by and were enjoying wine and barbecue, I ended up showing them the stuff in the UHaul and I've sold the red chair that is Oh So Lovelly. Perhaps when he is sober he may not want it any more. : )
-in any case, came home and konked out
-unpacked the UHaul Tuesday, returned it
-now I have ALL of the computer stuff I own in original boxes or swaddled in padded wrap sitting in my living room, breakfast room, and studio
-I also have about 13 hard drives to return, and one RAID enclosure
-before I do that, I need to copy off all data from the questionable RAID and drives
-I need to consolidate all the footage in one place and start organizing it
-I need to then make a backup copy of that data
-then I can put my world back in order - I "broke up" an 8 drive RAID to make two four drive RAIDs
-THEN I can start organizing it all and getting it in proper shape
-THEN I can start doing meaningful analysis
-so it'll take me some time before I have footage analysis, like next week
-in the meantime, check out DVInfo.net's thread, and at some point Adam Wilt will have some stuff to say about it too.

MEANINGFUL DATA STARTS HERE:

Some observations from the shoot:

-plan further ahead than I did. For anyone else crazy enough to do a 6 Mac live camera shoot and have to buy, assemble, borrow, configure, etc. to get it working, the day before the shoot is camera capture testing, the day before that is setting up/hooking up/plugging in computer gear, the day before that is transport, the day before that is packing & padding and boxing up. Don't forget to make a packing list of everything you've taken to location, and if there is more than one person's gear involved, don't forget to label EVERY SINGLE LITTLE PIECE OF GEAR with a piece of tape with your initials or some other identifying mark that you care about getting back. It isn't that anyone is going to steal it, just that at the end of the shoot when everyone's dead tired and stuff is getting unplugged...who's is this?

-at last year's shoot, I had a little pad that each operator took notes in for timecode in, timecode out, settings, scene and take #, etc. This year we decided to centralize this data capture and Li was in charge of it, just one person. One thing that didn't happen was timecode in/out records for each camera for each shot - it would have been nice to have so that I'd know WHERE to go find a shot now that I'm capturing! My fault for not finding a way to incorporate that, either on the record sheets that Li was using, or giving each operator another pad to write scene/take/timecode in/timecode out on.

-along those lines, I forgot until towards the end a handy way to differentiate multiple cameras when all shooting the same thing - assign #s to those cameras, and at the beginning of the take, have people stick their hands in front of the lens with that many fingers held out. It doesn't have to be in focus, you just have to be able to count how many fiingers. Obviously, five is the max you can do with one hand. : )

-we were VERY LUCKY that everything worked as well as it did and we only had a few snafus

-the AJA HD-10A will only do 59.94 frame rates; the Multibridge Extreme can also so 50Hz frame rates, so plan accordingly UPDATE - THIS IS WRONG - apparently, I just didn't set it up correctly. Our unit was not labelled 50i/60i. I'll go back and look and double check, but I clearly didn't use it to it's fullest abilities. My bad.

-the AJA HD-10A is small, light, and easily taped to the sticks to hang underneath with cable strain relief; the MB-X is not

-the AJA HD-10A has a convenient HD-SDI passthrough that helps for other monitoring activities as well. The MB-X does too, and has a component analog out as well

-the issue of POSSIBLE (unproven as yet) performance differences on the MB-X for analog to digital conversion is being vigorously investigated - BlackMagic is aware of the potential but unproven issue but we need to table the issue until after NAB, they are getting ready to come from Australia to no time. They are adamant that there is no underperformance issue. I did happen to capture the 24p JVC GY-HD100U footage through both the MB-X and HD10A so I'll be able to do some analysis and see if I can tell the difference, either visually or via scopes etc. Further update - Grant Petty himself emailed to tell me they are using the exact same conversion chip as used in the HD10A, so there should be no significant differences.

-I found it very interesting that I was able to capture uncompressed 10 bit 1080i60 footage (160 MB/sec data rate) and 10 bit 720p60 (150 MB/sec) on 4 drive arrays without apparent problems. I'll have to analyze the footage for dropped frames, but in our capture tests when I had "Abort capture on dropped frames" for everybody's Mac, they did OK. Now, in order to do that, I had pre-tested all the arrays and partitioned them at the point where I guessed performance would fall below 200 MB/sec, and assigned the capture scratch to the faster of the two partitions. OH, and other huge point in that success - they were ALL FRESHLY INITIALIZED/CREATED ARRAYS. So there were ZERO files already on those arrays. And I don't mean that the arrays were emptied off and trash emptied, I mean RE-INITIALIZED. I've seen performance differences between emptied vs. re-initialized arrays. I'm not sure what voodoo is going on there, but it made a difference. So freshly stripe your arrays before mission critical, e.g. live, capture.

-uncompressed capture is trickier than it would first appear - what you get out of the cameras is NOT 24fps coming down the wire, be it HD-SDI or component analog.

What you get coming down the wire from these cameras, be in HD analog or HD-SDI

Panasonic HVX200 -
When shooting:
720p24 - you get 720p60 coming down the wire with 2:3:3:2 or 2:3:2:3 repeating pattern, not sure which at the moment, gotta look it up - that'll make life interesting in post...

720p60 - is 720p60 coming down the wire - whew! At least there it is 1:1

1080p24 - comes down the wire as 1080i60 with classic 3:2 pulldown, mixed fields. I dub this 1080p24on60

1080i60 - comes across as 1080i60, 1:1, so that the fields will match between live and tape capture
...so clearly a little research or documentation to be done here

This camera has no HD-SDI output, so we used either the Multibridge Extreme (I dub it MB-X for short) or an AJA HD-10A to convert the HD analog outputs to HD-SDI for input into either the MB-X, an AJA Kona2 card, or a DeckLink HD Pro Dual Link card (have to check notes to see exactly what used)

JVC GY-HD100U
When shooting:

720p24 - you get 720p60 coming down the wire, with either a 2:3:3:2 or 2:3:2:3 frame repeating pattern. Gotta research that to find out which. UPDATE-WRONG! It's a 1:1:1:2 pattern according to David of Cineform

720p30 mode - comes down the wire as 720p60, and I think you're actually getting 60 true fps down the wire, no frame repeats (I need to confirm that by examining footage, but that's what I recall from NAB last year at the demo). So that also has exposure ramifications - you're limited to the exposure times/shutter speeds of a 60p shooting system even though you're only recording 720p30 to tape.

This camera had no HD-SDI out, so we used an AJA HD10A converter to convert to HD-SDI for ingest on a BMD or AJA card.

Sony Z1U - we blew off Cineframe since it has been documented that it suxors (throws out half the vertical resolution), so we shot 50i and 60i instead. They come down the wire, predictably, as 1080i50 and 1080i60. I'll be messing around with some deinterlacing and retiming tools to see what kind of a 24p result I can get.

Since this camera has no HD-SDI outputs for uncompressed capture, we used an AJA HD10A converter for 60i footage, then the MB-X for 1080i50 shots. We didn't capture one series of 24p shots (we used 50i when others were shooting 24p) when we realized the AJA HD10A couldn't do 50 Hz (or perhaps it can and we didn't know how to switch it to do so - anyone know? We were too rushed on set to dig out the manual). We switched it with another capture station, I think the GY-HD100U, and henceforth used the MB-X to capture 1080i50 from Z1U.

Canon XL H1
When shooting:

1080-24F: even though it has a 24F (not 24p) mode, it does NOT send a 1080pSF signal down the HD-SDI as a Sony F900 might. Instead, it adds classic 3:2 pulldown, so you get 1080i60 coming down the wire, what I dub 1080p24on60 (acutally, 1080p24on60i would be more accurate, but since there is no 1080p60 in widespread use, except for the Sony SRW-1 decks, and possibly more at NAB 2006, I'm going to let it slide for now).

1080i60 - comes as normal 1080i60 down the HD-SDI wire

...but I can't recall if this was the camera that DIDN'T send audio down the HD-SDI. I'll have to check my notes and footage to verify.

Sony F350
When shooting:

1080p24 - surprise! Even though it has a true 24p mode, and HD-SDI, it STILL sends 1080i60 down the HD-SDI cable, not 1080pSF, even though it is a Sony CineAlta. Harrumph. Classic 3:2 pulldown added that has to be removed in post. I see this as a clear example of market segmentation, and it is the lack of these kinds of features that makes me a bit dour about seeing that "CineAlta" badge on the side. But is has HD-SDI, which the F330 does not (right? Too many camera facts in my head right now), so that's a big plus.

1080i60 - comes down the HD-SDI nice and clean as you'd expect.

Oh, and audio was included on HD-SDI - we checked.

Panasonic Varicam
When shooting:

720p24 - For this test, I had just installed the AJA Kona3 card that AJA was kind enough to loan me. It has a preset that specifically mentions Varicam for 720p23.98, so I used that to capture. We were so run and gun that I didn't have time to test as carefully as I would have liked. HOPEFULLY it detects the "flags" in the signal coming down the wire that mark original from duplicate frames (when shooting 24p, the camera is correctly sampling time at 24 times a second, but repeats frames on the HD-SDI signal, marking those that are originals rather than duplicates for extraction). But doing some quickie captures and playback it looked pretty good, I'll just need to double check. If I did something wrong, or the Varicam or Kona3 doesn't act or wasn't configured the way I wanted, my 24 frames a second may not be the 24 per second that I wanted, I might have been "off cadence" and have some kind of stuttery playback with incorrect frames showing (repeats instead of originals, or originals skipped). Again, I'll have to hook everything back up and play it back and compare tape to live capture to see if I did it right.

720p60 - OK, this one should be easy enough so long as I used the right preset - 720p60 shot, 720p60 coming down the HD-SDI.

Post Considerations for tape/P2/XDCAM HD

At this time, all I have to use if Final Cut Pro 5.0.4, I don't have the 5.1 update yet. So here are my capture options:

Panasonic HVX200
all formats - connect a FireWire cable to the camera with the P2 cards inside that contain the footage you want. File==>Import==>P2, and then you get a list of clips to highlight and click import. They come in with random file names, so you have to sift and add metadata. Slates essential!

JVC GY-HD100U
-connect via FireWire
-720p24 does NOT work with Final Cut Pro 5.0.4, and I've read no indication that v5.1 does any better. Use LumiereHD instead is my best advice, they have support for capture and playback from this camera
-720p30 (NOT 60, 60fps only comes out the analog ports) - HDV capture works just fine over FireWire at 720p30 with this camera. Can use Make New Clip on Start/Stop, HUGELY helpful

UPDATE-WRONG! It's a 1:1:1:2 pattern according to David of Cineform for 720p24. So for now, I just captured over the HD-SDI to uncompressed, I'll come back with LumiereHD if I can get a camera to work with - anybody in Austin area have one they can loan me for a day or so?

Sony Z1U
-connect FireWire cable
-boot camera as 50 or 60Hz as needed
-use FCP presets to capture, not a problem at all
-CineFrame, which we didn't use, comes in as 1080i60 with 3:2 pulldown

Canon XL H1
-connect FireWire
-1080i50 - captures fine over FireWire using 1080i50 HDV FCP preset
-1080i60 - captures fine over FireWire using 1080i60 HDV FCP preset
-1080-24F, however, does not - I captured over HD-SDI as 1080p24on60i, will have to remove pulldown in post

Sony F350 XDCAM HD
-since there is no XDCAM HD native support for this in either FCP 5.0.4 or 5.1, had to capture over HD-SDI.
1080p24 - comes in as 1080p24on60i, so gotta remove 3:2 pulldown in post, same challenges as live capture
1080i60 - comes in as 1080i60 over HD-SDI. Again, frustrating that codec support and drivers would let me use GigE or FireWire or USB on this camera, but I can't...yet.

Panasonic Varicam (rev H we used)
-use Panasonic 1200A deck, you CANNOT capture using FireWire from the camera. The camera lacks FireWire and 9 pin deck control, so any kind of controlled capture has to be done with a deck. The 1200A is inexpensive (as HD decks go, about $30K with HD-SDI and FireWire boards installed list price). FireWire is an OPTION on this deck, make sure it's installed if you're renting. I'm planning on some thorough testing - capture BlackMagic, capture AJA, capture FireWire, capture analog for fun too, and see if there are any differences. Anyway, here's what I expect is the usual route:
HD-SDI:
-connect HD-SDI and 9 pin deck control cables, or can even use FireWire for deck control (did it the other week myself so I can confirm this works)
-log and capture in a "normal" fashion for 720p60
-use the 720p23.98 preset for the 24p capture to remove redundant frames
FireWire:
-connect via FireWire, use presets for either 720p24 or 720p60 and it works just fine.

POST CONSIDERATIONS FOR LIVE CAPTURED FOOTAGE

Now that it has all been captured, I need to sit down and foodle with how to extract the 3:2 pulldown from the 1080p24on60i footage, or 2:3:3:2 or 2:3:2:3 frame repeats from the 720p24on60p footage. Since timecode does NOT get captured correctly when doing live capture this way (even if timecode were on the HD-SDI from the cameras that have built in HD-SDIs, and not all do include timecode on HD-SDI, FCP can't de-embed that timecode and read it), the usual system won't work.

Normally, when trying to remove 3:2 pulldown, you can count on the pattern (3:2 pulldown on 480i60 or 1080i60 or the 2:3:3:2 or 2:3:2:3 on 720p60) to start on either the :00 and :05 frame counts for 60i formats, or at least the :00 frame counts for 2:3:3:2 for 720p, or :00's and :05's for the 2:3:2:3 720p60 formats (See kids? Isn't this FUN!).

UPDATE-WRONG! JVC GY-HD100U for 720p24 - it's a 1:1:1:2 pattern according to David of Cineform, not 2:3:3:2 or 2:3:2:3

...BUUUUUUUUUUUUUUT since our captures are "wild" and the timecode for each capture starts at "00:00:00:00" regardless of what the timecode on the camera said....I am hoserated - each and every shot will have to be analyzed to figure out where the start of the pulldown/padding pattern is. NOT a recommended workflow for bulk work.

The only time 1080p24 capture has been relatively easy is when using an F900 that spits out 1080pSF down the HD-SDI (and by the way, pSF stands for, I think, progressive Segmented Frames).

So all my source captures of 160 MB/sec will be processed down to about 130 MB/sec after 3:2 pulldown is removed from 1080p24on60i. The 720p24on60 should shrink more considerably, down to around 60 MB/sec I think. But yet more huge files to store and manipulate.

Before the really nitty gritty comparisons can be done, I need to process ALL live footage down to their true 24p resolution for playback, and to add the correct metadata to all - so that I'll have source footage, 24p versions, 50i converted to 24p, etc. and be able to compare all this stuff. And have log notes and a database with all settings (iris, shutter, etc.).

That, my chilluns, is gonna take a while, and I don't even know if I'll be done with that stuff before I leave for NAB next Thursday. (I'm going for fun for two days, then the Digital Cinema Summit for two days, then the trade show madness begins. There's an extra twist about the trade show I'll share sometime before the show starts, too....)

OK, THERE! That ought to give you enough to think about until tomorrow's Next Exciting Episode...

-mike