.comment-link {margin-left:.6em;}

High Definition Video for Independent Filmmakers
A How To Guide for Digital Filmmakers
Welcome all! This is my blog to share my latest research,
thoughts, etc. on utilizing HD for independent filmmaking.

YES, I am available for consulting
Contact me at mike@hdforindies.com

All content copyright 2004-2007 Mike Curtis.

Friday, January 18, 2008

On Storage...and our digital memories. 

So I'm off email for a bit whilst backing up my laptop drive - I am writing a bunch of stuff about storage for another publication (more on that soon), and I knew was short on drive space, but didn't realize HOW short - I went and looked, and had about ONE gigabyte free.

When you have more RAM than free disc space, it is time to free up some disc space. I did the usual digging to find Pfhat Media loitering about - did I have any full DVD rips sitting around, or collosal uncompressed video, or video projects sitting on the drive?

Nope.

Then I went looking to see if I'd done a Full Install with Final Cut Studio 2 (if you install All In, it is 55 (!) GB).

Nope - I'd already cleaned house there.

Then I went into my Projects folder (I always have a folder at the root level of the data drive (which has to be boot drive on laptop) to see if I had any fat folders I could trash or archive of a client project that was gone. Nope - Projects was all of 1.83GB - just a lot of little bitty stuff.

Then I went looking in my User folder, and found the culprits - Pictures (which encompasses years of tradeshow pictures and videos) was 35GB, Music (which of course includes my cutdown/favorite/most recently purchased iTunes Library) was 16GB. So that's over 50GB of personal files right there. When you factor in the size of the OS and Applications and Libraries folders...that makes for a very full 120GB drive (formats to about 112GB).

So I'm now installing my 3rd (or is it fourth?) hard drive in the same laptop - I started with an 80, I can't recall whether it died and was replaced with same or I immediately upgraded to this 120GB, and now I'm bumping up to a 200GB. THAT should hold me for a bit.

And yeah, I AM getting one of those Time Capsule doohickeys, it was time to upgrade my router anyway.

I was also on the cusp of getting a LARGE capacity tape backup system, I need to take a hard look at all that uncompressed HD footage and decide whether it really needs to be kept or not at this point.

This is all to say...we keep accumulating more and more stuff. In the big philosophical sense, our memories are increasingly digital these days, and a hard drive failure can be catastrophic in terms of vacation photos, organized/ripped music (legal and otherwise), etc. Back Up Your Stuff!

With big tape backup systems getting small studio price feasible, and Time Machine/Time Capsule out there, there's no excuse not to have one of these systems.

On a more micro scale, keep in mind that your digital memories are ever growing, and managing them can become more complex. I have 2 G4s, 3 G5s, an Octo Mac (1st gen, not the newbies), and a MacBook in the house...and they all have, or have had, an iPhoto Library on them. Figuring out how to get all that in one place, with all the metadata intact, is something I haven't figured out quite yet. How to get the original and modified pictures, with ratings, notes, keywords, etc., all in one place, from 6 or 7 different Macs, seems pretty non-trivial and I haven't found good tools to do so yet. Burning to a DVD and importing seems to lose a lot of stuff I want - anybody got any good experiences to share on that front? Since they called it iLife '08 when it came out last summer, I'll take that as a pretty good indication we're not seeing a new iLife until MWSF next year - so no joy/help/clue forthcoming from Apple as far as I can tell.

They've done a good job at helping folks organize data up to this point. Maybe Back To My Mac can be of some help, once I finally move my laptop to Leopard (the full drive one reason I hadn't made the switch yet).

OK, enough rambling for now...there's a broader thing to be discussed about long term digital archives, bit rot, graceful organic vs non-graceful binary failure, etc...but you just hink on that and I'll come back to it later.

I gotsta go get on a plane....

Labels: , ,

Comments:
Mike, just a note about syncing iPhoto libraries...personally I like and use Aperture for my photo needs and if you decide that's a better way to go, you can always create multiple Aperture libraries on each of your machines, but have one library 'to rule them all' and simply merge the smaller ones with the larger one on a semi-regular basis. The steps can be found here: http://whatisnoise.com/2007/12/howto-merge-aperture-libraries.html

good luck
 
With regards to Final Cut Studio 2, how much of the space is taken up by the apps and how much is taken up by the Apple Loops and Motion Libraries.
 
leland - it is 90% library stuff - just the apps fit on one DVD....

-mike
 
I use the app WhatSize (http://www.id-design.com/software/whatsize/) to scan my drives from time to time and report where all the space went (because it's always magically going away it seems). I am nearly always surprised at what the application finds. About two days ago I managed to free up about 70 GB of no longer necessary (and already backed-up) project files.

It's also a good way to dig into your FCP Studio templates and remove the PAL stuff if you are so inclined.

Matt Jeppsen
www.FreshDV.com
 
Yes I second WhatSize. It does an excellent job in an easy to use interface.

I have having the same problem but with my Dual 2.7 G5 tower. And the problem is I have had issues trying to get SATA II drives to work in it, so I am worried about picking one up and having it not work, but the 250GB internal is almost completely full. I want to get a 500 and use carbon copy cloner, and then replace the original
 
The command "df" will show all partition's free space.

"Real men don't do backups. They just upload their stuff to the internet"
Linus Torvalds
 
I had a dual G5 2.7 till recently - the only drives that had issues were from Western Digital, but removing a jumper took care of that. I usually buy Seagate/Hitachi and have no issues with those to date.
 
i just last week did some digital re-organization myself. i dont have quite as much data as you (my clients wont pay for uncompressed) and only 2 real computers. a 17" 1.5ghz powerbook g4 and a first gen 17" macbook pro. then for storage i use a 500 gig caldigit firewire vr drive, and 3 or 4 misc other firewire drives. my powerbooks HD went bad around xmass time so i dropped a 160 in. this has now become my on the road, in the car capture solution for dv and hdv. pretty handy i might add. but this had prompted me to organize all the data i had scattered across both computers and all those drives. i was able to get rid of about 100 gigs of duplicate files, 60 gigs of duplicate music and 10 gigs of duplicate pictures. and i cant even calculate all the disconnected render files i found to projects i never remember starting. i even tried using my airport extreme base station (the one without the hard drive your about to get) to plug a usb drive into to pass files around, but boy is that prohibitively slow. all i can recommend that for is families passing mp3s back and forth. i tried to figure out a system for passing large files (like captured footage) back and forth quickly, but the only thing that seems to work fast and consistently is actually dismounting the caldigit drive and then plugging it into the other computer. theirs got to be a sexier year 2008 mac way. i have just yet to discover it.
 
> This is all to say...we keep accumulating more and more stuff. In the big
> philosophical sense, our memories are increasingly digital these days, and a
> hard drive failure can be catastrophic in terms of vacation photos,
> organized/ripped music (legal and otherwise), etc. Back Up Your Stuff!

I echoed your sentiments a couple weeks ago:
http://techthoughts.org/2008/01/04/drobo-worth-the-hype/

> With big tape backup systems getting small studio price feasible, and Time
> Machine/Time Capsule out there, there's no excuse not to have one of these
> systems.

With the low cost of hard drives, tape backup systems really have little appeal. How fast can you restore 750 GB from tape, versus 750 GB from a plug-n-play external hard drive?

> On a more micro scale, keep in mind that your digital memories are ever
> growing, and managing them can become more complex. I have 2 G4s, 3 G5s, an
> Octo Mac (1st gen, not the newbies), and a MacBook in the house...and they all
> have, or have had, an iPhoto Library on them. Figuring out how to get all that
> in one place, with all the metadata intact, is something I haven't figured out
> quite yet. How to get the original and modified pictures, with ratings, notes,
> keywords, etc., all in one place, from 6 or 7 different Macs, seems pretty
> non-trivial and I haven't found good tools to do so yet.

Actually, it's pretty easy to do if you point them all at one Networked Attached Storage system (NAS). iTunes, iPhoto, ec libraries can be shared across computers, and machines. The NAS I highlight in my blog is also UPnP aware so a network device (not a computer) can easily access photos and music and movies from the server.

Anthony Burokas
TechThoughts.org
 
The one I use to see what's eating up space on my hard drive is called 'Disk Inventory X' and is nice because it gives you a visual representation of your hard drive and what is eating up what. http://www.derlien.com/. and its free!
 
Post a Comment


Links to this post:

Create a Link

This page is powered by Blogger. Isn't yours?

Listed on BlogShares