Chase Jarvis TECH: Complete Workflow and Backup for Photo + Video

This may well be the most important behind-the-scenes video we’ve made to date. Not because it’s fancy or sexy, but because it covers arguably the most essential information on a set of topics that every photo and video person should understand: workflow, storage and backup of your precious images. This video covers all the ins and outs, the theory and the details of our complete photo and video workflow from capture to archive and everything in between. So whether you’re a seasoned pro, an aspiring amateur, or just starting out in photography or video we’ve worked hard to make this worth your time.

While there is no “right” digital workflow, ours has been shaped in the professional environment over the past 10 years. More importantly the techniques we use, the gear, the methodology, everything discussed herein been time tested and entirely scalable depending on budget, experience and where you’re at personally in photography and video, wheter you’ve got millions of images or just hundreds. Ultimately, this is a template, a summary of our experience piled into one video and one blog post. Photography, video, and filmmaking are my passions, but they’re also my livelihood, so–as you’d imagine–I take workflow, storage and backup solutions very seriously.

There are, of course, a number of ways to do this stuff, but several key concepts remain consistent no matter what your level of experience. In recent years, workflow, storage and backup has probably been the most requested topic I’ve been asked to cover…and rightly so. It’s important. While I generally don’t actively ask, I’m going with a different approach on this one. Since I’m hoping to re-ignite the dialogue on this entire subject, I would love for you to embed, share, retweet, re-post or share this with your friends. Arguably these principles–especially the parts on storage and backup of data, apply to everyone, not just us photogs. It’s my goal that this becomes a living reference, a launchpad for further discussions on this sorta nerdy–but important–topic. I fully expect there to be questions, counterpoints, and alternative views. It’s all welcome. The comments section of this post will be another place for great information and discussion.

In addition to the video above, there’s a HUGE RESOURCE of written workflow and backup after the jump. You’ll find additional images, a downloadable pdf, even B&H links to all our workflow gear after the jump, so please dig in…

WORKFLOW. Here’s ours:

Here is a downloadable pdf of the above image for your later perusal. Enjoy.

STORAGE AND BACKUP theory. Some general theory behind backing up your work.

1. Make your work ORGANIZED. You should be able to easily navigate, save, and locate files an organized folder structure. For starters, I recommend what’s commonly called reverse-date naming, combined with some convenient, recognizable text. For example, if I shot images for an Apple campaign today, June 22, 2010, on my #1 Nikon D3s camera, I would rename the images using the convention 20100622_Apple_1_[camera file name] or similar and put them in a folder referencing that info as such. We use Aperture to rename our stuff, but there are lots of software options that do this for you. The important takeaway from this point is whatever you do, make it organized. That goes for files, folders and overall folder structure. Be sure to AVOID folder names like NEW PICTURES, or YESTERDAY’S PARTY.

2. Choose the right STORAGE MEDIUM. Use portable, external hard drives and–perhaps even RAIDed storage–at your home, office or studio to store your work. I do not use DVDs or CDs. There is all sorts of data that says use this or that or gold plated whatevers. I find that research incomplete and flawed. We use hard drives. And LOTS of them. We love G-Tech. They make great, affordable drives in a range of sizes, from 256G to full enterprise quality RAIDs. Note that we chose G-Tech from a variety of possible partners because of their high quality product and company commitment to the pro photo and video space. We now work with G-Tech helping test their gear and ideas, so consider that within this context, but be aware that this comes after having used nearly every manufacturer’s product at some point in the past . G-Tech is our choice, but you can feel reasonably comfortable using most any name brand hard drive. As a general rule, if you can afford it, purchase more storage than you think you’ll need. Also note that, relatively speaking, you’re purchasing according to economies of scale. Thus, a 200GB hard drive may cost $100, or $0.50/GB; whereas 500GB hard drive may cost $150, or $0.30/GB.

3. Keep a CLEAN COPY OF THE ORIGINAL DATA. Before uploading your images into some proprietary viewing software I recommend copying the original data to a sacred place, in a sound file structure, where it’s not altered. I strongly recommend this is a separate drive from your computer’s hard drive. Upload or copy images onto your computer’s hard drive or into your preferred viewing or editing software for manipulation or reference ONLY AFTER you have a clean copy saved (and never altered) somewhere safe.

4. Make it REDUNDANT. In order for your backup protocol to be effective, it’s absolutely crucial that your files be in at least two different locations as soon after creating the images as possible. Creating two copies of the original data is the most important step in backing up data. However unlikely, hard drives and memory cards do sometimes FAIL. Don’t subject yourself to having only one copy of your precious photos or videos. It’s not worth it.

5. Keep ‘em SEPARATE. Remember why you keep originals of your will in the bank’s safe-deposit box and copies at home? This is a similar concept. Now that you’ve got two separate copies, on separate drive, with the exact same data on them, do your best to keep them separated. Try keeping one at home and one at the office. Or what at your house and one at your moms. This is the most far-reaching component of the backup protocol and protects you from the more extreme events like theft or fire. Statistically, it’s unlikely that this will happen, however it’s the best way to truly protect yourself from catastrophic loss.

6. Use DILIGENCE. A backup strategy is only effective if you can maintain it. Even if you’re not a pro photographer, keeping extra copies of your files according to a well organized, established protocol will help keep your precious files safe for the long haul.

SOME STORAGE AND BACKUP SPECIFICS:

ON SITE PORTION OF THE SOLUTION: Our studio runs a network of many computers linked together at a hub which speaks directly to a central file serving computer. This “server” can be any computer really, in our case it’s 2 Apple X Serves. The server’s job is to retrieve files for the rest of the computers on the network. This server’s external hard drive is the focus of this post. In your case, this might be just a single drive, or a small RAID solution you can buy from most electronic stores. In our case, we upload all our data onto G-Speed FC-XL Raids by G-Tech. This is a giant hard drive (64TB) that has written data seamlessly over a number of different drives in an array. This is fancy terminology that basically means that the drives all sync together to act like one drive, but in reality they’re separate drives arranged in such a way that if one drive fails, the server can identify it and, upon replacing the defunct drive, re-create data that was on the dead drive. It circumvents the horror of all your data living on a single bulk hard drive and failing. By spreading the data over several drives, you’re minimizing your risks. If one drive fails, you’re covered; and theoretically, multiple separate drives are far less likely to die at the same time. Redundancy is the key. Whether you’ve got an enterprise quality solution, or even just a two hard drives daisy-chained together, make it redundant…

OFF SITE PORTION OF THE SOLUTION: Now, the RAID takes care of any on-site single drive failures. You’re backed up at the studio. But what about a fire? What if the entire building gets crushed in an earthquake, RAID and all? If you take your photos really seriously–professionally or as an advanced amateur, you should have at least one copy of your data at a secure location off site. In our case, we use raided G-Speed 8TB drives for all data. We do NOT recommend DVD’s or CD’s. They are more volatile than hard disks. The data for all jobs gets put directly onto these G-Speeds and gets archived off-site. Thus, we’re backed up in case of drive failure AND in case of a dramatic catastrophe. Generally speaking, we’re betting, as all backup systems do, that our redundancy measures will out perform even the most disastrous situations that occur.

OTHER DATA? Note: The above is our solution for the RAW photo data that is created in intense bursts of large piles of data (shoots), not usually a small daily trickle. All our images live in their original, unaltered state on the RAIDs and off site. But what about client work, adjusted image drafts, delivered images, post production in progress, invoices, production docs, etc, and all the other data that gets changed or updated on a day to day, “trickle” basis? We call this our LIVE (rhymes with hive) WORK and it’s handled in a slightly different manner. It still lives (in a separate partition) on our RAID, and thus had built-in, on-site redundancy, but to remedy the off-site portion of the equation we use Apple’s Time Machine software to write a copy of this portion of our data onto two hardrives that are in rotation, while one is on-site backing up the data every night, the other is offsite, secure. Then, every 7 days, we swap out the drives. What this accomplishes is important and based on the same basic principle of redundancy that we use for storing our raw photo data. If the studio gets firebombed, then we still have all our raw, untouched data offsite and also all our LIVE work offsite…the most we’d be out in such a catastrophe is 7 days worth of pixel pushing.

Also, we backup each and every workstation in our studio with its own, separate hard drive using Apple’s Time Machine. It writes a backup of your computer every hour for 24 hours and every day until the drive is full. As a rule of thumb, the size of the Time Machine drive should be twice the size of your computer’s hard drive. These backups are important if you have a computer hard drive failure OR if you errantly delete a file. Time Machine has a nifty interface where you can go back into various earlier ‘snap shots’ of your harddrive and find that file you accidentally deleted. Don’t rely on this as a strategy for saving data, but in a pinch this can be a life saver.

Lastly…THE GEAR LIST
We know you love it, so here ya go:

• Nikon D3S
• SanDisk 32GB Extreme Pro CF Card
• Apple MacBook Pro 17″
• SanDisk Firewire 800 Reader
• Apple Aperture 3
• G-Tech 256GB G-Drive Mini SSD
• Apple Final Cut Studio
• Apple MacPro Desktop Workstation
• G-Tech 4TB G-Safe
• Apple X Serve Servers
• G-Tech G-Speed FC XL RAID
• 8TB G-Speed ES
• Adobe CS5 Master Collection
• G-Tech 2TB G-Drive

Thanks very much for staying with me for this whole thing. Hope you find this to be a great resource. If so, please share it with your friends. Also, please feel free to subscribe to my blog, Facebook, Twitter and YouTube channels if you like anything you’ve read here. Feedback and dialogue is always welcome.