Back the F*#% Up!

The last few months has been quite a growth spurt in the new New York office. Relocating here was a big jump, but even more so setting up a proper digital asset management system. Its a work in progress at this point. I got a big kick in the ass this weekend, when I found out my good friend Fred Egan had his house burn to the ground. Luckily his office was left in tact and he was able to recover his photography equipment and images, but I think it's a good in my face warning that it could happen to anyone, including me at any time.

I've hired Jeff Holt, to come in and help me build, set up an IT infrastructure, and implement a back up system both on site and off site. The above sketch is a general outline of what we have in progress of set up and it's still evolving. It's very intricate. Designing an IT infrastructure is very custom to what you have an how much you have to spend, but after this great awakening with Fred, it's pushing me even more to get things in order, especially an off-site backup.

The main idea is to have a mirrored backup that is onsite and that are equal sizes, then to have the same data mirrored off site somewhere. Hard drives go bad completely at random, so a constant mirroring of data on-site is a necessity. In talking with an Apple IT consultant friend of mine, he swears by tape drive backups because they last at least 30 years. Backing up on to secondary hard drives is great if you have a method of spinning them up once a year to keep the liquid bearings from drying up. If you let them go 3-5 years, they dry and the HD is toast. They're meant to stay in use.

The trick is to build a system that is expandable and accessible. Obviously if you're serious about being a photographer, the more you shoot, the more your digital assets are going to grow. Now a days with the digital sensors getting larger and larger the RAW images take up quite a bit of space. I am also constantly needing to access images for portfolio uses, projects, books, website, blog posts etc. so having a system where you can easily access everything easily takes quite a bit of design.

The system I have set up is 2 x 8 bay enclosures in a J-Bod(individual drives) setup with 1TB drives in each bay totaling 16TB. Each bay from one enclosure is mirrored to the parallel bay in the second enclosure for the redundant onsite data. These are the archive drives where jobs are dropped once they are finished and archived. We have a 2 bay enclosure with a RAID 0 that acts as a production drive for jobs that are in progress. The archived jobs will then be backed up onto tape and sent offsite. This system is hovering $10K.

Creating a system that is customized to fit your needs and budget can be pretty tricky and techie. Jeff has set up a few consulting packages to design the right system for you if you need it. His packages go from over the phone consulting to in-house setup. For more info send him an email: Jeff Holt

Another part of backing up is HAVING insurance. Unfortunately for our friend Fred, he had no insurance which resulted in a total loss. Irresponsible? Maybe so, but shit happens. I know I go through life thinking these things will probably never happen to me, and knock on wood they don't, but this is a sure realization that they can happen to anyone any time. I know what it's like to be strapped for cash and not finding the funds to pay for insurance premiums, but this is definitely a hard reminder. Our friend Jory Cordy has set up a donation page if you feel so inclined to help Fred out as he lost his living possessions.

In regards to insurance, one thing to be careful of is that a lot of renters/home owners insurances won't cover your gear if it's for business use and it values over a certain amount. It's good to have separate policies. I have a basic renters policy with Liberty Mutual, and a Business policy with The Hartford Agency. http://www.thehartford.com Business insurance gives you liability coverage and and inland marine coverage on your gear world wide. The premium for the business insurance varies on where you live and how much gear value you want covered. It IS essential!

Comments

[Casey Templeton]

My team and I try to live by the mind set that "It doesn't exist until it exist in two places!" Thanks for sharing.

[claude etienne]

Hi Nick,

I discovered your blog about a month ago, and it's become one of my favorites. What do you think about less expensive options such as Drobo, and online backup solutions such as Photoshelter? Do you think they are sufficiently robust as a Digital Asset Management system? On another note I'm looking forward to your first book.

[Greer Rivera]

I need to get insurance! So sad it took Fred Egan's situation to get everyone's ass in gear!

[Terence]

Any reason why you left the two 8-bay towers as JBOD as opposed to using Raid 5?

[Dave Hodgkinson]

As someone who occasionally plays a sysadmin on TV, I have a deep, deep mistrust of tapes. Too often they've been a write-only medium and of relatively low capacity compared to disks.

There are two reasons to take a backup:

1. Recovering from an "oops I deleted a file"
2. Recovering from a disaster

Having a local backup mirror catches option 1.

Do you add more data in 24 hours than you can sync (with compression if your RAW files are not) offsite over, say a 1 or 2 megabit line?

Second site, more disks, rsync.

Covers option 2.

Hate tapes.

[nickonken]

@Claude I'm not a fan of Drobo. You have to have two of them for a true back up and if the internal hardware fails, you can't pull the drive and hook it up to recover because it's a proprietary unit. Online backup is hard when you're talking at least 4TB of storage. Bandwidth is a problem. Then if you need to recover the data, you have to download it all. It would take days/months.

@Terence Jbod is easier to catalog and I can pull drives out to store them. The catalog system will know what drive a specific shoot or image is on.

@Dave I'm not quite sure I understand your comment, but the tapes are for a third offsite backup. My apple consultant friend swears by them. If you only write once to them, they are supposed to last 30 years. Hard drives you have to spin up at least once a year to get them to last longer than 3-5 years. The tapes are meant to recover data if my office burns down. Not for day to day backup. That's what the mirrored HDs are for.

I don't add that much data every 24 hours, but when I do it's many gigs worth of data. Some of my recent shoots have reached up to 90GB. Too much for an internet connection.

[FieldsNellie34]

Every one understands that our life is very expensive, nevertheless different people need money for different stuff and not every person earns enough money. Thus to get some loan or just sba loan would be good solution.