So Seattle weather went from 50 degrees to 85 degrees overnight Friday and we all went from shivering to sweating! It's too hot to be sitting outside so we're both sitting inside getting a little work done. Well, I should really say 'work' as neither of us are actually doing anything productive for the business. Both of us are feverishly scanning.
We've got the Memorial for Kimberly's Dad (see here) coming up next weekend in Chicago so Kimberly's putting together a slide-show of his life. This involves scanning a bunch of very old photos, negatives, and slides and then laboriously touching them up to remove all the evidence of the ravages of time – dust, scratches, discoloration from old paper and mounts when acid-free wasn't the norm. After scanning she's using software called Adobe Elements which can do *incredible* things to restore images.
Many people say that if your house burns down, the only *really* irreplacable things are photos – everything else is just stuff. A few months ago I started to realize that between the two of us, we have an awful lot of film photos – for instance, Kimberly has literally more than 10000 slides from dive trips over the last 10 years – if something were to happen, that's a lot of memories to lose in one go (we estimate we've got 30000 film frames between us).
So – I bought a combo slide/negative scanner. I did lots of research before deciding on the Nikon Super CoolScan 5000ED – a little pricey but the reviews seem to justify the price. I've mostly scanned old (20-50 years) slides and negatives so far and the software the Nikon has to automatically put color back and remove all the imperfections is again just *incredible* with the results it gets. Now that I know the scanner is really top-notch, I've picked up the SF-210 Slide Feeder so I can load 50 of Kimberly's slides at one time and walk away for a few hours. Still – I'm looking at months and months of having the scanner buzzing away next to me while I work.
What's the point of this blog post then? Well, it's a little rambling but after Kimberly's recent corruption nightmares (see here) I started thinking a lot about making sure we had backups of everything we think is important. I realized that not all the data we want to preserve is already in digital format – which makes it impossible to just backup (there's no way to just make a quick copy of negatives). I'm sure a lot of you out there reading this are just like us – you've got a bunch of pre-digital photos that are slowly degrading and need to be scanned to be preserved – and may already be embarked on a months-long or years-long effort to scan them all.
Apart from the realization that I need to convert all this stuff to digital data to allow backing it up, the question then becomes – how can I be sure that I *really* have a backup of it all in the event of a disaster? Here are the options:
- Multiple copies of the data on different hard-drives
- Copies of the data on DVDs/USB-drives in a fire-safe
- Copies of the data on DVDs/drives in someone else's house
- Copies of the data on DVDs/drives in a safe-deposit box
- Copies of the data in the 'cloud' somewhere
If I'm really paranoid I'd probably do all of #1 through #4 – and given our experiences over the last few months, I'm sure that's what I'll end up doing!
But should I go with DVDs or hard-drives? Kimberly and I both have 1TB external Maxtor hard-drives that either have failed or show signs of failing (there's a class action lawsuit against Maxtor as I type). We both have multiple 250GB Western Digital USB drives that we travel with – 9 in total when we're together! However hard-drives aren't infallible at all – as Kimberly's in-flight corruption experience (for which I was unjustly blamed :-)) showed us. So what about DVDs? At 9GB each maximum, and with me scanning at 17.8MB per frame for say, 30000 frames, that would be 58 DVDs (to store a total of 521GB of data). Wow! And that's not even including the digital photos we have – Kimberly just reminded me that she took 6000 alone on our drive trip to Indonesia over Christmas 2006.
So it quickly gets a little overwhelming to think about and plan for. However, without any planning and forethought, if a disaster were to happen, we'd lose all our photos.
Same goes for business data in a database – without any planning, without any backups, you lose the lot in the event of a disaster.
Cheers
PS Kimberly just posted a little follow-up (see here) with a FANTASTIC image of her Grandfather sitting on the P-51 that he flew while a fighter-pilot during World-War II.
5 thoughts on “Do backups apply to just digital data? And how many backups should you have?”
Hi Paul!
I can’t keep up with most of your posts, but this one I can relate to! I have so many digital files and so many of them represent some of people’s most important moments (wedding photos) that the nightmare you describe is the one I have all the time! I finally landed on a backup / redundancy plan a couple of years ago:
Connected to my computer I have several external HHD’s (I use Lacie FW800). I use 1TB drives now, but I used to use 500gb drives. I use an internal disk as my ‘working’ drive and buy a trio of externals every year as my ’20XX’ drive. I always have one external attached to my desktop and a sync is run 2x/day. About once a week I’ll swap out the one attached to my desktop with the one I keep in my fireproof safe (usually right after I download the CF cards from a wedding). About once a month I swap the one attached to my desktop with the one I keep off-site. Oh, and I never delete images from CF cards until I know that they have been written to 2 or more disks.
This works well for disaster recovery. I use Adobe Lightroom for image management… it’s a non-destructive program and keeps a history of changes, so it suffices as a historical backup.
My backup plan is more than what most people would need, but you can be assured that your wedding images will be safe for a long time :-)
– John
Paul,
Having read your post and Kimberly’s post on the recent drive failures (I feel your pain), I can follow your interest in backup and protection of personal data.
I have heard that the three rules of real estate are: location, location, and location. Similarly, I have also heard that the three rules of IT are: backup, backup, and backup. We as IT practitioners shouldn’t have to be told that, but we are often guilty of not practicing it in all situations or as often as should be done (especially with our own personal data – myself included). Your posts are a good wake-up call for greater awareness of the consequences and alternatives.
On a similar topic, I’m curious if you have explored or had any feedback on the concept of a flash hard disk – not a USB flash drive, but an internal bootable drive (especially for laptops) as a traditional hard disk replacement. They have been out a short while and are still a bit pricey, but are certainly more shock-resistant that hard disks, due to their all-electronic non-mechanical nature. I haven’t personally tried them, but as the price for flash memory in general continues to drop, I suspect that they will evolve from a niche into a mainstream option before long. While a hard drive in any environment can fail (desktop, server, laptop), the mobile nature of laptops offer more opportunities for “excessive shock events” than the more static environments of the server or desktop.
Besides the pricing issues (cost per unit storage), other issues I have heard of or have further questions on:
– Performance: Claims to have better random I/O performance than hard disks (due to no mechanical seek or latency time, and a relatively small access time per request), but poorer sequential I/O performance (no leveraging of adjacent requests – every request is treated as if it was random). Impact of database fragmentation on performance is supposed to be less, for reasons described above.
– Lifetime reliability issues: Flash memory is supposed to have a limit to rewriting a given cell / byte / block – somewhere in the 1 million range, which is a big difference from hard disks. Mechanisms (hardware?, software?) are supposedly in place to relocate rewritten data to different locations to “wear-balance” the rewrite lifetime of all locations within a device. Some of the pros and cons I have read on this issue are fuzzy – hard to sort out marketing hype from reality.
Regardless of the current downsides, I suspect that these issues (price, performance, lifetime reliability, etc.) will continue to be favorably addressed and provide better storage alternatives for many environments.
While music players (iPod, Zune, etc.) all started out as hard disk-based devices, they too face the same “rugged mobile environment” durability / reliability issues as described for laptops (if not worse). The fact that music players are offering progressively larger storage capacities using flash memory and in more models (albeit at a price / capacity premium over hard disk storage models) is another indication of this trend. I find it interesting that a consumer device like a music player and their PC sync process provides automatic device-level backup protection – a great starter protection process that should be supplemented by secondary backups to external hard disk(s) (to protect against a PC hard disk failure – I know, I’m “preaching to the choir!”).
I am interested in your thoughts on flash disks as a component of a better data storage / protection solution – more from an "outage avoidance" than an "outage response" perspective for certain environments.
Scott R.
Hey Scott – I love the idea of flash-based drives for laptops – especially after the problems that Kimberly’s had over the last few months. My only concern to date has been the rewrite limit that you mention above. Until I hear that this is no longer an issue – I’ll be steering clear of them in favor of multiple USB-based external drives (we travel with 8 of the little guys :-)
Thanks for the comment!
I’m almost as paranoide as you…
I use Mozybackup for desktops as well as the free cloudbackup thats comes with Norton 360.
For server stuff and pictures etc, Amazon s3, its insanely cheap to store GBs it gets pricy if you up load and download a lot so I backup to an xraid nas box and then monthly backup to S3. beware though their upload bandwidth is not great it takes about 30hrs to backup.