Automate Your Backups

mooninitehd_20070204.jpg
There's a classic horror story that keeps me from sleeping at night sometimes. I've heard it told a few different ways. I've even told the story myself more than once, but Phil's version that he posted yesterday morning was one of the most frightening:

A couple weeks ago a flood hit my apartment/office area and soaked the desktop system, monitors, equipment *and* back up drive (along with a ton of other stuff) - luckily I have a daily back up on a Powerbook. But, of course the Powerbook decided to completely stop working while at our ETSY event before that could be backed up too. Zapping the PRAM revealed the hard drive failed, so the usual steps of Disk Util, TechTool and then finally drive removal and DiskWarrior were attempted - for the most part the drive seems completely dead - there might be a chance to recover some data under linux, or from a data recovery shop, but it's not looking good.

According his latest update, the backup drives dried out okay and appear to be working fine, so I guess that means he's managed to survive the perfect storm, but it got me thinking - how many of us ever keep a regular, daily backup in the first place? I've suffered several near-misses in the past, and I'm still guilty of not keeping good backups.

Never Again
So, February isn't too late for a new year's resolution. Don't go another day without your important files backed up. Let's sit down for 15 minutes, right now, and set up an automated backup system for ourselves. All you need is an external hard disk or a remote server with sufficient storage for a couple copies of your data. Based on Phil's story, you might want to situate your backup system on an elevated surface and not beneath any water pipes.

We're not focusing on a perfect backup solution here, with off-site, fire proof, vault storage. Don't let the nay-sayers stop you with the long list of things that can go wrong with a simple back-up solution, or explanations of how to do it the "right way". In 15 minutes you are going to be significantly more protected from data loss, and this will give you the time you need to relax and find a good price on your fire proof vault.

Focus On What's Critical
Don't worry about your gigs and gigs of MP3s and ripped DVDs. Find your work documents, your tax info, family photos, etc. Locate all the stuff you can't replace, the stuff that could make you contemplate terminal activity if it were to be lost. Move all this to a central location. I just keep everything important in subfolders of my desktop, but this could be your documents folder, or wherever. Your MP3s and video podcasts go elsewhere. You can think about how to back that stuff up when you've got the important stuff taken care of.

Windows Users - Get Cygwin
Cygwin is a unix-like environment for windows. It'll let you do all the cool things that your mac and linux buddies do out of the box.

Make and Test a Backup Script
We'll use rsync to do our backups for us. Edit a file called backup.sh and put something like this in it:
#!/bin/bash
rsync -rb /Users/username/Desktop/ /Volumes/External/desktopbackup/
rsync -rb /Users/username/Documents/ /Volumes/External/documentsbackup/

I'm on a Mac, so adjust your paths to match that of your system and the location of your important documents.

The -r flag is for recursive, so it'll copy everything from my Desktop and Documents folders, including all subfolders. The -b flag means backup; rsync will rename all the destination files before copying over the new files. If you are backing up to a network server, you may want to skip this option, since it means the files are completely copied on every backup. I like it, however, because if I accidentally mess up a file, I have an extra day to get it off of backup before it's irretrievable.

If you are copying your files to a remote backup server (ie. rsync -r sourcedir username@servername:serverpath), you need to make sure this can happen in an automated fashion without manual authorization. To do this, you need to add the source machine's public key to the remote machine's allowed host list. Read the ssh manpage or check out these instructions to see how to do this.

Automate It
Once you know your backup script is working correctly, set up cron to run it nightly. Make a file called backupcron, and add something like this to it (make sure to use tabs, not spaces):

#minute hour mday month wday sh
30 23 * * * sh /Users/username/backup.sh

That'll run your backup script at 11:30 every night. Just adjust the time and path to match your system and preferences. Then type the following to add this to your crontab:

crontab backupcron

You can type "crontab -l" to check that it's in cron correctly.

Verify It
You're almost done. Check the backup location tomorrow morning and verify that your backup ran correctly and that everything made it okay and there is sufficient space, etc. You should now be able to sleep soundly, knowing that your important data is finally safe... well, unless your backup server goes up in flames and then someone steals your laptop and your upstairs neighbor's bathtub falls through the ceiling, full of water, onto your desktop machine. But hey, lightning never strikes twice, right?

FYI, my machine finished backing itself up while I wrote this article. What are you waiting for? Get to it!


Recent Entries

Comments

Oldest comments listed first.

Posted by: super_J_dynamite on February 5, 2007 at 11:36 AM

My $0.02:

1) If you're only going to use Cygwin for rsync, it would make more sense just to find a version of rsync compiled for Win32. Scheduling can be done with Windows' native scheduler.

2) If you're backing up to a USB drive, you probably have the bandwidth to just copy everything over in it's entirety (instead of using rsync to only transmit deltas) using xcopy.


Posted by: s2156945 on February 5, 2007 at 3:08 PM

Data Loss is mostly from human error. You don't always notice your error straight away (See "ohnosecond").

Rdiff-backup is a smart combination of rsync with rdiff to efficient store both a mirror of your data and change-sets to go back in history. You can simply delete the oldest changesets to save space.

See http://www.nongnu.org/rdiff-backup/


Leave a comment


Subscribe to MAKE!Subscribe to MAKE Magazine!

Subscribe today, save 42% and get web access to MAKE free. MAKE Digital Edition is available only to subscribers.

$34.95 / 1 year
(4 Quarterly Issues)

Subscribe now


Void your warranty, violate a user agreement, fry a circuit, blow a fuse, poke an eye out. Make: The risk-takers, the doers, the makers of things... Welcome to Make: Online!


CRAFT Maker Shed Maker Faire MAKE television




Check out more videos from MAKE.

MZ_Jameco-RobotBuild_RR.gif
MZ_DIYMovieMaking-RR.gif
Maker SHED

Connect with MAKE

Be a MAKE fan on Facebook MAKE on Facebook
Visit our Facebook page and become a fan of MAKE!
MAKE on Twitter MAKE on Twitter
Follow our MAKE tweets!
MAKE Flickr Pool MAKE on Flickr
Join our MAKE Flickr Pool!
    make_tips on Twitter



    MAKE Archives

    Make: Money

    Make: Science Room
    Subscribe to MAKE Magazine!

    Make: Online editors and authors!

    Gareth BranwynGareth Branwyn
    Editor-in-Chief


    Phillip TorronePhillip Torrone
    Senior Editor
    | Web | Twitter


    Becky SternBecky Stern
    Associate Editor
    | AIM | Twitter


    Marc de VinckMarc de Vinck
    Contributing Writer
    | AIM | Twitter


    John ParkJohn Park
    Contributing Writer
    | Twitter


    Sean RaganSean Ragan
    Contributing Writer
    | Twitter


    Matt MetsMatt Mets
    Contributing Writer
    | AIM | Twitter


    Dale DoughertyDale Dougherty
    Editor & Publisher
    | Twitter


    Shawn ConnallyShawn Connally
    Managing Editor
    | Twitter


    Goli MohammadiGoli Mohammadi
    Associate Managing Editor

    Kip KayKip Kay
    Weekend Projects
    | AIM | Twitter


    Collin CunninghamCollin Cunningham
    Contributing Writer
    | AIM | Twitter

    Adam FlahertyAdam Flaherty
    Contributing Writer
    | AIM | Twitter


    John BaichtalJohn Baichtal
    Contributing Writer
    | AIM | Twitter



    More contributors: Mark Frauenfelder (Editor-in-Chief, MAKE magazine), Kipp Bradford (Technical Consultant/Writer), Chris Connors (Education), Diana Eng (Guest Author), Peter Horvath (Intern), Brian Jepson (O'Reilly Media), Robert Bruce Thompson (Science Room)

    Suggest a Site!

    Advertise here with FM.

    Why advertise on MAKE?
    Read what folks are saying about us!

    Click here to advertise on MAKE!



    Current Podcast

    itunesdl.gif Weekend Project: Pinhole Panoramic Camera A sturdy, panoramic design pinhole camera that uses 120 film. Thanks go to Ross Orr for the original article in MAKE Volume 09. To download The Pinhole Panoramic Camera video video click here and subscribe in iTunes. Check out... More...

    Get the Make: Online sent via email
    Enter your email to receive Make: Online each day:



    Sign up for the Make: Newsletter

    Our Make: Newsletter covers news from maker Media, has original columns, Shed deals, and more! You can also read the archives of past issues.


     



    MAKE Fascination video series brought to you by Dow

    Make: Education
    MAKE: en EspaƱol MAKE: Japan
    Important please read


    Subscribe to MAKE Magazine!

    Recent Posts from the Craft: Blog