Simple 'cloud' backup with Amazon S3

posted in tech by jon on 2009-11-01

I've had an Amazon S3 account set up for some time, and kept meaning to do something with it. Tonight I had a few free minutes and decided to take a look around to see what kind of tools are available for using it.

Coalescing a few things I found on the web (plus a man page or two), here's how to simply set up backups using s3fs

(these steps assume that you're using ubuntu, that you've already set up fuse, and that you already have an S3 account)

First, install a a few prerequisites for s3fs 

sudo apt-get install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev

Second, download the s3 source using the link above and untar it 

user@example.com:/tmp $ curl http://s3fs.googlecode.com/files/s3fs-r177-source.tar.gz|tar -xzf -
user@example.com:/tmp $ cd s3fs

Third, compile it and put the resulting binary somewhere sensible 

user@example.com:/tmp/s3fs $ make && mv s3fs /usr/local/sbin

Next, create /etc/passwd-s3fs, containing your AWS keypair 

echo [ACCESS_KEY]:[SECRET_KEY] >/etc/passwd-s3fs

Now, you can execute 's3fs ' to mount an S3 bucket to the given path. Rather than doing this manually though, we'd like to have it done automatically at boot.

Happily, the 'mount' binary deals with unrecognized filesystem types by searching for a separate mount program at /sbin/mount.TYPE

Thus, to have your s3 bucket (called 'my-backup') mounted at startup, create a symlink from /usr/local/sbin/s3fs to /sbin/mount.s3fs 

ln -sf /usr/local/sbin/s3fs /sbin/mount.s3fs

and add an appropriate line to fstab 

echo "my-backup /backup s3fs use_cache=/tmp 0 0" >>/etc/fstab

Now, issuing 'mount /backup' should automatically mount your S3 bucket, and it should automatically be mounted at each boot.

Once this is working correctly, having offsite backups is as easy as a cronjob and rsync. Or, create a samba/nfs share at the same mount point, and have an easy way to provide backups to all the computers on your home network.

<-- Back