Backups?

[ciphergoth]

babysimon reacted with horror when I told him at the weekend that we still hadn't sorted backups out, and pressed a spare 250GB USB external drive into my hands.

What's everyone's favourite way of doing backups? We have a Windows and a Linux laptop to back up, both are switched on at unpredictable times. I just had a go at making brad's "brackup" go, but despite the great attraction of GPG-encrypted backups, it doesn't seem to be ready for prime time yet - the documentation isn't there yet, and it creates zillions of tiny encrypted files. I am resisting the temptation to write my own Python-based alternative. What do you use?

Update: I since found this rather nice solution.

Comments

[mskala.livejournal.com]

My home machine's main partitions are mirrored to two drives with software RAID-1. That protects against the most typical kind of hardware failure, where one drive dies. If that happens I can stick in a new drive and be up and running in a day with no loss. Important files (e.g. my thesis) get rsynced to another machine by a daily cron job. That protects against a lot of the scenarios the RAID doesn't. In the case of my thesis, it's going from my desktop machine at school to a fileserver which is itself backed up by the systems people here. The basic design goal: the backups have to happen without my intervention because if I were required to do something regularly, it wouldn't happen.

Then on a much slower schedule, when I feel like doing it, I burn either my home directory or my entire file tree to write-once optical media (CDs until recently, DVDs now). That's both for backup and as an archive, so I can go back and look at what I was doing in the past.

These measures have seen me through a number of hardware failures, software failures, and human errors over the years. They're certainly not perfect, but the big thing is that I can really keep them up on an ongoing basis, unlike some of the other backup measures I've seen people attempt, which end up not really being sustained.

[juliet]

I *think* you can make Bacula do backups for things like laptops that aren't always-on (I use this at work; might be overkill at home though).

I used to use rsync (+ anacron would do the trick) on Mac. No idea at all about Windows. (I use Time Machine now on the Mac, which rocks, but that's no use to you!).

[werenerd.livejournal.com]

I keep everything backed up on gmail. Most files are things I send anyway. The rare file that I'm not sending anywhere, I'll send it to myself if it's important.

[autopope.livejournal.com]

Time Machine on the Macs, rsync to keep everything in step across platforms and on the Linux boxen.

IIRC somone was trying to implement a Linux equivalent of TM -- ah, here we are: FlyBack. Caveat: I haven't used this (yet).

[ciphergoth.livejournal.com]

Anything *really* important - eg any recent code that I've more than fiddled with - will be in Mercurial, and there will be copies at home and at work. When the laptops were stolen, the BiCon accounts were saved by being in Monotone. But it's stuff like photos and music which take up a lot of space and aren't a good match for DVCS.

[figg.livejournal.com]

I've lost all my email before, but I guess using gmail means it will be unlikely.

Most of the code I have written I have uploaded elsewhere, so I'm not too attached to it either.

I've already lost my entire music collection twice, so that isn't too bad.

[topbit.livejournal.com]

My home server has software mirroring, and I also use http://www.rsnapshot.org/ - Rsync/ssh & hardlink based backups. I've got it to run 6 times per day, which are then promoted daily/weekly & monthly. The hardlinks mean that only new/updated files change from backup to backup. It also means I could pull from the backup files that were changed at the end of October. (and I have recovered a file I'd thought I'd lost from it before now as well - just a matter of find it in the hierarchy, and copy).

It's not so useful for very large files that change often (like video files), so I exclude those directories, but for smaller text files, and long-term static files that aren't intrinsically huge, it's a great option. I've got my /etc/ and the websites I'm developing in it.

Another option as well is an Amazon S3 account, but for that there is a Firefox plugin to upload & set permissions.

[topbit.livejournal.com]

Looks a lot like rsnapshot, but with a prettier front end as opposed to a .conf file and crontab entries.

[ciphergoth.livejournal.com]

With rsnapshot the server initiates the backups, which is less useful when the clients aren't always-on.

[topbit.livejournal.com]

well, you don't have to put it into a cronjob, just run it at will with a particular configuration file. Since it's just a script on the command line, it's easy enough to wrap it with other checks to see if the destination is online, and if so, do things like touch status files which can then kick off alerts if they are too old.

[zz]

rsync, internally inside my server, and using cygwin/cwrsync from windows machines from a batch file using windows' scheduling thingy.

[http://users.livejournal.com/_lj_sucks_/]

rdiff-backup and Time Machine.

[kimble.livejournal.com]

Everything of importance on RAID-1 in the server (the RAID's more about preventing downtime than anything else), nightly rsync scriptage roughly equivilent to rsnapshot doing incremental backups to a dedicated disk, and monthly manually-invoked (a cron job nags daily if more than 28 days since the last run) rsync from there to an external disk that lives on a shelf.

In combination with a lack of (mobile) laptops and sufficient paranoia to ensure that everything of importance is actually on the server.

[reddragdiva]

*cough* still nothing. I should really run rsync to the house server from crontab on each of the laptops.

[pengshui-master.livejournal.com]

Why don't you add an anacron job to the clients which ssh's into the server and kicks
the job off?

[bohemiancoast.livejournal.com]

Time Machine. It's amazingly reassuring.

[hairyears.livejournal.com]

Low tech. I do a manual backup onto an 80-Gig USB drive in the weekly 'maintenance hour' - a time I set aside for updates and the scheduled virus scan.

Just about any USB drive you buy will have rudimentary software for an advanced file copy operation from MyDocuments and any other folders you specify, eliminating duplicates and version-marking (or overwriting) updated files.

[daneel-olivaw.livejournal.com]

Cronned rsync. You can get an version of rsyncd that runs on Win boxes. I have a dumb linux box that grabs a copy of all the relevant data once per hour, and then syncs the lot overnight to somewhere offsite and trustworthy.

[henry_the_cow]

Personally I'm completely crap at backups - I copy photos onto CD-Rs, mail onto a USB stick, and music onto my MP3 player. But that's not why I'm posting here.

Several of my better organised friends use the <A HREF="http://www.cis.upenn.edu/~bcpierce/home.html>Unison</A> file synchroniser to sync between different machines, e.g. work and home machines. Updates on either machine are propagated to the other.

[hughe.livejournal.com]

i nightly cron an rsync from live disk to backup disks then unmount backup disk.

found it more useful than raid for when I decide to delete things i don't mean to.

I've always meant to do a incremental backup system, but never got round to it.

[lovelybug.livejournal.com]

I've already lost my entire music collection twice

Me too. Very annoying.

[figg.livejournal.com]

The trick is to give as many copies of it to your friends. I managed to get a large chunk back quite quickly.

That and googling for music is ridiculously easy at the moment.

[lovelybug.livejournal.com]

Indeed - I have got back a lot in the last six months

[reddragdiva]

Right, set up:

0 0,6,12,18 * * * /usr/bin/rsync -rl --password-file=****** /home/fun/ fun@192.168.1.3:/trollop/David/

Ubuntu includes anacron, so it will run fine.

(Kubuntu 8.04 comes with rsync 2.6.9 and I hand-installed rsync 3.0 on the server from source, but they seem to talk OK.)

[babysimon]

Although BackupPC has the annoying property that the clients have to trust the server, in other respects it's pretty spot on. It does all the same-file-unification, has a nice web GUI, works with both OSs, copes well with intermittently disconnected clients and you can ask me if you have any problems ;)

It also was obviously developed by someone who'd been bitten; it emails you automatically if it detects it's not working. Useful :)