[H-GEN] tape and cdrom backup strategies and musings

[Tony Nugent]

[ Humbug *General* list - semi-serious discussions about Humbug and  ]
[ Unix-related topics.  Please observe the list's charter.           ]
[ Worthwhile understanding: http://www.humbug.org.au/netiquette.html ]

I need to put together a revised and mostly (completely?) automated
backup regime for two network server boxes:
- an NT application (database) and (special use) file server, and
- a linux server doing just about everything else, including router,
  firewall, proxy, smtp/pop, dns, windows domain server, housing all
  the (windows) network user home directories and user network share
  data.  (samba is very busy:)

There is a cdrom burner available on a (windows) box elsewhere on
the local network, and a 4Gb (uncompressed) scsi tape drive lives in
the linux box, which also has around 60Gb of hard drive space
(perhaps half of that "free") over three hard drives (one is a large
scsi).  (The linux box happens to be running redhat 6.2, with all
updates, which is not so relevant to this discussion).

  The advantage with backing up onto cdrom is that it allows very
  easy and almost access to the data from any computer with a cdrom.
  It isn't so easy to access files backed up on tapes.  (The problem
  then becomes, which disk has the file(s) that I'm looking for? :)

I have things working ok right now (in a fashion), but I'm looking
for more ideas for how things can be managed, steamlined and more
automated.

If anyone has any pointers to URLs and other resources that may be
useful reference reading for me, I'd appreciate it.

My current backup regime for this particular network goes something
like this, essentially fairly simple.  Comments and suggestions are
welcome:

- Full tape backup of specific important ("unrecoverable") system
  directories on the linux server on a regular (weekly?) basis,
  especially /etc /root /usr/local and config and configuration data
  in /var (and so on).

- Less regular (monthly?) full system backups using mondo/mindi
  recovery into a series of bootable and self-installing cdroms.

  I haven't actually started using mondo as a serious system backup
  and recovery tool in this environment, but it works great at home
  -- it is a very cool utility.  BTW, if you don't know about mondo
  (and mindi), then check it out:
  	http://www.microwerks.net/~hugo/

  My current "system disaster recovery plan" is with a rescue boot
  disk and restoring the OS directly from tape that way.  Lots of
  tape changes, slow etc.  Mondo would be a better/faster way.

  All the important system config files are archived safely, so I am
  also in a position to do a "fresh-start" rebuild of a new system
  from scratch with installation disks.

  BTW, I'm using recent versions (rpms) of backup/restore for the
  tape backups.  The tape works with tomsrtbt (www.toms.net) as a
  useful linux bootdisk rescue recovery tool.

- Regular (daily) tape backups of all changed data since the last
  full weekly backup.  This includes both system and user data
  changes.  (This tape can last all week using a non-rewinding
  device file).

- Full weekly tape backups of user data going back to the beginning
  of a month or so (keep a history).

- Full monthly (manually-initiated) backups of new and changed user
  data (since the last cdrom backup) into one or more iso images.
  (The iso files are copies over the network onto the windows
  machine where the burner lives).

  The result is a series of cdroms with user and system data
  _changes_ for each month.  This seems to work well, and it can
  easily (usually) fit onto one 700Mb disk per monthly backup.

- Full cdrom backups of all user data on CDROMs are already
  available going back each month, with even older data archived
  onto to sets of cdrom (in case of failure, damage or loss).

- Full system backup of the NT server, done as a tarball over a
  windows share into the tape drive.
  
  Although this can sometimes be a pain with file locking problems.
  AT jobs on the nt box are doing things like getting its
  (specialised) database server to create a backup db file and back
  that up while excluding the "live" db file from the backup run.
  There are quite few files like this to deal with.  (btw, is there
  an easy way to get clean backups of nt (and windows) registry
  files?)

  The recovery plan (mostly untested, ouch) is via linux boot disk
  to reformat the drive (as FAT) and recover the data over the
  network with a tape restore and (NT boot/install disks).  (The NT
  can be built from scratch too, if necessary).  I need to find a
  linux recovery bootdisk that has ntfs-reading capabilities so that
  I can get "clean" snapshot of its filesystems, at the moment I'm
  just tarballing the raw partitions.

  I'd like to change this so that it is done using mondo.  I think
  it is possible to use it to build full cdrom boot recovery disks
  for windows boxes, I want to try it - has anyone got any
  experiences using it like this?

- Only weekly incremental backups of the data storage on the NT
  server are required, with that data included into the monthly
  cdrom backups.

- Tapes and duplicates of all cdrom backups are kept in a fireproof
  safe, along with a log book.  (Actually, all of it is kept in the
  safe, it is precious data that should not go travelling).  (A copy
  of old data should also be kept safe somewhere off-site).

- I have much of this automated as cron jobs, with emails going to
  myself and people in the network to notifiy of success/failure
  activity and when intervention or other things need to be done.


Does anyone have any good tips for organising good tape and data
backup log books?  (I'm not entirely happy with my current system).

What about tape rotation and archiving methodologies?

Are there any good utilities for indexing/database'ing the tape and
cdrom files for quick searching?  (Mysql with a php/apache
front-end?)

I'm aware of norton's ghost.  It's brilliant, it's commercial (and
expensive for the network-capable version), it works well with
managing linux, windows and ntfs partitions, and (last time I used
it) it was mostly biased for a dos/windows environment.

Cheers
Tony

--
* This is list (humbug) general handled by majordomo at lists.humbug.org.au .
* Postings to this list are only accepted from subscribed addresses of
* lists 'general' or 'general-post'.