[H-GEN] Excalibur patches for S3 backup

[Russell Stuart]

On Mon, 2009-07-06 at 10:16 +1000, Raymond Smith wrote:
> How confident are we really be that all sensitive data has been scrubbed
> from these images? 

As confident as I can be, of course.  And I have tried to get all and
sundry to check the list.  It ain't hard: just download the patch from:

  http://www.stuart.id.au/russell/files/pub/humbug-s3-rdiff-image-patch.tar.gz 

extract:

  /etc/rdiff-image/rdiff-image.conf

and have a look.  It would be a great help if a more of people did that,
and in particular people like you (hint, hint) who are familiar with how
things are done on excalibur and caliburn took a look.  With lots of
eyes all bugs are shallow.  But seriously, if it is important to you,
then you should take the time to check it.

Anyway, so far myself and Stephen (our current sysadmin who set up
excalibur) have looked over it long and hard.  I am hoping between us we
have not missed much.  Greg may have looked as well - I am not sure.

> For example, what about passwords in the wiki?

Already scrubbed.  As are the mailing list passwords.

> What about registry keys stored in comments in DNS zone files? 

?  I didn't see any.

> What about SSH public keys (while not strictly sensitive but I don't know if I
> really want everyone to have a copy all the same -- "So Mr Smith, your
> SSH key was found on this server. Would you like to explain that....")?

As you say, they aren't sensitive.  If you personally find it a problem
then create a new one for excalibur.

> I am still not comfortable with this idea of having the backups available
>  to all comers via HTTP. I would be much, much happier if their were
> some minimum level of control over this.

The idea was not to make the URL public.  But provided the backups are
freely download-able by financial Humbug member I don't care.  We can
put a password on them, issue a cert, encrypt them with a public key, or
whatever.

> Alternatively, perhaps we could invert the public backup. What if instead
> of a backup this was the base image. That is, we have a base image
>  which is read-only available and write-only to sysadmins. Anyone can
> mess with the image and submit a patch. The sysadmin team then
> approves and applies or rejects the patch. At some point a cron job rolls
>  out changes from the patch onto the main system. Files containing
> "sensitive" data would either be excluded from the patch process or we
> could try for clever mergey stuff.

I don't understand the difference between what you are suggesting and
what this patch does.

> These need to be stored somewhere that all members of the sysadmin
> team can access even in the event of a complete meltdown. We used
> to do this by putting stuff on hydra and caliburn. It would make sense
> for individuals to keep their own records as well.

If people backup the humbug images in other ways (eg by download them to
work on them), then they will have a copy of the S3 credentials.  We can
also put a copy in the Humbug bank box, or whatever.

The choice of S3 is part of the problem, of course, as it requires
credentials.  I got the distinct impression people preferred S3
precisely because it was a for-money service (and thus required
credentials).  I put up another suggestion: backup to a gmail IMAP
account (which has 7G).  That made aj twitch badly, and besides is
against GMail's terms of service.

Anyway, there is no reason I can't arrange for the backups to be pushed
to several locations.  There is nothing special about S3.  The only
thing need to make this happen is someone actually find & suggest the
said alternative locations.