[H-SASIG] rdiff-image-cron /etc/rdiff-image/rdiff-image.conf FAILED!

[Russell Stuart]

On Mon, 2010-02-01 at 18:44 -0500, root wrote:
> Hopefully, its output will give you a clue:
> 
>   rdiff-image-s3.py: S3 changed by someone else - 'humbug-excalibur-backup_base/20100131-223904_20100131-113903_base.rdiff.gz' added.
>   rdiff-image-s3.py: (continuing with backup.)
>   rdiff-image-cron: Backup completed, but S3 modified by someone else.

I am still not completely sure what is going on here.  Currently my best
guess is this is caused by issues on the S3 side.  S3 stores several
copies of each file at locations that are separated geographically, are
serviced by separate power stations and have different internet feeds.
This is to ensure your data is always available.

The downside of all this is the S3 stores can go out of sync if they
have connectivity issues.  When they do, each store can contain
different data and you see different things depending on what server you
happen to be connected to.  Mostly this isn't an issue as they always
keep the latest copy of your data and things sort them out in the end.
But for deletions they err on the side of caution, and if one site says
the data was deleted and another says it isn't, it reappears.

That is what I think is happening here, primarily because last time a
whole bunch of files reappeared.  The backup program interprets the
"reappearance" of files it deleted hours ago as someone stomping on its
data, and thus issues this warning.  A rogue backup on our side can't do
make many files appear at once as it just sends one file on each run.

This file reappearance problem is a "new feature" from S3.  It isn't a
real issue as every time the backup program it deletes any excess files
it sees.  I am hoping the problem it just goes away.  If it persists I
will alter the "it was modified by someone else" detection algorithm.