[rdiff-backup-users] Backup fails with "lost connection to the remote"

[Henti Smith]

Good day all.

I'm having a tough problem with backing up a server. We've been running
successful backups on the server, but due to storage constraints we dropped
the retention to just one day. We're using backupninja to manage the
backups for us, but the underlying backup tool is rdiff-backup.

Since then we've been having regression and timeout issues, and have not
been able to successfully backup the data (which is quiet large)

I've even tried deleting the destination location completely and starting
from scratch.

The backup command is :

/usr/bin/rdiff-backup --force --remote-schema \
'ssh -C -p 22 %s rdiff-backup --server' --print-statistics \
--exclude '/home/*/.gnupg' \
--exclude '/var/cache/backupninja/duplicity' \
--include '/var/backups' \
--include '/usr/local/bin' \
--include '/usr/local/sbin' \
--include '/var/lib/dpkg/status' \
--include '/var/lib/dpkg/status-old' \
--include '/var/lib/jenkins' \
--exclude '/*' / \
backups-jenkins-master-***@backups-01.internal.company.com::/srv/backups/
jenkins-master-01.internal.company.com/

The error we get is, Multiple lines of "UpdateError ${path_to_file} Updated
mirror temp file ${path_to_another_file} does not match source" ending
with:
Killed

Fatal Error: Lost connection to the
remote system


I'm really at a loss as to what is causing the problem. It's likely that
the files listed in the "does not match source" are changing during backup
as our Jenkins server is pretty busy. If this is the cause, how do I get
around that ?

Any help would be appreciated.

Regards
Henti

[Dominic Raferd]

Post by Henti Smith
Good day all.
I'm having a tough problem with backing up a server. We've been running
successful backups on the server, but due to storage constraints we dropped
the retention to just one day. We're using backupninja to manage the
backups for us, but the underlying backup tool is rdiff-backup.
Since then we've been having regression and timeout issues, and have not
been able to successfully backup the data (which is quiet large)
I've even tried deleting the destination location completely and starting
from scratch.
/usr/bin/rdiff-backup --force --remote-schema \
'ssh -C -p 22 %s rdiff-backup --server' --print-statistics \
--exclude '/home/*/.gnupg' \
--exclude '/var/cache/backupninja/duplicity' \
--include '/var/backups' \
--include '/usr/local/bin' \
--include '/usr/local/sbin' \
--include '/var/lib/dpkg/status' \
--include '/var/lib/dpkg/status-old' \
--include '/var/lib/jenkins' \
--exclude '/*' / \
jenkins-master-01.internal.company.com/
The error we get is, Multiple lines of "UpdateError ${path_to_file} Updated
mirror temp file ${path_to_another_file} does not match source" ending
Killed
Fatal Error: Lost connection to the
remote system

​You could try setting on the client (i.e. connection initiator) in
~/.ssh/config:
ServerAliveInterval 5

and on the server (i.e. responder) in ​/etc/ssh/sshd_config:
ClientAliveInterval 5

This is more likely to help if you are backing up over an external
connection.

Post by Henti Smith
I'm really at a loss as to what is causing the problem. It's likely that
the files listed in the "does not match source" are changing during backup
as our Jenkins server is pretty busy. If this is the cause, how do I get
around that ?

​If your Jenkins server's fs is on top of LVM, you can create an LVM
snapshot and then backup from that. Btrfs and zfs, for instance, offer
snapshot capabilities too - so does Windows NTFS. Ext[234] filesystems have
no built-in snapshot capability - you have to put them on an LVM logical
volume. Before you take the snapshot you need to ensure that the data you
are snapping is in a consistent state, of course.

Dominic
http://www.timedicer.co.uk
_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-***@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backu