Lire cette documentation en Français: BackupGuide

This document aims at backup management, mainly for virtual machines and HAG.

Koumbit sysadmins might want to check these pages instead:

Contents

  1. Virtual machines
    1. borg
      1. Backups information
      2. Access files in a backup
      3. Extract a specific file
    2. rdiff-backup
  2. HAG
    1. Files and NFS
    2. Courriels

Virtual machines

If you have a virtual server with us and have opted for the backup service, you have RdiffBackup configured with BackupNinja.

Do not modify backupninja configuration files in /etc/backup.d. They would be overridden by Puppet in the next 30 minutes and the changes would be lost!

Before doing any operation, make sure you have enough disk space!

borg

Backups information

List backups: 

 $ sudo borg list ssh://backup-example.koumbit.net@backup.koumbit.net/srv/example.koumbit.net/borg
...
2020-06-01T05:00:04                  Mon, 2020-06-01 05:00:06 [03db2303c044b0d1fc50be19190c3ce566b9f912c994e37dce3c445ef63cabdc]
2020-06-02T05:00:04                  Tue, 2020-06-02 05:00:06 [e3833850d9969e2b40135e2fac4f9d88d80e0b664b92f0251417fe91cb4f854f]

List full repository informations: 

 $ sudo borg info ssh://backup-example.koumbit.net@backup.koumbit.net/backup/example.koumbit.net//borg
...
Repository ID: 9bf455a159d5d3fb15e6b2e385ed4d4ffc7f46ff4ddde191afecd14046ce086c
Location: ssh://backup-example.koumbit.net@backup.koumbit.net/backup/example.koumbit.net/borg
Encrypted: No
Cache: /root/.cache/borg/9bf455a159d5d3fb15e6b2e385ed4d4ffc7f46ff4ddde191afecd14046ce086c
Security dir: /root/.config/borg/security/9bf455a159d5d3fb15e6b2e385ed4d4ffc7f46ff4ddde191afecd14046ce086c
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:                5.63 TB              5.24 TB            573.24 GB

                       Unique chunks         Total chunks
Chunk index:                  587053              5766243

List informations for a specific backup: 

 $ sudo borg info ssh://backup-example.koumbit.net@backup.koumbit.net/backup/example.koumbit.net//borg::2022-05-04T01:27:06
...
Archive name: 2022-05-04T01:27:06
Archive fingerprint: bc0efe69a527059d9a65ab55524d1ce239291a163a7d9a0b94cca19bea2b7d82
Comment: 
Hostname: example
Username: root
Time (start): Wed, 2022-05-04 01:27:08
Time (end): Wed, 2022-05-04 01:32:00
Duration: 4 minutes 51.88 seconds
Number of files: 545293
Command line: /usr/bin/borg create --stats --compression lz4 --exclude /dev --exclude '/home/*/.aMule' --exclude '/home/*/.beagle' --exclude '/home/*/.gnupg' --exclude '/home/*/.local/share/Trash' --exclude '/home/*/.thumbnails' --exclude '/home/*/.Trash' --exclude '/home/*/gtk-gnutella-downloads' --exclude /proc --exclude /run --exclude /sys --exclude /tmp --exclude /var/cache --exclude '/var/lib/ganeti/locks-*.tmp' --exclude '/var/lib/ganeti/queue/*' --exclude /var/lib/mysql --exclude /var/lib/postgresql --exclude /var/log --exclude /var/run --exclude /var/tmp 'ssh://backup-example.koumbit.net@backup.koumbit.net/backup/example.koumbit.net//borg::{now:%Y-%m-%dT%H:%M:%S}' /
Utilization of maximum supported archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              797.45 GB            742.22 GB            596.15 MB
All archives:                5.63 TB              5.24 TB            573.24 GB

                       Unique chunks         Total chunks
Chunk index:                  587053              5766243

Access files in a backup

Mount a specific backup as FUSE: 

$ sudo su
$ tmux
$ mktemp -d
/tmp/tmp.R7fMCamDDq
$ borg mount ssh://backup-example.koumbit.net@backup.koumbit.net/srv/example.koumbit.net/borg::2020-06-02T05:00:04 /tmp/tmp.R7fMCamDDq

Note Work in tmux. If the backup is large, the process could consume a lot of time / memory. (~16G RAM and 25min for 2To)

Afterward, files can be copied as you usually would (Eg. local rsync, cp -r, etc.)

Unmount a borg directory: 

sudo borg umount /tmp/tmp.R7fMCamDDq

Extract a specific file

Supposing you have a backup named borg, done on /srv/my_backuped_directory/, and want to extract /srv/my_backuped_directory/my_website/my_file.php in the current directory: 

sudo borg extract ssh://backup-example.koumbit.net@backup.koumbit.net/backup/example.koumbit.net//borg::2022-05-04T01:27:06 ./srv/my_backuped_directory/my_website/my_file.php

rdiff-backup

Backups are stored on a machine accessible at backup.koumbit.net. If we suppose your machine is foo.koumbit.net, your backup username would be backup-foo.koumbit.net@backup.koumbit.net. You can then build your backups URL by adding :: and the path to your backups. E.g.: backup-foo.koumbit.net@backup.koumbit.net::/backup/foo.koumbit.net/rdiff-backup/.

A few examples:

Get relevant configuration variables for a server

    cat /etc/backup.d/90_default-rdiff-backup.rdiff | grep -E "directory = |host =| user ="
List backups history
  • $ sudo rdiff-backup -l backup-foo.koumbit.net@backup.koumbit.net::/backup/foo.koumbit.net/rdiff-backup/
Found 7 increments:
    increments.2014-12-30T03:01:50-05:00.dir   Tue Dec 30 03:01:50 2014
    increments.2014-12-31T03:02:04-05:00.dir   Wed Dec 31 03:02:04 2014
    increments.2015-01-01T03:02:00-05:00.dir   Thu Jan  1 03:02:00 2015
    increments.2015-01-02T03:01:45-05:00.dir   Fri Jan  2 03:01:45 2015
    increments.2015-01-03T03:02:04-05:00.dir   Sat Jan  3 03:02:04 2015
    increments.2015-01-04T03:01:47-05:00.dir   Sun Jan  4 03:01:47 2015
    increments.2015-01-05T03:01:32-05:00.dir   Mon Jan  5 03:01:32 2015
Current mirror: Tue Jan  6 03:01:36 2015
Include disk space used (longer)
  • $ sudo rdiff-backup -l --list-increment-sizes backup-foo.koumbit.net@backup.koumbit.net::/backup/foo.koumbit.net/rdiff-backup/
Restore a directory
  • $ sudo rdiff-backup -r 2015-01-05T03:01:32-05:00 backup-foo.koumbit.net@backup.koumbit.net::/backup/foo.koumbit.net/rdiff-backup/home/antoine antoine

  • This will extract the user antoine's home directory as it was on January 5, 2015 at 3am in the current directory.

List backup contents
  • rdiff-backup --list-at-time 48h backup-foo.koumbit.net@backup.koumbit.net::/backup/foo.koumbit.net/rdiff-backup/

Note: the use of sudo is important here, as only the root user SSH key has access to the backups server. Also keep in mind the different time formats available: now (more recent backup), 48h (2 days ago) or 2015-01-05T03:01:32-05:00 (specific date).

Then sometimes you have to merge the directories you recovered with the ones that are still there. Especially with Mail, or example. Here is one I did: 

$ ls -la | grep Maildir
drwxr-xr-x 11 foo admins     4096 nov  5 23:28 Maildir
$ sudo rsync -Pavvu --dry-run Maildir/ /var/alternc/mail/m/machin.foo_dude.ca/Maildir/

Pay attention to the / at the end of the directory paths, rsync cares about those. And then run it without the dry-run for realzies. Then don't forget to go properly chown all the recovered files! 

$ sudo ls -la /var/alternc/mail/m/machin.foo_dude.ca/Maildir/
drwxr-xr-x 11 foo admins  4096 nov  5 23:28 .
drwxr-xr-x  3  2407 vmail   4096 jun 28  2010 ..
$ sudo chown -R 2407:vmail /var/alternc/mail/m/machin.foo_dude.ca/Maildir/

HAG

As mentionned on our https://www.koumbit.org/en/content/shared-hosting:

We can restore your data within this period. Data restoration can be selective and partial, and the cost may vary according to the complexity of the site and of the restoration process. A fee of $75 will be billed if the process takes less than 30 minutes; should the data restoration take more than 30 minutes, you will be billed according to our hourly rate of $120/h for a minimum of 1 hour.

If you need to restore files in a shared hosting account, please contact us!

Files and NFS

Voir: BackupGuide#HAG

Courriels

Voir: BackupMaintenance#Emails

BackupGuideEN (last edited 2023-04-04 11:55:21 by nina)