Press "Enter" to skip to content

Using borg for backing up nuxx.net

Published November 10, 2019

Last updated on June 26, 2020

For 10+ years I’ve been backing up the server hosting this website using a combination of rsync and some homegrown scripts to rotate out old backups. This was covered in Time Machine for… FreeBSD? and while it works well and has saved me on some occasions it had downsides.

Notably, the use of rsync’s --link-dest requires a destination filesystem that supports shared inodes, limiting me to Linux/BSD/macOS for backup storage. Also, while the rotation was decent, it wasn’t particularly robust nor fast. Because of how I used rsync to maintain ownership with --numeric-ids it also required more control on the destination than I preferred. And, the backups couldn’t easily be moved between destinations.

In the years since, a few backup packages have come out for doing similar remote backups over ssh, and this weekend I settled on BorgBackup (borg) automated with borgbackup.sh.

Borg and a single wrapper script (that I based on dailybackup.sh) executed from cron simplifies all of this; handling compression, encryption, deduplication, nightly backups, ssh tunneling, and pruning of old backups. More importantly, it removes the dependence on shared inodes for deduplication, eliminating the Linux/BSD/macOS destination requirement. This means my destination could be anything from a Windows box running OpenSSH to a NAS to rsync.net.

Here’s a brief overview of how I have it set up:

  1. Install BorgBackup on both the source and destination (currently Linux nuxx.net server and a Mac at home).
  2. Put a copy of borgbackup.sh on the source, make it executable, and restrict it just to your backup user. Edit the variables near the top as needed for your install, and the paths to back up in the create section.
  3. Create an account on the destination which accepts remote SSH connections via key auth from the backup user on the source.
  4. Restrict the remote ssh connection to running only the borg serve command, to a particular repository, and give it a storage quota:
    restrict,command="borg serve --restrict-to-repository /Volumes/Rusty/borg/servername.nuxx.net” from=“10.0.0.10” ssh-rsa AAAAB3[…snipped…] root@servername.nuxx.net
  5. On the remote server, turn off history (to keep the passphrase from ending up in your history; in bash: set +o history), set the BORG_REPO, BORG_PASSPHRASE, BORG_REMOTE_PATH, and BORG_RSH variables from the top of the script to test what the script defines:
    export BORG_REPO='ssh://user@server.example.com:220/path/to/repo/location/'
    export BORG_PASSPHRASE='PASSPHRASEGOESHERE'
    export BORG_REMOTE_PATH=/usr/local/bin/borg
    export BORG_RSH='ssh -oBatchMode=yes'
  6. From the source, initialize the repo on the destination (this uses the environment variables): borg init --encryption=repokey-blake2
  7. Perform the first backup as a test, backing up just a smallish test directory, such as /var/log: borg create --stats --list ::{hostname}-{now:%Y-%m-%dT%H:%M:%S} /var/log
  8. Remotely list the backups in the repo: borg list
  9. Remotely list the files in the backup: borg list ::backupname
  10. Test restoring a file: borg extract ::backupname var/log/syslog
  11. Back up the encryption key for the repo, just in case: borg key export ssh://user@server.example.com:220/path/to/repo/location/
  12. Define things to exclude in /etc/borg_exclude on the source per the patterns. In my I use shell-style patterns to exclude some cache directories:
    sh:**/cache/*
    sh:**/locks/*
    sh:**/tmp/*
  13. Create the log directory, in the example as /var/log/borg.
  14. Run the backup script. Tail the log file to watch it run, see that it creates a new archive, prunes old backups as needed, etc.
  15. Once this all works, put it in crontab to run every night.
  16. Enjoy your new simpler backups! And read the excellent Borg Documentation on each command to see just how easy restores can be.

Published in computers