Automate Borg Backup

Aug 27, 2019 . 4 min read

The previous post, Borg Backup, explained installation and basics of Borg, and this post introduces Pybs, a simple python script to automate backups in Linux systems. To automate the backups, Pybs uses Python, YAML, Borg and Systemd. For non-linux platforms use some other task scheduler in place of Systemd.

Installation

Download Pybs from Github and extract it in your home dir.

Pybs, by default, uses Borg repository at $HOME/backup/borgrepo located in user home directory, and for sake of explanation, we stick with it. However, for actual backup, the repository should be located in a separate disk or device, otherwise the backup is lost in case of system failure.

To run, Pybs needs Python and Python YAML. Install them with,

$ sudo apt install python python-yaml

Pybs Setup

In all files, the paths are hard-coded as /home/x. Edit files listed below and change all occurrences of /home/x to absolute path of your home directory.

    
pybs/backup.yaml

pybs/systemd/pybs-daily.env
pybs/systemd/pybs-daily.service

pybs/systemd/pybs-weekly.env
pybs/systemd/pybs-weekly.service
    

We explain the Pybs configuration bit later. Let’s run Pybs and do some backup.

Run Pybs as normal user

Create Borg repository with,

    
$ cd $HOME
$ mkdir backup
$ borg init -e none backup/borgrepo
    

Next, setup systemd daily backup.

    
$ mkdir -p .config/systemd/user
$ cp pybs/systemd/*.service .config/systemd/user
$ systemctl --user daemon-reload
$ systemctl --user start pybs-daily
        

To run Pybs service as normal user, create $HOME/.config/systemd/user dir and copy systemd service files to it. After reloading systemd daemon, start pybs-daily service. Wait for sometime and see the backup status with,

    
$ systemctl --user status pybs-daily
        

To enable timer, copy and start timer files with,

    
$ cp pybs/systemd/*.timer .config/systemd/user/
$ systemctl --user daemon-reload
$ systemctl --user enable pybs-daily.timer
$ systemctl --user start pybs-daily.timer    
$ systemctl --user list-timers
    

Run Pybs as super user

To backup system wide files and directories, run systemd scripts and timers as root. For that, first create Borg repository as explained above and then place service and timer files in /etc/systemd/system dir as,

## service

$ cp pybs/systemd/*.service /etc/systemd/system
$ systemctl daemon-reload
$ systemctl start pybs-daily
$ systemctl status pybs-daily    

## timer

$ cp pybs/systemd/*.timer /etc/systemd/system
$ systemctl daemon-reload
$ systemctl enable pybs-daily.timer
$ systemctl start pybs-daily.timer
$ systemctl list-timers    
    

Pybs Configuration

Let’s create a new monthly backup service to know Pybs configuration. In pybs/systemd dir, create a new env file and name it as you like with following content,

pybs/systemd/pybs-monthly.env

    
REPO_PATH=/home/x/backup/borgrepo
BASE_DIR=/home/x/pybs
    
BACKUP_LIST=backup.yml    
LABEL=foo

DELAY_START_SEC=120
        

Point environment variable REPO_PATH to absolute path of your borg repository and the BASE_DIR is absolute path of your pybs directory. The BACKUP_LIST is the name of the yaml file which specifies the list of directories or files to backup. The LABEL can be named anything and pybs use it select backups from the list file which has same label. For example, if label is foo then pybs selects only the items labeled as foo for backup. The DELAY_START_SEC is useful to delay the backup so as to reduce the load at the system startup.

Next, create systemd service file, which you may name as you like, in pybs/systemd.

pybs/systemd/pybs-monthly.service

    
[Unit]
Description=Borg Monthly Backup Service
    
[Service]
Type=simple
Nice=19
IOSchedulingClass=2
IOSchedulingPriority=7
    
EnvironmentFile=/home/x/pybs/systemd/pybs-monthly.env
    
ExecStart=/usr/bin/bash -c '${BASE_DIR}/script/backup.sh'

Set EnvironmentFile variable to absolute path of env file which you have created. Leave all other things as it is.

We also need a systemd timer.

pybs/systemd/pybs-monthly.timer

    
[Unit]
Description=Borg Monthly Backup Timer
    
[Timer]
OnCalendar=monthly
AccuracySec=1h
Persistent=true
    
[Install]
WantedBy=timers.target

The file should be named same as service file base name, but with .timer extension. The backup frequency is set with OnCalendar variable. Refer systemd documentation to know more about timers.

Finally, add backup entries for monthly backup. Label the backup sets with the label specified in env file.

pybs/backup.yml

    
backups:
    
  music:
    label: foo
    dirs:
      - /home/x/Music
    excludes:
      - /home/x/Music/*.wmv
    
  picts:
    label: weekly
    dirs:
      - /home/x/Pictures
    

The new backup service backups only the Music dir excluding all wmv files.

Thats all there is to configure. To run new backup service, copy and enable service and timer units as explained in Run Pybs