Automated Raspberry Pi Backup with rsync
A couple of weeks ago, I made an attempt to reduce USB hard drive noise by idling it automatically. It didn’t work out that well – the drive seems to be possessed, turns itself on seemingly randomly, and the logs tell me nothing. I figured a better solution would be to mount it only when it’s needed, which coincides with my decision to use it just for backups.
This post is a part of my Homelab Series. See the index here.
Backup Script
I asked ChatGPT to write the backup script for me and I feel empty. It was just too easy. I missed the dopamine kick from figuring out the arcane arts of shell scripting, ugh. And because I was very lazy at prompting, it took several attempts to make it do exactly what I want. And again, instead of a sense of satisfaction or learning, it felt like a frustrating encounter with a incredibly talented but woefully unimaginative person.
And after that, the universe heard my request for a challenge and for two straight days, I wasn’t able to figure out why rsync
wouldn’t accept my exclude lists. I finally figured out a solution but I have no idea why the original approach (using an array with --exclude=
option) didn’t work.
Anyway, here’s the script. It stops all docker containers, mounts the external drive, performs 3 backup operations, restarts docker containers, unmounts the drive and sends a report to my phone. It even has a dry run mode – simply run sudo backup.sh test for a test run with more logging and some actions (like unmounting) disabled.
#!/bin/bash
# Check for root privileges
[[ $EUID -ne 0 ]] && { echo "This script must be run as root."; exit 1; }
# Paths
backupDrive="/dev/sda2"
backupDriveMount="/mnt/media"
backupPath="$backupDriveMount/Backups"
# Define backup sources and destinations
declare -a backups=(
"/boot/ sd-boot" # This means: backup /boot/ to $backupPath/sd-boot
"/ sd-root"
"/mnt/ssd-sandisk/ ssd-sandisk"
)
# Set the locale to avoid issues with rsync output
export LC_ALL=C
# Pushover credentials
tokenUser="Your Pushover User Token"
tokenApp="Your Pushover App Token"
# Log file location
logFile="/var/log/backup.log"
# Check if log file exists
if [ ! -f "$logFile" ]; then
touch "$logFile"
else
# Get the current date and the file's modification date in seconds since epoch
currentTime=$(date +%s)
fileModificationTime=$(stat -c %Y "$logFile")
# Calculate the age of the file in seconds
fileAge=$((currentTime - fileModificationTime))
# Calculate the number of seconds in 3 months (approximately 90 days)
threeMonthsInSeconds=$((90 * 24 * 60 * 60))
# Check if the file is older than 3 months
if [ $fileAge -gt $threeMonthsInSeconds ]; then
rm "$logFile"
touch "$logFile"
fi
fi
# Check for 'test' argument to enable dry run
dryRunFlag=""
[[ "$1" == "test" ]] && dryRunFlag="--dry-run -P"
# Log messages
logMessage() {
local message="$1"
echo "$(date '+%Y/%m/%d %H:%M:%S') - $message" | tee -a "$logFile"
}
# Initialize backup report for Pushover notification
backupReport=""
# Send Pushover notifications
sendNotification() {
local title="$1"
local message="$2"
curl -s \
--form-string "token=$tokenApp" \
--form-string "user=$tokenUser" \
--form-string "title=$title" \
--form-string "message=$message" \
https://api.pushover.net/1/messages.json >/dev/null
}
# Perform rsync backup
backup() {
local source="$1"
local description="$2"
local destination="$backupPath/$description/"
local options=""
[ -f "$description-exclude.txt" ] && options="--exclude-from=$description-exclude.txt"
logMessage "Starting backup of ${description}: ${source} -> ${destination}"
rsyncOutput=$(rsync -avzh $dryRunFlag --delete --stats ${options:+"$options"} "$source" "$destination" 2>&1)
if [[ $? -ne 0 ]]; then
sendNotification "Pi Backup ❌" "Failed to back up ${description}."
logMessage "Error during backup of ${description}."
logMessage "$rsyncOutput"
else
# Extract the number of regular files transferred
numFilesTransferred=$(echo "$rsyncOutput" | awk '/Number of regular files transferred/{print $NF}')
logMessage "Completed backup of ${description}. Files transferred: ${numFilesTransferred}."
backupReport+="${description}: ${numFilesTransferred} files\n"
fi
}
# Unmount backup hard drive
unmountMedia() {
if ! sudo umount $backupDriveMount; then
logMessage "$backupDriveMount is busy. Attempting lazy unmount."
if ! sudo umount -l $backupDriveMount; then
sendNotification "Pi Backup ❌" "Failed to unmount $backupDriveMount."
logMessage "Lazy unmount of $backupDriveMount failed."
exit 1
else
logMessage "Lazy unmount of $backupDriveMount succeeded."
fi
else
logMessage "Unmounted $backupDriveMount."
fi
}
#############################################
if [[ "$1" != "test" ]]; then
# Stop all docker containers
logMessage "Stopping all docker containers…"
docker stop $(docker ps -q)
fi
# Mount backup hard drive
if ! mountpoint -q $backupDriveMount; then
if ! sudo mount $backupDrive $backupDriveMount; then
sendNotification "Pi Backup ❌" "Failed to mount $backupDrive at $backupDriveMount."
exit 1
fi
else
logMessage "$backupDriveMount is already mounted."
fi
# Ensure the backup directory exists
sudo mkdir -p $backupPath
# Perform backups
for backupItem in "${backups[@]}"; do
IFS=' ' read -r source description <<< "$backupItem"
backup "$source" "$description"
done
if [[ "$1" != "test" ]]; then
# Unmount backup drive
unmountMedia
# Restart docker containers
logMessage "Restarting all docker containers."
docker start $(docker ps -a -q)
fi
# Send success notification
sendNotification "Pi Backup ✅" "$(printf "$backupReport")"
logMessage "$backupReport"
Excluding Stuff
If you want to exclude directories or files from backup, create a text file in the same directory as the script, which has to be named the same as backup destination with -exclude
suffix. Backup script will pick it up automatically.
Example time! If you declare your backup like this:
declare -a backups=(
"/ root-sd-card"
)
Your exclude file needs to be named root-sd-card-exclude.txt
. Here’s how I set up mine:
SD Card Root (sd-root-exclude.txt):
*DS_Store
/dev/*
/home/pe8er/.cache/*
/home/pe8er/.vscodium-server/*
/lost+found
/media/*
/mnt/*
/proc/*
/run/*
/sys/*
/tmp/*
/var/log/*
/var/swap
/var/tmp/*
logs*
logs/*
SSD Drive (ssd-sandisk-exclude.txt):
.caltrash
.stversions
*DS_Store
/Downloads/*
logs*
logs/*
Automate Backup
Thank god for crontab.guru. I want to run the script every Saturday at 1 AM and apparently this does it:
sudo crontab -e
0 1 * * 6 /home/pe8er/backup.sh
Don’t Forget
To remove backup drive’s entry from /etc/fstab
so that it doesn’t mount on boot anymore.