KPTree Wiki

This has been more difficult than I would have thought. I have used rsnapshot the past couple of years (2020-21), but have never really been happy with it for many reasons. The main reason is that it has never worked well, the rsync is reasonably fast, but the rsnapshot takes far too long. Other problems include; difficulties in set-up, and concerns will reliability. Recently I noticed that Debian no longer includes it in its repositories as of Debian 11 due to poor maintenance….. So I have no moved to Restic, which is described here: Back-up Server Setup.

overlayfs

I was disappointed with overlayfs, it does not seem to allow for a true merge file system, just overlay file system. The information available to set up was some of the worst I have found to date for Linux in the modern day. It is the only solution built in the Kernel. I looks like it may be of some use with containers, such as Docker. It does not really interest me at this time and I am not using.

tl;dr;

mergerfs

In the end I has stopped using mergerfs. There are other options to combine drives, e.g. lvm or simplely using symlinks.

tl;dr;

Mergerfs looks more like what I wish to achieve it is a way to pool drive into a common mount point. Below are some links with information and and possible setups.

Down load the current correct version of mergerfs from: https://github.com/trapexit/mergerfs/releases/, hint look at the link of the file of interest….: wget https://github.com/trapexit/mergerfs/releases/download/2.29.0/mergerfs_2.29.0.ubuntu-xenial_amd64.deb

Then load it into linux: sudo dpkg -i mergerfs_2.29.0.ubuntu-xenial_amd64.deb

Then create a mount point directory, e.g. /srv/storage, sudo mkdir /srv/storage sudo vim /etc/fstab

UUID=d426f5e0-7fbd-4e7f-915e-b7c78eb70877 /                   ext4    errors=remount-ro 0       1
# swap was on /dev/sdd5 during installation
UUID=e3442da3-4eaf-477d-94af-6ace66097a54        none           swap    sw              0       0
UUID=2fedae7e-8214-4fec-a6dd-85d7026d474f       /media/Disk1    ext4    defaults        0       0
UUID=38f52a66-fe96-4eb4-a794-6fe1d02bab4a       /media/Disk2    ext4    defaults        0       0
UUID=1cb1509c-ec09-46cd-ab30-23f3be5b554e       /media/Disk3    ext4    defaults        0       0
UUID=2ba18a1e-d48d-4122-bff8-30c2bc359749       /media/Disk4    ext4    defaults        0       0
/media/Disk1:/media/Disk2 /srv/storage fuse.mergerfs use_ino,defaults,allow_other,minfreespace=50G,fsname=mergerfs 0 0
#/media/Disk1           /export/Disk1   bind    bind    0       0
#/media/Disk2           /export/Disk2   bind    bind    0       0
/media/Disk4            /export/Disk4   bind    bind    0       0
/home/shared            /export/shared  bind    bind    0       0

Directly mount from CLI: sudo mergerfs -o defaults,allow_other,use_ino,fsname=mergerFS /media/Disk1:/media/Disk2 /srv/storage

To unmount: sudo umount /srv/storage

To check mounted disks df -Th

Filesystem     Type           Size  Used Avail Use% Mounted on
udev           devtmpfs        16G     0   16G   0% /dev
tmpfs          tmpfs          3.2G  2.7M  3.2G   1% /run
/dev/sdd1      ext4           443G  161G  260G  39% /
tmpfs          tmpfs           16G     0   16G   0% /dev/shm
tmpfs          tmpfs          5.0M     0  5.0M   0% /run/lock
tmpfs          tmpfs           16G     0   16G   0% /sys/fs/cgroup
mergerfs       fuse.mergerfs   11T  177M   11T   1% /srv/storage
/dev/sdb1      ext4           5.5T   89M  5.2T   1% /media/Disk2
/dev/sda1      ext4           5.5T   89M  5.2T   1% /media/Disk3
/dev/sdc1      ext4           5.5T   89M  5.2T   1% /media/Disk1
/dev/sde1      ext4           9.1T  1.2T  7.5T  14% /media/Disk4
tmpfs          tmpfs          3.2G     0  3.2G   0% /run/user/1000

LVM

With the availability of Gparted to adjust partitions on the run I do not know if LVM is required any more. It does however look like a flexible system.

Command	example	LVM	PV	VG	LG	Description (from man)
s	`pvs`	✘	✔	✔	✔	Display information about physical volumes
display	`pvdisplay`	✘	✔	✔	✔	Display various attributes of physical volume(s)
create	`pvcreate`	✘	✔	✔	✔	Initialize physical volume(s) for use by LVM
move	`pvmove`	✘	✔	✔	✘	Move extents from one physical volume to another
rename	`vgrename`	✘	✘	✔	✔	Rename a volume group
change	`lvchange`	✔	✔	✔	✔	Change the attributes of logical volume(s)
extend	`lvextend`	✘	✘	✔	✔	Add space to a logical volume
reduce	`lvreduce`	✘	✘	✔	✔	Reduce the size of a logical volume
resize	`lvresize`	✘	✔	✘	✔	Resize a logical volume
split	`vgsplit`	✘	✘	✔	✘	Move physical volumes into a new or existing volume group
merge	`vgmerge`	✘	✘	✔	✘	Merge volume groups
convert	`vgconvert`	✘	✘	✔	✘	Change volume group metadata format
import	`vgimport`	✘	✘	✔	✘	Register exported volume group with system
export	`vgexport`	✘	✘	✔	✘	Unregister volume group(s) from the system
importclone	`vgimportclone`	✘	✘	✔	✘	Import a VG from cloned PVs
cfgbackup	`vgcfgbackup`	✘	✘	✔	✘	Backup volume group configuration(s)
cfgrestore	`vgcfgrestore`	✘	✘	✔	✘	Restore volume group configuration
ck	`vgck`	✘	✔	✔	✘	Check the consistency of volume group(s)
scan	`vgscan`	1	✔	✔	✔	Search for all volume groups
remove	`vgremove`	✘	✔	✔	✔	Remove volume group(s)

1. diskscan

Links:

NFS

To check available server drives use sudo showmount -e 192.168.1.5

To mount a NFS manually:

sudo mount -t nfs -o rw,vers=4 192.168.1.5:/Disk1 /mnt/backup1
sudo mount -t nfs -o rw,vers=4 192.168.1.5:/Disk2 /mnt/backup2

Where:

-t nfs : type nfs
-o : options, options are separated by commas with no spaces
- rw : read / write
- vers=4 : version 4 of nfs
192.168.1.5:/Disk1 is the remote exported NFS mount point (/export/Disk1 is no longer accepted syntax in NFS4.)
/mnt/backup1 : is the local mount point, note this directory must exist, ideally empty, any existing contents are usually invisible and otherwise ignored

To unmount:

sudo umount /mnt/backup1
sudo umount /mnt/backup2

Where:

/mnt/backup1 is the local mount point to be unmounted

Possible mount script

#!/bin/bash
 
#input 1 is the proposed mount point
 
if [ -z "$1" ]
then 
   exit 11  #no mount point input, exit with error
else
   mount1=$1 #Use mount point input
fi
 
if grep -qs "$mount1" /proc/mounts; then
  exit 0  #Already mounted, exit with success
else
  mount "$mount1"
  if [ $? -eq 0 ]; then
   exit 0  # Successfully mounted, exit with success
  else
   exit 12 # mount failed, exit with error
  fi
fi

Created script to mount backup1 sudo bash ~/Myscripts/mount_backup1.sh to run

''sudo vim ~/Myscripts/mount_backup1.sh''

#!/bin/bash
#Script to mount NFS remote Disk1 to local /mnt/backup1
 
if grep -qs "/mnt/backup1" /proc/mounts; then
  exit 0  #Already mounted, exit with success
else
  mount -t nfs -o rw,vers=4 192.168.1.5:/export/Disk1 /mnt/backup1
  if [ $? -eq 0 ]; then
   exit 0  # Successfully mounted, exit with success
  else
   exit 12 # mount failed, exit with error
  fi
fi

Created script to mount backup2 sudo bash ~/Myscripts/mount_backup2.sh to run

''sudo vim ~/Myscripts/mount_backup2.sh''

#!/bin/bash
#Script to mount NFS remote Disk2 to local /mnt/backup2
 
if grep -qs "/mnt/backup2" /proc/mounts; then
  exit 0  #Already mounted, exit with success
else
  mount -t nfs -o rw,vers=4 192.168.1.5:/export/Disk2 /mnt/backup2
  if [ $? -eq 0 ]; then
   exit 0  # Successfully mounted, exit with success
  else
   exit 12 # mount failed, exit with error
  fi
fi

As we intend to shutdown the backup server it is good to unmount the back-up drives first.

Created script to unmount backup1 sudo ~/Myscripts/unmount_backup1.sh

''sudo vim ~/Myscripts/unmount_backup1.sh''

#!/bin/bash
#Script to mount NFS remote Disk2 to local /mnt/backup2
 
if grep -qs "/mnt/backup1" /proc/mounts; then
  umount /mnt/backup1
  if [ $? -eq 0 ]; then
   exit 0  # Successfully unmounted, exit with success
  else
   exit 12 # umount failed, exit with error
  fi
else
  exit 0 # Not actually mounted, exit with success
fi

Created script to unmount backup2 sudo ~/Myscripts/unmount_backup2.sh

''sudo vim ~/Myscripts/unmount_backup2.sh''

#!/bin/bash
#Script to mount NFS remote Disk2 to local /mnt/backup2
 
if grep -qs "/mnt/backup2" /proc/mounts; then
  umount /mnt/backup2
  if [ $? -eq 0 ]; then
   exit 0  # Successfully unmounted, exit with success
  else
   exit 12 # umount failed, exit with error
  fi
else
  exit 0 # Not actually mounted, exit with success
fi

Backup main Media drive (Rsync / Rsnapshot method)

Sadly I have found Rsnapshot and Rsync is not suitable for large backup service. Rsnapshot is simply too slow and I am concerned with reliability. Rsnapshot configuration is overly complex, with strange requirements such as tab for white space. As of Debian 11 it is no longer even included in the Debian repositories for lack of maintenance. It feels effectively abandoned.

Superseded, tl;dr;

I do not use Rsnapshot anymore. I just do not feel comfortable using it.
The following is the command I use to backup my main media drive: sudo rsync -av -u --delete /media/disk2/Media/ baumkp@192.168.1.5::all/media/Disk4/disk2 --exclude=snapraid.* --exclude=.Trash* --exclude=lost* >tmp.txt

I have rsync setup running as a daemon on 192.168.1.5 with login details for baumkp setup on the sever accepting input from the local area network to just the one address 192.168.1.10. On the remote machine 192.168.1.5: sudo vim /etc/rsyncd.conf

# create new
  
# any name you like
[all]
# destination directory to copy
path = /
# hosts you allow to access
hosts allow = 192.168.1.10
#hosts deny = *
list = true
uid = root
gid = root
read only = false
#auth users = baumkp
#secrets file = /etc/rsyncd.secrets

sudo vim /etc/rsyncd.secrets

username:userpassword

Use systemd systemctl from control of the deamon: sudo systemctl status rsync.service [status, start, stop, restart, reload, disable, enable] I have enabled so the rsync deamon this normally running.

The local directory /media/disk2/Media/ contents are to be backed up.

rsnapshot backups

I have 2 rsnapshot backup configurations;

/etc/rsnapshot1.conf To run locally: sudo rsnapshot -c /etc/rsnapshot1.conf daily

sudo vim /etc/rsnapshot1.conf

with all comments

#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULE:        #
#                                               #
# This file requires tabs between elements      #
#                                               #
#################################################
 
#######################
# CONFIG FILE VERSION #
#######################
 
config_version  1.2
 
###########################
# SNAPSHOT ROOT DIRECTORY #
###########################
 
# All snapshots will be stored under this root directory.
#
# snapshot_root /var/cache/rsnapshot/ #default 
snapshot_root   /mnt/backup1
 
# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root 1
 
#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################
 
# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp          /bin/cp
 
# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm          /bin/rm
 
# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync       /usr/bin/rsync
 
# Uncomment this to enable remote ssh backups over rsync.
#
#cmd_ssh        /usr/bin/ssh
 
# Comment this out to disable syslog support.
#
cmd_logger      /usr/bin/logger
 
# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
cmd_du          /usr/bin/du
 
# Uncomment this to specify the path to rsnapshot-diff.
#
cmd_rsnapshot_diff      /usr/bin/rsnapshot-diff
 
# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec    /path/to/preexec/script
 
# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec   /path/to/postexec/script
 
# Paths to lvcreate, lvremove, mount and umount commands, for use with
# Linux LVMs.
#
#linux_lvm_cmd_lvcreate /sbin/lvcreate
#linux_lvm_cmd_lvremove /sbin/lvremove
#linux_lvm_cmd_mount    /bin/mount
#linux_lvm_cmd_umount   /bin/umount
#########################################
#     BACKUP LEVELS / INTERVALS         #
# Must be unique and in ascending order #
# e.g. alpha, beta, gamma, etc.         #
#########################################
 
#retain alpha   6
#retain beta    7
#retain gamma   4
#retain delta   3
 
#retain hourly  6
retain  daily   7
retain  weekly  4
retain  monthly 3
 
 
############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################
 
# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      Everything
#
verbose         2
 
# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
# If you want the rsync output, you have to set it to 4
#
loglevel        3
 
# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
logfile         /var/log/rsnapshot.log
 
# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile        /var/run/rsnapshot.pid
 
# By default, rsnapshot check lockfile, check if PID is running
# and if not, consider lockfile as stale, then start
# Enabling this stop rsnapshot if PID in lockfile is not running
#
#stop_on_stale_lockfile         0
 
# Default rsync args. All rsync commands have at least these options set.
#
rsync_short_args        -av
rsync_long_args --delete        --numeric-ids   --relative      --delete-excluded
 
# ssh has no args passed by default, but you can specify some here.
#
#ssh_args       -p 22
 
# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
du_args         -csh
 
# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs         0
 
# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns. 
# 
#include        ???
#include        ???
exclude         /lost+found
exclude         snapraid.*
exclude         lost*
exclude         .Trash*
exclude         /proc/*
exclude         /run/*
exclude         /tmp/*
exclude         /VMs/*
 
# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file   /path/to/include/file
#exclude_file   /path/to/exclude/file
 
# If your version of rsync supports --link-dest, consider enabling this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest      0
 
# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot alpha"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first     0
 
# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes       0
 
# Number of rsync re-tries. If you experience any network problems or
# network card issues that tend to cause ssh to fail with errors like
# "Corrupted MAC on input", for example, set this to a non-zero value
# to have the rsync operation re-tried.
#
#rsync_numtries 0
 
# LVM parameters. Used to backup with creating lvm snapshot before backup
# and removing it after. This should ensure consistency of data in some special
# cases
#
# LVM snapshot(s) size (lvcreate --size option).
#
#linux_lvm_snapshotsize 100M
 
# Name to be used when creating the LVM logical volume snapshot(s).
#
#linux_lvm_snapshotname rsnapshot
 
# Path to the LVM Volume Groups.
#
#linux_lvm_vgpath       /dev
 
# Mount point to use to temporarily mount the snapshot(s).
#
#linux_lvm_mountpath    /path/to/mount/lvm/snapshot/during/backup
 
###############################
### BACKUP POINTS / SCRIPTS ###
###############################
 
# LOCALHOST
backup  /media/disk1/KarlData   KarlData/
backup  /home/                  home/
#backup /home/          localhost/
#backup /etc/           localhost/
#backup /usr/local/     localhost/
#backup /var/log/rsnapshot              localhost/
#backup /etc/passwd     localhost/
#backup /home/foo/My Documents/         localhost/
#backup /foo/bar/       localhost/      one_fs=1, rsync_short_args=-urltvpog
#backup_script  /usr/local/bin/backup_pgsql.sh  localhost/postgres/
# You must set linux_lvm_* parameters below before using lvm snapshots
#backup lvm://vg0/xen-home/     lvm-vg0/xen-home/
 
# EXAMPLE.COM
#backup_exec    /bin/date "+ backup of example.com started at %c"
#backup root@example.com:/home/ example.com/    +rsync_long_args=--bwlimit=16,exclude=core
#backup root@example.com:/etc/  example.com/    exclude=mtab,exclude=core
#backup_exec    ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"
#backup root@example.com:/var/db/dump/  example.com/
#backup_exec    /bin/date "+ backup of example.com ended at %c"
 
# CVS.SOURCEFORGE.NET
#backup_script  /usr/local/bin/backup_rsnapshot_cvsroot.sh      rsnapshot.cvs.sourceforge.net/
 
# RSYNC.SAMBA.ORG
#backup rsync://rsync.samba.org/rsyncftp/       rsync.samba.org/rsyncftp/

with no comments

config_version  1.2

snapshot_root   /mnt/backup1
cmd_cp          /bin/cp

cmd_rm          /bin/rm

cmd_rsync       /usr/bin/rsync

cmd_logger      /usr/bin/logger

cmd_du          /usr/bin/du

cmd_rsnapshot_diff      /usr/bin/rsnapshot-diff

retain  daily   7
retain  weekly  4
retain  monthly 3

verbose         2

loglevel        3

logfile         /var/log/rsnapshot.log

lockfile        /var/run/rsnapshot.pid

rsync_short_args        -av
rsync_long_args --delete        --numeric-ids   --relative      --delete-excluded

exclude         /lost+found
exclude         snapraid.*
exclude         lost*
exclude         .Trash*
exclude         /proc/*
exclude         /run/*
exclude         /tmp/*
exclude         /VMs/*

backup  /media/disk1/KarlData   KarlData/
backup  /home/                  home/

Remember! !The rsnapshot configuration file separates with tabs not spaces!

/etc/rsnapshot2.conf To run locally: sudo rsnapshot -c /etc/rsnapshot2.conf daily

sudo vim /etc/rsnapshot2.conf

with all comments

#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULE:        #
#                                               #
# This file requires tabs between elements      #
#                                               #
#################################################
 
#######################
# CONFIG FILE VERSION #
#######################
 
config_version  1.2
 
###########################
# SNAPSHOT ROOT DIRECTORY #
###########################
 
# All snapshots will be stored under this root directory.
#
# snapshot_root /var/cache/rsnapshot/ #default 
snapshot_root   /mnt/backup2
 
# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root 1
 
#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################
 
# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp          /bin/cp
 
# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm          /bin/rm
 
# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync       /usr/bin/rsync
 
# Uncomment this to enable remote ssh backups over rsync.
#
#cmd_ssh        /usr/bin/ssh
 
# Comment this out to disable syslog support.
#
cmd_logger      /usr/bin/logger
 
# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
cmd_du          /usr/bin/du
 
# Uncomment this to specify the path to rsnapshot-diff.
#
cmd_rsnapshot_diff      /usr/bin/rsnapshot-diff
 
# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec    /path/to/preexec/script
 
# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec   /path/to/postexec/script
 
# Paths to lvcreate, lvremove, mount and umount commands, for use with
# Linux LVMs.
#
#linux_lvm_cmd_lvcreate /sbin/lvcreate
#linux_lvm_cmd_lvremove /sbin/lvremove
#linux_lvm_cmd_mount    /bin/mount
#linux_lvm_cmd_umount   /bin/umount
 
#########################################
#     BACKUP LEVELS / INTERVALS         #
# Must be unique and in ascending order #
# e.g. alpha, beta, gamma, etc.         #
#########################################
#retain alpha   6
#retain beta    7
#retain gamma   4
#retain delta   3
 
#retain hourly  6
retain  daily   7
retain  weekly  4
retain  monthly 3
 
 
############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################
 
# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      Everything
#
verbose         2
 
# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
# If you want the rsync output, you have to set it to 4
#
loglevel        3
 
# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
logfile         /var/log/rsnapshot.log
 
# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile        /var/run/rsnapshot.pid
 
# By default, rsnapshot check lockfile, check if PID is running
# and if not, consider lockfile as stale, then start
# Enabling this stop rsnapshot if PID in lockfile is not running
#
#stop_on_stale_lockfile         0
 
# Default rsync args. All rsync commands have at least these options set.
#
rsync_short_args        -av
rsync_long_args --delete        --numeric-ids   --relative      --delete-excluded
 
# ssh has no args passed by default, but you can specify some here.
#
#ssh_args       -p 22
 
# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
du_args         -csh
 
# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs         0
 
# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns. 
# 
#include        ???
#include        ???
exclude         /lost+found
exclude         snapraid.*
exclude         lost*
exclude         .Trash*
exclude         /proc/*
exclude         /run/*
exclude         /tmp/*
exclude         /VMs/*
 
# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file   /path/to/include/file
#exclude_file   /path/to/exclude/file
 
# If your version of rsync supports --link-dest, consider enabling this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest      0
 
# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot alpha"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first     0
 
# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes       0
 
# Number of rsync re-tries. If you experience any network problems or
# network card issues that tend to cause ssh to fail with errors like
# "Corrupted MAC on input", for example, set this to a non-zero value
# to have the rsync operation re-tried.
#
#rsync_numtries 0
 
# LVM parameters. Used to backup with creating lvm snapshot before backup
# and removing it after. This should ensure consistency of data in some special
# cases
#
# LVM snapshot(s) size (lvcreate --size option).
#
#linux_lvm_snapshotsize 100M
 
# Name to be used when creating the LVM logical volume snapshot(s).
#
#linux_lvm_snapshotname rsnapshot
 
# Path to the LVM Volume Groups.
#
#linux_lvm_vgpath       /dev
 
# Mount point to use to temporarily mount the snapshot(s).
#
#linux_lvm_mountpath    /path/to/mount/lvm/snapshot/during/backup
 
###############################
### BACKUP POINTS / SCRIPTS ###
###############################
 
# LOCALHOST
backup  /media/disk1/Media      Media/
#backup /home/          localhost/
#backup /home/          localhost/
#backup /etc/           localhost/
#backup /usr/local/     localhost/
#backup /var/log/rsnapshot              localhost/
#backup /etc/passwd     localhost/
#backup /home/foo/My Documents/         localhost/
#backup /foo/bar/       localhost/      one_fs=1, rsync_short_args=-urltvpog
#backup_script  /usr/local/bin/backup_pgsql.sh  localhost/postgres/
# You must set linux_lvm_* parameters below before using lvm snapshots
#backup lvm://vg0/xen-home/     lvm-vg0/xen-home/
 
# EXAMPLE.COM
#backup_exec    /bin/date "+ backup of example.com started at %c"
#backup root@example.com:/home/ example.com/    +rsync_long_args=--bwlimit=16,exclude=core
#backup root@example.com:/etc/  example.com/    exclude=mtab,exclude=core
#backup_exec    ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"
#backup root@example.com:/var/db/dump/  example.com/
#backup_exec    /bin/date "+ backup of example.com ended at %c"
 
# CVS.SOURCEFORGE.NET
#backup_script  /usr/local/bin/backup_rsnapshot_cvsroot.sh      rsnapshot.cvs.sourceforge.net/
 
# RSYNC.SAMBA.ORG
#backup rsync://rsync.samba.org/rsyncftp/       rsync.samba.org/rsyncftp/

with no comments

config_version  1.2

snapshot_root   /mnt/backup2
cmd_cp          /bin/cp

cmd_rm          /bin/rm

cmd_rsync       /usr/bin/rsync

cmd_logger      /usr/bin/logger

cmd_du          /usr/bin/du

cmd_rsnapshot_diff      /usr/bin/rsnapshot-diff

retain  daily   7
retain  weekly  4
retain  monthly 3

verbose         2

loglevel        3

logfile         /var/log/rsnapshot.log

lockfile        /var/run/rsnapshot.pid

rsync_short_args        -av
rsync_long_args --delete        --numeric-ids   --relative      --delete-excluded

exclude         /lost+found
exclude         snapraid.*
exclude         lost*
exclude         .Trash*
exclude         /proc/*
exclude         /run/*
exclude         /tmp/*
exclude         /VMs/*

backup  /media/disk1/Media      Media/

Remember! !The rsnapshot configuration file separates with tabs not spaces!

As these backup are made to nfs mounts it is very important the at nfs drive are mounted, or the backups will go into the local mount points and fill the local disks.

String the Scripts Together into a Backup Master Script (Rsnapshot)

Superseded, tl;dr;

sudo vim ~/Myscripts/server_backup.sh

server_backup.sh code

#!/bin/bash
 
if [ "$1" != "" ]; then
    case "$1" in
        weekly | monthly | quarterly)
               period="$1"
        ;;
        *)
               period="weekly"
        ;;
    esac
else
    period="weekly"
fi
 
strf='/var/log/backup.log'
str0='/home/baumkp'
str1='/Myscripts/'
ipmi_start='ipmi_start_05_40.sh'
ipmi_stop='ipmi_stop_05_40.sh'
# $rtn  is the saved global error code from last command (failed)
# $func1 is the glabal of function name that failed 
# $period is the rsnapshot period to be used: weekly, monthly, or quarterly : default is weekly 
 
function successtype 
{
# This function performs the success response
# $rtn is the inputed return
# $strf is the log file
# function input 1:
  # 0 : return on error with error status
  # 1 : (default & not 0) exit on error with error status
 
if [ "$1" != "" ]; then
    case "$1" in
        0)
               return_type="$1"
        ;;
        *)
               return_type=1
        ;;
    esac
else
    return_type=1
fi
 
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
 
if [ "$rtn" == "0" ]
  then
    echo "Successful"  >>"$strf"
    return "$rtn"
  else
    echo "Unsuccessful" "$rtn"  >>"$strf"
    if [ "$return_type" == "0" ]
      then
        return "$rtn"
      else
        exit "$rtn"
    fi
fi
}
 
#Script to Startup remote IPMI Server
 
echo "--------------------------------------------------"  >>"$strf"
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "Start Backup Server" "$period"   >>"$strf"
 
func1="Remote Backup Server Started"
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "$func1" | awk '{printf "%s  ", $0}'           >>"$strf"
cmd1="${str0}${str1}${ipmi_start}"
bash "$cmd1"
rtn=$?
#rtn=10  # for error on test
successtype
 
#Script to mount remote backup drive
func1="Mount /mntbackup1"
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "$func1" | awk '{printf "%s  ", $0}'           >>"$strf"
 
if grep -qs "/mnt/backup1" /proc/mounts; then
    #Already mounted, log success
    echo "already mounted"   >>"$strf"
  else
    mount -t nfs -o rw,vers=4 192.168.1.5:/export/Disk1 /mnt/backup1
    rtn=$?
    successtype
  fi
 
function run_cmd
{
  # use global $strf for log file
  # use global $cmd1 for bash command to be run
  # use global $str3 for log descriptor string
  # use global $rtn to store function return value
 
  echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
  echo "${str3}" | awk '{printf "%s ", $0}'   >>"$strf"
 
  eval "${cmd1}"
  #echo "$cmd1" | awk '{printf "%s  ", $0}'           >>"$strf" # test script
  rtn=$?
  #rtn=10 # for error on test
  successtype 0
  return "$rtn"
}
 
#Script to run Rsnapshot Karl Data 1a
cmd1="rsync -a -u --delete '/media/disk1/KarlData/Karl Data 1' 'baumkp@192.168.1.5::all/media/Disk1/Karl Data 1/.sync' --exclude=snapraid.* --exclude=.Trash* --exclude=lost*"
str3="rsync /media/disk1/KarlData/Karl Data 1"
run_cmd
cmd1="rsnapshot -c /etc/rsnapshot1a.conf ${period}"
str3="/etc/rsnapshot1a"
#backup  /media/disk1/KarlData/Karl Data 1       KarlData/Karl Data 1
run_cmd
 
 
#Script to run Rsnapshot Karl Data 1b
cmd1="rsync -a -u --delete '/media/disk1/KarlData/Karl Data 2' 'baumkp@192.168.1.5::all/media/Disk1/Karl Data 2/.sync' --exclude=snapraid.* --exclude=.Trash* --exclude=lost*"
str3="rsync /media/disk1/KarlData/Karl Data 2"
run_cmd
cmd1="rsnapshot -c /etc/rsnapshot1b.conf ${period}"
str3="/etc/rsnapshot1b"
#backup  /media/disk1/KarlData/Karl Data 2       KarlData/Karl Data 2
run_cmd
 
#Script to run Rsnapshot Karl Data 1c
cmd1="rsync -a -u --delete '/home/' 'baumkp@192.168.1.5::all/media/Disk1/home/.sync' --exclude=snapraid.* --exclude=.Trash* --exclude=lost*"
str3="rsync /home/"
run_cmd
cmd1="rsnapshot -c /etc/rsnapshot1c.conf ${period}"
str3="/etc/rsnapshot1c"
#backup  /home/      home/
run_cmd
 
#Script to unmount NFS on /mnt/backup1
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
if grep -qs "/mnt/backup1" /proc/mounts; then
  /bin/umount /mnt/backup1
  if [ $? -eq 0 ]; then
    echo "Successfully unmounted /mnt/backup1"     >>"$strf"
  else
    echo "umount /mnt/backup1 failed"     >>"$strf"
  fi
else
  echo "/mnt/backup1 not actually mounted"     >>"$strf"
fi
 
#Script to mount remote backup drive
func1="Mount /mntbackup2"
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "$func1" | awk '{printf "%s  ", $0}'           >>"$strf"
 
if grep -qs "/mnt/backup2" /proc/mounts; then
    #Already mounted, log success
    echo "already mounted"   >>"$strf"
  else
    mount -t nfs -o rw,vers=4 192.168.1.5:/export/Disk2 /mnt/backup2
    rtn=$?
    successtype
  fi
 
#Script to run Rsnapshot Karl Data 2
cmd1="rsync -a -u --delete '/media/disk1/Media' 'baumkp@192.168.1.5::all/media/Disk2/Media/.sync' --exclude=snapraid.* --exclude=.Trash* --exclude=lost*"
str3="rsync /media/disk1/Media"
run_cmd
cmd1="/usr/bin/rsnapshot -c /etc/rsnapshot2.conf ${period}"
str3="rsnapshot2"
#backup  /media/disk1/Media      Media/
run_cmd
 
#Script to unmount NFS on /mnt/backup2
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
if grep -qs "/mnt/backup2" /proc/mounts; then
  /bin/umount /mnt/backup2
  if [ $? -eq 0 ]; then
    echo "Successfully unmounted /mnt/backup2"     >>"$strf"
  else
    echo "umount /mnt/backup2 failed"     >>"$strf"
  fi
else
  echo "/mnt/backup2 not actually mounted"     >>"$strf"
fi
 
#Script to mount remote Storage drive
func1="Mount /mnt/Sorage"
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "$func1" | awk '{printf "%s  ", $0}'           >>"$strf"
 
if grep -qs "/mnt/Storage" /proc/mounts; then
    #Already mounted, log success
    echo "already mounted"   >>"$strf"
  else
    mount -t nfs -o rw,vers=4 192.168.1.5:/export/Disk4 /mnt/storage
    rtn=$?
    successtype
  fi
 
#rsync main media files to backup (rsync daemon is enabled service on backup server)
cmd1="rsync -a -u --delete /media/disk2/Media/ baumkp@192.168.1.5::all/media/Disk4/Media/.sync --exclude=snapraid.* --exclude=.Trash* --exclude=lost*"
str3="rsync /media/disk2/Media/"
run_cmd
cmd1="/usr/bin/rsnapshot -c /etc/rsnapshot4.conf ${period}"
str3="rsnapshot4"
#backup  /media/disk2/Media      Media/
#run_cmd # decided not to history backup this one.
 
#Script to unmount NFS on /mnt/storage
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
if grep -qs "/mnt/storage" /proc/mounts; then
  /bin/umount /mnt/storage
  if [ $? -eq 0 ]; then
     echo "Successfully unmounted /mnt/storage"     >>"$strf"
  else
     echo "umount /mnt/storage failed"     >>"$strf"
  fi
else
  echo "/mnt/storage not actually mounted"     >>"$strf"
fi
 
#Script to shutdown 
 
func1="Remote Backup Server Stop"
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "$func1" | awk '{printf "%s  ", $0}'           >>"$strf"
cmd1="${str0}${str1}${ipmi_stop}"
bash "$cmd1"
rtn=$?
#rtn=10  # for error on test
successtype
 
 
echo $(date +"%Y-%m-%d %H:%M:%S") | awk '{printf "%s  ", $0}'           >>"$strf"
echo "Backup Server Script Ended"   >>"$strf"
 
exit 0

← Prev

Start page

Next →

Back-up Server Old Setup

Table of Contents