{{tag>rsync backup linux debian rsnapshot}} ======Rsync====== These are my rsync notes. I also used to use rsnapshot, however I was unhappy with the performance of this application and moved to restic. The original rsnapshot notes have been rolled up and moved to the bottom of this page. =====Rsync Notes===== ====Main rsync command usage summary==== A trailing slash (/) at the end of the first argument in the rsync command: ''rsync -a dir1/ dir2'', mean “the contents of dir1”. The alternative, without the trailing slash, would place dir1, including the directory, within dir2. This would create a hierarchy that looks like: ~/dir2/dir1/[files] So; ***''rsync -a dir1/ dir2'' copies the contents only of dir1 into dir2** ***''rsync -a dir1 dir2'' copies dir1 and its contents into dir2** Where *''-a'' is the flag for archive. Keeps original dates and permissions on copy and also recurses, copied sub-directories and content. *''-u'' only copies source files where they are newer than destination files (based upon file modification dates) *''--delete'' deletes files on the destination where they are not on the source. **//Use with care!//** *''-n'' is for dry run, no changes are made, but proposed output is seen *''-P'' keeps partially copies files and reports progress *''-h''' human readable file sizes *''-v'' increase output verbosity *''-z'' compress files on transfer, note does not necessarily increase transfer speeds, dependent on file types transferred and CPU speeds Always double-check your arguments before executing an rsync command. Rsync provides a method for doing this by passing the ''-n'' or ''%%-–%%dry-run'' options. The -v flag (for verbose) is also necessary to get the appropriate output: ====rsync daemon==== ++++tl;dr;| rsync can be installed as a daemon on a computer instance To install rsync daemon ''sudo apt install rsync'' ''sudo vim /etc/rsyncd.conf'' # create new # any name you like [all] # destination directory to copy path = / # hosts you allow to access hosts allow = 192.168.1.5 hosts deny = * list = true uid = root gid = root read only = false # auth users = baumkp # secrets file = /etc/rsyncd.secrets ''sudo vim /etc/rsyncd.secrets'' username:userpassword ''sudo systemctl status rsync.service'' [status, start, stop, restart, reload, disable, enable] ++++ ====Notes on Compression option==== Compression option -z can slow down file transfer notes as well as some examples I used ++++tl;dr;| ''export RSYNC_SKIP_COMPRESS=3fr/3g2/3gp/3gpp/7z/aac/ace/amr/apk/appx/appxbundle/arc/arj/arw/asf/avi/bz2/cab/cr2/crypt[5678]/dat/dcr/deb/dmg/drc/ear/erf/flac/flv/gif/gpg/gz/iiq/iso/jar/jp2/jpeg/jpg/k25/kdc/lz/lzma/lzo/m4[apv]/mef/mkv/mos/mov/mp[34]/mpeg/mp[gv]/msi/nef/oga/ogg/ogv/opus/orf/pef/png/qt/rar/rpm/rw2/rzip/s7z/sfx/sr2/srf/svgz/t[gb]z/tlz/txz/vob/wim/wma/wmv/xz/zip'' ''rsync %%--%%skip-compress=$RSYNC_SKIP_COMPRESS .....'' ''sudo rsync %%--%%dry-run -av -u --delete /media/Disk1/ baumkp@192.168.1.10::all/media/disk1 --exclude=snapraid.* %%--%%exclude=.Trash* %%--%%exclude=lost* >tmp.txt'' ''sudo rsync -av -Puh %%--%%delete /media/Disk1/ baumkp@192.168.1.10::all/media/disk1 %%--%%exclude=snapraid.* %%--%%exclude=.Trash* %%--%%exclude=lost* >tmp.txt'' Do not use from old to new on disk 2! ''xxx sudo rsync x %%--%%dry-run -avz -u %%--%%delete /media/Disk2/ baumkp@192.168.1.10::all/media/disk2 %%--%%exclude=snapraid.* %%--%%exclude=.Trash* >tmp.txt'' ''xxx sudo rsync x -a -Puh %%--%%delete /media/Disk2/ baumkp@192.168.1.10::all/media/disk2 %%--%%exclude=snapraid.* %%--%%exclude=.Trash* >tmp.txt'' ''sudo rsync %%--%%dry-run -avz -u %%--%%delete /Disk2/ baumkp@192.168.1.10::all/media/disk2 %%--%%exclude=snapraid.* %%--%%exclude=.Trash* >tmp.txt'' ''sudo rsync %%--%%dry-run -avz -u %%--%%delete ~/Myscripts/ /home/shared/Myscripts'' ''sudo rsync x %%--%%dry-run -avz -u /home/shared/temp/ baumkp@192.168.1.10::all/home/shared/html_kptree.net %%--%%exclude=.Trash* >tmp.txt'' ''sudo rsync x -a -Puh /home/shared/temp/ baumkp@192.168.1.10::all/home/shared/html_kptree.net %%--%%exclude=.Trash* >tmp.txt'' ++++ ====My web page copy bash batch==== ''sudo vim Myscripts/rsync_shared.sh'' or to run ''sudo bash Myscripts/rsync_shared.sh'' #Note this is a local only script and takes no additional batch inputs when run! ++++code| #!/bin/bash wwwpath='/home/shared/www/html' workpath='/home/shared/html_kptree.net/' cmd="rsync -ptoguv --chown=root:www-data --chmod=a+rwx,g+rwx,o-wx" $cmd ${workpath}/styles.css ${wwwpath} $cmd ${workpath}/w3.css ${wwwpath} $cmd ${workpath}/index.html ${wwwpath} $cmd ${workpath}/email_server_w3.html ${wwwpath} ++++ ====some rsync flag definitions==== Simply type ''rsync'' at terminal to see a complete list ^Flag Short^ Flag Long ^Description^ |''-z''| ''%%--%%compress'' |compress file data during the transfer| |''-v''| ''%%--%%verbose'' |increase verbosity| |''-a''| ''%%--%%archive'' |archive mode; equals -rlptgoD (no -H,-A,-X)| |''-r''| ''%%--%%recursive'' |recurse into directories| |''-l''| ''%%--%%links'' |copy symlinks as symlinks| |''-p''| ''%%--%%perms'' |preserve permissions| |''-t''| ''%%--%%times'' |preserve modification times| |''-o''| ''%%--%%owner'' |preserve owner (super-user only)| |''-g''| ''%%--%%group'' |preserve group| |''-D''| |same as --devices --specials| | | ''%%--%%devices'' |preserve device files (super-user only)| | | ''%%--%%specials'' |preserve special files| |''-H''| ''%%--%%hard-links'' |preserve hard links *| |''-A''| ''%%--%%acls'' |preserve ACLs (implies -p)| |''-X''| ''%%--%%xattrs'' |preserve extended attributes| |''-n''| ''%%--%%dry-run'' |perform a trial run with no changes made| | | ''%%--%%delete'' |delete extraneous files from destination| |''-P''| |same as %%--partial --progress%%| | | ''%%--%%partial'' |keep partially transferred files| | | ''%%--%%progress'' |show progress during transfer| |''-h''| ''%%--%%human-readable'' |output numbers in a human-readable format| |''-u''| ''%%--%%update'' |skip files that are newer on the receiver| | | ''%%--%%exclude=PATTERN'' |exclude files matching PATTERN| | | ''%%--%%numeric-ids'' |don't map uid/gid values by user/group name| |''-R''| ''%%--%%relative'' |Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames.| | | ''%%--%%delete-excluded'' |also delete excluded files from dest dirs| Note(s): * * preservation of hard links is important when moving across rsnapshots backups between disks. ---- ====Rsync related links==== *https://linux.die.net/man/1/rsync *https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories-on-a-vps *https://www.tecmint.com/rsync-local-remote-file-synchronization-commands/ *https://download.samba.org/pub/rsync/rsyncd.conf.html *https://linux.die.net/man/5/rsyncd.conf *https://www.server-world.info/en/note?os=Ubuntu_18.04&p=rsync *https://www.atlantic.net/hipaa-compliant-cloud-storage/how-to-setup-rsync-daemon-linux-server/ *https://mywiki.wooledge.org/BashFAQ *https://serverfault.com/questions/564385/rsync-command-issues-owner-and-group-permissions-doesn%C2%B4t-change *https://linoxide.com/usr-mgmt/simple-examples-to-explain-linux-file-permissions/ *https://linoxide.com/how-tos/howto-show-file-attributes-in-linux/ ---- ====Rsnapshot==== These are some rsnapshot and related rsync notes. They are really long! Sadly the original source was only onlie for a few years. ++++Rsnapshot and Old Rsync notes, really tl;dr;| I do not use rsnapshot anymore. I just do not feel comfortable using it.\\ \\ Some Links no longer originally available: [[https://web.archive.org/web/20180911154431/http://jeffskinnerbox.me/posts/2014/Feb/08/network-backups-via-rsync-and-rsnapshot|Jeff Skinnerbox Network Back-ups via Rsync and Rsnapshot]] [[https://web.archive.org/web/20190515212443/http://jeffskinnerbox.me/posts/2014/Dec/21/cheatsheet-vim-commands/|Jeff Skinnerbox Cheatsheet: Vim Commands]] [[https://web.archive.org/web/20180828185512/http://jeffskinnerbox.me/posts/2014/May/12/conky-for-ubuntu/|Jeff Skinnerbox Conky for Ubuntu]] =====Rsync and Rsnapshot===== [[http://rsync.samba.org/|Rsync]] is one of the most widely used Linux backup solutions, and in fact, it is often used by [[http://en.wikipedia.org/wiki/Rsync|other backup tools]] as a foundational component. Rsync stands for remote synchronization tool. Rsync is a UN*X command line utility, containing a [[http://wiki.r1soft.com/display/TP/rsync+Backup|special protocol and algorithm]], that synchronizes files and directories from one location to another while minimizing data transfer by using fast incremental file transfers. (rsync has also been ported to Microsoft Windows, Mac, and other operating systems.) The first time, rsync sends all the data over the network to your backup machine. The benefit comes the next time you backup. Instead of sending all the files again, rsync only transfers files that have been changed. If no files were changed, no files get transferred. And when you want to recover data, you transfer just the specific files that you want back to your machine (using rsync or scp or telnet or whatever). It creates a full-blown snapshot for each run, but makes extensive use of [[http://www.linfo.org/hard_link.html|hard links]] for files that haven’t changed, thereby greatly reduce the disk space required. That means you get efficient, lightweight, incremental backups that look like exactly like full-blown copies (in every way). This makes Rsnapshot a great choice for server backups. This easy-to-use utility is commonly used for backing up data, but can synchronize files for any other purpose you choose to use it for. Remote sync can be better than other backup methods because of its speed, and because it doesn’t require any special permissions to execute an rsync command. With just a small knowledge of the command line, you can be backing up in no time with rsync. You can also use [[http://en.wikipedia.org/wiki/Cron|cron]] to periodically create backups. To top it off, it also has a graphics front-end tool, call [[http://www.opbyte.it/grsync/|grsync]], and [[http://www.rsnapshot.org/|rsnapshot]] to create scheduled incremental backups, if so desired. =====Install rsync, grsync, and rsnapshot===== Rsync should already be installed on most Linux system. You can install it, and the [[http://www.opbyte.it/grsync/|grsync]] & [[http://www.rsnapshot.org/|rsnapshot]] tools, using this command: sudo apt-get install rsync grsync rsnapshot =====Select Backup Storage Location===== I want to create directory for the backups that are readable by all users, but writable by only root. To do this, do the following: mkdir /mnt/backup chmod a+rwx /mnt/backup chmod o-w /mnt/backup =====Getting Familiar with rsync and Preparatory Work===== In this section, we'll learn about some of rsync's features and get some foundational stuff done. ====Dry Running a rsync Command==== Rsync has the potential of really messing up things and then doing an undo can be a tedious job. Rsync can be run in a "dry run" mode to show you the output of the command. For example, this will make a copy of ''~/tmp'': rsync -azv --dry-run /home/jeff/tmp/ /home/jeff/tmp/copy-of-tmp If the output shows exactly what you want to do, then you can remove ''%%-–%%dry-run'' option from your command and re-run it for real. ====Test rsync on Local System==== In an effort to try out rsync in the raw, I'll create some ad hoc backups of my desktop's home directory. I want to preserves file timestamp, symbolic links, permissions, and owner/group. This can be done using the rsync option ''-a''. Additional options used are ''-z'' to enable compression, and ''-v'' to provide verbose status output. sudo rsync -azv /home/jeff/src/rtl-sdr/ /mnt/backup/rsync-test1 The ''sudo'' is required since I have made ''/mnt/backup'' writable only by root. The presence or absence of the terminating ''/'' within the rsync command line is important. The following two commands produce different results: sudo rsync -azv /home/jeff/Documents/"My Maps"/ /mnt/backup/rsync-test2 sudo rsync -azv /home/jeff/Documents/"My Maps" /mnt/backup/rsync-test3 The first takes the sources files and sub-directories and writes them directly to the destination directory. The second does the same but the files and directories go within a directory called ''My Maps''. This is an important subtlety to remember! ====Testing rsync on a Remote System==== In this case, my Linux desktop machine is the local system and RPi is the remote machine. The Linux desktop has the external drive used for backups. sudo rsync -azv pi@RedRPi:/home/pi/src/nRF24L01 /mnt/backup/rsync-test4 This will first prompt you for a password for the sudo. You will get a subsequent prompt for the login as pi on the RedRPi system, and then the data will be transferred to the external drive. But when performing beyond your the firewall, you should use ssh, as shown below: sudo rsync -azv -e ssh pi@RedRPi:/home/pi/src/ /mnt/backup/rsync-test5 To setup ssh with rsync so that it can run automatically without prompting for the password, checkout "[[http://www.thegeekstuff.com/2011/07/rsync-over-ssh-without-password/|How to Setup Rsync with SSH on UNIX / Linux (rsync without password)]]". This is sometime you will want to do to support automatic, unattended backups. Basically, your going to do the following on the machine that will store the backups: cd $HOME mkdir .ssh chmod 755 .ssh ssh-keygen -N "" -f /$HOME/.ssh/id_rsa ssh-copy-id -i /$HOME/.ssh/id_rsa.pub pi@RedRPi ssh-copy-id -i /$HOME/.ssh/id_rsa.pub pi@BlackRPi sudo bash cd /root mkdir .ssh chmod 755 .ssh ssh-keygen -N "" -f /root/.ssh/id_rsa ssh-copy-id -i /root/.ssh/id_rsa.pub pi@RedRPi ssh-copy-id -i /root/.ssh/id_rsa.pub pi@BlackRPi exit Now repeat the example above: sudo rsync -azv -e ssh pi@RedRPi:/home/pi/src/ /mnt/backup/rsync-test5a You should no longer be promoted for a password, except by the first ''sudo''. (Once rsync is run under a root login, this ''sudo'' will no longer be required). ====Exclude Files / Directories from rsync==== In a typical backup situation, you might want to exclude one or more files (or directories) from the backup. You might also want to exclude a specific file type from rsync. Let's run a backup of pi@RedRPi:/home/pi/ but leave out the ''tmp'' and ''src'' directories. sudo rsync -azv -e ssh --exclude 'tmp' --exclude 'src' pi@RedRPi:/home/pi/ /mnt/backup/rsync-test6 You could also create a file that contains the exclude list ''tmp'' and ''src''. This would look like: sudo rsync -azv --exclude-from 'exclude-list.txt' pi@RedRPi:/home/pi/ /mnt/backup/rsync-test6 ====Running rsync as Root==== The above examples works fine as long as the source login (in this case pi on the RedRPi system) has the appropriate permissions. For example, this will fail: sudo rsync -azv -e ssh pi@RedRPi:/root/ /mnt/backup/rsync-test It fails because the user pi doesn't have ''root'' permissions and the directory ''/root/'' can only be entered by the user ''root''. You might want to attempt to do the following: sudo rsync -azv -e ssh root@RedRPi:/root/ /mnt/backup/rsync-test Now your asking to login to the remote system as user root. In this case you'll be prompted to provide the RedRPi's root password, but the root login may not have a password. The way around all this is to use the ''%%--%%rsync-path'' option for rsync. This will work: sudo rsync -azv -e ssh --rsync-path="sudo rsync" pi@RedRPi:/root /mnt/backup/rsync-test The ''%%--%%rsync-path'' parameter specifies a path to rsync on the remote machine. So ''rsync'' will be executed using ''sudo'', giving it root user access privileges, and now the use of the pi login will gain access to the ''/root'' directory. That is, on the remote machine, rsync is being run as root! This all requires the proper set-up of ssh with password-less access, discussed earlier. =====Configuration of ssh to Eliminate Password Prompt and Warning Messages===== Proper configuration of ssh will be needed to assure routine warning messages5 don't disrupt the smooth operation of rsync / rsnapshot. With the SSH protocol, it is the SSH client's responsibility to verify the identity of the host to which it is connecting. The host identify is established by its SSH host key. Typically, the host key is auto-created during initial SSH installation setup. SSH protocol is designed to verify the host key against a local file (that file being ~/.ssh/known_hosts) to ensure the integrity of the remote server. By default, the user is prompted to accept a new key or warned when the host key changes (like after a server upgrade, DHCP server has given that IP address to a different machine from the one that had it last time). This is a nice defense against Man-In-The-Middle attacks, but it plays havoc on scripts. If a prompt occurs, your script stops and waits for input. There are two ways we can eliminate this. One is at the script level and the other is at the system level. If we want to continue to prompt for host key checks, then we can add the configuration to our script. This can be done with OpenSSH’s -o option. Here’s an example in which we run the hostname command on a remote server: ssh -o StrictHostKeyChecking=no \ -o UserKnownHostsFile=/dev/null \ -q user@host /usr/bin/hostname -s To set this configuration system-wide, place these entries in ''ssh_config'': StrictHostKeyChecking no UserKnownHostsFile /dev/null LogLevel QUIET For our backup utility, we need to add the appropriate parameters to one of the ssh configuration files listed below: ***System-wide SSH client configuration files (i.e. /etc/ssh/ssh_config -** This files set the default configuration for all users of OpenSSH clients on that desktop/laptop and it must be readable by all users on the system. ***User-specific SSH client configuration files (i.e. $HOME/.ssh/config -** This is user's own configuration file which, overrides the settings in the global client configuration file, /etc/ssh/ssh_config. Add this to ''/etc/ssh/ssh_conf'' file to disable the host key checking and the warnings6 for all users who access systems who's names end in 'RPi'. Host *RPi StrictHostKeyChecking no UserKnownHostsFile=/dev/null In addition, add the following to the ''$HOME/.ssh/config'' file: UserKnownHostsFile ~/.ssh/known_hosts Make these changes active by restarting the ssh services via the command ''sudo service ssh restart''. With ssh properly configured, a command like sudo rsync -azv -e ssh --rsync-path="sudo rsync" pi@BlackRPi:/root/.rpi-firmware/ /mnt/backup/rsync-test8 This should run without requesting passwords, it will run on the remote as the root user, and warning messages about differing "ECDSA host key" should not get in your way. It will run without user intervention; just what we need for automated backups! =====Creating Backup Users on Remote Servers===== On all the remote systems (i.e. RedRPi and BlackRPi) you need to create the login and establish its ssh authentication keys. sudo adduser backup_user --disabled-password -u 400 sudo mkdir /home/backup_user/.ssh sudo chmod 700 /home/backup_user/.ssh Now create a rsync wrapper script: sudo mkdir /home/backup_user/bin sudo vim /home/backup_user/bin/rsync-wrapper.sh Enter the following in this file to create the script: #!/bin/bash # This script is a wrapper around rsync, so that rsync can run as root, # and at the same time, eliminate any security risks. # Location of the Log file LOG="/home/backup_user/backup.log" # Place in the log file information concerning the execution of this script echo "rsync wrapper script executed on `date`" >> $LOG echo "options passed to rsync: " $@ >> $LOG echo "---------------------------------------------" >> $LOG # Now execute the script /usr/bin/sudo /usr/bin/rsync $@ Now change the mode and owner of this file: sudo chown backup_user:backup_user /home/backup_user/* sudo chown backup_user:backup_user /home/backup_user/bin/rsync-wrapper.sh sudo chmod 754 /home/backup_user/bin/rsync-wrapper.sh ''backup_user'' is not root, and therefore, ''rsync'' can't freely move through the whole directory system , write files, and such. Linux does provide a way to give a user limited super-user privileges. To do this for the ''backup_user'', configure ''sudo'' such that it can run the rsynctool as root without being being prompted for a password. For security reasons, we what only the these tools to have ''sudo'' privileges. Using ''visudo'' command, edit the ''/etc/sudoers'' file by adding the following to the bottom of the file. # allows this user to not need a password to sudo the specified command(s) backup_user ALL=NOPASSWD: /usr/bin/rsync =====Creating Backup Users for Backup Server===== On the host server (i.e. desktop) you need to create the login (with a UID of less that 500) which will run the rsync / rsnapshot utilities and establish its ''ssh'' authentication keys. sudo adduser backup_user -u 400 sudo chown backup_user:backup_user /home/backup_user/* sudo su backup_user cd ~ ssh-keygen -N "" -f ~/.ssh/id_rsa ssh-copy-id -i ~/.ssh/id_rsa.pub backup_user@RedRPi ssh-copy-id -i ~/.ssh/id_rsa.pub backup_user@BlackRPi exit And just like the remote systems ''backup_user'' is not root, and therefore, the utilities it uses (''rsync'' and ''rsnapshot'') can't freely move through the whole directory system, write files, and such. Again using the ''visudo'' command, edit the ''/etc/sudoers'' file by adding the following to the bottom of the file. # allows this user to not need a password to sudo the specified command(s) backup_user ALL=NOPASSWD: /usr/bin/rsync backup_user ALL=NOPASSWD: /usr/bin/rsnapshot =====Configuring rsnapshot for Incremental Backups===== Our mission here is to use rsnapshot to create backups of both normal and protected/restricted files from one server to another over ''ssh'' without enabling remote root access to either server while maintaining original file attributes and permissions. Where rsync does the actual file backups, rsnapshot is responsible for the overall management of the backups. In my case, I want it to schedule a nested and rotating series of incremental backups for my systems. I want the schedule and the backup increments to be: *Every 4 hours, create an incremental backup, and store all of them for the past 24 hours *Create a daily incremental backups (from the last hourly backup), and store a daily backup for the past 7 days *Create a weekly incremental backups, and store them for the past 4 weeks *Create monthly incremental backups, and store for the past 3 months When making the backups, the contents of ''/dev'', ''/proc'', ''/sys'', ''/tmp'', and ''/run'' should be excluded because they are populated at boot (while the directories themselves are not created). The file ''/lost+found'' is filesystem-specific and doesn't need to be copied. (Note: If you plan on backing up your system somewhere other than ''/mnt'' or ''/media'' don't forget to add it to the list, to avoid an infinite loop.) rsnapshot also provides a non-command-line method for excluding file, which is what I'm using. Specifically, I have defined exclusion files for Linux and Windows based systems. For Linux, the file is called ''rsync-exclude-desktop'' and ''rsync-exclude-RPi'' and look something like this: - /var/lib/pacman/sync/* - /lost+found - /media/* - /cdrom/* - /proc/* - /mnt/* - /run/* - /tmp/* - /sys/* - /dev/* For Windows, the file is called ''rsync-exclude-windows'': - /cygdrive/c/System\ Volume\ Information/* - /cygdrive/c/Windows/System32/winevt/* - /cygdrive/c/Windows/System32/catroot/* - /cygdrive/c/Windows/System32/catroot2/* - /cygdrive/c/Windows/System32/config/* - /cygdrive/c/Windows/ServiceProfiles/* - /cygdrive/c/ProgramData/Microsoft/* - /cygdrive/c/Windows/System32/wdi/* - /cygdrive/c/Windows/System32/wfp/* - /cygdrive/c/Windows/System32/sru/* - /cygdrive/c/proc/sys/Device/* - /cygdrive/c/proc/registry* - /cygdrive/c/Windows/Logs/* - /cygdrive/c/hiberfil.sys - /cygdrive/c/pagefile.sys - /cygdrive/c/swapfile.sys - /proc/sys/Sessions/ - UsrClass.dat - ntuser* - NTUSER* - *Cache* - *cache* - *Lock* - *lock* - *LOG* - *log* - *.tmp To implement the above backup rules (and many others), you need to edit a configuration file located at ''/etc/rsnapshot.conf''. **Note: This file must only have tabs between arguments. No spaces.** (It's a quirk of rsnapshot that it requires tabs.) The configuration elements I edited are below (''/usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz'' can be used as a starting point): # location where backups will be stored snapshot_root /mnt/backup/ # rsync command executed on the remote system cmd_rsync /usr/bin/rsync # incremental backup rules retain hourly 6 retain daily 7 retain weekly 4 retain monthly 3 # rsnapshot's log file logfile /var/log/rsnapshot.log # All rsync commands have at least these options set. rsync_short_args -aev rsync_long_args --delete --numeric-ids --relative --delete-excluded # ssh args passed ssh_args -i /home/backup_user/.ssh/id_rsa # systems to be backed up, what high level directory name is to be used # and the additional arguments to pass to rsync backup / desktop/ exclude_file=/home/backup_user/rsync-exclude-desktop backup backup_user@RedRPi:/ RedRPi/ exclude_file=/home/backup_user/rsync-exclude-RPi,+rsync_long_args=--rsync-path=/home/backup_user/bin/rsync-wrapper.sh backup backup_user@BlackRPi:/ BlackRPi/ exclude_file=/home/backup_user/rsync-exclude-RPi,+rsync_long_args=--rsync-path=/home/backup_user/bin/rsync-wrapper.sh backup Sara@SaraPC:/ SaraPC/ exclude_file=/home/backup_user/rsync-exclude-windows,+rsync_long_args=--fake-super In my backup scheme, I have several remote systems (i.e. RedRPi, BlackRPi, and SaraPC), and a backup server itself (i.e. desktop). All system will use a common user called ''backup_user'', who's only purpose is to support the backup process. From the backup system, we'll be able to logon to each remote system using the ''backup_user'' ssh public key. The main trick is to set sudoers on the remote system such that it allow rsync root access to ''backup_user'', and it tell rsnapshot to use additional parameters when calling rsync. This all required to maintain the security of the remote systems, and was discussed in the text above. Use the rsnapshot ''-c'' parameter to use a different configuration file, for instance if you want to run to 2 different backup locations. If the -c parameter is not used then the default configuration file ''/etc/rsnapshot.conf'' is used. Example to use differnt configuration file ''/usr/bin/rsnapshot -c /etc/rsnapshot.backup1.conf -t daily'' =====Testing rsnapshot===== rsnapshot provides an easy way to check that your configuration file doesn't contain any syntax errors. Simply type: sudo rsnapshot configtest If all is well, it will say ‘syntax ok’. If there is a problem, it will spit errors at you. If you get errors, check that you have separated items in the file using tabs (spaces are not allowed in the configuration file). The final step to test your configuration is to run rsnapshot in test mode, using the ''-t'' option: sudo rsnapshot -t hourly This tells rsnapshot to simulate an "hourly" backup. It should print out the commands it will perform when it runs for real. If all is well, then remove the ''-t'' and run it for real to create the initial backup. This initial run is likely to run a long time, and shouldn't be do via cron as discussed below. =====Scheduling (Automating) Backups in crontab===== Linux [[https://en.wikipedia.org/wiki/Cron|cron]] is used to schedule commands to be executed periodically. You can setup commands or scripts, which will be repeatedly run at a set time. For crontab, I created a wrapper script and placed it in ''/home/backup_user/bin/rsnapshot-wrapper.sh'': #!/bin/bash # This script is a wrapper around rsnapshot, so status information is logged # and notification are made via Pushover app. It also makes sure the log file # doesn't grow too large. # Capture scripts start time STARTTIME=`date +%s` # Location of the Log file LOG="/home/backup_user/backup.log" # Number of lines in the log file required to trigger it's truncation MAXLINES=7000 # Number of lines remaining in the log file after its truncated MINLINES=2000 # Calculate the size of the Log file LOGSIZE=$( wc $LOG | awk '{ print $1 }' ) # Place in the log file information concerning the execution of this script echo -e "\n\n+++++ rsnapshot script started on `date` +++++" >> $LOG echo "options passed to rsnapshot: " $@ >> $LOG # Now execute the script /usr/bin/sudo /usr/bin/rsnapshot "$@" >> $LOG 2>&1 EXITSTATUS=$? # Capture scripts end time and calculate run time ENDTIME=`date +%s` INTERVAL=$((ENDTIME - STARTTIME)) RUNTIME="$(($INTERVAL / 60)) min. $(($INTERVAL % 60)) sec." # Place time-stamped completion message in the Log echo "rsnapshot exited with status $EXITSTATUS" >> $LOG echo "+++++ rsnapshot ran" $RUNTIME "and script completed on `date` +++++" >> $LOG # Send status notification to Pushover app /home/jeff/bin/apprise -t "Incremental Backup Status" -m "Filesystem backup took $RUNTIME and completed on `date` with exit status $EXITSTATUS. Log file $LOG has $LOGSIZE lines." # Truncate the log file if needed if [ $LOGSIZE -ge $MAXLINES ] then LINECUT=`expr $LOGSIZE - $MINLINES` sed -i "1,$LINECUT d" $LOG sed -i "1i //////////////// File truncated here on `date`. //////////////// " $LOG fi Sometime, such as when upgrading my operating system, I want to perform a true full backup. That is, a true copy and not using [[https://www.linfo.org/hard_link.html|hard links]] to files that haven't changed. To do this, I create the script below, which even prompts me for the system I wish to backup. #!/bin/bash #set -x # arguments to be used with rsync ARGS="-aev --delete --numeric-ids --relative --delete-excluded" EXCL_DESKTOP="--exclude-from=/home/backup_user/rsync-exclude-desktop --exclude=mnt/backup" EXCL_RPI="--exclude-from=/home/backup_user/rsync-exclude-RPi" EXCL_WINDOWS="--exclude-from=/home/backup_user/rsync-exclude-windows" REMOTE_WRAPPER="--rsync-path=/home/backup_user/bin/rsync-wrapper.sh" SSH_ARGS="--rsh=\"/usr/bin/ssh -o CheckHostIP=no -i /home/backup_user/.ssh/id_rsa\"" # Location of the Log file LOG="/home/backup_user/backup.log" DIR=`date +%Y_%b_%d_%H_%M` while true; do read -p "What would you like to backup? ('desktop', 'RedRPi', 'BlackRPi', or 'SaraPC') " answer case $answer in 'desktop' ) ARGUMENTS="$ARGS $EXCL_DESKTOP" SOURCE="/." DESTINATION="/mnt/backup/full-backup/desktop/$DIR" break;; 'RedRPi' ) ARGUMENTS="$ARGS $EXCL_RPI $REMOTE_WRAPPER $SSH_ARGS" SOURCE="backup_user@RedRPi:/" DESTINATION="/mnt/backup/full-backup/RedRPi/$DIR" break;; 'BlackRPi' ) ARGUMENTS="$ARGS $EXCL_RPI $REMOTE_WRAPPER $SSH_ARGS" SOURCE="backup_user@BlackRPi:/" DESTINATION="/mnt/backup/full-backup/BlackRPi/$DIR" break;; 'SaraPC' ) ARGUMENTS="$ARGS $EXCL_WINDOWS --fake-super $SSH_ARGS" SOURCE="Sara@SaraPC:/" DESTINATION="/mnt/backup/full-backup/SaraPC/$DIR" break;; [Qq]* ) echo "Exiting script." exit;; * ) echo "Please answer 'desktop', 'RedRPi', 'BlackRPi', 'SaraPC' or 'q' to quit";; esac done # Capture scripts start time STARTTIME=`date +%s` # Place in the log file information concerning the execution of this script echo -e "\n\n####" $answer "full-backup script started on `date` ####" >> $LOG echo -n "options passed to rsync: " >> $LOG echo "$ARGUMENTS $SOURCE $DESTINATION" >> $LOG # Now execute the script, also capture scripts end time, calculate run time, # and send status notification to Pushover app ( STARTTIME=`date +%s` ; eval /usr/bin/rsync "$ARGUMENTS $SOURCE $DESTINATION" >> $LOG 2>&1 ;\ EXITSTATUS=$? ;\ echo "rsync terminated with exit status $EXITSTATUS." >> $LOG ;\ echo "####" $answer "full-backup script completed at `date` ####" >> $LOG ;\ ENDTIME=`date +%s` ;\ INTERVAL=$((ENDTIME - STARTTIME)) ;\ RUNTIME="$(($INTERVAL / 60)) min. $(($INTERVAL % 60)) sec." ;\ /home/jeff/bin/apprise -t "Full Backup Status" -m "Filesystem backup took $RUNTIME and completed on `date` with exit status $EXITSTATUS." ) & echo "Full Backup Underway From '$SOURCE' To '$DESTINATION'" Under the ''backup_user'' account I established the Cron job for the backups. You can use ''sudo crontab -l'' to list the contents of crontab. To update it, use ''crontab -e'' and enter the following (Also, restart cron with ''sudo service cron restart'' to make sure the changes are in effect): # Each task to run has to be defined through a single line # indicating with different fields when the task will be run # and what command to run for the task # # To define the time you can provide concrete values for # minute (m), hour (h), day of month (dom), month (mon), # and day of week (dow) or use '*' in these fields (for 'any').# # Notice that tasks will be started based on the cron's system # daemon's notion of time and timezones. # # Output of the crontab jobs (including errors) is sent through # email to the user the crontab file belongs to (unless redirected). # # For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # Instead of the first five fields, one of eight special strings may be applied: # string meaning # ------ ------- # @reboot Run once, at startup. # @yearly Run once a year, "0 0 1 1 *". # @annually (same as @yearly) # @monthly Run once a month, "0 0 1 * *". # @weekly Run once a week, "0 0 * * 0". # @daily Run once a day, "0 0 * * *". # @midnight (same as @daily) # @hourly Run once an hour, "0 * * * *". # # Examples # @reboot #Runs at boot # @yearly #Runs once a year [0 0 1 1 *] # # # # For more information see the manual pages of crontab(5) and cron(8) # # +------------- minutes (0 - 59) # | +----------- hour (0 - 23) # | | +--------- day of month (1 - 31) # | | | +------- month (1 - 12) # | | | | +----- day of week (0 - 6) (Sunday = 0) # | | | | | +--- command to be executed # | | | | | | # | | | | | | # m h dom mon dow # minute hour day-of-month month day-of-week command 0 */4 * * * /home/backup_user/bin/rsnapshot-wrapper.sh hourly 30 2 * * * /home/backup_user/bin/rsnapshot-wrapper.sh daily 30 10 * * 6 /home/backup_user/bin/rsnapshot-wrapper.sh weekly 30 14 1 * * /home/backup_user/bin/rsnapshot-wrapper.sh monthly Finally, within some of the scripts above, you'll the utility ''apprise''. This utility make use of the push notification [[https://pushover.net/|Pushover]]: #!/bin/bash # This utility does a push notification to the service Pushover - https://pushover.net/ APPTOKEN="" USERKEY="" # Parse command line options USAGE="Usage: `basename $0` [-h] -t title -m message" while getopts ht:m: OPT; do case "$OPT" in h) echo $USAGE exit 0 ;; t) TITLE=$OPTARG ;; m) MESSAGE=$OPTARG ;; \?) # getopts issues an error message echo $USAGE >&2 exit 1 ;; esac done # Send message to Pushover to create notification curl --silent \ -F "token=$APPTOKEN" \ -F "user=$USERKEY" \ -F "message=$MESSAGE" \ -F "device=desktop" \ -F "title=$TITLE" \ -F "sound=spacealarm" \ https://api.pushover.net/1/messages.json | grep '"status":1' > /dev/null # Check the exit code to identify an error if [ $? -eq 0 ]; then exit 0 else echo "apprise failed" exit 1 fi =====Increased Security===== The final step is to lock all this down. To increase the security of the overall scheme, on the remote systems and on the local system, remove the user password from the ''backup_user'' and set the shell to a NOOP command. # increase security by deleting password and remove login shell sudo passwd --delete backup_user sudo usermod -s /bin/false backup_user =====Monitoring rsnapshot===== Monitoring and testing is an essential part of any backup procedure. There's nothing worse than finding out that your backup hasn't been working for six months on the day that your system crashes. rsnapshot has a log file that records warnings and error messages. You can find/open it here: ''/var/log/rsnapshot.log'' Inspect this log regularly to make sure your backup jobs are running smoothly. You can increase or decrease the level of logging detail in the configuration file ''/etc/rsnapshot.conf''. =====Backup from Windows Machines to Linux===== Backing up Windows systems has its own special challenges, but utilities exist to help. [[https://www.cygwin.com/|Cygwin]] is a a collection of tools that provide a Linux look and feel environment for Windows. If you install it, you can use linux commands and services on your Windows. Cygwin contains a package with [[http://cygwin.com/cgi-bin2/package-cat.cgi?file=x86%2Frsync%2Frsync-3.0.9-1&grep=rsync|its version of rsync]] to make backups from the Windows based computer. With Cygwin's rsync and the rsnapshot running on the Linux backup server, you can set up a remote backup for the Microsoft Windows box. To do the install the cygwin ''rsync'', ''ssh'', etc. tools required to support the rsnapshot backup, you'll follow a similar design pattern used above for the Linux boxes. Follow this procedure: ====Install Cygwin Tools==== *From the [[http://www.cygwin.com/|cygwin site]], download and run the setup program. (Make sure to keep the setup program, since it is used to install additional Cygwin packages, if you so desire.) *Run your cygwin ''setup.exe'' and expand the categories to find "rsync" and "ssh". You'll find them under the "Net" packages. *When the install is complete, the ''rsync'' and ''ssh'' programs (and many more) will be located in ''C:\cygwin64\bin''. (This is equivalent to ''/bin'' when your running a ''bash'' shell in cygwin.) *In Windows, open a [[https://www.howtogeek.com/howto/windows-vista/run-a-command-as-administrator-from-the-windows-vista-run-box/|Command Prompt (Admin) window]]. Within this window, get a bash shell via the command ''C:\cygwin64\bin\bash''. Next execute ''export Path=/bin:$PATH''. ====Get ssh Operational==== *Now make a home directory for the cygwin user, in my case this was ''mkdir /home/Sara'' and then ''cd /home/Sara''. *Create a ''ssh'' public/private keys with the following command: ''ssh-keygen -t rsa''. *Now start up the ''ssh'' services by following the instructions of "[[http://lifehacker.com/205090/geek-to-live--set-up-a-personal-home-ssh-server|Geek to Live: Set up a personal, home SSH server]]". *Find out the name of the windows system via ''hostname'' (it's SaraPC). *In order to access the ''ssh'' server from the backup Linux box, the ''ssh'' port of 22 must be open in the Windows Firewall. You can check ports status by attempting connect via ''ssh'' (e.g. ''ssh Sara@SaraPC''). If this command appears to hang or time out (as it did for me), the port is likely blocked. You'll need to go to your [[http://windows.microsoft.com/en-us/windows/open-port-windows-firewall#1TC=windows-7|Windows Firewall]] and open port 22. *Transfer the public key for the ''backup_user'' on the Linux Backup server to ''/home/Sara/.ssh/authorized_keys''. *Make sure that in the files ''/etc/ssh/sshd_config'' and ''/etc/ssh/ssh_config'' that you have ''RSAAuthentication yes'' and ''PubKeyAuthentication yes''. ====Test ssh and rsync===== *On the Linux Backup server, login as the ''backup_user'' user. *Test the connection via ''ssh Sara@SaraPC'' in a terminal window on the Linux box (note to self: Case is important!). You should login without a password. *Test rsync via the command ''sudo rsync -azv --fake-super Sara@SaraPC:/home/Sara /mnt/backup/rsync-test9''. You'll want to use the ''%%--%%fake-super'' option to suppress some [[https://digitaldj.net/blog/2011/04/07/rsync-3-0-8-windows-and-chown/|issues with group ids]]. *Test the ''/etc/rsnapshot.conf'' given earlier by running ''rsnapshot hourly''. =====Restore Files with rsnapshot===== Restoring files are an no brainier because the backups are plain directories. You can open a file browser or a terminal, enter a snapshot from a few hours/days/weeks/months ago, find a working directory where your files are store, and copy them to where their needed. In other words, you can use any regular tools on any snapshot. No need to “revert” or “restore” files from backup, or run any special software. This is mighty convenient, intuitive, and fool proof. =====Sources===== I found these articles useful for writing this post and setting up my rsych / rsnapshot backup scheme. ====rsync==== *[[https://www.thegeekstuff.com/2010/09/rsync-command-examples/|How to Backup Linux? 15 rsync Command Examples]] *[[http://www.tecmint.com/rsync-local-remote-file-synchronization-commands/|Rsync (Remote Sync): 10 Practical Examples of Rsync Command in Linux]] *[[https://www.thegeekstuff.com/2011/01/rsync-exclude-files-and-folders/|6 rsync Examples to Exclude Multiple Files and Directories using exclude-from]] *[[https://www.thegeekstuff.com/2011/07/rsync-over-ssh-without-password/|How to Setup Rsync with SSH on UNIX / Linux (rsync without password)]] *[[http://www.mikerubel.org/computers/rsync_snapshots/|Easy Automated Snapshot-Style Backups with Linux and Rsync]] *[[https://wiki.archlinux.org/index.php/Full_System_Backup_with_rsync|Full System Backup with rsync]] ====rsnapshot==== *[[https://kamaradski.com/1245/how-to-install-and-setup-rsnapshot-incremental-remote-backup|How to install and setup Rsnapshot incremental remote backup]] *[[http://www.cyberciti.biz/faq/linux-rsnapshot-backup-howto/|Debian / Ubuntu Linux Install and Configure Remote Filesystem Snapshot with rsnapshot Incremental Backup Utility]] *[[http://www.tecmint.com/rsnapshot-a-file-system-backup-utility-for-linux/|Rsnapshot (Rsync Based) – A Local/Remote File System Backup Utility for Linux]] *[[http://www.rsnapshot.org/howto/1.2/rsnapshot-HOWTO.en.html#create_the_config_file|rsnapshot HOWTO]] *[[http://www.cyberciti.biz/faq/restoring-backup-files-with-rsnapshot-unix-linux-bsd-appleosx/|UNIX / Linux: Rsnapshot Restore Backups]] *[[https://sites.google.com/site/rsync2u/home/rsync-tutorial/the-exclude-from-option|The "exclude-from" and "recursive" options]] *[[http://www.evbackup.com/support-misc-redundant-backups-with-rsnapshot/|Implementing Redundant Backups with rsnapshot]] ====rsnapshot with root access==== *[[http://technokracy.net/2011/01/07/root_sudo_rsnapshot/|Root, Sudo, and Rsnapshot]] *[[http://dev.kprod.net/?q=linux-backup-rsnapshot-no-root|Backup remote Linux hosts without root access, using rsnapshot]] *[[http://derek.simkowiak.net/backing-up-multiple-servers-with-rsnapshot/|Backing Up Multiple Servers with Rsnapshot]] ====rsync and rsnapshot with Windows==== *[[http://www.smellems.com/tiki-read_article.php?articleId=14|Backup from Windows to Linux with Rsync and SSH]] *[[http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html|Installing ssh and rsync on a Windows machine: minimalist approach]] *[[http://www.trueblade.com/knowledge/using-rsync-and-cygwin-to-sync-files-from-a-linux-server-to-a-windows-notebook-pc|Using rsync and cygwin to Sync Files from a Linux Server to a Windows Notebook PC]] *[[http://terokarvinen.com/rsync_from_windows.html|Rsync from Windows]] ====SSH==== *[[http://www.symantec.com/connect/articles/ssh-host-key-protection|SSH Host Key Protection]] *[[http://linuxcommando.blogspot.com/2008/10/how-to-disable-ssh-host-key-checking.html|How to disable SSH host key checking]] NOTE: [[http://www.lostsaloon.com/technology/understanding-file-linking-hard-link-vs-symbolic-link-in-linux/|Hard links are similar to symlinks]]. They are normally created using the ''ln'' command but without the ''-s'' switch. A hard link is when two file entries point to the same [[https://en.wikipedia.org/wiki/Inode|inode]] and [[https://en.wikipedia.org/wiki/Block_(data_storage)|disk blocks]]. Unlike [[http://en.wikipedia.org/wiki/Symbolic_link|symlinks]] there isn't a file and a pointer to the file, but rather two links to the same file. If you delete either entry the other will remain and will still contain the data. NOTE: This assumes local permission on the remote systems to access the files that are to be backed up. Where the backup user does not have local permission to access files, the only reasonable alternative may be key-based authentication over SSH for root. NOTE: The command ''ssh-copy-id'' may not work if your attempting use the root login instant of pi on the remote machine. In this case, you'll need to manually move the public key to its destination. The article "[[http://www.tecmint.com/ssh-passwordless-login-using-ssh-keygen-in-5-easy-steps/|Create SSH Passwordless Login Using SSH Keygen]] can show you how to do this. NOTE: Because of an abundance of paranoia, remote root login over ssh session is disabled in some Linux distributions for [[http://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server-best-practices.html|security reasons]. By default, the Root account password is locked in Ubuntu and most other version of Linux. The account has no password, even though it prompts you to provide one. This means that you cannot login as Root directly or use the su command to become the Root user. However, since the Root account physically exists it is still possible to run programs with root-level privileges. This is where [[https://help.ubuntu.com/community/RootSudo|sudo]] comes in - it allows authorized users (normally "Administrative" users) to run certain programs as Root without having to know the root password. NOTE: One of the most common warnings you will likely encounter is a host key error which may not happen until long after you have made your first connection to a host. If any of several identifying features of the host change, a new host key could be created and if that happens your ssh client will let you know by refusing to log you into that system without first prompting you to confirm the warning, or may even force you to provide the login's password. It can be confusing on how to clean this up, but these two articles can help: [[http://xmodulo.com/2013/05/how-to-accept-ssh-host-keys-automatically-on-linux.html|How to accept ssh host keys automatically on Linux]] and “[[https://stackoverflow.com/questions/9299651/warning-permanently-added-to-the-list-of-known-hosts-message-from-git|Warning: Permanently added to the list of known hosts]]” message from Git. NOTE: The suppression of this warning does reduce the security profile of the system. Instead of suppressing the warning for all ssh access to *RPi, you could implement this functionality [[http://www.garron.me/en/linux/remove-offending-key-ssh.html|solely for the ''rsync'' command]] by using the option ''-e "ssh -o CheckHostIP=no -o StrictHostKeyChecking=no"''. See [[http://security.stackexchange.com/questions/10532/ecdsa-keys-changed-ssh-insecure-now|ECDSA Keys Changed, SSH insecure now]]. NOTE: The sudoers configuration file enables a huge amount of configurability, including but not limited to: enabling root commands only from the invoking terminal; not requiring a password for certain commands; requiring a password per user or group; requiring re-entry of a password every time or never requiring a password at all for a particular command line. It can also be configured to permit passing arguments or multiple commands, and even supports commands with regular expressions. NOTE: I choose a UID of 400 so that the ''backup_user'' would not appear on the Ubuntu login screen list. To hide a user from the Ubuntu login screen list, you should be able to add the name to the hidden-users list in the file ''/etc/lightdm/users.conf'', but there is a [[http://www.cyberciti.biz/faq/howto-change-rename-user-name-id/|problem]]. The is an alternative, and that is to choose a UID value less than 500 (See the "minimum-uid" in ''/etc/lightdm/users.conf''). NOTE: To automate rsnapshot backup to a remote servers, you'll also need to set up key-based authentication over SSH on the remote machines that you want to backup, so that they can be accessed without need for a password login. And you need to make sure this arrangement survives a reboot. To accomplish this, you will need to create an SSH public and private keys to authenticate on the rsnapshot server. This was all discussed earlier. NOTE: The configuration file requires tabs between elements and all directories require a trailing slash. Just open the configuration file using a text editor such as ''vim'' or ''gedit'' but be careful. Many editors are set to convert any tabs entered by the user to spaces. This can be a source of great confusion and frustration! NOTE: The cron service (daemon) runs in the background and constantly checks the ''/etc/crontab'' file, and ''/etc/cron.*/'' directories. It also checks the ''/var/spool/cron/'' directory. Review the article [[https://help.ubuntu.com/community/CronHowto|CronHowTo]] to see how you can schedule and run tasks in the background automatically at regular intervals using crontab files. Crontab ( **cron table** ) is a file which contains the schedule of cron entries to be run and at specified times. (If the system isn't running 24x7, consider using [[http://www.thegeekstuff.com/2011/05/anacron-examples/|anacron]]). ++++ <- home_server:home_server_setup:other_services:nfs|Prev ^ home_server:home_server_setup:other_services:index|Start page ^ home_server:home_server_setup:other_services:Tmux|Next ->