Sunday, May 30, 2010

Simple Backup for Spare-Time-Projects

In my spare time -- the remainder of the time my child, wife and job don't demand ;) -- I do some programming for private projects.

I was searching a very simple backup solution considering that I add new stuff just once in a while. The requirements:

  • Check once a day if I added new stuff
  • Just in case new or changed files are detected do the backup
  • Store the last 12 files locally on the development server
  • All backups should be stored on a remote FTP server

Finally, I wrote the following shell-script that does the job:
(If you like to use it just change the variable at the beginning and the FTP commands below):

#!/bin/bash

# Adjust those parameters to your needs
 
REPO_PATH="/var/myproject/"
BACKUP_PATH="/mybackup/myproject/"
BACKUP_FILE_NAME="myproject_backup"
TIME_FILE="${BACKUP_PATH}last_backup_time"
NR_OF_BACKUPS=12
BACKUP_EXCLUDES="/log/"


BACKUP_FILE="${BACKUP_PATH}${BACKUP_FILE_NAME}"

if [ ! -f "$TIME_FILE" ]
then
  echo "No $TIME_FILE found. Create one!"
  touch -d 731211 $TIME_FILE
fi

CHANGED_FILES=`find $REPO_PATH -newer $TIME_FILE | grep -v "$BACKUP_EXCLUDES" | wc -l`

echo "Found $CHANGED_FILES files in backup folder!"

if [ "$CHANGED_FILES" -gt "0" ]
then
  # do the backup
  tar -cjf "${BACKUP_FILE}_$(date +%F_%H%M%S).tar.bz2" -C $REPO_PATH . && touch $TIME_FILE && echo "Backup succeeded!" || echo "Backup FAILED !!!" 
  # put tar on FTP-Backup
  echo Putting backup-tar on FTP-Server
  cd $BACKUP_PATH
  TAR_FILE=`ls -tr1 *.tar* | tail -1`
  ftp -n -i my-ftp-server.hosteurope.de <<EOF
user ftpuser ftppasswd
cd remoteDir
put $TAR_FILE
quit
EOF
  # should old ones be removed ?
  COUNT_BACKUPS=`ls -l ${BACKUP_FILE}* | wc -l`
  if [ "$COUNT_BACKUPS" -gt "$NR_OF_BACKUPS" ]
  then
    echo Found more then $NR_OF_BACKUPS backups. Delete the oldest one.
    ls -t1 ${BACKUP_FILE}* | tail -1 | xargs rm
  fi
fi

Then add the script to your cronjobs. For example:

15 4 * * * /usr/local/bin/mybackup >/mybackup/myproject/backup.log 2>>/mybackup/myproject/error_backup.log

One final note:
If you do backups of "transactional" databases or version-control-systems, don't use the simple tar-backup-command inside the script. Instead use a "transaction-safe" application specific tool. For example in case of doing a Subversion backup use something like:

hot-backup.py --archive-type=bz2 --num-backups=12 $REPO_PATH $BACKUP_PATH && touch $TIME_FILE && echo "Backup succeeded!" || echo "Backup FAILED !!!"