CommunityData:Backups (nada): Difference between revisions

From CommunityData
No edit summary
Line 42: Line 42:
  /usr/local/sbin/mysql_backup_full
  /usr/local/sbin/mysql_backup_full
  /usr/local/sbin/mysql_backup_inc
  /usr/local/sbin/mysql_backup_inc
== Minimizing Backup Size ==
If you have large datasets that are unlikely to change or be replaced, store a copy of these data in the <code>/com/</code> directory in Hyak and keep the files in <code>/home/<YOUR NAME>/nobackup</code> and then symlink them from a more convenient location.

Revision as of 02:24, 23 December 2015

Nada has about 14TB of available disk space. Although we want to backup thing everything by default, backing up all 14TB would cost about $140/month. As a result, we should be smart about what/how we back things up.

Backups on Nada

Nada backups are systemwide backups using Duplicity. The backups are incremental backups using rdiff backup (think rsync), are encrypted using GPG, and then are stored in Google Nearline storage which costs about $0.01/GB. Backups scripts run once each week.

Everything is backed up except for the directories listed in /root/duplicitity_exclude. This page may not be up to date but the following files are excluded at the time that this page was written:

/mnt
/media
/mit
/nonexistent
/openafs_cache_fs
/tmp
/var/log
/var/lib/mysql
/var/lib/mongodb
/var/lib/redis
/var/lib/postgresql
/var/spool
/var/tmp
/var/cache
/lost+found
/lolo
/cdrom
/floppy
/proc
/sys
/root/.cache
/root/nobackup
/home/*/.cache
/home/*/nobackup
/home/awjordan

Backing up Databases

MySQL Backups

MySQL databases are backed up using a separate MySQL incremental backup script that calls Percona XtraBackup. These incremental MySQL backups are created once each week before the duplicity backup script is run. To add a new MySQL database to the backup list, you must edit the following files:

/usr/local/sbin/mysql_backup_full
/usr/local/sbin/mysql_backup_inc

Minimizing Backup Size

If you have large datasets that are unlikely to change or be replaced, store a copy of these data in the /com/ directory in Hyak and keep the files in /home/<YOUR NAME>/nobackup and then symlink them from a more convenient location.