In order to use our backup system you need to purchase backup
storage (the pricing is
listed online), then choose which files and directories are to be backed up
and which are to be skipped. The details of the process are discussed
below. The backups are executed daily during the night, so any changes made to
the configuration will be reflected in backups the next day. The backup servers
are located in Weill Hall.
Purchase Backup Storage
- First time users must start by purchasing backup storage by clicking on the "Purchase Backup Credit Account" button on the bottom of the "My Storage" page.
- Backup Storage is purchased in 1 TB-year increments, similarly as
main storage. How long your purchased storage will last depends on
the backup size - it is similar to storage-quota relation (see the
main storage page for details). This means if you purchase 1TB-year of backup
storage and your backup size is 0.5 TB then your 1TB-year of purchased backup storage will last 2 years.
If you backup size is 2TB then your 1TB-year purchase will expire after 6 months.
- Backup Storage Usage is calculated and reported every day on your "My Storage" page
and your remaining backup storage and expiration date recomputed
- A default name is given to your new Backup Credit Account after you accept the Purchase and an invoice is created
(the name may be changed after the purchase from the status table on the "My Storage" page).
Add Directories/Files to Backup Storage
- Click on the "Add or modify Backup Storage" button on the bottom of the "My Storage" page.
- This will bring up the "Add Backup Storage" Page.
- Here you enter the Directory you want to backup in the Text Box and click on the Add button
- Use the default Server: Network Storage to backup your directories which start with /home.
- You may change the "Server: Network Storage"" text box to the name of any specific server which you can access and where you have files that require backup.
This typically applies to BioHPC Lab hosted servers.
- Once a Directory has been added you will be able to "Manage Excludes" and edit the "Retention", "Frequency" and "MinSave" Values.
Exclude Directories/Files from Backup Directories
- Click on the "Manage Excludes" button to list the Directories/Files in the "Backup Directories"
- Click on the "Exclude checkbox" to remove the Directories/Files from the Backup Process.
- Repeat the "Add Directories/Files to Backup Storage" step for all directories you would like add to your Backup Account
- Check the "My Storage" page regularly for current Backup Credit Account Status.
You will be notified by e-mail when your purchased backup storage is about
How Does Backup Work
Using the procedure described below, the user specifies one or more directories they wish to
back up. Each such directory becomes a backup root. A typical example would be your home directory,
although it is also possible to specify other directories, located on hosted servers. Each backup root
will be backed up entirely (reccursively with all files and subdirectories) except subdirectories or files
When backup of a given directory is being done for the first time, the entire directory (except exclusions)
will be copied to the backup server, i.e., its current snpashot will be created reflecting the directory's
state at backup time. Next time the backup runs, this current snapshot will be updated, i.e., files removed,
added, or changed by the user in the meantime in the source directory will be also removed from, added to, or changed
in the current snapshot.
However, the files that have been removed, as well as previous versions of those that changed, will be saved on
the backup server in a backup snapshot labeled with backup date and time. The backup snapshot contains only
files that have been changed or removed by the user from the source directory since the previous backup cycle.
Subsequent backup cycles will update the current snapshot, create new backup snapshots, and remove the older ones.
Thus, the backup server will always contain the current snpashot, reflecting the state of the directory from before
the latest backup, plus a number of dated backup snapshots containing files changed or removed between previous backup cycles.
Multiple snapshots facilitate retrieval of old versions of all files, whenever needed.
The maximum age of the backup snapshots to be kept is configurable by the user.
Meaning of the Backup Parameters
Backup is controled by three parameters, set individually for each backup root directory:
- Retension: age (in days) of the oldest version of the directory to be kept
- Frequency: backup frequency (e.g., setting this to 3 means backup of this directory will be run every 3 days)
- MinSave: minimum number of old versions of the directpry to be saved always, regardless of age
(prevents a possibility of all previous versions being erased if the original directory is not changed for longer
than Retension days)
Typical Backup Scenarios
Depending on your needs, you may consider two basic backup strategies: back up most, exclude some
and back up some, exclude most.
- back up most, exclude some: Specify some top-level directory (such as your homoe directory)
as backup root, possibly with a few exclusions. The advantage is that all changes you make to this
directory (except excluded parts) will be reflected in the backup without you taking any extra effort.
However, if you add some large files which you did not really intend to back up but forget to exclude them,
they will be copied to the backup server and you will be charged for space-time they occupy.
- back up some, exclude most: Backup only one (or more) individual subdirectory of your home directory,
the content of which you consier most important. To do this, you need to specify this subdirectory (rather
than your entire home directory) as backup root. The advantage is that changes you make outside of backup root
will not junk up the backup. However, if any of these changes are important
but you forget to copy or move them into backup root, these changes will not be reflected in the backup.
Accessing Your Backup
Backup directories exported from backup server are mounted on our login nodes, cbsulogin and cbsulogin2.
Each user-specified backup root has a corresponding location under /backups/backup1 on both login nodes.
This location reflects the owner, source server, and backup root. The picture below shows three examples, with different
parts of the path color-coded for clarity.
My Storage page
The first two backup roots are within home directories located on Network Storage. The last
backup root is located on a hosted server cbsubscb02.
Each of these locations is, in turn, organized in current snapshot and backup snapshot directories. For example,
listing the content of the first of the directories above will show
The directory current contains the current snapshot, whereas the bak_* directories (each marked with the
date) contain files changed or deleted between the date of the directory and the backup cycle preceding it.