Following up with my last post about why the CDP MySQL Add-On is so important to getting a good backup of your MySQL database I have been asked a lot: “What good is the point-in-time snapshot anyways if it’s not good enough for MySQL? What about XYZ application?”

Beauty of the Snapshot is in the Eye of the Beholder

I think it’s hard to talk about volume snapshots without talking about who you are and what applications you use.  If you are a database admin you have one view, a Linux server admin a different view, and a Windows server admin yet a third view.

Windows Server Admins

You know you need some way to get a backup copy of open files as Windows enforces file locking and windows application typically lock files for reading.  When you shop for backup software you are used to making sure it has some kind of support for backing up open files.  Before MS Volume Shadow Copy you may have purchased expensive third party tools like OTM (open transaction manager) to solve this problem with your backups in the past.

Linux Server Admins

You probably never thought much about snapshots or consistency of files during backup.  On Linux the root user can read any file it wants and at any time regardless if the file is being written to or not.  You can even read locked files. Basically you are not protected from yourself unlike Windows where file locking really locks files. 

Database Administrators

You know your database needs more than just a file copy when it comes to backup.  You perform a scheduled database dump at a minimum, you likely purchase add-ons from your Windows/Linux file backup vendor that supports your database (e.g. R1Soft).  You may even purchase a specialized commercial database backup tool made just for your database.

What is a point-in-time File System Snapshot (aka Open File backup)?

Windows

The CDP Agent performs an online point-in-time snapshot using the Microsoft Volume Shadow Copy Service (VSS).

For the file system this means:

1. CDP Agent requests a snapshot from the VSS service
2. Volume Shadow Copy first notifies all applications that have asked to be notified there is an impending backup operation coming and to write critical data to disk.
3. VSS locks and flushes the file system.  Writes cached in memory and not yet written to disk are now flushed through to disk storage.
4. The file system is unlocked
5. Applications are notified the snapshot is complete and to continue writing as normal
6. CDP Agent reads backup data from the snapshot

Changes to the file system (writes to a file, new files, etc) that happen after the snapshot don’t appear in the backup.

For applications this means:

On Windows we have the MS VSS Service which provides just about all the snapshot features you could ever want.  (Linux really is lacking here).  And while we love to hate VSS when it does not work… it is very powerful and credit is due here.

Any application that decided to support VSS can inform VSS that it should be informed when there is a backup operation so it can write un-saved changes in memory to disk. MS VSS calls this a VSS writer.

So the question to ask on Windows is: Does my application integrate with Microsoft VSS.  Microsoft calls this a VSS Writer.

Linux

Linux has no Volume Shadow Copy Service and no snapshot support in the popular file systems (ext2/3/4, reiserfs, XFS, etc).  To bring VSS type volume snapshots to the Linux world R1Soft developed a special device driver (kernel module) for Linux that can perform a point-in-time snapshot of any Linux block device. Now that’s not all the CDP Linux device driver does… it has some magic that perform asynchronous replication… making backup windows very short… that is another matter.

For the file system this means:

1. The CDP Agent calls a ioctl call on the CDP driver to request a snapshot
2. If the MySQL add-on is installed MySQL is locked and flushed here
3. CDP driver locks and flushes the file system.  Writes cached in memory and not yet written to disk are now flushed through to disk storage.
4. If the MySQL add-on is installed MySQL is unlocked here
5. CDP driver unlocks the file system
6. CDP Agent reads backup data from the snapshot

Changes to the file system (writes to a file, new files, etc) that happen after the snapshot don’t appear in the backup.

So the question to ask on Linux is: Is my application safe with or without just a point-in-time snapshot.  And does my Linux application need special treatment during a backup.

Linux –  backup applications that do NO Snapshot like tar, rsync, bacula, amanda are they safe?

No, tar, rsync, bacula, amanda and just about every other Linux backup application* besides R1Soft CDP are reading from files while other applications are writing to them.  They are actually WORSE OFF.  There is no point-in-time snapshot so you get a mix match of changes to the file system in your backup and there is no protection at all to backing up open files.  If you use these utilities to backup Linux you are almost guaranteeing corrupt files!  Sorry, the truth is ugly I know… best we are aware.

What does this Snapshot mean for applications?

It depends on the application and how it does disk I/O and what is doing. Here is a summary of some popular applications that I have done the homework for you on:

Windows

Life is easy as most critical apps support VSS and MS did all the hard work.

Linux

R1Soft has made your life much better by giving you a point-in-time snapshot… life is still complicated because there is no VSS and few application developers take time to solve the consistent backup problem.

*Acronis True Image and Linux backup applications that use LVM to perform snapshots assuming all your data is on LVM Logical Volumes are also capable of performing point-in-time snapshots.  Zamanda can use LVM snapshots and this is good if all your data lives on LVM and you set aside special snapshot storage ahead of time.

**PostgreSQL is safe to copy at any time because of the way its magnificent write journal works. See http://www.r1soft.com/linux-cdp/cdp-server-20/postgresql/ for more details.

***Postfix mail spool files in my opinion are safe with only a file system snapshot with the caveat that mail spool files may have partially written or deleted messages.