• View
• Edit
• History
• Attach
• Print
• Backlinks

BackupWithRsync

Updated: Tue Jul 10 22:33:06 CDT 2012 by tamouse

Description and Usage

Backups are a hugely important thing when you have a wiki full of information. PmWiki is not a difficult system to keep backed up, as everything is stored in flat files on the server. There have been a few solutions proposed. This method uses the a rolling backup method, using a hard-linking scheme to only pull over the changes in each backup iteration.

Obtaining wikibackups.pl

Download wikibackups.zipΔ to the system which will run the script and store the backups, and unzip into a safe place.

Alternately, you can make a clone from the git repository at gitorious and run make in the directory.

Dependencies

Wikibackups.pl uses the following perl modules:

• YAML::Syck
• Data::Dumper
• Getopt::Std (bundled in perl dist)

Obtain the first two from cpan.org and install as usual.

Installation

• copy wikibackups.pl to some place on your PATH, such as /usr/local/bin.
• copy wikibackups.1 to a useful man path (/usr/local/share/man/man1 perhaps).
• copy sample.wbu.rc to $HOME/.wbu.rc

Configuration

To configure wikibackups,

• edit the configuration file in $HOME/.wbu.rc by putting in the wikiname, SOURCE, and TARGET parameters, and adjusting exclusions as you see fit. You can put information for multiple wikis in the same configuration file, just replicate the structure that is there and name each wiki appropriately.
  • note that $HOME/.wbu.rc is a YAML file. MAKE SURE YOU DON'T USE ANY TABS IN THIS FILE
• ensure the TARGET directory exists; create it if it does not.

Running the script

    wikibackups.pl [-c config] [-n] [-v] [-d]

    wikibackups.pl [-h]

    wikibackups.pl [-V]

• -c config - specify an alternate configuration file. The default is "$HOME/.wbu.rc".
• -n - dry run only. If used with -v will show the list of files which rsync would have transfered.
• -v - verbose mode. Will print out what is going on during the backup, including the output from the rsync command.
• -d - debug mode. Will print out all sorts of internal structure information. Useful for checking your configuration file. If you want to check your configuration, make sure to set -n and -v as well.
• -h - help. Prints a usage message.
• -V - version. Prints the script version.

Testing

I strongly encourage thorough testing of the backup mechanism before automating it. In addition, make sure you can recover files you want from the backups generated.

Maintenance

This recipe will create backup versions, but it won't delete any. You may want to have something that will prune off old backup versions. The entire backup set should not take that much space unless your wiki experiences a great deal of entropy, since using the hard linking feature of *nix only takes extra space with files actually change, but it can be disconcerting to see many directories.

Further Info

See the man page, html page or txt page in the wikibackups installation for more information on configuring and running the backup.

Updated: tamouse October 23, 2011, at 02:26 PM

rsync is an extremely useful solution to backing up files to or from a remote location, provided you have ssh access to the remote machine. The rsync man page lists tons of options and examples for doing this well.

A simple example of using rsync to replicate your wiki:

    $ rsync -av user@server:path/to/wiki/ local/path/to/backup

This will bring over everything, including the pmwiki software and such, which you probably don't really need in a backup situation.

Inclusions and Exclusions: rsync filtering

Luckily, rsync let's you include and exclude things by using an --exclude-from parameter that you can drop file glob patterns into.

It's a bit arcane, and the man page is less than helpful, but I've come up with a set of filter rules that seem to work for backing up the important content of your wiki.

Create a file called, say, exclusions, and fill it with:

    # Exclude stuff from pmwiki except some
    + /cookbook/
    + /wiki.d/
    - /wiki.d/.flock
    - /wiki.d/.pageindex
    - /wiki.d/.lastmod
    - /wiki.d/*,del-*
    - /wiki.d/*/*,del-*
    + /pub/
    + /local/
    + /uploads/
    - /*
    - **~
    - **,v
    - **.bak
    - **.tgz
    - **.zip
    - **.gz
    - **.Z

This will allow the files matching include patters (prefix "+") and prevent the files matching exclude patterns (prefix "-") from being included in the rsync. Note well that the include patterns need to come before the exclude patterns. Here's what's going on above:

This last line here is really the key to the whole thing. If you don't have this line, you'll still get most everything in the installation. It basically says "exclude everything else from here".

What follows are more miscelaneous matching patterns for various kinds of files that may appear in the included directories up above.

(Note: you may have good reason not to exclude some of the types of files I've shown above. Feel free to modify the above list; add and subtract entries as needed.)

Once you have your exclude file set up, then run the following command:

    $ rsync -av --exclude-from=exclusions user@server:path/to/wiki local/path/to/backup

Rsync has a nice parameter, --dry-run, which lets you see what it's going to do before you commit to the download. This is great for checking yout inclusions and exclusions to make sure nothing comes over that you don't want.

There are many more options to rsync that can make it the ideal backup solution, including making incremental and full backups, etc.

For a more robust backup setup using rsync, see http://www.sanitarium.net/golug/rsync_backups_2010.htm as it outlines a rather nice rolling backup. -- tamouse July 10, 2012, at 10:59 AM

Just keep in mind the 3 most important things:

BACKUPS, BACKUPS, BACKUPS

Notes

Change log / Release notes

• Version 0.1: 2012-07-10

If the recipe has multiple releases, then release notes can be placed here. Note that it's often easier for people to work with "release dates" instead of "version numbers".

See also

• PmWiki:BackupAndRestore
• Cookbook:BackupPages

Contributors

• Tamouse

Comments

See discussion at BackupWithRsync-Talk