Recent Changes - Search:
Cookbook /

TwoWayMirroringWithRsync

Summary: How to keep identical copies of a wiki on two systems, one hosted and the other local
Version:
Prerequisites:
Status:
Maintainer:
Categories: SystemTools

Question

  • I'd like to keep identical copies of my wiki on two systems: One hosted and the other local. How do I do that?

Answer

  • This solution will accomplish the above goal, but with some limitations. It is not a true two-way syncronization. It works by mirroring the majority of your site from your hosted wiki to your local wiki, and then mirroring just one group (called "Local") from your local wiki back to your host. In other words, changes you make on your hosted wiki to the "Local" group will be lost after sync, and changes you make on your local wiki to anything except the "Local" group will also be lost.
  • The following instructions were written for Mac OS X, but the general idea should work for all systems.
  • The solution uses Rsync and will require access to your host via the terminal. If you don't have access, you should request it.
  1. First, backup your entire PM wiki folder from your host to your local computer, then place it in the ~/Sites folder. You can now access your PM wiki installation from the url: http://127.0.0.1/~User/PMWiki where "user" is your username and the name of the PM Wiki folder is the same as what you have used on your system.
    1. Before you can properly use your local mirror of PM Wiki you will need to adjust the config.php file as well as fix permissions so that your wiki.d folder is writable by PM Wiki.
    2. I recommend setting a different theme for your local version so you know which is which.
    3. You may also wish to set the Admin Password for the local version as follows (only do this if you are the only person who has access to your local system). It will make it so you don't need passwords on your local system:
      • $DefaultPasswords['admin'] = '';
  2. Now, create a new group on your local system version of PM wiki called "Local"
  3. To test your sync use the following command in the terminal (you might wish to make changes on both systems to see how they are handled): 
    • rsync -nrtl "username@hosteddomain.tld:/path/to/PMwiki/wiki.d" \
/Users/username/Sites/pmwiki  -e ssh --progress --exclude=".flock" \
--exclude="Local.*"; \
rsync -nrtl /Users/username/Sites/pmwiki/wiki.d/Local.* \
"username@hosteddomain.tld:/path/to/PMwiki/wiki.d" -e ssh --progress
  4. Make sure it works the way you want. You might wish to backup your wiki before running the following script. This will run the mirroring for real: 
    • rsync -rtl "username@hosteddomain.tld:/path/to/PMwiki/wiki.d" \
/Users/username/Sites/pmwiki  -e ssh --progress --exclude=".flock" \
--exclude="Local.*"; \
rsync -rtl /Users/username/Sites/pmwiki/wiki.d/Local.* \
"username@hosteddomain.tld:/path/to/PMwiki/wiki.d" -e ssh --progress

Notes

  • This recipe was last tested on PmWiki version: 2.0.12
  • This recipe requires at least PmWiki version: and (any other recipes)
  • This recipe, version...... was installed here the...(date)

Releases

Comments

I've taken this a slightly different route using Linux. First a few considerations:

  1. I want to make a local mirror backup of my entire web space.
  2. I don't have immediate concerns for using my wiki locally. (I might in the future, but I will address this when the need / desire arises.)
  3. I wanted a quick / easy way to make an incremental backup. The biggest concern here is that I don't have to use cpanel, or something else to make a zip / tar and download a massive file.
  4. I don't use any databases, or anything that might be using any external storage on my site. Everything is literally under the home directory on my host.
  5. I am doing this on my laptop, using wireless connections (as mentioned below), so I need a semi-automated process to make backups easy.

Unfortunately, I am using wireless freebie connections at coffeehouses currently. If I were on a more permanent connection, I would consider doing a couple more things:

  1. I would implement ssh keys so I could connect to my host without having to enter a password all the time. (If someone is interested in this, I can get instructions on how to set up the keys.)
  2. I would set up the following as a cron job, have it run every few (say 3 or 6) hours to keep my mirror as fresh as possible. (Implementation of this technique is left as an excercise for the user. <g>)

This technique is very simple. Basically, since I have my own linux system, and my host is running linux, I mirror the account configuration on my local system:

  • Create an account of the same name as the account on my host.
    • Make certain the shell is set to something other than the shell that your host uses. I personally use zsh for my accounts. Most hosts seem to use bash or csh.
  • Create matching groups.
    • On your hosts system, you can check the groups by typing 'groups' at a shell prompt. My host has me in a group that is the same name as my account.
  • Create the home directory for the new user.
    • Again: you want to make the permissions / settings the same as your hosts.
    • Note: many systems will do this automatically for you, however there is a caveat to remember: you don't want to have the skeleton files your system places in your home directory. If there are any .* files, remove them.

With this set up, we get to the fun / simple part... This is where the shell change becomes important. Since I use zsh, I will use that in this part of the example:

  1. Log in to your new, local account.
  2. Create a new startup file for your shell (in my case, .zshrc) in your favorite editor (vi, pico, jed, emacs, whatever.)
  3. Add an alias the shell startup file. I created an alias called sync with the following command: 
alias sync='rsync -rtl "webacct@mywebhost.com:/home/webacct/" /home/webacct/ \
-e ssh --progress --exclude=".flock" --exclude=".zshrc"\;'
  1. Replace webacct and mywebhost with the correct information for your web host and account.
  2. Change the --exclude=".zshrc" to reflect the shell you are using.
  3. Save and exit your editor, then source your configuration file:
. .zshrc
or
source .zshrc
  1. Now, go ahead and run the sync alias.

Tada! A complete local mirror of your host account. Anytime you need to update your mirror, you just log into the local account, and run the sync command. (You could, in fact, add this to your login for the account, and have it automatically sync -- but there are a few tricks to that (such as time when you don't have an internet connection, so I'm leaving that as an excercise for the user.)

  • Has anyone figured out how to do this with windows? Or do I need to get Cygwin and start learning?

See Also

  • WikiCVSStorage offers a CVS based solution. For those familiar with CVS it might work even better. I haven't tested it myself.

Contributors

Kerim Friedman
George De Bruin Local mirror of webhost account.