It’s a pretty polished software package, but out of the box it tends to assume that you are creating something like Wikipedia that is visible to (and editable by) the whole wide world. If that’s not what you want, it requires some tuning, which I’ll describe in detail after the jump.

Why MediaWiki?

An obvious question is, if the MediaWiki software needs tweaking to do what I want, why do I use it and not some other software? A few reasons:

• It’s just a Wiki. I’ve tried doing similar things with other packages (such as Tiki Wiki), and ended up wasting lots of time fighting with or trying to turn off features I didn’t want or need. A CMS or blogging package or groupware suite are great if you need those things, but they all tend to assume a certain workflow and division of responsibility. If you are just trying to create a Wiki, you’ll waste time swimming against the current.
• The Wikitext markup language is reasonably intuitive (simple things are simple, and new users can start contributing right away) but still powerful enough to create complex pages. It’s incredibly well-tested, and doesn’t flake out in odd corner-cases. There’s a huge volume of example pages out there.
• It’s decently fast even on a modest computer, without resorting to esoteric fastcgi and caching tricks.
• There are lots of useful plugins available.
• Because of the huge user base, most “How do I…?” questions are answered with a simple web search.

Prerequisites

The remainder of this article will assume that you’re starting with a fresh install of MediaWiki, and have an admin user set up (and know its password). You’ll also need the ability to edit files under the MediaWiki install tree (via command-line access or FTP or any other means).

While the methods I describe should be reasonably general, future releases may involve different ways of doing things. The process described below was tested and is known to work with the following software versions:

• MediaWiki 1.16.0
• PHP 5.3.3 (cgi-fcgi)
• MySQL 4.1.20
• Perl 5.8.8
• Apache 2.2.8

Restricting Access to Registered Users Only

The first step in creating a private Wiki (and one which is well-documented elsewhere) is to make it so that only logged-in users can view or edit pages (other than the login page), and so that only sysops can create new accounts.

Edit your LocalSettings.php to include the following lines (I made all manual additions at the bottom of the file):

# Disable reading by anonymous users
$wgGroupPermissions['*']['read'] = false;

# But allow access to the login page or there will be no way to log in!
$wgWhitelistRead = array("Special:Userlogin", "MediaWiki:Common.css",
"MediaWiki:Common.js", "MediaWiki:Monobook.css", "MediaWiki:Monobook.js", "-");

# Disable anonymous editing
$wgGroupPermissions['*']['edit'] = false;

# Prevent new user registrations except by sysops
$wgGroupPermissions['*']['createaccount'] = false;

Disabling “Remember Passwords” in MediaWiki

That’s a good start, but there’s still a big problem: the “Remember my login on this computer” checkbox on the the login form. If the user checks this (and users, being users, will check it), it stores a password-equivalent token in a cookie on the user’s browser. That means that anyone with access to the user’s computer (which, for ‘doze boxes, is likely the whole Internet) can use your Wiki as that user.

Having the browser automatically log the user in isn’t quite the same as having no password at all, but it’s pretty bad. It’s giving up a huge degree of security for a very minor convenience. If what you’re protecting has any value at all, you don’t want it.

So how to get rid of this misfeature? Add the following to your LocalSettings.php:

# Disable "remember password" on login page:
require_once('extensions/NoRememberAuthPlugin.php');
$wgAuth = new NoRememberAuthPlugin();

# Disable "remember password" on user preferences page:
$wgHooks['GetPreferences'][] = 'NoRememberPrefHook';
function NoRememberPrefHook($user, &$preferences) {
    unset($preferences['rememberpassword']);
    return true;
}

Add a file extensions/NoRememberAuthPlugin.php containing the following:

<?php
require_once ('AuthPlugin.php');
class NoRememberAuthPlugin extends AuthPlugin {
   function modifyUITemplate(& $template) {
      //disable 'remember me' box
      $template->set('remember', false);
      $template->set('canremember', false);
   }
}
?>

Notice that we also keep users from turning “remember me” back on in their preferences, and disable it (at the next login) for any users who had it enabled before we made the change.

Also notice that all the modifications so far have been edits to LocalSettings.php and the creation of a new extension. These are all things that should survive a MediaWiki upgrade intact.

Disabling “Remember Passwords” in the Browser

Now that we’ve stopped the MediaWiki software from sabotaging its own security, our next challenge is the user’s web browser. The browser may have has its own separate “remember password” (aka “form fill in” or “wallet”) feature, and some users leave that turned on. We can’t really check for or enforce a particular setting; there’s no way, from the server end, to tell if the credentials were supplied via keyboard or autocompleted by the browser.

One approach we could use (and which I did use, on a previous project) is to randomize the URL of the login page or the field names or both. The browser can still “remember” the password, but won’t know to use it on the next login. On a scale from good to bad, this is bad: password-equivalent data is still sitting on the user’s disk. It’s also not easy (for me) to implement within MediaWiki.

Fortunately, there is another option: we can put a hint in the page source to tell the browser to turn off “remember password” for a particular form. We do this by adding an autocomplete attribute to the form element, with a value of off.

There are a couple disadvantages to this approach: First, autocomplete=”off” is not part of any web standard (though it is supported in both Gecko-engine browsers like Firefox and in IE). Second, I didn’t see any obvious way to add it to the necessary places using LocalSettings.php or an extension.

There are two places where the form-fill-in behavior is problematic: on the user login screen, and on the screen where the user can change his password. For the user login screen, edit the file includes/templates/Userlogin.php, find the form with name=”userlogin” and add the autocomplete=”off” attribute. (You may want to back up the file first.) Here is part of a context diff from my installation:

-<form name="userlogin" method="post" action="<?php $this->text('action') ?>">
+<form name="userlogin" method="post" action="<?php $this->text('action') ?>" autocomplete="off">

For the password change, edit the file includes/specials/SpecialResetpass.php. Look for the code that builds an array with an element ‘id’ => ‘mw-resetpass-form’ and add a new element ‘autocomplete’ => ‘off’. Another context diff:

-                    'id' => 'mw-resetpass-form' ) ) . "\n" .
+                    'id' => 'mw-resetpass-form',
+                    'autocomplete' => 'off' ) ) . "\n" .

When you are testing this (and you should test it), remember to turn the “remember password” feature of your browser back off when you are done.

Enforcing Password Quality

Another weak link in the chain is password quality. People tend to pick bad, easily-guessed passwords. While we would like for them to not do that, it requires a delicate balancing act. If the password rules are too strict, users won’t be able to pick passwords they can remember, and will end up writing them down (which is worse in some ways than picking a weak password). If we pick really silly rules, it can dramatically narrow the search space for an attacker trying to find passwords by brute force.

On the other hand, using password quality rules has another benefit beyond making sure passwords are harder to guess: it helps ensure that users don’t use the same password for multiple unrelated sites.

There are MediaWiki extensions (like SafeCreate) which enforce password rules, but I didn’t really like the way any of them work (and my PHP skills aren’t very strong). So, instead I came up with a way to call an arbitrary, external program to test potential new passwords. Here is the code (to be added to LocalSettings.php):

# Enforce strong passwords when password is changed:
$wgHooks['PrefsPasswordAudit'][] = 'ChkStrongPassword';
function ChkStrongPassword($user, $newPass, $error) {
   $output = array();

   if($error !== 'success') { return true; }
   putenv("CHKPASS=$newPass");
   exec("/home/mywikiuser/bin/chkpass", $output, $rtn);
   putenv("CHKPASS");
   if($rtn) {
      throw new PasswordError(implode($output));
   }

   return true;
}

We are calling the program /home/mywikiuser/bin/chkpass to check passwords. This program expects the environment variable CHKPASS to be set to the potential new password. (We pass it in via the environment rather than the command line because other interactive users on the server may be able to see the command-line options of running jobs. I used an environment variable rather than stdin because it appeared easier to do in PHP.)

If the password meets our requirements, the chkpass program should return success (exit status 0) and emit no output. If the password fails to meet requirements, chkpass should return failure (exit status non-0) and emit to stdout a human-readable text description of what is wrong.

Here is a link to my sample implementation: chkpass (1397B Perl script)

Requiring HTTPS

Even if users have to authenticate using strong passwords that they’ve committed to memory, that accomplishes little if those passwords are flying over the ‘net in cleartext — or if the content protected by those passwords is sent in the clear.

Fortunately, there is a relatively simple solution: make sure we use HTTPS for every transaction. If you’re using Apache 2.x, this can be accomplished  by putting something like the following in the .htaccess file for your Wiki:

# If HTTPS, require a strong cipher
SSLOptions           +StrictRequire
SSLRequire           %{SSL_CIPHER_USEKEYSIZE} >= 128

# Insist on HTTPS always
RewriteEngine        on
RewriteCond          %{HTTPS} !=on
RewriteRule          .* - [F]

With the above in place, any access attempt using plain old HTTP will be met with a “403 Forbidden” error. Note that you’ll need an SSL certificate to use HTTPS, either self-signed or certified by a CA. If the former, users will need to manually establish the chain of trust for your cert the first time they access your site.

One bad change you may have noticed is that a handful of the posts here are now password-protected. This isn’t something I’m especially happy about, since these posts are about things I think are cool, and which I want to share with everyone. But someone smart (and with a significant stake in the matter) made the case that these posts also leaked information which could put me and others at risk.

I’ve tried to adopt the most measured response that still fixes the problem: hide only the posts I have to, and put those behind a wall rather than destroy them entirely. While I hope this is the last time I’ll have to do this retroactively, we don’t always get what we want or expect.

If you see a password-protected article, and have reason to believe I know and trust you, then you can always send email and ask for the password. Unless that trust is already established, though, you’re wasting your time — if it were something I could show to just anybody, I’d already be doing that.

Unfortunately, people mostly either know this already, or won’t be convinced of it until they learn from the school of bitter experience. So, this isn’t a post to try to convince you to take backups; it’s a post about how I do it, presented in the hopes that it’ll make doing so easier and safer. (As with many of my computer-related posts, the actual implementation is somewhat specific to UNIX-like systems, though many of the general principles apply universally.)

Goals

There are a few different requirements that a backup solution needs to meet — for me, anyway; your requirements may differ. These include:

• reliability — Write-only backups are no fun. Part of reliability is testability. You can’t count a backup solution as reliable unless you can and do periodically check (by actually retrieving files) that it works. If you didn’t personally get real data out of the backup, it didn’t happen. Reliability claims of hardware or software vendors aren’t enough. Success status reported by the backup software is not enough. (Actual example: expensive enterprise-level backup software saying “complete success” day after day, when all it was configured to back up was one empty directory.)
• redundancy — While having a reliable backup system means that there’s a good chance that you’ll be able to recover what you need from a single backup disk or tape, redundancy means you have more than one such thing, and that they’re kept in different places. If all your backups are in the same building, you may be out of luck if that building burns or floods or is raided by thieves. Even backups in different places within the same city or region can all be lost if there is a widespread disaster like a hurricane.
• capacity — The backup target should be big enough to hold all my important files. For that matter, it should be big enough to hold my unimportant files too, since I don’t want to have to pick and choose. Now, there are certainly files you can re-generate or re-install from some other source. But if you’re making decisions about what to omit (to save space), it’ll inevitably bite you. Example: Why keep all those .mp3 or .flac files? They’re huge, and you can always re-rip your CD collection. That’s fine, unless the reason you’re restoring is the house fire that also destroyed your CDs.
• portability — Backups are worthless if the rare machine you need to read the media or the special software you need to restore the data were destroyed. Plan around the idea that you’re in a small town with the backup medium in your hand, a limited budget and a need to restore your data in the next few days — maybe on a public-access or borrowed computer.
• security — If your backup media is stolen or seized, the perpetrator should be prevented — by strong, peer-reviewed encryption – from gaining access to your files.
• ease of use — If making backups is a hassle, it’ll be skipped. If it isn’t automatic, it’ll be forgotten. It’s also important that restoring files be reasonably easy. On the one hand, you shouldn’t delete or overwrite files you want to keep, and you should always think before typing the command. On the other, mistakes are inevitable, and time you don’t have to spend recovering from an error is time you can spend on something productive.
• ability to keep multiple generations — Sometimes a file gets overwritten or corrupted, and stays that way for a while before you notice anything wrong. In such cases, the most recent backup will just be a faithful copy of the bad data. It’s useful to have the ability to keep multiple generations of backups, spread out over time, and to restore from one of the older ones if you like.

Physical Medium

I’ve tried a number of different systems over the years, starting with “save to two different floppy disks”, then QIC tapes, various helical-scan systems, and optical media (CD-ROM and DVD-ROM). I’ve found tapes to be prohibitively expensive, painfully slow and, unless you are absolutely religious about head cleaning and replacing media on schedule, prone to failure. Optical media have limited capacity, and the most reliable such media are write-once.

My current solution involves using plain old ATA (or SATA) hard disks in USB drive enclosures. These are fast, capacious enough to hold multiple snapshots of my entire system, hot-pluggable and more reliable than all but the very best tapes (the drives for which I cannot afford). They’re self-contained, so I can take them (or mail them) to off-site storage. They’re cheap and use commodity hardware, so I can have a bunch of backup devices, and buy more if I think I need them.

Software

I keep one such drive enclosure connected to my homebox, and use a cron script to run nightly backups. The backup device has a single partition formatted as an encrypted  journaling filesystem (ext3 or reiserfs) — more details about this in a later section. The backup software is a simple Bourne-shell script I wrote:

Bourne shell script (3KB)

Note that there are configuration options within the body of the script which you must edit; do not just run this without first adapting it to your site.

There are substantial comments within the script itself explaining what it does in detail. In essence, it uses rsync(1) to make a snapshot of a filesystem. It uses the --link-dest= option to opportunistically make hard links for files which have not changed since the previous backup generation.

For example, say you have a 4GB movie file “cat.avi”. You make a first backup, and it puts a copy of cat.avi on the backup device. You make a second backup, and cat.avi hasn’t changed (in terms of contents, name or directory). The second backup will hard-link to cat.avi in the first backup. Now each backup contains cat.avi (with a link count of 2), and it only takes up 4GB of space on the backup (not 8GB).

If you edit the movie (say, by using a compositing program to add captions), and make a third backup on the same device, rsync will notice the change and make a fresh copy (without disturbing the previous two backups).

This combines the advantages of full backups (you need only look in one place to get the latest data, or any specific generation of data) and incremental backups (speed, efficient use of storage space on backup medium).

Because the backup medium is just a plain old mounted filesystem, I can navigate it and use the contents using all the same tools I use for my regular files. I can plug the enclosure into any modern Linux box with a USB port, mount the filesystem, and read my files without having to install anything special. I can do a full restore onto commodity hardware using any one of the various bootable rescue disks. Because not very many files change each night, backups after the first one are extremely fast, and a single drive can hold dozens of generations.

Encryption

My backup media are small, portable, and a tempting target for theft or seizure. They are also — because of the requirement for geographic redundancy — kept in various places where the physical security is maybe not the best.

As a result, I don’t want mere possession of one of my backup disks to be sufficient to allow someone to read my files. I also don’t want to depend on a key or token that might be destroyed or stolen in the same disaster that caused the need to restore from backup in the first place. So, I depend on encryption (using a pass-phrase I know, which is recorded nowhere) to protect the backups.

I don’t want a filesystem-level encryption solution, since there are (surprisingly common) cases where metadata like the names, sizes and timestamps of files and directories are as privacy-sensitive as the contents of the files; solutions which protect only the file contents are insufficient.

I also consider proprietary closed-source “encryption” products to be a contradiction in terms.  You have to take the word of the vendor that the product works, and that they haven’t included back doors at the behest of some criminal enterprise (governmental or otherwise). This is one of the reasons I disdain “cloud” backup schemes: they generally ask you to trust some black-box alleged crypto, sight-unseen. (If you must use such a service, mitigate the risk by storing only big undifferentiated blobs of data you’ve encrypted yourself, using real open-source peer-reviewed crypto.)

Finally, I want something where the decryption software is ubiquitous, thoroughly tested, actively developed, and easy to come by if I have to do a “bare metal” restore.

Fortunately LUKS (Linux Unified Key Setup) is an excellent solution to all of the above concerns. Plenty of detail is available on the project homepage. A short description of the moving parts:

• block device driver — abstracts storage hardware (like a USB hard disk) as a standard UNIX block device with a uniform API
• device mapper — a generic framework for mapping one block device onto another
• dm-crypt — a device mapper target that provides encryption
• LUKS — provides standardized key management

When you plug in your drive enclosure, you get a block device file like /dev/sdb1 for the first (and perhaps only) partition on the disk. That partition (representing what’s on the physical disk) is full of encrypted hash, statistically indistinguishable from purely random numbers.

Using the device mapper, dm-crypt target and cryptographic functions built in to the kernel (probably via the cryptsetup(8) command), you get another (virtual) block device file like /dev/mapper/backup, which acts as a cleartext version of your physical device. That is to say, when you read from it, a block of encrypted data is read from the physical disk, decrypted, and presented to you. When you write, your cleartext data is encrypted then written to the physical device.

This is handy, because you can use the device mapper virtual block device like any other block device. In particular, you can make a filesystem on it, mount it, and then it is just another directory.

Details depend on your distribution, but the essential steps are running cryptsetup luksFormat to set up the key block, encryption scheme and passphrase, then cryptsetup luksOpen to use your passphrase to unlock a key and create a mapping. A tutorial for Ubuntu 8.04 is a representative example.

Some distros (like Ubuntu 9.04) have very nice integration of this process — all you have to do is plug in a device with an encrypted partition; you’ll be prompted for the passphrase, and your filesystem will be mounted automatically.

Data Protection

Encryption is an essential component of data protection, but isn’t a complete solution by itself. A complete discussion is beyond the scope of this post, but some things to think about:

If your threat matrix includes scenarios where the attacker has you as well as your backup media, consider using a duress password in combination with decoy content. This will let you appear to comply with an attacker who threatens you with harm unless you disclose your data, while simultaneously making the data unrecoverable forever (even if you change your mind or the ruse is discovered). The beerbottle project is an example implementation.

Data is only protected if every copy is protected. While backups deserve special consideration because they are have to be in places where physical security is less well-controlled, consider also encrypting your primary filesystem. (This is especially important for laptops, which get travel to all manner of sketchy places.) An unencrypted swap partition may be full of things you’d rather keep to yourself.

Attackers routinely steal running machines, both host and UPS, dragging the lot back to where memory contents can be dumped and the contents of mounted filesystems examined. A simple limit switch can cut power if the case is lifted; easy to defeat, but effective if the attacker doesn’t expect it. If you’re the suspicious sort, there are trembler switches, accelerometers, magnetometers, and really any kind of electronic widget that can detect a condition which is reasonably likely to only arise if your box is being abducted. Creativity counts for a lot; an off-the-shelf solution will have an off-the-shelf countermeasure.

Controlling Writes

I keep my backup device mounted most of the time. It’s nice, because users have direct access (subject to the same file permissions as the original) to all the recent generations of backup. The one disadvantage, and it’s a big one, is that if a user can write to a file he can destroy the backups of that file. Since part of the purpose of backups is to protect users from their own errors, this is a serious problem.

My solution is to mount the backup filesystem on a mount point that only root can reach, and use that to perform the backups. I have a read-only bind mount accessible to users. A user can read a file if the mode bits allow it, but can’t write a file no matter what the permissions are. Example commands:

mount -t auto /dev/mapper/backup /private-mnt/backup
mount --bind /private-mnt/backup /mnt
mount -o remount,ro /mnt

In the above example, the directory /private-mnt/ is owned by root and has mode 0700 (read, write, execute only by owner). Non-root users cannot do anything beneath /private-mnt/ (including traverse the backup mount point therein). Anyone can traverse directories, read files and even run programs under /mnt/ (subject to individual file permissions), but can modify nothing (due to the read-only status of the filesystem).

Scheduling

Running the backup script can be done through a simple nightly cron job. Mounting the device involves a necessary manual step: entering the pass phrase, which can obviously not be kept in persistent storage. Rotating backup devices between home and remote locations also requires human effort.

I’m still finding the right compromise. Right now, I run (automated) backups nightly, and rotate the devices every few weeks. Each rotation requires remounting and typing the passphrase. If I have to reboot for some reason (like a lengthy power outage or a kernel update) that also requires a remount — but such events are few and far between.

Testing

If you’re using cron, it’s relatively easy to set things up so you get mail if the backup script fails for whatever reason. Actually reading the files is simple enough — just go into the mounted backup filesystem and do whatever you need to do (either as part of a random audit, or because you made a mess and actually need to).

If you want a more extensive test, it isn’t difficult to do bitwise comparisons of the backups filesystem (or portions thereof) versus your running system. Just bear in mind that not all differences are errors; some files are supposed to change, after all.

Problems and Limitations

The chief problem I’ve encountered so far is inelegant handling of hard links in the source data. If foo and bar are hard links which refer to the same underlying data, a given generation of my backup will contain two copies of that data. It may be that I can solve this by being smarter about what options I pass to rsync. In some cases, using symlinks instead might be a workaround.

Another potential hassle is automating the process of selectively throwing away older generations of backups. (Actually getting rid of a generation is not a problem — you just remove the directory tree.) Right now, I deal with this by re-formatting the drives at a certain point in the rotation. It wouldn’t be a bad idea to write a script that would do something like keep all backups less than a month old, then single generations at two, three, six and twelve months. I just haven’t done this yet.

While there is a facility for excluding certain data on the running filesystem from the backup (using the rsync –filter=dir-merge option), it is not exactly straightforward to use. (However, it’s dead easy to check the next day’s backups to see if you got the set of files you expected.)

This backup scheme is resilient in the face of single bits of backup media failing (in addition to whatever failure caused you to need to restore from backup in the first place). Beware of monoculture vulnerabilities in your hardware, though. I found a bargain on a loss-leader drive plus enclosure combo at a local shop, and bought several. All had power supplies which failed after a relatively short period of use. Buy enclosures from different vendors, and make sure the drives inside are different makes as well. (There are lots of stories about RAID arrays where all the drives came from the same lot, and all failed within hours of one another. Heed such stories.)

Some USB drive enclosures play silly mapping games, using a portion of your drive for their own nefarious purposes. Avoid these, as it is possible that if you transfer the drive to another enclosure, you’ll be unable to see your partition table without vigorous hackery. In fact, I’d suggest trying the drive-swap game right away when you get a new enclosure. Enclosures purchased alone seem less likely to exhibit annoying incompatibilities than those sold bundled with a drive.

This backup scheme is extremely simpleminded, in that it works by just copying all the files one at a time, whenever it is run. This works great for most things, but can be a problem for things that are in the midst of being updated when that copy is made. Relational databases are especially troublesome in this regard. You can work around this by programmatically dumping the database contents to a flat file just prior to running the backup. (Both MySQL and PostgreSQL offer means of doing this without shutting down the database.)

USB drive enclosures, with a few exceptions, do not support the commands needed to access reliability information available on the underlying drive hardware (SMART and the like). While tools like smartctl only give you advance notice of a fraction of drive failures, some warning is better than none. Hotplug ATA or SCSI gets around this problem, but adds others (chiefly expense and difficulty finding compatible hardware on short notice).

This backup scheme makes it much less likely you’ll lose data by accident, but complicates life if for some reason you need to get rid of all the copies of a given file.

Conclusion

I doubt that anyone else will want to use exactly the backup scheme I use, but I hope that some of the ideas here will help some of my readers. I also hope that you’ll point out my inevitable mistakes, and make suggestions if you have ideas for improvements.

Back up your data. It doesn’t matter how, so long as it’s complete, current and actually tested. More of your life than you think is on your computer. Think about someone being interviewed after a house fire — if all the people and pets are safe, their big worry is usually things like family photo albums. You have files on your computer that are analogous to — and in some cases, literally are — those photo albums. And they depend on a complicated, delicate machine that was mass-produced as cheaply as possible.

Protecting your files requires conscious action on your part. It isn’t hard, but if you don’t take it, you’re living your digital life on borrowed time.

If you are not particularly pleased with the idea of a company like Microsoft making money from your creative work, then my strong suggestion is to create a robots.txt file in the root of your web-space, with contents not unlike:

User-agent: msnbot
Disallow: /

User-agent: *
Disllaow:

The robots.txt is a voluntary standard which allows web page authors to exclude search engines from part or all of their sites. There’s a helpful website that has details about why you might want to do this, and how to go about it.

Of course, there have been some allegations that Bing isn’t honoring the robots.txt standard. But announcing that they’re unwelcome is a fine symbolic act, even should they fail to honor your wishes.