Archive for the 'sysadmin' Category

Secure File Delete on Mac OS X

Under Linux I always used wipe within a terminal to securely delete my files, and I was about to purchase “ShredIt X” to essentially perform the same thing under Mac OS X when I came across a comment on a software download forum mentioning srm.

srm is a Unix command-line tool to securely delete a file. It is installed by default for us Mac OS X users too, just open up a Terminal and type the following:

srm my-file.doc

Quick, right? You did type srm, not just rm, right? srm is designed to operate just like rm so it takes the same command line switches or options, but rather than just deleting the file (or the link between what is essentially an index of files and the actual location of the data on the physical disk which in reality is exactly what a normal delete does (hence it’s lightning quick)) it looks at the data on the physical disk and overwrites it.

Now, the US Department of Defence specifies certain requirements about the overwriting process to ensure the chances of recovery are limited. Basically hard disks are like magnets and file data is stored as magnetic fingerprints (yes, this is highly simplified) on those disks. Consider a fridge door with magnetic letters spelling out a message. Same difference. However, remove those letters and although the information cannot be read by the naked eye specialist data recovery firms (and law enforcement agencies) have methods and tools to retrieve the fingerprints just like they were ghosts or shadows of the original data. Very clever, very scary.

Anyway, the DoD specification reads quite simple compared with a data overwriting algorithm specified by a guy called Guttman which basically entails random data repeatedly written over the data thirty six times. The DoD I think specifies six.

There is a disadvantage to using Guttman’s method: The process is a whole lot slower than the DoD’s method and obviously orders of magnitude slower than a standard delete. Swings and roundabouts, as they say.

I prefer the more secure route. And srm uses the Guttman method by default. So that .doc file above really did go, unless you forgot the s.

Now, if you think you have already removed via Trash or rm files that you wish you’d securely deleted, there is a get out clause. You can ask Disk Utility to securely delete your free disk space. When a file is removed the normal (non-secure) way, the physical space on disk although not overwritten is now available for other files to be written to. Otherwise when you deleted a 1 gigabyte file you’re disk free space wouldn’t go up by the same amount! However, once it has been overwritten (in whole or in part) by future files like videos, emails, documents or resources used by your operating system, the physical area on the disk may or may not right now be in use. Areas not in use will be securely erased, areas in use obviously not.

Thank you Apple and open source software developers.


Linux Backups

The past several years has seen a big rise in Linux on the server. It runs email, databases and web sites. All of whom happen to be (in the vast majority of cases) incredibly important applications for businesses and individuals alike.

Yet the documentation often sucks, and there is usually a lack of information on how to back up your data and restore it.

What I would like to see is every software package installed on Linux like these having a notice somewhere on backups. A reference to a web site or manual page would be a minimum. For MySQL there is a section of their documentation web site. For apache it usually involves backing up a set of files. But by default nothing really exists which really doesn’t make sense.

How about a set of wizards. Break down most installations into a set of standard questions, and out comes a short set of instructions for creating a backup and similarly short but sweet set of instructions for restoration.

Sure some of this is highly dependant on the site, but variations always apply.

Cyrus IMAP Backups

So I spent part of last week working on Cyrus IMAP backups. The intention was simple: If we’re rsyncing the files to a backup store it should be possible to prove that they work.

Not necessarily.

The idea was to take our mailspool backup from a Cyrus 2.1 install and import into 2.2. The Ubuntu upgrade guide told me to upgrade the databases. What databases? I inherited this thing without documentation. All I could see were email files in a tree like directory structure. Anyway, apparently I should follow the 1.5 to 2.x upgrade instructions. I ran a series of commands from the guide knowing I could re-restore if things went badly. Which they did, having no idea what I was doing I decided to re-restore.

However, a more detailed look into what was wrong was needed. And I discovered that some files were missing. Yes apparently another directory file of configuration and status information was on the production server. It made sense, there had to be permissions saved somewhere.

So I added this config /var/lib/cyrus/ directory to the rsync script and restored again. This time ensuring that after re-installing that Cyrus 2.1 (given up trying 2.2) packages I restored without rsync’s -u flag which doesn’t overwrite newer files. Kind important for restoring old consistent files.

Restoring the /var/spool/cyrus and /var/lib/cyrus directories took almost four hours. Then came the /etc/cyrus.conf and /etc/imapd.conf and not forgetting the /etc/sasldb2 file. Oh and because this machine had a different hostname to the production server I needed to add root@newserver and myaccount@newserver to the SASL database: saslpasswd2 root@newserver not difficult to do.

Finally it was up and running, and I connected from another LAN desktop. I could re-subscribe to my folders and see mail I had already read. Fantastic.

Only one problem to “prove” how to do now: backup and (more importantly) partial mailbox restoration. Oh no! Our sales mailbox got wiped. Do you have a backup? Yes from 5am. Well that will have to do. But I don’t know how to restore that mailbox without restoring the others too.

Guess: Restore the email directory structures and run within cyradm ‘reconstruct mailboxname’. Fingers crossed for when I try this later.

Summary of what to do (full backup and full restoration):

rsync -zarle ssh /var/spool/cyrus/*
rsync -zarle ssh /var/lib/cyrus/*
rsync -zarle ssh /etc/*

To restore: The opposite of the above. Literally. You might want to selectively restore imapd.conf and cyrus.conf for obvious reasons.