I am more than a bit jealous of Brester Kahle’s Archive.Org and for a long time have wanted a tool that would let me create something along the lines of a personal Internet archive. Since storage is dirt cheap, as I’ve ranted and raved about, why not just automatically save ever web page I ever visit?
There are some commercial programs that come close, but not close enough and besides they all tend to be IE only.
Last night, however, I decided to see if there might be any extensions for Mozilla Firefox that allow something like this, and lo and behold I ran across Ken Schutte’s excellent Slogger extension.
This is exactly what I wanted. I have configured Slogger so that every time I visit a page, it automatically saves that entire page. It also appends the page to a log list that shows the page name, the time I visited it, the original URL, and a link to a local copy. It uses a naming convention that uses the date down to the milliseconds for file names to avoid duplicate file name problems, etc.
You don’t have to configure it this way. Slogger can be set up just to keep a detailed history, or it could be configured to save pages at the press of a button rather than automatically.
There are only a couple drawbacks that I noticed in the current version of the software.
First, if you are using tabbed browsing (and if you’re using Firefox you’d be crazy not to), Slogger can only save the page in the active tab. So if you have Firefox set up to load new tabs in the background, Slogger can’t automatically capture those background tabs (in fact, when you load a new background tab it will simply make another copy of the page in the active tab). I just configured Firefox to load new tabs in the foreground — a bit of a change, but nothing I can’t adapt to.
Second, although storage is certainly cheap, making copies of every single web page visited can still chew through hard drive space very quickly. Today has been a very light web surfing day for me, but Slogger has added about 105mb worth of files. I’d imagine on a typical day I’d be looking at 300-400mb of files. That ends up at about about 146 gigabytes/year. Since I only buy external drives these days, that’s an annual storage cost of about $154 at today’s hard drive prices. That’s still less than $.50/day, as far as I’m concerned it’s still incredibly cheap, but, as always, your milage may vary.