Jorn Barger, whose Robot Wisdom web log I visit every day, recently had an interesting essay on an idea that seems to be gaining speed; namely that the huge processor speed + huge hard drives + broadband = a revolution in how people store and access information.
Managing all that information, however, starts to be a problem. In fact, managing the relatively small amount of information I post on the web is often a big problem. I estimate I post an average of 5 articles a day overall, with many different themes and topics. Figuring out how to organize them is an enormous headache, much less actually finding the time to do it.
Barger apparently has the same problem. Barger writes,
Webloggers have a headstart on the challenge of building their own, because they’ve started archiving the best links, with annotations. They can go back thru these archives and sort the links by topic– my netlit portal was a first try at this, using the categories: fun, art, media, issues, net, tech, science, history, search, shop.
But I feel like that experiment was a near-complete failure– I hardly ever use them myself. Esthetically, they’re just too noisy. (I trimmed almost all the pullquotes when I sorted them, which may have been a mistake, and I started rewriting my blurbs, but it didn’t help that much, imho.)
A big problem that Barger hints is also the fact that often relations between topics are “fuzzy.” Sometimes I want to view information on Cuban executions as part of a Cuba topics page — other times I want to read about it as part of a death penalty topics page (not to mention as part of a Fidel Castro biography page). Once you get more than a few of these, the whole endeavor becomes a nightmare.
Conversant goes a long way to solving this problem with what it calls an Advanced Query Page type. The search page on this site is a simple Advanced Query Page, but there is a lot of power behind the hood. For example the new books page I added the other day looks like a typical web page, but it in fact is an Advanced Query Page and is dynamically generated.
In this case, all the page does is perform a search looking for anything I’ve categorized as being about “Books”, and then returns the web pages it finds in a neat ordered list. I could easily insert prefatory text explaining my love of books, etc., as well. And once I’ve got it set I don’t have to worry about updating the page — it will take care of itself.
Plus, I could easily do more refined pages. For example, if I had written a lot of articles about a specific computer game such as “Diablo II,” I could easily create a search that simply looked for that phrase where it occurs in web pages and return a list of them.
This makes it very easy to take any HTML document base and quickly create a very detailed, sophisticated directory of those documents. Powerful stuff.