Conversant and a Common Problem with Dynamic Sites

Tim Perdue has written a short article on overcoming a common problem with dynamic web serving systems — in his case PHPBuilder. The problem is that dynamic, database driven sites tend to end up with a lot of stuff in the URL (usually question marks and ampersands) that prevent search engines from indexing them properly. Most search engines hit an ampersand or question mark in a URL and just move on without indexing. For most such content systems you have to find a workaround like Perdue has with PHPBuilder.

One of the things I like about Conversant is that I never run into this problem. The only area where I ever see question marks and ampersands is when running search queries, and it’s even relatively easy to set up a specific search query that could be indexed. For example, http://brian.carnell.com/recent_messages is a hard coded search query that returns every new message posted within the past two weeks and sorts them in reverse chronological order (most recent message first). And it does it with no ampersands or question marks and took about 10 minutes to configure, with most of that being my tampering with the HTML template for the returned results.

Just another example of how thoroughly the good folks at Macrobyte really have wrapped their heads around the issues and problems confronting people who want to manage complicated web sites but don’t have piles of dot.com cash just lying around to blow on excessively expensive content management systems or a technical support person to get something like PHPBuilder or Zope or whatever running and updated.

Conversant Newbies Site

Mark Morgan started a Conversant Newbies site the other day which he summarizes as, “A place to put really cool stuff we come up with.”

One of the sort of double-edged swords with Conversant is its incredible flexibility. On the one hand this is astounding because to an extent not possible in any other tools I’ve used, I can build a web solution tailored to how I work and how I want to present information, while Mark or others can do things entirely differently.

On the other hand regardless of how easy it is to use, at some point once you get past a certain level of features, things can get pretty confusing (on the flip side of that, though, given its flexibility it is far easier to keep all of the options of Conversant in my head than it was remember how to configure four or five CGI scripts to do similar functions).

One of the things I’ve been working on, for example, is writing documentation for reguler site visitors who are used to using things like the Ultimate Bulletin Board. In my opinion, writing good software documentation is the most difficult writing task, period.

Leave to Work Out, and Everything Changes

All I did was spend a hour in the gym and when I come back I learn the Israelis decided to escalate the Middle Eastern conflict into full-scale war, and then Greg Pierce, inspired by Mark Morgan’s flat discussion hack, went ahead and demoed a version that he’s working on that uses some new Conversant code — and it looks great.

Give Me More Options

Okay, an email arrives in my mailbox a few minutes ago from Mark Morgan (one of the excellent things about Conversant is you can subscribe to these sites via e-mail) pointing out that Dave Winer’s Manila CMS has added a flat discussion view to its toolset.

This is one of the few things on the short list of things that I’d like to do on my sites but that Conversant can’t do yet … sort of. Morgan himself hacked a pretty good implementation of a flat view within Conversant that I plan on implementing soon, and Seth Dillingham did say that a flat discussion view is on the list of things to add to Conversant, though it was on schedule for a Q1 2001 date since there are other features that are being worked on.

One of the things I keep congratulating Seth on is the fact that when Conversant implements a feature, generally it has a pretty open structure so I can implement a feature the way I want. Personally I’m amazed that this isn’t the default attitude of software developers, but in discussing Manila’s implementation of the flat discussion group view, Winer illustrates the typical attitude of developers toward users.

Somebody asked Winer how he would handle letting people break long threads into multiple pages, which is a pretty standard feature in flat threaded discussion group software. Winer’s response,

First, there’s no limit to the size of a browser page.

Second, when the page gets “full”, start a new topic.

Yuck. First, although there is no limit to the size of the browser page, some of us have difficulty keeping thing straight when the page gets huge. Also, I don’t know about Winer but I have had people write me from remote locations — one person was accessing my site from a remote part of Alaska — where getting reliable 28.8k connections is iffy and huge pages would pretty much destroy the ability for them to participate in a meaningful way.

As for starting new topics when a page gets “full”, personally I’ve visited sites that did this, and I really hate that approach. Lots of people swear by it, but I think it’s self-defeating to have say 12 different thread pages going on about the same topic.

But beyond whether Winer or I are right about the “best” way to format a flat discussion group thread (and this is all a matter of personal preference), why not just write your software so that you support whatever option the administrator and/or user is most comfortable with? Since the page being served in this case is dynamic anyway, why not give administrators, and preferably users, control over how many messages they are served? It seems like such a small thing, why impose the “one true interface” on users?

The other thing I dislike about traditional flat discussion groups, which Winer’s uses, is they tend to order the messages in strict chronological order. This is helpful the first time you read the thread, but is a real pain in the butt, in my opinion, when you’re revisiting a thread. I much prefer ordering the messages in reverse chronological order so I always see the newest messages first. Others vehemently argue for the opposite. Again, let administrators, and preferably users, make that choice.

Mark refers to leaving out these sort of options as “only partway doing things.” Sometimes developers initially only partially implement a feature with the intent of adding on and fully implementing other features later, which is fine, but the goal to my mind should always be to make features as flexible as possible, especially in something like a CMS system.

The Beauty of Labels

Conversant uses something called labels which are essentially configurable categories for stories. You may have noticed that some stories on this site have images to the right while others have broken image icons.

You can click on those to find related stories on this web site. If you click on the CD next to the story about the MP3.Com bill, for example, you’ll get a list of all the stories I’ve written about music over the past few months.

The broken images are there because I’ve been sick and haven’t had time to find good images for all of the labels I’ve set up. I should be able to take care of that this weekend.

This is another thing I really like about Conversant — all of my sites, and many of the sites I read on the Internet, are basically stuff I find interesting. As you may have noticed, I find a lot of things interesting. Unfortunately when you write 5,000 words a day or so as I’ve been doing recently, it can get difficult to keep track of everything. Labels are just one more feature that makes it easy for me to track down information I need long after I’ve forgotten that I even wrote it.

Conversant: It’s Like Having a Paid Assistant

I have always been a compulsive writer. I have been writing 10 to 20 pages a day for so long that it is second nature. Maintaining a web site, however, is definitely not second nature.

The real pain for me was always managing a site’s structure. On a static site this can be a nightmare. Suppose, for example, I write an article about Fidel Castro criticizing Bill Clinton over U.S. free trade policies. Where do I put the file that contains that story? In a directory with other stories about Castro? Clinton? Free trade? And then I have to remember where I put it six months from now. An obvious solution would have been to just create an /articles/ directory and sequentially number all articles, but again this is difficult to manage when I might write 200 stories for a site in a year. Next year there’s going to be no way I will remember that the story on Castro and U.S. free trade policy is articles/2000/000035.html. That will never happen.

As I mentioned the other day, with Conversant this becomes effortless since I can link to that story automatically just by typing in the headline and surrounding it with pipes (the “|” character) and forget about the underlying structure.

Similarly tomorrow morning when I wake up and post a new article for this site, I do not have to worry about cutting and pasting the old articles to an archive and then re-uploading half a dozen files — the software takes care of archiving and keeping track of everything automatically.

The upshot: more time for writing. I conservatively estimate that when I was using Dreamweaver, for every hour I spent on writing, I had to spend about 15 to 20 minutes in web site maintenance, which was a serious incentive not to write. Every new article I wrote meant a new headache in updating the site. With Conversant, there was a relatively time consuming initial startup period in converting a couple thousand pages to Conversant which took about 30 to 40 hours overall. On a daily basis, however, I estimate that for every hour I spend writing I only have to spend 3 to 5 minutes maintaining the site. In addition I can concentrate that in an hour or two at the end of the week. It is very nice to spend an hour on Sunday night taking care of the administrative tasks rather than 90 minutes or more each day.

On Monday, for example, I wrote almost 7,000 words which is a lot even for me (that is about 30 typed pages). There is simply no way I could have possibly done that and got all of those articles up on my static web site in only a 16 hour day. With Conversant, I finished everything with plenty of time to go home and catch up on my taped episodes of “The Power Puff Girls.”

I an pretty much a one person operation — my wife does some editing for me occasionally — but Conversant makes it feel like I’ve got a paid assistant who I can hand off all of the routine, boring stuff to while I work on what I enjoy, updating the content. It is nice not to be up at 10:30 p.m. trying to upload changed pages to a static server.