Month: February 2009

Charles Stross on Data Preservation

Charles Stross writes about a problem near and dear to my heart — data preservation.

Stross starts off by mentioning this Observer article by British Library head Lynne Brindley, but Brindley annoys with material like this,

The 2000 Sydney Olympics was the first truly online games with more 150 websites, but these sites disappeared overnight at the end of the games and the only record is held by the National Library of Australia.

. . .

People often assume that commercial organisations such as Google are collecting and archiving this kind of material – they are not. The task of capturing our online intellectual heritage and preserving it for the long term falls, quite rightly, to the same libraries and archives that have over centuries systematically collected books, periodicals, newspapers and recordings and which remain available in perpetuity, thanks to these institutions.

Not once does she mention Archive.org, the nonprofit driven effort to archive the web which has been doing for years what librarians like Brindley are finally getting serious about doing. And, yes, Archive.org has a copy of the Sydney Olympic websites.

But, of course, replication is just part of the problem and probably the easiest to solve these days. The cost of building large scale data storage is getting cheaper all the time. Yes, there’s more data being produced, but the ability to store and use that data is scaling well.

Ensuring the data will still be useful 10 years from now is another matter entirely. Yes, the web is mostly HTML and JPEGs, but there is plenty of nonstandard data from Word and PowerPoint files to Javascript and Flash files created using very different versions of software and often requiring specific versions of programs to all work properly together.

Stross looks at that problem from a personal level, noting the times he’s switched from one platform or software to another and the inherent difficulty in taking your data with you during that transition,

In the space of six years, I went through five word processing packages. Being naive at the time I didn’t export my files into ASCII when I moved from CP/M and LocoScript to MS-DOS. I learned better, and when I switched from Sprint to Word I halfway ASCII-fied those files; they’re a bit weird, but if I really wanted to I could get into them with Perl and mangle them into something editable. Along the way, I lost the 3″ floppies from the PCW. Then I had a hard disk die on me — in those days, the MTBF of hard drives was around 10,000 hours — and it took the only copy of most of the early work with it.

Score to 1993: two years’ work is 90% lost. And a subsequent five years’ work is accessible, kinda-sorta, if I want to strip out all the formatting codes and revert to raw ASCII.

Been there, though not quite to the extent as Stross — I made sure to have printouts of everything on the proprietary word processors I used back in the 1980s, though having PDF scans of everything is significantly less useful than just having plain ASCII files.

Having been burned by data loss, Stross describes how he tries to prevent that from happening again,

As a matter of personal policy, for those activities that involve creating data, I aim to use only software that is (a) cross-platform, (b) uses open or well-published file formats, and (c) ideally is free software.

This is in some ways a handicap; Thunderbird (my mail client of choice) and OpenOffice aren’t as colourful and feature-rich as, say, Apple’s Mail.app or Microsoft’s latest Word. However …

Firstly, they run on Macs, Linux systems, Windows PCs, and even on some other minority platforms. This protects my data from being held to ransom by an operating system vendor.

Secondly, they use open file formats. Thunderbird stores mailboxes internally in mbox format, with a secondary file to provide metadata. (This means I can claw back my email if I ever decide to abandon the platform.) OpenOffice uses OASIS, an ISO standard for word processing files (XML, style sheet, and other sub-files stored within a zip archive, if you need to go digging inside one). I can rip my raw and bleeding text right out of an OASIS file using command line tools if I need to. (Or simply tell OpenOffice to export it into RTF.)

Thirdly, they’re both open source projects and thus the developers have no incentive to lock me in so that they can charge me rent. I don’t mind paying for software; where an essential piece of free software has a tipjar on the developer’s website, I will on occasion use it. And I’m writing this screed on a Mac, running OS/X; itself a proprietary platform. But the software I use for my work is open — because these projects are technology driven rather than marketing driven, so they’ve got no motivation to lock me in and no reason to force me onto a compulsory (and expensive) upgrade treadmill.

I’ll make exceptions to this personal policy if no tool exists for the job that meets my criteria — but given a choice between a second-rate tool that doesn’t try to steal my data and blackmail me into paying rent and a first-rate tool that locks me in, I’ll take the honest one every time. And I’ll make a big exception to it for activities that don’t involve acts of creation on my part. I see no reason not to use proprietary games consoles, or ebook readers that display files in a non-extractable format (as opposed to DRM, which is just plain evil all of the time). But if I created a work I damn well own it, and I’ll go back to using a manual typewriter if necessary, rather than let a large corporation pry it from my possession and charge me rent for access to it.

All very good ideas. For text, I do everything in a text editor. If I need to pretty it up I’ll then import it into Open Office or something, but there’s something especially useful about plain old ASCII.

And for those situations where a closed-source, proprietary file format solution is the only possibility, make damn sure you have a relatively open alternative representation of your work. I use a modified CAD program, for example, and I have no idea whether or not the file format is closed or open, but I damn well make sure I have JPEG and PDF versions of every project I’ve done in case the company goes belly up and 10 years from now I don’t have anything to run the original files through.

How to Show You’re Not to Be Taken Seriously

I’m going to pick on Daniel Florien for a minute because I absolutely detest the basic form of his blog post, How to Stump Anti-Abortionists With One Question. Florien is hardly the only person to offer one of these “how to stump xyz with one questions” guides. I’ve seen how to stump pro-abortionists, atheists, Christians, Darwinists, creationists, etc. etc. in one question essays.

For the most part, the argument that is allegedly a show stopper is surprisingly lame. Consider, for example, what Florien considers to be a show stopping argument against anti-abortionists,

Just ask them this:

If abortion was illegal, what should be done with the women who have illegal abortions?

Now watch their faces as the cognitive dissonance sets in. They believe abortion to be murder. Murder deserves severe punishment. Thus, women who have illegal abortions should receive severe punishment — like life in prison or the death penalty. That’s the logical conclusion.

That’s just stupid on a number of levels.

First of all, it simply does not logically follow that if abortion is wrong that it should result in “severe punishment — like life in prison or the death penalty.” That’s not a logical conclusion, that’s a straw man.

In fact, prior to Roe v. Wade, when abortion was illegal in most states, women were rarely prosecuted for having illegal abortions. Rather if there was a prosecution, the state would generally go after the doctor performing the procedure. Similarly, abortion is banned throughout Ireland, and yet prosecutions of women who have illegal abortions are extremely rare there.

On the other hand, it is also certainly possible for anti-abortion states to prosecute women. Portugal, after all, does actively prosecute women who have illegal abortions as well as the doctors who perform them. Unfortunately for Florien, the penalties are from from his Draconian straw man — for example, a couple of women acquitted of having abortions in 2005 faced up to three years in jail if they had been convicted.

Apparently Florien needs to first convince both Ireland and Portugal that they simply cannot ban abortion without sentencing women to life in prison or the death penalty.

I’m sure there are, nonetheless, anti-abortionists who may be stumped by this lame argument, just as I assume there are jurors who may be convinced by the Chewbacca defense. All Florin’s post shows is that he takes the arguments for or against abortion about as seriously as those dimwits he seeks to stump. Personally, I wouldn’t be bragging about that.

Finally, Florien felt the need to update his blog post with the usual tripe from “pro-choice” types,

First, a clarification. I’m not pro-abortion. I support the legalization of abortion. I do not like abortion nor do I usually counsel it. However, I do think it should be available for women who want it, especially if they were raped or have zero interest in caring for a child.

I would like abortion to be safe, legal, and rare. We can all work together on the rare part.

God, I hate this line of thinking. Can you imagine this nonsense being deployed in any other medical contest? I’m not pro-viagra. I would like viagra to be safe, legal and rare. Why do “pro-choicers” always feel the need to apologize for their support of legalized abortion in this way?

I am pro-abortion. I want abortion to be safe and legal and occur in whatever frequency women freely choose.

Anyone adding the “and rare” clause needs to define a) just what sort of abortion rate would qualify as rare, and b) how they would go about reducing the number of abortions from the >1 million annually in the United States to whatever rate is acceptable as “rare.”

Scooby Doo Gang as Call of Cthulhu Characters

Someone has helpfully drawn up Call of Cthulhu character sheets for the Scooby Doo gang. I can just see the end of that session.

Velma: (ripping off sheet) No, the ghost was really Cthulhu in a sheet with an automated fog machine.

Cthulhu: And I’d have been able to rise from R’leyh if it hadn’t been for you meddling kids.

Apple Will Control the Horizontal. Apple Will Control the Vertical.

Apple iPhoneMashable has an interesting post about Apple rejecting a game for its iPhone App Store because the game involves throwing shoes at a caricatured image of former President George W. Bush. Mashable quotes the developer as describing the game this way,

1. You hold the IPhone like a shoe,
2. Press on Start button
3. System, starts saying 3, 2, 1 and Go
4. You, swing your arm ( as you are throwing ) without throwing.
5. Application will figure out how hard did you throw the shoe at him and it may or may miss the cartoon.
6. It shoes the total number of shoes that have been hit at the cartoon.
7. We started with Bush, but next in line were Bin-Laden and other figures as well ( apparent from the splash screen).

But, of course, Apple controls what applications users can run on the iPod Touch/iPhones they own, so it refuses to allow users to purchase this application. The AppStore’s TOS includes ridiculous language that,

Applications must not contain any obscene, pornographic, offensive or defamatory content or materials of any kind (text, graphics, images, photographs, etc.), or other content or materials that in Apple’s reasonable judgement may be found objectionable by iPhone or iPod touch users.

Well, of course — why would you ever let users decide what kind of applications they want to buy and run, when Uncle Steve and his cronies are more than capable of making that decision for you.

The Independent: Greatest Threat to Journalism Since Polio

The Independent has a profile of Eminem’s publicity for his upcoming new album which include this claim,

As his fame grew, Eminem became a magnet for controversy. He was accused of glorifying misogyny, homophobia, bad language and violence. He was arrested on gun charges. On one occasion, George Bush, upset by his lack of respect for the forces of conservatism, labelled him: “The greatest threat to America’s children since polio.”

How stupid is the reporter who wrote that? That has urban legend written all over it. And, per Snopes,

Given that:

  • None of these articles [from 2001 when the quote first appears] contains any details about when (or where, or under what circumstances) President Bush allegedly described Eminem as “the most dangerous threat to American children since polio.”
  • All of the articles containing this putative quote come from newspapers published in the UK.
  • We haven’t yet turned up even a single article from a U.S. newspaper which includes this quote (other than brief references to its having been mentioned in British newspapers), even though President Bush and Eminem are both Americans and major media figures.

We’re guessing that this was a spurious “quote” fabricated by someone for publicity purposes (nothing piques curiosity about a person more than the President’s declaring him to be a dangerous enemy), or to poke fun at President Bush. Since no one else has been able to verify the authenticity of this quote, we’re assigning it a “False” rating.

Many of us who live in the United States can remember the withering criticism directed at George H. Bush for his moronic statements about The Simpsons. If George W. Bush had really said something as stupid as that, it would have been major news in the United States.