Back in March, the Long Now Foundation blog featured an extremely long post republishing two articles and a paper concerned with the potential loss of data caused by the increasing speed at which storage technologies become obsolete and, soon thereafter, difficult to access.
Of the three pieces, Jennifer Stilles’s look at the National Archives’ efforts to preserve/recover data stored in obsolete formats was the most interesting. It seems clear from Stilles piece that the crux of the problem is the constant drive for technological innovation which produces products that are ever better but also, too often, ever more incompatible with previous formats. Moreover, this is a problem that started long before the current digital computer age,
On the wall are the internal organs of a film projector from the 1930s; the old heads have been mounted to play together with modern reels. “Twenty-eight different kinds of movie sound-tracking systems were devised during the 1930s and 1940s, trying to improve the quality of sound tracks,” Mayn explained. “Most of them are unique and incompatible.” This particular one used something called “push-pull” technology, in which the sound signal was split onto two different tracks. The technology was meant to cancel out noise distortion, but the two tracks must play in near-perfect synchrony. “If it is played back properly, it is better than a standard optical track, but if it is played back even a little bit improperly, it is far, far worse,” Mayn said. In the mid-1980s at a theater in downtown Washington, he was able to actually use this reconfigured projector to show several reels of push-pull film containing the trials of top Nazi leaders at Nuremberg. And the lab has transferred some 1800 reels of push-pull tape onto new negatives.
Wow. That fits nicely with one of the main problems with data storage today once you get past the physical media — the plethora of file formats and an odd lack of recognition that this is even a problem.
Microsoft rolls out yet another proprietary format for Office? Everybody simply upgrades without a second thought, because if you don’t all of a sudden you’re receiving file attachments you can’t open. Much of this is driven, I suspect, by the view that most data production is largely ephermal. Are we really going to be want to be able to open this report in Word 2003 format 10 years from now? Of course, I’ve also seen the fallout from that where people run around trying to find some way to open that 10 year old file which is suddenly extremely important due to issues with a specific vendor or contract, etc.
The current state of data preservation efforts remind me of the documentary “The Chances of the World Changing.” The documentary follows turtle enthusiasts who, given the lack of any coordinated effort to preserve endangered turtles, create their own ad hoc network of mini-Arks. They buy up individual turtles from overseas, and store them in warehouses, basements, garages, etc., moving the turtles around when one or another enthuisast burns out or runs out of cash. And they hope they’ll be able to keep the turtles going and around until they’re able to get others to see the need for a permanent, formal preservation effort.