I really like Silicon Fornesics’ hard drive transporter for 3.5″ hard drives, but I’ve got 10-12 hard drives stuck in a locking drawer, and the bulk from 10 or 12 of the hard drive transporters would be a bit much. Enter Silicon Forensics’ Hard Drive Shipping Case:
Holds 12 hard drives in a foam padded case suitable for shipping, if you wanted to — though I just want a nice storage solution for a bunch of loose drives.
This thing goes for $129.99 and weighs 8 lbs. I can’t wait to get one.
I knew someone who several years ago wrote a book … on his laptop … for a year … and never backed it up or retained a print copy. You can probably guess what happened next.
Almost as bad are these folks who relied on a cloud-based company to store backups of episodes of the children’s show they produced. One malicious employee later, and (per the Register),
CyberLynk had fired an employee called Michael Scott Jewson and, according to a Honolulu courthouse news report, one month after being given the boot, Jewson accessed CyberLynk servers and wiped out 304GB of data, including 14 Zodiac Island episodes, a full season of the show.
The Zodiac Island producers were based in Hawaii, and Cyberlynk in Wisconsin. A cloud-based service is probably a very good solution for a television production team to share assets among disperse groups all working on a television show, but as a primary backup as well? Seriously?
Especially considering the small size of the dataset involved. Local backup of 304gb would have been dirt cheap. Having a cloud-based backup for convenience or as an alternative in case of a local disaster is a good idea, but I can’t see ever giving up local backups entirely unless the dataset is too large to do so meaningfully (if they were dealing with 100s of terabytes, then maybe I’d understand why they weren’t doing local backups as well, but 304gb…puhleeze).
FreeFileSync is a free and open source tool for syncing directories and files. I use it primarily to mirror my main personal data drive — which clocks in at about 3 million files in 1.1 terabytes – to a local backup.
In the past, I’ve actually paid for commercial sync tools and this blows them all away. It tears through the compare and sync very quickly, and is extremely configurable if you want to go beyond simple mirroring.
I rely on this daily, and the best praise I can give it is that I just hit the Synchronize button and forget about it.
Paul Querna wrote an interesting post back in June about “forever storage” — data storage that could potentially be stable for civilization-spanning eras of time,
Not everyone will believe we can keep growing technology at the pace we have, nor that we might be able to stop death and diseases in our generation, but I do believe we are in the age where information created and stored today, could survive forever.
…
There are small technical challenges, like how would you write to media intended to last thousands of years, where would you store it all, and how would you pass on access to this data to whomever you desire, but I think they are all solvable.
If you can store your body in cryogenic storage for thousands of years, why can’t you store your data; Not just for yourself, but for your descendants.
One of the interesting questions here is just how well previous civilizations have done at information preservation. On the one hand, I can still read the full text of Sophocles’ Oedipus the King even though it was written about 2,400 years ago. On the other hand, even though he was one of the most famous and successful Athenian playwrights, only 7 of the 123 plays he wrote survives in complete form.
It would be interesting to estimate what percentage of written data generated by civilizations prior to the 15th century survived to be readable today. I’d be surprised if more than 5 percent of such material survived, and suspect something like 0.5% is excessively optimistic (if anyone knows of any published estimates of long-term information survival, please send me a link or reference).
Add to that we’re relying primarily on magnetic-based form factor-based hard drives for large scale storage which is technology that has been around for just 32 years now.
One possibility is to use something like the Rosetta disc or some of the physical, non-dye based optical solutions which should last a very long time, though at exorbitant prices (there’s no way I’m putting my 25tb or so personal data archive on them).
One of the commenters to Querna’s post highlights the data storage of memristors which are capable of storing data at much higher densities than existing hard drives and are nonvolatile (sort of like flash memory today). Moreover, Stanley Williams — who first invented the memristor — has said that the lifespan of memristors could be for periods far longer than mere millenia. Now all we need are for memristors to become cheap and widely available!
And don’t forget that even with memristors or something like it, at the moment your data is stuck in this lonely gravity well in some third-rate planetary system that is vulnerable to a number of potential catastrophes. What we need are autonomous, reproducing memristor-bots that can spread that data archive throughout the universe (now that’s cloud-based computing).
First heard about Millenniata’s M-Arc archival DVD system when the Long Now blog mentioned that the product was actually shipping.
Millenniata claims its M-Arc DVDs are backwards-compatible with existing DVD technologies, but rather than using a laser to heat up a photosensitive dye, the M-Arc uses a mechanical process to make scratches in a physical layer that M-Arc claims will last potentially for centuries if stored properly. According to a brief summary on the manufacturer’s site, the M-Arc:
Preserves data for centuries with physical changes in data layer Constructed with rock-hard materials known to last for centuries Backwards-compatible on all standard DVD drives Functions like a standard DVD with a capacity of 4.7 GB Exclusively written by the M-Writer™ Drive
The Millenniata site doesn’t list any prices, but Long Now reports $1,700 for the writer and $16-$25 per 4.7gb disc depending on the quantity.
Since there can never be enough Star Wars merchandise (apparently), Star Wars flash drives seemed the obvious way for Lucascorp to go. Along with Yoda, a Boba Fett, C-3PO, Darth Vader and others will be available in October 2009. Unfortunately there was a bug in the Han Solo version — it turned out you had to insert the Greedo flash drive first before it would work.
Via the Long Now blog, I ran across this article describing researchers’ efforts to create a data storage system to last a thousand years or more. Rather than rely on optical or magnetic media, both of which can be corrupted fairly easily over decades — much less hundreds of years — of storage, Japanese researchers proposed a system of stacked wafers composed of mask ROM,
Thus, the researchers proposed the idea of saving data on the mask ROM with electron-beam direct-writing technology, stocking the wafers and packaging them with SiO2 to form a “slate.” When a wafer (reader) for reading data is attached to the slate, it becomes possible to supply power and communicate signals by wireless.
If four 15-inch wafers made by using 45nm CMOS technology are stacked, the memory capacity will be 2.5 Tbits.
As long as humidity levels are kept low, the proposed device would have a lifespan of hundreds of years.
The Long Now blog adds a nice twist on how to solve a problem inherent with any such device — a thousand years from now, how will our descendants know how to access the data?
If someone finds this disk 1,000 years from now, how will they know how to access the information? We think a microetched instruction manual might do very nicely.
Nice. That’s how I plan to backup my World of Warcraft videos.
I really wish there was a video online somewhere of Rose White’s presentation on data storage at 25C3, The Infinite Library: Storage and Access of Pornographic Information.
Of course, it has always been a pain to store pornography — and so we have the cultural trope of a stash of magazines “under the mattress” or in a box hidden in the closet. But as the sex industry shifts toward digital publication at every level, we might imagine that mere storage will become a problem of the past, or, at least, a problem related to legacy materials (books, magazines, videos, comic books, photographs, etc.). Cheap, massive storage media means no more problem, right?
Well, reviewers of porn find that they quickly amass more material than they will ever have time to peruse; librarians who need to provide access to controversial and poorly cataloged material end up overwhelmed; even casual collectors of pornography still need some way to keep track of what they have.
Toward that end, I am doing preliminary research on how people store and access their digital pornography collections. In my early interviews, I have already encountered a fascinating mix of responses; one person has said they store their porn “in the cloud,” while another explained his detailed system for hiding digital porn files from his partner.
Back in March, the Long Now Foundation blog featured an extremely long post republishing two articles and a paper concerned with the potential loss of data caused by the increasing speed at which storage technologies become obsolete and, soon thereafter, difficult to access.
Of the three pieces, Jennifer Stilles’s look at the National Archives’ efforts to preserve/recover data stored in obsolete formats was the most interesting. It seems clear from Stilles piece that the crux of the problem is the constant drive for technological innovation which produces products that are ever better but also, too often, ever more incompatible with previous formats. Moreover, this is a problem that started long before the current digital computer age,
On the wall are the internal organs of a film projector from the 1930s; the old heads have been mounted to play together with modern reels. “Twenty-eight different kinds of movie sound-tracking systems were devised during the 1930s and 1940s, trying to improve the quality of sound tracks,” Mayn explained. “Most of them are unique and incompatible.” This particular one used something called “push-pull” technology, in which the sound signal was split onto two different tracks. The technology was meant to cancel out noise distortion, but the two tracks must play in near-perfect synchrony. “If it is played back properly, it is better than a standard optical track, but if it is played back even a little bit improperly, it is far, far worse,” Mayn said. In the mid-1980s at a theater in downtown Washington, he was able to actually use this reconfigured projector to show several reels of push-pull film containing the trials of top Nazi leaders at Nuremberg. And the lab has transferred some 1800 reels of push-pull tape onto new negatives.
Wow. That fits nicely with one of the main problems with data storage today once you get past the physical media — the plethora of file formats and an odd lack of recognition that this is even a problem.
Microsoft rolls out yet another proprietary format for Office? Everybody simply upgrades without a second thought, because if you don’t all of a sudden you’re receiving file attachments you can’t open. Much of this is driven, I suspect, by the view that most data production is largely ephermal. Are we really going to be want to be able to open this report in Word 2003 format 10 years from now? Of course, I’ve also seen the fallout from that where people run around trying to find some way to open that 10 year old file which is suddenly extremely important due to issues with a specific vendor or contract, etc.
The current state of data preservation efforts remind me of the documentary “The Chances of the World Changing.” The documentary follows turtle enthusiasts who, given the lack of any coordinated effort to preserve endangered turtles, create their own ad hoc network of mini-Arks. They buy up individual turtles from overseas, and store them in warehouses, basements, garages, etc., moving the turtles around when one or another enthuisast burns out or runs out of cash. And they hope they’ll be able to keep the turtles going and around until they’re able to get others to see the need for a permanent, formal preservation effort.
I couldn’t agree more with Jason Scott’s essay, Fuck the Cloud — if you are entrusting important data to a service that you don’t control and don’t have a migration path out, you are a fool.
Because if you’re not asking what stuff means anything to you, then you’re a sucker, ready to throw your stuff down at the nearest gaping hole that proclaims it is a free service (or ad-supported service), quietly flinging you past an End User License Agreement that indicates that, at the end of the day, you might as well as dragged all this stuff to the trash. If it goes, it’s gone.
. . .
Contrast, though, when people are dumping hundreds of hours a year into the Cloud. Blowing out photos. Entering day after day of entries. Sharing memories, talking about subjects that matter to them. Linking friends or commenting on statuses or trading twitters or what have you. This is a big piece, a very big piece of what is probably important stuff.
Don’t trust the Cloud to safekeep this stuff. Hell yeah, use the Cloud, blow whatever you want into the Cloud. The Internet’s a big copy machine, as they say. Blow copies into the Cloud. But please:
- Don’t blow anything into the Cloud that you don’t have a personal copy of.
- Insult, berate and make fun of any company that offers you something like a “sharing” site that makes you push stuff in that you can’t make copies out of or which you can’t export stuff out of. They will burble about technology issues. They are fucking lying. They might go off further about business models. They are fucking stupid. Make fun of these people, and their shitty little Cloud Cities running on low-grade cooking fat and dreams. They will die and they will take your stuff into the hole. Don’t let them.
- Recognize a Cloud when you see it. Are you paying for these services? No? You are a sucker. You are giving people stuff for free. I pay for Vimeo and I pay for Flickr and a couple other things. This makes me a customer. Neither of these places get my only copy of anything.
- If you want to take advantage of the froth, like with YouTube or so Google Video (oh wait! Google Video is going off the air!) then do so, but recognize that these are not Services. These are not dependable enterprises. These are parties. And parties are fun and parties and cool and you meet neat people at parties but parties are not a home.
I think this is a much bigger short term problem than the sort of more basic data preservation problems. People are dumping all of their data into different services and coming to rely on those services without ever thinking, “what if this company goes out of business next year?” In many cases, people won’t even realize just how much they’re dependent on other people providing them access to their data until that disappears.
Personally, I do use a lot of cloud services, but I am also fairly obsessive (ok, ridiculously obsessive) about making sure I have personal copies of everything so the day those services go asking for a bailout I’m not stuck wondering whether I’m going to be able to get my data back or not.
This is also one of the reasons those offering such services need to be pressured to adopt open standards so it is simple and straightforward to create local copies of any data and/or migrate to another service, whether it be on another web service or on a server the user controls. Most of the sort of web services today that Scott is bitching about seem to think that locking their customers into their specific service is the way to go, emulating the Microsoft’s of the traditional software market.
