Internet pioneer Vint Cerf predicts the future, fears Word-DOCALYPSE

As has already been said, backwards compatibility and data death has been a perceived problem for years. Perhaps the Doomsday Project was the first 'popular news' item with the death of the hardware (laser disc) making the media unreadable.

Not having the software to decode the data is a second problem.

"Use ODF", "Use XML"? You're missing the entire point folks. However open the format is, just because the data structures exist with software to decode them today doesn't mean it will exist in 50 ... or even 5 years time. Pick some of the 'totally compatible' formats from history and try to read them - for instance wordstar format on a 360kb disk used all over the place ... until word .doc replaced it. I might be able to at the moment but that's only a 30 years old data set ...

So if you can a) find a drive that will take the disc and b) find an os that will read the disk format and c) find software that will decode the file you're fine ...

Perhaps there should be a relatively small, open library of data formats established somewhere - disk structures, storage formats, software encoding techniques et al? These can then be applied to data as required rather than having to preserve the tools themselves...

I'm now off to read a book - which I just have to open to decode it ...

Total non-issue

Scientists don't collect data shrouded in proprietary formats like PowerPoint files. If you can't access the raw figures, then you're not doing science!

rasters

PNG and TIFF (caveat: use a standardized codec) are decent for long-term storage. Most of the public records I've seen are black-and-white TIFFs with CCITT G4 compression (like faxes). These are things like property deeds and engineering drawings, stamped and signed by a bunch of people then scanned. Everything else gets thrown away after a few years, especially the Word docs and CAD drawings. Pretty decent system.

Text

Yes ASCII is a "format", but it's a simple one which is likely to survive a long time. I'm still trying to understand what all this other stuff is really good for. Why is everybody so bone headed about needing all that word processor stuff? I think the answer is that there is much less interest in what a document says than what it looks like.

Encryption is the real danger

If anyone wanted to read a non-encrypted document in some old format or other, I doubt that they'd need to be qualified to work at GCHQ in order to "break" the (non-)code. It's just reverse-engineering something that actually isn't designed to conceal.

Strong encryption, on the other hand, means that once the decryption key is gone, so is the document.

Relevance

If there's a document worth preserving, I'm pretty sure it'll get preserved in a way that future generations can read it.

The problem is that 99.999% of the stuff we generate today is worthless. Hell, even the design notes of big-hitting games like Prince of Persia on the Atari (I believe) are pretty useless and boring, even to those interested. Same for the Mac paint app that had it's original source released recently. There probably exists more useful information in a reverse-engineering of the program itself than anything contained in the design documents that could ever be found and preserved. And this is from an industry where the whole product is digital, not just a document or two about it.

The fact is that I have email in front of me now going back to 1999. I have archives for a few years before that. I have code I wrote when I was a teenager. I have huge essays and articles and documents I've written over all that time. The number of times I have to refer to anything older than a year? So small as to be worthless, and usually just because of poor organisation or convenience rather than being a vital requirement. I can't imagine that most of what goes through a computer database needs to stay around for that long, and the stuff that does ends up being on paper in archives for a few decades at most. There's just no need for it.

As Terry Pratchett says: Digital archeologists of the future? Get a real job. DELETE.

Not for nuthin'

Amongst the reference material I gathered while learning x86 assembly is a file called "386intel.txt", circa 1986, which is just as readable now as it was then. And, if it for some reason weren't, it would be trivial to write software to make it readable.

Not that I much care about future generations knowing the inner workings of ancient CPUs, but I guess the moral of the story is: create files in propriety formats at your own risk, if you care about them being readable in the future, that is.

So Mr Cerf hasn't heard of virtualisation?

What about .org files?

Distant "fear"

Relax V. Most species that have ever existed are extinct. The amount of historical data we have prior to the printing press is paltry compared to what was written but we still seem to make up pretty good stories about the past and presume that what we have is more important than what we've lost (which can't be known).

The premise of the philosophy of progress is that the past is well, outdated and of little value. We want to know the future not what has already happened. Knowing the past doesn't prepare us for a future that is always new, always changing. So if you are a progressive type, it's a non-issue.

This is really a function of human development. As humans get older their attention always turns more to the past than the future primarily because they have little future left. Older people (and I am one) always think that the way it was was better than the way it is or is going to be. Not true. It will suck for people in the future just as it did for us in the past. It'll just suck in different ways.

Holding on to all this big data just means one ends up with a bigger haystack in which one has to search for the needle.

As for Doris Goodwin's book on Lincoln, we needed another book on Lincoln like a whole in the head. It's like getting another book on Jesus or Plato. Studying the past (I was a history major) is a great way to escape the present but it doesn't mean beans for the future anymore. When things changed very little for a thousand years, the past may have had some actual use but now, not really.

it's about science

He actually gave good example: "So years from now, when you have a new theory, you won't be able to go back and look at the older data"

There is stupendous mountain of scientific data collected every day (e.g. CERN alone collects 15PB per annum) and yet there is little warranty that this information will be of any use to future generations. Since referring to old experiments to verify new theory is established, and very useful practice, this is actually important.

Reply to: Citizen 140924376

Due to a lack of decipherable information (Files in state sanctioned format Ab19-0-7 are unavilable prior to 2025), we are unable to validate your date of birth, nationality, and any work history prior to 2025.

Therefore your retirement/disabilty claim is rejected.

Failure to produce the relevant files (In state-sanctioned format Ab19-0-7) within 30 days will result in your reclassification as Non-person, and any files on you dated from 2025 to present will be deleted.

Have a nice day, and remember, The Council Loves And Cares For You.

It's a harder problem that you might think...

There are really two problems here, one is the problem of retrieval over a time span of about one human, say 80 years, and the other is about designing systems that permit information retrieval over really long spans, like 10,000 years.

We are barely getting to grips with the first one, but we have at least learnt some lessons. For example, binary blobs are really bad, whereas text formats are pretty good. Not perfect (how many people can still read EBCDIC?) but fairly trivially decodable.

The other problem fairly rapidly descends into a spiral argument even if you assume the existence of a long-lived storage medium. (Aside: clay tablets seem to do remarkably well!!) Let's say you write something in French or English. Who's to say in 10K years anyone will understand that language? And if you write a decoder, what do you write it in? There is a project called Rosetta (see the WP page) that tries to tackle this issue, but it's not easy.

Finally, and then I'll shut up: the quality *today* of content or items has no bearing of the importance in the *far future* of the same. Think how much we have learnt through archaeologists digging through middens (i.e. sh**-heaps) and turning up bits of junk that tell us so much. Whose to say that a future digital archaeologist won't unearth "Charlie Bit Me" and explain how late 20th century families worked?

Office formats were always more like memory dumps than archival formats

Office file formats, no matter what office suite or version, were never meant to be archival formats. They were more like save games, little "memory dumps" allowing you to continue the game where you left off, no more no less. In fact some early systems even just dumped the memory onto diskette. (i.e. the Canon Cat) That's why such formats have non-portable options like OLE objects which are nearly impossible to open on another computer. If such a file ever moves from one computer to another you are screwed.

If you want to have something you want to be able to read in a few years or send to someone else, you must use archival formats. Those formats must be as trivially simple as possible. Possible candidates for archiving "printed" documents are TIFF (bitmap format, supports multiple pages) and archival grade PDF (special PDF without all of those useless features). Be sure to include a dump of the text in a separate text file so it's trivial to search. You don't need to change things in your archive. If you want a newer version re-create it again.

Never ever ever store data in file formats you cannot read yourself. Complex (binary) file formats are acceptable only as long as they don't have to be backed up. That's why SQL-Servers tend to store their dumps as simple text files.