The Web Going Dark – Preserving and Serving Aging Websites

Adrienne LaFrance writes in the Atlantic in an article called “Raiders of the Lost Web” about vanishing websites and the recovery of one particularly important investigative reporting piece, “The Crossing.” Along the way, LaFrance discusses the state of web archiving through brief interviews with Jason Scott and Brewster Kahle of Internet Archive as well as Alexander Rose of the Long Now Foundation. The problems of preserving everything are still insurmountable. The problems of just preserving and serving valuable websites, those curated say or deemed to be of ongoing importance, is possible but not yet done well.

I’m glad to see the continued attention to this problem. With the help of support from IBM research grant entitled “Keeping the Historic Web Alive and Accessible: preserving and serving,” ibiblio is involved in using virtualization to keep older versions of older software open and accessible.

Paul Jones' NeXT on which Tim Berners-Lee installed an early web page in 1991

In fact, I’ve kept open and accessible a version of Tim Berners-Lee’s earliest public webpage from 1991. You can read about the context of the page in an Atlantic article from 2013.

Despite the great efforts of Internet Archive and others, archivists attempts at preserving websites have largely been a collection of web page collected periodically by such successful software as’s Heritrix. While page collectors provide a good service especially for sites that do not allow access to the database that drive them, they fail on several fronts.

First, a complex site can never be completely captured by a screen capture program or even a video. The number of potential pages and interactions cannot be duplicated in such a manner.

Second, user interactions with a complex site cannot be duplicated by a captured set of static pages. Understanding interactions is important to the use of archival material.

Third, the context of use and interactions is lost when the user experience is changed from one of interactivity and customization to one of simply retrieving selected static pages.

Even worse are archival activities that attempt to preserve by atomizing the objects from the pages removing images, audio, texts, etc and placing them in new decontextualized object databases completely unrelated to their original context.

We propose, using ibiblio as an example, to attack this preservation problem by using machine virtualization and virtual machine management tools. One suite of tools that particularly interest us are NCSU’s Virtual Computing Lab known as VCL. VCL is in use at over 200 institutions and has a strong support structure. Open source and licensed under the Apache project licenses, VCL is mostly used to provide temporary virtual machine for training computer scientists. We and our partners at NC State see a new use for VCL as a way to manage and present older servers Running older software in a secure yet living and interactive way.

I hope that we and those others of us working on digital preservation and in virtualization can provide a framework that benefits the creators, the archivist, and publics present and future.

Look for some initial results to be posted here soon.

(Originally blogged at The Real Paul Jones)

Septembrrrrr 19 is International Talk Like A Pirate Day


And ibiblio has resources to assist you in your piratical practice:

Bret Harte’s Queen of Pirate Isle I first knew her as the Queen of the Pirate Isle. To the best of my recollection she had no reasonable right to that title. She was only nine years old, inclined to plumpness and good humour, deprecated violence and had never been to sea. Need it be added that she did not live in an island and that her name was “Polly.”

Roger McGuinn sings Henry Martin

Oh no! Oh no! cried Henry Martin,
That thing it never could be,
For I am turned robber all on the salt sea
Salt sea, salt sea.
For to maintain my brothers and me.

Maturin Murray Ballou’s Fanny Campbell, The Female Pirate Captain: A Tale of The Revolution Some fears were entertained by Fanny and her family, touching the captures she had made, inasmuch as, strictly speaking, she had laid herself liable to the charge of piracy, and Fanny, in the eyes of the law, was actually a Female Pirate Captain.

A Project Gutenberg Bookshelf – Pirates, Buccaneers, Corsairs, etc. Amongst which are Thrilling Narratives of Mutiny, Murder and Piracy A weird series of tales of shipwreck and disaster, from the earliest part of the century to the present time, with accounts of providential escapes and heart-rending fatalities.

And a mid-20th Century USS Pirate

Tor Challenge and ibiblio

TOR Challenge logo

ibiblio was one of several sites recognized by the Electronic Frontier Foundation and the TOR Project for the long uptimes and significant bandwidth dedicated to our TOR Relay. 98.96% uptime over a year including downtime for upgrades and maintenance is something to brag about — even without the rewards of stickers, t-shirts and the like. More statistics about ibibUNC0 here.

Tor is one of the strongest tools for protecting privacy online. It’s used by millions of people worldwide to route around Internet censorship and browse the Web with anonymity. Whistleblowers, journalists, activists, and everyday people rely on Tor.

As a joint project of the University of North Carolina’s School of Information and Library Science and of the School of Media and Journalism, ibiblio is proud and glad to help contribute to the Tor network.

Special thanks to ibiblians Dayton Bobbitt and Cristóbal Palmer for their continuing work on ibiblio’s Tor contributions.

The Tor Challenge was a joint campaign of EFF, the Free Software Foundation, the Freedom of the Press Foundation, and the Tor Project.