In the earliest days of the World Wide Web, very few people were thinking about digital preservation. But a quest to revisit websites from the distant past—to, say, look up the online syllabus of a college course taken in the 1990s, or to search for an old MySpace profile—might come up short. Because those older pages were built using technology that present-day web browsers can’t read or access, they would likely appear as broken links, or only partially loaded.
Applying the same logic, today’s websites could someday become unreachable, too. For example, with the announcement by computer software company Adobe that it will soon phase out its Flash Player, the multimedia software that was the backbone of videos, games, and other web applications of the 1990s and 2000s, many of those older websites will be rendered obsolete.
Enter the mission of digital archivists, who have been working to preserve the massive daily deluge of dynamic web content that comprises today’s web—including social media feeds about major events, YouTube videos, and other interactive elements—for future use and scholarship. Using tools called web crawlers to “capture” webpages, the goal is to save important Internet content, before that content changes or it can no longer be reached.
“We are again and again pushed and challenged and asked to broaden our software development work based on the really exciting ways that artists break things or make new things.”
“Some technologies which are common now, like [Adobe] Flash, will become obsolete and go away over the course of the coming years. Due to upcoming changes in tools used to create and access websites within the next few years, there will be a lot of websites that aren't going to behave like they do today,” says Anna Perricci, the associate director of strategic partnerships for Webrecorder, a web archiving tool developed by the New York City arts organization Rhizome.
In order to address this, Rhizome teamed up with developer Ilya Kreymer in 2016 to build Webrecorder through a two-year, $600,000 grant from The Andrew W. Mellon Foundation. Kreymer formerly worked on the Wayback Machine, an early digital archiving tool launched by the San Francisco non-profit Internet Archive. Kreymer and his team have since built Webrecorder as free, open-source software capable of capturing web content such as videos and social media profiles with their interactivity intact. That means a user can actually browse a social media profile, scroll through and search a social media feed, or watch a YouTube video embedded in a page that has been archived by Webrecorder.
In addition, Webrecorder is also able to emulate the older versions of common web browsers like Firefox and Chrome, so that pages of websites leveraging older technology can be captured as a functional copy that looks and feels like the original. This way, “you can perform an artifact without having to update it or having to change it in any way,” explains Rhizome executive director Zachary Kaplan. He says this approach is different from other archiving software, which has crawled the web to capture a vast collection of static, HTML pages. Webrecorder, on the other hand, captures the way an individual Internet user would want to explore a website.Webrecorder is a free, open-source software capable of capturing web content such as videos and social media profiles with their interactivity intact. Screen shot courtesy of Rhizome.
“It's important that web archiving captures the context around a given artifact, and it's built to capture that context and give the user-eye view,” Kaplan says. “What is an individual route through a website or through a digital web experience? That's something that Webrecorder's really good at creating.”
Webrecorder is being used for digital archiving projects by arts and academic institutions like Carnegie Hall, the Harvard Law School Library, Stanford University Press, and the New York Art Resources Consortium. In 2018, Rhizome received an additional $1 million grant from Mellon to continue to develop Webrecorder.
Rhizome also uses Webrecorder to preserve and present digital art, including the works in its Net Art Anthology, a two-year online exhibition of 100 works of digital art spanning from the 1980s to present-day.
“Digital art, it's a little bit like the greyhound that inspires the horses to race,” says Kaplan of Rhizome’s long history of digital art preservation and its continued development of Webrecorder—it launched the ArtsBase archive back in 1999, for preserving digital art from all over the world. “We are again and again pushed and challenged and asked to broaden our software development work based on the really exciting ways that artists break things or make new things.”