Remembrance of Links Past -- Maciej Ceglowski

Since Pinboard has collected a lot of bookmarks at this point, I thought it would be interesting to actually run the numbers on link rot - the depressing phenomenon in which perfectly healthy URLs stop working just a few years after appearing online.

Link rot in my own bookmarks is what first inspired me to create Pinboard, a personal archive disguised as a social bookmarking site. As I’ve shilled before, Pinboard is the only website that will store full page content for the kind of champagne-swilling fat cats willing to pay us a $25/year fee.

But while link rot motivated me to build the site, until recently I did not have enough user data to actually quantify the problem. I was particularly curious to see whether link rot would be linear with time, or if links would turn out to have a half-life, like plutonium. Here’s what I found:

[…]

Here are some of my own open questions regarding link rot:

• How many of these dead URLs are findable on archive.org?
• What is the attrition rate for shortened links?
• Is there a simple programmatic way to detect parked domains?
• Given just a URL, can we make any intelligent guesses about its vulnerability to link rot?

Link rot is a constant of the universe, like gravity.

Maciej doesn’t mention URL shorteners at all, which is a large factor in link rot on Twitter. Perhaps Pinboard resolves all links at the time of bookmark creation, which eliminates a lot of the 301 headaches that could arise from URL shorteners going out of business.

I wonder what Hilary Mason has to offer on this topic?

(via Audrey Watters, 2105)