TagCloud Growing Pains

A few weeks back, I tried to put a TagCloud up at /Message, and I got the endless “This cloud has no data message” for about a week and a half, so I took the widget off the blog.

Turns out this is just another growing pain of the Blogosphere, which is approaching the size of the Milky Way:

[from tagcloud : Message: State of the TagCloud - Dec 2005 by John Herren]

It’s not you, it’s us.

Really. I’ve had many people write in lately confused or upset about TagCloud. It seems like lots of folks have created clouds but continue to see the dreaded “This cloud has no data” message. When we launched the service in June, TagCloud was able to provide analysis within minutes. Now it’s taking several days. The simple fact is that the size of our userbase has grown to the point that we have a backlog of feeds to analyze. If you see a feed listed on your cloud’s list, then our system likes it. We can grab it and analyze it. We’re just way behind right now. You’ve done everything correctly.

The problem is, we’ve outgrown our current hosting setup. The database is too big, the site traffic is too high, and we’re basically mirroring a nice subsection of the blogosphere. We’re also assuming the traffic of the sites that put a TagCloud on their pages. Think of the name of the most popular tag-related website. I love that site. Since TagCloud does automated tagging, I’m quite positive that we are storing a lot more tags that those guys. Despite all of these challenges, the good news is, right now this is a hardware scalabilty issue.

Like nearly every successful blog-related service, TagCloud is bogging down because of all the blogging going down.

I will wait for the upgrade.