Dealing With Capacity Overload and "Laying Off" Features

on louisgray.com from 31 weeks ago

Hey J. Phil and Louis...we had to disable them simply because of the sheer number of calls we were sending them.

We call the services in a very different way than FriendFeed and others, mainly because our goal isn't just to aggregate you and your profiles, but rather, aggregate your friends on that service. The end result is a single dashboard with all of your friends' activity elsewhere.

What this does, is cause a whole lot of extra calls that we have to make to the services. Some of the APIs that we call are prepared for this, and deliver most all of that information in a single, or maybe two calls. However, in the case of del.icio.us, Digg, Last.fm and YouTube, the process was like this:

1 call for your activities
1 call for your friends list
x calls for x friends

In a very short period of time, we might call a service like Digg 102 times in 1 second (let's say, if you had 100 friends).

To remedy this, we've worked with the services to find alternative ways of calling them, as well as have completely re-written our architecture and ways that we call the services from our end. When we launch 2.0, it will reflect this, and all of those services will be added back in immediately. The new infrastructure also allows for us to call RSS based sites and services, as well as opens us up to many other services. We're expecting to launch the 2.0 features very soon (I can't give a specific date, but I can say that we're almost done testing, and pretty much just awaiting deployment now)

I hope that could answer a few of your questions.

Matt Galigan
indiekid