Crossing the streams – large numbers of Twitter updates

Chris Bilson (@cbilson) had a good description regarding my post about Twitter’s scaling/architecture challenge.

Kevin Rose and Leo Laporte tweet at the same time = crossing the streams”

I dunno if Proton Packs have exponential load challenges, but the end result for a server can feel similar. Is my post I pointed out that Twitter has to determine delivery options and potentially deliver between 100 million and 1 billion updates per day.

But that’s in a day. 1 billion messages in a day are a piece of cake when spread over 24 hours. What if 1 billion messages have to be delivered in an hour? Or all at once?

Take my list of the top-10 Twitter accounts and imagine them all at TED, WWDC, Google I/O, or your local unconference. These ten users, if each sends an update around the same time create 321,928 messages that need delivery (total number of followers for top-10 accounts). This is an awesome amount of message delivery. If those ten users live-blog or get conversational and send ten updates in an hour… 3,219,280 (again, that’s from only 10 users).

I don’t illustrate this to state it’s these power user’s fault. Absolutely the opposite. They’re generating amazing amounts of traffic, which is a wonderful thing, and the algorithms are the problem.

It’s possible to optimize algorithms and modify systems for maximum performance. I bring up Twitter’s challenges because I’m wondering if this is a challenge beyond present day computing.

To open some minds, here’s an impossibility often overlooked: Huge numbers in a deck of cards (just to show impossibilities can stem from small initial numbers).

Google Contact API finally here

Google Code

Cart dropped a note about long anticipated API addition at Google (so long that most people forgot about they might even still be working on it). Google now has a contact API. This is huge news because up until now only Yahoo! offered a good synchronization service.

I’ve been a Yahoo! Address Book fan for years because of their service, especially how it integrates with their Go service. With Google’s API we’ll see apps that start synchronization between these two, as well as any other service out there. This is fantastic!

Sun acquiring MySQL – Terrorism wins with the OSAMa stack

That’s a cheap dig, but I heard this news this morning and couldn’t resist. “Shockwaves” don’t describe it, but it’s a big shocker for the web dev community this week. What will it mean for those of us that implement MySQL? Probably very little actually.

Sun will keep it open and we’ll still have our LAMPs. They’ll have greater influence over features which optimize their Java or OpenSolaris products. We can hope they’ll fuel faster development and maybe we’ll get some added goodies with their combo.

At least we know where to find OSAMa. Maybe their first task will be to market it with a different stack name?