Flow – Sol Young

Returning to “Traditional” use of Twitter

Posted on August 9, 2008 by Sol Young with 6 Comments

After using Twitter as my push-based latest-news system for five months, I’ve gone back to the “traditional” use of Twitter. Without IM and large follower functionality, Twitter offers no way to experience a flow of tweets.

“What have I done!?”

I’ve gone back to the traditional use of Twitter. The method more than 95% of the userbase uses it for. I now use it to stay in touch with the people I’ve met and know personally, rather than using Twitter as a medium for info aggregation. It’s not possible to use Twitter how I did in the past.

If you know my series on flow (it kicked off here), you know what I was doing and how cool it was. I got the idea partially from Robert Scoble’s entry, The Secret to Twitter. His use was brilliant and it worked amazingly well!

Back in March of ’08 I began following any interesting person I thought to be intelligent and putting out informative tweets. Primarily I found people in the software development, new media, aviation, library science, and management arenas. I ended up following 6,218 people at the high (last week). Everyone’s updates were viewed in IM and I would see an amazing flow of information.

Usually hundreds of tweets per minute, forcing me to read very quickly and get a quick read on the blogging, technology, and media areas in a short period of time. It allowed me to find articles and posts that would have filtered in slowly on RSS (arguably, if I had more than my 632 RSS feeds I’d find more information here, too).

It was great. Flip on iChat over breakfast and watch the flow while eating granola and yogurt. An ideal start to the day.

But in the last week I’ve culled over 4,000. The removed are people who don’t follow me and who I never met in real life. The chance of our interaction is very small, and if we meet I’ll follow.

I’m looking forward to having more intimate interaction with friends and followers. Focus will shift more towards FriendFeed and Google Reader (RSS).

Posted in Blogging, Flow, FriendFeed, Micro-blogging, Microblogging, RSS, Social Networking, Twitter

Twitter’s one-to-many scaling impossible?

Posted on June 27, 2008June 27, 2008 by Sol Young with 5 Comments

Twitter has been having all kinds of scaling challenges. There have been hundreds, if not thousands, of posts on the subject. Dave Winer pushed an idea for a decentralized Twitter (and has since admitted the power of Twitter is in its centrality). There is a single, simple, reason for Twitter’s challenges – Math is against them.

The facility of communication on the Twitter service is absolutely outstanding. I’ve written extensively about using it to receive an amazing amount of quality information in my series on flow.

I originally questioned the scaling ability of the service prior to SXSW, but when the service held up I went back to the drawing board to make sure my numbers were correct.

Before continuing, let’s establish the basics about the service so the math will make sense…

Each Twitter account can follow any other Twitter account (bear with me and forget those accounts with private updates).
Messages travel in one direction, from the updater to the follower.
Each account has updates from other accounts it follows placed in its timeline.
A Twitter account can selectively receive pushed updates immediately via instant messenger and SMS in addition to having an update added to its timeline.
An update added to an account’s timeline may or may not be push based (lets assume it’s demand driven, or pull based).
An update sent to an account from an account denoted as SMS or IM announcement is push based (there is no other way to send an update – it must be actively pushed from the server).
The mere possibility of an update needing to be pushed requires the system to check with each follower’s settings, thus requiring analysis of each follower for each update.

A warm-up equation

If there are one hundred (100) users and each user follows ten (10) fellow users, and each user sends ten (10) updates per day, assuming all updates are push-based, how many updates are sent?

Posted in Architecture, Blogging, Flow, Math, Micro-blogging, Microblogging, Social Networking, Software Development, Twitter, XMPP

Twitter did some Spring cleaning – stale accounts pruned?

Posted on April 22, 2008April 22, 2008 by Sol Young with 3 Comments

While doing my typical searches for new and interesting people on Twitter to add to the flow, I noticed something indicative of Spring cleaning. You see, when you search Twitter, you usually get pages of people who haven’t updated in a year or accounts with zero updates – ever – and six months stale.

None of those cases seem to be true for my latest search results. There are a couple accounts with no updates in a year, but they’ve got a lot of updates, so they’d likely not be pruned. Either Twitter created a better search algorithm (unlikely, since the results are haphazard and not chronological) or they pruned the dead accounts (makes a lot of sense – I myself got an old account). When I’ve worked at community driven companies, we’ve done plenty of account trimmings.

Twitter doing prunings makes a lot of sense. This is a benefit to the user base, and a huge benefit to Twitter’s load. If this is truly the case, you might do a search for your favorite name about now… And if it isn’t, at least rejoice in a better mechanism to find the people you’re looking for.

Posted in Flow, Prediction, Twitter, Web Services

Flow – Jabber/XMPP as an RSS over HTTP replacement

Posted on April 18, 2008 by Sol Young with 6 Comments

Twitter on XMPP is just the beginning…

Courtesy NASA Glenn Research Center

I’ve been using Twitter as a main source of news and entertainment (it’s entertaining and informative to have commentary coming in with links, events, articles, and photos). Most everything pertinent to my areas of interest are discussed, so the latest news is passed around as discussion.

As my series on flow describes, my Twitter stream is received through a GTalk client and I’m receiving about 30 to 40 tweets per minute.

This is a lot of incoming information. A lot more than one could read and keep up with all day. It’s valuable for periods of time… Jump in to the river, jump out. This is sort of like news.

Now, I love RSS. I spend a good hour per day reading feeds. I believe it will be the standard in syndication for years to come. And maybe it will be the format passed over XMPP channels, too. In using Twitter for my flow of information I have discovered how amazing real-time updates of news can be, and how HTTP (the current method of pulling RSS feeds from various servers) isn’t powerful enough.

Imagine Google Reader being push based. Instead of periodically receiving items every five, ten, or fifteen minutes. You receive new blog entries, articles, etc, within milliseconds of their publication. This becomes amazingly powerful because you are no longer reading what happened, you are participating in what is happening.

Comment systems become conversation engines. Discussions and exchanges of information become natural, rather than one-way.

HTTP and web services, with their beautiful RESTfulness, won’t be going away. They have a very effective place for on-demand pulls of data. What I’m describing is a move away from HTTP and web services which currently poll – the enablement of FriendFeed, Twitter, blogs, and news services to fire off announcements on a push basis…

Nobody wants to wait three minutes before receiving their next round of updates. We want it when it happens.

Posted in Architecture, Flow, FriendFeed, RSS, Web Services, XMPP