Flow – Jabber/XMPP as an RSS over HTTP replacement

Twitter on XMPP is just the beginning…

Speed of Light

Courtesy NASA Glenn Research Center


I’ve been using Twitter as a main source of news and entertainment (it’s entertaining and informative to have commentary coming in with links, events, articles, and photos). Most everything pertinent to my areas of interest are discussed, so the latest news is passed around as discussion.

As my series on flow describes, my Twitter stream is received through a GTalk client and I’m receiving about 30 to 40 tweets per minute.

This is a lot of incoming information. A lot more than one could read and keep up with all day. It’s valuable for periods of time… Jump in to the river, jump out. This is sort of like news.

Now, I love RSS. I spend a good hour per day reading feeds. I believe it will be the standard in syndication for years to come. And maybe it will be the format passed over XMPP channels, too. In using Twitter for my flow of information I have discovered how amazing real-time updates of news can be, and how HTTP (the current method of pulling RSS feeds from various servers) isn’t powerful enough.

Imagine Google Reader being push based. Instead of periodically receiving items every five, ten, or fifteen minutes. You receive new blog entries, articles, etc, within milliseconds of their publication. This becomes amazingly powerful because you are no longer reading what happened, you are participating in what is happening.

Comment systems become conversation engines. Discussions and exchanges of information become natural, rather than one-way.

HTTP and web services, with their beautiful RESTfulness, won’t be going away. They have a very effective place for on-demand pulls of data. What I’m describing is a move away from HTTP and web services which currently poll – the enablement of FriendFeed, Twitter, blogs, and news services to fire off announcements on a push basis…

Nobody wants to wait three minutes before receiving their next round of updates. We want it when it happens.

Back in my routine – daily running, RSS+flow, and dev

Latest run results… Almost to my goal of < 7:00 pace…

Today is special… It marks the first day of getting back in to my preferred routine. Here it is:

  • Get up early and hit a good run.
  • Read RSS (and now some flow) over breakfast.
  • Drive to the office (while continuing to read RSS).
  • Have a productive day of dev.
  • Get home and enjoy some afternoon/evening time with family.
  • Read more, code more, free time before bed…

For the last few months my routine has been thrashed. Very productive, but insane…

  • Wake up just in time leave for the office.
  • Have a productive day of dev.
  • Get home and follow up with emails and later dev projects.
  • Do demolition/remodeling/construction with my wife.
  • Fall in to bed, catch 15 minutes of news.

The latter routine has accomplished a remodeled home, most recently the bathroom, but hasn’t allowed for much mental expansion or physical fitness.

What you miss in the flow

Meeka telescope Towers Numar.jpg

(interior picture from The Towers of Numar, by Michael Gagne)

It’s been almost two weeks since I started using Twitter as a primary source of news, links, and other fascinating bits of information. The approach has been awesome and I’ve discovered a ton of people and sites which I now return to. It’s been eye opening.

But I’ve been missing sites previously frequented. The time I’ve spent in the flow cut in to time spent reading feeds and visiting sites. And while my Google Reader feeds are grossly limited compared to the nearly 5,000 people I follow on Twitter, there is still some attachment and familiarity that goes missing.

I share my Google Reader items as a feed (RSS) or on a page (HTML), and of course it’s aggregated on my FriendFeed. I am ‘sol‘ on Twitter.

Flow – Day 9 – I switched to iChat for Twitter XMPP

iChat Count 386 – 7 minutes


When following a lot of friends in a flow environment and using XMPP, one sees the above numbers in less than ten minutes. I’d been using Adium, but Adium doesn’t smooth scroll between each received tweet. It constantly jerks messages upwards and has made it virtually impossible to have a meaningful experience. There are often times when I want to read each incoming tweet. A good, smooth, reading experience was needed.

iChat has a slightly smoother hit at each received message, and is therefore much more enjoyable to read. The interface is customizable enough, but nothing quite as nice as some of Adium’s minimal themes.

I was mostly hesitant to switch since Adium has outstanding AppleScript support. I’ve been thinking of prototyping something (given a couple hours – someday). Apparently iChat has something even better which I should have known about… Callbacks! A script can fire for each received message.

This will make dynamic, real-time, filtering a reality.

iChat AppleScript

The start of something very cool…

Flow – Day 9 – Open it up

I’m used to the speed of the flow and it’s slow. It’s time to open it up and look for five-figures…

Useful link: flow entries

Follow me on Twitter: sol

Open it up

I read the flow of XMPP Twitter traffic with breakfast and in the evenings. I then scan it when checking email or if I catch a lot of added traffic on the IM window. The part which most people don’t understand is how this translates and how it’s even immaginable to distinguish signal from noise here.

It’s easy. I’m now following over 4,000 fellow Twitterers (Twitterites? Twitterans?). The TPM (Tweets Per Minute) ranges between 20 and 35. This equates to the Twitterers I’m following announcing, approximately, once every two hours (obviously some are once a day and some are every 10 minutes).

Reading the flow at this rate is easy. You have tweets coming in 24 hours per day, but you absolutely can’t follow it the entire time. Feeling like you have to read every Twitter announcement your friends send is the first psychological obstacle to get over. Once you get beyond that feeling of needing to maintain control, you free yourself to dip in to the news of the moment as reported by everybody.

To ensure I’m not missing any messages specifically to me, I keep a browser tab open (usually immediately to the right of my GMail tab) to the Twitter Replies page.

The main trick to keeping a strong signal is being selective in who you follow. By tuning this early, you avoid needing as much filtration later. To date I have only filtered out a single spammer account.

One last point is that some feel this approach is a pull technique in which I’m getting, but not giving back. I  disagree. I submit my status and the special news and information I come by. I encourage people to follow me so they’ll be able to have an insight in to my thought processes and activities.

Given the present rate of flow, I see 10,000 as the next step. It’ll take a while to get there with a selective approach. In the meantime I’m interested in metrics and whether Twitter will continue to be a best source of this data.

Any service could provide an XMPP flow… Imagine Facebook, MySpace, Pownce, etc, offering an XMPP feed of updates. FriendFeed with an XMPP flavor would be incredible.