techno.blog("Dion")

FriendFeed is marching on. I prefer it to Twitter these days, but of course the users are still mainly on Twitter. The best of both worlds for me is to run FriendFeed as my “client” yet have Twitter users happily following and not knowing or caring that, that is the case.

We may have the pieces to actually do this right now:

Posting to Twitter

FriendFeed added support for posting activities to Twitter for you. People immediately worry about spamming Twitter with this stuff, so you should think about that and turn off a bunch of services. I currently use TwitterFeed, which I will turn off if I use this feature (as FriendFeed will handle that as I turn on my blog service).

Seeing your @replies

FriendFeed is one of the few companies who have access to the XMPP firehose. They should be able to grok the replies (e.g. @dalmaer) and merge those into the comment stream.

In the meantime, DeWitt has a great hack:

I did this, registering “To Dion” as the friend. It is ugly as it doesn’t merge into your flow, you can’t easily tell who has replied, and it is one big link… but it is good enough.

Friend / Follower Syncing

The people I follow on FriendFeed and Twitter isn’t in sync, and it should be. I haven’t caught up on FriendFeed. The beauty of the imaginary friends feature is that you can add Twitter folks even if they don’t have FriendFeed accounts yet. Ideally, I could tell FriendFeed to do the syncing and followee management for me.

For now though, I could try this sync script and see if it does the trick.

Good times for FriendFeed. I also think that it shows how great their real-time Web experience it is, auto updating in front of my eyes. If they implement these features natively, I will be a total convert.

This is incredibly anal, but I appreciated it. When Facebook launched their new look and feel, it started out on a beta site, and then migrated to become the default. For what feels like a looooong time, I have seen this header:

The problem is that I get it. I know now. Let me get rid of the header please!

Twitter on the other hand did a really good job with this. When they launched the election subsite I started to see this:

Once I internalized that I could click on the ” X ” close icon and it would minimize to this:

Minimized allows me to get back to it if I want, but without taking much real estate. There have been some issues where it doesn’t seem to remember that setting, but in general I like this kind of implementation. On the right hand side they have a little box for their new look and feel that says “Notice something different? Read about the changes.” I wish that had an ” X ” too, without minimize, as again… I know it is new, and I don’t need to read about it anymore :)

First, I apologize for writing a post with a title that has both FriendFeed and Twitter in it. I understand that most people will hit delete as soon as they can as they are sick of hearing about them.

However, I feel like writing this so I will.

I have dabbled in FriendFeed, but although I thought I should really like it, I never got into it and stayed in Twitter land. It took me awhile to really get the right feel for Twitter though, so I decided to really try to spend time with FriendFeed this week and see how I feel at the end of it.

I think I found the zen a little, but there are good and bad things:

Participation

Once you participate in conversations on the items you see how much nicer it is compared to @replies thrown this way and that in your stream. This is a little like going to conversations in Gmail giving you real threading in email instead of random replies.

One side effect is that you don’t do as many posts yourself, and do more comments on other peoples.

The down side though, is the way the river flows through. With Twitter it is easy, it flows on up. Here though, again a little like email, threads jump around as people comment. I find this pretty weird, as suddenly an item from 4 months ago shows up just because some random bloke “liked” it.

I do find that I actually click on “like” here though, whereas I never “favourite” on Twitter. I don’t know why. I think it is because you actually SEE when people like something so you are motivated to play the game.

Go Away A-List!

One nice viral (organic @kmarks?) feature is that they show you items one friend away (friend of a friend). This seems great, and can make a nice way to find people that you would like to follow. Seeing their stuff allows you to judge that, whereas “find friends” doesn’t.

The downside here though is that I can’t get away from Robert Scoble. No offense to Robert. The problem is that even after I unsubscribed to Robert because most of the posts had something to do about him…. I still get his posts because of this:

Since so many people connect to him and other A-lists, chances are that some of my friends are in that chain too. Argh! There doesn’t seem to be a way to explicitly mute these, and using ‘hide’ a lot doesn’t seem to tell the engine to do it itself. “Oh, wait, Dion has hidden this guys stuff a lot so lets lay off.” The net effect is that the A-list folk have way too much of the feed.

Another couple of random things. For some reason I couldn’t get Flickr to work as a service. The interface asks for your nickname or email, so I tried both: my Flickr username (dalmaer), and my email, both personal and the Yahoo! one. No cigar. Finally, I went over to Flickr and duh, the email was foo+flickr@almaer.com in my settings. I plug that in, and FriendFeed found it.

Finally, I am still looking for a bit more. I still want The River to be a tool that I live in, instead of something that sits over there. I hope the good folk at FF push ahead :) I do find that I am very rarely in Facebook now. My social group isn’t really there anymore, and my Dad is on there a lot more than I. The new iPhone app is awesome though, and could become your contact list.

Thanks for putting up with me for this post. I keep thinking that I sound like Steve Gilmor going on about @replies and the power of track.

Sometimes you can get in the zone just enough to be productive on a plane. On my flight to Mexico City yesterday, I created Endpoint a project that contains a server proxy, JavaScript client, and Greasemonkey Script with a mission. The mission is to take a URL, work out if it is a redirect (via a Location: header), and then return the final endpoint for it.

Why did I do this?

I was brainstorming functionality for a Twitter client with James Strachan (he is working on gtwit) and we talked about how annoying tinyurl / is.gd / snurl / you name it URLs are. They don’t tell you where you are going, and you could get Rick Rolled (if you are lucky) or much much worse.

So, I wanted to create a library, and one client (Greasemonkey) to test it out. Then anyone else could use it too to resolve directly from their Web pages.

How does it work

You load up the JavaScript via script src and then you can call resolve, passing the URL and a callback that will get the result. A few examples:

// Simple version
Endpoint.resolve('http://snurl.com/2luj3', function(url) { 
  alert(url); 
});
 
// Using the original URL to work out if it has changed
Endpoint.resolve(
  document.getElementById('testurl').value, 
  function(url, orig) { 
    alert(url); 
    alert(Endpoint.isRedirecting(url, orig));
  }
);
 
// How it is used in the Twitter Endpoint Resolver
Endpoint.resolve(url, function(resulturl, originalurl) {
  if (!Endpoint.isRedirecting(resulturl, originalurl)) return;
 
  newtext = newtext.replace(originalurl, resulturl, "g");
  jQuery(el).html(newtext);
});

Under the hood, a bunch of stuff is happening. I would love to be able to just use XMLHttpRequest to dynamically hit the URL and look at the headers, but the same-origin policy stops me.

This is why I have the server proxy, which returns a JSONP callback.

When you call resolve(url, callback) the script tag is created on the fly and added to the DOM. The callback function is all handled to allow multiple calls, and then the chain unravels.

Here you can see it all in action, showing how my Twitter stream will go through and the URLs will dynamically change from their tinyurl versions to whereyouaregoing.com:

I wanted to use App Engine to host the server proxy, but unfortunately I can’t work out how to do that yet. You have access to the URLFetch API to access resources from App Engine. Unfortunately for me, one of the features is that it understands redirects and just goes on through to the full resource itself, with no way to get the endpoint from the headers in the response.

It was also interesting to read Steve Gilmor talk about these services all be it in a post that is hard to actually understand ;)

Also, Simon Willison just put up a simple service on App Engine, json-time, that “exposes Python’s pytz timezone library over JSON.” I think that we will see a lot of these types of mini-Web services hosted on App Engine. Taking Python utility and making services from its goodness is an obvious choice.

@al3x told us about the Twitter architecture, and what says it all is:

Amid the huge number of “oh no twitter is down. make it faster!” posts, we have some good ones.

The answer isn’t “Use PHP” ;)

If I was Twitter I wouldn’t be looking for Erlang as the answer, but I would be interested in talking to Joe Armstrong. I wouldn’t jump to Java as the answer, but I would be reaching to talk to Cameron Purdy of Tangosol Coherence (now Oracle). These people have seen systems that make Twitter look like a toy in comparison, and it is the knowledge that is more valuable than any technology.

If you think about contorting a typical LAMP stack to run Twitter you quickly shudder. Having a database layer, even with master/slaves, is scary.

Twitter needs a Mom, and it looks like it is finally getting one. With true message oriented middleware, and money to get the systems they need, they should be fine. As Cedric says this isn’t an original problem.

The system of messages shouldn’t be living in bottleneck databases. Instead, they can be flowing through a river of distributed cache systems. The Jabber side of the house shouldn’t be able to “bring down” the entire website. The beauty of publish subscribe and messaging is that you can throttle things nicely. You shouldn’t be “running out of database connections.” You can tune the number of listeners for the “track” command for example, and if it is getting abused you limit its resources. Sure, this may mean that you get messages to people a little later, but who cares. If messages got a little slower would people even realise? Compare that to the birds lifting the animal to safety message.

In fact, if you think about systems such as the stock exchange. You will realise that you rarely get truly real-time access. Most of the time you are on a delay of some kind, and that is fine. Through the distributed caching architecture you can push out messages to other systems to do their work. One of these systems will be the website itself. Twitter.com is just another client to the messaging backbone. Even if the backbone is in trouble, the website can still show the current view of the world, and could even still batch up work to be done.

I was talking to another startup that is migrating away from a database backed system, and soon the entire real-time world will be in a huge distributed cache. I am sure that Twitter will be moving there too.

Currently, I still feed bad for the engineers. I have been there; The point where you are the limits of your current architecture and you know it can tank at any time. You are firefighting all day and night, and thus don’t even have much time to fix anything at all. It is hard work. It is tiring work. It is demoralizing work.

However, I know that Alex and the rest of the crew will pull through their current situation, which after all came about thanks to the amount of love that its users have for the service, and one day the new architecture will be there in a way where we will look back and remember the early days, where downtime was such an issue.

Thanks for all the hard work guys. I can’t wait to be tweeting on a fully loosely coupled architecture, talking to one of your Moms!

I like the idea of TwitterFone. It would be nice to be out and quickly call in a tweet. I got an invite, and tried it out this week, and then remembered that voice recognition is really hard. Doesn’t it always seem to be one of those technologies that is “10 years out.”

My test tweet occurred after Man U beat Chelsea for the Champions League trophy.

I said:

and I saw the following in my twitter stream a few minutes later:

To be fair, they did pretty darn well. It is hard to expect them to know something like Manchester United. Unfortunately, even though they did well, it would be hard for anyone to understand what I was saying.

I then thought back to Twitter Translate. In that case we are translating between languages, and it is also often “off”. However, you can normally work it out, and there is a key difference. I expect that if I translate something that it will be a bit off, and I will cope with that. However, something like TwitterFone is happening without the consumer knowing, and thus it comes across as gobbledy gook as they don’t have the context (other than seeing “twitterfone” if they look carefully).

It seems that there are many use cases where it is OK that translations of some kind are “good enough”, and others where it does matter. You could get into trouble too, such as the comic above which messes up the “d” for direct message and changes it slightly so you say something totally different.

You could go from: “I like yoghurt and peas” to: “It hurts when I pee”.

Then there is the 140 character issue. It is impossible to do the math as you are speaking a sentence, so you will often run on.

Nice idea, but we need the technology to get to the next level for this kind of input to be used in a case like this. Using it for GOOG-411? Perfect… you can keep saying the same thing again.

Every other day you see a post like this or this.

I wonder if the downtime that Twitter has, may have been a good thing in some ways. It gets people riled up, and we all rant about how it is crucial that we have a Twitter that is up. But, it isn’t down enough to make people switch to Pownce, which doesn’t seem to go down (or at least you don’t hear about it) so it therefore doesn’t get in the news.

I wish we could have a parallel universe that could compare the growth of a service with zero downtime versus a touch of it, in this case.

I am having a lot of fun with the AJAX Language API. Last week I talked about the translation bookmarklet that lets you translate anything that you select in the browser.

This time, I whipped up Twitter Translate, which watches the tweets in the page, and if the content isn’t your native tongue automatically converts it and replaces it inline. It then adds a mini logo with “translated from …” which you can click on to see the original text:

It is probably easiest to quickly see it in action:

I think that I am so excited about this API as it is a vertical service that you can just use for free. Think back on how you would be able to integrate language into your applications in the past. You would either:

Now, we can use this service for free, and the best part? It will keep getting better behind the scenes without us having to do a thing!

Some people, apparently erroneously, said that Twitter is testing ads mid-stream just like clients such as Twitterific do.

I didn’t think this was new, as when I look at my stream I see something like that image above. Isn’t it already ads? :)

Twitter is the watering hole for those of us who don’t work at the same company, and we are Beacon’ing all the time.

NOTE: This isn’t a bad thing at all. And, I know that I do it all the time!

Before I start, I have to get it out that the thinking in question took place at 5am. I have been enjoying time in Europe, getting to meet various developers on the On Air tour that Adobe was kind enough to have me speak at. Since I was in Europe for such as short period of time, and due to a few work matters, I ended up staying somewhat on US time. This never quite works, and I think I end up with my body clock tick-tocking somewhere over the Atlantic. If I ever had to crash land on that tiny American runway on the side of a volcano, I am sure I would sleep fantastically at 10pm.

Anyway, to the matter at hand. These emails drive me nuts:

Title: Bob Harris via Twitter to me

"Some random content in 140 characters or less"

Bob Harris / bobh

--
follow me at http://twitter.com/bobh
reply on the web at http://twitter.com/direct_messages/create/bobh
send me a direct message from your phone or IM: D BOBH your message here.
turn off these email notifications at: http://twitter.com/account/notifications

You get them from Facebook too (thankfully they put the random content in there for some of the content), and many other services out there.

What is wrong with them? This is how they come across to me:

I got angry one night (after some dark and stormys, white russians, …) and wanted to fix it.

This lead me down the path of Greasemonkey. How about greasing up the wheels like this:

if the title of the message has / from Twitter to /
  when pressing the "r" key to reply or clicking on the reply button
    open up IM with "d [get reply to]" (grab /IM: D \w+/)
      now you can put in the message

The problem is that tying into all of the actions can be a pain, and it is a little bit annoying to be running this on every email. Oh, and what about the other sites! We don’t want to have to repeat this for every service that doesn’t care about me, do we?

After all of this over-engineering it seemed obvious that I shouldn’t be lubbing up the ape, but Twitter should handle this for me, and thus everyone that uses Twitter.

Instead of Twitter emailing me as <noreply@twitter.com> how about if a gentler, more Oprah-like Twitter greeted me as <pleasereplytoyourmate@twitter.com>. Then the email becomes:

Title: Bob Harris via Twitter to me

"Some random content in 140 characters or less"

Bob Harris / bobh

--
simply reply to this email, and the first 140 characters will be posted

or,

follow me at http://twitter.com/bobh
reply on the web at http://twitter.com/direct_messages/create/bobh
send me a direct message from your phone or IM: D BOBH your message here.
turn off these email notifications at: http://twitter.com/account/notifications

The 140 character limit could be one of the reasons that they don’t do this? Stripping could cut the message up, but if this was the case, I would get the monkey out again and write something to help enforce the limit by letting the user know as they types too much.

Having nice extension points to email sounds pretty interesting to me too. What if Gmail adding to the Greasemonkey JavaScript API so you could add event listeners to events such as: getting new mail, pre-reply, post-reply, typing a subject, adding contacts.

Think of the possibilities.

But, I still want services to take email seriously as an interface to them, as does Dopplr and some others. After all, doesn’t Stallman use email to browse the Web?