techno.blog("Dion")

I have had the pleasure of spending time with members of the V8 team such as Kevin Milikin and Kasper Lund, and Brendan Eich of Mozilla and TraceMonkey. I haven’t had a chance to talk with the great folks behind SquirrelFish Extreme yet, which looks very good too.

It has been great to talk to these guys about the way their implementations work, and to look at differences too. What has been particularly exciting has been how they have loved each others work. There is a huge amount of respect between all of the teams, and they are jazzed to be making the JavaScript world scream.

One common issue that I have found has been benchmarks. They all use slightly different benchmarks and can all agree that:

SunSpider has known bugs in it which are not changed because they have a baseline with IE. Are you kidding me? It is also very Regex heavy, and also focuses on other areas. The methodology of just adding up all of the timings also doesn’t help, and totally skews results (you could do the important things fast, and the obscure things slow, and still look bad). IE can’t even run the tests correctly, and thus their numbers are messed up too.

Dromeo is interesting, but has flaws too. The timing seems to be broken as only V8 gives you precision that doesn’t get rounded down, and thus they get penalized. Also, it uses libraries which do “if browserX do X elsif … else ….” which means different engines actually go down totally different paths.

This is the tip of the iceberg. Benchmarks are notoriously hard to build in a way that measures practical performance versus micro meaninglessness.

It seems like everyone acknowledges that JS benchmark life is pretty bad, so we can either keep that in mind, or we can get everyone together to have an attempt to do a better job. Doing the practical thing can be tough too, as there are too forms of practical:

Time for SPEC JSVM?

Brendan Eich promised that trace based JIT’s will give us killer JavaScript speed and now we have seen the fruit of his labour with TraceMonkey (adds trace smarts to Tamarin-tracing).

A lot has been said already, but I am really excited about much more than the “look at how we run Sun Spider”.

The DOM matters

Although it may not sure in todays code drop, Brendan gets that you have to care about the DOM and not running while(1) { cheasyTest(); }:

Canvas

You know Canvas? It’s days are coming… fast, as John Resig mentions:

Opensource matters

None of this could be accomplished without the great side of opensource. TraceMonkey uses Tamarin-tracing, which enables it to build on amazing code.

TraceMonkey itself is opensource, which enables the SquirrelFish team say, to check it out. And, vice versa. This will keep the competition going along nicely.

This kind of competition is phenomenal for us, the consumers of the Web. Performance is key in 2008, and Web developers get to laugh all the way to the bank.

Thanks guys!

After listening to Brendan talk about typing on the latest Open Web Podcast episode on ECMAScript Harmony it got me thinking again about optional typing.

It has always bugged me a little to think of type information in my nice clean dynamic languages (Ruby, JavaScript, Python, Perl, etc.).

Looking at even the simplest of code like this (taken from Mike Chamber’s XMPP server in ActionScript 3)

var room:Room = new Room(connection);

It irks me. Just work out that it is a room already won’t you? I know you can do it? You CAN do it. So, leave it out in this case.

Although the type information is optional, it seems that a lot of the code that I have seen in AS3 puts types in all over the place. I somewhat like the idea of using types when you really need them, such as for clear documentation of a core library, or some performance issue that you find is an issue (note: that becomes an issue, not one that is assumed!)

Once you open the door though, can people play that way? Is giving the programmer a bit switch of “shall I put the type in here?” every few seconds a good thing? Especially when the tools try to put the type in all the time?

I also found it very interesting when Brendan talked about how adding type information can actually hurt performance, sharing an example of when var foo:int doesn’t do what you actually need. For me, performance is pretty much out of the window, now, and definitely in the future.

So, I lean towards not needing it. But then I flip over the cards and see stuff like this (note: a mock example):

/**
 * @type Room
 */
var room = new Room(connection);

Ugh. Now we have the worst of all words. We are documenting the code out of band. This is often done for some jsdoc-like tool that will generate docs for you. The problems are:

This alone makes me think that I would rather have optional types just to avoid any hacks like this.

You?

I had a post in my queue relating JavaScript 2 to Perl 6. They are both in very rough spots indeed, yet JavaScript 2 has a couple of features that have me a lot more worried than I am about Perl 6:

I have been hacking more and more on JavaScript (1.0, Brendans toy!) recently, and the more I do so, the more I like it. I worked out some years ago that it was the DOM and cross browser issues that I didn’t like, not the language.

I do wish that there was a standard library for JavaScript (Dojo does a good job). I do find myself reinventing the wheel a lot when it comes to the Object world. This week I will be in a Class.create() kinda mood. Other times I would fancy some new function() magic, and other times a bit of (function() { functions; return { ... } })(). Now I try to use the right tool for the job, but the problem is reading other peoples code.

Take a peek at the Firebug source and you will see the following at the top of each module:

FBL.ns(function() { with (FBL) {

Joe Hewitt likes with() :)

This is where it starts to feel a lot like Perl. Many ways to do one thing, and when that one thing is so core as packaging and modularization, it shows through. This is also a reason why we are in pain with components. If someone writes a great component for Dojo but you are using Prototype, you curse. What is the real difference? The darn packaging! dojo.hitch() versus function.bind(). These changes go so deep though, that it would be quicker to write from scratch.

I would love an import/include statement. Finding myself dynamically creating script elements to do a script src is starting to get a little silly. The language itself should just support that.

JavaScript 2 is helping us with packaging, but in my opinion, it used to have too much in it. Programming units, packages, namespaces, that is just too much to wrap your head around.

I am also not a fan of the optional static typing. That is fine for ActionScript, but I don’t think JavaScript needs it. This is a scripting language, something that can fit in a onclick="..." and I think the world is moving more towards the dynamic world (there is still a place for static). How about just allowing ActionScript to be written in the browser? Why try to boil the ocean with JavaScript, as Java has tried to do in the past. Work on the runtime, and allow multiple languages. This is already being done with IronMonkey and the like…. and Microsoft is doing a great job with that tactic in Silverlight. Work on an object model that can be shared, and then we can all go to town.

var self = this;

I see that too much too, even with the binders and the hitchers. The magic of var foo jumping to the top of the scope is weird too. But, all in all, JavaScript is pretty damn good! In fact, JavaScript 1.8, and what I hear about 1.9 is really nice too. I am all for cleaning up what needs to be cleaned up, and working on other stuff.

At this point in my draft I sat in the corner and cried, as I thought JS2 was in epic trouble, and that the walls were going to fall down. I didn’t know what to do. We had Brendan++ on one side, and Doug/Microsoft on the other (simplified version). I sit in between. Without some Harmony, what would happen to the Web?

Well, the reason that I can post this is because I heard that the ECMAScript meeting in Oslo was very promising indeed, and some Harmony may come out of it. Hopefully the news will get through the official process, and out in public as soon as possible, as there is new hope for us. I am sure there is still a long road, but with great people working together, we’ll get ‘er done.

Ever since Aaron Boodman uses the back and forth saw animation to demonstrate how the Gears WorkerPool can keep your UI responsive with the browser, I have used it for testing just that. It turns out the human eye is quite good at the “oh man look at how badly that jumps” kind of test.

To make it simple, I have componentized it, allowing you to drop in a kitt.js and a new Kitt() to be on your way.

This is the minimum that you need to kick one off. The new Kitt will look for an element with the default id of ‘kitt’. Since it isn’t found, one is created and appended to the end of the body.

What about expanding this?

You can tell Kitt to do more, such as giving it an explicit id, colors to use, whether to use the automatic start and stop buttons, etc. Here is an example that turns off the buttons and programatically starts and stops the animation.

var evilkitt = new Kitt({id: 'evilkitt', animateBackgroundColor: 'red', hideButtons: true });
evilkitt.start();
setTimeout(function() { evilkitt.stop(); }, 6000);

With this simple little component at work, I can drop it in on the variety of performance testing work that I am doing more and more of these days (hopefully something to share one of these days).

Now, I need to make it use SoundManager2 and the sound of Kitt! ;)

For more information, check out the Google Code project that hosts the code.

Sometimes you can get in the zone just enough to be productive on a plane. On my flight to Mexico City yesterday, I created Endpoint a project that contains a server proxy, JavaScript client, and Greasemonkey Script with a mission. The mission is to take a URL, work out if it is a redirect (via a Location: header), and then return the final endpoint for it.

Why did I do this?

I was brainstorming functionality for a Twitter client with James Strachan (he is working on gtwit) and we talked about how annoying tinyurl / is.gd / snurl / you name it URLs are. They don’t tell you where you are going, and you could get Rick Rolled (if you are lucky) or much much worse.

So, I wanted to create a library, and one client (Greasemonkey) to test it out. Then anyone else could use it too to resolve directly from their Web pages.

How does it work

You load up the JavaScript via script src and then you can call resolve, passing the URL and a callback that will get the result. A few examples:

// Simple version
Endpoint.resolve('http://snurl.com/2luj3', function(url) { 
  alert(url); 
});
 
// Using the original URL to work out if it has changed
Endpoint.resolve(
  document.getElementById('testurl').value, 
  function(url, orig) { 
    alert(url); 
    alert(Endpoint.isRedirecting(url, orig));
  }
);
 
// How it is used in the Twitter Endpoint Resolver
Endpoint.resolve(url, function(resulturl, originalurl) {
  if (!Endpoint.isRedirecting(resulturl, originalurl)) return;
 
  newtext = newtext.replace(originalurl, resulturl, "g");
  jQuery(el).html(newtext);
});

Under the hood, a bunch of stuff is happening. I would love to be able to just use XMLHttpRequest to dynamically hit the URL and look at the headers, but the same-origin policy stops me.

This is why I have the server proxy, which returns a JSONP callback.

When you call resolve(url, callback) the script tag is created on the fly and added to the DOM. The callback function is all handled to allow multiple calls, and then the chain unravels.

Here you can see it all in action, showing how my Twitter stream will go through and the URLs will dynamically change from their tinyurl versions to whereyouaregoing.com:

I wanted to use App Engine to host the server proxy, but unfortunately I can’t work out how to do that yet. You have access to the URLFetch API to access resources from App Engine. Unfortunately for me, one of the features is that it understands redirects and just goes on through to the full resource itself, with no way to get the endpoint from the headers in the response.

It was also interesting to read Steve Gilmor talk about these services all be it in a post that is hard to actually understand ;)

Also, Simon Willison just put up a simple service on App Engine, json-time, that “exposes Python’s pytz timezone library over JSON.” I think that we will see a lot of these types of mini-Web services hosted on App Engine. Taking Python utility and making services from its goodness is an obvious choice.

I recently built an example of the Form History Pattern using an Addressbook case study.

I found myself talking about App Engine on the On Air tour, so I decided to change the example to not store the data locally with Gears, but instead to save it away into the cloud via App Engine.

Why did people want to hear about App Engine at an Adobe conference? I think that Jonathan Schwartz got it when he mentioned rich internet back-ends. As you build rich clients, you suddenly realise that the promised benefits of web services can kick into gear nicely.

Back to the sample. The architecture change was quite simple. Where I was doing a local DB save, I would call a back-end service such as /loadcontacts or /savecontact depending on the task.

You can view and download the full App Engine project code for this sample, and you can see it running live on App Engine

Check out the video below, or view the high quality version (recommended) to see the code walk through, and the various tools that App Engine gives you to develop, debug, and monitor your application running at a few thousand feet.

I am having a lot of fun with the AJAX Language API. Last week I talked about the translation bookmarklet that lets you translate anything that you select in the browser.

This time, I whipped up Twitter Translate, which watches the tweets in the page, and if the content isn’t your native tongue automatically converts it and replaces it inline. It then adds a mini logo with “translated from …” which you can click on to see the original text:

It is probably easiest to quickly see it in action:

I think that I am so excited about this API as it is a vertical service that you can just use for free. Think back on how you would be able to integrate language into your applications in the past. You would either:

Now, we can use this service for free, and the best part? It will keep getting better behind the scenes without us having to do a thing!

Every now and then, normally when talking to a libertarian, I think about how we are actually all connected to each other. It is impossible to sandbox yourself from society which leads me to conclude that I need to embrace it and do what I can to work out what kind of society we want to be.

With the current Ext JS debacle, you get reminded of how connected your project and business are to other people. Just because you own a company, doesn’t mean that you control it. When I think about my own company, Google, I realize that the most important currency is user trust. It doesn’t matter how many PhDs and great technology releases we have, if we ever lost some of the trust. I think that Google has earned its reputation, but all it would take is something that goes against what we have stood for so far, and we could lose it just as fast. I actually like this fact, as it keeps us honest.

It is a little like your tennis ranking. A rolling year of past performance is what really matters here. It doesn’t matter if you won that grand slam one year and one month ago. This is why every tournament matters. A bad showing loses points.

Of course, with user trust it is a lot more nuanced, and the graph is more exponential (the longer you go back in time, the less it matters).

Anyway, enough side tracking. When you have a software project that is a library for developers, your end users are those developers. If the project is open source, then there is a clear communication of the rules through your license. This is why open source licensing is so important. It allows you to have a simple contract saying “this is what you can and can’t do”. As a developer I can see GPL, BSD, Apache, and I know right away what kind of community this is, and how I can play a role. It isn’t about one license being better or worse than another. It is about communicating rights.

If you are fortunate enough to gain a real community, where other developers are participating, then the game starts to change. Now you have people who are invested in your project, maybe building on it for you, or evangelizing it, writing documentation, or creating their own business. At this point you really start to see what kind of project it is going to be, above and beyond the source code licensing. This is the Open Community side of a project. It can range from: only people who work for company X contribute in anyway, to: active commiters from all over the Web. This paints a picture of the project as a whole, and will have large effects on project, including who uses it. This is all about governance.

This also comes into play in other ways. When you think of Apache, or the Dojo foundation, you know about the legal protection that comes through the process. You know that everyone has signed a CLA, and that the history of the code is clean and well known. This has a huge effect on getting large companies into the game (This is why companies like IBM and Sun are so involved in Dojo IMO).

Now that you have users of various stripes, and a community with varied roles, you also have connections through out. If you then change the open source license for your project, the contract in the community has changed. When you make a change you not only need a good reason, but it has to be transparent, and you obviously have to get all of your ducks in a row to even be able to pull it off (e.g. depending on the change you may need every author of a line of code to get involved).

With Ext JS, there was a strange situation. The original license of LGPL-ish was very confusing, which lead to a confused community. Some kind o change was required, and clarity needed to be brought in. Unfortunately, it seems that the move to GPL has caused more chaos and confusion. Developers who poured a lot of time into the community (e.g. by creating GWT-Ext) are upset. The chaos can rip the community apart and you end up with a true lose-lose. Jack has spent far too much time and grey hairs on this one, instead of writing great code and growing his business.

So, it acts as a reminder, that the community is all connected. Everyone may not be equal, but make sure that communication is incredibly clear at all times to make sure that something like this doesn’t happen.

I really liked getting the Ajax Language API out into developers hands as god knows we shouldn’t have to worry about translations. Now we can use the API and have the Google back-end do all of the work.

I have recently had a couple of scenarios where I really wanted a quick translation. I had a few twitter messages pass through my stream in French and Spanish. I had the answer to some technical issues show up on foreign forums.

So, I decided to create a Translate bookmarklet that allows me to select any foreign text, click on the bookmark, and a little window pops up with the English translation if it can work it out. Automatic translation is far from perfect yet, but for many scenarios you can easily get the gist (e.g. you wouldn’t want to automatically convert a book).

This is how I created the bookmarklet:

The source

First, I have the raw JavaScript source that will become the bookmarklet. There are a few sections of the code. First, we setup a method that will go off and call the Ajax Language API, passing in the translation and language that we want. This is where you would change the language code for non-English.

if (!window['apiLoaded']) {
  window.apiLoaded = function() {
    var language = "en";
    var text = window.getSelection().toString();
    if (text) {
      google.load("language", "1", { "callback" : function() {
        google.language.detect(text, function(dresult) {
          if (!dresult.error && dresult.language) {
            google.language.translate(text, dresult.language, language, function(tresult) {
              if (tresult.translation) {
                translationWindow(tresult, dresult);
              } else {
                alert('No translation found for "' + text + '" guessing the language: ' + dresult.language);
              }
            });
          }
        });
      }});
    }
  };
}

Then we setup a method that is able to display a window showing the result. I used the Prototype UI Window object if available, and good old alert() if not:

if (!window['translationWindow']) {
  window.translationWindow = function(tresult, dresult) {
    if (window['UI']) {
      new UI.Window({theme:  "black_hud",
                   shadow: true, 
                   width:  350,
                   height: 100}).setContent("<div style='padding:6px'>" + tresult.translation + "</div>")
                   .setHeader("English Translation")
                   .setFooter("Language detected: " + dresult.language)
                   .center({top: 20}).show();
    } else {
      alert(tresult.translation + " [lang = " + dresult.language + "]");
    }
  }
}

Next, we load the Prototype UI window code, and accompanying CSS resources by dynamically adding the resources to the DOM:

if (!window['UI']) {
  var pw = document.createElement('script');
  pw.src = 'http://almaer.com/downloads/protowindow/protowin.js';
  pw.type = "text/javascript";
  document.getElementsByTagName('body')[0].appendChild(pw);
 
  var pwdefault = document.createElement('link');
  pwdefault.setAttribute('rel', 'stylesheet');
  pwdefault.setAttribute('type', 'text/css');
  pwdefault.setAttribute('href', 'http://almaer.com/downloads/protowindow/themes/window.css');
  document.getElementsByTagName('body')[0].appendChild(pwdefault);
 
  var pwblack = document.createElement('link');
  pwblack.setAttribute('rel', 'stylesheet');
  pwblack.setAttribute('type', 'text/css');
  pwblack.setAttribute('href', 'http://almaer.com/downloads/protowindow/themes/black_hud.css');
  document.getElementsByTagName('body')[0].appendChild(pwblack);
}

Finally, we load the Google API loader, and use the dynamic loading option with the ?callback=apiLoaded. This kicks off the main driver that we saw first, and if it is already loaded we call it directly (for multiple translations on the same page).

if (!window['google']) {
  var s = document.createElement('script');
  s.src = 'http://www.google.com/jsapi?callback=apiLoaded';
  s.type = "text/javascript";
  document.getElementsByTagName('body')[0].appendChild(s);
} else {
  apiLoaded();
};

“Compilation”

This is the raw form, and we need to get the bookmarklet form, which you can just use right away if you are wanting English. For this, I use John Grubber’s makebookmarklet Perl script to do the conversion.

The Server

The Prototype UI code lives on the server, so I put a striped down version over there which just contains a combined Prototype + Window JavaScript file, and just the one theme CSS set.

In Action

Unsure what I am talking about? Just watch it in action:

UPDATE: I also implemented Twitter Translate to automatically convert tweets to your language.