With a little help from with

You know the old saying:

with (JavaScript) {
    metaprogramming.is('possible');
}

I’m going to leave the discussion of what constitutes metaprogramming to another day (read: never), but what I will say is that I’m becoming more interested in DSLs and fluent interfaces. I want the code I write to work at a very high level, where it describes what I’m trying to achieve in terms that anyone else (read: me, six months later) can understand. I particularly like this example of a JavaScript DSL for describing state machines:

var Machine = FSM.build(function(fsm) { with (fsm) {
  onUnexpectedEvent(function() { ... });

  state('start', 'initial')
    .event('go')
      .goesTo('middle')
      .doing(function() { ... })
      .doing('phew')
    .event('run')
      .goesTo('finish')
    .onExiting(function() { ... });

  state('middle')
    .onUnexpectedEvent(function() { ... })
    .onEntering(function() { ... })
    .event('back')
      .goesTo('start')
      .onlyIf(function() { return true_or_false })
    .event('go')
      .goesTo('finish');

  state('finish', 'final');
}});

This example, along with various testing/speccing libraries, make heavy use of the much-maligned with statement. with essentially works as follows: inside the with block, all variable names are first looked up in the object passed to with, before being looked up in the parent scope. so:

var object = {
    name: 'James',
    getName: function() {
        return this.name;
    }
};

var theName, name;

with (object) {
    name = 'Bob';
    theName = getName();
}

// theName == "Bob"
// name == ""
// object.name == "Bob"
// object.theName === undefined

Now, clearly this can cause serious headaches if overused, and I would never recommend writing your business logic using with statements. But, for writing descriptions of things — like specs, or state machine definitions, or class definitions — it can be quite helpful. Using JS.Class, you can implement some of Ruby’s class definition syntax in 20 lines of code:

JS.Ruby = (function() {
  
  var extendDSL = function(builder, source) {
    for (var method in source) {
      if (!builder[method] && typeof source[method] == 'function')
        builder[method] = source[method].bind(source);
    }
  };
  
  var ClassBuilder = function(klass) {
    this.def  = klass.method('instanceMethod');
    this.self = {def: klass.method('classMethod')};
    
    extendDSL(this, klass);
    
    this.extend = function(source) {
      klass.extend(source);
      extendDSL(this, klass);
    };
  };
  
  return function(klass, define) {
    define(new ClassBuilder(klass));
  };
})();

That implements a DSL that, with a little help from with, lets you write JavaScript classes that look like Ruby ones, with all the power and flexibility that entails (let’s pretend you’re porting ActiveRecord to JavaScript):

ActiveRecord.Base = JS.Class();
JS.Ruby(ActiveRecord.Base, function(Ruby) { with(Ruby) {

  extend(ActiveRecord.Validations);
  
  validatesPresenceOf('name');
  validatesLengthOf('password', {minimum: 8});
  
  with (self) {
    def('create', function(attrs) {
      var base = new this();
      base.setAttributes(attrs || {});
      return base;
    });
  }
  
  def('initialize', function() {
    this.attributes = {};
  });
  
  def('setAttributes', function(attrs) {
    for (var key in attrs)
      this.attributes[key] = attrs[key];
  });
  
  var methodName = function(string) {
    return string.replace(/_(\w)/g, function(match, s) {
      return s.toUpperCase();
    });
  };
  
  // We're in a function, so any kind of logic goes
  ['name', 'email', 'password'].forEach(function(name) {
    def(methodName(name), function() {
      return this.attributes[name];
    });
    
    def(methodName('set_' + name), function(value) {
      this.attributes[name] = value;
    });
  });
}});

The whole class definition takes place inside a closure, so you can create any helper code you want in there and it won’t touch the global namespace. You have more control over the ordering of method definitions and mixin inclusions, and you can define methods dynamically to your heart’s content. The class behaves as you’d expect:

var me = ActiveRecord.Base.create({name: 'James'});
me.setAttributes({password: 'fizzbuzz'});
me.setEmail('james@jcoglan.com');
me.name() == 'James'  // true

The moral of this story is that with can be useful occasionally, especially when you want to describe or define something without repeating yourself too much. It’s useful for writing DSLs for doing such jobs, but don’t go sprinkling it around inside your actual methods. You’ll just get confused.

JS.Class 1.5 is now out

Hot on the heels of Ojay comes a new release of JS.Class, my Ruby-inspired JavaScript library for class-based OOP. Ojay is itself based on JS.Class, and has influenced the design of some of its new features. There has been one small change to the 1.0 API, and a stack of additional modules added in; it’s not really a 1.1 release or a 2.0 release, hence 1.5.

The one change to the 1.0 API is that _super is now called callSuper. Ojay uses a compression strategy that obfuscates ‘private’ variables (this will be in the next PackR release, as and when Packer 3.1 comes out). You do need to be very careful when doing this, though, so I figured I should make super not look like a private variable just in case.

Now, onto the new stuff. This release adds tons of new features to play with, namely:

  • Implementations of several design patterns in such a way that minimises how much repetitive boilerplate you need to write to use them. (I’m becoming more and more convinced that repetition is one of the largest and most easily removable causes of software bugs.) Patterns include decorator, virtual proxy, observer and command.
  • Versions of Ruby’s Comparable, Enumerable and Observable modules.
  • Linked lists. They’re like arrays, but much better suited to removal, shift operations and reordering.
  • MethodChain, formerly known as ChainCollector. This module on its own is responsible for some of Ojay’s key features.

So, what are you waiting for? Go and download it and tell me what you think.

Announcing Ojay, the nice way to use YUI

I’ve been wanting to talk about this project for weeks if not months, and now I finally can. the OTHER media (the web shop I work for) is open-sourcing Ojay, a project I’ve been developing on-and-off since I started at the company back in October. It’s a wrapper for the core DOM, event, animation and Ajax modules in YUI, providing a very concise, expressive API to what is, truth be told, a finely crafted library with a verbose, repetitive API.

The project started out as a replacement for DED|Chain, which we’d been using for a while but has not kept pace with the changes in YUI. When I started at this job, I’d been using Prototype for a year, and jQuery a little before that and was somewhat surprised at the amount of work I had to do to build anything with YUI. It does a sterling job of smoothing over browser inconsistencies and has some really nice features, they’re just not very nice to use. Ojay evolved out of my desire to get back the expressiveness I’d enjoyed with other libraries, while keeping access to YUI’s feature set.

To me, expressiveness is not just ‘syntactic sugar’, a tool to help you type fewer lines of code. It’s a core factor in how maintainable and well-designed your codebase is. We have a team made up of Java devs, a few Ruby enthusiasts and a couple of JavaScripters, and it’s crucially important to us that anyone can look at a piece of code and figure out what it does. Code should tell a story. Allow me to quote Reg Braithwaite:

One problem with the for loop is that it can only handle one loop at a time. We have to nest loops to work with two lists at once. This is patently wrong: there’s nothing inherently nested about what we’re trying to do. We can demonstrate this easily: try calling a colleague on the telephone and explaining what we want as succinctly as possible. Do you say “We want a loop inside a loop and inside of that an if, and…”?

No, we say, “We want to count the number of employees that have been with the company longer than their departments have existed.” There’s no discussion of nesting.

… The difference is more than just semantics, or counting characters, or the alleged pleasure of fooling around with closures. …

We have an example of the “Telephone Test:” when code very closely resembles how you would explain your solution over the telephone, we often say it is “very high level.” The usual case is that such code expresses a lot more what and a lot less how. The concern of what has been very clearly separated from the concern of how: you can’t even see the how if you don’t go looking for it. In general, we think this is a good thing.

And so it is with Ojay. Take this example from the documentation site:

$('a#run-this-code').on('click', $.stopEvent)
    .setContent('Running...')
    ._('#example').animate({
        height:     {to: 0},
        opacity:    {from: 1, to: 0}
    }, 0.5)
    ._($.HTTP).GET('/service/hello.html')
    .insertInto('#example')
    ._('#example').animate({
        height:     {to: 80},
        opacity:    {from: 0, to: 1}
    }, 0.7);

It says what you want to happen, with very little how. There are at least three pieces of asynchronous behaviour in this code: we want the code to fire when a certain link is clicked, then when the first animation is over we want to request some data from the server, then when the data arrives we want to insert it into the document and perform another animation. The usual way to handle asynchronous behaviour is using callback functions, but I’ve not used the word function but once in the above example. That would involve nesting things, and — hey, you guessed — there’s nothing inherently nested about what we’re trying to do. What we’re trying to do is carry out a list of instructions: wait for an event, then animate something, then make an Ajax request. And that’s exactly what the code looks like. I’m willing to bet that, with no knowledge of Ojay itself, most programmers with a passing familiarity with client-side scripting could guess what the above code does.

Update, in response to some comments on Ajaxian… The alternative way to write the above would be:

$('a#run-this-code').on('click', function(link, e) {
    e.stopEvent();
    link.setContent('Running...');
    var example = $('#example');
    example.animate({
        height:     {to: 0},
        opacity:    {from: 1, to: 0}
    }, 0.5, {
        after: function() {
            $.HTTP.GET('/service/hello.html', {}, {
                onSuccess: function(transport) {
                    example.setContent(transport.responseText);
                    example.animate({
                        height:     {to: 80},
                        opacity:    {from: 0, to: 1}
                    }, 0.7);
                }
            })
        }
    });
});

The point being that Ojay’s chaining mechanism handles asynchronous behaviour for you, rather than synchronously calling methods on the same base object. This approach means that, as well as being shorter and easier to follow, it’s much easier to change and debug: if you want to remove one of the asynchronous steps in the sequence, you only need to modify code in one place, rather than two or more as you would in the second example. (End update)

This is exactly the sort of stuff I’ve been alluding to when talking about ChainCollector. (It’s now called MethodChain, and will be part of the next release of JS.Class, which Ojay itself is based on.) Ojay uses this technique extensively to let you handle asynchronous coding as if it were synchronous, so that your code more clearly expresses what you’re trying to do. One particularly expressive part of the library is its HTML module:

Ojay.HTML.ul({id: 'list'}, function(HTML) {
    HTML.li('Item one');
    HTML.li('Item two');
    ['Cats', 'Dogs', 'Pets'].forEach(HTML.method('li'));
})

// -> <ul id="list">
          <li>Item one</li>
          <li>Item two</li>
          <li>Cats</li>
          <li>Dogs</li>
          <li>Pets</li>
      </ul>

Yes, people have done this sort of thing before, but I knew it could be made even cleaner. Most of Ojay is essentially an attempt to push JavaScript as far as I can, to make it get out of the way and let me say what I mean.

One other major area of frustration we’ve paved over with Ojay is YAHOO.util.History. It’s a great tool, but it encourages bad design, or at least makes good design fairly tricky. Ojay includes a wrapper for YAHOO.util.History that distills it down to one function call:

$.History.manage(myObject, 'itsName');

myObject has to implement a couple of special methods for this to work, but it’s not that hard and it will improve the design of your code: the concern of having an object know its state has been separated from the concern of getting the browser to remember that state. Your object will work with or without history management — it can just be bolted on as and when you feel like it.

That’s really all I wanted to say about it for now. Chances are I’ll be demonstrating some example code here, and digging into Ojay’s innards a little along the way. For now, I’d be thrilled if you’d download Ojay and give it a whirl. Our code is hosted on Google Code, and we have a mailing list should you want to get in touch.

How to help your users out as a library designer

Seems my functional posts have been generating an obscene amount of traffic lately (thank you, Reg). In which spirit, here’s a little suggestion for anyone designing a JavaScript library.

I was reading about Oliver Steele’s MIT-licensed Functional library earlier, and came across this little gem (modified slightly to my whims):

/**
 * Returns a function identical to this function except that
 * it prints its arguments on entry and its return value on exit.
 * This is useful for debugging function-level programs.
 */
Function.prototype.traced = function(name, logger) {
    var self = this, name = name || this, logger = logger || 'info';
    return function() {
        window.console && console[logger](name,
                'called on', this, 'with', arguments);
        var result = self.apply(this, arguments);
        window.console && console[logger](name, '->', result);
        return result;
    }
}

To which I say, never mind function-level programs (though there’s a voice in my ear whispering something about learning Haskell). It’s useful for debugging just about anything. There’s nothing stopping you using this with procedural code or class-based OO programs.

var jibber = {
    age: 68,
    getAge: function(n) {
        return "It's, er... " + (this.age/n) + ". I think.";
    }.traced('getAge()')
};

jibber.getAge(2);

// console prints...
// getAge() called on Object age=68 with [2]
// getAge() -> It's, er... 34. I think.

So we see that we can use it to log method calls, not just bare functions. And it’s not even just useful for debugging. As a library author, you could use it to log interesting points in your codebase, or to highlight upcoming changes to an API in a future release:

MyClass.prototype.jibber = function() {
    return this.wobble.apply(this, arguments);
}.traced('DEPRECATED: use wobble() instead. jibber()', 'warn');

So, before releasing version 2.0 of your library, you could put out a 1.5.x release that just added a bunch of deprecation messages to make it easier for your users to upgrade from 1.5 to 2.0.

Let’s get committed

Right, I know I said I’d do this a long time ago, but I just got round to checking up on some of my Rails patches with the hopes of persuading one or two of you that they might be worth supporting and getting committed to the core. This post concerns a couple of patches that, a couple of days ago, I updated to make sure they will apply against the current head version of Rails.

The first is one which totally saved my life a while ago while building DesalData. I had to allow users to graph data of water plants going back decades, in real time in their browsers. On a cheap shared server with one database. I figured that the best way to do this was to ask MySQL for all the required SUMs and MAXes in one query, but Rails only lets you make one calculation per query using ActiveRecord::Base.calculate. So, I set about patching it to allow multiple calculations at once, and ended up with client code that looked like this:

Person.calculate(:how_many => [:count, '*'],
    :total_age => [:sum, :age])
#=> {:how_many => 12, :total_age => 387}

Person.calculate({:how_many => [:count, '*'],
    :total_age => [:sum, :age]},
    :conditions => ['age < ?', 30])
#=> {:how_many => 7, :total_age => 94}

(These taken from my Rails ticket, not the app itself!) Needless to say, what with complaints from all sides about Rails performing badly and not being ‘enterprise-ready’ (whatever that means), I figured this would be a useful addition. So far a couple of people seem to agree with me, but I need a couple more votes of support to verify the patch and get it committed. I’ve tested it with MySQL 5 and SQLite 3, so if you feel like testing it yourself with some other DB and making sure it works, head over to the ticket and show me some love.

A related (and overlapping) ticket involves the addition of a :distinct_records option to ActiveRecord::Base.find. (See the ticket.) I’ll quote from the ticket rather than trying to explain it all again:

Let’s say you want to find the sum of the prices of a bunch of products in your database, and the criteria used for selecting products involve using :include to bring in associated tables. Those LEFT OUTER JOIN statements may result in the base record (Product) being duplicated for every association found. e.g. if product number 1 has three comments associated with it, then Product.sum(:price, :include => :comments, :conditions => ...) will include product number 1 in three rows of the SQL result set. I only want its price added to the sum once, but I can’t use the :distinct option because the prices I’m adding up aren’t necessarily unique.

This really caused me a huge headache with filtering data correctly, and these two patches combined are the result of days and days of banging my head against the desk trying to get my app to work properly. I’d be thrilled if you’d go and support this patch.

That is all, for now. I’d also be chuffed if the nested has_many :through patch gets committed, so go vote for it too.