The If Works This dirt was a building before

Events: they’re not just for the DOM, you know

In Evented, Functional, JS.Class, JavaScript, Ojay on 21 February 2010 with 4 comments

This post is part of a series on event-driven programming. The complete series is:

Over recent months we’ve seen the major JavaScript libraries talking up their event support. Back in October Luke Smith spoke about the YUI3 event system, and more recently Yehuda Katz gave a screencast on evented programming with jQuery. Dan Webb took him to task for over-zealous use of data-* attributes, arguing that the DOM is not the place to model your domain. I have a similar but more broad problem with the screencast, which is that I think the DOM is not the place for your application’s event model.

Yehuda’s talk runs through an example of creating a tabs widget using jQuery. The assertion throughout is: wouldn’t it be nice if you could treat the events in the UI widget – events like changing the visible pane, clicking a tab etc. – just like you treat DOM events. This is true up to a point, in that having a consistent event API is nice, but it fails by trying to overload the DOM with too much responsibility and creates a leaky abstraction in the process.

The initial proposition is: imagine if HTML natively supported tabs. What would that look like? One suggestion is:

<tabs>
  <tab pane="first">First</tab>
  <tab pane="second" selected="selected">
    Second
  </tab>
  <tab pane="third">Third</tab>
</tabs>

<pane name="first">Some content</pane>
<pane name="second">Some content</pane>
<pane name="third">Some content</pane>

<script type="text/javascript">
  $("tabs").bind("change", function() {
    $(this).attr("panes").hide();
    $(this).attr("selected").show();
  });
</script>

Now, the above elements don’t exist in HTML so instead it’s proposed that we get as close as we can in real HTML like so:

<ul class="tabs">
  <li data-pane="first">First</li>
  <li data-pane="second" class="selected">
    Second
  </li>
  <li data-pane="third">Third</li>
</ul>

<div class="pane" id="first">Some content</div>
<div class="pane" id="second">Some content</div>
<div class="pane" id="third">Some content</div>

<script type="text/javascript">
  // When tab selection changes, show all
  // the panes then hide the selected one
  $("ul.tabs").bind("change", function() {
    $(this).attr("panes").hide();
    $(this).attr("selected").show();
  });
</script>

This won’t work out of the box so some plumbing is needed to make ul elements support change events and tab-related attributes:

// Trigger 'change' when 'click' fires
$("ul.tabs").click(function() {
  $(this).trigger("change");
});

// On change, set the selected pane and
// toggle some class names
$("ul.tabs").bind("change", function(e) {
  $(this).attr("selected", e.target);

  $(e.target)
    .addClass("selected")
    .siblings()
    .removeClass("selected");
});

$(document).ready(function() {
  // Set the 'panes' attribute by getting the divs
  // referenced by the li 'data-pane' attributes
  $("ul.tabs").attr("panes", function() {
    return $("li", this).map(function() {
      return $("#" + $(this).attr("data-pane"));
    });
  });

  $("ul.tabs").trigger("change");
});

Don’t try this code out; even after correcting some of the more obvious typos from the screencast I still can’t get it to work. There are missing attributes (e.g. the selected attribute is never set), and in any case the DOM stringifies anything you set as an attribute so the expression $(this).attr("panes") just returns "[object Object]" rather than a list of DOM nodes.

But the thing I really want to focus on here is evented programming, and how in JavaScript there’s this unwritten assumption that if you need events, you need the DOM. In fact Yehuda nails it with this quote:

The nice thing about the browser code that we’re implementing is that it’s very familiar to you. You know how to do $("ul.tabs").click(), you know how to bind to change and add classes and get siblings. This is code that you’re already familiar with.

This reminds me of the classic quote, “When your hammer is C++, everything begins to look like a thumb.” Just because you know how one thing works, that doesn’t make it a good model for every programming task. Think about what an event is for a second. An event is just a point in time at which some change takes place. Yes, the DOM fires events all the time as changes are made to it by scripts and user interaction; these events are what power GUIs on the web. But not every event in your application needs to live in the DOM.

Let’s take a concrete example. Suppose part of your client-side app is a module that integrates your application with Facebook Connect. Your requirements list includes such items as:

The spirit of Yehuda’s talk is right in that we should translate this into real code that’s close to how we wrote the requirements (by the way, TDD is a great way to get into this habbit):

$("a.facebook-connect").click(function() {
  Facebook.loginAndRedirect("/");
  return false;
});

Facebook.on("login", function() {
  KM.record("Login with Facebook");
});

Facebook.on("cancel", function() {
  KM.record("Cancel Facebook login");
});

When we get around to implementing our Facebook module, how should we support these events? The first response of most JavaScripters seems to be, “Use the DOM.” But what if your events have nothing to do with the DOM? This module is a wrapper around what is essentially a web service that provides authentication and data APIs. If you had to pick a DOM node to fire these events, which would you pick? Would the fact that DOM events bubble and can be cancelled cause you problems? Do these concepts even make sense for the task at hand? My experience is that for most application-level events, they do not.

The way I like to think of the DOM is as an output device, and nothing more. I model my UI widgets and their events in pure JavaScript, providing API methods for everything the widget can do and all the events it supports. Internally, the widget takes care of hacking the DOM in whatever ways are needed to get the right look and feel. This has several benefits:

Part of Yehuda’s argument is that an object-oriented approach to GUIs doesn’t work, that writing code like

Tabs = new Class({
  initialize: function(node) {
    this.node = node;
  },
  select: function(tab) {
    // perform selection
    // behavior
  }
});

is bad because method calls assume synchronous rather than event-driven behaviour. This does not have to be the case, and as I outlined above objects are a great way to package up components and give them scripting APIs for easy integration. With that in mind, I’m going to spend this week on a series of articles on event-driven programming and common patterns for combining it with objects. The first will be on implementing events without any help from the DOM – see you tomorrow.

Faye: a Comet client and server for Node.js and Rack

In Evented, Faye, Functional, JavaScript, Node, Ruby on 2 February 2010 with 12 comments

I’m doing my traditional birthday software announcement a little early this year, mostly because I really want to get this out and partly because I’m doing a lot of little bits of work on old projects at the moment and this is the only fancy new thing I’ve got to show.

Spurred on by the sheer volume of awesome emanating from Music Hack Day over the weekend – a few Songkickers went over, I caught the demos via the live stream – I decided to revisit a project I worked on just before the London MHD about six months ago. My hack was this little web service that exposed your iTunes library over HTTP and let you broadcast what you were listening to to other browsers via Comet. The Comet part of it was handled by an early version of Faye, a Ruby server and JavaScript client I cooked up to provide client-to-client messaging.

The Ruby version still exists, and just gained support for Thin’s async response feature, making it totally non-blocking when used behind Thin. The more exciting news is that I’ve also ported the backend to Node.js, a JavaScript environment optimised for non-blocking asynchronous I/O. This makes it much better at serving highly concurrent Comet requests since it uses an event loop rather than blocking threads to respond to each request. It’s a straight port of the Ruby version, which is entirely event-driven except for the web server interface, since Rack expects a synchronous return to respond to requests.

Anyway, onto the code. The backends are trivial to set up, and both currently reside in a single process and keep all channel data in memory. This means you can’t use it behind Passenger (which spawns multiple Ruby processes to serve an app) but I’m hoping some bright spark can help with scaling it. Consider it a fun and reasonably nippy toy right now, I guess. To set up a backend:

// JavaScript
var faye = require('./faye');
var comet = new faye.NodeAdapter({mount: '/comet'});

# Ruby
require 'faye'
comet = Faye::RackAdapter.new(:mount => '/comet')

More complete usage examples are available in the Git repo. Both these backends provide an endpoint at http://0.0.0.0/comet that speaks the Bayeux protocol, and clients can use this to route messages between each other.

Setting up the client is a snap:

<script type="text/javascript" src="/comet.js"></script>
<script type="text/javascript">
  CometClient = new Faye.Client('/comet');
  CometClient.connect();
</script>

Each client (i.e. a JavaScript program running in a browser) can subscribe to named channels and publish arbitrary JavaScript objects to channels. Each message is distributed by the server to all the clients subscribed to that channel, and clients use a callback to handle messages from the channel.

// I can publish a message ...
CometClient.publish('/my/channel', {msg: 'Hello world!'});

// .. and you can pick it up
CometClient.subscribe('/my/channel', function(message) {
  alert(message.msg);
});

Using this it’s absolutely trivial to make simple client-to-client messaging apps – the repo contains the obligatory chat demo, which I’ve modelled on Twitter to illustrate different uses of subscriptions. You don’t need any server-side code for this aside from starting the Faye backend up to handle message routing. The next step would be to provide a client interface on the server side so your backend code can send data out to clients, but we’ll save that for a future release. Contributions in the form of scaling advice and server-side client suggestions would be very much appreciated, in the meantime go have a play with the code.

Adding a dynamic defmacro to Heist

In Functional, Heist, Lisp, Metaprogramming, Ruby on 20 January 2010 with no comments

I’ve just picked up the opening chapters of Let Over Lambda, which describes itself as a book on macro programming – particularly Common Lisp macro programming. One of the early macros given in the book is unit-of-time which looks like this:

(defmacro unit-of-time (value unit)
  `(* ,value
      ,(case unit
         ((s) 1)
         ((m) 60)
         ((h) 3600)
         ((d) 86400))))

In Common Lisp (in so far as I’ve been able to gather in one evening), macros are expanded in an imperative fashion using the built-in CL list manipulation functions. That is, if I call

(unit-of-time 5 m)

CL will bind the integer 5 to parameter value, and the symbol m to unit, then evaluate the list expression inside the macro, in this case giving (* 5 60). That is to say, (unit-of-time 5 m) is replaced with (* 5 60), and the latter is then evaluated at runtime to give the final result. Within a defmacro definition you can use the built-in CL functions – in this case, quasiquotation and the case form – to manipulate source code passed into the macro and transform it into something else. You’re directly manipulating source code by using defmacro.

Scheme’s syntax-rules macro system takes a more declarative approach to macros and provides a pattern-matching language for transforming source expressions into lower-level function calls. A Scheme implementation of the above might look like this:

(define-syntax unit-of-time (syntax-rules ()
  ((unit-of-time value unit)
   (* value (case 'unit
              ((s) 1)
              ((m) 60)
              ((h) 3600)
              ((d) 86400))))))

Superficially this looks almost the same, but there’s an important difference lurking beneath the surface. Whereas the CL macro will expand (unit-of-time 5 m) as (* 5 60), the Scheme version will expand it as:

(* 5 (case 'm
       ((s) 1)
       ((m) 60)
       ((h) 3600)
       ((d) 86400)))

That is, Scheme slots the macro parameters into a template, which is used to replace the original expression. You don’t manipulate the source yourself, you use this templating system to declare transformation patterns. Now a smart compiler might spot that that case expression evaluates to a constant and optimise it, but suppose we’re using a toy interpreter that we’re rather fond of but isn’t all that clever. We want to make Scheme inline the conversion factor. We’d need to write the macro like this:

(define-syntax unit-of-time (syntax-rules (s m h d)
  ((unit-of-time value s) value)
  ((unit-of-time value m) (* value 60))
  ((unit-of-time value h) (* value 3600))
  ((unit-of-time value d) (* value 86400))))

This way, Scheme will pick one of the matching patterns and use the corresponding template with the factor hard-coded. The s case even avoids an unnecessary multiplication.

Still it’s clear that CL is better suited to some types of macro writing: it gives you full access to the source code as a Lisp data structure that you can manipulate with Lisp’s list processing library. Scheme only allows certain rigid transformations using its pattern language, making certain things awkward to express. However, Common Lisp’s power is blunted slightly by most Lisps’ distinction between compile time and run time. Your code is first compiled, one stage of which is expansion of all macros, and once this is finished the expanded source is executed. This means that a defmacro can only use functions that are built into the CL environment. User-defined functions are not available until run time and so cannot be used to manipulate source code.

Heist, being the naïve so-and-so that it is, only has run time. Macro calls are expanded as they are encountered at run time, their expansions being inlined into the source tree once processed. Macros, special forms, built-in and user-defined functions all have first-class status, and are implemented as different types of Function that we call as we evaluate code. This means, if you were to add defmacro to Heist, it would be called at run time and therefore have access to any user-defined function.

So let’s go and implement it. The latest Heist update (0.3.2) added the ability to easily load Ruby files from Scheme land, meaning I can write a Ruby file called defmacro.rb and load it in Scheme using (load "defmacro"). In your Ruby files you can use define and syntax to add functions to the Scheme environment.

To begin with, we’re going to need a new class to model CL-style macros. It should inherit from the Heist Function class so it receives all the references it needs from the interpreter; we just need to override the call method to implement defmacro behaviour.

class CommonLispMacro < Heist::Runtime::Function
  def call(context, cells)
    scope = Heist::Runtime::Scope.new(@scope)
    cells.to_a.each_with_index do |cell, i|
      scope[@formals[i]] = cell
    end
    expression = @body.map { |e| Heist.evaluate(e, scope) }.last
    Expansion.new(expression)
  end
end

A quick walk-through: Heist will call this type of function with context (an object representing the scope the function call is made from) and cells which is a Lisp list of the unevaluated argument expressions to the function. Heist Function objects have a @scope, which is the scope they are defined in, an array of the names of their arguments in @formals and a Lisp list of the expressions they contain in @body. We begin by making a new Scope in which to evaluate the function’s innards. Then we take the unevaluated argument expressions and assign them in the new scope using the variable names from @formals. We evaluate all the expressions in @body and return the last one, wrapped in an Expansion. The @body should return a list structure that could be evaluated as Scheme source code.

What’s this Expansion object? Well, we don’t want the expanded expression to be the macro call’s return value. We want Heist to insert this new expression into the source code, replacing the old expression, and then evaluate it to get the final value. To make this happen, we need to return an object of type Macro::Expansion, and this object should have an expression method. Heist takes care of all the rewriting and further evaluation as long as your function returns one of these. Our Expansion class is just a dumb wrapper around an expression and doesn’t need to do a lot of the expansion work that Heist’s macro system usually does.

class CommonLispMacro
  class Expansion < Heist::Runtime::Macro::Expansion
    attr_reader :expression
    def initialize(expression)
      @expression = expression
    end
  end
end

With these defined, all that’s left is to provide a front-end for making these functions from Scheme. In Heist, a Function is constructed using the current scope, a list of argument names and the function body.

syntax('defmacro') do |scope, cells|
  scope[cells.car] = CommonLispMacro.new(scope, cells.cdr.car, cells.cdr.cdr)
end

In the expression (defmacro foo (one two) (* one two)), cells.car refers to foo, cells.cdr.car to (one two) and cells.cdr.cdr to ((* one two)) – remember the body can contain more than one expression. Now you can drop the above Ruby in defmacro.rb, boot up Heist and use CL-style macros:

(load "defmacro")

(defmacro unit-of-time (value unit)
  `(* ,value
      ,(case unit
             ((s) 1)
             ((m) 60)
             ((h) 3600)
             ((d) 86400))))

(unit-of-time 5 s)
; => 5

(unit-of-time 5 m)
; => 300

Cross-process metaprogramming on the cheap

In JavaScript, Metaprogramming, Rails, Ruby, Testing on 14 January 2010 with 1 comment

I will preface my first post of the new decade by saying: this is not by any means elegant. It’s an egregious hack, but it may come in handy for those of you using Culerity for testing your Rails front-end using JavaScript. This is not so much about JavaScript as about dealing with the multitude of processes involved in the testing setup.

For those of you unfamiliar with it, Culerity is a Ruby gem that lets you use Celerity within Cucumber. What’s Celerity? It’s a JRuby wrapper for HtmlUnit, which is a web browser simulator written in Java. HtmlUnit can load web pages and run JavaScript just like a real browser, except it’s “headless”, meaning it has no GUI.

Installing all these tools is a breeze (especially now that JRuby is in the Ubuntu repos) but what’s going on behind the scenes is fairly tricky:

Now recently I’ve been working on Acceptance, which is a JavaScript validation API that comes with a Rails plugin that generates client-side validation code from ActiveRecord validations. I want its integration tests to look something like:

  Scenario: Leaving a required checkbox unchecked
    Given the Account class validates acceptance of terms
    When I visit "/accounts/new"
    And I press "Submit"
    Then I should see "Terms must be accepted"

The item I really want to concentrate on is the first step. When your app and your tests are running the same Ruby process, you might be able to implement the step as follows:

Given /^the Account class validates acceptance of (\S+)$/ do |field|
  ::Account = Class.new(ActiveRecord::Base) do
    validates_acceptance_of field
  end
end

But under Culerity, your app and your tests run in separate processes. This means your dynamic creation of the Account class in process B won’t be visible to the application running in process A, so you cannot script the behaviour of the app like this. You’re going to have to do something sneaky.

One thing you can do is “send” code from your tests to your app by storing it in some globally visible resource, such as the filesystem. For more advanced uses you may want to use proper inter-process messaging but for our purposes dumping code in the filesystem will suffice (I told you this was on the cheap). Let’s change our step definition to write the validation code to a file:

# features/step_definitions/validation_steps.rb

require 'fileutils'
require 'find'

VALIDATION_CONFIG = File.dirname(__FILE__) + '/../../config/validation/'
FileUtils.mkdir_p(VALIDATION_CONFIG)

Given /^the Account class validates acceptance of (\S+)$/ do |field|
  File.open(VALIDATION_CONFIG + 'account.rb', 'w') do |f|
    f.write < <-CODE
    class Account
      validates_acceptance_of :#{field}
    end
    CODE
  end
end

The other half of this trick relies on the fact that Culerity runs the app in process A in development mode, so all the models will get reloaded on every request. This means we can put some code in app/models/account.rb to dynamically load the validation code we’ve dumped in the filesystem:

class Account < ActiveRecord::Base
end

file = File.dirname(__FILE__) + '/../../config/validation/account.rb'
load file if File.file?(file)

And to stop state leaking into other tests, we need to clean up after ourselves in our step definitions. We need to remove any validation files that may have been created during each test, so that next time the models are reloaded they won’t pick up any extra validations:

# features/step_definitions/validation_steps.rb

After { Given "I remove all validations" }

Given /^I remove all validations$/ do
  Find.find(VALIDATION_CONFIG) do |path|
    File.delete(path) if File.file?(path)
  end
end

And that’s just about enough setup to let you inject code into a running Rails app while you’re testing it. Maybe one day Rails validations will be modelled in such a way that you can reflect on them and add/remove them at runtime, but for now this will have to do.

Load everything asynchronously

In Functional, Helium, JS.Class, JavaScript, Metaprogramming on 18 December 2009 with no comments

A couple weeks ago there was rather a lot of excitement over the fact that Google released a new Analytics snippet that loads the tracking library asynchronously. This is indeed great news, for reasons pored over in the aforelinked articles. But let’s take a closer look at Google’s implementation:

  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-XXXXX-X']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script');
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' :
        'http://www') + '.google-analytics.com/ga.js';
    ga.setAttribute('async', 'true');
    document.documentElement.firstChild.appendChild(ga);
  })();

For simple use cases this is fine, barring some quibbles over Google’s obfuscatory naming scheme for Analytics’ global public API objects and their bizarre use of underscored methods throughout. But say you have a page where you’re doing a lot of custom event tracking. The various event handlers in your page should be completely independent of each other, but this snippet introduces some complications. Each handler will need to figure out whether Analytics is loaded: do I call something synchronously using _gat and pageTracker, or do I push items onto _gaq? Analytics makes this slightly friendlier by replacing _gaq on load with an object whose push() method executes the given command immediately, but we’re still left with the problem of global state. Since we want all our event handlers to be independent and async-safe, each one will have to check whether we have a global _gaq object already and make one if we don’t:

  $('a').on('click', function() {
    window._gaq = window._gaq || [];
    _gaq.push(['_trackPageView', '/path/to/virtual/url']);
  });

This however is very brittle since it assumes that if Analytics is not yet loaded then it will be at some point, at which juncture your tracking command will be executed. To fix this properly you’d need to have a function that could wrap code in a block that checked whether Analytics was loaded before running the code, and that made sure Analytics was only loaded once. Something like this:

  writeToAnalytics = (function() {
    var loaded = false;

    var ensureLoaded = function() {
      if (loaded) return;
      loaded = true;
      var ga = document.createElement('script');
      ga.src = ('https:' == document.location.protocol ? 'https://ssl' :
          'http://www') + '.google-analytics.com/ga.js';
      ga.setAttribute('async', 'true');
      document.documentElement.firstChild.appendChild(ga);
    };

    return function(command) {
      window._gaq = window._gaq || [];
      _gaq.push(command);
      ensureLoaded();
    };
  })();

  $('a').on('click', function() {
    writeToAnalytics(['_trackPageView', '/path/to/virtual/url']);
  });

This may not seem like a terrible burden but it illustrates a larger point: having many independent events rely on global state or execution order does not scale to large codebases as it becomes increasingly hard to manage dependencies between functions that ought to be completely decoupled.

Thankfully, since its 2.1 release back in June JS.Class has allowed us to load just about anything we like asynchronously, including Analytics (Helium even ships with functions for this). Let’s have a look at the old blocking Analytics loader and figure out how to make it async:

  <script type="text/javascript">
  var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
  document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
  </script>
  <script type="text/javascript">
  try {
  var pageTracker = _gat._getTracker("UA-XXXXXX-X");
  pageTracker._trackPageview();
  } catch(err) {}</script>

We can use JS.Packages to write a loader function for this that will let us load Analytics on demand without using any global state checks. First, notice that the first script picks a domain to load Analytics from, then writes a script tag with the appropriate URL. The loaded script contains the Analytics _gat object. We can encapsulate this as:

  JS.Packages(function() { with(this) {

      var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
      file(gaJsHost + 'google-analytics.com/ga.js')
          .provides('_gat');
  }});

We can then add a custom loader function that uses our Analytics ID to make a pageTracker object, as done by Google’s second script. Our completed loader code looks like this:

  JS.Packages(function() { with(this) {

      var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
      file(gaJsHost + 'google-analytics.com/ga.js')
          .provides('_gat');

      loader(function(cb) {
          try {
              window.pageTracker = _gat._getTracker("UA-XXXXXX-X");
          } catch(err) {}
          cb();
      })  .provides('pageTracker')
          .requires('_gat');
  }});

We need to call cb() when our custom loader is done loading its objects to tell JS.Packages the loader has completed. Also note how we assign to window.pageTracker to make the object global, and we don’t call _trackPageview() here since our only concern is loading the object.

Now that JS.Packages knows how to load pageTracker we can just require() it wherever we need it, and JS.Packages will make sure it is only loaded if needed, and that it is only loaded once. Our event handler then becomes:

  $('a').on('click', function() {
    require('pageTracker', function() {
      pageTracker._trackPageView('/path/to/virtual/url');
    });
  });

You could even tidy this up using some higher-order functional programming:

  requiring = function() {
    var args = arguments;
    return function() { require.apply(this, args) };
  };

  $('a').on('click', requiring('pageTracker', function() {
    pageTracker._trackPageView('/path/to/virtual/url');
  }));

We’ve eliminated all knowledge about how the Analytics library gets loaded from our event handler and put the loading logic in one place: the JS.Packages configuration. Hopefully you can see that this scales to more complex codebases better than Google’s default snippets, and you can see how to write async loaders for any library you choose.

One final remark; notice how we were able to use JS.Packages to transform the legacy Analytics loader into an asynchronous loader without modifying our use of the Analytics API (aside from wrapping it in require() blocks). This strategy removes the need for the _gaq command queue since JS.Packages makes sure an object is loaded before letting you use it. I’m seeing far too many JavaScript libraries being tweaked to accommodate async loading, requiring developers to use a queue or hook into some custom event to run their code. We can have async loading without modifying our libraries, and we can do it in a way that is almost transparent to application code such that it scales very nicely with the size of your codebase. This is what JS.Packages provides, and I’d much rather we focus on a general-purpose package loader than have every framework out there solve the problem in its own idiosynchratic way.

Testing command-line apps with Cucumber

In Ruby, Testing on 14 November 2009 with no comments

I recently wrote a tiny little tool called Claw to help me work on large codebases in gEdit. It provides a terminal that lets you search for files by name and content using very minimal syntax, and it numbers the search results so you can just type a number to open the file using any shell command. In the last few days I converted its tests from Test::Unit to Cucumber and added a lot more coverage, and found Cucumber to be very nice for testing command-line UIs.

Test::Unit and RSpec work fine when you’re just calling methods and testing their return values, stuff like assert [:foo, :bar].include?(:foo) or "ruby".upcase.should == "RUBY". The thing about command-line apps is that they mostly output their results as side effects, such as writing to standard out, modifying files, making network calls etc. Testing these things often involves stubs and mock objects and these can make your tests harder to follow, but Cucumber lets you hide all that behind easy-to-read prose.

To illustrate, let’s write a little app ourselves using TDD. Let’s make a few directories:

mkdir tdd-terminal
cd tdd-terminal
mkdir lib
mkdir -p features/step_definitions
mkdir -p features/support

Let’s write a spec for our program. This is a stripped down version of the Claw spec that contains enough details to illustrate how to test command-line apps: we’ve got some visible output to verify, and a side effect of running the program: it should open files for us when we issue certain commands.

# features/search_for_files.feature

Feature: Search for files

  Background:
    Given I start the app with "-c gedit"

  Scenario: Find files by name
    Given I enter "foo"
    Then I should see
    """
    1. foo.txt
    """

  Scenario: Open a matching file
    Given I enter "foo"
    And I enter "1"
    Then "foo.txt" should be open in "gedit"

Let’s run the feature using Cucumber:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/search_for_files.feature:4

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/search_for_files.feature:7
    Then I should see                     # features/search_for_files.feature:8
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/search_for_files.feature:14
    And I enter "1"                          # features/search_for_files.feature:15
    Then "foo.txt" should be open in "gedit" # features/search_for_files.feature:16

2 scenarios (2 undefined)
7 steps (7 undefined)
0m0.041s

You can implement step definitions for undefined steps with these snippets:

Given /^I start the app with "([^\"]*)"$/ do |arg1|
  pending
end

Given /^I enter "([^\"]*)"$/ do |arg1|
  pending
end

Then /^I should see$/ do |string|
  pending
end

Then /^"([^\"]*)" should be open in "([^\"]*)"$/ do |arg1, arg2|
  pending
end

Take the generated step definitions and copy them into features/step_definitions/terminal_steps.rb. Re-running the tests you should see the steps marked as pending or skipped rather than undefined. Let’s implement the first step, the one that starts our app.

Given /^I start the app with "([^\"]*)"$/ do |command|
  @io  = StringIO.new
  @app = Terminal.new(command.split(/\s+/), @io)
end

Here we take the command-line argument string and split it up to mimic what we’d expect to receive in ARGV when running the app for real. We also make an IO object and give it to the app: instead of calling Kernel#puts our app will call puts on whatever object we pass in. In production this would be Kernel but for testing we want an object that we can read from so we can check what the app prints. Run the tests again:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1
      uninitialized constant Terminal (NameError)
      ./features/step_definitions/terminal_steps.rb:3:in `/^I start the app with "([^\"]*)"$/'
      features/search_for_files.feature:4:in `Given I start the app with "-c gedit"'

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:6
    Then I should see                     # features/step_definitions/terminal_steps.rb:10
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:6
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:6
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:14

Failing Scenarios:
cucumber features/search_for_files.feature:6 # Scenario: Find files by name

2 scenarios (1 failed, 1 skipped)
7 steps (1 failed, 6 skipped)

We need to go and implement our Terminal class to get the first step working. First, enter the following in features/support/env.rb:

# features/support/env.rb

require File.dirname(__FILE__) + '/../../lib/terminal'
require 'spec/mocks'

This tells Cucumber to load our application and RSpec’s mocking library before running the tests. We need to give it something to load, so let’s put the following in lib/terminal.rb:

# lib/terminal.rb

require 'oyster'

class Terminal
  BIN_SPEC = Oyster.spec do
    string :command
  end

  def initialize(argv, io)
    @options = BIN_SPEC.parse(argv)
    @stdout  = io
  end
end

The constructor takes an array of command-line input (should be ARGV in production) and an output device to puts to. I’m using my Oyster gem to parse the input into an options hash, and storing a reference to the output device for later use. This should be enough code to turn our first step green:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:6
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:7:in `/^I enter "([^\"]*)"$/'
      features/search_for_files.feature:7:in `Given I enter "foo"'
    Then I should see                     # features/step_definitions/terminal_steps.rb:10
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:6
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:7:in `/^I enter "([^\"]*)"$/'
      features/search_for_files.feature:14:in `Given I enter "foo"'
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:6
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:14

2 scenarios (2 pending)
7 steps (3 skipped, 2 pending, 2 passed)

Let’s implement the next step, “Given I enter”. This step just needs to send the command to the app, the app just needs to receive the command to get the step to pass.

# features/step_definitions/terminal_steps.rb
Given /^I enter "([^\"]*)"$/ do |command|
  @app.interpret(command)
end

# lib/terminal.rb
  def interpret(command)
  end

Running the tests now gives:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:6
    Then I should see                     # features/step_definitions/terminal_steps.rb:10
      """
      1. foo.txt
      """
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:11:in `/^I should see$/'
      features/search_for_files.feature:8:in `Then I should see'

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:6
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:6
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:14
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:15:in `/^"([^\"]*)" should be open in "([^\"]*)"$/'
      features/search_for_files.feature:16:in `Then "foo.txt" should be open in "gedit"'

2 scenarios (2 pending)
7 steps (2 pending, 5 passed)

Now I’m going to tackle the remaining pending steps in one go since their implementation overlaps somewhat. First we need to implement the “I should see” step, which will just read from our @io object to find out what the app has printed:

# features/step_definitions/terminal_steps.rb

Then /^I should see$/ do |string|
  @io.rewind
  @io.read.gsub(/\n*$/, "").should == string
end

To implement the step that checks that a file was openned, we want to know that the app shelled out to the relevant program to tell it to open the file. To support this, I’m going to stub out the system call method on the app and collect any commands we’d normally send to the shell. In our first step:

# features/step_definitions/terminal_steps.rb

Given /^I start the app with "([^\"]*)"$/ do |command|
  @io  = StringIO.new
  @app = Terminal.new(command.split(/\s+/), @io)

  @commands = []
  @app.stub('`') { |cmd| @commands << cmd }
end

Then to implement the “should be open” step we can inspect this array to see if the right command was called:

# features/step_definitions/terminal_steps.rb

Then /^"([^\"]*)" should be open in "([^\"]*)"$/ do |path, program|
  @commands.should include("#{program} #{path}")
end

One more test run:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:9
    Then I should see                     # features/step_definitions/terminal_steps.rb:13
      """
      1. foo.txt
      """
      expected: "1. foo.txt",
           got: "" (using ==)

       Diff:
      @@ -1,2 +1 @@
      -1. foo.txt
       (Spec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/terminal_steps.rb:15:in `/^I should see$/'
      features/search_for_files.feature:8:in `Then I should see'

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:9
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:9
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:18
      expected [] to include "gedit foo.txt" (Spec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/terminal_steps.rb:19:in `/^"([^\"]*)" should be open in "([^\"]*)"$/'
      features/search_for_files.feature:16:in `Then "foo.txt" should be open in "gedit"'

Failing Scenarios:
cucumber features/search_for_files.feature:6 # Scenario: Find files by name
cucumber features/search_for_files.feature:13 # Scenario: Open a matching file

2 scenarios (2 failed)
7 steps (2 failed, 5 passed)

Finally, we implement code in the app to make these steps pass. I’m not going to actually write the search logic, just the logic to print output and open files so that our tests pass.

# lib/terminal.rb

require 'oyster'

class Terminal
  BIN_SPEC = Oyster.spec do
    string :command
  end

  def initialize(argv, io)
    @options = BIN_SPEC.parse(argv)
    @stdout  = io
    @results = []
  end

  def interpret(command)
    case command
    when /^\d+$/ then open_result(command.to_i - 1)
    else
      @results = command.split(/\s+/).map { |f| "#{f}.txt" }
      print_results
    end
  end

  def open_result(index)
    `#{ @options[:command] } #{ @results[index] }`
  end

  def print_results
    @results.each_with_index do |result, i|
      @stdout.puts "#{ i+1 }. #{ result }"
    end
  end
end

We should get a nice green list of cukes now:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:9
    Then I should see                     # features/step_definitions/terminal_steps.rb:13
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:9
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:9
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:18

2 scenarios (2 passed)
7 steps (7 passed)

Now that you have a well-tested frontend for your app, you can flesh it out with business logic, adding more tests using “Given I enter”, “I should see” as you go. Finally, you can easily add an executable for your project that supplies real input/output objects and runs the application. In bin/terminal

#!/usr/bin/env ruby

require 'rubygems'
require 'readline'
require File.dirname(__FILE__) + '/../lib/terminal'

app = Terminal.new(ARGV, Kernel)
loop { app.interpret(Readline.readline('> ')) }

Just chmod +x bin/terminal and run it, and you should see it running just like your tests say it should.

Helium: a package server for JavaScript

In Git, Helium, JS.Class, JavaScript, Metaprogramming, Ojay, PackR, Ruby, Yahoo! UI on 2 November 2009 with 7 comments

Last week, my former employer theOTHERmedia open-sourced the last project I worked on there: Helium. It’s a web application that lets you deploy JavaScript packages from Git and load them on-demand into any website by including a single script tag. There’s been a lot of innovation in JavaScript deployment recently, and Helium fits a particular set of needs that I think most web agencies will be all too used to struggling with.

First, some background about the problems we were trying to solve. theOTHERmedia is a design and development agency with dozens of clients and a lot of code to keep under control. Their JavaScript stack is based around YUI and Ojay, as well as other common tools like Google Maps, Analytics etc. As part of client work, we inevitably ended up producing a lot of reusable components that didn’t really fit into the Ojay core but nevertheless needed to be shared between client projects.

Because they had no obvious place to live, these little libraries ended up being copy-pasted from project to project, producing a maintenance headache. If I fixed a bug in one copy, I’d have to track down all the other copies to patch them. Also, some clients required some small customisations: too small to merit generalising the code to accommodate them, but large enough that managed branching, merging and cherry-picking between branches would really help with maintenance.

The main day-to-day problems I’d run into were that these projects became hard to maintain, hard to track down, and that it wasn’t clear how to set them up. Developers would copy a component into their project and come to me when it didn’t work: inevitably some dependency or config setting was missing. I wanted to make it as transparent as possible to load any JavaScript object you liked into a site without running into these sorts of problems, and I wanted each library to be maintained in one repository and farmed out to client sites as required. I should be able to apply a bug fix and see it show up in all our clients’ sites immediately.

The solution to the maintenance problem was obvious: these libraries should have been under their own version control, and the need to fork libraries for certain clients and keep the forks up-to-date with bug fixes made Git the perfect choice for this. Git makes branching and merging easy, and lets you cherry-pick changes between branches if you don’t want to do a full merge. Problem one solved.

The second problem is deployment: we need to get the libraries out of Git, build them and distribute them to client sites. Clients need to specify which version (i.e. which branch or tag) of each library they want to use. And, dependencies should be handled automatically: I’m a big fan of package managers like aptitude on Ubuntu or RubyGems in the Ruby community as they make it easy to get software running without worrying about its internal dependencies.

This is where an number of other tools come in. Helium is really a wrapper around a set of smaller components that were all designed to be flexible enough that they can be trivially integrated. The main “front end” of Helium is powered by JS.Packages, a pure-JavaScript dependency manager that ships with JS.Class. It works at a very high level, letting you load JavaScript objects by name rather than by URL, figuring out dependencies for you and downloading files in parallel where possible. For example, it lets me write this:

require('YAHOO.util.Selector', 'GMap2', function() {
    var links = YAHOO.util.Selector.query('a'),
        map = new GMap2(container);

    // ...
});

This is very powerful, but it requires some configuration to tell JS.Packages where to find objects, what other objects they depend on and so on. To make the above work, you need the following configuration:

JS.Packages(function() { with(this) {
    file('http://yui.yahooapis.com/2.8.0/build/yahoo-dom-event/yahoo-dom-event.js')
        .provides('YAHOO',
                  'YAHOO.util.Dom',
                  'YAHOO.util.Event');

    file('http://yui.yahooapis.com/2.8.0/build/selector/selector-min.js')
        .provides('YAHOO.util.Selector')
        .requires('YAHOO.util.Dom');

    loader(function(cb) {
        var url = 'http://www.google.com/jsapi?key=' + Helium.GOOGLE_API_KEY;
        load(url, cb);
    })  .provides('google.load');

    loader(function(cb) { google.load('maps', '2.x', {callback: cb}) })
        .provides('GMap2', 'GClientGeocoder',
                  'GEvent', 'GLatLng', 'GMarker')
        .requires('google.load');
}});

Aside from the fact that writing this is fairly tedious, I didn’t want these config files to live in the client projects where they’d need just as much maintenance as the original libraries, especially since different branches of a library often have different dependencies.

Now most of this code tends to be of the file/provides/requires variety rather than the custom loader function variety, and this is code that’s very easy to generate. The final piece of the puzzle, Jake, is what ties this all together.

Jake is a build tool for JavaScript projects that mostly handles code concatenation and minification for larger JavaScript projects. The neat thing about it is that it lets you embed metadata (such as dependency information) and has an event system to let you know when build files are generated and where they live on disk. This means you can use it to build a manifest of all the files a project build generates, which JavaScript objects each file contains and which objects it depends on, just by placing some metadata in each project’s build configuration. We’ve now moved this data out of the client projects and into a central location: each library manages its own dependency data, and Helium uses Jake to extract this and build a manifest of all the libraries on your server and what their dependencies are.

Putting it all together, the whole process pulls projects out of Git, uses Jake to build every branch of each project and extract its dependency data, then uses this to generate a JS.Packages manifest: it’s this file that client sites include in their head section:

<script type="text/javascript"
        src="http://helium.yourcompany.com/js/helium.js">
        </script>

You then need to tell Helium which branch/tag of each library to use, and thereafter you can use require() to load any object you like. On the client side, Helium essentially acts as a versioning layer on top of JS.Packages.

<script type="text/javascript">
Helium.use('yui', '2.7.0');
Helium.use('ojay', '0.4.1');
</script>

The intent is for Helium users to host their own installations of it, since providing a single package manifest for all the JavaScript libraries on the web would produce a huge file and obviate the benefit of on-demand loading. If you want to check it out, the best place to start is the documentation on GitHub, which covers a ton of stuff I’ve not even mentioned here. In particular it explains how to make your own JavaScript projects Helium-deployable, a process I’ve tried hard to make as easy as possible. I’d love to see this adopted as a package distribution system for JavaScript, so if you have any feedback on how it can be improved, get over to GitHub and let us know!

Getting started with Cucumber, RSpec, Webrat and multiruby

In Metaprogramming, Rails, Ruby, Testing on 3 October 2009 with 6 comments

After a couple years off from full-time Ruby/Rails work, I’m getting back into it having just joined the development team at Songkick. Much as I’ve tried to keep my hand in with the Ruby world by hacking on stuff like Heist, Siren and Jake, a few things have passed me by. In particular, I’ve not been doing any big Rails apps for a while so I’m a little behind on the current testing toolset. Songkick uses Cucumber for acceptance testing, with help from RSpec, Webrat and factory_girl. The test suite is huge, so needless to say I’ve spent a lot of time getting up to speed on these tools so I can get to work.

While these tools have great support for various web frameworks, I wanted to get down to basics so I can try them out for non-Rails (and non-web) projects. I’m going to go through all the setup you need to start writing Cucumber features by writing a quick test for checking Google search results.

First up, you’re going to need a bunch of gems. Webrat uses Nokogiri for parsing HTML documents, and we need a couple of Ubuntu packages to support that:

$ sudo aptitude install libxslt1-dev libxml2-dev

Then you need to install the required gems. You’ll need to get multiruby set up, then run these:

$ sudo gem install cucumber rspec webrat mechanize
$ multiruby -S gem install cucumber rspec webrat mechanize

Now we have the libraries we need, we have to set up some directories that Cucumber expects to find things in:

$ mkdir features
$ mkdir features/step_definitions
$ mkdir features/support

We can now run Cucumber and it will tell us we’ve got no tests:

$ cucumber
0 scenarios
0 steps
0m0.000s

Let’s get testing. We start by writing features in Cucumber’s plain-text language to describe what we want our app to do. It’s common to put some text at the start of a feature to explain why it’s being added. Save the following as features/search.feature:

Feature: Search Google
  In order to make sure people can find my documentation
  I want to check it is listed on the first page in Google

  Scenario: Searching for JS.Class docs
    Given I have opened "http://www.google.com/"
    When I search for "JS.Class"
    Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript"

Running Cucumber again will tell us we’ve not defined the steps: we’ve not said what any of the above text means:

$ cucumber
Feature: Search Google
  In order to make sure people can find my documentation
  I want to check it is listed on the first page in Google

  Scenario: Searching for JS.Class docs                                                                    # features/search.feature:5
    Given I have opened "http://www.google.com/"                                                           # features/search.feature:6
    When I search for "JS.Class"                                                                           # features/search.feature:7
    Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript" # features/search.feature:8

1 scenario (1 undefined)
3 steps (3 undefined)
0m0.002s

You can implement step definitions for undefined steps with these snippets:

Given /^I have opened "([^\"]*)"$/ do |arg1|
  pending
end

When /^I search for "([^\"]*)"$/ do |arg1|
  pending
end

Then /^I should see a link to "([^\"]*)" with text "([^\"]*)"$/ do |arg1, arg2|
  pending
end

We have to fix these by adding step definitions to step_definitions/search_steps.rb: copy the above definitions into the file then run Cucumber:

$ cucumber
Feature: Search Google
  In order to make sure people can find my documentation
  I want to check it is listed on the first page in Google

  Scenario: Searching for JS.Class docs                                                                    # features/search.feature:5
    Given I have opened "http://www.google.com/"                                                           # features/step_definitions/search_steps.rb:1
      TODO (Cucumber::Pending)
      features/search.feature:6:in `Given I have opened "http://www.google.com/"'
    When I search for "JS.Class"                                                                           # features/step_definitions/search_steps.rb:5
    Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript" # features/step_definitions/search_steps.rb:9

1 scenario (1 pending)
3 steps (2 skipped, 1 pending)
0m0.002s

The first step will be marked yellow to indicate that it’s pending, and the remaining steps will be blue to show they’ve been skipped. Let’s try to turn the first step green. In step_definitions/search_steps.rb, change the first definition to the following:

Given /^I have opened "([^\"]*)"$/ do |url|
  visit url
end

visit is a method provided by Webrat for getting web pages. We need to tell Cucumber to load Webrat and include its methods in the testing environment, which we do by putting this in support/env.rb:

require 'webrat'

Webrat.configure do |config|
  config.mode = :mechanize
end

World(Webrat::Methods)
World(Webrat::Matchers)

Webrat doesn’t actually make requests itself, it provides adapters to other systems such as Rails and Sinatra for calling their web stacks with a uniform API. We just want to query the web, so we’re using telling Webrat to use Mechanize to fetch pages. We then mix two modules into the Cucumber World, the context that all your steps run in. Webrat::Methods provides methods like visit and click_button for navigating the web, and Webrat::Matchers provides things like have_selector and contains for use with RSpec’s should interface.

Running our features now gives the following:

 $ cucumber
Feature: Search Google
  In order to make sure people can find my documentation
  I want to check it is listed on the first page in Google

  Scenario: Searching for JS.Class docs                                                                    # features/search.feature:5
    Given I have opened "http://www.google.com/"                                                           # features/step_definitions/search_steps.rb:1
    When I search for "JS.Class"                                                                           # features/step_definitions/search_steps.rb:5
      TODO (Cucumber::Pending)
      features/search.feature:7:in `When I search for "JS.Class"'
    Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript" # features/step_definitions/search_steps.rb:9

1 scenario (1 pending)
3 steps (1 skipped, 1 pending, 1 passed)
0m0.624s

Great, one passing step! Let’s move onto the next one: we want to tell Webrat to fill in Google’s search form and submit. Replace the second step with:

When /^I search for "([^\"]*)"$/ do |query|
  fill_in "q", :with => query
  click_button "Google Search"
end

We’ll be able to access the HTTP response after the form submission using Webrat’s response_body method in the next step. Run the features again:

$ cucumber
Feature: Search Google
  In order to make sure people can find my documentation
  I want to check it is listed on the first page in Google

  Scenario: Searching for JS.Class docs                                                                    # features/search.feature:5
    Given I have opened "http://www.google.com/"                                                           # features/step_definitions/search_steps.rb:1
    When I search for "JS.Class"                                                                           # features/step_definitions/search_steps.rb:5
    Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript" # features/step_definitions/search_steps.rb:10
      TODO (Cucumber::Pending)
      features/search.feature:8:in `Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript"'

1 scenario (1 pending)
3 steps (1 pending, 2 passed)
0m0.481s

Only one step left. In the Then step, we want to use RSpec to check some condition is true. We want to check that a link exists with the URL we’ve given, and that link should contain the given text. Webrat uses Nokogiri to let you query the response using CSS selectors:

Then /^I should see a link to "([^\"]*)" with text "([^\"]*)"$/ do |url, text|
  response_body.should have_selector("a[href='#{ url }']") do |element|
    element.should contain(text)
  end
end

One more test run:

$ cucumber
Feature: Search Google
  In order to make sure people can find my documentation
  I want to check it is listed on the first page in Google

  Scenario: Searching for JS.Class docs                                                                    # features/search.feature:5
    Given I have opened "http://www.google.com/"                                                           # features/step_definitions/search_steps.rb:1
    When I search for "JS.Class"                                                                           # features/step_definitions/search_steps.rb:5
    Then I should see a link to "http://jsclass.jcoglan.com/" with text "JS.Class - Ruby-style JavaScript" # features/step_definitions/search_steps.rb:10

1 scenario (1 passed)
3 steps (3 passed)
0m0.537s

And a final check with multiruby:

$ multiruby -S cucumber

TOTAL RESULT = 0 failures out of 3

Passed: 1.9.1-p129, 1.8.7-p160, 1.8.6-p368
Failed:

Hopefully that’s enough of an introduction to get you started: the main problem I had was figuring out which libraries were providing which testing features and what setup was required. There’s a ton of helpful information in the Cucumber wiki to move beyond the basics.

Bluff 0.3.6: tooltips, dot graphs and sundry bug fixes

In Bluff, JS.Class, JavaScript, Ruby on 14 September 2009 with 3 comments

A year to the day since the initial release, and following the new release of Gruff a couple of weeks ago, I’m happy to announce the release of Bluff 0.3.6.

This is largely a maintenance release that pulls in patches that have gone into Gruff in the past twelve months, but it does introduce a couple of new features. We’ve added the Dot graph type that’s new in the latest Gruff release, and we’ve also added tooltip support to line and bar graphs, which you can see in action on the home page as we speak. This gives you a little mouseover-triggered bubble over data points so users can see the exact value. (Side note: pie charts are now labelled with raw values rather than percentages.) This feature was kindly sponsored by CrimsonJet, makers of Appstatz which seems to be making nice use of Bluff.

The only other major changes are now optional settings, all of which are listed in the API reference. They include title_margin, legend_margin, dot_radius, line_width, bar_spacing, and hide_labels_less_than. As usual, get the download from the Bluff home page.

Heist 0.3: complete set of Scheme data types

In Functional, Heist, Lisp, Metaprogramming, Ruby on 24 August 2009 with no comments

I actually tagged the 0.3.0 release of Heist, my Ruby Scheme implementation, about a month back, mostly to get it off my desk for a while. I’ve made a few minor tweaks and released 0.3.1 over the weekend, so now’s as good a time as any to go over what’s new.

The major milestone for this release was to complete the set of R5RS data types: Heist now supports vectors (including macro and quasiquotation support), characters and strings, including full libraries for each type. These work largely as advertised by the spec, although vector handling is slightly idiosyncratic.

Some Schemes require vectors to be quoted, as in '#(1 2 3). This makes the vector a constant (i.e. it is immutable), and every evaluation of it returns the same object. That is (in mzscheme):

> (define (f) '#(1 2 3))
> (define a (f))
> (define b (f))
> (eq? a b)
#t

> (define (f) (vector 1 2 3))
> (define a (f))
> (define b (f))
> (eq? a b)
#f

So we see that vector allocates a new object every time, but the quoted literal does not. This is still true in Heist, but Heist also allows non-quoted vectors. These return a new mutable object when evaluated.

(define (f) '#(1 2 3))
(define (g) #(1 2 3))

(define a (f)) (define b (f))
; => #(1 2 3)
(eq? a b)
; => #t
(vector-set! a 0 9)
; [error] Cannot modify vector constant

(define c (g)) (define d (g))
; => #(1 2 3)
(eq? c d)
; => #f
(vector-set! c 0 9)
c
; => #(9 2 3)
d
; => #(1 2 3)

The reason for this is that unquoted vectors must be allowed — they are part of the syntax-rules spec. But, making them immutable makes performing macro transformations very hard; particularly inlining the transformation into the AST is impossible since it involves setting an attribute or changing a cell on my now-frozen Heist::Runtime::Vector instance. So they need to be mutable (left unfrozen, in Ruby terms). If an unquoted literal evaluated to the same object every time, then the vector-set! procedure would be able to modify the parse tree, which is clearly undesirable. So, unquoted vector literals evaluate to a mutable copy of themselves.

With the Scheme type system completed, where do we go from here? The only pieces of R5RS I’ve not implemented so far are the file I/O procedures, and some of the more exotic continuation stuff like call-with-values and dynamic-wind (I’d love to figure these out but so far they’ve eluded me). My main reason for developing this was to help me with SICP, so I plan to keep going with that and fill in any blanks the book requires along the way.

I may provide faster Ruby implementations of the built-in procedures currently written in Scheme, but that’s not a priority. Paul Graham will tell you that Lisp “came about … as the byproduct of an attempt to axiomatize computation”, and Scheme is no exception. If you write a Scheme yourself, you soon start to find that there do not need to be very many functions built into the core language: a lot of the R5RS library can be written in Scheme itself. Most of the syntactic functions can be written as macros, and many of the data types just need a constructor, accessors and mutators and the user can build the rest on top. Witness the list functions: the core provides cons, car, cdr, set-car!, set-cdr! and pair?, and all the other ‘built-in’ list functions are written in Scheme. SICP chapter 2 reinforces this pattern of data abstraction and is well worth reading as a lesson in robust program design.

So, for the time being at least I’m happy letting Heist be a neat little example of this axiomatization idea at work. I’d check out the source now if I were you, before I get bored and decide the performance needs kicking up a notch.

← Before After →

About

James Coglan is a web developer at Songkick in London, UK. He writes here about Ruby, JavaScript, API and language design, Scheme and anything else that seems relevant. He has a few open-source projects, a GitHub profile and some photographs.

Recently

Et Cetera


© The If Works. Powered by WordPress using the DePo Skinny Theme.