I recently wrote a tiny little tool called Claw to help me work on large codebases in gEdit. It provides a terminal that lets you search for files by name and content using very minimal syntax, and it numbers the search results so you can just type a number to open the file using any shell command. In the last few days I converted its tests from Test::Unit to Cucumber and added a lot more coverage, and found Cucumber to be very nice for testing command-line UIs.

Test::Unit and RSpec work fine when you’re just calling methods and testing their return values, stuff like assert [:foo, :bar].include?(:foo) or "ruby".upcase.should == "RUBY". The thing about command-line apps is that they mostly output their results as side effects, such as writing to standard out, modifying files, making network calls etc. Testing these things often involves stubs and mock objects and these can make your tests harder to follow, but Cucumber lets you hide all that behind easy-to-read prose.

To illustrate, let’s write a little app ourselves using TDD. Let’s make a few directories:

mkdir tdd-terminal
cd tdd-terminal
mkdir lib
mkdir -p features/step_definitions
mkdir -p features/support

Let’s write a spec for our program. This is a stripped down version of the Claw spec that contains enough details to illustrate how to test command-line apps: we’ve got some visible output to verify, and a side effect of running the program: it should open files for us when we issue certain commands.

# features/search_for_files.feature

Feature: Search for files
  
  Background:
    Given I start the app with "-c gedit"
  
  Scenario: Find files by name
    Given I enter "foo"
    Then I should see
    """
    1. foo.txt
    """
  
  Scenario: Open a matching file
    Given I enter "foo"
    And I enter "1"
    Then "foo.txt" should be open in "gedit"

Let’s run the feature using Cucumber:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/search_for_files.feature:4

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/search_for_files.feature:7
    Then I should see                     # features/search_for_files.feature:8
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/search_for_files.feature:14
    And I enter "1"                          # features/search_for_files.feature:15
    Then "foo.txt" should be open in "gedit" # features/search_for_files.feature:16

2 scenarios (2 undefined)
7 steps (7 undefined)
0m0.041s

You can implement step definitions for undefined steps with these snippets:

Given /^I start the app with "([^\"]*)"$/ do |arg1|
  pending
end

Given /^I enter "([^\"]*)"$/ do |arg1|
  pending
end

Then /^I should see$/ do |string|
  pending
end

Then /^"([^\"]*)" should be open in "([^\"]*)"$/ do |arg1, arg2|
  pending
end

Take the generated step definitions and copy them into features/step_definitions/terminal_steps.rb. Re-running the tests you should see the steps marked as pending or skipped rather than undefined. Let’s implement the first step, the one that starts our app.

Given /^I start the app with "([^\"]*)"$/ do |command|
  @io  = StringIO.new
  @app = Terminal.new(command.split(/\s+/), @io)
end

Here we take the command-line argument string and split it up to mimic what we’d expect to receive in ARGV when running the app for real. We also make an IO object and give it to the app: instead of calling Kernel#puts our app will call puts on whatever object we pass in. In production this would be Kernel but for testing we want an object that we can read from so we can check what the app prints. Run the tests again:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1
      uninitialized constant Terminal (NameError)
      ./features/step_definitions/terminal_steps.rb:3:in `/^I start the app with "([^\"]*)"$/'
      features/search_for_files.feature:4:in `Given I start the app with "-c gedit"'

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:6
    Then I should see                     # features/step_definitions/terminal_steps.rb:10
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:6
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:6
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:14

Failing Scenarios:
cucumber features/search_for_files.feature:6 # Scenario: Find files by name

2 scenarios (1 failed, 1 skipped)
7 steps (1 failed, 6 skipped)

We need to go and implement our Terminal class to get the first step working. First, enter the following in features/support/env.rb:

# features/support/env.rb

require File.dirname(__FILE__) + '/../../lib/terminal'
require 'spec/mocks'

This tells Cucumber to load our application and RSpec’s mocking library before running the tests. We need to give it something to load, so let’s put the following in lib/terminal.rb:

# lib/terminal.rb

require 'oyster'

class Terminal
  BIN_SPEC = Oyster.spec do
    string :command
  end
  
  def initialize(argv, io)
    @options = BIN_SPEC.parse(argv)
    @stdout  = io
  end
end

The constructor takes an array of command-line input (should be ARGV in production) and an output device to puts to. I’m using my Oyster gem to parse the input into an options hash, and storing a reference to the output device for later use. This should be enough code to turn our first step green:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:6
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:7:in `/^I enter "([^\"]*)"$/'
      features/search_for_files.feature:7:in `Given I enter "foo"'
    Then I should see                     # features/step_definitions/terminal_steps.rb:10
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:6
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:7:in `/^I enter "([^\"]*)"$/'
      features/search_for_files.feature:14:in `Given I enter "foo"'
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:6
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:14

2 scenarios (2 pending)
7 steps (3 skipped, 2 pending, 2 passed)

Let’s implement the next step, “Given I enter”. This step just needs to send the command to the app, the app just needs to receive the command to get the step to pass.

# features/step_definitions/terminal_steps.rb
Given /^I enter "([^\"]*)"$/ do |command|
  @app.interpret(command)
end

# lib/terminal.rb
  def interpret(command)
  end

Running the tests now gives:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:6
    Then I should see                     # features/step_definitions/terminal_steps.rb:10
      """
      1. foo.txt
      """
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:11:in `/^I should see$/'
      features/search_for_files.feature:8:in `Then I should see'

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:6
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:6
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:14
      TODO (Cucumber::Pending)
      ./features/step_definitions/terminal_steps.rb:15:in `/^"([^\"]*)" should be open in "([^\"]*)"$/'
      features/search_for_files.feature:16:in `Then "foo.txt" should be open in "gedit"'

2 scenarios (2 pending)
7 steps (2 pending, 5 passed)

Now I’m going to tackle the remaining pending steps in one go since their implementation overlaps somewhat. First we need to implement the “I should see” step, which will just read from our @io object to find out what the app has printed:

# features/step_definitions/terminal_steps.rb

Then /^I should see$/ do |string|
  @io.rewind
  @io.read.gsub(/\n*$/, "").should == string
end

To implement the step that checks that a file was openned, we want to know that the app shelled out to the relevant program to tell it to open the file. To support this, I’m going to stub out the system call method on the app and collect any commands we’d normally send to the shell. In our first step:

# features/step_definitions/terminal_steps.rb

Given /^I start the app with "([^\"]*)"$/ do |command|
  @io  = StringIO.new
  @app = Terminal.new(command.split(/\s+/), @io)
  
  @commands = []
  @app.stub('`') { |cmd| @commands << cmd }
end

Then to implement the “should be open” step we can inspect this array to see if the right command was called:

# features/step_definitions/terminal_steps.rb

Then /^"([^\"]*)" should be open in "([^\"]*)"$/ do |path, program|
  @commands.should include("#{program} #{path}")
end

One more test run:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:9
    Then I should see                     # features/step_definitions/terminal_steps.rb:13
      """
      1. foo.txt
      """
      expected: "1. foo.txt",
           got: "" (using ==)
      
       Diff:
      @@ -1,2 +1 @@
      -1. foo.txt
       (Spec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/terminal_steps.rb:15:in `/^I should see$/'
      features/search_for_files.feature:8:in `Then I should see'

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:9
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:9
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:18
      expected [] to include "gedit foo.txt" (Spec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/terminal_steps.rb:19:in `/^"([^\"]*)" should be open in "([^\"]*)"$/'
      features/search_for_files.feature:16:in `Then "foo.txt" should be open in "gedit"'

Failing Scenarios:
cucumber features/search_for_files.feature:6 # Scenario: Find files by name
cucumber features/search_for_files.feature:13 # Scenario: Open a matching file

2 scenarios (2 failed)
7 steps (2 failed, 5 passed)

Finally, we implement code in the app to make these steps pass. I’m not going to actually write the search logic, just the logic to print output and open files so that our tests pass.

# lib/terminal.rb

require 'oyster'

class Terminal
  BIN_SPEC = Oyster.spec do
    string :command
  end
  
  def initialize(argv, io)
    @options = BIN_SPEC.parse(argv)
    @stdout  = io
    @results = []
  end
  
  def interpret(command)
    case command
    when /^\d+$/ then open_result(command.to_i - 1)
    else
      @results = command.split(/\s+/).map { |f| "#{f}.txt" }
      print_results
    end
  end
  
  def open_result(index)
    `#{ @options[:command] } #{ @results[index] }`
  end
  
  def print_results
    @results.each_with_index do |result, i|
      @stdout.puts "#{ i+1 }. #{ result }"
    end
  end
end

We should get a nice green list of cukes now:

Feature: Search for files

  Background:                             # features/search_for_files.feature:3
    Given I start the app with "-c gedit" # features/step_definitions/terminal_steps.rb:1

  Scenario: Find files by name            # features/search_for_files.feature:6
    Given I enter "foo"                   # features/step_definitions/terminal_steps.rb:9
    Then I should see                     # features/step_definitions/terminal_steps.rb:13
      """
      1. foo.txt
      """

  Scenario: Open a matching file             # features/search_for_files.feature:13
    Given I enter "foo"                      # features/step_definitions/terminal_steps.rb:9
    And I enter "1"                          # features/step_definitions/terminal_steps.rb:9
    Then "foo.txt" should be open in "gedit" # features/step_definitions/terminal_steps.rb:18

2 scenarios (2 passed)
7 steps (7 passed)

Now that you have a well-tested frontend for your app, you can flesh it out with business logic, adding more tests using “Given I enter”, “I should see” as you go. Finally, you can easily add an executable for your project that supplies real input/output objects and runs the application. In bin/terminal

#!/usr/bin/env ruby

require 'rubygems'
require 'readline'
require File.dirname(__FILE__) + '/../lib/terminal'

app = Terminal.new(ARGV, Kernel)
loop { app.interpret(Readline.readline('> ')) }

Just chmod +x bin/terminal and run it, and you should see it running just like your tests say it should.