Source map support added to Packr and Jake

I’m a little late announcing this, in fact I held off until I’d been using this code for a couple of months to make sure there weren’t any glaring surprises. But the good news is, all the JavaScript libraries I ship from now on will come with source maps, and yours can too.

What’s a source map, you ask. Well, as this article explains, a source map is just a metadata file that maps locations in a minified (or otherwise compiled) JavaScript file back to locations in the source code the author is working on. This means you can load minified/compiled code into a browser, and when log messages or exceptions occur, the browser can show you the filename and line number in your source code that such events come from, rather than somewhere in line 1 of whatever huge bundle of code you’re actually deploying. This is really useful.

Telling the browser about this metadata is really easy. If you look at the latest release of Faye, you’ll see it contains three client-side files:

  • faye-browser.js – the source code
  • faye-browser-min.js – the minified code
  • faye-browser-min.js.map – the source map

faye-browser-min.js is what gets loaded in the browser. At the end of the file is a single line comment that looks like this:

//@ sourceMappingURL=faye-browser-min.js.map

This line tells the browser to use the file faye-browser-min.js.map as the source map for the JavaScript file that includes this comment. sourceMappingURL is resolved relative to the URL of the containing script, so as long as the script and the map are in the same directory this directive works fine.

The file faye-browser-min.js.map looks like this:

{
  "version": 3,
  "file": "faye-browser-min.js",
  "sourceRoot": "",
  "sources": ["faye-browser.js"],
  "names": ["_advice", "_callback", "_callbacks", "_cancelled", "_cbCount", "_channels", ...],
  "mappings": "AAAA,IAAI,MAAQ,OAAO,QAAU,SAAW,QACxC,GAAI,OAAO,UAAY,WAAY,OAAO,KAAO,KAEjD,KAAK..."
}

The sources field refers to the source code the map relates to: a source map can describe how one file was concatenated from a set of source files. The browser uses the sources and mappings data to show you the source code instead of the minified code when you use the web inspector, and it only loads the map and the source code if you have the inspector open so it doesn’t add latency for normal users.

Now if you want to use this new browser feature on your own projects, I have a couple of tools you should look at. First is Packr, my port of Dead Edwards’ Packer tool in Ruby. Say you have two source files:

# example_a.js

1. // When the minified code is loaded into a browser, you should see the call to
2. // console.log() attributed to example_a.js:4
3. 
4. console.log('Hello from file A');
# example_b.js

1. var display = function(message) {
2.   alert(message + ' from file B');
3. };
4. 
5. display('Ahoy there');

You can run this in the shell:

packr example_a.js example_b.js -o min/example-min.js -h '/* Copyright 2012 */' --shrink-vars

And it will generate a minified file and a source map for you:

# example-min.js

/* Copyright 2012 */
console.log('Hello from file A');var display=function(a){alert(a+' from file B')};display('Ahoy there');
//@ sourceMappingURL=example-min.js.map
# example-min.js.map

{
  "version": 3,
  "file": "example-min.js",
  "sourceRoot": "",
  "sources": ["../example_a.js", "../example_b.js"],
  "names": ["message"],
  "mappings": ";AAGA,QAAQ,KAAK,MAAM,KAAK,KAAK,ICH7B,IAAI,QAAU,SAASA,GACrB,MAAMA,IAAY,KAAK,KAAK,KAG9B,SAAS,KAAK;"
}

Packr lets you remove whitespace, compress variable names, use base-62 encoding, insert header comments on minified output, and can be used as a CLI or as a Ruby library. See the readme for more info.

Second is a tool I’ve been using to build all my JavaScript projects for a few years now, Jake. Jake is based on Packr, but is designed for larger projects. It’s designed to let you describe the structure of a project using a simple YAML file, and build the whole thing easily with one shell command. Based on Packr’s source map support, I’ve added the ability to tell Jake to generate source maps for one build based on the output of another. For example, say your build produces both unminified (src) and minified (min) copies of your code; then you can tell Jake to generate a source map for the min build that refers to code in the src build as follows:

---
builds:
  src:
    minify: false
  min:
    minify: true
    shrink_vars: true
    source_map: src

The source_map: src line means, generate a source map for min files that refers to locations in the corresponding src file. This is how the Faye files above are generated. Again, the Jake readme has more information.

We’re likely to see source map support in other tools soon; adding it to Packr was fairly easy because of Packr’s very simple model for compressing code. Basically it’s a regex-replacement-based compressor rather than a full JavaScript parser, so source maps could be added as a post-processing step rather than being based on AST annotations requiring deep integration. Unfortunately this means that if you like omitting semicolons from your code you might have to wait a little longer to use this stuff.