Flagger 0.9.5 is out

Flagger got a little bug fix yesterday, in that it now recognises after_mark_as_* callbacks if they are private methods. This should really have been there in the beginning, I just left a couple of true arguments out somewhere. Get it like this:

script plugin intstall


Make Capistrano compress your JavaScript and CSS automatically

I finally gave Capistrano a whirl this morning by setting up a dummy application and getting a deployment to work. I must say, once it’s up and running it makes updating your application about as easy as it conceivably could be, and must save you loads of time if you have your app installed on multiple machines.

Thing is, when pushing code out to the server, I need to make sure my JavaScript and CSS is nicely packed up so I can serve gzipped files to supported browsers. The first step is having AssetPackager build its packages, and the second is finding all the .js and .css files in my app and making compressed copies. This can all be done automatically by adding the following lines to your :after_update_code task in deploy.rb (line wraps marked \:

task :after_update_code, :roles => [:app, :db, :web] do
  # ...
  run "cd #{release_path} && rake asset:packager:build_all"
  run "for file in $(find #{release_path}/public/javascripts \\
      -name '*.js'); do gzip $file -f -c > $file.gz; done"
  run "for file in $(find #{release_path}/public/stylesheets \\
      -name '*.css'); do gzip $file -f -c > $file.gz; done"

All this does is invoke the AssetPackager build task, then it goes into your JS and CSS directories in turn and finds all the JS and CSS files. It loops through the files it finds and makes gzipped copies with ‘.gz’ appended to the filename – the orignial files are left intact so you can serve them to browsers that don’t support gzipped content.

Variable or reference?

One thing that’s tripped me up recently, and is the cause of some bugs in Sylvester that I’m ironing out, is JavaScript’s handling of array and object variables. When you create an array — [3,4,5] — or an object — {foo: 12, bar: 'something'}, the variable name you give it is not a storage vessel for the value you’re assigning, it’s a direct reference to the value in memory. For example, with strings you can do this:

var foo = 'a string';
var bar = foo;
bar = 'something else';

// foo is still 'a string'

So by saying bar = foo, you’re telling the compiler, “take the value stored in the variable foo and copy it into the variable bar“. The value ‘a string’ is stored in memory twice — once for each variable, until bar is reassigned.

Not so with arrays. Take the following:

var foo = ['apples', 'oranges', 'pears'];
var bar = foo;
bar[2] = 'kiwis';

// foo is now ['apples', 'oranges', 'kiwis']

Now when you say bar = foo, you are not copying foo‘s value into the space in memory represented by bar. There remains only one copy of the array in memory, and both foo and bar are references to it. If you want to copy an array, you need to use the following:

var foo = ['apples', 'oranges', 'pears'];

var bar = foo.slice();
bar[2] = 'kiwis';

// foo is ['apples', 'oranges', 'pears']

The reference vs. value, um, thing (I won’t call it a problem as there are some very useful things you can do with it) also means that any function that takes an array or an object as an argument receives the original variable you pass to it, not a copy of its value. Take a look at the following:

function doSomethingTo(arg) {
  var localArg = arg;

var foo = ['this', 'is', 'an', 'array'];

// foo is now ['this', 'is', 'an', 'array', 'extra!']

You might be fooled into thinking that assigning the incoming argument to a local variable and pushing values onto that means that the argument you pass will not be modified, but this is not the case. To leave the argument untouched by the function call, you need to explicitly copy its value in memory using localArg = arg.slice() or by looping through its elements.

The place where this specifically comes up in Sylvester is allowing methods to accept both Vector objects (with an elements property) and arrays as arguments. Some methods need to modify the element array after grabbing it from the argument, and the effect of this depends on what you’re trying to do.

function vectorMethod(vector) {
  var V = vector.elements || vector;
  V = Vector.create([3,4,5]);

var foo = $V([9,8,7]);

// foo is $V([9,8,7,10]);

A fairly useless method, but it illustrates the point. The first line just grabs the elements propery if it exists, otherwise it grabs the raw argument. In the second line, V is thus a reference to the array fed into the function. We call push(10) on this reference, which means we push 10 onto the one copy of the array living in memory, so foo mirrors this change. In the third line, V is reassigned. It is no longer a reference to the argument, but a reference to the newly created object — Vector.create([3,4,5]). This does not assign this new object to the function’s argument.

To sum up, then: array/object variables do not store values, they refer to them. Calling methods on these references modifies the return value of all connected references, but reassigning to references leaves all connected variables unchanged.

When to use plain arrays, or, know when you’re repeating yourself

UPDATE: since writing this, Sylvester continues to be tweaked for performance. Some of the below may no longer apply, and you should always do your own testing for performance to find out what works for you. I’m leaving this article up to illustrate the broader points it has to make.

Now that Sylvester allows you to use plain old arrays pretty much anywhere you can use a vector as an argument, how do you decide which one to use? The trick is to know what the the method you’re passing the array/vector to is going to do with it. In the vast majority of cases where vectors are used as arguments, the method is only interested in the vector’s elements. In fact, what often happens is something like this:

someVectorMethod: function(vector) {
  var elements = vector.elements || vector;
  // ...

So elements ends up being an array, whether you pass an array or a vector. You’ll find passing a vector is slightly faster due to the nature of the || operator, and that might lead you to do the following:

// Bad bad code:
var foo = myVector.add($V([2,9,4]));

As well as being ugly, this is really inefficient. You’re calling $V for no reason at all, and the time that takes to run far outweighs the time saving of passing a vector. In this situation,

// Much nicer:
var bar = myVector.add([2,9,4]);

is both cleaner and faster.

It’s only sensible to pass a vector if you’ve already got one in memory as the result of another calculation:

var somePoint = lineA.intersectionWith(PlaneB);

// This is faster...
var foo = myVector.add(somePoint);

// than this...
var bar = myVector.add(somePoint.elements);

For matrices, the reverse is true. Because the Matrix class is written to be able to cope with accepting vectors and matrices as arguments, and to treat vectors as column matrices for multiplication, its methods can’t just use arg.elements || arg to get the necessary array of elements. Matrix arrays are nested, whereas vector ones aren’t. If arg is [9,2,6] (a vector), it must be converted to [[9],[2],[6]] to behave as a column matrix. So, most matrix methods call Matrix.create if they receive a non-matrix argument, and this sorts out the conversion. This means that, if you’ve got a vector you’re going to subject to lots of matrix methods, you’re better off converting it to a matrix beforehand to save all those conversion calls. Even if you’re only calling one matrix method, you’ll find that converting the argument to a matrix before passing it over will save you some time overall. Strange but true.

JavaScript function generation

Justin Palmer shows me something I hadn’t seen before: generating functions dynamically by assigning properties to the window object. He applies it to creating shortcuts for the new DOM Builder in Prototype, but it could be applied to any situation where you have a bunch of similar functions with a few small differences between them. Don’t write them all out: just write a template that does different things depending on the method name. You needn’t use the window object either: you can use any object you want to add methods to. This results in more legible code than if you wrote one function that changed its behaviour based on the arguments, and that’s exactly what Justin does here. You get $form({id: 'foo'}) rather than Element('form', {id: 'foo'}).

It’s not quite Ruby’s method_missing, but it does demonstrate that JavaScript can be flexible enough to mimic the programming style of other languages quite nicely.

Sylvester 0.1.1 released

As I’m sure I’ve mentioned before, Sylvester 0.1.0 was slow. Very very slow in fact. With that in mind, today sees the release of version 0.1.1. Here’s what’s changed.

First and foremost: the vast majority of the framework has got much faster. Some notable highlights (speed comparisons are approximate and may depend on your computer):

  • Matrix.Rotation in 3D is 50 times (yes, fifty, as in 5000%) faster than in 0.1.0. As a result, Vector#rotate is about 60 times faster, and Line#rotate and Plane#rotate are 20 to 30 times faster. 0.1.0 was using some mighty stupid routines to work out rotation matrices; the new implementation uses a quick closed-form algorithm.
  • Reflection in planes is now around 10 times faster for vectors, lines and planes.
  • Matrix equality testing is now 20 times faster and multiplication is 6 times faster (based on 3×3 matrices). Trace calculation is 100 times faster.
  • Many other routines are between 2 and 6 times faster than before.

One change that really helped to speed the whole thing up was removing value-checking from the vector and matrix setElements methods. If you pass non-numeric values to the vector and matrix constructors, you will not get null any more, but you’ll find that method calls on the objects you get back probably break. Why you’d be using non-numeric values I don’t know, but you might find it useful, maybe for doing searches.

There are a couple of new methods:

  • Vector#each can be used to iterate over the elements of a vector.
  • Plane#isPerpendicularTo can be used to test whether two planes are at right angles to each other. I’ve not implemented this for lines, as skew lines make the definition of ‘at right angles’ a little fuzzy.

A bug has been fixed: Matrix#augment was chopping columns off due to some row/column indexes being writted the wrong way round.

And finally, 0.1.0 has some ambiguity regarding use of vectors versus plain arrays. In 0.1.1, any method that takes a vector as an argument will also accept an array. There are situations when you’ll get faster code by taking advantage of this. Methods that take matrices as arguments also accept plain arrays but you need to be careful with this feature. I’ll be writing more about use of arrays shortly.

Go and download Sylvester and let me know how you get on.

Compress JavaScript and CSS without touching your application code

UPDATE: Safari doesn’t seem to play well with this — I’ve modified the code so that Safari is served regular uncompressed files. Also, I realised that the code went about things in a slightly roundabout way, so I’ve shuffled it around a little.

What with the fact that optimising Sylvester for speed is causing its brief, legible internals to balloon into reams of hard-coded arrays and unrolled loops, and the fact the fact that Prototype‘s latest release has put on a few extra chins (1.5.1 is 97kb compared to 1.5.0′s 70kb), I thought it was about time I figured out how to serve my JavaScript like a grown-up. Most of what’s written online seems to be focused on specific platforms (PHP, mostly) and I thought something a little more generally useful was needed.

The general approach to gzipping your web content seems to run as follows:

  • Check the user agent will accept gzip-encoded content.
  • Have your application code filter its output via a gzip function, or have Apache do this for you.
  • Fiddle around with .htaccess to add the correct content type.

The problem with doing all that is that, well, you have to do all that. Compressing assets on the fly is probably not the most efficient way of doing things, and I’m not a huge fan of having your application deal with content delivery — the server should be doing that.

There is a way to have all this handled in a few lines of .htaccess, providing you put in a couple of minutes effort compressing your files by hand. Let’s say you have prototype.js sitting on your server. Gzip it (use 7-zip or similar if you’re running Windows) to give you a file called prototype.js.gz. Stick this in the same directory on your server. If you’re comfotable with the command line, you can just SSH into your box and do, for example:

gzip foo.js -c > foo.js.gz

Then, add the following to .htaccess in your document root (preceed this with RewriteEngine On if you don’t have it somewhere already):

AddEncoding gzip .gz
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)$ $1.gz [QSA,L]

The first line tells the server that files with .gz extensions should be served with the gzip encoding-type, so the browser knows what to do with them. The second line checks that the browser will accept gzipped content — the follwing lines will not be executed if this test fails. We exclude Safari as it doesn’t interpret the gzipped content correctly. We check that this gzipped version of the file exists (fourth line), and if it does, we append .gz to the requested filename.

With this in place, all you have to do is upload gzipped copies of your files to your server and Apache will serve whichever version of the file is most appropriate — you don’t have to change your <script> tags or any application code.

One final platform-specific extra for those of you on Rails: I cannot recommend AssetPackager highly enough. The base package for an app I’m currently working on includes Prototype and various commonly used widgets of mine, some of which are packed. The package totals 125kb as separate files, 99kb using AssetPackager, and a measly 26kb when you throw gzip into the pot.

It’s just FizzBuzz all over again

Interesting (and troubling) discussion over at Coding Horror. It seems that some of Jeff’s readers, the majority of whom I assume to be developers of some sort or another, can’t get their heads around some probability theory. I’ll admit that I needed a little help on my way to solving the problem: reading the second phrasing of the problem in the linked article made it seem much simpler. I think that people generally find it easier to think in terms of whole numbers of people and split them into groups rather than thinking in terms of nebulous probabilities.

That’s fine. My problem is that a group of guys that (I assume) write software for a living can’t follow the solution even when it’s spelled out several times for them, instead arguing that the intuitive answer is correct when it’s really no more than a guess. These are supposed to be smart, well-educated logical people, and it’s really very surprising to see that they don’t get this sort of thing. I’m no genius myself, but give me a pen and paper and I can at least try and follow an argument. I realise that much software development work doesn’t really involve much heavy-duty maths — I just think guys that write code should be able to figure this stuff out. I wonder if these are the same people who can’t write FizzBuzz?