When bugs become features

Been a while since we had a good rant, what with Zed calming down and Giles talking about winding up his blog. I had a great post lined up on browser performance and why all the advice you’ve heard is wrong and so on and so on, and I’d gathered tons of data and sunk days of work into it and then I found out there was a performance bottleneck in one of the tests that had wildly skewed the results. Having fixed it, I was left with nothing controversial to say, which is a shame, though the whole process did help me make a pretty big performance boost to one of my projects. Harder to make a song and dance about, but better in the end.

So, to make up for the absence of a post that would have left my face largely covered in egg, I’m going to rant about this:

$ ~/.multiruby/install/1.8.6-p368/bin/irb
>> module A; end
>> class C; include A end
>> module B; def foo; "B#foo" end end
>> module A; include B end
>> class D; include A; end
>> D.new.foo
=> "B#foo"
>> C.new.foo
NoMethodError: undefined method `foo' for #<C:0xb7e0d834>
	from (irb):7
	from :0

$ ~/.multiruby/install/1.8.7-p160/bin/irb 
>> module A; end
>> class C; include A end
>> module B; def foo; "B#foo" end end
>> module A; include B end
>> class D; include A; end
>> D.new.foo
=> "B#foo"
>> C.new.foo
NoMethodError: undefined method `foo' for #<C:0xb7e47a84>
	from (irb):7
	from :0

$ ~/.multiruby/install/1.9.1-p129/bin/irb 
>> module A; end
>> class C; include A end
>> module B; def foo; "B#foo" end end
>> module A; include B end
>> class D; include A; end
>> D.new.foo
=> "B#foo"
>> C.new.foo
NoMethodError: undefined method `foo' for #<C:0x8b5fb50>
	from (irb):7
	from ~/.multiruby/install/1.9.1-p129/bin/irb:12:in `<main>'

$ ~/projects/rubinius/bin/rbx
>> module A; end
>> class C; include A end
>> module B; def foo; "B#foo" end end
>> module A; include B end
>> class D; include A; end
>> D.new.foo
=> "B\#foo"
>> C.new.foo
NoMethodError: No method 'foo' on an instance of C.
   from Kernel(C)#foo (method_missing) at kernel/delta/kernel.rb:47
   from Object# (__eval__) at (irb):7

$ jirb
>> module A; end
>> class C; include A end
>> module B; def foo; "B#foo" end end
>> module A; include B end
>> class D; include A; end
>> D.new.foo
=> "B#foo"
>> C.new.foo
NoMethodError: undefined method `foo' for #<C:0x5d855f>

That’s five current major Ruby implementations (well, strictly four different VMs), all with the same bug. And it is a bug. It’s recognised as a problem, and I doubt anyone would design a system to produce results like this, where the inheritance tree looks different depending on who you ask and when you ask them:

>> C.ancestors
=> [C, A, Object, Kernel]
>> A.ancestors
=> [A, B]
>> C.send :include, A
>> C.ancestors
=> [C, A, B, Object, Kernel]

What I find curious is that this same problem has crept into four different VMs, as though put in deliberately to match the original buggy version. All these implementations have the same ancestry problems, and all of them make eigenclasses (those things you get by doing class << obj; self; end) second-class citizens that don’t appear in ancestor lists. Yes, they were initially considered an implementation detail, which I suppose is why the syntax for getting at them is so weird, but once the feature has been exposed people start poking around with it and expect it to work across platforms. You start getting strange diagrams that confuse people because the things they seem to imply about Module#ancestors don’t seem to hold true in practise. Folk argue on mailing lists about whether mixins are referenced or copied, whether they affect ancestry, method lookups, singleton methods, all three or none of the above. And don’t even get me started on this:

>> class C; end
>> class D; include C; end
TypeError: wrong argument type Class (expected Module)
	from (irb):2:in `include'
	from (irb):2
	from :0

>> module Kernel
>>   def eigenclass
>>     class << self; self; end
>>   end
>> end

>> eigenclass
=> #<Class:#<Object:0xb7dea94c>>

>> eigenclass.ancestors
=> [Object, Kernel]

>> eigenclass.new
TypeError: can't create instance of virtual class
	from (irb):8:in `new'
	from (irb):8
	from :0

>> Class.new(eigenclass)
TypeError: can't make subclass of virtual class
	from (irb):9:in `initialize'
	from (irb):9:in `new'
	from (irb):9
	from :0

Now really it’s not fair of me to slag off the guys writing these runtimes, since they’re almost certainly smarter than me, what they’re doing is very hard, I have almost no idea what I’m talking about and the problem I’m ranting about is of barely any practical importance to anybody. But I do think that if you were starting a new Ruby you might try to scrub some of the existing design warts off the language and make it more consistent.

I’m going to gloat for a second and tell you that I’ve just managed to fix some of these problems. In JS.Class, you can’t instantiate eigenclasses because they’re not classes (they’re modules), but you can happily use them as mixins:

$ rhino
Rhino 1.7 release 1 2008 10 20
js> load('js.class/build/min/core.js');

js> A = new JS.Module('A');
js> C = new JS.Class('C', { include: A });
js> B = new JS.Module('B', { foo: function() { return 'B#foo' } });
js> A.include(B);
js> D = new JS.Class('D', { include: A });
js> new D().foo();
B#foo
js> new C().foo();
B#foo

js> C.ancestors().map(function(m) { return m.displayName })
Kernel,B,A,C

js> C = new JS.Class();
js> D = new JS.Class({ include: C });
js> C.define('foo', function() { return 'C#foo' });
js> new D().foo()
C#foo

js> var obj = new C();
js> obj.extend({ bar: function() { return 'obj.bar' } });
js> obj.bar()
obj.bar
js> D.include(obj.__eigen__());
js> new D().bar();
obj.bar

This is in the 2.1 release, which should be coming out very very soon indeed. 2.0 came out last August, and it took about nine months for me to get around to fixing this. But, in that whole time, JS.Class never had the weird ancestry problems described above because it was designed from the outset to appear as consistent as possible; that’s why all its inheritance semantics, including classical parent-child inheritance and singleton methods, is implemented using modules and mixins. I hesitate to say Ruby could be this way because truthfully I don’t have the experience to know what kind of problems it would cause. Because of its internal design, my implementation was fairly straightforward but had performance problems, which are largely mitigated by caching ancestries and method lookups. I hear this is the reason Ruby still has the problem, though I’ve not seen any hard data on how much of a performance hit it would cause.

But I will say that Ruby should be this way, and it would be easier to understand if it were. For inheritance, you just have a big tree of mixins – every module has a list of methods and can include zero or more other modules – and some rules for scanning the tree for method implementations. If this sounds appealing to you, here’s something to get you started. Unfortunately, it looks like Ruby’s warts have made it into the spec, so it’s not likely they’ll be fixed any time soon.