IncludeByDefault progress

IncludeByDefault, as mentioned in my last post, hit some snags with ActiveRecord generating duplicate table aliases when doing cascaded includes, e.g.

Tag.find(8).posts.find(:all, :include => :tags)

So, I set out to work around it, only to run into further problems. I went with option C: let find operations get all the way to the database, and then catch StatementInvalid if it is thrown and convert problematic :includes to :joins fragments by hand.

But, turns out Rails won’t let you use the :joins option on finds scoped on HABTM associations (like the one above), meaning I had to hack support for that in order to get my workaround for the duplicate naming bug to work. Even then, eager loading refuses to work properly – asking for the tags of any of the returned posts will go and fetch them from the database, even though the find operation included JOIN fragments to load the tags.

The other complication is that, if you have duplicate many-to-many links in your database, Rails returns incorrect result sets if you use :limit and :include, as long as the :include contains no has_many or has_and_belongs_to_many associations. Which means that converting troublesome :includes (those that raise StatementInvalid in the example up top, and are to-many associations) to :joins triggers this bug. I had to hack in a way for ActiveRecord to remember to select unique records properly, but this still only works as long as you :include suitable associations to begin with.

I have been round and round in circles intercepting various bits of the ActiveRecord call stack trying to get rid of this, but it’s whack-a-mole: fixing the eager loading thing sets off the unique records problem, or makes the call stack retry the database calls too many times, or drops the association on the floor so you can’t even read it never mind eager load it. I’ve got the plugin to a state where:

  • If your :include doesn’t cause any SQL problems, it runs just fine.
  • If it raises StatementInvalid, we’ll rewrite it. The association causing the problem probably won’t get eager loaded, but your query will at least run.
  • You will avoid the duplicate links problem as long as your :include includes a to-many association.

All these apply whether the :include is added using include_by_default or explicitly in your find call – the plugin will catch the exceptions either way. If anyone knows a way to get all this to work without actually including large parts of rewritten Rails (so far there’s one actual method overwrite – the rest are wrappers that catch exceptions) then I’d love to hear about it.