The Hidden Costs of Metaprogramming

Metaprogramming sounds like a very fancy word, but is it any good?

It can be useful, but many people don’t realize that using metaprogramming has some costs.

Just so we are on the same page…

What is metaprogramming exactly?

I define metaprogramming as using any method that:

• Alters the structure of your code (like define_method)
• Runs a string as if it was part of your actual Ruby code (like instance_eval)
• Does something as a reaction to some event (like method_missing)

So what are the costs of metaprogramming? I classify them into 3 groups:

• Speed
• Readability
• Searchability

Note: You could also say that there is a fourth group: Security. The reason for that are the eval methods, which don’t make any kind of security checks on what is being passed in. You have to do that yourself.

Let’s explore each of those in more detail!

Speed

The first cost is speed because most metaprogramming methods are slower than regular methods.

Here is some benchmarking code:

require 'benchmark/ips'

class Thing
  def method_missing(name, *args)
  end

  def normal_method
  end

  define_method(:speak) {}
end

t = Thing.new

Benchmark.ips do |x|
  x.report("normal method")  { t.normal_method }
  x.report("missing method") { t.abc }
  x.report("defined method") { t.speak }

  x.compare!
end

The results (Ruby 2.2.4):

normal method:   7344529.4 i/s
defined method:  5766584.9 i/s - 1.34x  slower
missing method:  4777911.7 i/s - 1.54x  slower

As you can see both metaprogramming methods (define_method & method_missing) are quite a bit slower than the normal method.

Here is something interesting I discovered…

The results above are from Ruby 2.2.4, but if you run these benchmarks on Ruby 2.3 or Ruby 2.4 it looks like these methods are getting slower!

Ruby 2.4 benchmark results:

normal method:   8252851.6 i/s
defined method:  6153202.9 i/s - 1.39x  slower
missing method:  4557376.3 i/s - 1.87x  slower

I ran this benchmark several times to make sure it wasn’t a fluke.

But if you pay attention & look at the iterations per second (i/s) it seems like regular methods got faster since Ruby 2.3. That’s the reason method_missing looks a lot slower 🙂

Readability

Error messages can be less than helpful when using the instance_eval / class_eval methods.

Take a look at the following code:

class Thing
  class_eval("def self.foo; raise 'something went wrong'; end")
end

Thing.foo

This will result in the following error:

(eval):1:in 'foo': 'something went wrong...' (RuntimeError)

Notice that we are missing the file name (it says eval instead) & the correct line number. The good news is that there is a fix for this, these eval methods take two extra parameters:

• a file name
• a line number

Using the built-in constants __FILE__ & __LINE__ as the parameters for class_eval you will get the correct information in the error message.

Example:

class Thing
  class_eval(
    "def foo; raise 'something went right'; end",
    __FILE__,
    __LINE__
  )
end

Why isn’t this the default?

I don’t know, but it’s something to keep in mind if you are going to use these methods 🙂

Searchability

Metaprogramming methods make your code less searchable, less accessible (via worse documentation) & harder to debug.

If you are looking for a method definition you won’t be able to do CTRL+F (or whatever shortcut you use) to find a method defined via metaprogramming, especially if the method’s name is built at run-time.

The following example defines 3 methods using metaprogramming:

class RubyBlog
  def create_post_tags
    types = ['computer_science', 'tools', 'advanced_ruby']

    types.each do |type|
      define_singleton_method(type + "_tag") { puts "This post is about #{type}" }
    end
  end
end

rb = RubyBlog.new

rb.create_post_tags
rb.computer_science_tag

Tools that generate documentation (like Yard or RDoc) can’t find these methods & list them.

These tools use a technique called “Static Analysis” to find classes & methods. This technique can only find methods that are defined directly (using the def syntax).

Try running yard doc with the last example, you will see that the only method found is create_post_tags.

It looks like this:

There is a way to tell yard to document extra methods, using the @method tag, but that is not always practical.

Example:

class Thing
  # @method build_report
  define_method(:build_report)
end

Also if you are going to use a tool like grep, ack, or your editor to search for method definitions, it’s going to be harder to find metaprogramming methods than regular methods.

“I don’t think Sidekiq uses any metaprogramming at all because I find it obscures the code more than it helps 95% of the time.” – Mike Perham, Creator of Sidekiq

Conclusion

Not everything is bad about metaprogramming. It can be useful in the right situations to make your code more flexible.

Just be aware of the extra costs so you can make better decisions.

Don’t forget to share this post if you found it useful 🙂