RubyGuides
Share this post!

The Hidden Costs of Metaprogramming

Metaprogramming sounds like a very fancy word, but is it any good?

It can be useful, but many people don’t realize that using metaprogramming has some costs.

Just so we are on the same page…

What is metaprogramming exactly?

I define metaprogramming as using any method that:

  • Alters the structure of your code (like define_method)
  • Runs a string as if it was part of your actual Ruby code (like instance_eval)
  • Does something as a reaction to some event (like method_missing)

So what are the costs of metaprogramming? I classify them into 3 groups:

  • Speed
  • Readability
  • Searchability

Note: You could also say that there is a fourth group: Security. The reason for that are the eval methods, which don’t make any kind of security checks on what is being passed in. You have to do that yourself.

Let’s explore each of those in more detail!

Speed

The first cost is speed because most metaprogramming methods are slower than regular methods.

Here is some benchmarking code:

The results (Ruby 2.2.4):

As you can see both metaprogramming methods (define_method & method_missing) are quite a bit slower than the normal method.

Here is something interesting I discovered…

The results above are from Ruby 2.2.4, but if you run these benchmarks on Ruby 2.3 or Ruby 2.4 it looks like these methods are getting slower!

Ruby 2.4 benchmark results:

I ran this benchmark several times to make sure it wasn’t a fluke.

But if you pay attention & look at the iterations per second (i/s) it seems like regular methods got faster since Ruby 2.3. That’s the reason method_missing looks a lot slower πŸ™‚

Readability

Error messages can be less than helpful when using the instance_eval / class_eval methods.

Take a look at the following code:

This will result in the following error:

Notice that we are missing the file name (it says eval instead) & the correct line number. The good news is that there is a fix for this, these eval methods take two extra parameters:

  • a file name
  • a line number

Using the built-in constants __FILE__ & __LINE__ as the parameters for class_eval you will get the correct information in the error message.

Example:

Why isn’t this the default?

I don’t know, but it’s something to keep in mind if you are going to use these methods πŸ™‚

Searchability

Metaprogramming methods make your code less searchable, less accessible (via worse documentation) & harder to debug.

If you are looking for a method definition you won’t be able to do CTRL+F (or whatever shortcut you use) to find a method defined via metaprogramming, especially if the method’s name is built at run-time.

The following example defines 3 methods using metaprogramming:

Tools that generate documentation (like Yard or RDoc) can’t find these methods & list them.

These tools use a technique called “Static Analysis” to find classes & methods. This technique can only find methods that are defined directly (using the def syntax).

Try running yard doc with the last example, you will see that the only method found is create_post_tags.

It looks like this:

yard example

There is a way to tell yard to document extra methods, using the @method tag, but that is not always practical.

Example:

Also if you are going to use a tool like grep, ack, or your editor to search for method definitions, it’s going to be harder to find metaprogramming methods than regular methods.

“I don’t think Sidekiq uses any metaprogramming at all because I find it obscures the code more than it helps 95% of the time.” – Mike Perham, Creator of Sidekiq

Conclusion

Not everything is bad about metaprogramming. It can be useful in the right situations to make your code more flexible.

Just be aware of the extra costs so you can make better decisions.

Don’t forget to share this post if you found it useful πŸ™‚

13 comments
Adis says a few months ago

You have a typo on first benchmark. You missed Thing initialization.

    Jesus Castello says a few months ago

    You are right! Thanks for pointing that out, just fixed it πŸ™‚

Noah says a few months ago

One way to think about the security cost: it’s a side effect of complexity. You point out that readability/searchability is lower (true.) But complexity is also higher – you need more understanding of the code to reason about what it’s doing.

Security cost is a side effect of complexity cost, which you’re expressing as readability/searchability cost.

Nasos Psarrakos says a few months ago

Great post Jesus.

Keep it up πŸ˜‰

    Jesus Castello says a few months ago

    Thank you πŸ™‚

Manish johari says a few months ago

Great post Jesus, Thank you for sharing.

    Jesus Castello says a few months ago

    Thanks for reading πŸ™‚

Benoit Daloze says a few months ago

Interestingly, with the latest TruffleRuby (http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html), the speed cost is essentially non-existent.
Using the benchmark above and adding x.iterations = 2 to make it more stable I get:

…/graalvm-0.25/bin/ruby -v bench.rb

Also, the speed is in the order of 200 millions calls per second!

    Jesus Castello says a few months ago

    Yeah, that’s interesting. Thanks for sharing!

    funny_falcon says a few months ago

    200M calls/sec – it was optimized out? just couple of guards, and benchmark code overhead?

Robert Pankowecki says a few months ago

I also decided to stop using meta code that much for similar reasons (except speed which is already good enough in Ruby) http://blog.arkency.com/2017/02/ruby-code-i-no-longer-write/

Jay says a few months ago

I am running an older version of jruby (9.1.2.0) and the normal vs defined method is also within error. Missing method is still about 1.5x times slower. With all the JIT-ing that ruby does I think that accounts for the negligible difference in the dynamic method creation.

    Jesus Castello says a few months ago

    Thanks for sharing your results πŸ™‚

Comments are closed