13 comments
You have a typo on first benchmark. You missed Thing initialization.
You are right! Thanks for pointing that out, just fixed it 🙂
One way to think about the security cost: it’s a side effect of complexity. You point out that readability/searchability is lower (true.) But complexity is also higher – you need more understanding of the code to reason about what it’s doing.
Security cost is a side effect of complexity cost, which you’re expressing as readability/searchability cost.
Thank you 🙂
Thanks for reading 🙂
Interestingly, with the latest TruffleRuby (http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html), the speed cost is essentially non-existent.
Using the benchmark above and adding x.iterations = 2 to make it more stable I get:
…/graalvm-0.25/bin/ruby -v bench.rb
truffleruby 0.25, like ruby 2.3.3 [linux-x86_64] Warming up -------------------------------------- normal method 436.542k i/100ms missing method 2.301M i/100ms defined method 2.202M i/100ms normal method 2.421M i/100ms missing method 2.527M i/100ms defined method 2.950M i/100ms Calculating ------------------------------------- normal method 189.232M (± 8.2%) i/s - 900.678M in 5.006158s missing method 208.035M (± 2.0%) i/s - 1.041B in 5.006995s defined method 216.816M (± 6.2%) i/s - 1.077B in 4.994133s normal method 189.811M (± 5.5%) i/s - 946.680M in 5.009507s missing method 201.205M (± 8.5%) i/s - 995.617M in 5.001431s defined method 207.183M (±11.7%) i/s - 1.018B in 5.004248s Comparison: defined method: 207182797.2 i/s missing method: 201204588.3 i/s - same-ish: difference falls within error normal method: 189811442.8 i/s - same-ish: difference falls within error
Also, the speed is in the order of 200 millions calls per second!
Yeah, that’s interesting. Thanks for sharing!
200M calls/sec – it was optimized out? just couple of guards, and benchmark code overhead?
I also decided to stop using meta code that much for similar reasons (except speed which is already good enough in Ruby) http://blog.arkency.com/2017/02/ruby-code-i-no-longer-write/
I am running an older version of jruby (9.1.2.0) and the normal vs defined method is also within error. Missing method is still about 1.5x times slower. With all the JIT-ing that ruby does I think that accounts for the negligible difference in the dynamic method creation.
Warming up -------------------------------------- normal method 202.307k i/100ms missing method 196.264k i/100ms defined method 213.748k i/100ms Calculating ------------------------------------- normal method 16.209M (±10.7%) i/s - 79.911M in 4.995190s missing method 10.914M (± 9.3%) i/s - 53.973M in 4.998669s defined method 14.672M (±10.5%) i/s - 71.819M in 5.001494s Comparison: normal method: 16208767.3 i/s defined method: 14671736.4 i/s - same-ish: difference falls within error missing method: 10914121.6 i/s - 1.49x slower
Thanks for sharing your results 🙂