Erik's Engineering

something alliterative

Ruby Start-up Times

edit:

I got the reasons for things slowing down wrong. Things get slower, but not for the reasons I thought. José Valim explains what's actually going on. I can confirm that if I take the largest of my test apps and nuke all the helpers it will speed up to roughly the speed of the 200 scaffold version, though the response times are pretty inconsistent. If I clear out the routes file, startup times and response times go down to the same level as the tiniest 1 scaffold app.


This is mainly a follow-up to my previous post.   Several people asked about the performance of Ruby Enterprise Edition, and one person mentioned start-up times.

I thought I'd make use of the test applications I'd created and gather a little more data, this time looking at start-up times and including REE.

REE Dev performance

Here's a quick update of the graph from my previous post with REE added.   You can see that in this respect it performs almost identically to Ruby 1.9.2.  jRuby is scaling a lot better than the others, though MRI outperforms it for smaller applications.

 

rails-scaling-ree-1000

 

Startup Times

Startup times are a bit different.  I timed these using a quick test script:

#!/bin/bash
echo puts Time.now > test_script
time rails runner test_script
time rails runner test_script
time rails runner test_script

 

You can see that this basically grabs the time taken to start up the rails environment, do a tiny bit of work, and shut down.  I'm assuming that shutdown is near-instantaneous so that the bulk of what we're observing is start up time.

Here's what I came up with:

rails-startup-performance-graph

It looks like jRuby isn't really in the running.  For a bare-minimum rails app, jRuby needs 5.5 seconds, while ruby 1.9.2 needs only 1 second.  jRuby then scales better than Ruby 1.8.7, but you have to get really big before jRuby starts actually being faster.  Even then it's not as fast as Ruby 1.9.2.  REE starts out mirroring Ruby 1.8.7, but it looks like towards the end it's doing noticeably worse.  That might just be noise in the data, though.  We'd probably have to go up to 1500 or 2000 models to say for sure.

All in all, it looks like Ruby 1.9.2 is the king of startup times.   It seems that start up time scales differently than dev mode response time, and that a workflow that avoids having to re-start the ruby interpreter is a big win.

A Note About Scale

I'm looking at how things change when you go from a tiny app to a really huge one. The smallest app I'm talking about takes up about 600k on disk, and the largest is about 55M. 55M is absolutely huge, especially since it doesn't have any images or other assets bulking it up.

# scaffolds size (k)
1 596
50 3460
100 6268
200 11880
300 17504
500 28716
750 42732
1000 56756

No matter how you slice it, 56M is a lot of code. Hopefully this will make it a little easier to compare my numbers which are based on synthetic test applications to real-world applications.

Published on 19/02/2011 at 18h15 .

  • By Hongli Lai 20/02/2011 at 08h30

    Label your graphs. What are the axes?


  • By fred 20/02/2011 at 13h49

    Erik, may want to update for p180. Though I doubt there will be a difference.

    1.9.2 rocks. Ruby-head would have been nice to see on this (1.9.3)


  • By Eric Hodel 20/02/2011 at 16h09

    What do your graphs mean? The axes aren’t labeled.


  • By Jay Godse 21/02/2011 at 13h02

    These numbers are consistent with what I have read about the performance of MRI 1.9.2.

    Do you have a chart for memory usage of these platforms as well?


  • By Erik 27/03/2011 at 14h45

    Axes: The vertical axes are always seconds, the horizontal axes are always the number of scaffolds in the app.

    p180/HEAD: I’m afraid I just don’t have time to keep adding more cases, especially when Jose Valim was nice enough to explain exactly what’s going on.

    Memory usage: :( It can really make a big difference, but the problems that inspired this didn’t include memory usage so I didn’t think to capture that.

    And now to investigate why I didn’t get email notification of comments.


Comment Ruby Start-up Times

Trackbacks are disabled

Powered by Typo – Thème Frédéric de Villamil | Photo Glenn