Celerity: first impressions

Last week at the Test Automation Workshop (TAW) here in Australia, the topic of being able to run automated tests ‘headless’ came up, and I mentioned the Celerity project: a headless Watir port. I decided to have a play with Celerity tonight to see how easy it is to get up and running, and also look at it as a way to quickly run Watir scripts.

Installation

The installation was fairly straightforward. You need a Java 6 JDK, as well as the JRuby binaries. You’ll need to update two environment variables, namely add JRuby to your path, and update your JAVA_HOME variable. Installing Celerity involves a JRuby gem: "jruby -S gem install celerity". You can also install it from github, but I don’t know the difference between the two.

Availability of Gems

Ruby gems are supported in JRuby, but only if they don’t use C libraries. This means that Watir won’t work, nor will Roo. I use Roo to define test cases in spreadsheets, so it means I can’t do a comparison of execution times at the moment, at least until I get these test cases out of spreadsheets.

Using Celerity as a simple load testing tool instead

At TAW, Kelvin Ross thought Celerity sounded promising as a load testing tool, considering it was headless and lightweight. I thought I would give a simple google search script a try, running in both Celerity, and Watir, with 30 concurrent users.

Running under Watir

CPU peaked at 100% for the entire run, and each page varied considerable but took on average 10 seconds to load.

Running under Celerity

CPU use was normal, and each page took just over 1 second to load (minimal variance).

Conclusion

Celerity seems a promising way to execute basic load tests using a headless browser. The benefit of Celerity is support for javascript execution in the browser, the downside at the moment is lack of support for some ruby gems. If you could run Watir under JRuby, I could have used a single script.

Scripts Used
Celerity Script

require 'thread'
require "rubygems"
require "celerity"

def test_google
  browser = Celerity::Browser.new
  browser.goto('http://www.google.com')
  browser.text_field(:name, 'q').value = 'Celerity'
  start_time = Time.now
  browser.button(:name, 'btnG').click
  end_time = Time.now
  puts end_time - start_time
  browser.close
end

threads = []
30.times do
  threads << Thread.new {test_google}
end
threads.each {|x| x.join}

Watir Script

require 'thread'
require 'watir'
require 'watir/ie'

def test_google
  browser = Watir::IE.start('http://www.google.com')
  browser.text_field(:name, 'q').value = 'Celerity'
  start_time = Time.now
  browser.button(:name, 'btnG').click
  end_time = Time.now
  puts end_time - start_time
  browser.close
end

threads = []
30.times do
  threads << Thread.new {test_google}
end
threads.each {|x| x.join}

Author: Alister Scott

Alister is an Excellence Wrangler for Automattic.

7 thoughts on “Celerity: first impressions”

  1. I’m really glad to see people considering alternatives besides standard protocol-focussed tools (JMeter, Apache Bench, etc) when it comes to load testing. Celerity is backed by HtmlUnit, a very nice browser simulator. It has a few quirks (ie: doesn’t do concurrent object download, some JS libs break it, etc) but it’s improving every day.

    At BrowserMob, we chose to simply use the power of EC2 to allow us to drive as many real browsers as possible. As you noted, running 30 on a single machine will break it, so we ended up taking a very conservative path: 1 browser per 1 CPU core + 1 GB of RAM. The great thing about EC2 is the prices are still pretty low, so we can pass the savings on to our customers and be one of the lowest cost load testing services out there.

    Like

  2. I find this entry as very useful. I’m looking for a way to run celerity on few machines on the same time to do basic load testing. Does celerity support such a feature? Is this feasible idea?
    Alister – Your post are extremely handy. keep on update with great ideas.

    Many Thanks,

    Guy

    Like

    1. You could simply use a version control system and check out the same script onto multiple machines.
      The script could have a loop from/until a certain time to ensure synchronization.

      Like

    2. Guy,
      My company, BrowserMob, is now rolling out a load testing solution that includes support for something very similar to Celerity. So in addition to our real browser load testing service (which is based on Selenium) and our virtual user testing (which is raw HTTP), we’ll now have this option as well.

      If you’re interested in trying it out (we’re looking for beta testers), please let us know by dropping a note to support@browsermob.com

      Patrick

      Like

  3. Hi,

    This is really useful. I am doing a spike to use celerity for some of our automated tests. However Celerity hangs forever when trying to click a button. Any thoughts on this please? I am using jruby1.5.6 on a fedora box.

    My sample code:

    require ‘rubygems’
    require “celerity”
    require “rspec”
    browser = Celerity::Browser.new

    browser.goto(url)
    browser.text_field(:name, ‘username’).set(‘username’)
    browser.text_field(:name, ‘password’).set(‘password’)
    browser.button(:id=>’signinButton’, :title=>’Sign in’).click
    browser.title.should == expected_title

    Like

Comments are closed.