Running your watir-webdriver tests in the cloud, for free!

  • What if you could run unlimited Watir WebDriver tests in the cloud? Check.
  • What if the Watir WebDriver tests would run automatically as soon as you pushed a change to github? Check.
  • What if you would have a full visual history of results with embedded screenshots on failure? Check.
  • What if all of this was free?* Checkmate.

I’ve spent a bit of time over the last week working out how to do this. Here are the basics of what you need to do:

And here’s the detailed instructions.

Set up an Amazon EC2 micro instance running Ubuntu.

  1. First you need to sign up for an Amazon AWS account. This means you’re eligible for a free-tier micro instance for a year.
  2. Once you have an account set up, you need to launch a new instance. I found a free tier eligible Ubuntu image (11.04 Natty 64 bit desktop) and launched that.
  3. You will also want to create an elastic IP and associate it to your instance so that if you reboot your machine, you will have the same IP address. This is done through the AWS console under Elastic IPs.
  4. While you’re here, you’ll want to edit your machine’s security group and open up port 22 for SSH, and 80 for HTTP.
  5. This gives you secure shell (SSH) access to this machine using the provided key, and user ‘ubuntu’:
    ssh -i your-key-name.pem ubuntu@your-ip-address
  6. Everything you will do to configure this machine will be through this SSH session, so polish up your unix command line skills!

Set up Jenkins on your machine

There is a useful page for installing Jenkins on Ubuntu.

wget -q -O - | sudo apt-key add -
sudo sh -c 'echo deb binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo aptitude update
sudo aptitude install jenkins

Make Jenkins available on port 80 so that you don’t need to specify port

Jenkins installs by default on port 80. Ubuntu won’t let applications run on port 80 unless they’re running as root, so it’s best to set up an Apache 2 proxy to port 80 to 8080.

sudo aptitude install apache2
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod vhost_alias
sudo a2dissite default

Then create a file called jenkins in /etc/apache2/sites-available

	ServerAdmin webmaster@localhost
	ServerAlias ci
	ProxyRequests Off
		Order deny,allow
		Allow from all
	ProxyPreserveHost on
	ProxyPass / http://localhost:8080/

Then run the following commands:

sudo a2ensite jenkins
sudo apache2ctl restart

Password Protect Jenkins

You should go to your Jenkins site (accessible directly at your instance’s IP address through a web browser), and create an account, and then configure the security of Jenkins.

Install Jenkins Plugins

You will need to install the following Jenkins plugins

  • Github: to integrate to Github SCM
  • Rake: to run ruby rake tasks that run Watir-WebDriver tests
  • Green balls: because blue balls are just plain wrong

Install RVM for the Jenkins user

First we’ll need to install git

sudo apt-get install git

Jenkins will need to be able to run ruby, so we’ll install RVM as the Jenkins user.

To run as the jenkins user, we’ll use the sudo command, with the -Hiu arguments to load the home directory and bash profile:

sudo -Hiu jenkins

Once we are user Jenkins, we’ll install RVM using Git.

bash < <(curl -s

Now we need to work out what Ubuntu packages Ruby needs, which is easily done via RVM.

rvm notes

which gives me something like

For Ruby (MRI, Rubinius, & REE)  you should install the following OS dependencies:
/usr/bin/apt-get install build-essential bison openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake

So, we can log-out as the Jenkins user (control-D) and install the following as ubuntu

sudo apt-get install build-essential bison openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake

Once we’ve done this, we’ll want to run the following as the Jenkins user (
sudo -Hiu jenkins) to install Ruby 1.9.2.

rvm pkg install zlib
rvm install 1.9.2 --with-zlib-dir=$rvm_path/usr

Running headless Watir-WebDriver tests

I choose a desktop version of Ubuntu, so it’ll already have Firefox installed, but if you don’t, you can install it by:

sudo apt-get install firefox

To run our Watir WebDriver tests headlessly using the headless gem, we’ll need xvfb

sudo apt-get install xvfb

Configuring Jenkins to run tests via Rake

You add a new build in Jenkins where you can specify the github repository location.

As we’ve installed the rake plugin, we can configure a new Jenkins project to use an RVM ruby install (in my case ruby-1.9.2-p290@watirmelon-cucumber).

I simply set up a default task in rake, which runs all my cucumber tests. This generates a results.html file which is captured as an artifact, and also creates and captures junit xml results, which are used to show test summary information.I also capture any file created under the ‘screenshots’ directory.

Summary and Outcome

I have set up both my WatirMelonCucumber and EtsyWatirWebDriver projects on

My Jenkins Dashboard looks something like this:

Please feel to leave a comment below and let me know what you think.

* Free for one year using an free tier EC2 micro instance

Author: Alister Scott

Alister is an Excellence Wrangler for Automattic.

11 thoughts on “Running your watir-webdriver tests in the cloud, for free!”

  1. Thanks Alister for sharing this…its really good!!! This is something I wanted to try over the bank holiday weekend as well and this post is just in time…

    Excellent stuff!!!



  2. It’s very common to use Xvfb to run browser tests on headless servers. However, for those hard to debug failures I often find it helpful to to be able to watch the tests run, and therefore prefer to use Xvnc.


    1. Couldn’t you just run it locally and see what the problem is? That’s how I would rather debug.
      How do you VNC into a publicly accessible server? Tunnel VNC through SSH?


  3. Alister, this was a lot of fun to work through! I could just use this as a challenge when interviewing candidates. Just a couple things came up:

    Elastic IP addresses are $0.01 per hour.

    I added rules to the default security group, not to a new group. As an AWS first-timer, took me a bit to figure out I should just use the default group.

    I used the public DNS name, found it worth noting the DNS name and the actual IP address probably won’t match. In the AWS Console, I right-clicked on the instance, and selected Connect, and it generated the command I should run.

    This jenkins file here: worked on the first try, Did the tags get parsed out of the example?

    Thanks again, for blogging about cool things,



  4. Alister, I think you have to be careful when you say ‘for free’. Indeed, I am pretty sure that the image you recommend using is an EBS backed one. I followed your steps ages ago (really great stuff BTW) – mainly because I wanted to look at AMIs, how they work, and how you can script some of the config – and realised that I had a new EBS snapshot when I created my Jenkins AMI. Later I got rid of everything and my AWS account has just past the date when I can use a free EC2 again, so I can’t try now with the same image. But you might want to check your AWS bills and the AWS Console dashboard to see if there is an EBS volume attached to your Jenkins EC2. I am pretty sure there should be (probably a 8GB one), in which case you have to pay for it, regardless of whether you use your Jenkins or not. Cheers. JR


      1. Interesting, out of curiosity did you try to remove the volume and restart the EC2, cos it should not work without the volume.
        BTW, I found that Jenkins now has a plugin that spins off EC2 slaves, I will try it!


  5. Thanks mate. Much appreciated. Saved me literally days of messing around. If you find yourself in Sydney, look me up and I’ll buy the beer.

    One thing I had to do when installing rvm as the jenkins user was to manually add rvm to my bash profile, using this script pilfered from the rvm website:

    user$ echo ‘[[ -s “$HOME/.rvm/scripts/rvm” ]] && . “$HOME/.rvm/scripts/rvm” # Load RVM function’ >> ~/.bash_profile

    Additionally, as Željko noted, your Apache2 config snippet is missing the XML tags.


  6. This is great. I have a beginner question though… when you talk about your rake file with defaults generating some results.html, etc. do you have any info on what that looks like? I’m kinda lost there.


  7. I tried this over the past few days and I hit a problem I just couldn’t crack…. so I went to my local box (instead of AWS) which is windows… and the same thing (slightly different error saying the same thing is thrown there as well…. and yet it works locally from the command line.

    Here’s the problem. I got everything working with your tutorial. But when I try and build the cucumber tests in ubunutu I would get: unable to obtain stable firefox connection in 60 seconds.

    I tried every suggestion I found online… upgrade webdriver, upgrade rake, upgrade the browser, downgrade all of the above…. no matter what I try, this error blocks me.

    So I went to my local windows to try and debug. I can run rake run locally and it works perfectly. But when Jenkins does it, I get this error on the first Cucumber test:
    Given a user clicks the features tab # features/step_definitions/features_tests.rb.rb:1
    Timeout::Error (Timeout::Error)
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/protocol.rb:146:in `rescue in rbuf_fill’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/protocol.rb:140:in `rbuf_fill’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/protocol.rb:122:in `readuntil’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/protocol.rb:132:in `readline’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:2562:in `read_status_line’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:2551:in `read_new’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:1319:in `block in transport_request’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:1316:in `catch’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:1316:in `transport_request’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:1293:in `request’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:1286:in `block in request’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:745:in `start’
    C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:1284:in `request’
    ./features/step_definitions/features_tests.rb.rb:3:in `/^a user clicks the features tab$/’

    So I cut the rake out entirely and just did a call to run cucumber features… same result.

    I’ve also tried Windows suggestions, such as making sure the jenkins service login is allowed to interact with desktop objects… no change. I’ve seen others in the same state/error with no resolution.

    So it’s not the OS, it’s not the Rake File… not the webdriver, not the browser… any ideas?


Comments are closed.