Whilst I set up five agents to run these tests in parallel across the different browsers, the build was as slow as its slowest member (much like a buffalo heard) which happened to be IE8 (followed closely by IE9).
Initially the execution times looked something like this:
- Chrome ~50 secs
- Firefox ~1m 10 secs
- IE9 ~4 m 30 secs
- IE8 ~5 mins
I was wondering why on earth it was taking so long, when Simon Stewart pointed out how screenshots work in the IE Driver. I set up the tests to take a screenshot at the end of each scenario, which meant each browser was taking about 18 screenshots per test run.
I didn’t know but the IE Driver maximizes then restores the IE window every time it takes a screenshot, and it also parses the entire DOM to take a screenshot. This is why it was taking so long to execute the tests.
I removed the screenshots from the IE runs and was able to reduce both IE8 and IE9 to just over 2 minutes execution time. Not the best, after all it’s over twice as slow as Chrome, but better than 5 minutes previously!
In the future, I’ll avoid taking any screenshots using IE Driver wherever possible.