AMA: CodeceptJS support for Safari and IE?

Sahana Asks…

We area VOD startup and we have web app, mobile apps and TV apps. I am writing acceptance tests for web app now and chose codeceptjs framework since we have our website’s front end code in Javascript. We have dockerised the processes and docker images for codeceptjs webdriver IO is availble only for chrome and firefox browsers. How can I handle Safari , Internet Explorer browsers ? Looks like CodeceptJs does not support IE and Safari browsers. Do you have any suggestion?

My Response…

I’ve never personally found the return on investment of getting automated tests running across Internet Explorer and  Safari to be worthwhile as in my experience this took more effort than the bugs it found. So I personally stick to running our full e2e test suite in our most used browser (Chrome) and supplementing this with exploratory testing on all other browsers.

In saying that the reason you won’t be able to use Docker containers for these purposes is that they’re Linux and Internet Explorer requires Microsoft Windows and Safari requires Apple macOS to be able to run. To be able to use these for your existing automated tests you can sign up to a on-demand browser service like SauceLabs and use the remote WebDriver protocol to execute your tests.

Do you REALLY need to run your WebDriver tests in IE?

I recently read that Microsoft are now on board to officially support Selenium WebDriver from Internet Explorer (IE) 11+

Whilst I welcome the news, I try to avoid running WebDriver tests in Internet Explorer completely for the following reasons:

  • Internet Explorer is a very non-testable browser. Whilst everyone agrees testability of your app is paramount, testability of its run-time container, the browser, is equally important. Settings such as security zones, proxies and auto-complete in IE must be manually configured on each machine instead of being programmatically specified by profiles in Firefox and Chrome; and
  • Because IE has historically been so hard to test, WebDriver’s support for IE is much less mature and much less stable and efficient than Firefox and Chrome

The only way automated UI tests can succeed (and the chances of success aren’t high to begin with), is if they are fast and consistent. WebDriver against IE is neither (I see it more of a problem with IE than WebDriver). So if you want to use WebDriver, don’t test against IE, test against Firefox or Chrome.

But, In my role as a consultant, I continually hear managers say that we must run our WebDriver automated tests in Internet Explorer. There’s usually one or two reasons given:

  1. Our web app is for internal staff only and our only supported browser is IE (which is usually IE8); and/or
  2. Our web app (or the one we pay for) has been specifically coded to work only in IE and therefore it’s not possible to test in another browser.

You need to explain that your WebDriver automated tests aren’t the only tests you’ll run against your app. In a corporate environment (such as those who only support IE8), chances are you’ll have a period of business acceptance testing or user acceptance testing. This will be conducted by users in the browser they use, so this straight away mitigates the risk of only running your automated tests against a non-IE browser.

From my experience testing many applications against older versions of IE, the one thing that doesn’t work well (and causes web apps to break) is not the HTML but JavaScript support. If your app contains a decent amount of JavaScript you could write some JavaScript tests in a tool like js-test-driver and run these automatically against older versions of IE automatically. That way you can be assured your JavaScript is working without having to deal with IE/WebDriver issues (and slow running tests).

As for applications specifically coded to work in IE. Web standards exist for a reason and in my opinion it’s crazy to develop a web app that is tied to the implementation of a browser by a single vendor. Microsoft made IE11 purposely report itself to a web server as not being IE so Microsoft can avoid this exact situation happening in the future.

Chances are if your app is hard-coded to only work in IE then it won’t work in IE11 anyway. If it works in IE11, then it’ll work in Chrome and Firefox as they all follow web standards, and you can run your WebDriver tests reliably now.

I believe you’re better off not having any automated UI tests if you there’s a mandate in place that you must run them against IE. If you can’t automatically test your app in Firefox or Chrome, I believe you’re better off spending your time manually testing your app in IE than trying to maintain a test suite that will never be efficient or reliable.

Elegantly handling basic browser authentication with Watir

In programming, I find there’s little that compares to the feeling of deleting code; like finding a one or two-line solution that means you can remove a good-sized chunk of code no longer needed.

A fellow ThoughtWorker (who I shall call the Awesome-est Tom™) showed me a solution to handling the basic browser authentication in Internet Explorer, which effectively meant I could delete lines and lines of ruby code that called the AutoIt dll to handle the modal authentication dialog.

Internet Explorer Basic Authentication Dialog
Internet Explorer Basic Authentication Dialog

The code I had was loosely based upon the solutions available on the Watir wiki, all which roughly use the same technique of locating and authenticating the authentication dialog using AutoIt. Because the dialog is modal, the code I had used to run in a separate thread, and this caused me problems with debugging, and I find if I can avoid AutoIt, then my Watir tests run a lot more reliably.

Tom’s Occam’s Razor solution simply embeds the username and password in the URL that usually triggers an authentication dialog.

Instead of using:

Watir::Browser.start "http://yoururl.com"

you use:

Watir::Browser.start "http://username:password@yoururl.com"

This means you don’t see the authentication dialog at all! No more messy AutoIt code. The only tricky part is ensuring that Internet Explorer allows you to do this. Since Internet Explorer 6, this has been disabled by default, but it’s a simple matter of enabling the functionality by setting two registry keys.

This can easily be done running a .reg file you can create, that sets the required values:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME_PASSWORD_DISABLE]
"iexplore.exe"=dword:00000000
"explorer.exe"=dword:00000000

You then run this file from the command line: regedit.exe /s IESecurityURL.reg and Robert is your father’s brother.