Watir is a great library, but to use it to its full potential you need to create your own framework to drive it.
As with any context driven approach, the framework/solution you decide to implement has to suit your own environment. One approach that I have used successfully in multiple projects (with tweaks for each) is the business driven approach.
All the software projects I have worked on have had one main purpose, that is to support a business need. This business need may be to allow people to easily travel from country to country, or in contrast, allow enthusiasts to efficiently buy books online. Creating a test suite that is focussed around these real business activities will clarify the link from these to the the user tests you are writing and running. This also compliments a risk based testing approach as it is often easy to get business people to express the important and risky areas of the business, this being what will be tested first.
The concept of my framework is that the functionality of the software is divided into functions, each of which has user tests defined and scripted. These user tests can be easily strung together to test entire business scenarios, or soap opera scenarios if you are so inclined.
The first thing to do is split the areas of the software up into functions. You can then define user tests for each function. User tests are something that a user will do or currently does. They have to be real world. They usually consist of some inputs (data), some processing and an outcome. If the outcome is positive, usually there is output, for example, an ID. If the outcome is negative, usually there is an error message.
Performing this activity for the Depot Rails Application creates something like:
User Tests: Empty Cart, Add Book to Cart, Check Out
User Tests: Log On, Ship All Book Orders, Ship Some Book Orders, Add User, Delete User
This activity can be done for any kind of requirements and even if requirements don’t exist. You can do this from a functional requirements specification, use cases/user stories (easier), and you can do this (albeit less easily) from an existing system interface or prototype.
Your functions then become modules of Watir code, and the user tests become methods in these modules. Each method takes some data, and returns an outcome (it worked or it didn’t) and optionally an error or an output value.
For example, this is an empty Customer module with just the methods defined.
module Customer URL = 'http://localhost:3000/store/' def Customer.add_book(book_title) end def Customer.check_out(customer_name, customer_email, customer_address, customer_payment_method) end def Customer.empty_cart end end
For efficiency and usability, I always ensure that each user test is home based, meaning that it starts and finishes on the same home page/screen. This avoids ‘state’ problems occurring, so a series of user tests can be called to test a business scenario without worrying about where each user test starts and ends. This also avoids repetitious and inefficient startup/logging in/logging out for each user test. Don’t laugh, I have often seen this done.Your user tests should accommodate positive and negative tests in the same method. For example, our ‘Customer.check_out(…)’ user test should be able to test both valid and invalid credit card numbers. This is done by making the method return the outcome and then determing the result outside the method depending on your expected outcome.For example, although the method may return an error, we could be expecting this in our negative test so therefore our test actually passes.I have seen many people write specific methods to test specific error messages. Don’t be tempted to write ‘Customer.check_out_valid_card(…)’ and ‘Customer.check_out_invalid_card(…)’. This leads to an unmaintainable set of user tests due to the repetition of code required. Limiting the number of methods also makes it easier to define business scenarios as there are a limited number of methods for a business scenario tester to choose from.
Once you have defined modules and methods you need to define the business scenarios which involve running user stories providing data (positive and negative) as well as expected outcomes. It is best to use a data presentation language to do this.
Excel is very common data presentation language for designing user tests and business scenarios but versioning excel files can be problematic due to their binary nature.
A wiki is excellent for defining user tests and business scenarios in wiki tables as wikis have in built versioning, a centralised, accessible and flexible nature, and are generally easy to use.
In the coming weeks I will discuss how to set up a wiki page in Confluence as a business scenario which includes a series of user tests and then how to dynamically call the ruby methods and determine the test results from the method outcomes.