Quality assurance testing your e-publishing website with Selenium

As quality assurance assistant and junior developer for Semantico I spend a lot of time developing and implementing QA tests. Testing an e-publishing website can be time consuming.

Even a simple test of search functionality has several steps; go to the designated URL, log in, search, verify the search results, check hit highlighting, start an advanced search, verify those results, check hit highlighting again, start another search with a different term… well, you get the idea. It’s repetitive. And there are many aspects to test, not just the search facilities.

After a while you might find yourself thinking that there should be a more efficient way to test your website. You can hire someone to do all your QA tests for you of course. But will they test your website in an efficient way? Will they follow your test cases to the letter; the tests you have spent hours designing, editing and documenting? If only there were an automated tool to do all this – and one which did not require a degree in Computer Science to operate.

Enter Selenium

At Semantico we do extensive testing. To get the most out of budgets and man hours we now use Selenium IDE for the repetitive tasks.

The official website describes Selenium as “a suite of tools to automate web app testing across many platforms”. You can think of Selenium as a little piece of software that does not mind doing your testing for you. It will test your website exactly the way you want it to be tested, as many times as you wish. You just need to tell it what to test and how. Selenium is a life saving tool and can cut out a lot of time, effort and money. It’s got no price tag (it’s free!) and is fairly easy to learn. It comes in different flavours but we’ll only talk about the Firefox add-on version here because that’s the version we know best.

Installation and tutorials

Download Selenium IDE and let’s get started. Install it on Firefox, restart your browser and you are ready. Selenium now lives in Firefox’s Tools menu and has a minimal interface. If you have ever recorded macros in an office suite of applications (like OpenOffice or Microsoft Office for example) then using Selenium will be a breeze. If you haven’t, it will only take you a couple of hours to learn how to use it efficiently.

I have found two very useful tutorials so far, one from the official Selenium Wiki and one from the Software Testing Geek website . Both tutorials require Flash to run. These are only a couple of tutorials from the many available on the web. A quick search on your favourite search engine should help you find even more.

After that, you are ready to start recording your test cases and then share them with your colleagues if you want, so that they can do some QA work when you are not available.

Using Selenium

Using Selenium is fairly straightforward. You start Firefox, fire up Selenium IDE and it is already in record mode. As soon as you start clicking on links, typing in text boxes and pressing buttons, Selenium will record these actions. The simplest thing to do when you first run Selenium is use it to run a search on a website. Once you have initiated the search and the search results are displayed on the page, press the ‘Record’ button to deactivate it. Afterwards, you can run the newly-created test case and Selenium will play it back on demand, repeating your actions in exactly the same order as you have taken them.

Translating your test cases

Selenium stores your commands in its own programming language. This language, called Selenese, uses English words and can be easily read by non-programmers. Once you’ve created your test cases you can translate them into Selenese and they are ready to be used whenever you need them. This is where you can realise the true power of this tool. You simply run Selenium and it will handle the rest. You can either run a test suite or individual test cases. If there are any errors, Selenium will report them, along with an explanation, in a log file which is directly accessible from the programme’s interface.

Selenium’s interface will not win any beauty contests. But it’s incredibly easy to use and, most importantly of all, is constantly giving you helpful feedback on what is happening. Brains over beauty. Simple.

Most automated tools we have tried in the past have had one common problem: they will not let you change the base URL. The base URL is essentially the URL of your website. But as all experienced QA testers know, the URL can change. One day you are testing something on a test environment, later that day you are testing something on the live site. Implementing your test cases multiple times for each URL can be time consuming and inefficient. With Selenium IDE you can choose not to store a base URL, which makes it very versatile and easy to use on different environments.

Sharing your test cases

Once you are happy with the implementation of your test cases and you have separated them in test suites, you are able to share them with your colleagues and across different departments. The files are small enough to email, or you can save them to a shared drive.

It is not a panacea

No matter how powerful a testing tool is, it will never entirely substitute the work done by a QA team. Selenium IDE is great for testing specific functionalities of your website and can deal with most of your test cases – but it is not ideal for checking presentation-related issues. You cannot use it, for instance, to test whether your website looks the same on a wide-screen monitor, or to check the text on a particular menu for correct alignment.

Also, it is a Firefox add-on. Unless you install the server-based version of Selenium, Selenium RC on a server, you can only check your website on Firefox. There are CSS issues between browsers that you also need to examine before you give the developers the green light to go live.

Lastly, if there is one area that needs improving it’s that Selenium is a bit weak on dynamic content. If, for example, one of your pages involves different sections that are presented in tabs with JavaScript, Selenium will not always work.

Recap

Using Selenium to improve your tests on your website has many benefits: you save time, money and resources. Your tests are done in a fraction of the time it used to take you to fully test a website. Your test cases become more flexible, more dynamic and of course, you never forget to test anything.

I’ve found Selenium to be an exceptionally useful tool when it comes to regression tests and rapid functional tests. It’s not here to replace testing done by humans, but it does a great job of assisting and complementing human testing.