As quality assurance assistant and junior developer for Semantico I spend a lot of time developing and implementing QA tests. Testing an e-publishing website can be time consuming.

Even a simple test of search functionality has several steps; go to the designated URL, log in, search, verify the search results, check hit highlighting, start an advanced search, verify those results, check hit highlighting again, start another search with a different term… well, you get the idea. It’s repetitive. And there are many aspects to test, not just the search facilities.

After a while you might find yourself thinking that there should be a more efficient way to test your website. You can hire someone to do all your QA tests for you of course. But will they test your website in an efficient way? Will they follow your test cases to the letter; the tests you have spent hours designing, editing and documenting? If only there were an automated tool to do all this – and one which did not require a degree in Computer Science to operate.

Enter Selenium

At Semantico we do extensive testing. To get the most out of budgets and man hours we now use Selenium IDE for the repetitive tasks.

The official website describes Selenium as “a suite of tools to automate web app testing across many platforms”. You can think of Selenium as a little piece of software that does not mind doing your testing for you. It will test your website exactly the way you want it to be tested, as many times as you wish. You just need to tell it what to test and how. Selenium is a life saving tool and can cut out a lot of time, effort and money. It’s got no price tag (it’s free!) and is fairly easy to learn. It comes in different flavours but we’ll only talk about the Firefox add-on version here because that’s the version we know best.

Installation and tutorials

Download Selenium IDE and let’s get started. Install it on Firefox, restart your browser and you are ready. Selenium now lives in Firefox’s Tools menu and has a minimal interface. If you have ever recorded macros in an office suite of applications (like OpenOffice or Microsoft Office for example) then using Selenium will be a breeze. If you haven’t, it will only take you a couple of hours to learn how to use it efficiently.

I have found two very useful tutorials so far, one from the official Selenium Wiki and one from the Software Testing Geek website . Both tutorials require Flash to run. These are only a couple of tutorials from the many available on the web. A quick search on your favourite search engine should help you find even more.

After that, you are ready to start recording your test cases and then share them with your colleagues if you want, so that they can do some QA work when you are not available.

Using Selenium

Using Selenium is fairly straightforward. You start Firefox, fire up Selenium IDE and it is already in record mode. As soon as you start clicking on links, typing in text boxes and pressing buttons, Selenium will record these actions. The simplest thing to do when you first run Selenium is use it to run a search on a website. Once you have initiated the search and the search results are displayed on the page, press the ‘Record’ button to deactivate it. Afterwards, you can run the newly-created test case and Selenium will play it back on demand, repeating your actions in exactly the same order as you have taken them.

Translating your test cases

Selenium stores your commands in its own programming language. This language, called Selenese, uses English words and can be easily read by non-programmers. Once you’ve created your test cases you can translate them into Selenese and they are ready to be used whenever you need them. This is where you can realise the true power of this tool. You simply run Selenium and it will handle the rest. You can either run a test suite or individual test cases. If there are any errors, Selenium will report them, along with an explanation, in a log file which is directly accessible from the programme’s interface.

Selenium’s interface will not win any beauty contests. But it’s incredibly easy to use and, most importantly of all, is constantly giving you helpful feedback on what is happening. Brains over beauty. Simple.

Most automated tools we have tried in the past have had one common problem: they will not let you change the base URL. The base URL is essentially the URL of your website. But as all experienced QA testers know, the URL can change. One day you are testing something on a test environment, later that day you are testing something on the live site. Implementing your test cases multiple times for each URL can be time consuming and inefficient. With Selenium IDE you can choose not to store a base URL, which makes it very versatile and easy to use on different environments.

Sharing your test cases

Once you are happy with the implementation of your test cases and you have separated them in test suites, you are able to share them with your colleagues and across different departments. The files are small enough to email, or you can save them to a shared drive.

It is not a panacea

No matter how powerful a testing tool is, it will never entirely substitute the work done by a QA team. Selenium IDE is great for testing specific functionalities of your website and can deal with most of your test cases – but it is not ideal for checking presentation-related issues. You cannot use it, for instance, to test whether your website looks the same on a wide-screen monitor, or to check the text on a particular menu for correct alignment.

Also, it is a Firefox add-on. Unless you install the server-based version of Selenium, Selenium RC on a server, you can only check your website on Firefox. There are CSS issues between browsers that you also need to examine before you give the developers the green light to go live.

Lastly, if there is one area that needs improving it’s that Selenium is a bit weak on dynamic content. If, for example, one of your pages involves different sections that are presented in tabs with JavaScript, Selenium will not always work.

Recap

Using Selenium to improve your tests on your website has many benefits: you save time, money and resources. Your tests are done in a fraction of the time it used to take you to fully test a website. Your test cases become more flexible, more dynamic and of course, you never forget to test anything.

I’ve found Selenium to be an exceptionally useful tool when it comes to regression tests and rapid functional tests. It’s not here to replace testing done by humans, but it does a great job of assisting and complementing human testing.

To be fair, it’s not always the developer’s fault. There are times when their hands are tied, even though they may have the best intentions at heart, by budget constraints – or by a customer who is not willing to pay a little extra money for features that will make their website more accessible. There is also the lack of tools in their working environment which would enable them to fully test a web site (the cost of these tools is not inconsiderable). The aforementioned problems and hindrances can all add up to the development of a product that does not meet accessibility standards.

The Standards

The standards themselves can be migraine-inducing and can some times cancel each other out. And there are so many of them. Different countries have their own set of standards and sets of laws; the UK has the Disability Discrimination Act, the USA has Section 508 of the Rehabilitation Act, Japan has the Japanese Industrial Standards X 8341-3 and the W3C has a set of its own. Which one is the best? Which one should you be focusing on? Should you be using online or desktop tools to test your website?

In a nutshell, your best tool is simple common sense. Look into the web site you are developing, identify its structure and content, and the answer should present itself. If you feel you do not have the necessary skills and knowledge to improve the accessibility of your site alone however, you may want to consider hiring an experienced professional to do it for you. Both online and desktop tools, either free or not, are very helpful but they lack the human factor, and an experienced eye.

The sooner you start implementing accessible features in the development process, the better. It will save you and your customers time and money but in return, you will earn the users’ trust and loyalty. If they can use the web site you have created without difficulties, they will come back again and again.

Screen readers

One extremely helpful tool to test your web site for accessibility is a screen reader. Screen readers are applications which you install on your workstation and they will “read” the contents of a screen (dialogue boxes, application interfaces and web pages) out loud to you. They are mostly used by users who have visual impairments and thankfully, they are also used by a small percentage of web developers for testing according to a recent survey conducted by WebAIM.

The Windows operating system has one set up out of the box called Narrator, so does Mac OS X Leopard in the form of VoiceOver, and Ubuntu, a popular Linux distribution also comes with its own screen reader, named Orca pre-installed.

NVDA

Personally, I have never found Narrator to be very easy to use and have encountered some problems while using it for testing web sites so I started looking for an alternative solution. The answer to my problem came in the form of  “NVDA” (NonVisual Desktop Access). And best of all? It is absolutely open source and free. Of course, there are other products out there such as the ever-popular JAWS, but this small application can save you (or your business) money and it works really well for a piece of software that is still in its infancy.

The NVDA web site offers extensive documentation on how to install and use the application, which can also be run from a USB stick (portable version). As expected from a screen reader, most of the functions are accessed via keyboard shortcuts which take some time getting used to if you have not used such a programme before. But once you master the shortcuts you will realise how powerful this tool is in helping you build web sites and web applications that are easily accessible by visually impaired users and it will help you identify any problems that a screen reader use might face while trying to use your web site.

That is not the end of your journey though… Do not forget that there are other impairments such as colour blindness, auditory impairments, cognitive disabilities and  movement disorders such as the Parkinson’s disease, to name but a few. These can cause difficulties to many who use the Internet on a daily basis and using a variety of tools to check your web sites can help make your site more accessible for them, also allowing you to achieve your goals faster and more efficiently. It might seem difficult to follow all these guidelines and to use these tools at first but it soon becomes second nature. And the end result is a web site that can be accessed by everyone without difficulties.

Good luck!