One of the tools I have been using is Selenium. Selenium is a test framework for web applications. Selenium is composed of 3 different projects:
• Selenium IDE: A FireFox plugin that allows creation, recording and editing of tests for Selenium.
• Selenium RC: A web server and a set of classes to allow integrating Selenium tests in test applications like MbUnit.
To illustrate the concepts I’m going to use a sample. I have a control called ListControlMediator, that is used to connect two DropDownList controls in order to move ListItems from a DropDownList to the other. It can be used to reorder ListItems too. The following image shows two DropDownLists with a ListControlMediator:
The identifier is shown near the controls on the page.
The first thing will be to create a page to perform the tests, as the one shown above. The only functionality of the page shown above is the action to perform when the “Send” button is clicked. When the “Send” button is clicked, the text of the items of the ListBox2 is shown in the lResult label (separated with spaces).
The second thing to do is to create a test of the web page. In this step, there are two main options:
• Create the test by hand.
• Record the test with the Selenium IDE.
I usually record the test with the Selenium IDE and then adjust it a bit. To use the Selenium IDE just open FireFox and select Tools->Selenium IDE. Use the record button to start recording actions:
For example, after some actions in the page I got the following actions recorded:
As you can see, an action can have up to 3 attributes: Command, Target and Value. If you click on an action, you can select a command from the combo box. There are a lot of commands available, but you can learn them when you need them. The reference at the bottom of the Selenium IDE shows the help for the selected command, and the arguments it takes (Target is the first argument and Value is the second argument).
A cool thing about the Selenium IDE is that it adds an option in the FireFox context menu called “Show All Available Commands”. When you right click in an element of the page, it will show a list of the most common commands that can be applied to that element. That is very helpful for newbies.
After a test has been recorded, we can save it to replay it later (using File->Save Test). As a test is not more than a list of actions, and an action can have up to 3 attributes, tests are saved as HTML pages with a table. Each row in the table is an action, and has three columns, one per attribute of the action.
Once you have some tests saved, you can use the Selenium core to test them. Several tests are grouped in a “Test Suite”. A test suite is also an HTML file with a table, where each row in the table has only one column, where a link to the test of the suite is specified.
To use the Selenium core you will have to copy the “core” directory of the selenium core distribution to the root folder of your website. In the selenium core distribution, there is also a directory called “tests” where there are a lot of tests of a test suite. Create a directory called “tests” in the root folder of your website and copy the TestSuite.html file found in the “tests” directory of the selenium core distribution. Edit the TestSuite.html file, leaving only a row in the table that points to the test saved with the Selenium IDE (copy the saved test to the “tests” directory of your website). Some tests may not work in some browsers, so you can add that information to the row. For example, if you add unless="browserVersion.isSafari" to some rows and you try to run the tests using Safari, those tests won’t be executed.
To run a test suite using the Selenium core you have to point the browser to the /core/TestRunner.html page:
And select the test suite to run on the left. After the test suite has been selected, the tests contained in the test suite are shown on the left. When you click on a test the top center part of the browser shows the actions of the selected test. In the right there are some controls to run the tests. The bottom part of the browser is where the test will be run:
The TestRunner.html page accept some parameters to select the test suite, run the test automatically, etc, but the truth is that we use Selenium RC we won’t need to mess with the TestRunner anymore.
To use Selenium RC for our web tests we don’t need to copy the core or tests from the Selenium Core distribution.
Selenium Remote Control (RC) is basically a web server and the ThoughtWorks.Selenium.Core.dll assembly.
The ThoughtWorks.Selenium.Core.dll contains types to interact with the Selenium Server, so with .NET code we can create new sessions, and perform actions with it, testing our web sites with real browsers, using our favorite testing framework (MbUnit in my case).
To start the Selenium Server you need to have JRE 1.5.0 or higher installed and type:
java.exe -jar selenium-server.jar
The server listens to port 4444 by default.
However, to automate this I have added Assembly level SetUp and TearDown methods to my web tests so the server is started before all the tests and shutdown after the tests:
From .NET you have to use the DefaultSelenium class to create a browser session. In my tests, I have a base class for web tests that has a SetUp and TearDown methods that automatically open a close a browser session:
ServerUrl, ServerPort and BaseUrl are read from the application configuration file. The Browser property is passed to the constructor of the base class for web tests.
The actual test should use perform actions in the browser session that has been opened in the SetUp method. The Selenium IDE has an option to save a test in C# format, that converts an action to a call to the browser session object, so it can be used easily from code.
Let see some code of a test (don’t try to compare it with the example of the Selenium IDE, it is different):
As you can see, it is just another way to express the actions.
Some thing to clarify is the CheckPageError method. Unfortunately if an ASPX page throws an exception, the assembly ThoughtWorks.Selenium.Core.dll does not throw that exception and returns OK. This is because the Selenium Server requests a page and receives a response, so it doesn’t see any problem. This is very unfortunate and I hope it will be fixed in future releases. For now what I’m doing is to check if the body of the page starts with “Server Error”. This works for me because my test website doesn’t handle custom errors and the default error handling is displayed. You may have to create your own custom CheckPageError method.
Another thing that doesn’t work as I’d expect is the Selenium.GetSelectOptions method. If the select element is empty, it returns a string array with one item that contains an empty string instead of returning an empty string array. Hopefully this will be also fixed in the future.
Note that to be able to run the tests with Selenium RC you’ll probably need to use IIS instead of the built in server of Visual Studio.
I usually test my stuff in IE and FireFox. I inherit from the class where the test methods are defined as shown here:
If you want to test with more browsers probably this will get ugly and you will have to work out a better way to run the tests without inheritance.
To test the Server Side mode in the ListControlMediator, the only required changes to the test method is to place Selenium.WaitForPageToLoad instructions after each click, so the Selenium Core waits for the postback to complete before continuing processing the test.
I hope this helps some people to enter in the world of real web tests.