Agile Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 637 posts at DZone. You can read more from them at their website. View Full User Profile

Testing web applications with Selenium

06.15.2010
| 16646 views |
  • submit to reddit

There is a common problem between many testing harnesses: they are different from the real client (in the case of web applications a browser). Zend_Test, HttpUnit and similar tools perform fake HTTP requests (that may go or not go over the TCP/IP stack), and check an HTTP response in HTML or XML to make sure your application is producing the correct results.

Another problem is that usually JavaScript is not executed by these tools (and if it's executed, like in HttpUnit's case, it is a subset of the real JavaScript available in browsers and may lack some objects or methods in the DOM.)

Selenium uses real browsers

Let's talk about Selenium instead. Selenium instances manage real browsers, like Firefox, and send commands telling them how to crawl web applications in their behalf. This way, Selenium is easily able to execute JavaScript in every form (there is no Rhino or similar alternate implementation which supports only some features.). There is no replication of code between common HTTP clients and the testing harness.

Still, you need to run your app in a sandbox, so that you can recreate an initial state that the tests start from, and that you can throw away after having checked with assertions the results embedded in the generated pages.

As a Firefox extension

There are different distributions of Selenium. Selenium IDE is a record and replay tool: basically you install it as a Firefox extension, visit an host that contains the site or application to test, and save what you do in the browser as HTML files (much like with FitNesse acceptance tests).

You can then replay a test suite in the browser or via an Api (discussed later), on any host (specifying at that time localhost, stagingserver, etc.)

An issue that came to my mind is that Selenium IDE is not adapt to Test-Driven Development: there must be an existing user interface before you can record a test case. Therefore you lose the benefits of testability as a design tool (not graphic design but application design.)

That said, I'm still learning how to test-drive an AJAX interface, and maybe Test-Driven Development is not that useful for GUIs. However, Acceptance Test-Driven Development with HTML and CSS was a charm (in tests I checked that XPath or CSS expressions selected an element which contained a particular text, or that a table or a list had N elements...)

As a server to refer to in your builds

By the way, Selenium IDE is not the only way to produce a test for Selenium, nor the only way to run it. Let's move towards some kind of build automation, that is run via command line or automatically instead of at the push of a browser button.

Selenium RC (Remote Control) is a server that launches browsers as requested by its clients. Clients can be xUnit test cases in your language of choice (JUnit, PHPUNit), that access the Selenium Api provided as a library in the very Selenium RC package. This is radically better in case you're test-driving the user interface.

You can launch Selenium RC at every test run, passing it the test suite in HTML format, or simply leave it in execution on a server (possibly in Continuous Integration). I run it as a daemon on the Ubuntu machine that performs the build.

When using Selenium RC, you'll actually see Firefox launched and terminated for test runs on the machine that is hosting it, and a walktrough of the application will be executed for you at any time. You can even film it, manually or programmatically, and in case of a test regression rewatch it to see what went wrong. Selenium RC is obviously heavier than HttpUnit and other lower-level tools, but actually seeing a browser access the application is much more powerful and revealing.

Java Api

In case you're using Java for your tests, SeleneseTestCase is the class you're going to refer to (you'll find the Jar in the package that contains Selenium RC.) You don't need to always extend it, you can just subclass it one time, add a getSelenium() method and compose it in your acceptance tests.

Here is my SeleniumIntegrationTest (checks that the Api and Selenium RC are installed and working, and my wrapper of SeleneseTestCase to use it as a collaborator in your test instead of as a superclass.

//SeleniumIntegrationTest.java
package it.polimi.chansonnier.test;

import java.io.File;

import junit.framework.TestCase;

import com.thoughtworks.selenium.*;

public class SeleniumIntegrationTest extends TestCase {
WrappableSeleneseTestCase wrapped;
Selenium selenium;


public void setUp() throws Exception {
wrapped = new WrappableSeleneseTestCase();
wrapped.setUp("http://www.google.com/", "*chrome");
selenium = wrapped.getSelenium();
}

public void tearDown() throws Exception {
wrapped.tearDown();
}

public void testSeleniumRCIsStartedAndWorks() throws Exception {
selenium.open("/");
wrapped.verifyTrue(selenium.isTextPresent("Google"));
}
}

//WrappableSeleneseTestCase.java
package it.polimi.chansonnier.test;

import com.thoughtworks.selenium.SeleneseTestCase;
import com.thoughtworks.selenium.Selenium;

public class WrappableSeleneseTestCase extends SeleneseTestCase {
public Selenium getSelenium() {
return selenium;
}
}

In build servers

Selenium RC normally requires a graphical environment, since modern browsers won't launch without a display to render themselves in. But it is still useful to run Selenium RC in build automation, maybe in server without a GUI.

My situation was exactly that: I run my builds and test suites on an Ubuntu Server. But at least on Unix, servers haven't graphical interfaces. Fortunately, this is Linux and you can plug in software like Lego bricks: meet X virtual framebuffer, essentially a Fake testing pattern applied to XServer:

In the X Window System, Xvfb or X virtual framebuffer  is an X11 server that performs all graphical operations in memory, not showing any screen output. From the point of view of the client, it acts exactly like any other server, serving requests and sending events and errors as appropriate. However, no output is shown. This virtual server does not require the computer it is running on to even have a screen or any input device. Only a network layer is necessary.

Of course every server has a network layer, or you won't be able to access it by ssh.

Thus, I installed the xvfm and firefox packages on Ubuntu Server, and instead of running selenium RC server like this:

sudo java -jar selenium-server.jar -log /home/giorgio/log/selenium &

in my /etc/rc.local, I now run it like this:

sudo xvfb-run -e /home/giorgio/log/xvfb-run java -jar selenium-server.jar -log /home/giorgio/log/selenium &

If you install also the package x11-apps, you can make some nice photographs with xwd while Selenium RC accomplishes its job just fine:

Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Harry Sheng replied on Tue, 2010/06/15 - 8:25am

I've just tried the IDE in the last couple of days. In the case of NON-SPI application, it's really impressive.

But in my specific case, I was trying to use it against a backbase (SPI) application. It did not come out well. The problem is with the dynamic contents.  The action targets are embedded deep into the <span>, <tr>, <td>, <div>, etc. and without ID. The IDE recorded clicks on components, that it complains non-existence when replay; even after I replaced all click to clickAndWait.

One question I have is, to what extend it supports XPATH to find the action target component.

 Harry

Giorgio Sironi replied on Tue, 2010/06/15 - 10:44am in response to: Harry Sheng

In my experience everything I can do with HttpUnit (it's for Java) I can do with Selenium. XPath is supported but I use mostly ids to access elements so I never tried very long selectors.

Walter Bogaardt replied on Wed, 2010/06/16 - 12:21am

The general IDE (firefox plugin for selenium) does an ok job, but deep nested tags XPATH or regex are the way to go. At times, you even have to go so far as using Java Robot class to get some of the work you need selenuium to do.

You presented a pretty manual process of selenium testing. There is a more elagant way using Maven 2 plugin's to run your tests and start up your remote RCs. In fact, you could go beyond testing a simple page or two in a single browser by testing against multiple browsers in multiple OS's using Selenium grid.

 Overall, I prefer Selenium over HTTPUnit simply because Selenium is much more "user friendly" towards QA engineers. I was able to train basic QA folks in selenium much quicker than I could train them how to write HTTPUnit tests, which is more developer centric.

Giorgio Sironi replied on Thu, 2010/06/17 - 4:20am in response to: Walter Bogaardt

Deep nested XPath? I prefer the contrary, specifying the text to click on, since it is less likely to become brittle when the user interfaces changes over time.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.