[Tapestry Central] TestNG and Selenium

[Howard <hl...@gmail.com>]

I love working on client projects, because those help me really
understand how Tapestry gets used, and the problems people are running
in to. On site training is another good way to see where the theory
meets (or misses) the reality.
In any case, I'm working for a couple of clients right now for whom
testing is, rightfully, quite important. My normal approach is to write
unit tests to test specific error cases (or other unusual cases), and
then write integration tests to run through main use cases. I consider
this a balanced approach, that recognizes that a lot of what Tapestry
does is integration.
One of the reasons I like TestNG is that it seamlessly spans from unit
tests to integration tests. All of Tapestry's internal tests (about
1500 individual tests) are written using TestNG, and Tapestry includes
a base test case class for working with Selenium:
AbstractIntegrationTestSuite. This class does some useful things:
- Launches your application using Jetty
- Launches a SeleniumServer (which drives a web browser that can
exercise your application)
- Creates an instance of the Selenium client
- Implements all the methods of Selenium, redirecting each to the
Selenium instance
- Adds additional error reporting around any Selenium client calls that
fail
These are all useful things, but the class has gotten a little long in
the tooth ... it has a couple of critical short-comings:
- It runs your application using Jetty 5 (bundled with SeleniumServer)
- It starts and stops the stack (Selenium, SeleniumServer, Jetty)
around each class
For my current client, a couple of resources require JNDI, and so I'm
using Jetty 7 to run the application (at least in development, and
possibly in deployment as well). Fortunately, Jetty 5 uses the old
org.mortbay.jetty packages, and Jetty 7 uses the new org.eclipse.jetty
packages, so both versions of the server can co-exist within the same
application.
The larger problem is that I didn't want a single titanic test case for
my entire application; I wanted to break it up in other ways, by
Tapestry page initially.
I could create additional subclasses of AbstractIntegrationTestSuite,
but then the tests will spend a huge amount of time starting and
stopping Firefox and friends. I really want that stuff to start just
once.
What I've done is a bit of refactoring, by leveraging some features of
TestNG that I hadn't previously used.
The part of AbstractIntegrationTestSuite responsible for starting and
stopping the stack is broken out into its own class. This new class,
SeleniumLauncher, is responsible for starting and stopping the stack
around an entire TestNG test. In the TestNG terminology, a suite
contains multiple tests, and a test contains test cases (found in
individual classes, within scanned packages). The test case contains
test and configuration methods.
Here's what I've come up with:
package com.myclient.itest; import
org.apache.tapestry5.test.ErrorReportingCommandProcessor; import
org.eclipse.jetty.server.Server; import
org.openqa.selenium.server.RemoteControlConfiguration; import
org.openqa.selenium.server.SeleniumServer; import
org.testng.ITestContext; import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest; import com.myclient.RunJetty;
import com.thoughtworks.selenium.CommandProcessor; import
com.thoughtworks.selenium.DefaultSelenium; import
com.thoughtworks.selenium.HttpCommandProcessor; import
com.thoughtworks.selenium.Selenium; public class SeleniumLauncher {
public static final String SELENIUM_KEY = "myclient.selenium"; public
static final String BASE_URL_KEY = "myclient.base-url"; public static
final int JETTY_PORT = 9999; public static final String BROWSER_COMMAND
= "*firefox"; private Selenium selenium; private Server jettyServer;
private SeleniumServer seleniumServer; /** Starts the SeleniumServer,
the application, and the Selenium instance. */ @BeforeTest(alwaysRun =
true) public void setup(ITestContext context) throws Exception {
jettyServer = RunJetty.start(JETTY_PORT); seleniumServer = new
SeleniumServer(); seleniumServer.start(); String baseURL =
String.format("http://localhost:%d/", JETTY_PORT); CommandProcessor cp
= new HttpCommandProcessor("localhost",
RemoteControlConfiguration.DEFAULT_PORT, BROWSER_COMMAND, baseURL);
selenium = new DefaultSelenium(new ErrorReportingCommandProcessor(cp));
selenium.start(); context.setAttribute(SELENIUM_KEY, selenium);
context.setAttribute(BASE_URL_KEY, baseURL); } /** Shuts everything
down. */ @AfterTest(alwaysRun = true) public void cleanup() throws
Exception { if (selenium != null) { selenium.stop(); selenium = null; }
if (seleniumServer != null) { seleniumServer.stop(); seleniumServer =
null; } if (jettyServer != null) { jettyServer.stop(); jettyServer =
null; } } }

Notice that we're using the @BeforeTest and @AfterTest annotations;
that means any number of tests cases can execute using the same stack.
The stack is only started once.
Also, notice how we're using the ITestContext to communicate
information to the tests in the form of attributes. TestNG has a built
in form of dependency injection; any method that needs the ITestContext
can get it just by declaring a parameter of that type.
AbstractIntegrationTestSuite2 is the new base class for writing
integration tests:
package com.myclient.itest; import java.lang.reflect.Method; import
org.apache.tapestry5.test.AbstractIntegrationTestSuite; import
org.apache.tapestry5.test.RandomDataSource; import org.testng.Assert;
import org.testng.ITestContext; import
org.testng.annotations.AfterClass; import
org.testng.annotations.BeforeClass; import
org.testng.annotations.BeforeMethod; import
com.mchange.util.AssertException; import
com.thoughtworks.selenium.Selenium; public abstract class
AbstractIntegrationTestSuite2 extends Assert implements Selenium {
public static final String BROWSERBOT
= "selenium.browserbot.getCurrentWindow()"; public static final String
SUBMIT = "//input[@type='submit']"; /** * 15 seconds */ public static
final String PAGE_LOAD_TIMEOUT = "15000"; private Selenium selenium;
private String baseURL; protected String getBaseURL() { return
baseURL; } @BeforeClass public void setup(ITestContext context) {
selenium = (Selenium)
context .getAttribute(SeleniumLauncher.SELENIUM_KEY); baseURL =
(String)
context.getAttribute(SeleniumLauncher.BASE_URL_KEY); } @AfterClass
public void cleanup() { selenium = null; baseURL =
null; } @BeforeMethod public void indicateTestMethodName(Method
testMethod) { selenium.setContext(String.format("Running %s: %s",
testMethod .getDeclaringClass().getSimpleName(),
testMethod.getName() .replace("_", " "))); } /* Start of delegate
methods */ public void addCustomRequestHeader(String key, String value)
{ selenium.addCustomRequestHeader(key, value); } ... }

Inside the @BeforeClass-annotated method, we receive the test context
and extract the selenium instance and base URL put in there by
SeleniumLauncher.
The last piece of the puzzle is the code that launches Jetty. Normally,
I test my web applications using the Eclipse run-jetty-run plugin, but
RJR doesn't support the "Jetty Plus" functionality, including JNDI.
Thus I've created an application to run Jetty embedded:
package com.myclient; import org.eclipse.jetty.server.Server; import
org.eclipse.jetty.webapp.WebAppContext; public class RunJetty { public
static void main(String[] args) throws Exception { start().join(); }
public static Server start() throws Exception { return start(8080); }
public static Server start(int port) throws Exception { Server server =
new Server(port); WebAppContext webapp = new WebAppContext();
webapp.setContextPath("/"); webapp.setWar("src/main/webapp"); // Note:
Need jetty-plus and jetty-jndi on the classpath; otherwise //
jetty-web.xml (where datasources are configured) will not be // read.
server.setHandler(webapp); server.start(); return server; } }

This is all looking great. I expect to move this code into Tapestry 5.2
pretty soon. What I'm puzzling on is a couple of extra ideas:
- Better flexibility on starting up Jetty so that you can hook your own
custom Jetty server configuration in.
- Ability to run multiple browser agents, so that a single test suite
can execute against Internet Explorer, Firefox, Safari, etc. In many
cases, the same test method might be invoked multiple times, to test
against different browsers.
Anyway, this is just one of a number of very cool ideas I expect to
roll into Tapestry 5.2 in the near future.

--
Posted By Howard to Tapestry Central at 12/03/2009 09:37:00 AM