You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Jason Dillon <ja...@planet57.com> on 2006/09/02 03:46:09 UTC
Testsuite module, with basic/crude Selenium support
I've checked in a new top-level testsuite module and a few new
plugins to support it. This is only the start and I expect it to
change over the next few weeks (er maybe months) as momentum starts
to pick up.
I looked into Cargo, and while I think we should eventually use it to
start/stop the server... asis now its broke. We need to provide a
very simple and consistent way for Cargo to invoke some operations
via JMX, which I hinted to in previous emails. Once that is done, we
can patch Cargo and hopefully get that committed to support G... but
in the mean time I whipped up a simple start & stop mojo that uses
Ant to start and stop the server. Its very, very crude and we need
to fix stuff like logging output etc.
I also played around a little with Selenium to make some tests for
the console. It wants a special server process started, so I created
a selenium plugin which currently only starts that server in... well
an icky way, but should work for the moment. Need to ask the m2
folks how to do this better.
I created a console-testsuite module, which sets up the G server and
Selenium server in pre-integration-test, and then uses the maven-
invoker-plugin to invoke the basic module to actually run the tests.
Only one test class right now, SimpleLoginTest, which does just
that... logs in, logs in and then out and then another that logs in
and clicks some links.
I've got it all running from one mvn command and so far, on my mac,
it works fine. I'd imagine due to the location of the assembly that
Windows will not be able to run this... we'll need to fix this
later. But those of you lucky enough to have a Linux box or a mac,
you should be able to:
mvn clean && mvn install
cd testsuite
mvn
(the testsuite is not hooked up to the default build)
And you should see Firefox popup a few times as Selenium goes about
clicking on stuff. Right now its only really doing validation of the
console page's titles... but if everything works, then the invoker
should not complain and the build should complete.
A bunch more needs to be done... like creating a common test support
class for selenium tests... and probably switching these tests to use
testng, so that the selenium setup is done once, not for each test...
as it does now... so expect Firefox to start and stop a few times.
We need to have some helpers to pick out the console contents easily
so they can be validated... I read that selenium supports xpath or
something, but I have not actually tried any of that.
This is just a very, very crude proof of concept... needs to have
reporting features added, capture logs, blah, blah, blah.
Anyways, I committed this so we have a common base to work patches
off of.
One thing to note about the Selenium IDE, is that sometimes it does
not put into a waitForPageToLoad(), like if you record a login to the
console... you need to manually add that wait after the button is
clicked or it will hose the test.
I'm not married to any of this either... I just wanted something that
worked... or kinda worked to show that its not going to be that hard
to make some rich integration tests.
So, comments, suggestions, patches... all welcome.
--jason
Re: Testsuite module, with basic/crude Selenium support
Posted by Prasad Kashyap <go...@gmail.com>.
Please review the latest patch at
http://issues.apache.org/jira/browse/GERONIMO-2359#action_12433253
I have worked on all but one of the issues we discussed here. The
remaning issue is the one of delegating stuff to a JMXProxy.
I have issues getting my geronimo server to start. So I shall go and
chase that problem for now.
Module 18/20 org.apache.geronimo.configs/webconsole-jetty/1.2-SNAPSHOT/car
17:27:31,528 ERROR [GBeanInstanceState] Error while starting;
GBean is now in the FAILED state:
abstractName="org.apache.geronimo.configs/webconsole-jetty/1.2-SNAPSHOT/car?J2EEApplication=org.apache.geronimo.configs/webconsole-jetty/1.2-SNAPSHOT/car,WebModule=framework.war,j2eeType=Servlet,name=car-export-forward"
java.lang.ClassNotFoundException:
org.apache.geronimo.console.servlet.ContextForwardServlet in
classloader org.apache.geronimo.configs/webconsole-jetty_framework.war/1.2-SNAPSHOT/car
at org.apache.geronimo.kernel.config.MultiParentClassLoader.loadClass(MultiParentClassLoader.java:249)
Cheers
Prasad
On 9/6/06, Jason Dillon <ja...@planet57.com> wrote:
> On Sep 6, 2006, at 8:03 PM, Prasad Kashyap wrote:
> > Here are the use case scenarios I thought of -
> >
> > UC #1 : use goal in testsuites. The modules for deployment in this
> > execution typically comes from the testsupport project. For most
> > testcases we can build the modules with the plans inside them. A few
> > testcases that need to test a deployment with an external plan can
> > specify the plan in the config.
> >
> > UC #2: use goal by others. The modules for deployment in this
> > execution may come from a repository or be specified as a non-artifact
> > archive. Even here, the plan, if need be, can be specified in the
> > config along with the <modules>. However what I'd eventually like to
> > do here is be able to deploy a list of modules.
> >
> > To be able to deploy a list of modules, the #plan param needs to get
> > inside the #module param so that there can be a plan specified for
> > every module config.
>
> None of these cases require the moduleId and defaultModuleId bits,
> which was what I was hinting at. A list of modules, with optional
> plan such be fine.
>
> Sounds like that if you did want to do some selection, that it would
> be over one set of modules or another... but I still don't see the
> need for that.
>
>
> >> Why do we have 'distribute' and 'undeploy'? Why not 'deploy' and
> >> 'undeploy'?
> >
> > "distribute" and "deploy" are some of the options passed to the deploy
> > tool. The former option just "installs" the modules into G while the
> > latter "installs" and starts it. Now I went with distribute + start
> > sequence since that is what G had in the old itests. If we think we
> > can get rid of it and just use deploy, then I'll be glad to change it.
>
> I'd say, call the goal deploy, and add an optional startModule flag
> to indicate if distribute() or deploy() should be used.
>
>
> >> In DistributeModuleMojo, why is #plan a String and not a File? The
> >> plexus container will perform the equiv of your getFile() for you.
> >
> > I thought about it. But eventually I wanted to move the #plan inside
> > the #module. So I left it as String.
>
> You can still have a File in a helper object and Plexus will inject
> it. We should create a ModuleConfig, that extends from ArtifactItem
> (like AssemblyConfig) and add that field.
>
>
> >> Any reason why we have explicitly set the TCL in getDeploymentManager
> >> (), was the TCL that maven setup not correct?
> >
> > Hmm.. Not that I know of. Must be from legacy code. But I'll reserve
> > judgement on it until I investigate that further.
>
> Well, if we don't need to set it... lets not.
>
> --jason
>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
On Sep 6, 2006, at 8:03 PM, Prasad Kashyap wrote:
> Here are the use case scenarios I thought of -
>
> UC #1 : use goal in testsuites. The modules for deployment in this
> execution typically comes from the testsupport project. For most
> testcases we can build the modules with the plans inside them. A few
> testcases that need to test a deployment with an external plan can
> specify the plan in the config.
>
> UC #2: use goal by others. The modules for deployment in this
> execution may come from a repository or be specified as a non-artifact
> archive. Even here, the plan, if need be, can be specified in the
> config along with the <modules>. However what I'd eventually like to
> do here is be able to deploy a list of modules.
>
> To be able to deploy a list of modules, the #plan param needs to get
> inside the #module param so that there can be a plan specified for
> every module config.
None of these cases require the moduleId and defaultModuleId bits,
which was what I was hinting at. A list of modules, with optional
plan such be fine.
Sounds like that if you did want to do some selection, that it would
be over one set of modules or another... but I still don't see the
need for that.
>> Why do we have 'distribute' and 'undeploy'? Why not 'deploy' and
>> 'undeploy'?
>
> "distribute" and "deploy" are some of the options passed to the deploy
> tool. The former option just "installs" the modules into G while the
> latter "installs" and starts it. Now I went with distribute + start
> sequence since that is what G had in the old itests. If we think we
> can get rid of it and just use deploy, then I'll be glad to change it.
I'd say, call the goal deploy, and add an optional startModule flag
to indicate if distribute() or deploy() should be used.
>> In DistributeModuleMojo, why is #plan a String and not a File? The
>> plexus container will perform the equiv of your getFile() for you.
>
> I thought about it. But eventually I wanted to move the #plan inside
> the #module. So I left it as String.
You can still have a File in a helper object and Plexus will inject
it. We should create a ModuleConfig, that extends from ArtifactItem
(like AssemblyConfig) and add that field.
>> Any reason why we have explicitly set the TCL in getDeploymentManager
>> (), was the TCL that maven setup not correct?
>
> Hmm.. Not that I know of. Must be from legacy code. But I'll reserve
> judgement on it until I investigate that further.
Well, if we don't need to set it... lets not.
--jason
Re: Testsuite module, with basic/crude Selenium support
Posted by Prasad Kashyap <go...@gmail.com>.
Please see comments inline -
On 9/6/06, Jason Dillon <ja...@planet57.com> wrote:
> Cool, thanks :-) Some comments for ya...
>
> How does this work with archive + plan deployment? I'm not sure that
> the assembly artifact selection fits into the model if deploy/
> undeploy... but I'm not sure. The point of the selection for
> assemblies was to make it easier to change the dependency w/o having
> to type in the full artifact details. Seems like with the module +
> plan, you get half of that, but you will always have to configure the
> plan... kinda negating the ease of use. Do you have a use case for
> when you would want to pre-configure a set modules to deploy and then
> deploy them with an alias? It might help me understand why you added
> module selection here...
Here are the use case scenarios I thought of -
UC #1 : use goal in testsuites. The modules for deployment in this
execution typically comes from the testsupport project. For most
testcases we can build the modules with the plans inside them. A few
testcases that need to test a deployment with an external plan can
specify the plan in the config.
UC #2: use goal by others. The modules for deployment in this
execution may come from a repository or be specified as a non-artifact
archive. Even here, the plan, if need be, can be specified in the
config along with the <modules>. However what I'd eventually like to
do here is be able to deploy a list of modules.
To be able to deploy a list of modules, the #plan param needs to get
inside the #module param so that there can be a plan specified for
every module config.
>
> Why do we have 'distribute' and 'undeploy'? Why not 'deploy' and
> 'undeploy'?
"distribute" and "deploy" are some of the options passed to the deploy
tool. The former option just "installs" the modules into G while the
latter "installs" and starts it. Now I went with distribute + start
sequence since that is what G had in the old itests. If we think we
can get rid of it and just use deploy, then I'll be glad to change it.
>
> Is there anyway to get around needing to "new DeploymentFactoryImpl
> ();"? It would be nice to have the plugin free from importing system
> classes to deploy and delegate everything to a JMX proxy... so that
> the plugin has a better chance of working with newer G servers, as
> well as retaining compatibility with older ones).
I'll look into this.
>
> In DistributeModuleMojo, why is #plan a String and not a File? The
> plexus container will perform the equiv of your getFile() for you.
I thought about it. But eventually I wanted to move the #plan inside
the #module. So I left it as String.
>
> In DeployToolSupport, might want to just configure the deployer URI
> and not worry about building that URI from a hostname, port. Still
> need to add hostname (or raw URI) support to the other mojo's, so we
> should make this consistent... right now shutdown can only happen for
> localhost (see TODO).
Yeah, I saw the TODO there, which is why I thought I should fix it
here if I can :-) OK.. will change that to make it consistent.
>
> Any reason why we have explicitly set the TCL in getDeploymentManager
> (), was the TCL that maven setup not correct?
Hmm.. Not that I know of. Must be from legacy code. But I'll reserve
judgement on it until I investigate that further.
>
> And finally.. :-) I'd like to see more logging used, for both debug
> and info to show what is going on... for the goals at least... still
> need to resolve the artifact selection, and if that is going to be
> used for this, then we need to clean up the logging (and method
> names) for getAssemblyArtifact(), since that was assembly specific,
> and now it has changed to be more than that... still unsure that is
> good, but let me know what you think the use cases care.
Sure, I'll add more logging and debugging based on what we decide
about artifact resolution and selection. Yes, the method names needs
to ne refactored too if we go with this.
>
> * * *
>
> I hope you find all of my comments as constructive... I appreciate
> your work :-)
Absolutely. I appreciate the feedback, sincerely.
>
> --jason
Cheers
Prasad.
>
>
> On Sep 6, 2006, at 11:22 AM, Prasad Kashyap wrote:
>
> > I have uploaded the DeployToolSupport.patch to
> > http://issues.apache.org/jira/secure/attachment/12340282/
> > DeployToolSuport.patch
> >
> > This patch adds DistributeModuleMojo and UndeployModuleMojo to the
> > geronimo-maven-plugin. The deploy is done by conecting to the default
> > uri "deployer:geronimo:jmx"
> >
> > The patch abstracts the artifact resolution code from
> > InstallerMojoSupport.java up one level to ServerMojoSuport.java.
> > DeployToolSupport.java extends ServerMojoSupport.java.
> >
> > Similar config options that one uses to specify an assembly for
> > install can be reused to specify the modules for deploy/undeploy. The
> > getAssembly() code that has been abstracted to ServerMojoSupport.java
> > can now be reused to resolve the assembly artifacts or module
> > artifacts.
> >
> > Let me know your thoughts. I'm amenable to any suggestions.
> >
> > Cheers
> > Prasad.
> >
> >
> >
> >
> >
> > On 9/4/06, Jason Dillon <ja...@planet57.com> wrote:
> >>
> >> K, no worries. Though... looks like some time is going to be put
> >> into
> >> fixing up surefire in the week or so, so I don't think we will need
> >> 2.8-SNAPSHOT for too long.
> >>
> >> --jason
> >>
> >>
> >>
> >> On Sep 4, 2006, at 10:53 AM, Bill Dudney wrote:
> >> Hey Jason,
> >>
> >> Sorry for the false alarm, the tests don't run. Looks like 2.8-
> >> SNAPSHOT is
> >> what we will have to use until the surefire folks can do an update.
> >>
> >> TTFN,
> >>
> >> -bd-
> >>
> >>
> >> On Sep 4, 2006, at 10:34 AM, Jason Dillon wrote:
> >> Did the tests actually run? Make sure that it actually ran tests for
> >> basic/*
> >>
> >> --jason
> >>
> >>
> >>
> >> On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
> >> Hi All,
> >>
> >> The only down side I can see to testng is the reliance on surefire
> >> 2.8-SNAPSHOT.
> >>
> >> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-
> >> SNAPSHOT from
> >> http://people.apache.org/maven-snapshot-repository and
> >> everything seemed to work fine. Jason could you try that out so we
> >> can get
> >> rid of the 2.8-SNAPSHOT dependency?
> >>
> >> Apart from that I think testng is the way to go.
> >>
> >> TTFN,
> >>
> >> -bd-
> >>
> >> ------------------ begin patch ---------------------------
> >> Index: pom.xml
> >> ===================================================================
> >> --- pom.xml (revision 439918)
> >> +++ pom.xml (working copy)
> >> @@ -93,7 +93,7 @@
> >> <plugin>
> >> <groupId>org.apache.maven.plugins</groupId>
> >>
> >> <artifactId>maven-surefire-plugin</artifactId>
> >> - <version>2.8-SNAPSHOT</version>
> >> + <version>2.3-SNAPSHOT</version>
> >> </plugin>
> >> </plugins>
> >> </pluginManagement>
> >> @@ -116,14 +116,4 @@
> >> </repository>
> >> </repositories>
> >>
> >> - <!--
> >> - NOTE: This is the repository where maven-surefire-plugin:2.8-
> >> SNAPSHOT
> >> comes from.
> >> - -->
> >> - <pluginRepositories>
> >> - <pluginRepository>
> >> - <id>tapestry.javaforge</id>
> >> -
> >> <url>http://howardlewisship.com/repository</url>
> >> - </pluginRepository>
> >> - </pluginRepositories>
> >> -
> >> </project>
> >> ---------------------- end
> >> patch--------------------------------------
> >>
> >>
> >>
> >>
> >> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
> >> Ya, I see this from time to time... not really sure why it pops up
> >> sometimes
> >> and not at others. Its easy enough to fix... but I think that its
> >> much
> >> easier with ng.
> >>
> >> Still talking to maven folks about getting a release of surefire
> >> that works
> >> with testng + jdk14, but the 2.8-SNAPSHOT works for now...
> >>
> >> --jason
> >>
> >>
> >> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
> >> > So you got around this? This was not happening for me so I'm not
> >> sure
> >> > what is going on, it appears from the next message that you got
> >> it to
> >> > work.
> >> >
> >> > I did see an UndeclaredThrowableException a time or two but it was
> >> > always because the selenium server was not running.
> >> >
> >> > I'm going to dig into your testng stuff in the am and will post
> >> > thoughts.
> >> >
> >> > TTFN,
> >> >
> >> > bd-
> >> > On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
> >> >
> >> > > FYI, I applied your patch... and surefire barfs as I thought
> >> it might:
> >> > >
> >> > > <snip>
> >> > > java.lang.reflect.UndeclaredThrowableException
> >> > > at $Proxy0.addError(Unknown Source)
> >> > > at
> >> junit.framework.TestResult.addError(TestResult.java:36)
> >> > > at
> >> junit.framework.TestResult.runProtected(TestResult.java:
> >> > > 133)
> >> > > at junit.extensions.TestSetup.run(TestSetup.java:23)
> >> > > at
> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke
> >> > > (NativeMethodAccessorImpl.java :39)
> >> > > at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke
> >> > > (DelegatingMethodAccessorImpl.java:25)
> >> > > at java.lang.reflect.Method.invoke(Method.java:324)
> >> > > at
> >> org.apache.maven.surefire.junit.JUnitTestSet.execute
> >> > > (JUnitTestSet.java:210)
> >> > > at
> >> > >
> >> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTes
> >> t
> >> > > Set(AbstractDirectoryTestSuite.java:135)
> >> > > at
> >> > >
> >> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
> >> > > (AbstractDirectoryTestSuite.java:122)
> >> > > at
> >> org.apache.maven.surefire.Surefire.run(Surefire.java:129)
> >> > > at
> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke
> >> > > (NativeMethodAccessorImpl.java:39)
> >> > > at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke
> >> > > (DelegatingMethodAccessorImpl.java:25)
> >> > > at java.lang.reflect.Method.invoke (Method.java:324)
> >> > > at
> >> > >
> >> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
> >> > > (SurefireBooter.java:262)
> >> > > at
> >> org.apache.maven.surefire.booter.SurefireBooter.main
> >> > > (SurefireBooter.java:787)
> >> > > Caused by: java.lang.NoSuchMethodException:
> >> > >
> >> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName(
> >> )
> >> > > at java.lang.Class.getMethod(Class.java:986)
> >> > > at
> >> > >
> >> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStac
> >> k
> >> > > TraceWriter(TestListenerInvocationHandler.java:171)
> >> > > at
> >> > >
> >> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleA
> >> d
> >> > > dError(TestListenerInvocationHandler.java:160)
> >> > > at
> >> > >
> >> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
> >> > > (TestListenerInvocationHandler.java:134)
> >> > > ... 18 more
> >> > > </snip>
> >> > >
> >> > > --jason
> >> > >
> >> > >
> >> > > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> >> > >
> >> > >> Hi Jason,
> >> > >>
> >> > >> This is very cool indeed, thanks for putting the prototype
> >> > >> together. I just submitted a patch that makes firefox pop up
> >> only
> >> > >> once to run all the tests (it actually pops up twice, once
> >> during
> >> > >> selenium.start() and then once for the test suite but the first
> >> > >> time is for a split second).
> >> > >>
> >> > >> http://issues.apache.org/jira/browse/GERONIMO-2374
> >> > >>
> >> > >> It uses a JUnit TestDecorator to run setup only once for the
> >> whole
> >> > >> suite.
> >> > >>
> >> > >> IMO until we move to JDK 5 its not a great idea to move to
> >> TestNG,
> >> > >> you will get some benefits for sure but the source has to be
> >> > >> parsed at runtime to get at the meta-data ( i.e. the info
> >> about the
> >> > >> tests). Krufty IMO.
> >> > >>
> >> > >> The patch has a comment about invalid XHTML. I believe the
> >> > >> invalid XHTML part of the console is preventing the XPath find
> >> > >> from working.
> >> > >>
> >> > >> TTFN,
> >> > >>
> >> > >> -bd-
> >> > >>
> >> > >>
> >> > >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
> >> > >>
> >> > >>> I've checked in a new top-level testsuite module and a few new
> >> > >>> plugins to support it. This is only the start and I expect
> >> it to
> >> > >>> change over the next few weeks (er maybe months) as momentum
> >> > >>> starts to pick up.
> >> > >>>
> >> > >>> I looked into Cargo, and while I think we should eventually use
> >> > >>> it to start/stop the server... asis now its broke. We need to
> >> > >>> provide a very simple and consistent way for Cargo to invoke
> >> some
> >> > >>> operations via JMX, which I hinted to in previous emails. Once
> >> > >>> that is done, we can patch Cargo and hopefully get that
> >> committed
> >> > >>> to support G... but in the mean time I whipped up a simple
> >> start
> >> > >>> & stop mojo that uses Ant to start and stop the server. Its
> >> > >>> very, very crude and we need to fix stuff like logging
> >> output etc.
> >> > >>>
> >> > >>> I also played around a little with Selenium to make some tests
> >> > >>> for the console. It wants a special server process started,
> >> so I
> >> > >>> created a selenium plugin which currently only starts that
> >> server
> >> > >>> in... well an icky way, but should work for the moment.
> >> Need to
> >> > >>> ask the m2 folks how to do this better.
> >> > >>>
> >> > >>> I created a console-testsuite module, which sets up the G
> >> server
> >> > >>> and Selenium server in pre-integration-test, and then uses the
> >> > >>> maven-invoker-plugin to invoke the basic module to actually run
> >> > >>> the tests. Only one test class right now, SimpleLoginTest,
> >> which
> >> > >>> does just that... logs in, logs in and then out and then
> >> another
> >> > >>> that logs in and clicks some links.
> >> > >>>
> >> > >>> I've got it all running from one mvn command and so far, on my
> >> > >>> mac, it works fine. I'd imagine due to the location of the
> >> > >>> assembly that Windows will not be able to run this... we'll
> >> need
> >> > >>> to fix this later. But those of you lucky enough to have a
> >> Linux
> >> > >>> box or a mac, you should be able to:
> >> > >>>
> >> > >>> mvn clean && mvn install
> >> > >>> cd testsuite
> >> > >>> mvn
> >> > >>>
> >> > >>> (the testsuite is not hooked up to the default build)
> >> > >>>
> >> > >>> And you should see Firefox popup a few times as Selenium goes
> >> > >>> about clicking on stuff. Right now its only really doing
> >> > >>> validation of the console page's titles... but if everything
> >> > >>> works, then the invoker should not complain and the build
> >> should
> >> > >>> complete.
> >> > >>>
> >> > >>> A bunch more needs to be done... like creating a common test
> >> > >>> support class for selenium tests... and probably switching
> >> these
> >> > >>> tests to use testng, so that the selenium setup is done
> >> once, not
> >> > >>> for each test... as it does now... so expect Firefox to
> >> start and
> >> > >>> stop a few times. We need to have some helpers to pick out the
> >> > >>> console contents easily so they can be validated... I read that
> >> > >>> selenium supports xpath or something, but I have not actually
> >> > >>> tried any of that.
> >> > >>>
> >> > >>> This is just a very, very crude proof of concept... needs to
> >> have
> >> > >>> reporting features added, capture logs, blah, blah, blah.
> >> > >>>
> >> > >>> Anyways, I committed this so we have a common base to work
> >> > >>> patches off of.
> >> > >>>
> >> > >>> One thing to note about the Selenium IDE, is that sometimes it
> >> > >>> does not put into a waitForPageToLoad(), like if you record a
> >> > >>> login to the console... you need to manually add that wait
> >> after
> >> > >>> the button is clicked or it will hose the test.
> >> > >>>
> >> > >>> I'm not married to any of this either... I just wanted
> >> something
> >> > >>> that worked... or kinda worked to show that its not going to be
> >> > >>> that hard to make some rich integration tests.
> >> > >>>
> >> > >>> So, comments, suggestions, patches... all welcome.
> >> > >>>
> >> > >>> --jason
> >> > >>>
> >> > >>>
> >> > >>
> >> > >
> >> >
> >> >
> >>
> >>
> >>
> >>
> >>
> >>
>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
Cool, thanks :-) Some comments for ya...
How does this work with archive + plan deployment? I'm not sure that
the assembly artifact selection fits into the model if deploy/
undeploy... but I'm not sure. The point of the selection for
assemblies was to make it easier to change the dependency w/o having
to type in the full artifact details. Seems like with the module +
plan, you get half of that, but you will always have to configure the
plan... kinda negating the ease of use. Do you have a use case for
when you would want to pre-configure a set modules to deploy and then
deploy them with an alias? It might help me understand why you added
module selection here...
Why do we have 'distribute' and 'undeploy'? Why not 'deploy' and
'undeploy'?
Is there anyway to get around needing to "new DeploymentFactoryImpl
();"? It would be nice to have the plugin free from importing system
classes to deploy and delegate everything to a JMX proxy... so that
the plugin has a better chance of working with newer G servers, as
well as retaining compatibility with older ones).
In DistributeModuleMojo, why is #plan a String and not a File? The
plexus container will perform the equiv of your getFile() for you.
In DeployToolSupport, might want to just configure the deployer URI
and not worry about building that URI from a hostname, port. Still
need to add hostname (or raw URI) support to the other mojo's, so we
should make this consistent... right now shutdown can only happen for
localhost (see TODO).
Any reason why we have explicitly set the TCL in getDeploymentManager
(), was the TCL that maven setup not correct?
And finally.. :-) I'd like to see more logging used, for both debug
and info to show what is going on... for the goals at least... still
need to resolve the artifact selection, and if that is going to be
used for this, then we need to clean up the logging (and method
names) for getAssemblyArtifact(), since that was assembly specific,
and now it has changed to be more than that... still unsure that is
good, but let me know what you think the use cases care.
* * *
I hope you find all of my comments as constructive... I appreciate
your work :-)
--jason
On Sep 6, 2006, at 11:22 AM, Prasad Kashyap wrote:
> I have uploaded the DeployToolSupport.patch to
> http://issues.apache.org/jira/secure/attachment/12340282/
> DeployToolSuport.patch
>
> This patch adds DistributeModuleMojo and UndeployModuleMojo to the
> geronimo-maven-plugin. The deploy is done by conecting to the default
> uri "deployer:geronimo:jmx"
>
> The patch abstracts the artifact resolution code from
> InstallerMojoSupport.java up one level to ServerMojoSuport.java.
> DeployToolSupport.java extends ServerMojoSupport.java.
>
> Similar config options that one uses to specify an assembly for
> install can be reused to specify the modules for deploy/undeploy. The
> getAssembly() code that has been abstracted to ServerMojoSupport.java
> can now be reused to resolve the assembly artifacts or module
> artifacts.
>
> Let me know your thoughts. I'm amenable to any suggestions.
>
> Cheers
> Prasad.
>
>
>
>
>
> On 9/4/06, Jason Dillon <ja...@planet57.com> wrote:
>>
>> K, no worries. Though... looks like some time is going to be put
>> into
>> fixing up surefire in the week or so, so I don't think we will need
>> 2.8-SNAPSHOT for too long.
>>
>> --jason
>>
>>
>>
>> On Sep 4, 2006, at 10:53 AM, Bill Dudney wrote:
>> Hey Jason,
>>
>> Sorry for the false alarm, the tests don't run. Looks like 2.8-
>> SNAPSHOT is
>> what we will have to use until the surefire folks can do an update.
>>
>> TTFN,
>>
>> -bd-
>>
>>
>> On Sep 4, 2006, at 10:34 AM, Jason Dillon wrote:
>> Did the tests actually run? Make sure that it actually ran tests for
>> basic/*
>>
>> --jason
>>
>>
>>
>> On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
>> Hi All,
>>
>> The only down side I can see to testng is the reliance on surefire
>> 2.8-SNAPSHOT.
>>
>> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-
>> SNAPSHOT from
>> http://people.apache.org/maven-snapshot-repository and
>> everything seemed to work fine. Jason could you try that out so we
>> can get
>> rid of the 2.8-SNAPSHOT dependency?
>>
>> Apart from that I think testng is the way to go.
>>
>> TTFN,
>>
>> -bd-
>>
>> ------------------ begin patch ---------------------------
>> Index: pom.xml
>> ===================================================================
>> --- pom.xml (revision 439918)
>> +++ pom.xml (working copy)
>> @@ -93,7 +93,7 @@
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>>
>> <artifactId>maven-surefire-plugin</artifactId>
>> - <version>2.8-SNAPSHOT</version>
>> + <version>2.3-SNAPSHOT</version>
>> </plugin>
>> </plugins>
>> </pluginManagement>
>> @@ -116,14 +116,4 @@
>> </repository>
>> </repositories>
>>
>> - <!--
>> - NOTE: This is the repository where maven-surefire-plugin:2.8-
>> SNAPSHOT
>> comes from.
>> - -->
>> - <pluginRepositories>
>> - <pluginRepository>
>> - <id>tapestry.javaforge</id>
>> -
>> <url>http://howardlewisship.com/repository</url>
>> - </pluginRepository>
>> - </pluginRepositories>
>> -
>> </project>
>> ---------------------- end
>> patch--------------------------------------
>>
>>
>>
>>
>> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
>> Ya, I see this from time to time... not really sure why it pops up
>> sometimes
>> and not at others. Its easy enough to fix... but I think that its
>> much
>> easier with ng.
>>
>> Still talking to maven folks about getting a release of surefire
>> that works
>> with testng + jdk14, but the 2.8-SNAPSHOT works for now...
>>
>> --jason
>>
>>
>> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
>> > So you got around this? This was not happening for me so I'm not
>> sure
>> > what is going on, it appears from the next message that you got
>> it to
>> > work.
>> >
>> > I did see an UndeclaredThrowableException a time or two but it was
>> > always because the selenium server was not running.
>> >
>> > I'm going to dig into your testng stuff in the am and will post
>> > thoughts.
>> >
>> > TTFN,
>> >
>> > bd-
>> > On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>> >
>> > > FYI, I applied your patch... and surefire barfs as I thought
>> it might:
>> > >
>> > > <snip>
>> > > java.lang.reflect.UndeclaredThrowableException
>> > > at $Proxy0.addError(Unknown Source)
>> > > at
>> junit.framework.TestResult.addError(TestResult.java:36)
>> > > at
>> junit.framework.TestResult.runProtected(TestResult.java:
>> > > 133)
>> > > at junit.extensions.TestSetup.run(TestSetup.java:23)
>> > > at
>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > at sun.reflect.NativeMethodAccessorImpl.invoke
>> > > (NativeMethodAccessorImpl.java :39)
>> > > at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke
>> > > (DelegatingMethodAccessorImpl.java:25)
>> > > at java.lang.reflect.Method.invoke(Method.java:324)
>> > > at
>> org.apache.maven.surefire.junit.JUnitTestSet.execute
>> > > (JUnitTestSet.java:210)
>> > > at
>> > >
>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTes
>> t
>> > > Set(AbstractDirectoryTestSuite.java:135)
>> > > at
>> > >
>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
>> > > (AbstractDirectoryTestSuite.java:122)
>> > > at
>> org.apache.maven.surefire.Surefire.run(Surefire.java:129)
>> > > at
>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > at sun.reflect.NativeMethodAccessorImpl.invoke
>> > > (NativeMethodAccessorImpl.java:39)
>> > > at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke
>> > > (DelegatingMethodAccessorImpl.java:25)
>> > > at java.lang.reflect.Method.invoke (Method.java:324)
>> > > at
>> > >
>> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
>> > > (SurefireBooter.java:262)
>> > > at
>> org.apache.maven.surefire.booter.SurefireBooter.main
>> > > (SurefireBooter.java:787)
>> > > Caused by: java.lang.NoSuchMethodException:
>> > >
>> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName(
>> )
>> > > at java.lang.Class.getMethod(Class.java:986)
>> > > at
>> > >
>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStac
>> k
>> > > TraceWriter(TestListenerInvocationHandler.java:171)
>> > > at
>> > >
>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleA
>> d
>> > > dError(TestListenerInvocationHandler.java:160)
>> > > at
>> > >
>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
>> > > (TestListenerInvocationHandler.java:134)
>> > > ... 18 more
>> > > </snip>
>> > >
>> > > --jason
>> > >
>> > >
>> > > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>> > >
>> > >> Hi Jason,
>> > >>
>> > >> This is very cool indeed, thanks for putting the prototype
>> > >> together. I just submitted a patch that makes firefox pop up
>> only
>> > >> once to run all the tests (it actually pops up twice, once
>> during
>> > >> selenium.start() and then once for the test suite but the first
>> > >> time is for a split second).
>> > >>
>> > >> http://issues.apache.org/jira/browse/GERONIMO-2374
>> > >>
>> > >> It uses a JUnit TestDecorator to run setup only once for the
>> whole
>> > >> suite.
>> > >>
>> > >> IMO until we move to JDK 5 its not a great idea to move to
>> TestNG,
>> > >> you will get some benefits for sure but the source has to be
>> > >> parsed at runtime to get at the meta-data ( i.e. the info
>> about the
>> > >> tests). Krufty IMO.
>> > >>
>> > >> The patch has a comment about invalid XHTML. I believe the
>> > >> invalid XHTML part of the console is preventing the XPath find
>> > >> from working.
>> > >>
>> > >> TTFN,
>> > >>
>> > >> -bd-
>> > >>
>> > >>
>> > >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>> > >>
>> > >>> I've checked in a new top-level testsuite module and a few new
>> > >>> plugins to support it. This is only the start and I expect
>> it to
>> > >>> change over the next few weeks (er maybe months) as momentum
>> > >>> starts to pick up.
>> > >>>
>> > >>> I looked into Cargo, and while I think we should eventually use
>> > >>> it to start/stop the server... asis now its broke. We need to
>> > >>> provide a very simple and consistent way for Cargo to invoke
>> some
>> > >>> operations via JMX, which I hinted to in previous emails. Once
>> > >>> that is done, we can patch Cargo and hopefully get that
>> committed
>> > >>> to support G... but in the mean time I whipped up a simple
>> start
>> > >>> & stop mojo that uses Ant to start and stop the server. Its
>> > >>> very, very crude and we need to fix stuff like logging
>> output etc.
>> > >>>
>> > >>> I also played around a little with Selenium to make some tests
>> > >>> for the console. It wants a special server process started,
>> so I
>> > >>> created a selenium plugin which currently only starts that
>> server
>> > >>> in... well an icky way, but should work for the moment.
>> Need to
>> > >>> ask the m2 folks how to do this better.
>> > >>>
>> > >>> I created a console-testsuite module, which sets up the G
>> server
>> > >>> and Selenium server in pre-integration-test, and then uses the
>> > >>> maven-invoker-plugin to invoke the basic module to actually run
>> > >>> the tests. Only one test class right now, SimpleLoginTest,
>> which
>> > >>> does just that... logs in, logs in and then out and then
>> another
>> > >>> that logs in and clicks some links.
>> > >>>
>> > >>> I've got it all running from one mvn command and so far, on my
>> > >>> mac, it works fine. I'd imagine due to the location of the
>> > >>> assembly that Windows will not be able to run this... we'll
>> need
>> > >>> to fix this later. But those of you lucky enough to have a
>> Linux
>> > >>> box or a mac, you should be able to:
>> > >>>
>> > >>> mvn clean && mvn install
>> > >>> cd testsuite
>> > >>> mvn
>> > >>>
>> > >>> (the testsuite is not hooked up to the default build)
>> > >>>
>> > >>> And you should see Firefox popup a few times as Selenium goes
>> > >>> about clicking on stuff. Right now its only really doing
>> > >>> validation of the console page's titles... but if everything
>> > >>> works, then the invoker should not complain and the build
>> should
>> > >>> complete.
>> > >>>
>> > >>> A bunch more needs to be done... like creating a common test
>> > >>> support class for selenium tests... and probably switching
>> these
>> > >>> tests to use testng, so that the selenium setup is done
>> once, not
>> > >>> for each test... as it does now... so expect Firefox to
>> start and
>> > >>> stop a few times. We need to have some helpers to pick out the
>> > >>> console contents easily so they can be validated... I read that
>> > >>> selenium supports xpath or something, but I have not actually
>> > >>> tried any of that.
>> > >>>
>> > >>> This is just a very, very crude proof of concept... needs to
>> have
>> > >>> reporting features added, capture logs, blah, blah, blah.
>> > >>>
>> > >>> Anyways, I committed this so we have a common base to work
>> > >>> patches off of.
>> > >>>
>> > >>> One thing to note about the Selenium IDE, is that sometimes it
>> > >>> does not put into a waitForPageToLoad(), like if you record a
>> > >>> login to the console... you need to manually add that wait
>> after
>> > >>> the button is clicked or it will hose the test.
>> > >>>
>> > >>> I'm not married to any of this either... I just wanted
>> something
>> > >>> that worked... or kinda worked to show that its not going to be
>> > >>> that hard to make some rich integration tests.
>> > >>>
>> > >>> So, comments, suggestions, patches... all welcome.
>> > >>>
>> > >>> --jason
>> > >>>
>> > >>>
>> > >>
>> > >
>> >
>> >
>>
>>
>>
>>
>>
>>
Re: Testsuite module, with basic/crude Selenium support
Posted by Prasad Kashyap <go...@gmail.com>.
I have uploaded the DeployToolSupport.patch to
http://issues.apache.org/jira/secure/attachment/12340282/DeployToolSuport.patch
This patch adds DistributeModuleMojo and UndeployModuleMojo to the
geronimo-maven-plugin. The deploy is done by conecting to the default
uri "deployer:geronimo:jmx"
The patch abstracts the artifact resolution code from
InstallerMojoSupport.java up one level to ServerMojoSuport.java.
DeployToolSupport.java extends ServerMojoSupport.java.
Similar config options that one uses to specify an assembly for
install can be reused to specify the modules for deploy/undeploy. The
getAssembly() code that has been abstracted to ServerMojoSupport.java
can now be reused to resolve the assembly artifacts or module
artifacts.
Let me know your thoughts. I'm amenable to any suggestions.
Cheers
Prasad.
On 9/4/06, Jason Dillon <ja...@planet57.com> wrote:
>
> K, no worries. Though... looks like some time is going to be put into
> fixing up surefire in the week or so, so I don't think we will need
> 2.8-SNAPSHOT for too long.
>
> --jason
>
>
>
> On Sep 4, 2006, at 10:53 AM, Bill Dudney wrote:
> Hey Jason,
>
> Sorry for the false alarm, the tests don't run. Looks like 2.8-SNAPSHOT is
> what we will have to use until the surefire folks can do an update.
>
> TTFN,
>
> -bd-
>
>
> On Sep 4, 2006, at 10:34 AM, Jason Dillon wrote:
> Did the tests actually run? Make sure that it actually ran tests for
> basic/*
>
> --jason
>
>
>
> On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
> Hi All,
>
> The only down side I can see to testng is the reliance on surefire
> 2.8-SNAPSHOT.
>
> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-SNAPSHOT from
> http://people.apache.org/maven-snapshot-repository and
> everything seemed to work fine. Jason could you try that out so we can get
> rid of the 2.8-SNAPSHOT dependency?
>
> Apart from that I think testng is the way to go.
>
> TTFN,
>
> -bd-
>
> ------------------ begin patch ---------------------------
> Index: pom.xml
> ===================================================================
> --- pom.xml (revision 439918)
> +++ pom.xml (working copy)
> @@ -93,7 +93,7 @@
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
>
> <artifactId>maven-surefire-plugin</artifactId>
> - <version>2.8-SNAPSHOT</version>
> + <version>2.3-SNAPSHOT</version>
> </plugin>
> </plugins>
> </pluginManagement>
> @@ -116,14 +116,4 @@
> </repository>
> </repositories>
>
> - <!--
> - NOTE: This is the repository where maven-surefire-plugin:2.8-SNAPSHOT
> comes from.
> - -->
> - <pluginRepositories>
> - <pluginRepository>
> - <id>tapestry.javaforge</id>
> -
> <url>http://howardlewisship.com/repository</url>
> - </pluginRepository>
> - </pluginRepositories>
> -
> </project>
> ---------------------- end
> patch--------------------------------------
>
>
>
>
> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
> Ya, I see this from time to time... not really sure why it pops up sometimes
> and not at others. Its easy enough to fix... but I think that its much
> easier with ng.
>
> Still talking to maven folks about getting a release of surefire that works
> with testng + jdk14, but the 2.8-SNAPSHOT works for now...
>
> --jason
>
>
> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
> > So you got around this? This was not happening for me so I'm not sure
> > what is going on, it appears from the next message that you got it to
> > work.
> >
> > I did see an UndeclaredThrowableException a time or two but it was
> > always because the selenium server was not running.
> >
> > I'm going to dig into your testng stuff in the am and will post
> > thoughts.
> >
> > TTFN,
> >
> > bd-
> > On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
> >
> > > FYI, I applied your patch... and surefire barfs as I thought it might:
> > >
> > > <snip>
> > > java.lang.reflect.UndeclaredThrowableException
> > > at $Proxy0.addError(Unknown Source)
> > > at
> junit.framework.TestResult.addError(TestResult.java:36)
> > > at
> junit.framework.TestResult.runProtected(TestResult.java:
> > > 133)
> > > at junit.extensions.TestSetup.run(TestSetup.java:23)
> > > at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke
> > > (NativeMethodAccessorImpl.java :39)
> > > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke
> > > (DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java:324)
> > > at
> org.apache.maven.surefire.junit.JUnitTestSet.execute
> > > (JUnitTestSet.java:210)
> > > at
> > >
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> > > Set(AbstractDirectoryTestSuite.java:135)
> > > at
> > >
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
> > > (AbstractDirectoryTestSuite.java:122)
> > > at
> org.apache.maven.surefire.Surefire.run(Surefire.java:129)
> > > at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke
> > > (NativeMethodAccessorImpl.java:39)
> > > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke
> > > (DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke (Method.java:324)
> > > at
> > >
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
> > > (SurefireBooter.java:262)
> > > at
> org.apache.maven.surefire.booter.SurefireBooter.main
> > > (SurefireBooter.java:787)
> > > Caused by: java.lang.NoSuchMethodException:
> > >
> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName()
> > > at java.lang.Class.getMethod(Class.java:986)
> > > at
> > >
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStack
> > > TraceWriter(TestListenerInvocationHandler.java:171)
> > > at
> > >
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleAd
> > > dError(TestListenerInvocationHandler.java:160)
> > > at
> > >
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
> > > (TestListenerInvocationHandler.java:134)
> > > ... 18 more
> > > </snip>
> > >
> > > --jason
> > >
> > >
> > > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> > >
> > >> Hi Jason,
> > >>
> > >> This is very cool indeed, thanks for putting the prototype
> > >> together. I just submitted a patch that makes firefox pop up only
> > >> once to run all the tests (it actually pops up twice, once during
> > >> selenium.start() and then once for the test suite but the first
> > >> time is for a split second).
> > >>
> > >> http://issues.apache.org/jira/browse/GERONIMO-2374
> > >>
> > >> It uses a JUnit TestDecorator to run setup only once for the whole
> > >> suite.
> > >>
> > >> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> > >> you will get some benefits for sure but the source has to be
> > >> parsed at runtime to get at the meta-data ( i.e. the info about the
> > >> tests). Krufty IMO.
> > >>
> > >> The patch has a comment about invalid XHTML. I believe the
> > >> invalid XHTML part of the console is preventing the XPath find
> > >> from working.
> > >>
> > >> TTFN,
> > >>
> > >> -bd-
> > >>
> > >>
> > >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
> > >>
> > >>> I've checked in a new top-level testsuite module and a few new
> > >>> plugins to support it. This is only the start and I expect it to
> > >>> change over the next few weeks (er maybe months) as momentum
> > >>> starts to pick up.
> > >>>
> > >>> I looked into Cargo, and while I think we should eventually use
> > >>> it to start/stop the server... asis now its broke. We need to
> > >>> provide a very simple and consistent way for Cargo to invoke some
> > >>> operations via JMX, which I hinted to in previous emails. Once
> > >>> that is done, we can patch Cargo and hopefully get that committed
> > >>> to support G... but in the mean time I whipped up a simple start
> > >>> & stop mojo that uses Ant to start and stop the server. Its
> > >>> very, very crude and we need to fix stuff like logging output etc.
> > >>>
> > >>> I also played around a little with Selenium to make some tests
> > >>> for the console. It wants a special server process started, so I
> > >>> created a selenium plugin which currently only starts that server
> > >>> in... well an icky way, but should work for the moment. Need to
> > >>> ask the m2 folks how to do this better.
> > >>>
> > >>> I created a console-testsuite module, which sets up the G server
> > >>> and Selenium server in pre-integration-test, and then uses the
> > >>> maven-invoker-plugin to invoke the basic module to actually run
> > >>> the tests. Only one test class right now, SimpleLoginTest, which
> > >>> does just that... logs in, logs in and then out and then another
> > >>> that logs in and clicks some links.
> > >>>
> > >>> I've got it all running from one mvn command and so far, on my
> > >>> mac, it works fine. I'd imagine due to the location of the
> > >>> assembly that Windows will not be able to run this... we'll need
> > >>> to fix this later. But those of you lucky enough to have a Linux
> > >>> box or a mac, you should be able to:
> > >>>
> > >>> mvn clean && mvn install
> > >>> cd testsuite
> > >>> mvn
> > >>>
> > >>> (the testsuite is not hooked up to the default build)
> > >>>
> > >>> And you should see Firefox popup a few times as Selenium goes
> > >>> about clicking on stuff. Right now its only really doing
> > >>> validation of the console page's titles... but if everything
> > >>> works, then the invoker should not complain and the build should
> > >>> complete.
> > >>>
> > >>> A bunch more needs to be done... like creating a common test
> > >>> support class for selenium tests... and probably switching these
> > >>> tests to use testng, so that the selenium setup is done once, not
> > >>> for each test... as it does now... so expect Firefox to start and
> > >>> stop a few times. We need to have some helpers to pick out the
> > >>> console contents easily so they can be validated... I read that
> > >>> selenium supports xpath or something, but I have not actually
> > >>> tried any of that.
> > >>>
> > >>> This is just a very, very crude proof of concept... needs to have
> > >>> reporting features added, capture logs, blah, blah, blah.
> > >>>
> > >>> Anyways, I committed this so we have a common base to work
> > >>> patches off of.
> > >>>
> > >>> One thing to note about the Selenium IDE, is that sometimes it
> > >>> does not put into a waitForPageToLoad(), like if you record a
> > >>> login to the console... you need to manually add that wait after
> > >>> the button is clicked or it will hose the test.
> > >>>
> > >>> I'm not married to any of this either... I just wanted something
> > >>> that worked... or kinda worked to show that its not going to be
> > >>> that hard to make some rich integration tests.
> > >>>
> > >>> So, comments, suggestions, patches... all welcome.
> > >>>
> > >>> --jason
> > >>>
> > >>>
> > >>
> > >
> >
> >
>
>
>
>
>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
K, no worries. Though... looks like some time is going to be put
into fixing up surefire in the week or so, so I don't think we will
need 2.8-SNAPSHOT for too long.
--jason
On Sep 4, 2006, at 10:53 AM, Bill Dudney wrote:
> Hey Jason,
>
> Sorry for the false alarm, the tests don't run. Looks like 2.8-
> SNAPSHOT is what we will have to use until the surefire folks can
> do an update.
>
> TTFN,
>
> -bd-
>
> On Sep 4, 2006, at 10:34 AM, Jason Dillon wrote:
>
>> Did the tests actually run? Make sure that it actually ran tests
>> for basic/*
>>
>> --jason
>>
>>
>> On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
>>
>>> Hi All,
>>>
>>> The only down side I can see to testng is the reliance on
>>> surefire 2.8-SNAPSHOT.
>>>
>>> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-
>>> SNAPSHOT from http://people.apache.org/maven-snapshot-repository
>>> and everything seemed to work fine. Jason could you try that out
>>> so we can get rid of the 2.8-SNAPSHOT dependency?
>>>
>>> Apart from that I think testng is the way to go.
>>>
>>> TTFN,
>>>
>>> -bd-
>>>
>>> ------------------ begin patch ---------------------------
>>> Index: pom.xml
>>> ===================================================================
>>> --- pom.xml (revision 439918)
>>> +++ pom.xml (working copy)
>>> @@ -93,7 +93,7 @@
>>> <plugin>
>>> <groupId>org.apache.maven.plugins</groupId>
>>> <artifactId>maven-surefire-plugin</artifactId>
>>> - <version>2.8-SNAPSHOT</version>
>>> + <version>2.3-SNAPSHOT</version>
>>> </plugin>
>>> </plugins>
>>> </pluginManagement>
>>> @@ -116,14 +116,4 @@
>>> </repository>
>>> </repositories>
>>>
>>> - <!--
>>> - NOTE: This is the repository where maven-surefire-plugin:2.8-
>>> SNAPSHOT comes from.
>>> - -->
>>> - <pluginRepositories>
>>> - <pluginRepository>
>>> - <id>tapestry.javaforge</id>
>>> - <url>http://howardlewisship.com/repository</url>
>>> - </pluginRepository>
>>> - </pluginRepositories>
>>> -
>>> </project>
>>> ---------------------- end
>>> patch--------------------------------------
>>>
>>>
>>>
>>> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
>>>
>>>> Ya, I see this from time to time... not really sure why it pops
>>>> up sometimes and not at others. Its easy enough to fix... but I
>>>> think that its much easier with ng.
>>>>
>>>> Still talking to maven folks about getting a release of surefire
>>>> that works with testng + jdk14, but the 2.8-SNAPSHOT works for
>>>> now...
>>>>
>>>> --jason
>>>>
>>>>
>>>> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
>>>> So you got around this? This was not happening for me so I'm not
>>>> sure
>>>> what is going on, it appears from the next message that you got
>>>> it to
>>>> work.
>>>>
>>>> I did see an UndeclaredThrowableException a time or two but it was
>>>> always because the selenium server was not running.
>>>>
>>>> I'm going to dig into your testng stuff in the am and will post
>>>> thoughts.
>>>>
>>>> TTFN,
>>>>
>>>> bd-
>>>> On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>>>>
>>>> > FYI, I applied your patch... and surefire barfs as I thought
>>>> it might:
>>>> >
>>>> > <snip>
>>>> > java.lang.reflect.UndeclaredThrowableException
>>>> > at $Proxy0.addError(Unknown Source)
>>>> > at junit.framework.TestResult.addError(TestResult.java:
>>>> 36)
>>>> > at junit.framework.TestResult.runProtected
>>>> (TestResult.java:
>>>> > 133)
>>>> > at junit.extensions.TestSetup.run(TestSetup.java:23)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke
>>>> > (NativeMethodAccessorImpl.java :39)
>>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
>>>> > (DelegatingMethodAccessorImpl.java:25)
>>>> > at java.lang.reflect.Method.invoke(Method.java:324)
>>>> > at org.apache.maven.surefire.junit.JUnitTestSet.execute
>>>> > (JUnitTestSet.java:210)
>>>> > at
>>>> >
>>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeT
>>>> est
>>>> > Set(AbstractDirectoryTestSuite.java:135)
>>>> > at
>>>> >
>>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
>>>> > (AbstractDirectoryTestSuite.java:122)
>>>> > at org.apache.maven.surefire.Surefire.run
>>>> (Surefire.java:129)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke
>>>> > (NativeMethodAccessorImpl.java:39)
>>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
>>>> > (DelegatingMethodAccessorImpl.java:25)
>>>> > at java.lang.reflect.Method.invoke (Method.java:324)
>>>> > at
>>>> >
>>>> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
>>>> > (SurefireBooter.java:262)
>>>> > at org.apache.maven.surefire.booter.SurefireBooter.main
>>>> > (SurefireBooter.java:787)
>>>> > Caused by: java.lang.NoSuchMethodException:
>>>> >
>>>> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getNam
>>>> e()
>>>> > at java.lang.Class.getMethod(Class.java:986)
>>>> > at
>>>> >
>>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getSt
>>>> ack
>>>> > TraceWriter(TestListenerInvocationHandler.java:171)
>>>> > at
>>>> >
>>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handl
>>>> eAd
>>>> > dError(TestListenerInvocationHandler.java:160)
>>>> > at
>>>> >
>>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invok
>>>> e
>>>> > (TestListenerInvocationHandler.java:134)
>>>> > ... 18 more
>>>> > </snip>
>>>> >
>>>> > --jason
>>>> >
>>>> >
>>>> > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>>>> >
>>>> >> Hi Jason,
>>>> >>
>>>> >> This is very cool indeed, thanks for putting the prototype
>>>> >> together. I just submitted a patch that makes firefox pop up
>>>> only
>>>> >> once to run all the tests (it actually pops up twice, once
>>>> during
>>>> >> selenium.start() and then once for the test suite but the first
>>>> >> time is for a split second).
>>>> >>
>>>> >> http://issues.apache.org/jira/browse/GERONIMO-2374
>>>> >>
>>>> >> It uses a JUnit TestDecorator to run setup only once for the
>>>> whole
>>>> >> suite.
>>>> >>
>>>> >> IMO until we move to JDK 5 its not a great idea to move to
>>>> TestNG,
>>>> >> you will get some benefits for sure but the source has to be
>>>> >> parsed at runtime to get at the meta-data ( i.e. the info
>>>> about the
>>>> >> tests). Krufty IMO.
>>>> >>
>>>> >> The patch has a comment about invalid XHTML. I believe the
>>>> >> invalid XHTML part of the console is preventing the XPath find
>>>> >> from working.
>>>> >>
>>>> >> TTFN,
>>>> >>
>>>> >> -bd-
>>>> >>
>>>> >>
>>>> >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>>>> >>
>>>> >>> I've checked in a new top-level testsuite module and a few new
>>>> >>> plugins to support it. This is only the start and I expect
>>>> it to
>>>> >>> change over the next few weeks (er maybe months) as momentum
>>>> >>> starts to pick up.
>>>> >>>
>>>> >>> I looked into Cargo, and while I think we should eventually use
>>>> >>> it to start/stop the server... asis now its broke. We need to
>>>> >>> provide a very simple and consistent way for Cargo to invoke
>>>> some
>>>> >>> operations via JMX, which I hinted to in previous emails. Once
>>>> >>> that is done, we can patch Cargo and hopefully get that
>>>> committed
>>>> >>> to support G... but in the mean time I whipped up a simple
>>>> start
>>>> >>> & stop mojo that uses Ant to start and stop the server. Its
>>>> >>> very, very crude and we need to fix stuff like logging
>>>> output etc.
>>>> >>>
>>>> >>> I also played around a little with Selenium to make some tests
>>>> >>> for the console. It wants a special server process started,
>>>> so I
>>>> >>> created a selenium plugin which currently only starts that
>>>> server
>>>> >>> in... well an icky way, but should work for the moment.
>>>> Need to
>>>> >>> ask the m2 folks how to do this better.
>>>> >>>
>>>> >>> I created a console-testsuite module, which sets up the G
>>>> server
>>>> >>> and Selenium server in pre-integration-test, and then uses the
>>>> >>> maven-invoker-plugin to invoke the basic module to actually run
>>>> >>> the tests. Only one test class right now, SimpleLoginTest,
>>>> which
>>>> >>> does just that... logs in, logs in and then out and then
>>>> another
>>>> >>> that logs in and clicks some links.
>>>> >>>
>>>> >>> I've got it all running from one mvn command and so far, on my
>>>> >>> mac, it works fine. I'd imagine due to the location of the
>>>> >>> assembly that Windows will not be able to run this... we'll
>>>> need
>>>> >>> to fix this later. But those of you lucky enough to have a
>>>> Linux
>>>> >>> box or a mac, you should be able to:
>>>> >>>
>>>> >>> mvn clean && mvn install
>>>> >>> cd testsuite
>>>> >>> mvn
>>>> >>>
>>>> >>> (the testsuite is not hooked up to the default build)
>>>> >>>
>>>> >>> And you should see Firefox popup a few times as Selenium goes
>>>> >>> about clicking on stuff. Right now its only really doing
>>>> >>> validation of the console page's titles... but if everything
>>>> >>> works, then the invoker should not complain and the build
>>>> should
>>>> >>> complete.
>>>> >>>
>>>> >>> A bunch more needs to be done... like creating a common test
>>>> >>> support class for selenium tests... and probably switching
>>>> these
>>>> >>> tests to use testng, so that the selenium setup is done
>>>> once, not
>>>> >>> for each test... as it does now... so expect Firefox to
>>>> start and
>>>> >>> stop a few times. We need to have some helpers to pick out the
>>>> >>> console contents easily so they can be validated... I read that
>>>> >>> selenium supports xpath or something, but I have not actually
>>>> >>> tried any of that.
>>>> >>>
>>>> >>> This is just a very, very crude proof of concept... needs to
>>>> have
>>>> >>> reporting features added, capture logs, blah, blah, blah.
>>>> >>>
>>>> >>> Anyways, I committed this so we have a common base to work
>>>> >>> patches off of.
>>>> >>>
>>>> >>> One thing to note about the Selenium IDE, is that sometimes it
>>>> >>> does not put into a waitForPageToLoad(), like if you record a
>>>> >>> login to the console... you need to manually add that wait
>>>> after
>>>> >>> the button is clicked or it will hose the test.
>>>> >>>
>>>> >>> I'm not married to any of this either... I just wanted
>>>> something
>>>> >>> that worked... or kinda worked to show that its not going to be
>>>> >>> that hard to make some rich integration tests.
>>>> >>>
>>>> >>> So, comments, suggestions, patches... all welcome.
>>>> >>>
>>>> >>> --jason
>>>> >>>
>>>> >>>
>>>> >>
>>>> >
>>>>
>>>>
>>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Bill Dudney <bd...@mac.com>.
Hey Jason,
Sorry for the false alarm, the tests don't run. Looks like 2.8-
SNAPSHOT is what we will have to use until the surefire folks can do
an update.
TTFN,
-bd-
On Sep 4, 2006, at 10:34 AM, Jason Dillon wrote:
> Did the tests actually run? Make sure that it actually ran tests
> for basic/*
>
> --jason
>
>
> On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
>
>> Hi All,
>>
>> The only down side I can see to testng is the reliance on surefire
>> 2.8-SNAPSHOT.
>>
>> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-
>> SNAPSHOT from http://people.apache.org/maven-snapshot-repository
>> and everything seemed to work fine. Jason could you try that out
>> so we can get rid of the 2.8-SNAPSHOT dependency?
>>
>> Apart from that I think testng is the way to go.
>>
>> TTFN,
>>
>> -bd-
>>
>> ------------------ begin patch ---------------------------
>> Index: pom.xml
>> ===================================================================
>> --- pom.xml (revision 439918)
>> +++ pom.xml (working copy)
>> @@ -93,7 +93,7 @@
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-surefire-plugin</artifactId>
>> - <version>2.8-SNAPSHOT</version>
>> + <version>2.3-SNAPSHOT</version>
>> </plugin>
>> </plugins>
>> </pluginManagement>
>> @@ -116,14 +116,4 @@
>> </repository>
>> </repositories>
>>
>> - <!--
>> - NOTE: This is the repository where maven-surefire-plugin:2.8-
>> SNAPSHOT comes from.
>> - -->
>> - <pluginRepositories>
>> - <pluginRepository>
>> - <id>tapestry.javaforge</id>
>> - <url>http://howardlewisship.com/repository</url>
>> - </pluginRepository>
>> - </pluginRepositories>
>> -
>> </project>
>> ---------------------- end
>> patch--------------------------------------
>>
>>
>>
>> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
>>
>>> Ya, I see this from time to time... not really sure why it pops
>>> up sometimes and not at others. Its easy enough to fix... but I
>>> think that its much easier with ng.
>>>
>>> Still talking to maven folks about getting a release of surefire
>>> that works with testng + jdk14, but the 2.8-SNAPSHOT works for
>>> now...
>>>
>>> --jason
>>>
>>>
>>> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
>>> So you got around this? This was not happening for me so I'm not
>>> sure
>>> what is going on, it appears from the next message that you got
>>> it to
>>> work.
>>>
>>> I did see an UndeclaredThrowableException a time or two but it was
>>> always because the selenium server was not running.
>>>
>>> I'm going to dig into your testng stuff in the am and will post
>>> thoughts.
>>>
>>> TTFN,
>>>
>>> bd-
>>> On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>>>
>>> > FYI, I applied your patch... and surefire barfs as I thought it
>>> might:
>>> >
>>> > <snip>
>>> > java.lang.reflect.UndeclaredThrowableException
>>> > at $Proxy0.addError(Unknown Source)
>>> > at junit.framework.TestResult.addError(TestResult.java:36)
>>> > at junit.framework.TestResult.runProtected
>>> (TestResult.java:
>>> > 133)
>>> > at junit.extensions.TestSetup.run(TestSetup.java:23)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke
>>> > (NativeMethodAccessorImpl.java :39)
>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
>>> > (DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:324)
>>> > at org.apache.maven.surefire.junit.JUnitTestSet.execute
>>> > (JUnitTestSet.java:210)
>>> > at
>>> >
>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTe
>>> st
>>> > Set(AbstractDirectoryTestSuite.java:135)
>>> > at
>>> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
>>> > (AbstractDirectoryTestSuite.java:122)
>>> > at org.apache.maven.surefire.Surefire.run(Surefire.java:
>>> 129)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke
>>> > (NativeMethodAccessorImpl.java:39)
>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
>>> > (DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke (Method.java:324)
>>> > at
>>> > org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
>>> > (SurefireBooter.java:262)
>>> > at org.apache.maven.surefire.booter.SurefireBooter.main
>>> > (SurefireBooter.java:787)
>>> > Caused by: java.lang.NoSuchMethodException:
>>> >
>>> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName
>>> ()
>>> > at java.lang.Class.getMethod(Class.java:986)
>>> > at
>>> >
>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getSta
>>> ck
>>> > TraceWriter(TestListenerInvocationHandler.java:171)
>>> > at
>>> >
>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handle
>>> Ad
>>> > dError(TestListenerInvocationHandler.java:160)
>>> > at
>>> >
>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
>>> > (TestListenerInvocationHandler.java:134)
>>> > ... 18 more
>>> > </snip>
>>> >
>>> > --jason
>>> >
>>> >
>>> > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>>> >
>>> >> Hi Jason,
>>> >>
>>> >> This is very cool indeed, thanks for putting the prototype
>>> >> together. I just submitted a patch that makes firefox pop up only
>>> >> once to run all the tests (it actually pops up twice, once during
>>> >> selenium.start() and then once for the test suite but the first
>>> >> time is for a split second).
>>> >>
>>> >> http://issues.apache.org/jira/browse/GERONIMO-2374
>>> >>
>>> >> It uses a JUnit TestDecorator to run setup only once for the
>>> whole
>>> >> suite.
>>> >>
>>> >> IMO until we move to JDK 5 its not a great idea to move to
>>> TestNG,
>>> >> you will get some benefits for sure but the source has to be
>>> >> parsed at runtime to get at the meta-data ( i.e. the info
>>> about the
>>> >> tests). Krufty IMO.
>>> >>
>>> >> The patch has a comment about invalid XHTML. I believe the
>>> >> invalid XHTML part of the console is preventing the XPath find
>>> >> from working.
>>> >>
>>> >> TTFN,
>>> >>
>>> >> -bd-
>>> >>
>>> >>
>>> >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>>> >>
>>> >>> I've checked in a new top-level testsuite module and a few new
>>> >>> plugins to support it. This is only the start and I expect
>>> it to
>>> >>> change over the next few weeks (er maybe months) as momentum
>>> >>> starts to pick up.
>>> >>>
>>> >>> I looked into Cargo, and while I think we should eventually use
>>> >>> it to start/stop the server... asis now its broke. We need to
>>> >>> provide a very simple and consistent way for Cargo to invoke
>>> some
>>> >>> operations via JMX, which I hinted to in previous emails. Once
>>> >>> that is done, we can patch Cargo and hopefully get that
>>> committed
>>> >>> to support G... but in the mean time I whipped up a simple start
>>> >>> & stop mojo that uses Ant to start and stop the server. Its
>>> >>> very, very crude and we need to fix stuff like logging output
>>> etc.
>>> >>>
>>> >>> I also played around a little with Selenium to make some tests
>>> >>> for the console. It wants a special server process started,
>>> so I
>>> >>> created a selenium plugin which currently only starts that
>>> server
>>> >>> in... well an icky way, but should work for the moment. Need to
>>> >>> ask the m2 folks how to do this better.
>>> >>>
>>> >>> I created a console-testsuite module, which sets up the G server
>>> >>> and Selenium server in pre-integration-test, and then uses the
>>> >>> maven-invoker-plugin to invoke the basic module to actually run
>>> >>> the tests. Only one test class right now, SimpleLoginTest,
>>> which
>>> >>> does just that... logs in, logs in and then out and then another
>>> >>> that logs in and clicks some links.
>>> >>>
>>> >>> I've got it all running from one mvn command and so far, on my
>>> >>> mac, it works fine. I'd imagine due to the location of the
>>> >>> assembly that Windows will not be able to run this... we'll need
>>> >>> to fix this later. But those of you lucky enough to have a
>>> Linux
>>> >>> box or a mac, you should be able to:
>>> >>>
>>> >>> mvn clean && mvn install
>>> >>> cd testsuite
>>> >>> mvn
>>> >>>
>>> >>> (the testsuite is not hooked up to the default build)
>>> >>>
>>> >>> And you should see Firefox popup a few times as Selenium goes
>>> >>> about clicking on stuff. Right now its only really doing
>>> >>> validation of the console page's titles... but if everything
>>> >>> works, then the invoker should not complain and the build should
>>> >>> complete.
>>> >>>
>>> >>> A bunch more needs to be done... like creating a common test
>>> >>> support class for selenium tests... and probably switching these
>>> >>> tests to use testng, so that the selenium setup is done once,
>>> not
>>> >>> for each test... as it does now... so expect Firefox to start
>>> and
>>> >>> stop a few times. We need to have some helpers to pick out the
>>> >>> console contents easily so they can be validated... I read that
>>> >>> selenium supports xpath or something, but I have not actually
>>> >>> tried any of that.
>>> >>>
>>> >>> This is just a very, very crude proof of concept... needs to
>>> have
>>> >>> reporting features added, capture logs, blah, blah, blah.
>>> >>>
>>> >>> Anyways, I committed this so we have a common base to work
>>> >>> patches off of.
>>> >>>
>>> >>> One thing to note about the Selenium IDE, is that sometimes it
>>> >>> does not put into a waitForPageToLoad(), like if you record a
>>> >>> login to the console... you need to manually add that wait after
>>> >>> the button is clicked or it will hose the test.
>>> >>>
>>> >>> I'm not married to any of this either... I just wanted something
>>> >>> that worked... or kinda worked to show that its not going to be
>>> >>> that hard to make some rich integration tests.
>>> >>>
>>> >>> So, comments, suggestions, patches... all welcome.
>>> >>>
>>> >>> --jason
>>> >>>
>>> >>>
>>> >>
>>> >
>>>
>>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
Did the tests actually run? Make sure that it actually ran tests for
basic/*
--jason
On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
> Hi All,
>
> The only down side I can see to testng is the reliance on surefire
> 2.8-SNAPSHOT.
>
> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-
> SNAPSHOT from http://people.apache.org/maven-snapshot-repository
> and everything seemed to work fine. Jason could you try that out so
> we can get rid of the 2.8-SNAPSHOT dependency?
>
> Apart from that I think testng is the way to go.
>
> TTFN,
>
> -bd-
>
> ------------------ begin patch ---------------------------
> Index: pom.xml
> ===================================================================
> --- pom.xml (revision 439918)
> +++ pom.xml (working copy)
> @@ -93,7 +93,7 @@
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> - <version>2.8-SNAPSHOT</version>
> + <version>2.3-SNAPSHOT</version>
> </plugin>
> </plugins>
> </pluginManagement>
> @@ -116,14 +116,4 @@
> </repository>
> </repositories>
>
> - <!--
> - NOTE: This is the repository where maven-surefire-plugin:2.8-
> SNAPSHOT comes from.
> - -->
> - <pluginRepositories>
> - <pluginRepository>
> - <id>tapestry.javaforge</id>
> - <url>http://howardlewisship.com/repository</url>
> - </pluginRepository>
> - </pluginRepositories>
> -
> </project>
> ---------------------- end patch--------------------------------------
>
>
>
> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
>
>> Ya, I see this from time to time... not really sure why it pops up
>> sometimes and not at others. Its easy enough to fix... but I
>> think that its much easier with ng.
>>
>> Still talking to maven folks about getting a release of surefire
>> that works with testng + jdk14, but the 2.8-SNAPSHOT works for now...
>>
>> --jason
>>
>>
>> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
>> So you got around this? This was not happening for me so I'm not sure
>> what is going on, it appears from the next message that you got it to
>> work.
>>
>> I did see an UndeclaredThrowableException a time or two but it was
>> always because the selenium server was not running.
>>
>> I'm going to dig into your testng stuff in the am and will post
>> thoughts.
>>
>> TTFN,
>>
>> bd-
>> On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>>
>> > FYI, I applied your patch... and surefire barfs as I thought it
>> might:
>> >
>> > <snip>
>> > java.lang.reflect.UndeclaredThrowableException
>> > at $Proxy0.addError(Unknown Source)
>> > at junit.framework.TestResult.addError(TestResult.java:36)
>> > at junit.framework.TestResult.runProtected(TestResult.java:
>> > 133)
>> > at junit.extensions.TestSetup.run(TestSetup.java:23)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke
>> > (NativeMethodAccessorImpl.java :39)
>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
>> > (DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:324)
>> > at org.apache.maven.surefire.junit.JUnitTestSet.execute
>> > (JUnitTestSet.java:210)
>> > at
>> >
>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTes
>> t
>> > Set(AbstractDirectoryTestSuite.java:135)
>> > at
>> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
>> > (AbstractDirectoryTestSuite.java:122)
>> > at org.apache.maven.surefire.Surefire.run(Surefire.java:
>> 129)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke
>> > (NativeMethodAccessorImpl.java:39)
>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
>> > (DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke (Method.java:324)
>> > at
>> > org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
>> > (SurefireBooter.java:262)
>> > at org.apache.maven.surefire.booter.SurefireBooter.main
>> > (SurefireBooter.java:787)
>> > Caused by: java.lang.NoSuchMethodException:
>> >
>> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName(
>> )
>> > at java.lang.Class.getMethod(Class.java:986)
>> > at
>> >
>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStac
>> k
>> > TraceWriter(TestListenerInvocationHandler.java:171)
>> > at
>> >
>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleA
>> d
>> > dError(TestListenerInvocationHandler.java:160)
>> > at
>> >
>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
>> > (TestListenerInvocationHandler.java:134)
>> > ... 18 more
>> > </snip>
>> >
>> > --jason
>> >
>> >
>> > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>> >
>> >> Hi Jason,
>> >>
>> >> This is very cool indeed, thanks for putting the prototype
>> >> together. I just submitted a patch that makes firefox pop up only
>> >> once to run all the tests (it actually pops up twice, once during
>> >> selenium.start() and then once for the test suite but the first
>> >> time is for a split second).
>> >>
>> >> http://issues.apache.org/jira/browse/GERONIMO-2374
>> >>
>> >> It uses a JUnit TestDecorator to run setup only once for the whole
>> >> suite.
>> >>
>> >> IMO until we move to JDK 5 its not a great idea to move to TestNG,
>> >> you will get some benefits for sure but the source has to be
>> >> parsed at runtime to get at the meta-data ( i.e. the info about
>> the
>> >> tests). Krufty IMO.
>> >>
>> >> The patch has a comment about invalid XHTML. I believe the
>> >> invalid XHTML part of the console is preventing the XPath find
>> >> from working.
>> >>
>> >> TTFN,
>> >>
>> >> -bd-
>> >>
>> >>
>> >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>> >>
>> >>> I've checked in a new top-level testsuite module and a few new
>> >>> plugins to support it. This is only the start and I expect it to
>> >>> change over the next few weeks (er maybe months) as momentum
>> >>> starts to pick up.
>> >>>
>> >>> I looked into Cargo, and while I think we should eventually use
>> >>> it to start/stop the server... asis now its broke. We need to
>> >>> provide a very simple and consistent way for Cargo to invoke some
>> >>> operations via JMX, which I hinted to in previous emails. Once
>> >>> that is done, we can patch Cargo and hopefully get that committed
>> >>> to support G... but in the mean time I whipped up a simple start
>> >>> & stop mojo that uses Ant to start and stop the server. Its
>> >>> very, very crude and we need to fix stuff like logging output
>> etc.
>> >>>
>> >>> I also played around a little with Selenium to make some tests
>> >>> for the console. It wants a special server process started, so I
>> >>> created a selenium plugin which currently only starts that server
>> >>> in... well an icky way, but should work for the moment. Need to
>> >>> ask the m2 folks how to do this better.
>> >>>
>> >>> I created a console-testsuite module, which sets up the G server
>> >>> and Selenium server in pre-integration-test, and then uses the
>> >>> maven-invoker-plugin to invoke the basic module to actually run
>> >>> the tests. Only one test class right now, SimpleLoginTest, which
>> >>> does just that... logs in, logs in and then out and then another
>> >>> that logs in and clicks some links.
>> >>>
>> >>> I've got it all running from one mvn command and so far, on my
>> >>> mac, it works fine. I'd imagine due to the location of the
>> >>> assembly that Windows will not be able to run this... we'll need
>> >>> to fix this later. But those of you lucky enough to have a Linux
>> >>> box or a mac, you should be able to:
>> >>>
>> >>> mvn clean && mvn install
>> >>> cd testsuite
>> >>> mvn
>> >>>
>> >>> (the testsuite is not hooked up to the default build)
>> >>>
>> >>> And you should see Firefox popup a few times as Selenium goes
>> >>> about clicking on stuff. Right now its only really doing
>> >>> validation of the console page's titles... but if everything
>> >>> works, then the invoker should not complain and the build should
>> >>> complete.
>> >>>
>> >>> A bunch more needs to be done... like creating a common test
>> >>> support class for selenium tests... and probably switching these
>> >>> tests to use testng, so that the selenium setup is done once, not
>> >>> for each test... as it does now... so expect Firefox to start and
>> >>> stop a few times. We need to have some helpers to pick out the
>> >>> console contents easily so they can be validated... I read that
>> >>> selenium supports xpath or something, but I have not actually
>> >>> tried any of that.
>> >>>
>> >>> This is just a very, very crude proof of concept... needs to have
>> >>> reporting features added, capture logs, blah, blah, blah.
>> >>>
>> >>> Anyways, I committed this so we have a common base to work
>> >>> patches off of.
>> >>>
>> >>> One thing to note about the Selenium IDE, is that sometimes it
>> >>> does not put into a waitForPageToLoad(), like if you record a
>> >>> login to the console... you need to manually add that wait after
>> >>> the button is clicked or it will hose the test.
>> >>>
>> >>> I'm not married to any of this either... I just wanted something
>> >>> that worked... or kinda worked to show that its not going to be
>> >>> that hard to make some rich integration tests.
>> >>>
>> >>> So, comments, suggestions, patches... all welcome.
>> >>>
>> >>> --jason
>> >>>
>> >>>
>> >>
>> >
>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Bill Dudney <bd...@apache.org>.
Hi All,
The only down side I can see to testng is the reliance on surefire
2.8-SNAPSHOT.
The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-SNAPSHOT
from http://people.apache.org/maven-snapshot-repository and
everything seemed to work fine. Jason could you try that out so we
can get rid of the 2.8-SNAPSHOT dependency?
Apart from that I think testng is the way to go.
TTFN,
-bd-
------------------ begin patch ---------------------------
Index: pom.xml
===================================================================
--- pom.xml (revision 439918)
+++ pom.xml (working copy)
@@ -93,7 +93,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>2.8-SNAPSHOT</version>
+ <version>2.3-SNAPSHOT</version>
</plugin>
</plugins>
</pluginManagement>
@@ -116,14 +116,4 @@
</repository>
</repositories>
- <!--
- NOTE: This is the repository where maven-surefire-plugin:2.8-
SNAPSHOT comes from.
- -->
- <pluginRepositories>
- <pluginRepository>
- <id>tapestry.javaforge</id>
- <url>http://howardlewisship.com/repository</url>
- </pluginRepository>
- </pluginRepositories>
-
</project>
---------------------- end patch--------------------------------------
On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
> Ya, I see this from time to time... not really sure why it pops up
> sometimes and not at others. Its easy enough to fix... but I think
> that its much easier with ng.
>
> Still talking to maven folks about getting a release of surefire
> that works with testng + jdk14, but the 2.8-SNAPSHOT works for now...
>
> --jason
>
>
> On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
> So you got around this? This was not happening for me so I'm not sure
> what is going on, it appears from the next message that you got it to
> work.
>
> I did see an UndeclaredThrowableException a time or two but it was
> always because the selenium server was not running.
>
> I'm going to dig into your testng stuff in the am and will post
> thoughts.
>
> TTFN,
>
> bd-
> On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>
> > FYI, I applied your patch... and surefire barfs as I thought it
> might:
> >
> > <snip>
> > java.lang.reflect.UndeclaredThrowableException
> > at $Proxy0.addError(Unknown Source)
> > at junit.framework.TestResult.addError(TestResult.java:36)
> > at junit.framework.TestResult.runProtected(TestResult.java:
> > 133)
> > at junit.extensions.TestSetup.run(TestSetup.java:23)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke
> > (NativeMethodAccessorImpl.java :39)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
> > (DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:324)
> > at org.apache.maven.surefire.junit.JUnitTestSet.execute
> > (JUnitTestSet.java:210)
> > at
> >
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> > Set(AbstractDirectoryTestSuite.java:135)
> > at
> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
> > (AbstractDirectoryTestSuite.java:122)
> > at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke
> > (NativeMethodAccessorImpl.java:39)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
> > (DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke (Method.java:324)
> > at
> > org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
> > (SurefireBooter.java:262)
> > at org.apache.maven.surefire.booter.SurefireBooter.main
> > (SurefireBooter.java:787)
> > Caused by: java.lang.NoSuchMethodException:
> >
> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName()
> > at java.lang.Class.getMethod(Class.java:986)
> > at
> >
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStack
> > TraceWriter(TestListenerInvocationHandler.java:171)
> > at
> >
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleAd
> > dError(TestListenerInvocationHandler.java:160)
> > at
> > org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
> > (TestListenerInvocationHandler.java:134)
> > ... 18 more
> > </snip>
> >
> > --jason
> >
> >
> > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> >
> >> Hi Jason,
> >>
> >> This is very cool indeed, thanks for putting the prototype
> >> together. I just submitted a patch that makes firefox pop up only
> >> once to run all the tests (it actually pops up twice, once during
> >> selenium.start() and then once for the test suite but the first
> >> time is for a split second).
> >>
> >> http://issues.apache.org/jira/browse/GERONIMO-2374
> >>
> >> It uses a JUnit TestDecorator to run setup only once for the whole
> >> suite.
> >>
> >> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> >> you will get some benefits for sure but the source has to be
> >> parsed at runtime to get at the meta-data ( i.e. the info about the
> >> tests). Krufty IMO.
> >>
> >> The patch has a comment about invalid XHTML. I believe the
> >> invalid XHTML part of the console is preventing the XPath find
> >> from working.
> >>
> >> TTFN,
> >>
> >> -bd-
> >>
> >>
> >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
> >>
> >>> I've checked in a new top-level testsuite module and a few new
> >>> plugins to support it. This is only the start and I expect it to
> >>> change over the next few weeks (er maybe months) as momentum
> >>> starts to pick up.
> >>>
> >>> I looked into Cargo, and while I think we should eventually use
> >>> it to start/stop the server... asis now its broke. We need to
> >>> provide a very simple and consistent way for Cargo to invoke some
> >>> operations via JMX, which I hinted to in previous emails. Once
> >>> that is done, we can patch Cargo and hopefully get that committed
> >>> to support G... but in the mean time I whipped up a simple start
> >>> & stop mojo that uses Ant to start and stop the server. Its
> >>> very, very crude and we need to fix stuff like logging output etc.
> >>>
> >>> I also played around a little with Selenium to make some tests
> >>> for the console. It wants a special server process started, so I
> >>> created a selenium plugin which currently only starts that server
> >>> in... well an icky way, but should work for the moment. Need to
> >>> ask the m2 folks how to do this better.
> >>>
> >>> I created a console-testsuite module, which sets up the G server
> >>> and Selenium server in pre-integration-test, and then uses the
> >>> maven-invoker-plugin to invoke the basic module to actually run
> >>> the tests. Only one test class right now, SimpleLoginTest, which
> >>> does just that... logs in, logs in and then out and then another
> >>> that logs in and clicks some links.
> >>>
> >>> I've got it all running from one mvn command and so far, on my
> >>> mac, it works fine. I'd imagine due to the location of the
> >>> assembly that Windows will not be able to run this... we'll need
> >>> to fix this later. But those of you lucky enough to have a Linux
> >>> box or a mac, you should be able to:
> >>>
> >>> mvn clean && mvn install
> >>> cd testsuite
> >>> mvn
> >>>
> >>> (the testsuite is not hooked up to the default build)
> >>>
> >>> And you should see Firefox popup a few times as Selenium goes
> >>> about clicking on stuff. Right now its only really doing
> >>> validation of the console page's titles... but if everything
> >>> works, then the invoker should not complain and the build should
> >>> complete.
> >>>
> >>> A bunch more needs to be done... like creating a common test
> >>> support class for selenium tests... and probably switching these
> >>> tests to use testng, so that the selenium setup is done once, not
> >>> for each test... as it does now... so expect Firefox to start and
> >>> stop a few times. We need to have some helpers to pick out the
> >>> console contents easily so they can be validated... I read that
> >>> selenium supports xpath or something, but I have not actually
> >>> tried any of that.
> >>>
> >>> This is just a very, very crude proof of concept... needs to have
> >>> reporting features added, capture logs, blah, blah, blah.
> >>>
> >>> Anyways, I committed this so we have a common base to work
> >>> patches off of.
> >>>
> >>> One thing to note about the Selenium IDE, is that sometimes it
> >>> does not put into a waitForPageToLoad(), like if you record a
> >>> login to the console... you need to manually add that wait after
> >>> the button is clicked or it will hose the test.
> >>>
> >>> I'm not married to any of this either... I just wanted something
> >>> that worked... or kinda worked to show that its not going to be
> >>> that hard to make some rich integration tests.
> >>>
> >>> So, comments, suggestions, patches... all welcome.
> >>>
> >>> --jason
> >>>
> >>>
> >>
> >
>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
Ya, I see this from time to time... not really sure why it pops up sometimes
and not at others. Its easy enough to fix... but I think that its much
easier with ng.
Still talking to maven folks about getting a release of surefire that works
with testng + jdk14, but the 2.8-SNAPSHOT works for now...
--jason
On 9/3/06, Bill Dudney <bd...@apache.org> wrote:
>
> So you got around this? This was not happening for me so I'm not sure
> what is going on, it appears from the next message that you got it to
> work.
>
> I did see an UndeclaredThrowableException a time or two but it was
> always because the selenium server was not running.
>
> I'm going to dig into your testng stuff in the am and will post
> thoughts.
>
> TTFN,
>
> bd-
> On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>
> > FYI, I applied your patch... and surefire barfs as I thought it might:
> >
> > <snip>
> > java.lang.reflect.UndeclaredThrowableException
> > at $Proxy0.addError(Unknown Source)
> > at junit.framework.TestResult.addError(TestResult.java:36)
> > at junit.framework.TestResult.runProtected(TestResult.java:
> > 133)
> > at junit.extensions.TestSetup.run(TestSetup.java:23)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke
> > (NativeMethodAccessorImpl.java:39)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
> > (DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:324)
> > at org.apache.maven.surefire.junit.JUnitTestSet.execute
> > (JUnitTestSet.java:210)
> > at
> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> > Set(AbstractDirectoryTestSuite.java:135)
> > at
> > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
> > (AbstractDirectoryTestSuite.java:122)
> > at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke
> > (NativeMethodAccessorImpl.java:39)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke
> > (DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:324)
> > at
> > org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
> > (SurefireBooter.java:262)
> > at org.apache.maven.surefire.booter.SurefireBooter.main
> > (SurefireBooter.java:787)
> > Caused by: java.lang.NoSuchMethodException:
> > org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName()
> > at java.lang.Class.getMethod(Class.java:986)
> > at
> > org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStack
> > TraceWriter(TestListenerInvocationHandler.java:171)
> > at
> > org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleAd
> > dError(TestListenerInvocationHandler.java:160)
> > at
> > org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
> > (TestListenerInvocationHandler.java:134)
> > ... 18 more
> > </snip>
> >
> > --jason
> >
> >
> > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> >
> >> Hi Jason,
> >>
> >> This is very cool indeed, thanks for putting the prototype
> >> together. I just submitted a patch that makes firefox pop up only
> >> once to run all the tests (it actually pops up twice, once during
> >> selenium.start() and then once for the test suite but the first
> >> time is for a split second).
> >>
> >> http://issues.apache.org/jira/browse/GERONIMO-2374
> >>
> >> It uses a JUnit TestDecorator to run setup only once for the whole
> >> suite.
> >>
> >> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> >> you will get some benefits for sure but the source has to be
> >> parsed at runtime to get at the meta-data (i.e. the info about the
> >> tests). Krufty IMO.
> >>
> >> The patch has a comment about invalid XHTML. I believe the
> >> invalid XHTML part of the console is preventing the XPath find
> >> from working.
> >>
> >> TTFN,
> >>
> >> -bd-
> >>
> >>
> >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
> >>
> >>> I've checked in a new top-level testsuite module and a few new
> >>> plugins to support it. This is only the start and I expect it to
> >>> change over the next few weeks (er maybe months) as momentum
> >>> starts to pick up.
> >>>
> >>> I looked into Cargo, and while I think we should eventually use
> >>> it to start/stop the server... asis now its broke. We need to
> >>> provide a very simple and consistent way for Cargo to invoke some
> >>> operations via JMX, which I hinted to in previous emails. Once
> >>> that is done, we can patch Cargo and hopefully get that committed
> >>> to support G... but in the mean time I whipped up a simple start
> >>> & stop mojo that uses Ant to start and stop the server. Its
> >>> very, very crude and we need to fix stuff like logging output etc.
> >>>
> >>> I also played around a little with Selenium to make some tests
> >>> for the console. It wants a special server process started, so I
> >>> created a selenium plugin which currently only starts that server
> >>> in... well an icky way, but should work for the moment. Need to
> >>> ask the m2 folks how to do this better.
> >>>
> >>> I created a console-testsuite module, which sets up the G server
> >>> and Selenium server in pre-integration-test, and then uses the
> >>> maven-invoker-plugin to invoke the basic module to actually run
> >>> the tests. Only one test class right now, SimpleLoginTest, which
> >>> does just that... logs in, logs in and then out and then another
> >>> that logs in and clicks some links.
> >>>
> >>> I've got it all running from one mvn command and so far, on my
> >>> mac, it works fine. I'd imagine due to the location of the
> >>> assembly that Windows will not be able to run this... we'll need
> >>> to fix this later. But those of you lucky enough to have a Linux
> >>> box or a mac, you should be able to:
> >>>
> >>> mvn clean && mvn install
> >>> cd testsuite
> >>> mvn
> >>>
> >>> (the testsuite is not hooked up to the default build)
> >>>
> >>> And you should see Firefox popup a few times as Selenium goes
> >>> about clicking on stuff. Right now its only really doing
> >>> validation of the console page's titles... but if everything
> >>> works, then the invoker should not complain and the build should
> >>> complete.
> >>>
> >>> A bunch more needs to be done... like creating a common test
> >>> support class for selenium tests... and probably switching these
> >>> tests to use testng, so that the selenium setup is done once, not
> >>> for each test... as it does now... so expect Firefox to start and
> >>> stop a few times. We need to have some helpers to pick out the
> >>> console contents easily so they can be validated... I read that
> >>> selenium supports xpath or something, but I have not actually
> >>> tried any of that.
> >>>
> >>> This is just a very, very crude proof of concept... needs to have
> >>> reporting features added, capture logs, blah, blah, blah.
> >>>
> >>> Anyways, I committed this so we have a common base to work
> >>> patches off of.
> >>>
> >>> One thing to note about the Selenium IDE, is that sometimes it
> >>> does not put into a waitForPageToLoad(), like if you record a
> >>> login to the console... you need to manually add that wait after
> >>> the button is clicked or it will hose the test.
> >>>
> >>> I'm not married to any of this either... I just wanted something
> >>> that worked... or kinda worked to show that its not going to be
> >>> that hard to make some rich integration tests.
> >>>
> >>> So, comments, suggestions, patches... all welcome.
> >>>
> >>> --jason
> >>>
> >>>
> >>
> >
>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Bill Dudney <bd...@apache.org>.
So you got around this? This was not happening for me so I'm not sure
what is going on, it appears from the next message that you got it to
work.
I did see an UndeclaredThrowableException a time or two but it was
always because the selenium server was not running.
I'm going to dig into your testng stuff in the am and will post
thoughts.
TTFN,
bd-
On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
> FYI, I applied your patch... and surefire barfs as I thought it might:
>
> <snip>
> java.lang.reflect.UndeclaredThrowableException
> at $Proxy0.addError(Unknown Source)
> at junit.framework.TestResult.addError(TestResult.java:36)
> at junit.framework.TestResult.runProtected(TestResult.java:
> 133)
> at junit.extensions.TestSetup.run(TestSetup.java:23)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:324)
> at org.apache.maven.surefire.junit.JUnitTestSet.execute
> (JUnitTestSet.java:210)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> Set(AbstractDirectoryTestSuite.java:135)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
> (AbstractDirectoryTestSuite.java:122)
> at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:324)
> at
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
> (SurefireBooter.java:262)
> at org.apache.maven.surefire.booter.SurefireBooter.main
> (SurefireBooter.java:787)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName()
> at java.lang.Class.getMethod(Class.java:986)
> at
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStack
> TraceWriter(TestListenerInvocationHandler.java:171)
> at
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleAd
> dError(TestListenerInvocationHandler.java:160)
> at
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
> (TestListenerInvocationHandler.java:134)
> ... 18 more
> </snip>
>
> --jason
>
>
> On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>
>> Hi Jason,
>>
>> This is very cool indeed, thanks for putting the prototype
>> together. I just submitted a patch that makes firefox pop up only
>> once to run all the tests (it actually pops up twice, once during
>> selenium.start() and then once for the test suite but the first
>> time is for a split second).
>>
>> http://issues.apache.org/jira/browse/GERONIMO-2374
>>
>> It uses a JUnit TestDecorator to run setup only once for the whole
>> suite.
>>
>> IMO until we move to JDK 5 its not a great idea to move to TestNG,
>> you will get some benefits for sure but the source has to be
>> parsed at runtime to get at the meta-data (i.e. the info about the
>> tests). Krufty IMO.
>>
>> The patch has a comment about invalid XHTML. I believe the
>> invalid XHTML part of the console is preventing the XPath find
>> from working.
>>
>> TTFN,
>>
>> -bd-
>>
>>
>> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>>
>>> I've checked in a new top-level testsuite module and a few new
>>> plugins to support it. This is only the start and I expect it to
>>> change over the next few weeks (er maybe months) as momentum
>>> starts to pick up.
>>>
>>> I looked into Cargo, and while I think we should eventually use
>>> it to start/stop the server... asis now its broke. We need to
>>> provide a very simple and consistent way for Cargo to invoke some
>>> operations via JMX, which I hinted to in previous emails. Once
>>> that is done, we can patch Cargo and hopefully get that committed
>>> to support G... but in the mean time I whipped up a simple start
>>> & stop mojo that uses Ant to start and stop the server. Its
>>> very, very crude and we need to fix stuff like logging output etc.
>>>
>>> I also played around a little with Selenium to make some tests
>>> for the console. It wants a special server process started, so I
>>> created a selenium plugin which currently only starts that server
>>> in... well an icky way, but should work for the moment. Need to
>>> ask the m2 folks how to do this better.
>>>
>>> I created a console-testsuite module, which sets up the G server
>>> and Selenium server in pre-integration-test, and then uses the
>>> maven-invoker-plugin to invoke the basic module to actually run
>>> the tests. Only one test class right now, SimpleLoginTest, which
>>> does just that... logs in, logs in and then out and then another
>>> that logs in and clicks some links.
>>>
>>> I've got it all running from one mvn command and so far, on my
>>> mac, it works fine. I'd imagine due to the location of the
>>> assembly that Windows will not be able to run this... we'll need
>>> to fix this later. But those of you lucky enough to have a Linux
>>> box or a mac, you should be able to:
>>>
>>> mvn clean && mvn install
>>> cd testsuite
>>> mvn
>>>
>>> (the testsuite is not hooked up to the default build)
>>>
>>> And you should see Firefox popup a few times as Selenium goes
>>> about clicking on stuff. Right now its only really doing
>>> validation of the console page's titles... but if everything
>>> works, then the invoker should not complain and the build should
>>> complete.
>>>
>>> A bunch more needs to be done... like creating a common test
>>> support class for selenium tests... and probably switching these
>>> tests to use testng, so that the selenium setup is done once, not
>>> for each test... as it does now... so expect Firefox to start and
>>> stop a few times. We need to have some helpers to pick out the
>>> console contents easily so they can be validated... I read that
>>> selenium supports xpath or something, but I have not actually
>>> tried any of that.
>>>
>>> This is just a very, very crude proof of concept... needs to have
>>> reporting features added, capture logs, blah, blah, blah.
>>>
>>> Anyways, I committed this so we have a common base to work
>>> patches off of.
>>>
>>> One thing to note about the Selenium IDE, is that sometimes it
>>> does not put into a waitForPageToLoad(), like if you record a
>>> login to the console... you need to manually add that wait after
>>> the button is clicked or it will hose the test.
>>>
>>> I'm not married to any of this either... I just wanted something
>>> that worked... or kinda worked to show that its not going to be
>>> that hard to make some rich integration tests.
>>>
>>> So, comments, suggestions, patches... all welcome.
>>>
>>> --jason
>>>
>>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
FYI, I applied your patch... and surefire barfs as I thought it might:
<snip>
java.lang.reflect.UndeclaredThrowableException
at $Proxy0.addError(Unknown Source)
at junit.framework.TestResult.addError(TestResult.java:36)
at junit.framework.TestResult.runProtected(TestResult.java:133)
at junit.extensions.TestSetup.run(TestSetup.java:23)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.apache.maven.surefire.junit.JUnitTestSet.execute
(JUnitTestSet.java:210)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSe
t(AbstractDirectoryTestSuite.java:135)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
(AbstractDirectoryTestSuite.java:122)
at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at
org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
(SurefireBooter.java:262)
at org.apache.maven.surefire.booter.SurefireBooter.main
(SurefireBooter.java:787)
Caused by: java.lang.NoSuchMethodException:
org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName()
at java.lang.Class.getMethod(Class.java:986)
at
org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStackTr
aceWriter(TestListenerInvocationHandler.java:171)
at
org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleAddE
rror(TestListenerInvocationHandler.java:160)
at
org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
(TestListenerInvocationHandler.java:134)
... 18 more
</snip>
--jason
On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> Hi Jason,
>
> This is very cool indeed, thanks for putting the prototype
> together. I just submitted a patch that makes firefox pop up only
> once to run all the tests (it actually pops up twice, once during
> selenium.start() and then once for the test suite but the first
> time is for a split second).
>
> http://issues.apache.org/jira/browse/GERONIMO-2374
>
> It uses a JUnit TestDecorator to run setup only once for the whole
> suite.
>
> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> you will get some benefits for sure but the source has to be parsed
> at runtime to get at the meta-data (i.e. the info about the tests).
> Krufty IMO.
>
> The patch has a comment about invalid XHTML. I believe the invalid
> XHTML part of the console is preventing the XPath find from working.
>
> TTFN,
>
> -bd-
>
>
> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>
>> I've checked in a new top-level testsuite module and a few new
>> plugins to support it. This is only the start and I expect it to
>> change over the next few weeks (er maybe months) as momentum
>> starts to pick up.
>>
>> I looked into Cargo, and while I think we should eventually use it
>> to start/stop the server... asis now its broke. We need to
>> provide a very simple and consistent way for Cargo to invoke some
>> operations via JMX, which I hinted to in previous emails. Once
>> that is done, we can patch Cargo and hopefully get that committed
>> to support G... but in the mean time I whipped up a simple start &
>> stop mojo that uses Ant to start and stop the server. Its very,
>> very crude and we need to fix stuff like logging output etc.
>>
>> I also played around a little with Selenium to make some tests for
>> the console. It wants a special server process started, so I
>> created a selenium plugin which currently only starts that server
>> in... well an icky way, but should work for the moment. Need to
>> ask the m2 folks how to do this better.
>>
>> I created a console-testsuite module, which sets up the G server
>> and Selenium server in pre-integration-test, and then uses the
>> maven-invoker-plugin to invoke the basic module to actually run
>> the tests. Only one test class right now, SimpleLoginTest, which
>> does just that... logs in, logs in and then out and then another
>> that logs in and clicks some links.
>>
>> I've got it all running from one mvn command and so far, on my
>> mac, it works fine. I'd imagine due to the location of the
>> assembly that Windows will not be able to run this... we'll need
>> to fix this later. But those of you lucky enough to have a Linux
>> box or a mac, you should be able to:
>>
>> mvn clean && mvn install
>> cd testsuite
>> mvn
>>
>> (the testsuite is not hooked up to the default build)
>>
>> And you should see Firefox popup a few times as Selenium goes
>> about clicking on stuff. Right now its only really doing
>> validation of the console page's titles... but if everything
>> works, then the invoker should not complain and the build should
>> complete.
>>
>> A bunch more needs to be done... like creating a common test
>> support class for selenium tests... and probably switching these
>> tests to use testng, so that the selenium setup is done once, not
>> for each test... as it does now... so expect Firefox to start and
>> stop a few times. We need to have some helpers to pick out the
>> console contents easily so they can be validated... I read that
>> selenium supports xpath or something, but I have not actually
>> tried any of that.
>>
>> This is just a very, very crude proof of concept... needs to have
>> reporting features added, capture logs, blah, blah, blah.
>>
>> Anyways, I committed this so we have a common base to work patches
>> off of.
>>
>> One thing to note about the Selenium IDE, is that sometimes it
>> does not put into a waitForPageToLoad(), like if you record a
>> login to the console... you need to manually add that wait after
>> the button is clicked or it will hose the test.
>>
>> I'm not married to any of this either... I just wanted something
>> that worked... or kinda worked to show that its not going to be
>> that hard to make some rich integration tests.
>>
>> So, comments, suggestions, patches... all welcome.
>>
>> --jason
>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> This is very cool indeed, thanks for putting the prototype
> together. I just submitted a patch that makes firefox pop up only
> once to run all the tests (it actually pops up twice, once during
> selenium.start() and then once for the test suite but the first
> time is for a split second).
Cool.
> It uses a JUnit TestDecorator to run setup only once for the whole
> suite.
>
> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> you will get some benefits for sure but the source has to be parsed
> at runtime to get at the meta-data (i.e. the info about the tests).
> Krufty IMO.
Eh... I actually think using extensions to JUnit (especially since
surefire can freak out on them, like it is doing in OpenEJB2) is more
icky then having TestNG parse some javadoc comments to determine tests.
Plus the ability to inject configuration data into tests also seems
very appealing. That could be used to easily pass in the browser
string to use, then configure the set of tests to run against FF and IE.
I think it is worth a try, since this is all new tests... IMO its a
great time to do an evaluation of TestNG.
> The patch has a comment about invalid XHTML. I believe the invalid
> XHTML part of the console is preventing the XPath find from working.
Any idea how to fix this... seems like we need to get this working so
we can actually perform real content tests. Maybe need to hookup
jtidy as an output filter... or I think if we were using sitemesh to
render top look then it might reformat.
--jason
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
I think I'm going to make 2 test groups for now... one to test testng
and one to test the junit decorator bits... then we can see which
works better.
I've been playing with console-testsuite today using testng and it
works well... nice to have tags to specify what methods to run before
the suite and after the suite... I think we might have to do some
magic with JUnit to make that work.
I also hooked up selenium to be able to use some custom extensions...
specifically the ability to removeCookie(), but we can add more as
needed.
And, I changed the way that the server starting goals work, so that
you can run them on the command-line with maven to start them up,
which is much better to help develop. For example you can:
mvn selenium:start &
mvn geronimo:start &
...
mvn geronimo:stop
This will default to starting up Jetty + J2EE, but if you want to
test with Tomcat + J2EE, you can:
mvn geronimo:start -DassemblyId=tomcat
...
mvn geronimo:stop -DassemblyId=tomcat
jetty-minimal and tomcat-minimal can also be used with artifactId,
though until I get a better way to tell when the server is fully
started via JMX, background mode for these assemblies won't work.
They can still be background'd for testsuite usage, though needs to
have the background flag set to true.
The geronimo:start goal is also smart enough to detect when a new
assembly is available, so the next time you run it it will re-
install... else it will just reuse the previously extracted assembly.
Probably need to expose a command-line param to allow the basedir to
be changed... for windows folks... since the default is to unzip
under console-testsuite/target/, which I'd guess is probably going to
be a tad to long for windows.
Plugin docs have been updated if anyone is interested:
http://geronimo.apache.org/maven/server/maven-plugins/geronimo-
maven-plugin/index.html
http://geronimo.apache.org/maven/server/maven-plugins/selenium-
maven-plugin/index.html
Finally... geronimo:start and stop are usable from the main tree as
well, which makes it nice and easy to build and run the server:
mvn install
mvn geronimo:start -Dverbose=true
* * *
So I'm going to svn cp console-testsuite/basic to console-testsuite/
basic-junit and then apply your patch to that tree, and then commit
the changes to basic that I have done to use testng, plus some other
support/helper classes.
Oh, and I did get some xpath validation working... but the xpath
string is horrible... I used XPather (http://xpath.alephzarro.com/)
to select some text and give me the xpath. It did work but ends up
like:
<snip>
assertEquals("Server Info", selenium.getText(
"xpath=/html/body/table[@id='rootfragment']/tbody/tr[2]/td/
table/tbody/tr[2]/td[4]/table/tbody/tr[1]/td/table/tbody/tr[1]/td/div/
table/tbody/tr/td[2]/table/tbody/tr/td[1]/strong"));
</snip>
We may need to insert some id's into the console's pages to allow us
to root to the major panels.
--jason
On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> Hi Jason,
>
> This is very cool indeed, thanks for putting the prototype
> together. I just submitted a patch that makes firefox pop up only
> once to run all the tests (it actually pops up twice, once during
> selenium.start() and then once for the test suite but the first
> time is for a split second).
>
> http://issues.apache.org/jira/browse/GERONIMO-2374
>
> It uses a JUnit TestDecorator to run setup only once for the whole
> suite.
>
> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> you will get some benefits for sure but the source has to be parsed
> at runtime to get at the meta-data (i.e. the info about the tests).
> Krufty IMO.
>
> The patch has a comment about invalid XHTML. I believe the invalid
> XHTML part of the console is preventing the XPath find from working.
>
> TTFN,
>
> -bd-
>
>
> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>
>> I've checked in a new top-level testsuite module and a few new
>> plugins to support it. This is only the start and I expect it to
>> change over the next few weeks (er maybe months) as momentum
>> starts to pick up.
>>
>> I looked into Cargo, and while I think we should eventually use it
>> to start/stop the server... asis now its broke. We need to
>> provide a very simple and consistent way for Cargo to invoke some
>> operations via JMX, which I hinted to in previous emails. Once
>> that is done, we can patch Cargo and hopefully get that committed
>> to support G... but in the mean time I whipped up a simple start &
>> stop mojo that uses Ant to start and stop the server. Its very,
>> very crude and we need to fix stuff like logging output etc.
>>
>> I also played around a little with Selenium to make some tests for
>> the console. It wants a special server process started, so I
>> created a selenium plugin which currently only starts that server
>> in... well an icky way, but should work for the moment. Need to
>> ask the m2 folks how to do this better.
>>
>> I created a console-testsuite module, which sets up the G server
>> and Selenium server in pre-integration-test, and then uses the
>> maven-invoker-plugin to invoke the basic module to actually run
>> the tests. Only one test class right now, SimpleLoginTest, which
>> does just that... logs in, logs in and then out and then another
>> that logs in and clicks some links.
>>
>> I've got it all running from one mvn command and so far, on my
>> mac, it works fine. I'd imagine due to the location of the
>> assembly that Windows will not be able to run this... we'll need
>> to fix this later. But those of you lucky enough to have a Linux
>> box or a mac, you should be able to:
>>
>> mvn clean && mvn install
>> cd testsuite
>> mvn
>>
>> (the testsuite is not hooked up to the default build)
>>
>> And you should see Firefox popup a few times as Selenium goes
>> about clicking on stuff. Right now its only really doing
>> validation of the console page's titles... but if everything
>> works, then the invoker should not complain and the build should
>> complete.
>>
>> A bunch more needs to be done... like creating a common test
>> support class for selenium tests... and probably switching these
>> tests to use testng, so that the selenium setup is done once, not
>> for each test... as it does now... so expect Firefox to start and
>> stop a few times. We need to have some helpers to pick out the
>> console contents easily so they can be validated... I read that
>> selenium supports xpath or something, but I have not actually
>> tried any of that.
>>
>> This is just a very, very crude proof of concept... needs to have
>> reporting features added, capture logs, blah, blah, blah.
>>
>> Anyways, I committed this so we have a common base to work patches
>> off of.
>>
>> One thing to note about the Selenium IDE, is that sometimes it
>> does not put into a waitForPageToLoad(), like if you record a
>> login to the console... you need to manually add that wait after
>> the button is clicked or it will hose the test.
>>
>> I'm not married to any of this either... I just wanted something
>> that worked... or kinda worked to show that its not going to be
>> that hard to make some rich integration tests.
>>
>> So, comments, suggestions, patches... all welcome.
>>
>> --jason
>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
And... FYI, as I thought, the decorator setup is per-class... I added
a TestLogin2Test that is just a plain copy to see what it would do.
Anyways, looks like basic (testng) and basic-junit work, so I'm gonna
commit your patch and we can discus further which is the best option.
--jason
On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> Hi Jason,
>
> This is very cool indeed, thanks for putting the prototype
> together. I just submitted a patch that makes firefox pop up only
> once to run all the tests (it actually pops up twice, once during
> selenium.start() and then once for the test suite but the first
> time is for a split second).
>
> http://issues.apache.org/jira/browse/GERONIMO-2374
>
> It uses a JUnit TestDecorator to run setup only once for the whole
> suite.
>
> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> you will get some benefits for sure but the source has to be parsed
> at runtime to get at the meta-data (i.e. the info about the tests).
> Krufty IMO.
>
> The patch has a comment about invalid XHTML. I believe the invalid
> XHTML part of the console is preventing the XPath find from working.
>
> TTFN,
>
> -bd-
>
>
> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>
>> I've checked in a new top-level testsuite module and a few new
>> plugins to support it. This is only the start and I expect it to
>> change over the next few weeks (er maybe months) as momentum
>> starts to pick up.
>>
>> I looked into Cargo, and while I think we should eventually use it
>> to start/stop the server... asis now its broke. We need to
>> provide a very simple and consistent way for Cargo to invoke some
>> operations via JMX, which I hinted to in previous emails. Once
>> that is done, we can patch Cargo and hopefully get that committed
>> to support G... but in the mean time I whipped up a simple start &
>> stop mojo that uses Ant to start and stop the server. Its very,
>> very crude and we need to fix stuff like logging output etc.
>>
>> I also played around a little with Selenium to make some tests for
>> the console. It wants a special server process started, so I
>> created a selenium plugin which currently only starts that server
>> in... well an icky way, but should work for the moment. Need to
>> ask the m2 folks how to do this better.
>>
>> I created a console-testsuite module, which sets up the G server
>> and Selenium server in pre-integration-test, and then uses the
>> maven-invoker-plugin to invoke the basic module to actually run
>> the tests. Only one test class right now, SimpleLoginTest, which
>> does just that... logs in, logs in and then out and then another
>> that logs in and clicks some links.
>>
>> I've got it all running from one mvn command and so far, on my
>> mac, it works fine. I'd imagine due to the location of the
>> assembly that Windows will not be able to run this... we'll need
>> to fix this later. But those of you lucky enough to have a Linux
>> box or a mac, you should be able to:
>>
>> mvn clean && mvn install
>> cd testsuite
>> mvn
>>
>> (the testsuite is not hooked up to the default build)
>>
>> And you should see Firefox popup a few times as Selenium goes
>> about clicking on stuff. Right now its only really doing
>> validation of the console page's titles... but if everything
>> works, then the invoker should not complain and the build should
>> complete.
>>
>> A bunch more needs to be done... like creating a common test
>> support class for selenium tests... and probably switching these
>> tests to use testng, so that the selenium setup is done once, not
>> for each test... as it does now... so expect Firefox to start and
>> stop a few times. We need to have some helpers to pick out the
>> console contents easily so they can be validated... I read that
>> selenium supports xpath or something, but I have not actually
>> tried any of that.
>>
>> This is just a very, very crude proof of concept... needs to have
>> reporting features added, capture logs, blah, blah, blah.
>>
>> Anyways, I committed this so we have a common base to work patches
>> off of.
>>
>> One thing to note about the Selenium IDE, is that sometimes it
>> does not put into a waitForPageToLoad(), like if you record a
>> login to the console... you need to manually add that wait after
>> the button is clicked or it will hose the test.
>>
>> I'm not married to any of this either... I just wanted something
>> that worked... or kinda worked to show that its not going to be
>> that hard to make some rich integration tests.
>>
>> So, comments, suggestions, patches... all welcome.
>>
>> --jason
>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Jason Dillon <ja...@planet57.com>.
Looks like maven-surefire-plugin 2.2 is not happy with TestNG :-
( There is some docs ( http://testng.org/doc/maven.html ) that
suggest using 2.8-SNAPSHOT from this repo http://howardlewisship.com/
repository, which does work fine...
I posted a question to the maven dev list to find out what the status
is...
--jason
On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
> Hi Jason,
>
> This is very cool indeed, thanks for putting the prototype
> together. I just submitted a patch that makes firefox pop up only
> once to run all the tests (it actually pops up twice, once during
> selenium.start() and then once for the test suite but the first
> time is for a split second).
>
> http://issues.apache.org/jira/browse/GERONIMO-2374
>
> It uses a JUnit TestDecorator to run setup only once for the whole
> suite.
>
> IMO until we move to JDK 5 its not a great idea to move to TestNG,
> you will get some benefits for sure but the source has to be parsed
> at runtime to get at the meta-data (i.e. the info about the tests).
> Krufty IMO.
>
> The patch has a comment about invalid XHTML. I believe the invalid
> XHTML part of the console is preventing the XPath find from working.
>
> TTFN,
>
> -bd-
>
>
> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>
>> I've checked in a new top-level testsuite module and a few new
>> plugins to support it. This is only the start and I expect it to
>> change over the next few weeks (er maybe months) as momentum
>> starts to pick up.
>>
>> I looked into Cargo, and while I think we should eventually use it
>> to start/stop the server... asis now its broke. We need to
>> provide a very simple and consistent way for Cargo to invoke some
>> operations via JMX, which I hinted to in previous emails. Once
>> that is done, we can patch Cargo and hopefully get that committed
>> to support G... but in the mean time I whipped up a simple start &
>> stop mojo that uses Ant to start and stop the server. Its very,
>> very crude and we need to fix stuff like logging output etc.
>>
>> I also played around a little with Selenium to make some tests for
>> the console. It wants a special server process started, so I
>> created a selenium plugin which currently only starts that server
>> in... well an icky way, but should work for the moment. Need to
>> ask the m2 folks how to do this better.
>>
>> I created a console-testsuite module, which sets up the G server
>> and Selenium server in pre-integration-test, and then uses the
>> maven-invoker-plugin to invoke the basic module to actually run
>> the tests. Only one test class right now, SimpleLoginTest, which
>> does just that... logs in, logs in and then out and then another
>> that logs in and clicks some links.
>>
>> I've got it all running from one mvn command and so far, on my
>> mac, it works fine. I'd imagine due to the location of the
>> assembly that Windows will not be able to run this... we'll need
>> to fix this later. But those of you lucky enough to have a Linux
>> box or a mac, you should be able to:
>>
>> mvn clean && mvn install
>> cd testsuite
>> mvn
>>
>> (the testsuite is not hooked up to the default build)
>>
>> And you should see Firefox popup a few times as Selenium goes
>> about clicking on stuff. Right now its only really doing
>> validation of the console page's titles... but if everything
>> works, then the invoker should not complain and the build should
>> complete.
>>
>> A bunch more needs to be done... like creating a common test
>> support class for selenium tests... and probably switching these
>> tests to use testng, so that the selenium setup is done once, not
>> for each test... as it does now... so expect Firefox to start and
>> stop a few times. We need to have some helpers to pick out the
>> console contents easily so they can be validated... I read that
>> selenium supports xpath or something, but I have not actually
>> tried any of that.
>>
>> This is just a very, very crude proof of concept... needs to have
>> reporting features added, capture logs, blah, blah, blah.
>>
>> Anyways, I committed this so we have a common base to work patches
>> off of.
>>
>> One thing to note about the Selenium IDE, is that sometimes it
>> does not put into a waitForPageToLoad(), like if you record a
>> login to the console... you need to manually add that wait after
>> the button is clicked or it will hose the test.
>>
>> I'm not married to any of this either... I just wanted something
>> that worked... or kinda worked to show that its not going to be
>> that hard to make some rich integration tests.
>>
>> So, comments, suggestions, patches... all welcome.
>>
>> --jason
>>
>>
>
Re: Testsuite module, with basic/crude Selenium support
Posted by Bill Dudney <bd...@apache.org>.
Hi Jason,
This is very cool indeed, thanks for putting the prototype together.
I just submitted a patch that makes firefox pop up only once to run
all the tests (it actually pops up twice, once during selenium.start
() and then once for the test suite but the first time is for a split
second).
http://issues.apache.org/jira/browse/GERONIMO-2374
It uses a JUnit TestDecorator to run setup only once for the whole
suite.
IMO until we move to JDK 5 its not a great idea to move to TestNG,
you will get some benefits for sure but the source has to be parsed
at runtime to get at the meta-data (i.e. the info about the tests).
Krufty IMO.
The patch has a comment about invalid XHTML. I believe the invalid
XHTML part of the console is preventing the XPath find from working.
TTFN,
-bd-
On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
> I've checked in a new top-level testsuite module and a few new
> plugins to support it. This is only the start and I expect it to
> change over the next few weeks (er maybe months) as momentum starts
> to pick up.
>
> I looked into Cargo, and while I think we should eventually use it
> to start/stop the server... asis now its broke. We need to provide
> a very simple and consistent way for Cargo to invoke some
> operations via JMX, which I hinted to in previous emails. Once
> that is done, we can patch Cargo and hopefully get that committed
> to support G... but in the mean time I whipped up a simple start &
> stop mojo that uses Ant to start and stop the server. Its very,
> very crude and we need to fix stuff like logging output etc.
>
> I also played around a little with Selenium to make some tests for
> the console. It wants a special server process started, so I
> created a selenium plugin which currently only starts that server
> in... well an icky way, but should work for the moment. Need to
> ask the m2 folks how to do this better.
>
> I created a console-testsuite module, which sets up the G server
> and Selenium server in pre-integration-test, and then uses the
> maven-invoker-plugin to invoke the basic module to actually run the
> tests. Only one test class right now, SimpleLoginTest, which does
> just that... logs in, logs in and then out and then another that
> logs in and clicks some links.
>
> I've got it all running from one mvn command and so far, on my mac,
> it works fine. I'd imagine due to the location of the assembly
> that Windows will not be able to run this... we'll need to fix this
> later. But those of you lucky enough to have a Linux box or a mac,
> you should be able to:
>
> mvn clean && mvn install
> cd testsuite
> mvn
>
> (the testsuite is not hooked up to the default build)
>
> And you should see Firefox popup a few times as Selenium goes about
> clicking on stuff. Right now its only really doing validation of
> the console page's titles... but if everything works, then the
> invoker should not complain and the build should complete.
>
> A bunch more needs to be done... like creating a common test
> support class for selenium tests... and probably switching these
> tests to use testng, so that the selenium setup is done once, not
> for each test... as it does now... so expect Firefox to start and
> stop a few times. We need to have some helpers to pick out the
> console contents easily so they can be validated... I read that
> selenium supports xpath or something, but I have not actually tried
> any of that.
>
> This is just a very, very crude proof of concept... needs to have
> reporting features added, capture logs, blah, blah, blah.
>
> Anyways, I committed this so we have a common base to work patches
> off of.
>
> One thing to note about the Selenium IDE, is that sometimes it does
> not put into a waitForPageToLoad(), like if you record a login to
> the console... you need to manually add that wait after the button
> is clicked or it will hose the test.
>
> I'm not married to any of this either... I just wanted something
> that worked... or kinda worked to show that its not going to be
> that hard to make some rich integration tests.
>
> So, comments, suggestions, patches... all welcome.
>
> --jason
>
>