Thinking about running iTest aloud - need help figuring it all out

[Jacek Laskowski <ja...@laskowski.net.pl>]

Hi,

I'm thinking about iTest aloud and would appreciate your comments on it.

>From Maven2's standpoint iTest *is* a test so I can run mvn test
-Dtest=iTest. It however runs a bunch of openejb tests that are *not*
tests for maven2 and hence it's not possible to run them alone with
mvn -Dtest=... even though some of them might be if they were in
src/test/java (that would however break other things). They live in
their own modules in src/main/java as if they weren't tests. It makes
impossible to run them separately. One can run all itests or nothing
and what's most confusing to me is that it belongs to the openejb-core
module. The project structure makes the tests very hard to change and
run afterwards. The only way to run an itest after it's been changed
is as follows (provided o3 is an alias for changing a working
directory to where openejb is).

o3; cd itests/openejb-itests-interceptor-beans/; mvn clean install; \
o3; cd itests/openejb-itests-client/; mvn clean install; \
o3; cd itests/openejb-itests-app/; mvn clean install; \
o3; cd container/openejb-core/; mvn clean test -Dtest=iTest

It's overly complicated and time-consuming.

Am I right so far? If so, we need to fix it before 3.0 and I'd be
happy to work on it, but to be honest I don't have any idea how to fix
it. I think we need to move iTest to its own module. Let people use
properties to select what subset of tests should be executed. That's
for the start. I wish I could run mvn -Dtest=StatelessInterceptorTests
test without any trouble. It requires a move to src/test/java (or
pom.xml changes for appropriate plugins). What else?

Jacek

-- 
Jacek Laskowski
http://www.JacekLaskowski.pl

Re: Thinking about running iTest aloud - need help figuring it all out

[David Blevins <da...@visi.com>]

On Dec 22, 2007, at 9:07 AM, David Blevins wrote:

>
> On Dec 22, 2007, at 3:45 AM, Jacek Laskowski wrote:
>
>> On Dec 22, 2007 1:01 AM, David Blevins <da...@visi.com>  
>> wrote:
>>
>>> You could just:
>>>
>>> o3; cd itests/; mvn clean install;
>>> o3; cd container/openejb-core/; mvn clean test -Dtest=iTest
>>
>> Right. I kept adding more and more itests submodules and when I ended
>> up with all of them I forgot to run 'mvn clean install' from itests
>> dir. Thanks.
>>
>>>  - container/openejb-core/src/test/java/.../iTest
>>>  - container/openejb-core/src/test/java/.../ApplicationTest
>>>  - container/openejb-core/src/test/java/.../RedeployTest
>>>  - server/openejb-ejbd/src/test/java/.../RemoteiTest
>>>  - server/openejb-http/src/test/java/.../HttpEjbServerTest
>>>
>>> And though not automated, we need to run the itest ejb jar and  
>>> client
>>> jar here too before release:
>>>  - assembly/openejb-standalone/
>>>  - assembly/openejb-tomcat/
>>
>> Thanks for the list. I'm going to poke at it and having it listed
>> together helps much.
>
> Cool.  If any of your poking leads you to finding a way to automate  
> the running of the itests automatically in assembly, that'd be more  
> than excellent ;)
>
>>> We could probably rig something like that up.  I hacked up something
>>> that uses a -Ditest to filter, like so:
>>>
>>>  mvn clean install -Dtest=ApplicationTest - 
>>> Ditest=StatefulRmiIiopTests
>>
>> You beat me to it. I was about to have committed the changes of mine
>> and when I did 'svn up' I saw lots of changes in the files *I* had
>> changed. I didn't review your changes yet, but am wondering how
>> different your and my solutions are.
>
> I basically made the run() method of our TestSuite subclass filter  
> the tests.  I originally had a static filter method but it didn't  
> look like it was going to work out so well that way so sort of  
> switched mid stream.
>
>> The easier it is to write a itest for openejb the better. I remember
>> I've been struggling with it for a while and wouldn't be surprised to
>> hear alike from our end users.
>
> I would expect users to just right plain unit tests like we have in  
> our examples directory.  Our itests are definitely not the example  
> to follow.

s/right/write/

Re: Thinking about running iTest aloud - need help figuring it all out

[David Blevins <da...@visi.com>]

On Dec 22, 2007, at 3:45 AM, Jacek Laskowski wrote:

> On Dec 22, 2007 1:01 AM, David Blevins <da...@visi.com> wrote:
>
>> You could just:
>>
>> o3; cd itests/; mvn clean install;
>> o3; cd container/openejb-core/; mvn clean test -Dtest=iTest
>
> Right. I kept adding more and more itests submodules and when I ended
> up with all of them I forgot to run 'mvn clean install' from itests
> dir. Thanks.
>
>>   - container/openejb-core/src/test/java/.../iTest
>>   - container/openejb-core/src/test/java/.../ApplicationTest
>>   - container/openejb-core/src/test/java/.../RedeployTest
>>   - server/openejb-ejbd/src/test/java/.../RemoteiTest
>>   - server/openejb-http/src/test/java/.../HttpEjbServerTest
>>
>> And though not automated, we need to run the itest ejb jar and client
>> jar here too before release:
>>   - assembly/openejb-standalone/
>>   - assembly/openejb-tomcat/
>
> Thanks for the list. I'm going to poke at it and having it listed
> together helps much.

Cool.  If any of your poking leads you to finding a way to automate  
the running of the itests automatically in assembly, that'd be more  
than excellent ;)

>> We could probably rig something like that up.  I hacked up something
>> that uses a -Ditest to filter, like so:
>>
>>   mvn clean install -Dtest=ApplicationTest - 
>> Ditest=StatefulRmiIiopTests
>
> You beat me to it. I was about to have committed the changes of mine
> and when I did 'svn up' I saw lots of changes in the files *I* had
> changed. I didn't review your changes yet, but am wondering how
> different your and my solutions are.

I basically made the run() method of our TestSuite subclass filter the  
tests.  I originally had a static filter method but it didn't look  
like it was going to work out so well that way so sort of switched mid  
stream.

> The easier it is to write a itest for openejb the better. I remember
> I've been struggling with it for a while and wouldn't be surprised to
> hear alike from our end users.

I would expect users to just right plain unit tests like we have in  
our examples directory.  Our itests are definitely not the example to  
follow.

-David

Re: Thinking about running iTest aloud - need help figuring it all out

[Jacek Laskowski <ja...@laskowski.net.pl>]

On Dec 22, 2007 1:01 AM, David Blevins <da...@visi.com> wrote:

> You could just:
>
> o3; cd itests/; mvn clean install;
> o3; cd container/openejb-core/; mvn clean test -Dtest=iTest

Right. I kept adding more and more itests submodules and when I ended
up with all of them I forgot to run 'mvn clean install' from itests
dir. Thanks.

>    - container/openejb-core/src/test/java/.../iTest
>    - container/openejb-core/src/test/java/.../ApplicationTest
>    - container/openejb-core/src/test/java/.../RedeployTest
>    - server/openejb-ejbd/src/test/java/.../RemoteiTest
>    - server/openejb-http/src/test/java/.../HttpEjbServerTest
>
> And though not automated, we need to run the itest ejb jar and client
> jar here too before release:
>    - assembly/openejb-standalone/
>    - assembly/openejb-tomcat/

Thanks for the list. I'm going to poke at it and having it listed
together helps much.

> We could probably rig something like that up.  I hacked up something
> that uses a -Ditest to filter, like so:
>
>    mvn clean install -Dtest=ApplicationTest -Ditest=StatefulRmiIiopTests

You beat me to it. I was about to have committed the changes of mine
and when I did 'svn up' I saw lots of changes in the files *I* had
changed. I didn't review your changes yet, but am wondering how
different your and my solutions are.

The easier it is to write a itest for openejb the better. I remember
I've been struggling with it for a while and wouldn't be surprised to
hear alike from our end users.

Jacek

-- 
Jacek Laskowski
http://www.JacekLaskowski.pl

Re: Thinking about running iTest aloud - need help figuring it all out

[David Blevins <da...@visi.com>]

On Dec 21, 2007, at 2:00 AM, Jacek Laskowski wrote:

> Hi,
>
> I'm thinking about iTest aloud and would appreciate your comments on  
> it.
>
>> From Maven2's standpoint iTest *is* a test so I can run mvn test
> -Dtest=iTest. It however runs a bunch of openejb tests that are *not*
> tests for maven2 and hence it's not possible to run them alone with
> mvn -Dtest=... even though some of them might be if they were in
> src/test/java (that would however break other things). They live in
> their own modules in src/main/java as if they weren't tests. It makes
> impossible to run them separately. One can run all itests or nothing
> and what's most confusing to me is that it belongs to the openejb-core
> module. The project structure makes the tests very hard to change and
> run afterwards. The only way to run an itest after it's been changed
> is as follows (provided o3 is an alias for changing a working
> directory to where openejb is).
>
> o3; cd itests/openejb-itests-interceptor-beans/; mvn clean install; \
> o3; cd itests/openejb-itests-client/; mvn clean install; \
> o3; cd itests/openejb-itests-app/; mvn clean install; \
> o3; cd container/openejb-core/; mvn clean test -Dtest=iTest
>
> It's overly complicated and time-consuming.

You could just:

o3; cd itests/; mvn clean install;
o3; cd container/openejb-core/; mvn clean test -Dtest=iTest

Or in bash:

o3$ (cd itests/ && mvn clean install) && (cd container/openejb-core/  
&& mvn clean test -Dtest=iTest)

> Am I right so far?

Well if itests existed just to test the openejb-core jar, you'd  
definitely have something.  They're really just a plain ejb  
application (with standalone client jar (that can optionally be  
combined into a EAR)) with no bindings to OpenEJB specifically that  
can test anything that claims it can run ejb apps.  They're like our  
own tck.  We used to use them far more aggressively on the final  
binaries ([1] see Testing matrix).

We run the itests in about 5 different places in the build:

   - container/openejb-core/src/test/java/.../iTest
   - container/openejb-core/src/test/java/.../ApplicationTest
   - container/openejb-core/src/test/java/.../RedeployTest
   - server/openejb-ejbd/src/test/java/.../RemoteiTest
   - server/openejb-http/src/test/java/.../HttpEjbServerTest

And though not automated, we need to run the itest ejb jar and client  
jar here too before release:
   - assembly/openejb-standalone/
   - assembly/openejb-tomcat/

They are also included in the geronimo set of itegration tests.

> Let people use
> properties to select what subset of tests should be executed. That's
> for the start. I wish I could run mvn -Dtest=StatelessInterceptorTests
> test without any trouble.

We could probably rig something like that up.  I hacked up something  
that uses a -Ditest to filter, like so:

   mvn clean install -Dtest=ApplicationTest -Ditest=StatefulRmiIiopTests

That's at least a little bit better.  Admittedly, I've been commenting  
and uncommenting the 'addTest' methods for years....  Your idea is way  
better :)

-David

[1] http://cwiki.apache.org/OPENEJB/itests-overview.html