You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltaspike.apache.org by Mark Struberg <st...@yahoo.de> on 2011/12/16 09:38:02 UTC
basic decisions: test setup
Hi!
Another round of discussion - this time about how to setup our unit testing and integration testing.
I think it's pretty much clear that we will gonna use Arquillian for all our tests. That was also fundamental part of the incubator proposal.
And instead of inventing yet another abstraction layer, I'd favour to just use Arquillian and contribute all things we need to this project.
So here a formal vote
1.)
for using Arquillian as test integration framework.
+1 from me
2.)
What do we like to use from the unit tests itself? TestNG or JUnit?
JUnit has better Arquillian integration and TestNG is better for 'real world' projects, because it allows to define test dependencies.
So which one to take?
But we should definitely only use 1 of the two exclusively!
3.)
'Integrated' tests vs 'Integration Tests'
We have 2 options to test our projects
3.a.) create a full unit test in each module (e.g. deltaspike/core/impl) and add a profile for each and every server (maybe we can trim this down with pom imports?).
Then run the build with one after each other:
$> mvn clean test -Powb
$> mvn clean test -Pweld
$> mvn clean test -Powb-tc
$> mvn clean test -Pweld-tc
$> mvn clean test -Pgf31
$> mvn clean test -Pas7
$> mvn clean test -Ptomee
This might pretty much blow up our poms...
3.b) create a full unit test in each module (e.g. deltaspike/core/impl) and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
Then add a deltaspike/test/base/core with the very basic parent stuff and 1 module for each integrated container, e.g.
deltaspike/test/weld-tc
deltaspike/test/owb-tc
deltaspike/test/weld-jetty
deltaspike/test/owb-jetty
deltaspike/test/tomee
deltaspike/test/gf31
deltaspike/test/geronimo
deltaspike/test/websphere
deltaspike/test/as7
deltaspike/test/as6
etc.
The initial setup costs are higher, but it would be pretty easy to add new containers that way.
Which route shall we take?
LieGrue,
strub
Re: basic decisions: test setup
Posted by Gerhard Petracek <ge...@gmail.com>.
#1) +1
#2) +1 for using only one of them +0 for both
#3) +1 for 3b) because we can define multiple default test-sets (for
different containers)
regards,
gerhard
2011/12/16 Mark Struberg <st...@yahoo.de>
> Hi!
>
> Another round of discussion - this time about how to setup our unit
> testing and integration testing.
>
> I think it's pretty much clear that we will gonna use Arquillian for all
> our tests. That was also fundamental part of the incubator proposal.
> And instead of inventing yet another abstraction layer, I'd favour to just
> use Arquillian and contribute all things we need to this project.
> So here a formal vote
>
> 1.)
>
> for using Arquillian as test integration framework.
> +1 from me
>
>
>
> 2.)
>
> What do we like to use from the unit tests itself? TestNG or JUnit?
> JUnit has better Arquillian integration and TestNG is better for 'real
> world' projects, because it allows to define test dependencies.
> So which one to take?
> But we should definitely only use 1 of the two exclusively!
>
> 3.)
> 'Integrated' tests vs 'Integration Tests'
> We have 2 options to test our projects
>
> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
> and add a profile for each and every server (maybe we can trim this down
> with pom imports?).
> Then run the build with one after each other:
>
> $> mvn clean test -Powb
> $> mvn clean test -Pweld
> $> mvn clean test -Powb-tc
> $> mvn clean test -Pweld-tc
> $> mvn clean test -Pgf31
> $> mvn clean test -Pas7
> $> mvn clean test -Ptomee
>
> This might pretty much blow up our poms...
>
>
> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
> Then add a deltaspike/test/base/core with the very basic parent stuff and
> 1 module for each integrated container, e.g.
> deltaspike/test/weld-tc
> deltaspike/test/owb-tc
> deltaspike/test/weld-jetty
> deltaspike/test/owb-jetty
> deltaspike/test/tomee
> deltaspike/test/gf31
> deltaspike/test/geronimo
> deltaspike/test/websphere
> deltaspike/test/as7
> deltaspike/test/as6
> etc.
>
> The initial setup costs are higher, but it would be pretty easy to add new
> containers that way.
>
> Which route shall we take?
>
>
> LieGrue,
> strub
>
>
Re: basic decisions: test setup
Posted by Marius Bogoevici <ma...@gmail.com>.
On 2011-12-17, at 4:18 AM, Jason Porter wrote:
> I've wanted that as well. Probably something we could fairly easily add or have as an add-on in one our extensions. Mark, you mentioned you have some arquillian extensions written up, anything that could help there?
>
> Sent from my iPhone
>
> On Dec 16, 2011, at 23:55, Christian Kaltepoth <ch...@kaltepoth.de> wrote:
>
>> I would like to add some "non-binding" comments here regarding 3).
>>
>> Actually I really like the concept Jason described and which is
>> currently used in Seam3. Having worked with it a bit I can confirm
>> that it runs fine. And somehow it's the best of both worlds (in regard
>> to 3a vs 3b).
>>
>> That the full integration test suite with all containers cannot run in
>> a single Maven invocation is IMHO not a real problem. We could write a
>> simple script for this.
>>
>> The only thing that could be optimized in this setup is the separation
>> of container dependent tests via package name. But perhaps this could
>> be solved by Arquillian? What I would really like to see is the
>> ability to configure a test so it is only executed in a specific
>> container (something similar to @TargetsContainer). This way tests
>> that require to be run in a full container could be executed only in
>> JBoss and Glassfish but would be skipped in embedded Weld.
>>
>> Christian
>>
>>
>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>> Comments inline.
>>>
>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>>>
>>>> Hi!
>>>>
>>>> Another round of discussion - this time about how to setup our unit
>>>> testing and integration testing.
>>>>
>>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>>> our tests. That was also fundamental part of the incubator proposal.
>>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>>> use Arquillian and contribute all things we need to this project.
>>>> So here a formal vote
>>>>
>>>> 1.)
>>>>
>>>> for using Arquillian as test integration framework.
>>>> +1 from me
>>>>
>>>
>>> +1 from me
>>>
>>>
>>>>
>>>> 2.)
>>>>
>>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>>> world' projects, because it allows to define test dependencies.
>>>> So which one to take?
>>>> But we should definitely only use 1 of the two exclusively!
>>>>
>>>
>>> +1 for using one exclusively
>>>
>>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>>> certainly the easier of the two to use with Arquillian.
>>>
>>> Also, are we open to having other languages for tests or using other
>>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>>> scala, etc.?
>>>
>>>
>>>> 3.)
>>>> 'Integrated' tests vs 'Integration Tests'
>>>> We have 2 options to test our projects
>>>>
>>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>>> and add a profile for each and every server (maybe we can trim this down
>>>> with pom imports?).
>>>> Then run the build with one after each other:
>>>>
>>>> $> mvn clean test -Powb
>>>> $> mvn clean test -Pweld
>>>> $> mvn clean test -Powb-tc
>>>> $> mvn clean test -Pweld-tc
>>>> $> mvn clean test -Pgf31
>>>> $> mvn clean test -Pas7
>>>> $> mvn clean test -Ptomee
>>>>
>>>> This might pretty much blow up our poms...
>>>>
>>>>
>>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>>> 1 module for each integrated container, e.g.
>>>> deltaspike/test/weld-tc
>>>> deltaspike/test/owb-tc
>>>> deltaspike/test/weld-jetty
>>>> deltaspike/test/owb-jetty
>>>> deltaspike/test/tomee
>>>> deltaspike/test/gf31
>>>> deltaspike/test/geronimo
>>>> deltaspike/test/websphere
>>>> deltaspike/test/as7
>>>> deltaspike/test/as6
>>>> etc.
>>>>
>>>> The initial setup costs are higher, but it would be pretty easy to add new
>>>> containers that way.
>>>>
>>>> Which route shall we take?
>>>>
>>>
>>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>>> We have regular unit tests (with mockito, stubs, etc) under
>>> impl/src/test/... and then we have a separate module called testsuite which
>>> contained all the integration tests. We used pom imports to get all the
>>> containers setup correctly and then had different profiles which pulled in
>>> the poms and also configured surefire for that server. Inside the code
>>> structure we separated the servers by package: common, jbossas, glassfish,
>>> etc. That way all of the tests were in one location, they were easy to
>>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>>> two downsides we had were 1) the extra config in the pom and 2) you can't
>>> run the full integration suite in one maven execution. We spent quite a bit
>>> of time examining options and trying some of them out. This was the best we
>>> came up with, and also the one Aslak thought would work best as well.
There is another benefit of a separate test suite: since the actual artifacts are
dependencies, you are able to test that the entire module artifact is sound. This
is especially important in the case of CDI extensions, where getting the packaging
structure right (i.e. code, META-INF services) is a requirement.
I used that feature extensively in Seam-Spring testing and it worked very well.
>>>
>>>
>>>> LieGrue,
>>>> strub
>>>>
>>>>
>>>
>>>
>>> --
>>> Jason Porter
>>> http://lightguard-jp.blogspot.com
>>> http://twitter.com/lightguardjp
>>>
>>> Software Engineer
>>> Open Source Advocate
>>> Author of Seam Catch - Next Generation Java Exception Handling
>>>
>>> PGP key id: 926CCFF5
>>> PGP key available at: keyserver.net, pgp.mit.edu
>>
>>
>>
>> --
>> Christian Kaltepoth
>> Blog: http://chkal.blogspot.com/
>> Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Jason Porter <li...@gmail.com>.
I've wanted that as well. Probably something we could fairly easily add or have as an add-on in one our extensions. Mark, you mentioned you have some arquillian extensions written up, anything that could help there?
Sent from my iPhone
On Dec 16, 2011, at 23:55, Christian Kaltepoth <ch...@kaltepoth.de> wrote:
> I would like to add some "non-binding" comments here regarding 3).
>
> Actually I really like the concept Jason described and which is
> currently used in Seam3. Having worked with it a bit I can confirm
> that it runs fine. And somehow it's the best of both worlds (in regard
> to 3a vs 3b).
>
> That the full integration test suite with all containers cannot run in
> a single Maven invocation is IMHO not a real problem. We could write a
> simple script for this.
>
> The only thing that could be optimized in this setup is the separation
> of container dependent tests via package name. But perhaps this could
> be solved by Arquillian? What I would really like to see is the
> ability to configure a test so it is only executed in a specific
> container (something similar to @TargetsContainer). This way tests
> that require to be run in a full container could be executed only in
> JBoss and Glassfish but would be skipped in embedded Weld.
>
> Christian
>
>
> 2011/12/16 Jason Porter <li...@gmail.com>:
>> Comments inline.
>>
>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>>
>>> Hi!
>>>
>>> Another round of discussion - this time about how to setup our unit
>>> testing and integration testing.
>>>
>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>> our tests. That was also fundamental part of the incubator proposal.
>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>> use Arquillian and contribute all things we need to this project.
>>> So here a formal vote
>>>
>>> 1.)
>>>
>>> for using Arquillian as test integration framework.
>>> +1 from me
>>>
>>
>> +1 from me
>>
>>
>>>
>>> 2.)
>>>
>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>> world' projects, because it allows to define test dependencies.
>>> So which one to take?
>>> But we should definitely only use 1 of the two exclusively!
>>>
>>
>> +1 for using one exclusively
>>
>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>> certainly the easier of the two to use with Arquillian.
>>
>> Also, are we open to having other languages for tests or using other
>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>> scala, etc.?
>>
>>
>>> 3.)
>>> 'Integrated' tests vs 'Integration Tests'
>>> We have 2 options to test our projects
>>>
>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add a profile for each and every server (maybe we can trim this down
>>> with pom imports?).
>>> Then run the build with one after each other:
>>>
>>> $> mvn clean test -Powb
>>> $> mvn clean test -Pweld
>>> $> mvn clean test -Powb-tc
>>> $> mvn clean test -Pweld-tc
>>> $> mvn clean test -Pgf31
>>> $> mvn clean test -Pas7
>>> $> mvn clean test -Ptomee
>>>
>>> This might pretty much blow up our poms...
>>>
>>>
>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>> 1 module for each integrated container, e.g.
>>> deltaspike/test/weld-tc
>>> deltaspike/test/owb-tc
>>> deltaspike/test/weld-jetty
>>> deltaspike/test/owb-jetty
>>> deltaspike/test/tomee
>>> deltaspike/test/gf31
>>> deltaspike/test/geronimo
>>> deltaspike/test/websphere
>>> deltaspike/test/as7
>>> deltaspike/test/as6
>>> etc.
>>>
>>> The initial setup costs are higher, but it would be pretty easy to add new
>>> containers that way.
>>>
>>> Which route shall we take?
>>>
>>
>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>> We have regular unit tests (with mockito, stubs, etc) under
>> impl/src/test/... and then we have a separate module called testsuite which
>> contained all the integration tests. We used pom imports to get all the
>> containers setup correctly and then had different profiles which pulled in
>> the poms and also configured surefire for that server. Inside the code
>> structure we separated the servers by package: common, jbossas, glassfish,
>> etc. That way all of the tests were in one location, they were easy to
>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>> two downsides we had were 1) the extra config in the pom and 2) you can't
>> run the full integration suite in one maven execution. We spent quite a bit
>> of time examining options and trying some of them out. This was the best we
>> came up with, and also the one Aslak thought would work best as well.
>>
>>
>>> LieGrue,
>>> strub
>>>
>>>
>>
>>
>> --
>> Jason Porter
>> http://lightguard-jp.blogspot.com
>> http://twitter.com/lightguardjp
>>
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next Generation Java Exception Handling
>>
>> PGP key id: 926CCFF5
>> PGP key available at: keyserver.net, pgp.mit.edu
>
>
>
> --
> Christian Kaltepoth
> Blog: http://chkal.blogspot.com/
> Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Gerhard Petracek <ge...@gmail.com>.
hi mark,
that's very similar to the idea implemented by manila :)
regards,
gerhard
2011/12/17 Jason Porter <li...@gmail.com>
> I like the idea, however, that would be a very core change to both core
> arquillian and the containers. I don't think we'll get that anytime soon.
> Maybe a simple package convention by environment will work for now?
>
> Sent from my iPhone
>
> On Dec 17, 2011, at 9:19, Mark Struberg <st...@yahoo.de> wrote:
>
> > true, we would need something like
> >
> > @RunInEnvironments({Environment.EeLight, Environment.EeFull,
> Environment.Servlet})
> >
> > @RunInEnvironments(Environment.JavaSE)
> > @RunInEnvironments(Environments.ALL)
> >
> > LieGrue,
> > strub
> >
> >
> >
> > ----- Original Message -----
> >> From: Christian Kaltepoth <ch...@kaltepoth.de>
> >> To: deltaspike-dev@incubator.apache.org
> >> Cc:
> >> Sent: Saturday, December 17, 2011 5:04 PM
> >> Subject: Re: basic decisions: test setup
> >>
> >> @Mark:
> >>
> >> But what about tests that should run in EE-light, EE-full and Servlet
> >> but not in SE. Or tests that should run in EE-light/full but not in
> >> Servlet and SE. I think this could get very complex.
> >>
> >> Christian
> >>
> >>
> >> 2011/12/17 Mark Struberg <st...@yahoo.de>:
> >>> We might solve this simply with a package naming rule. And then we
> >> configure surefire to just run the tests with the respective package?
> >>>
> >>> Not the most advanced solution, but will work.
> >>>
> >>>
> >>> LieGrue,
> >>> strub
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
> >>>> To: deltaspike-dev@incubator.apache.org
> >>>> Cc:
> >>>> Sent: Saturday, December 17, 2011 4:43 PM
> >>>> Subject: Re: basic decisions: test setup
> >>>>
> >>>> @Mark:
> >>>>
> >>>> Actually @TargetsContainer is an existing feature of Arquillian. It
> >>>> allows to build container specific deployments.
> >>>>
> >>>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
> >>>>
> >>>> Unfortunately it won't fit our needs because the test completely
> >> fails
> >>>> if there is no matching deployment for the container the test is
> >>>> executed against. So I guess we have to build something new for this.
> >>>>
> >>>> I really like your suggestion regarding the four environments. It
> >>>> absolutely make sense to declare environment constraints (like SE,
> >>>> Servlet, EE, etc.) instead of listing concrete containers.
> >>>>
> >>>> Christian
> >>>>
> >>>>
> >>>>
> >>>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
> >>>>> Hi Christian!
> >>>>>
> >>>>>
> >>>>> @TargetContainer sounds good, but I'd rather make it a
> >>>>>
> >>>>>
> >>>>> @TargetEnvironment()
> >>>>>
> >>>>> with values: SE, Servlet, EE-light, EE-full
> >>>>>
> >>>>> The excludes should be done in the single test integration
> >> modules, and the
> >>>> goal should be to get down to zero excluded tests for all containers.
> >>>>>
> >>>>>
> >>>>> @Jason: that mixture for the tests looks fine to me as well. Can
> >> you please
> >>>> hack a sample structure in the DeltaSpikePlayground git repo?
> >>>>>
> >>>>>
> >>>>> LieGrue,
> >>>>> strub
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> ________________________________
> >>>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
> >>>>>> To: deltaspike-dev@incubator.apache.org
> >>>>>> Sent: Saturday, December 17, 2011 7:55 AM
> >>>>>> Subject: Re: basic decisions: test setup
> >>>>>>
> >>>>>> I would like to add some "non-binding" comments here
> >> regarding
> >>>> 3).
> >>>>>>
> >>>>>> Actually I really like the concept Jason described and which is
> >>>>>> currently used in Seam3. Having worked with it a bit I can
> >> confirm
> >>>>>> that it runs fine. And somehow it's the best of both worlds
> >> (in
> >>>> regard
> >>>>>> to 3a vs 3b).
> >>>>>>
> >>>>>> That the full integration test suite with all containers cannot
> >> run in
> >>>>>> a single Maven invocation is IMHO not a real problem. We could
> >> write a
> >>>>>> simple script for this.
> >>>>>>
> >>>>>> The only thing that could be optimized in this setup is the
> >> separation
> >>>>>> of container dependent tests via package name. But perhaps this
> >> could
> >>>>>> be solved by Arquillian? What I would really like to see is the
> >>>>>> ability to configure a test so it is only executed in a
> >> specific
> >>>>>> container (something similar to @TargetsContainer). This way
> >> tests
> >>>>>> that require to be run in a full container could be executed
> >> only in
> >>>>>> JBoss and Glassfish but would be skipped in embedded Weld.
> >>>>>>
> >>>>>> Christian
> >>>>>>
> >>>>>>
> >>>>>> 2011/12/16 Jason Porter <li...@gmail.com>:
> >>>>>>> Comments inline.
> >>>>>>>
> >>>>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg
> >>>> <st...@yahoo.de> wrote:
> >>>>>>>
> >>>>>>>> Hi!
> >>>>>>>>
> >>>>>>>> Another round of discussion - this time about how to
> >> setup our
> >>>> unit
> >>>>>>>> testing and integration testing.
> >>>>>>>>
> >>>>>>>> I think it's pretty much clear that we will gonna
> >> use
> >>>> Arquillian for all
> >>>>>>>> our tests. That was also fundamental part of the
> >> incubator
> >>>> proposal.
> >>>>>>>> And instead of inventing yet another abstraction
> >> layer, I'd
> >>>> favour to just
> >>>>>>>> use Arquillian and contribute all things we need to
> >> this
> >>>> project.
> >>>>>>>> So here a formal vote
> >>>>>>>>
> >>>>>>>> 1.)
> >>>>>>>>
> >>>>>>>> for using Arquillian as test integration framework.
> >>>>>>>> +1 from me
> >>>>>>>>
> >>>>>>>
> >>>>>>> +1 from me
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> 2.)
> >>>>>>>>
> >>>>>>>> What do we like to use from the unit tests itself?
> >> TestNG or
> >>>> JUnit?
> >>>>>>>> JUnit has better Arquillian integration and TestNG is
> >> better
> >>>> for 'real
> >>>>>>>> world' projects, because it allows to define test
> >>>> dependencies.
> >>>>>>>> So which one to take?
> >>>>>>>> But we should definitely only use 1 of the two
> >> exclusively!
> >>>>>>>>
> >>>>>>>
> >>>>>>> +1 for using one exclusively
> >>>>>>>
> >>>>>>> As for which, I'm also +0 I like both for different
> >> reasons,
> >>>> but JUnit is
> >>>>>>> certainly the easier of the two to use with Arquillian.
> >>>>>>>
> >>>>>>> Also, are we open to having other languages for tests or
> >> using
> >>>> other
> >>>>>>> testing structures such as spock (it works with
> >> Arquillian, BTW),
> >>>> RSpec,
> >>>>>>> scala, etc.?
> >>>>>>>
> >>>>>>>
> >>>>>>>> 3.)
> >>>>>>>> 'Integrated' tests vs 'Integration
> >> Tests'
> >>>>>>>> We have 2 options to test our projects
> >>>>>>>>
> >>>>>>>> 3.a.) create a full unit test in each module (e.g.
> >>>> deltaspike/core/impl)
> >>>>>>>> and add a profile for each and every server (maybe we
> >> can trim
> >>>> this down
> >>>>>>>> with pom imports?).
> >>>>>>>> Then run the build with one after each other:
> >>>>>>>>
> >>>>>>>> $> mvn clean test -Powb
> >>>>>>>> $> mvn clean test -Pweld
> >>>>>>>> $> mvn clean test -Powb-tc
> >>>>>>>> $> mvn clean test -Pweld-tc
> >>>>>>>> $> mvn clean test -Pgf31
> >>>>>>>> $> mvn clean test -Pas7
> >>>>>>>> $> mvn clean test -Ptomee
> >>>>>>>>
> >>>>>>>> This might pretty much blow up our poms...
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 3.b) create a full unit test in each module (e.g.
> >>>> deltaspike/core/impl)
> >>>>>>>> and add only 2 profiles directly ('weld' and
> >>>> 'owb' (probably resin later))
> >>>>>>>> Then add a deltaspike/test/base/core with the very
> >> basic parent
> >>>> stuff and
> >>>>>>>> 1 module for each integrated container, e.g.
> >>>>>>>> deltaspike/test/weld-tc
> >>>>>>>> deltaspike/test/owb-tc
> >>>>>>>> deltaspike/test/weld-jetty
> >>>>>>>> deltaspike/test/owb-jetty
> >>>>>>>> deltaspike/test/tomee
> >>>>>>>> deltaspike/test/gf31
> >>>>>>>> deltaspike/test/geronimo
> >>>>>>>> deltaspike/test/websphere
> >>>>>>>> deltaspike/test/as7
> >>>>>>>> deltaspike/test/as6
> >>>>>>>> etc.
> >>>>>>>>
> >>>>>>>> The initial setup costs are higher, but it would be
> >> pretty easy
> >>>> to add new
> >>>>>>>> containers that way.
> >>>>>>>>
> >>>>>>>> Which route shall we take?
> >>>>>>>>
> >>>>>>>
> >>>>>>> In Seam 3 we have an approach which is a little bit of a
> >> mixture of
> >>>> both.
> >>>>>>> We have regular unit tests (with mockito, stubs, etc)
> >> under
> >>>>>>> impl/src/test/... and then we have a separate module
> >> called
> >>>> testsuite which
> >>>>>>> contained all the integration tests. We used pom imports
> >> to get all
> >>>> the
> >>>>>>> containers setup correctly and then had different profiles
> >> which
> >>>> pulled in
> >>>>>>> the poms and also configured surefire for that server.
> >> Inside the
> >>>> code
> >>>>>>> structure we separated the servers by package: common,
> >> jbossas,
> >>>> glassfish,
> >>>>>>> etc. That way all of the tests were in one location, they
> >> were easy
> >>>> to
> >>>>>>> execute, the tests stayed DRY, and the IDE wasn't much
> >> of a
> >>>> problem. The
> >>>>>>> two downsides we had were 1) the extra config in the pom
> >> and 2) you
> >>>> can't
> >>>>>>> run the full integration suite in one maven execution. We
> >> spent
> >>>> quite a bit
> >>>>>>> of time examining options and trying some of them out.
> >> This was the
> >>>> best we
> >>>>>>> came up with, and also the one Aslak thought would work
> >> best as
> >>>> well.
> >>>>>>>
> >>>>>>>
> >>>>>>>> LieGrue,
> >>>>>>>> strub
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Jason Porter
> >>>>>>> http://lightguard-jp.blogspot.com
> >>>>>>> http://twitter.com/lightguardjp
> >>>>>>>
> >>>>>>> Software Engineer
> >>>>>>> Open Source Advocate
> >>>>>>> Author of Seam Catch - Next Generation Java Exception
> >> Handling
> >>>>>>>
> >>>>>>> PGP key id: 926CCFF5
> >>>>>>> PGP key available at: keyserver.net, pgp.mit.edu
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Christian Kaltepoth
> >>>>>> Blog: http://chkal.blogspot.com/
> >>>>>> Twitter: http://twitter.com/chkal
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Christian Kaltepoth
> >>>> Blog: http://chkal.blogspot.com/
> >>>> Twitter: http://twitter.com/chkal
> >>>>
> >>
> >>
> >>
> >> --
> >> Christian Kaltepoth
> >> Blog: http://chkal.blogspot.com/
> >> Twitter: http://twitter.com/chkal
> >>
>
Re: basic decisions: test setup
Posted by Christian Kaltepoth <ch...@kaltepoth.de>.
I just thought about the package naming convention idea a bit further.
Isn't such an environment constraint always something like "requires
at least X"?
In this case we could do it this way:
If a test requires at least EE-light, it is placed in a package
*.eelight.* within the testsuite. All containers profiles that don't
support the minimum requirement (e.g. weld-embedded or tomcat) exclude
this package in the Surefire configuration. This would be very simply
to set up.
It's a bit similar to what Seam3 currently does but the idea is based
on environments instead of containers and the "requires at least"
concept makes it very simple to use.
Thoughts?
Christian
2011/12/17 Jason Porter <li...@gmail.com>:
> I like the idea, however, that would be a very core change to both core arquillian and the containers. I don't think we'll get that anytime soon. Maybe a simple package convention by environment will work for now?
>
> Sent from my iPhone
>
> On Dec 17, 2011, at 9:19, Mark Struberg <st...@yahoo.de> wrote:
>
>> true, we would need something like
>>
>> @RunInEnvironments({Environment.EeLight, Environment.EeFull, Environment.Servlet})
>>
>> @RunInEnvironments(Environment.JavaSE)
>> @RunInEnvironments(Environments.ALL)
>>
>> LieGrue,
>> strub
>>
>>
>>
>> ----- Original Message -----
>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>> To: deltaspike-dev@incubator.apache.org
>>> Cc:
>>> Sent: Saturday, December 17, 2011 5:04 PM
>>> Subject: Re: basic decisions: test setup
>>>
>>> @Mark:
>>>
>>> But what about tests that should run in EE-light, EE-full and Servlet
>>> but not in SE. Or tests that should run in EE-light/full but not in
>>> Servlet and SE. I think this could get very complex.
>>>
>>> Christian
>>>
>>>
>>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>>>> We might solve this simply with a package naming rule. And then we
>>> configure surefire to just run the tests with the respective package?
>>>>
>>>> Not the most advanced solution, but will work.
>>>>
>>>>
>>>> LieGrue,
>>>> strub
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>>>> To: deltaspike-dev@incubator.apache.org
>>>>> Cc:
>>>>> Sent: Saturday, December 17, 2011 4:43 PM
>>>>> Subject: Re: basic decisions: test setup
>>>>>
>>>>> @Mark:
>>>>>
>>>>> Actually @TargetsContainer is an existing feature of Arquillian. It
>>>>> allows to build container specific deployments.
>>>>>
>>>>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>>>>>
>>>>> Unfortunately it won't fit our needs because the test completely
>>> fails
>>>>> if there is no matching deployment for the container the test is
>>>>> executed against. So I guess we have to build something new for this.
>>>>>
>>>>> I really like your suggestion regarding the four environments. It
>>>>> absolutely make sense to declare environment constraints (like SE,
>>>>> Servlet, EE, etc.) instead of listing concrete containers.
>>>>>
>>>>> Christian
>>>>>
>>>>>
>>>>>
>>>>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>>>>>> Hi Christian!
>>>>>>
>>>>>>
>>>>>> @TargetContainer sounds good, but I'd rather make it a
>>>>>>
>>>>>>
>>>>>> @TargetEnvironment()
>>>>>>
>>>>>> with values: SE, Servlet, EE-light, EE-full
>>>>>>
>>>>>> The excludes should be done in the single test integration
>>> modules, and the
>>>>> goal should be to get down to zero excluded tests for all containers.
>>>>>>
>>>>>>
>>>>>> @Jason: that mixture for the tests looks fine to me as well. Can
>>> you please
>>>>> hack a sample structure in the DeltaSpikePlayground git repo?
>>>>>>
>>>>>>
>>>>>> LieGrue,
>>>>>> strub
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> ________________________________
>>>>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>>>>>> To: deltaspike-dev@incubator.apache.org
>>>>>>> Sent: Saturday, December 17, 2011 7:55 AM
>>>>>>> Subject: Re: basic decisions: test setup
>>>>>>>
>>>>>>> I would like to add some "non-binding" comments here
>>> regarding
>>>>> 3).
>>>>>>>
>>>>>>> Actually I really like the concept Jason described and which is
>>>>>>> currently used in Seam3. Having worked with it a bit I can
>>> confirm
>>>>>>> that it runs fine. And somehow it's the best of both worlds
>>> (in
>>>>> regard
>>>>>>> to 3a vs 3b).
>>>>>>>
>>>>>>> That the full integration test suite with all containers cannot
>>> run in
>>>>>>> a single Maven invocation is IMHO not a real problem. We could
>>> write a
>>>>>>> simple script for this.
>>>>>>>
>>>>>>> The only thing that could be optimized in this setup is the
>>> separation
>>>>>>> of container dependent tests via package name. But perhaps this
>>> could
>>>>>>> be solved by Arquillian? What I would really like to see is the
>>>>>>> ability to configure a test so it is only executed in a
>>> specific
>>>>>>> container (something similar to @TargetsContainer). This way
>>> tests
>>>>>>> that require to be run in a full container could be executed
>>> only in
>>>>>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>>>>>>
>>>>>>> Christian
>>>>>>>
>>>>>>>
>>>>>>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>>>>>>> Comments inline.
>>>>>>>>
>>>>>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg
>>>>> <st...@yahoo.de> wrote:
>>>>>>>>
>>>>>>>>> Hi!
>>>>>>>>>
>>>>>>>>> Another round of discussion - this time about how to
>>> setup our
>>>>> unit
>>>>>>>>> testing and integration testing.
>>>>>>>>>
>>>>>>>>> I think it's pretty much clear that we will gonna
>>> use
>>>>> Arquillian for all
>>>>>>>>> our tests. That was also fundamental part of the
>>> incubator
>>>>> proposal.
>>>>>>>>> And instead of inventing yet another abstraction
>>> layer, I'd
>>>>> favour to just
>>>>>>>>> use Arquillian and contribute all things we need to
>>> this
>>>>> project.
>>>>>>>>> So here a formal vote
>>>>>>>>>
>>>>>>>>> 1.)
>>>>>>>>>
>>>>>>>>> for using Arquillian as test integration framework.
>>>>>>>>> +1 from me
>>>>>>>>>
>>>>>>>>
>>>>>>>> +1 from me
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2.)
>>>>>>>>>
>>>>>>>>> What do we like to use from the unit tests itself?
>>> TestNG or
>>>>> JUnit?
>>>>>>>>> JUnit has better Arquillian integration and TestNG is
>>> better
>>>>> for 'real
>>>>>>>>> world' projects, because it allows to define test
>>>>> dependencies.
>>>>>>>>> So which one to take?
>>>>>>>>> But we should definitely only use 1 of the two
>>> exclusively!
>>>>>>>>>
>>>>>>>>
>>>>>>>> +1 for using one exclusively
>>>>>>>>
>>>>>>>> As for which, I'm also +0 I like both for different
>>> reasons,
>>>>> but JUnit is
>>>>>>>> certainly the easier of the two to use with Arquillian.
>>>>>>>>
>>>>>>>> Also, are we open to having other languages for tests or
>>> using
>>>>> other
>>>>>>>> testing structures such as spock (it works with
>>> Arquillian, BTW),
>>>>> RSpec,
>>>>>>>> scala, etc.?
>>>>>>>>
>>>>>>>>
>>>>>>>>> 3.)
>>>>>>>>> 'Integrated' tests vs 'Integration
>>> Tests'
>>>>>>>>> We have 2 options to test our projects
>>>>>>>>>
>>>>>>>>> 3.a.) create a full unit test in each module (e.g.
>>>>> deltaspike/core/impl)
>>>>>>>>> and add a profile for each and every server (maybe we
>>> can trim
>>>>> this down
>>>>>>>>> with pom imports?).
>>>>>>>>> Then run the build with one after each other:
>>>>>>>>>
>>>>>>>>> $> mvn clean test -Powb
>>>>>>>>> $> mvn clean test -Pweld
>>>>>>>>> $> mvn clean test -Powb-tc
>>>>>>>>> $> mvn clean test -Pweld-tc
>>>>>>>>> $> mvn clean test -Pgf31
>>>>>>>>> $> mvn clean test -Pas7
>>>>>>>>> $> mvn clean test -Ptomee
>>>>>>>>>
>>>>>>>>> This might pretty much blow up our poms...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 3.b) create a full unit test in each module (e.g.
>>>>> deltaspike/core/impl)
>>>>>>>>> and add only 2 profiles directly ('weld' and
>>>>> 'owb' (probably resin later))
>>>>>>>>> Then add a deltaspike/test/base/core with the very
>>> basic parent
>>>>> stuff and
>>>>>>>>> 1 module for each integrated container, e.g.
>>>>>>>>> deltaspike/test/weld-tc
>>>>>>>>> deltaspike/test/owb-tc
>>>>>>>>> deltaspike/test/weld-jetty
>>>>>>>>> deltaspike/test/owb-jetty
>>>>>>>>> deltaspike/test/tomee
>>>>>>>>> deltaspike/test/gf31
>>>>>>>>> deltaspike/test/geronimo
>>>>>>>>> deltaspike/test/websphere
>>>>>>>>> deltaspike/test/as7
>>>>>>>>> deltaspike/test/as6
>>>>>>>>> etc.
>>>>>>>>>
>>>>>>>>> The initial setup costs are higher, but it would be
>>> pretty easy
>>>>> to add new
>>>>>>>>> containers that way.
>>>>>>>>>
>>>>>>>>> Which route shall we take?
>>>>>>>>>
>>>>>>>>
>>>>>>>> In Seam 3 we have an approach which is a little bit of a
>>> mixture of
>>>>> both.
>>>>>>>> We have regular unit tests (with mockito, stubs, etc)
>>> under
>>>>>>>> impl/src/test/... and then we have a separate module
>>> called
>>>>> testsuite which
>>>>>>>> contained all the integration tests. We used pom imports
>>> to get all
>>>>> the
>>>>>>>> containers setup correctly and then had different profiles
>>> which
>>>>> pulled in
>>>>>>>> the poms and also configured surefire for that server.
>>> Inside the
>>>>> code
>>>>>>>> structure we separated the servers by package: common,
>>> jbossas,
>>>>> glassfish,
>>>>>>>> etc. That way all of the tests were in one location, they
>>> were easy
>>>>> to
>>>>>>>> execute, the tests stayed DRY, and the IDE wasn't much
>>> of a
>>>>> problem. The
>>>>>>>> two downsides we had were 1) the extra config in the pom
>>> and 2) you
>>>>> can't
>>>>>>>> run the full integration suite in one maven execution. We
>>> spent
>>>>> quite a bit
>>>>>>>> of time examining options and trying some of them out.
>>> This was the
>>>>> best we
>>>>>>>> came up with, and also the one Aslak thought would work
>>> best as
>>>>> well.
>>>>>>>>
>>>>>>>>
>>>>>>>>> LieGrue,
>>>>>>>>> strub
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Jason Porter
>>>>>>>> http://lightguard-jp.blogspot.com
>>>>>>>> http://twitter.com/lightguardjp
>>>>>>>>
>>>>>>>> Software Engineer
>>>>>>>> Open Source Advocate
>>>>>>>> Author of Seam Catch - Next Generation Java Exception
>>> Handling
>>>>>>>>
>>>>>>>> PGP key id: 926CCFF5
>>>>>>>> PGP key available at: keyserver.net, pgp.mit.edu
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Christian Kaltepoth
>>>>>>> Blog: http://chkal.blogspot.com/
>>>>>>> Twitter: http://twitter.com/chkal
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Christian Kaltepoth
>>>>> Blog: http://chkal.blogspot.com/
>>>>> Twitter: http://twitter.com/chkal
>>>>>
>>>
>>>
>>>
>>> --
>>> Christian Kaltepoth
>>> Blog: http://chkal.blogspot.com/
>>> Twitter: http://twitter.com/chkal
>>>
--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Jason Porter <li...@gmail.com>.
I like the idea, however, that would be a very core change to both core arquillian and the containers. I don't think we'll get that anytime soon. Maybe a simple package convention by environment will work for now?
Sent from my iPhone
On Dec 17, 2011, at 9:19, Mark Struberg <st...@yahoo.de> wrote:
> true, we would need something like
>
> @RunInEnvironments({Environment.EeLight, Environment.EeFull, Environment.Servlet})
>
> @RunInEnvironments(Environment.JavaSE)
> @RunInEnvironments(Environments.ALL)
>
> LieGrue,
> strub
>
>
>
> ----- Original Message -----
>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>> To: deltaspike-dev@incubator.apache.org
>> Cc:
>> Sent: Saturday, December 17, 2011 5:04 PM
>> Subject: Re: basic decisions: test setup
>>
>> @Mark:
>>
>> But what about tests that should run in EE-light, EE-full and Servlet
>> but not in SE. Or tests that should run in EE-light/full but not in
>> Servlet and SE. I think this could get very complex.
>>
>> Christian
>>
>>
>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>>> We might solve this simply with a package naming rule. And then we
>> configure surefire to just run the tests with the respective package?
>>>
>>> Not the most advanced solution, but will work.
>>>
>>>
>>> LieGrue,
>>> strub
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>>> To: deltaspike-dev@incubator.apache.org
>>>> Cc:
>>>> Sent: Saturday, December 17, 2011 4:43 PM
>>>> Subject: Re: basic decisions: test setup
>>>>
>>>> @Mark:
>>>>
>>>> Actually @TargetsContainer is an existing feature of Arquillian. It
>>>> allows to build container specific deployments.
>>>>
>>>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>>>>
>>>> Unfortunately it won't fit our needs because the test completely
>> fails
>>>> if there is no matching deployment for the container the test is
>>>> executed against. So I guess we have to build something new for this.
>>>>
>>>> I really like your suggestion regarding the four environments. It
>>>> absolutely make sense to declare environment constraints (like SE,
>>>> Servlet, EE, etc.) instead of listing concrete containers.
>>>>
>>>> Christian
>>>>
>>>>
>>>>
>>>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>>>>> Hi Christian!
>>>>>
>>>>>
>>>>> @TargetContainer sounds good, but I'd rather make it a
>>>>>
>>>>>
>>>>> @TargetEnvironment()
>>>>>
>>>>> with values: SE, Servlet, EE-light, EE-full
>>>>>
>>>>> The excludes should be done in the single test integration
>> modules, and the
>>>> goal should be to get down to zero excluded tests for all containers.
>>>>>
>>>>>
>>>>> @Jason: that mixture for the tests looks fine to me as well. Can
>> you please
>>>> hack a sample structure in the DeltaSpikePlayground git repo?
>>>>>
>>>>>
>>>>> LieGrue,
>>>>> strub
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> ________________________________
>>>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>>>>> To: deltaspike-dev@incubator.apache.org
>>>>>> Sent: Saturday, December 17, 2011 7:55 AM
>>>>>> Subject: Re: basic decisions: test setup
>>>>>>
>>>>>> I would like to add some "non-binding" comments here
>> regarding
>>>> 3).
>>>>>>
>>>>>> Actually I really like the concept Jason described and which is
>>>>>> currently used in Seam3. Having worked with it a bit I can
>> confirm
>>>>>> that it runs fine. And somehow it's the best of both worlds
>> (in
>>>> regard
>>>>>> to 3a vs 3b).
>>>>>>
>>>>>> That the full integration test suite with all containers cannot
>> run in
>>>>>> a single Maven invocation is IMHO not a real problem. We could
>> write a
>>>>>> simple script for this.
>>>>>>
>>>>>> The only thing that could be optimized in this setup is the
>> separation
>>>>>> of container dependent tests via package name. But perhaps this
>> could
>>>>>> be solved by Arquillian? What I would really like to see is the
>>>>>> ability to configure a test so it is only executed in a
>> specific
>>>>>> container (something similar to @TargetsContainer). This way
>> tests
>>>>>> that require to be run in a full container could be executed
>> only in
>>>>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>>>>>
>>>>>> Christian
>>>>>>
>>>>>>
>>>>>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>>>>>> Comments inline.
>>>>>>>
>>>>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg
>>>> <st...@yahoo.de> wrote:
>>>>>>>
>>>>>>>> Hi!
>>>>>>>>
>>>>>>>> Another round of discussion - this time about how to
>> setup our
>>>> unit
>>>>>>>> testing and integration testing.
>>>>>>>>
>>>>>>>> I think it's pretty much clear that we will gonna
>> use
>>>> Arquillian for all
>>>>>>>> our tests. That was also fundamental part of the
>> incubator
>>>> proposal.
>>>>>>>> And instead of inventing yet another abstraction
>> layer, I'd
>>>> favour to just
>>>>>>>> use Arquillian and contribute all things we need to
>> this
>>>> project.
>>>>>>>> So here a formal vote
>>>>>>>>
>>>>>>>> 1.)
>>>>>>>>
>>>>>>>> for using Arquillian as test integration framework.
>>>>>>>> +1 from me
>>>>>>>>
>>>>>>>
>>>>>>> +1 from me
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> 2.)
>>>>>>>>
>>>>>>>> What do we like to use from the unit tests itself?
>> TestNG or
>>>> JUnit?
>>>>>>>> JUnit has better Arquillian integration and TestNG is
>> better
>>>> for 'real
>>>>>>>> world' projects, because it allows to define test
>>>> dependencies.
>>>>>>>> So which one to take?
>>>>>>>> But we should definitely only use 1 of the two
>> exclusively!
>>>>>>>>
>>>>>>>
>>>>>>> +1 for using one exclusively
>>>>>>>
>>>>>>> As for which, I'm also +0 I like both for different
>> reasons,
>>>> but JUnit is
>>>>>>> certainly the easier of the two to use with Arquillian.
>>>>>>>
>>>>>>> Also, are we open to having other languages for tests or
>> using
>>>> other
>>>>>>> testing structures such as spock (it works with
>> Arquillian, BTW),
>>>> RSpec,
>>>>>>> scala, etc.?
>>>>>>>
>>>>>>>
>>>>>>>> 3.)
>>>>>>>> 'Integrated' tests vs 'Integration
>> Tests'
>>>>>>>> We have 2 options to test our projects
>>>>>>>>
>>>>>>>> 3.a.) create a full unit test in each module (e.g.
>>>> deltaspike/core/impl)
>>>>>>>> and add a profile for each and every server (maybe we
>> can trim
>>>> this down
>>>>>>>> with pom imports?).
>>>>>>>> Then run the build with one after each other:
>>>>>>>>
>>>>>>>> $> mvn clean test -Powb
>>>>>>>> $> mvn clean test -Pweld
>>>>>>>> $> mvn clean test -Powb-tc
>>>>>>>> $> mvn clean test -Pweld-tc
>>>>>>>> $> mvn clean test -Pgf31
>>>>>>>> $> mvn clean test -Pas7
>>>>>>>> $> mvn clean test -Ptomee
>>>>>>>>
>>>>>>>> This might pretty much blow up our poms...
>>>>>>>>
>>>>>>>>
>>>>>>>> 3.b) create a full unit test in each module (e.g.
>>>> deltaspike/core/impl)
>>>>>>>> and add only 2 profiles directly ('weld' and
>>>> 'owb' (probably resin later))
>>>>>>>> Then add a deltaspike/test/base/core with the very
>> basic parent
>>>> stuff and
>>>>>>>> 1 module for each integrated container, e.g.
>>>>>>>> deltaspike/test/weld-tc
>>>>>>>> deltaspike/test/owb-tc
>>>>>>>> deltaspike/test/weld-jetty
>>>>>>>> deltaspike/test/owb-jetty
>>>>>>>> deltaspike/test/tomee
>>>>>>>> deltaspike/test/gf31
>>>>>>>> deltaspike/test/geronimo
>>>>>>>> deltaspike/test/websphere
>>>>>>>> deltaspike/test/as7
>>>>>>>> deltaspike/test/as6
>>>>>>>> etc.
>>>>>>>>
>>>>>>>> The initial setup costs are higher, but it would be
>> pretty easy
>>>> to add new
>>>>>>>> containers that way.
>>>>>>>>
>>>>>>>> Which route shall we take?
>>>>>>>>
>>>>>>>
>>>>>>> In Seam 3 we have an approach which is a little bit of a
>> mixture of
>>>> both.
>>>>>>> We have regular unit tests (with mockito, stubs, etc)
>> under
>>>>>>> impl/src/test/... and then we have a separate module
>> called
>>>> testsuite which
>>>>>>> contained all the integration tests. We used pom imports
>> to get all
>>>> the
>>>>>>> containers setup correctly and then had different profiles
>> which
>>>> pulled in
>>>>>>> the poms and also configured surefire for that server.
>> Inside the
>>>> code
>>>>>>> structure we separated the servers by package: common,
>> jbossas,
>>>> glassfish,
>>>>>>> etc. That way all of the tests were in one location, they
>> were easy
>>>> to
>>>>>>> execute, the tests stayed DRY, and the IDE wasn't much
>> of a
>>>> problem. The
>>>>>>> two downsides we had were 1) the extra config in the pom
>> and 2) you
>>>> can't
>>>>>>> run the full integration suite in one maven execution. We
>> spent
>>>> quite a bit
>>>>>>> of time examining options and trying some of them out.
>> This was the
>>>> best we
>>>>>>> came up with, and also the one Aslak thought would work
>> best as
>>>> well.
>>>>>>>
>>>>>>>
>>>>>>>> LieGrue,
>>>>>>>> strub
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Jason Porter
>>>>>>> http://lightguard-jp.blogspot.com
>>>>>>> http://twitter.com/lightguardjp
>>>>>>>
>>>>>>> Software Engineer
>>>>>>> Open Source Advocate
>>>>>>> Author of Seam Catch - Next Generation Java Exception
>> Handling
>>>>>>>
>>>>>>> PGP key id: 926CCFF5
>>>>>>> PGP key available at: keyserver.net, pgp.mit.edu
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Christian Kaltepoth
>>>>>> Blog: http://chkal.blogspot.com/
>>>>>> Twitter: http://twitter.com/chkal
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Christian Kaltepoth
>>>> Blog: http://chkal.blogspot.com/
>>>> Twitter: http://twitter.com/chkal
>>>>
>>
>>
>>
>> --
>> Christian Kaltepoth
>> Blog: http://chkal.blogspot.com/
>> Twitter: http://twitter.com/chkal
>>
Re: basic decisions: test setup
Posted by Mark Struberg <st...@yahoo.de>.
true, we would need something like
@RunInEnvironments({Environment.EeLight, Environment.EeFull, Environment.Servlet})
@RunInEnvironments(Environment.JavaSE)
@RunInEnvironments(Environments.ALL)
LieGrue,
strub
----- Original Message -----
> From: Christian Kaltepoth <ch...@kaltepoth.de>
> To: deltaspike-dev@incubator.apache.org
> Cc:
> Sent: Saturday, December 17, 2011 5:04 PM
> Subject: Re: basic decisions: test setup
>
> @Mark:
>
> But what about tests that should run in EE-light, EE-full and Servlet
> but not in SE. Or tests that should run in EE-light/full but not in
> Servlet and SE. I think this could get very complex.
>
> Christian
>
>
> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>> We might solve this simply with a package naming rule. And then we
> configure surefire to just run the tests with the respective package?
>>
>> Not the most advanced solution, but will work.
>>
>>
>> LieGrue,
>> strub
>>
>>
>>
>> ----- Original Message -----
>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>> To: deltaspike-dev@incubator.apache.org
>>> Cc:
>>> Sent: Saturday, December 17, 2011 4:43 PM
>>> Subject: Re: basic decisions: test setup
>>>
>>> @Mark:
>>>
>>> Actually @TargetsContainer is an existing feature of Arquillian. It
>>> allows to build container specific deployments.
>>>
>>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>>>
>>> Unfortunately it won't fit our needs because the test completely
> fails
>>> if there is no matching deployment for the container the test is
>>> executed against. So I guess we have to build something new for this.
>>>
>>> I really like your suggestion regarding the four environments. It
>>> absolutely make sense to declare environment constraints (like SE,
>>> Servlet, EE, etc.) instead of listing concrete containers.
>>>
>>> Christian
>>>
>>>
>>>
>>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>>>> Hi Christian!
>>>>
>>>>
>>>> @TargetContainer sounds good, but I'd rather make it a
>>>>
>>>>
>>>> @TargetEnvironment()
>>>>
>>>> with values: SE, Servlet, EE-light, EE-full
>>>>
>>>> The excludes should be done in the single test integration
> modules, and the
>>> goal should be to get down to zero excluded tests for all containers.
>>>>
>>>>
>>>> @Jason: that mixture for the tests looks fine to me as well. Can
> you please
>>> hack a sample structure in the DeltaSpikePlayground git repo?
>>>>
>>>>
>>>> LieGrue,
>>>> strub
>>>>
>>>>
>>>>
>>>>
>>>>> ________________________________
>>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>>>> To: deltaspike-dev@incubator.apache.org
>>>>> Sent: Saturday, December 17, 2011 7:55 AM
>>>>> Subject: Re: basic decisions: test setup
>>>>>
>>>>> I would like to add some "non-binding" comments here
> regarding
>>> 3).
>>>>>
>>>>> Actually I really like the concept Jason described and which is
>>>>> currently used in Seam3. Having worked with it a bit I can
> confirm
>>>>> that it runs fine. And somehow it's the best of both worlds
> (in
>>> regard
>>>>> to 3a vs 3b).
>>>>>
>>>>> That the full integration test suite with all containers cannot
> run in
>>>>> a single Maven invocation is IMHO not a real problem. We could
> write a
>>>>> simple script for this.
>>>>>
>>>>> The only thing that could be optimized in this setup is the
> separation
>>>>> of container dependent tests via package name. But perhaps this
> could
>>>>> be solved by Arquillian? What I would really like to see is the
>>>>> ability to configure a test so it is only executed in a
> specific
>>>>> container (something similar to @TargetsContainer). This way
> tests
>>>>> that require to be run in a full container could be executed
> only in
>>>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>>>>
>>>>> Christian
>>>>>
>>>>>
>>>>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>>>>> Comments inline.
>>>>>>
>>>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg
>>> <st...@yahoo.de> wrote:
>>>>>>
>>>>>>> Hi!
>>>>>>>
>>>>>>> Another round of discussion - this time about how to
> setup our
>>> unit
>>>>>>> testing and integration testing.
>>>>>>>
>>>>>>> I think it's pretty much clear that we will gonna
> use
>>> Arquillian for all
>>>>>>> our tests. That was also fundamental part of the
> incubator
>>> proposal.
>>>>>>> And instead of inventing yet another abstraction
> layer, I'd
>>> favour to just
>>>>>>> use Arquillian and contribute all things we need to
> this
>>> project.
>>>>>>> So here a formal vote
>>>>>>>
>>>>>>> 1.)
>>>>>>>
>>>>>>> for using Arquillian as test integration framework.
>>>>>>> +1 from me
>>>>>>>
>>>>>>
>>>>>> +1 from me
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> 2.)
>>>>>>>
>>>>>>> What do we like to use from the unit tests itself?
> TestNG or
>>> JUnit?
>>>>>>> JUnit has better Arquillian integration and TestNG is
> better
>>> for 'real
>>>>>>> world' projects, because it allows to define test
>>> dependencies.
>>>>>>> So which one to take?
>>>>>>> But we should definitely only use 1 of the two
> exclusively!
>>>>>>>
>>>>>>
>>>>>> +1 for using one exclusively
>>>>>>
>>>>>> As for which, I'm also +0 I like both for different
> reasons,
>>> but JUnit is
>>>>>> certainly the easier of the two to use with Arquillian.
>>>>>>
>>>>>> Also, are we open to having other languages for tests or
> using
>>> other
>>>>>> testing structures such as spock (it works with
> Arquillian, BTW),
>>> RSpec,
>>>>>> scala, etc.?
>>>>>>
>>>>>>
>>>>>>> 3.)
>>>>>>> 'Integrated' tests vs 'Integration
> Tests'
>>>>>>> We have 2 options to test our projects
>>>>>>>
>>>>>>> 3.a.) create a full unit test in each module (e.g.
>>> deltaspike/core/impl)
>>>>>>> and add a profile for each and every server (maybe we
> can trim
>>> this down
>>>>>>> with pom imports?).
>>>>>>> Then run the build with one after each other:
>>>>>>>
>>>>>>> $> mvn clean test -Powb
>>>>>>> $> mvn clean test -Pweld
>>>>>>> $> mvn clean test -Powb-tc
>>>>>>> $> mvn clean test -Pweld-tc
>>>>>>> $> mvn clean test -Pgf31
>>>>>>> $> mvn clean test -Pas7
>>>>>>> $> mvn clean test -Ptomee
>>>>>>>
>>>>>>> This might pretty much blow up our poms...
>>>>>>>
>>>>>>>
>>>>>>> 3.b) create a full unit test in each module (e.g.
>>> deltaspike/core/impl)
>>>>>>> and add only 2 profiles directly ('weld' and
>>> 'owb' (probably resin later))
>>>>>>> Then add a deltaspike/test/base/core with the very
> basic parent
>>> stuff and
>>>>>>> 1 module for each integrated container, e.g.
>>>>>>> deltaspike/test/weld-tc
>>>>>>> deltaspike/test/owb-tc
>>>>>>> deltaspike/test/weld-jetty
>>>>>>> deltaspike/test/owb-jetty
>>>>>>> deltaspike/test/tomee
>>>>>>> deltaspike/test/gf31
>>>>>>> deltaspike/test/geronimo
>>>>>>> deltaspike/test/websphere
>>>>>>> deltaspike/test/as7
>>>>>>> deltaspike/test/as6
>>>>>>> etc.
>>>>>>>
>>>>>>> The initial setup costs are higher, but it would be
> pretty easy
>>> to add new
>>>>>>> containers that way.
>>>>>>>
>>>>>>> Which route shall we take?
>>>>>>>
>>>>>>
>>>>>> In Seam 3 we have an approach which is a little bit of a
> mixture of
>>> both.
>>>>>> We have regular unit tests (with mockito, stubs, etc)
> under
>>>>>> impl/src/test/... and then we have a separate module
> called
>>> testsuite which
>>>>>> contained all the integration tests. We used pom imports
> to get all
>>> the
>>>>>> containers setup correctly and then had different profiles
> which
>>> pulled in
>>>>>> the poms and also configured surefire for that server.
> Inside the
>>> code
>>>>>> structure we separated the servers by package: common,
> jbossas,
>>> glassfish,
>>>>>> etc. That way all of the tests were in one location, they
> were easy
>>> to
>>>>>> execute, the tests stayed DRY, and the IDE wasn't much
> of a
>>> problem. The
>>>>>> two downsides we had were 1) the extra config in the pom
> and 2) you
>>> can't
>>>>>> run the full integration suite in one maven execution. We
> spent
>>> quite a bit
>>>>>> of time examining options and trying some of them out.
> This was the
>>> best we
>>>>>> came up with, and also the one Aslak thought would work
> best as
>>> well.
>>>>>>
>>>>>>
>>>>>>> LieGrue,
>>>>>>> strub
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jason Porter
>>>>>> http://lightguard-jp.blogspot.com
>>>>>> http://twitter.com/lightguardjp
>>>>>>
>>>>>> Software Engineer
>>>>>> Open Source Advocate
>>>>>> Author of Seam Catch - Next Generation Java Exception
> Handling
>>>>>>
>>>>>> PGP key id: 926CCFF5
>>>>>> PGP key available at: keyserver.net, pgp.mit.edu
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Christian Kaltepoth
>>>>> Blog: http://chkal.blogspot.com/
>>>>> Twitter: http://twitter.com/chkal
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>> --
>>> Christian Kaltepoth
>>> Blog: http://chkal.blogspot.com/
>>> Twitter: http://twitter.com/chkal
>>>
>
>
>
> --
> Christian Kaltepoth
> Blog: http://chkal.blogspot.com/
> Twitter: http://twitter.com/chkal
>
Re: basic decisions: test setup
Posted by Christian Kaltepoth <ch...@kaltepoth.de>.
@Mark:
But what about tests that should run in EE-light, EE-full and Servlet
but not in SE. Or tests that should run in EE-light/full but not in
Servlet and SE. I think this could get very complex.
Christian
2011/12/17 Mark Struberg <st...@yahoo.de>:
> We might solve this simply with a package naming rule. And then we configure surefire to just run the tests with the respective package?
>
> Not the most advanced solution, but will work.
>
>
> LieGrue,
> strub
>
>
>
> ----- Original Message -----
>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>> To: deltaspike-dev@incubator.apache.org
>> Cc:
>> Sent: Saturday, December 17, 2011 4:43 PM
>> Subject: Re: basic decisions: test setup
>>
>> @Mark:
>>
>> Actually @TargetsContainer is an existing feature of Arquillian. It
>> allows to build container specific deployments.
>>
>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>>
>> Unfortunately it won't fit our needs because the test completely fails
>> if there is no matching deployment for the container the test is
>> executed against. So I guess we have to build something new for this.
>>
>> I really like your suggestion regarding the four environments. It
>> absolutely make sense to declare environment constraints (like SE,
>> Servlet, EE, etc.) instead of listing concrete containers.
>>
>> Christian
>>
>>
>>
>> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>>> Hi Christian!
>>>
>>>
>>> @TargetContainer sounds good, but I'd rather make it a
>>>
>>>
>>> @TargetEnvironment()
>>>
>>> with values: SE, Servlet, EE-light, EE-full
>>>
>>> The excludes should be done in the single test integration modules, and the
>> goal should be to get down to zero excluded tests for all containers.
>>>
>>>
>>> @Jason: that mixture for the tests looks fine to me as well. Can you please
>> hack a sample structure in the DeltaSpikePlayground git repo?
>>>
>>>
>>> LieGrue,
>>> strub
>>>
>>>
>>>
>>>
>>>> ________________________________
>>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>>> To: deltaspike-dev@incubator.apache.org
>>>> Sent: Saturday, December 17, 2011 7:55 AM
>>>> Subject: Re: basic decisions: test setup
>>>>
>>>> I would like to add some "non-binding" comments here regarding
>> 3).
>>>>
>>>> Actually I really like the concept Jason described and which is
>>>> currently used in Seam3. Having worked with it a bit I can confirm
>>>> that it runs fine. And somehow it's the best of both worlds (in
>> regard
>>>> to 3a vs 3b).
>>>>
>>>> That the full integration test suite with all containers cannot run in
>>>> a single Maven invocation is IMHO not a real problem. We could write a
>>>> simple script for this.
>>>>
>>>> The only thing that could be optimized in this setup is the separation
>>>> of container dependent tests via package name. But perhaps this could
>>>> be solved by Arquillian? What I would really like to see is the
>>>> ability to configure a test so it is only executed in a specific
>>>> container (something similar to @TargetsContainer). This way tests
>>>> that require to be run in a full container could be executed only in
>>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>>>
>>>> Christian
>>>>
>>>>
>>>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>>>> Comments inline.
>>>>>
>>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg
>> <st...@yahoo.de> wrote:
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> Another round of discussion - this time about how to setup our
>> unit
>>>>>> testing and integration testing.
>>>>>>
>>>>>> I think it's pretty much clear that we will gonna use
>> Arquillian for all
>>>>>> our tests. That was also fundamental part of the incubator
>> proposal.
>>>>>> And instead of inventing yet another abstraction layer, I'd
>> favour to just
>>>>>> use Arquillian and contribute all things we need to this
>> project.
>>>>>> So here a formal vote
>>>>>>
>>>>>> 1.)
>>>>>>
>>>>>> for using Arquillian as test integration framework.
>>>>>> +1 from me
>>>>>>
>>>>>
>>>>> +1 from me
>>>>>
>>>>>
>>>>>>
>>>>>> 2.)
>>>>>>
>>>>>> What do we like to use from the unit tests itself? TestNG or
>> JUnit?
>>>>>> JUnit has better Arquillian integration and TestNG is better
>> for 'real
>>>>>> world' projects, because it allows to define test
>> dependencies.
>>>>>> So which one to take?
>>>>>> But we should definitely only use 1 of the two exclusively!
>>>>>>
>>>>>
>>>>> +1 for using one exclusively
>>>>>
>>>>> As for which, I'm also +0 I like both for different reasons,
>> but JUnit is
>>>>> certainly the easier of the two to use with Arquillian.
>>>>>
>>>>> Also, are we open to having other languages for tests or using
>> other
>>>>> testing structures such as spock (it works with Arquillian, BTW),
>> RSpec,
>>>>> scala, etc.?
>>>>>
>>>>>
>>>>>> 3.)
>>>>>> 'Integrated' tests vs 'Integration Tests'
>>>>>> We have 2 options to test our projects
>>>>>>
>>>>>> 3.a.) create a full unit test in each module (e.g.
>> deltaspike/core/impl)
>>>>>> and add a profile for each and every server (maybe we can trim
>> this down
>>>>>> with pom imports?).
>>>>>> Then run the build with one after each other:
>>>>>>
>>>>>> $> mvn clean test -Powb
>>>>>> $> mvn clean test -Pweld
>>>>>> $> mvn clean test -Powb-tc
>>>>>> $> mvn clean test -Pweld-tc
>>>>>> $> mvn clean test -Pgf31
>>>>>> $> mvn clean test -Pas7
>>>>>> $> mvn clean test -Ptomee
>>>>>>
>>>>>> This might pretty much blow up our poms...
>>>>>>
>>>>>>
>>>>>> 3.b) create a full unit test in each module (e.g.
>> deltaspike/core/impl)
>>>>>> and add only 2 profiles directly ('weld' and
>> 'owb' (probably resin later))
>>>>>> Then add a deltaspike/test/base/core with the very basic parent
>> stuff and
>>>>>> 1 module for each integrated container, e.g.
>>>>>> deltaspike/test/weld-tc
>>>>>> deltaspike/test/owb-tc
>>>>>> deltaspike/test/weld-jetty
>>>>>> deltaspike/test/owb-jetty
>>>>>> deltaspike/test/tomee
>>>>>> deltaspike/test/gf31
>>>>>> deltaspike/test/geronimo
>>>>>> deltaspike/test/websphere
>>>>>> deltaspike/test/as7
>>>>>> deltaspike/test/as6
>>>>>> etc.
>>>>>>
>>>>>> The initial setup costs are higher, but it would be pretty easy
>> to add new
>>>>>> containers that way.
>>>>>>
>>>>>> Which route shall we take?
>>>>>>
>>>>>
>>>>> In Seam 3 we have an approach which is a little bit of a mixture of
>> both.
>>>>> We have regular unit tests (with mockito, stubs, etc) under
>>>>> impl/src/test/... and then we have a separate module called
>> testsuite which
>>>>> contained all the integration tests. We used pom imports to get all
>> the
>>>>> containers setup correctly and then had different profiles which
>> pulled in
>>>>> the poms and also configured surefire for that server. Inside the
>> code
>>>>> structure we separated the servers by package: common, jbossas,
>> glassfish,
>>>>> etc. That way all of the tests were in one location, they were easy
>> to
>>>>> execute, the tests stayed DRY, and the IDE wasn't much of a
>> problem. The
>>>>> two downsides we had were 1) the extra config in the pom and 2) you
>> can't
>>>>> run the full integration suite in one maven execution. We spent
>> quite a bit
>>>>> of time examining options and trying some of them out. This was the
>> best we
>>>>> came up with, and also the one Aslak thought would work best as
>> well.
>>>>>
>>>>>
>>>>>> LieGrue,
>>>>>> strub
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jason Porter
>>>>> http://lightguard-jp.blogspot.com
>>>>> http://twitter.com/lightguardjp
>>>>>
>>>>> Software Engineer
>>>>> Open Source Advocate
>>>>> Author of Seam Catch - Next Generation Java Exception Handling
>>>>>
>>>>> PGP key id: 926CCFF5
>>>>> PGP key available at: keyserver.net, pgp.mit.edu
>>>>
>>>>
>>>>
>>>> --
>>>> Christian Kaltepoth
>>>> Blog: http://chkal.blogspot.com/
>>>> Twitter: http://twitter.com/chkal
>>>>
>>>>
>>>>
>>
>>
>>
>> --
>> Christian Kaltepoth
>> Blog: http://chkal.blogspot.com/
>> Twitter: http://twitter.com/chkal
>>
--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Mark Struberg <st...@yahoo.de>.
We might solve this simply with a package naming rule. And then we configure surefire to just run the tests with the respective package?
Not the most advanced solution, but will work.
LieGrue,
strub
----- Original Message -----
> From: Christian Kaltepoth <ch...@kaltepoth.de>
> To: deltaspike-dev@incubator.apache.org
> Cc:
> Sent: Saturday, December 17, 2011 4:43 PM
> Subject: Re: basic decisions: test setup
>
> @Mark:
>
> Actually @TargetsContainer is an existing feature of Arquillian. It
> allows to build container specific deployments.
>
> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>
> Unfortunately it won't fit our needs because the test completely fails
> if there is no matching deployment for the container the test is
> executed against. So I guess we have to build something new for this.
>
> I really like your suggestion regarding the four environments. It
> absolutely make sense to declare environment constraints (like SE,
> Servlet, EE, etc.) instead of listing concrete containers.
>
> Christian
>
>
>
> 2011/12/17 Mark Struberg <st...@yahoo.de>:
>> Hi Christian!
>>
>>
>> @TargetContainer sounds good, but I'd rather make it a
>>
>>
>> @TargetEnvironment()
>>
>> with values: SE, Servlet, EE-light, EE-full
>>
>> The excludes should be done in the single test integration modules, and the
> goal should be to get down to zero excluded tests for all containers.
>>
>>
>> @Jason: that mixture for the tests looks fine to me as well. Can you please
> hack a sample structure in the DeltaSpikePlayground git repo?
>>
>>
>> LieGrue,
>> strub
>>
>>
>>
>>
>>> ________________________________
>>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>> To: deltaspike-dev@incubator.apache.org
>>> Sent: Saturday, December 17, 2011 7:55 AM
>>> Subject: Re: basic decisions: test setup
>>>
>>> I would like to add some "non-binding" comments here regarding
> 3).
>>>
>>> Actually I really like the concept Jason described and which is
>>> currently used in Seam3. Having worked with it a bit I can confirm
>>> that it runs fine. And somehow it's the best of both worlds (in
> regard
>>> to 3a vs 3b).
>>>
>>> That the full integration test suite with all containers cannot run in
>>> a single Maven invocation is IMHO not a real problem. We could write a
>>> simple script for this.
>>>
>>> The only thing that could be optimized in this setup is the separation
>>> of container dependent tests via package name. But perhaps this could
>>> be solved by Arquillian? What I would really like to see is the
>>> ability to configure a test so it is only executed in a specific
>>> container (something similar to @TargetsContainer). This way tests
>>> that require to be run in a full container could be executed only in
>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>>
>>> Christian
>>>
>>>
>>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>>> Comments inline.
>>>>
>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg
> <st...@yahoo.de> wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> Another round of discussion - this time about how to setup our
> unit
>>>>> testing and integration testing.
>>>>>
>>>>> I think it's pretty much clear that we will gonna use
> Arquillian for all
>>>>> our tests. That was also fundamental part of the incubator
> proposal.
>>>>> And instead of inventing yet another abstraction layer, I'd
> favour to just
>>>>> use Arquillian and contribute all things we need to this
> project.
>>>>> So here a formal vote
>>>>>
>>>>> 1.)
>>>>>
>>>>> for using Arquillian as test integration framework.
>>>>> +1 from me
>>>>>
>>>>
>>>> +1 from me
>>>>
>>>>
>>>>>
>>>>> 2.)
>>>>>
>>>>> What do we like to use from the unit tests itself? TestNG or
> JUnit?
>>>>> JUnit has better Arquillian integration and TestNG is better
> for 'real
>>>>> world' projects, because it allows to define test
> dependencies.
>>>>> So which one to take?
>>>>> But we should definitely only use 1 of the two exclusively!
>>>>>
>>>>
>>>> +1 for using one exclusively
>>>>
>>>> As for which, I'm also +0 I like both for different reasons,
> but JUnit is
>>>> certainly the easier of the two to use with Arquillian.
>>>>
>>>> Also, are we open to having other languages for tests or using
> other
>>>> testing structures such as spock (it works with Arquillian, BTW),
> RSpec,
>>>> scala, etc.?
>>>>
>>>>
>>>>> 3.)
>>>>> 'Integrated' tests vs 'Integration Tests'
>>>>> We have 2 options to test our projects
>>>>>
>>>>> 3.a.) create a full unit test in each module (e.g.
> deltaspike/core/impl)
>>>>> and add a profile for each and every server (maybe we can trim
> this down
>>>>> with pom imports?).
>>>>> Then run the build with one after each other:
>>>>>
>>>>> $> mvn clean test -Powb
>>>>> $> mvn clean test -Pweld
>>>>> $> mvn clean test -Powb-tc
>>>>> $> mvn clean test -Pweld-tc
>>>>> $> mvn clean test -Pgf31
>>>>> $> mvn clean test -Pas7
>>>>> $> mvn clean test -Ptomee
>>>>>
>>>>> This might pretty much blow up our poms...
>>>>>
>>>>>
>>>>> 3.b) create a full unit test in each module (e.g.
> deltaspike/core/impl)
>>>>> and add only 2 profiles directly ('weld' and
> 'owb' (probably resin later))
>>>>> Then add a deltaspike/test/base/core with the very basic parent
> stuff and
>>>>> 1 module for each integrated container, e.g.
>>>>> deltaspike/test/weld-tc
>>>>> deltaspike/test/owb-tc
>>>>> deltaspike/test/weld-jetty
>>>>> deltaspike/test/owb-jetty
>>>>> deltaspike/test/tomee
>>>>> deltaspike/test/gf31
>>>>> deltaspike/test/geronimo
>>>>> deltaspike/test/websphere
>>>>> deltaspike/test/as7
>>>>> deltaspike/test/as6
>>>>> etc.
>>>>>
>>>>> The initial setup costs are higher, but it would be pretty easy
> to add new
>>>>> containers that way.
>>>>>
>>>>> Which route shall we take?
>>>>>
>>>>
>>>> In Seam 3 we have an approach which is a little bit of a mixture of
> both.
>>>> We have regular unit tests (with mockito, stubs, etc) under
>>>> impl/src/test/... and then we have a separate module called
> testsuite which
>>>> contained all the integration tests. We used pom imports to get all
> the
>>>> containers setup correctly and then had different profiles which
> pulled in
>>>> the poms and also configured surefire for that server. Inside the
> code
>>>> structure we separated the servers by package: common, jbossas,
> glassfish,
>>>> etc. That way all of the tests were in one location, they were easy
> to
>>>> execute, the tests stayed DRY, and the IDE wasn't much of a
> problem. The
>>>> two downsides we had were 1) the extra config in the pom and 2) you
> can't
>>>> run the full integration suite in one maven execution. We spent
> quite a bit
>>>> of time examining options and trying some of them out. This was the
> best we
>>>> came up with, and also the one Aslak thought would work best as
> well.
>>>>
>>>>
>>>>> LieGrue,
>>>>> strub
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jason Porter
>>>> http://lightguard-jp.blogspot.com
>>>> http://twitter.com/lightguardjp
>>>>
>>>> Software Engineer
>>>> Open Source Advocate
>>>> Author of Seam Catch - Next Generation Java Exception Handling
>>>>
>>>> PGP key id: 926CCFF5
>>>> PGP key available at: keyserver.net, pgp.mit.edu
>>>
>>>
>>>
>>> --
>>> Christian Kaltepoth
>>> Blog: http://chkal.blogspot.com/
>>> Twitter: http://twitter.com/chkal
>>>
>>>
>>>
>
>
>
> --
> Christian Kaltepoth
> Blog: http://chkal.blogspot.com/
> Twitter: http://twitter.com/chkal
>
Re: basic decisions: test setup
Posted by Christian Kaltepoth <ch...@kaltepoth.de>.
@Mark:
Actually @TargetsContainer is an existing feature of Arquillian. It
allows to build container specific deployments.
https://docs.jboss.org/author/display/ARQ/Multiple+Containers
Unfortunately it won't fit our needs because the test completely fails
if there is no matching deployment for the container the test is
executed against. So I guess we have to build something new for this.
I really like your suggestion regarding the four environments. It
absolutely make sense to declare environment constraints (like SE,
Servlet, EE, etc.) instead of listing concrete containers.
Christian
2011/12/17 Mark Struberg <st...@yahoo.de>:
> Hi Christian!
>
>
> @TargetContainer sounds good, but I'd rather make it a
>
>
> @TargetEnvironment()
>
> with values: SE, Servlet, EE-light, EE-full
>
> The excludes should be done in the single test integration modules, and the goal should be to get down to zero excluded tests for all containers.
>
>
> @Jason: that mixture for the tests looks fine to me as well. Can you please hack a sample structure in the DeltaSpikePlayground git repo?
>
>
> LieGrue,
> strub
>
>
>
>
>>________________________________
>> From: Christian Kaltepoth <ch...@kaltepoth.de>
>>To: deltaspike-dev@incubator.apache.org
>>Sent: Saturday, December 17, 2011 7:55 AM
>>Subject: Re: basic decisions: test setup
>>
>>I would like to add some "non-binding" comments here regarding 3).
>>
>>Actually I really like the concept Jason described and which is
>>currently used in Seam3. Having worked with it a bit I can confirm
>>that it runs fine. And somehow it's the best of both worlds (in regard
>>to 3a vs 3b).
>>
>>That the full integration test suite with all containers cannot run in
>>a single Maven invocation is IMHO not a real problem. We could write a
>>simple script for this.
>>
>>The only thing that could be optimized in this setup is the separation
>>of container dependent tests via package name. But perhaps this could
>>be solved by Arquillian? What I would really like to see is the
>>ability to configure a test so it is only executed in a specific
>>container (something similar to @TargetsContainer). This way tests
>>that require to be run in a full container could be executed only in
>>JBoss and Glassfish but would be skipped in embedded Weld.
>>
>>Christian
>>
>>
>>2011/12/16 Jason Porter <li...@gmail.com>:
>>> Comments inline.
>>>
>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>>>
>>>> Hi!
>>>>
>>>> Another round of discussion - this time about how to setup our unit
>>>> testing and integration testing.
>>>>
>>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>>> our tests. That was also fundamental part of the incubator proposal.
>>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>>> use Arquillian and contribute all things we need to this project.
>>>> So here a formal vote
>>>>
>>>> 1.)
>>>>
>>>> for using Arquillian as test integration framework.
>>>> +1 from me
>>>>
>>>
>>> +1 from me
>>>
>>>
>>>>
>>>> 2.)
>>>>
>>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>>> world' projects, because it allows to define test dependencies.
>>>> So which one to take?
>>>> But we should definitely only use 1 of the two exclusively!
>>>>
>>>
>>> +1 for using one exclusively
>>>
>>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>>> certainly the easier of the two to use with Arquillian.
>>>
>>> Also, are we open to having other languages for tests or using other
>>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>>> scala, etc.?
>>>
>>>
>>>> 3.)
>>>> 'Integrated' tests vs 'Integration Tests'
>>>> We have 2 options to test our projects
>>>>
>>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>>> and add a profile for each and every server (maybe we can trim this down
>>>> with pom imports?).
>>>> Then run the build with one after each other:
>>>>
>>>> $> mvn clean test -Powb
>>>> $> mvn clean test -Pweld
>>>> $> mvn clean test -Powb-tc
>>>> $> mvn clean test -Pweld-tc
>>>> $> mvn clean test -Pgf31
>>>> $> mvn clean test -Pas7
>>>> $> mvn clean test -Ptomee
>>>>
>>>> This might pretty much blow up our poms...
>>>>
>>>>
>>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>>> 1 module for each integrated container, e.g.
>>>> deltaspike/test/weld-tc
>>>> deltaspike/test/owb-tc
>>>> deltaspike/test/weld-jetty
>>>> deltaspike/test/owb-jetty
>>>> deltaspike/test/tomee
>>>> deltaspike/test/gf31
>>>> deltaspike/test/geronimo
>>>> deltaspike/test/websphere
>>>> deltaspike/test/as7
>>>> deltaspike/test/as6
>>>> etc.
>>>>
>>>> The initial setup costs are higher, but it would be pretty easy to add new
>>>> containers that way.
>>>>
>>>> Which route shall we take?
>>>>
>>>
>>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>>> We have regular unit tests (with mockito, stubs, etc) under
>>> impl/src/test/... and then we have a separate module called testsuite which
>>> contained all the integration tests. We used pom imports to get all the
>>> containers setup correctly and then had different profiles which pulled in
>>> the poms and also configured surefire for that server. Inside the code
>>> structure we separated the servers by package: common, jbossas, glassfish,
>>> etc. That way all of the tests were in one location, they were easy to
>>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>>> two downsides we had were 1) the extra config in the pom and 2) you can't
>>> run the full integration suite in one maven execution. We spent quite a bit
>>> of time examining options and trying some of them out. This was the best we
>>> came up with, and also the one Aslak thought would work best as well.
>>>
>>>
>>>> LieGrue,
>>>> strub
>>>>
>>>>
>>>
>>>
>>> --
>>> Jason Porter
>>> http://lightguard-jp.blogspot.com
>>> http://twitter.com/lightguardjp
>>>
>>> Software Engineer
>>> Open Source Advocate
>>> Author of Seam Catch - Next Generation Java Exception Handling
>>>
>>> PGP key id: 926CCFF5
>>> PGP key available at: keyserver.net, pgp.mit.edu
>>
>>
>>
>>--
>>Christian Kaltepoth
>>Blog: http://chkal.blogspot.com/
>>Twitter: http://twitter.com/chkal
>>
>>
>>
--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Jason Porter <li...@gmail.com>.
Sure can do. Today is pretty booked for me, I may be able to do it tonight
though. If not, I'll do it on Monday.
On Sat, Dec 17, 2011 at 03:05, Mark Struberg <st...@yahoo.de> wrote:
> Hi Christian!
>
>
> @TargetContainer sounds good, but I'd rather make it a
>
>
> @TargetEnvironment()
>
> with values: SE, Servlet, EE-light, EE-full
>
> The excludes should be done in the single test integration modules, and
> the goal should be to get down to zero excluded tests for all containers.
>
>
> @Jason: that mixture for the tests looks fine to me as well. Can you
> please hack a sample structure in the DeltaSpikePlayground git repo?
>
>
> LieGrue,
> strub
>
>
>
>
> >________________________________
> > From: Christian Kaltepoth <ch...@kaltepoth.de>
> >To: deltaspike-dev@incubator.apache.org
> >Sent: Saturday, December 17, 2011 7:55 AM
> >Subject: Re: basic decisions: test setup
> >
> >I would like to add some "non-binding" comments here regarding 3).
> >
> >Actually I really like the concept Jason described and which is
> >currently used in Seam3. Having worked with it a bit I can confirm
> >that it runs fine. And somehow it's the best of both worlds (in regard
> >to 3a vs 3b).
> >
> >That the full integration test suite with all containers cannot run in
> >a single Maven invocation is IMHO not a real problem. We could write a
> >simple script for this.
> >
> >The only thing that could be optimized in this setup is the separation
> >of container dependent tests via package name. But perhaps this could
> >be solved by Arquillian? What I would really like to see is the
> >ability to configure a test so it is only executed in a specific
> >container (something similar to @TargetsContainer). This way tests
> >that require to be run in a full container could be executed only in
> >JBoss and Glassfish but would be skipped in embedded Weld.
> >
> >Christian
> >
> >
> >2011/12/16 Jason Porter <li...@gmail.com>:
> >> Comments inline.
> >>
> >> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
> >>
> >>> Hi!
> >>>
> >>> Another round of discussion - this time about how to setup our unit
> >>> testing and integration testing.
> >>>
> >>> I think it's pretty much clear that we will gonna use Arquillian for
> all
> >>> our tests. That was also fundamental part of the incubator proposal.
> >>> And instead of inventing yet another abstraction layer, I'd favour to
> just
> >>> use Arquillian and contribute all things we need to this project.
> >>> So here a formal vote
> >>>
> >>> 1.)
> >>>
> >>> for using Arquillian as test integration framework.
> >>> +1 from me
> >>>
> >>
> >> +1 from me
> >>
> >>
> >>>
> >>> 2.)
> >>>
> >>> What do we like to use from the unit tests itself? TestNG or JUnit?
> >>> JUnit has better Arquillian integration and TestNG is better for 'real
> >>> world' projects, because it allows to define test dependencies.
> >>> So which one to take?
> >>> But we should definitely only use 1 of the two exclusively!
> >>>
> >>
> >> +1 for using one exclusively
> >>
> >> As for which, I'm also +0 I like both for different reasons, but JUnit
> is
> >> certainly the easier of the two to use with Arquillian.
> >>
> >> Also, are we open to having other languages for tests or using other
> >> testing structures such as spock (it works with Arquillian, BTW), RSpec,
> >> scala, etc.?
> >>
> >>
> >>> 3.)
> >>> 'Integrated' tests vs 'Integration Tests'
> >>> We have 2 options to test our projects
> >>>
> >>> 3.a.) create a full unit test in each module (e.g.
> deltaspike/core/impl)
> >>> and add a profile for each and every server (maybe we can trim this
> down
> >>> with pom imports?).
> >>> Then run the build with one after each other:
> >>>
> >>> $> mvn clean test -Powb
> >>> $> mvn clean test -Pweld
> >>> $> mvn clean test -Powb-tc
> >>> $> mvn clean test -Pweld-tc
> >>> $> mvn clean test -Pgf31
> >>> $> mvn clean test -Pas7
> >>> $> mvn clean test -Ptomee
> >>>
> >>> This might pretty much blow up our poms...
> >>>
> >>>
> >>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
> >>> and add only 2 profiles directly ('weld' and 'owb' (probably resin
> later))
> >>> Then add a deltaspike/test/base/core with the very basic parent stuff
> and
> >>> 1 module for each integrated container, e.g.
> >>> deltaspike/test/weld-tc
> >>> deltaspike/test/owb-tc
> >>> deltaspike/test/weld-jetty
> >>> deltaspike/test/owb-jetty
> >>> deltaspike/test/tomee
> >>> deltaspike/test/gf31
> >>> deltaspike/test/geronimo
> >>> deltaspike/test/websphere
> >>> deltaspike/test/as7
> >>> deltaspike/test/as6
> >>> etc.
> >>>
> >>> The initial setup costs are higher, but it would be pretty easy to add
> new
> >>> containers that way.
> >>>
> >>> Which route shall we take?
> >>>
> >>
> >> In Seam 3 we have an approach which is a little bit of a mixture of
> both.
> >> We have regular unit tests (with mockito, stubs, etc) under
> >> impl/src/test/... and then we have a separate module called testsuite
> which
> >> contained all the integration tests. We used pom imports to get all the
> >> containers setup correctly and then had different profiles which pulled
> in
> >> the poms and also configured surefire for that server. Inside the code
> >> structure we separated the servers by package: common, jbossas,
> glassfish,
> >> etc. That way all of the tests were in one location, they were easy to
> >> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
> >> two downsides we had were 1) the extra config in the pom and 2) you
> can't
> >> run the full integration suite in one maven execution. We spent quite a
> bit
> >> of time examining options and trying some of them out. This was the
> best we
> >> came up with, and also the one Aslak thought would work best as well.
> >>
> >>
> >>> LieGrue,
> >>> strub
> >>>
> >>>
> >>
> >>
> >> --
> >> Jason Porter
> >> http://lightguard-jp.blogspot.com
> >> http://twitter.com/lightguardjp
> >>
> >> Software Engineer
> >> Open Source Advocate
> >> Author of Seam Catch - Next Generation Java Exception Handling
> >>
> >> PGP key id: 926CCFF5
> >> PGP key available at: keyserver.net, pgp.mit.edu
> >
> >
> >
> >--
> >Christian Kaltepoth
> >Blog: http://chkal.blogspot.com/
> >Twitter: http://twitter.com/chkal
> >
> >
> >
>
--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net, pgp.mit.edu
Re: basic decisions: test setup
Posted by Mark Struberg <st...@yahoo.de>.
Hi Christian!
@TargetContainer sounds good, but I'd rather make it a
@TargetEnvironment()
with values: SE, Servlet, EE-light, EE-full
The excludes should be done in the single test integration modules, and the goal should be to get down to zero excluded tests for all containers.
@Jason: that mixture for the tests looks fine to me as well. Can you please hack a sample structure in the DeltaSpikePlayground git repo?
LieGrue,
strub
>________________________________
> From: Christian Kaltepoth <ch...@kaltepoth.de>
>To: deltaspike-dev@incubator.apache.org
>Sent: Saturday, December 17, 2011 7:55 AM
>Subject: Re: basic decisions: test setup
>
>I would like to add some "non-binding" comments here regarding 3).
>
>Actually I really like the concept Jason described and which is
>currently used in Seam3. Having worked with it a bit I can confirm
>that it runs fine. And somehow it's the best of both worlds (in regard
>to 3a vs 3b).
>
>That the full integration test suite with all containers cannot run in
>a single Maven invocation is IMHO not a real problem. We could write a
>simple script for this.
>
>The only thing that could be optimized in this setup is the separation
>of container dependent tests via package name. But perhaps this could
>be solved by Arquillian? What I would really like to see is the
>ability to configure a test so it is only executed in a specific
>container (something similar to @TargetsContainer). This way tests
>that require to be run in a full container could be executed only in
>JBoss and Glassfish but would be skipped in embedded Weld.
>
>Christian
>
>
>2011/12/16 Jason Porter <li...@gmail.com>:
>> Comments inline.
>>
>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>>
>>> Hi!
>>>
>>> Another round of discussion - this time about how to setup our unit
>>> testing and integration testing.
>>>
>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>> our tests. That was also fundamental part of the incubator proposal.
>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>> use Arquillian and contribute all things we need to this project.
>>> So here a formal vote
>>>
>>> 1.)
>>>
>>> for using Arquillian as test integration framework.
>>> +1 from me
>>>
>>
>> +1 from me
>>
>>
>>>
>>> 2.)
>>>
>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>> world' projects, because it allows to define test dependencies.
>>> So which one to take?
>>> But we should definitely only use 1 of the two exclusively!
>>>
>>
>> +1 for using one exclusively
>>
>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>> certainly the easier of the two to use with Arquillian.
>>
>> Also, are we open to having other languages for tests or using other
>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>> scala, etc.?
>>
>>
>>> 3.)
>>> 'Integrated' tests vs 'Integration Tests'
>>> We have 2 options to test our projects
>>>
>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add a profile for each and every server (maybe we can trim this down
>>> with pom imports?).
>>> Then run the build with one after each other:
>>>
>>> $> mvn clean test -Powb
>>> $> mvn clean test -Pweld
>>> $> mvn clean test -Powb-tc
>>> $> mvn clean test -Pweld-tc
>>> $> mvn clean test -Pgf31
>>> $> mvn clean test -Pas7
>>> $> mvn clean test -Ptomee
>>>
>>> This might pretty much blow up our poms...
>>>
>>>
>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>> 1 module for each integrated container, e.g.
>>> deltaspike/test/weld-tc
>>> deltaspike/test/owb-tc
>>> deltaspike/test/weld-jetty
>>> deltaspike/test/owb-jetty
>>> deltaspike/test/tomee
>>> deltaspike/test/gf31
>>> deltaspike/test/geronimo
>>> deltaspike/test/websphere
>>> deltaspike/test/as7
>>> deltaspike/test/as6
>>> etc.
>>>
>>> The initial setup costs are higher, but it would be pretty easy to add new
>>> containers that way.
>>>
>>> Which route shall we take?
>>>
>>
>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>> We have regular unit tests (with mockito, stubs, etc) under
>> impl/src/test/... and then we have a separate module called testsuite which
>> contained all the integration tests. We used pom imports to get all the
>> containers setup correctly and then had different profiles which pulled in
>> the poms and also configured surefire for that server. Inside the code
>> structure we separated the servers by package: common, jbossas, glassfish,
>> etc. That way all of the tests were in one location, they were easy to
>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>> two downsides we had were 1) the extra config in the pom and 2) you can't
>> run the full integration suite in one maven execution. We spent quite a bit
>> of time examining options and trying some of them out. This was the best we
>> came up with, and also the one Aslak thought would work best as well.
>>
>>
>>> LieGrue,
>>> strub
>>>
>>>
>>
>>
>> --
>> Jason Porter
>> http://lightguard-jp.blogspot.com
>> http://twitter.com/lightguardjp
>>
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next Generation Java Exception Handling
>>
>> PGP key id: 926CCFF5
>> PGP key available at: keyserver.net, pgp.mit.edu
>
>
>
>--
>Christian Kaltepoth
>Blog: http://chkal.blogspot.com/
>Twitter: http://twitter.com/chkal
>
>
>
Re: basic decisions: test setup
Posted by Christian Kaltepoth <ch...@kaltepoth.de>.
OK, thanks! I just wanted to be formally correct! :)
Christian
2011/12/19 Matthias Wessendorf <ma...@apache.org>:
> On Sat, Dec 17, 2011 at 7:55 AM, Christian Kaltepoth
> <ch...@kaltepoth.de> wrote:
>> I would like to add some "non-binding" comments here regarding 3).
>
>
> Hey Chris,
>
> don't put too much weight in the binding VS non-binding - this shindig
> is all about founding and building a (strong) developers,
> community.
>
> :-)
>
>> Actually I really like the concept Jason described and which is
>> currently used in Seam3. Having worked with it a bit I can confirm
>> that it runs fine. And somehow it's the best of both worlds (in regard
>> to 3a vs 3b).
>>
>> That the full integration test suite with all containers cannot run in
>> a single Maven invocation is IMHO not a real problem. We could write a
>> simple script for this.
>>
>> The only thing that could be optimized in this setup is the separation
>> of container dependent tests via package name. But perhaps this could
>> be solved by Arquillian? What I would really like to see is the
>> ability to configure a test so it is only executed in a specific
>> container (something similar to @TargetsContainer). This way tests
>> that require to be run in a full container could be executed only in
>> JBoss and Glassfish but would be skipped in embedded Weld.
>>
>> Christian
>>
>>
>> 2011/12/16 Jason Porter <li...@gmail.com>:
>>> Comments inline.
>>>
>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>>>
>>>> Hi!
>>>>
>>>> Another round of discussion - this time about how to setup our unit
>>>> testing and integration testing.
>>>>
>>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>>> our tests. That was also fundamental part of the incubator proposal.
>>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>>> use Arquillian and contribute all things we need to this project.
>>>> So here a formal vote
>>>>
>>>> 1.)
>>>>
>>>> for using Arquillian as test integration framework.
>>>> +1 from me
>>>>
>>>
>>> +1 from me
>>>
>>>
>>>>
>>>> 2.)
>>>>
>>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>>> world' projects, because it allows to define test dependencies.
>>>> So which one to take?
>>>> But we should definitely only use 1 of the two exclusively!
>>>>
>>>
>>> +1 for using one exclusively
>>>
>>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>>> certainly the easier of the two to use with Arquillian.
>>>
>>> Also, are we open to having other languages for tests or using other
>>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>>> scala, etc.?
>>>
>>>
>>>> 3.)
>>>> 'Integrated' tests vs 'Integration Tests'
>>>> We have 2 options to test our projects
>>>>
>>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>>> and add a profile for each and every server (maybe we can trim this down
>>>> with pom imports?).
>>>> Then run the build with one after each other:
>>>>
>>>> $> mvn clean test -Powb
>>>> $> mvn clean test -Pweld
>>>> $> mvn clean test -Powb-tc
>>>> $> mvn clean test -Pweld-tc
>>>> $> mvn clean test -Pgf31
>>>> $> mvn clean test -Pas7
>>>> $> mvn clean test -Ptomee
>>>>
>>>> This might pretty much blow up our poms...
>>>>
>>>>
>>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>>> 1 module for each integrated container, e.g.
>>>> deltaspike/test/weld-tc
>>>> deltaspike/test/owb-tc
>>>> deltaspike/test/weld-jetty
>>>> deltaspike/test/owb-jetty
>>>> deltaspike/test/tomee
>>>> deltaspike/test/gf31
>>>> deltaspike/test/geronimo
>>>> deltaspike/test/websphere
>>>> deltaspike/test/as7
>>>> deltaspike/test/as6
>>>> etc.
>>>>
>>>> The initial setup costs are higher, but it would be pretty easy to add new
>>>> containers that way.
>>>>
>>>> Which route shall we take?
>>>>
>>>
>>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>>> We have regular unit tests (with mockito, stubs, etc) under
>>> impl/src/test/... and then we have a separate module called testsuite which
>>> contained all the integration tests. We used pom imports to get all the
>>> containers setup correctly and then had different profiles which pulled in
>>> the poms and also configured surefire for that server. Inside the code
>>> structure we separated the servers by package: common, jbossas, glassfish,
>>> etc. That way all of the tests were in one location, they were easy to
>>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>>> two downsides we had were 1) the extra config in the pom and 2) you can't
>>> run the full integration suite in one maven execution. We spent quite a bit
>>> of time examining options and trying some of them out. This was the best we
>>> came up with, and also the one Aslak thought would work best as well.
>>>
>>>
>>>> LieGrue,
>>>> strub
>>>>
>>>>
>>>
>>>
>>> --
>>> Jason Porter
>>> http://lightguard-jp.blogspot.com
>>> http://twitter.com/lightguardjp
>>>
>>> Software Engineer
>>> Open Source Advocate
>>> Author of Seam Catch - Next Generation Java Exception Handling
>>>
>>> PGP key id: 926CCFF5
>>> PGP key available at: keyserver.net, pgp.mit.edu
>>
>>
>>
>> --
>> Christian Kaltepoth
>> Blog: http://chkal.blogspot.com/
>> Twitter: http://twitter.com/chkal
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Matthias Wessendorf <ma...@apache.org>.
On Sat, Dec 17, 2011 at 7:55 AM, Christian Kaltepoth
<ch...@kaltepoth.de> wrote:
> I would like to add some "non-binding" comments here regarding 3).
Hey Chris,
don't put too much weight in the binding VS non-binding - this shindig
is all about founding and building a (strong) developers,
community.
:-)
> Actually I really like the concept Jason described and which is
> currently used in Seam3. Having worked with it a bit I can confirm
> that it runs fine. And somehow it's the best of both worlds (in regard
> to 3a vs 3b).
>
> That the full integration test suite with all containers cannot run in
> a single Maven invocation is IMHO not a real problem. We could write a
> simple script for this.
>
> The only thing that could be optimized in this setup is the separation
> of container dependent tests via package name. But perhaps this could
> be solved by Arquillian? What I would really like to see is the
> ability to configure a test so it is only executed in a specific
> container (something similar to @TargetsContainer). This way tests
> that require to be run in a full container could be executed only in
> JBoss and Glassfish but would be skipped in embedded Weld.
>
> Christian
>
>
> 2011/12/16 Jason Porter <li...@gmail.com>:
>> Comments inline.
>>
>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>>
>>> Hi!
>>>
>>> Another round of discussion - this time about how to setup our unit
>>> testing and integration testing.
>>>
>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>> our tests. That was also fundamental part of the incubator proposal.
>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>> use Arquillian and contribute all things we need to this project.
>>> So here a formal vote
>>>
>>> 1.)
>>>
>>> for using Arquillian as test integration framework.
>>> +1 from me
>>>
>>
>> +1 from me
>>
>>
>>>
>>> 2.)
>>>
>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>> world' projects, because it allows to define test dependencies.
>>> So which one to take?
>>> But we should definitely only use 1 of the two exclusively!
>>>
>>
>> +1 for using one exclusively
>>
>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>> certainly the easier of the two to use with Arquillian.
>>
>> Also, are we open to having other languages for tests or using other
>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>> scala, etc.?
>>
>>
>>> 3.)
>>> 'Integrated' tests vs 'Integration Tests'
>>> We have 2 options to test our projects
>>>
>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add a profile for each and every server (maybe we can trim this down
>>> with pom imports?).
>>> Then run the build with one after each other:
>>>
>>> $> mvn clean test -Powb
>>> $> mvn clean test -Pweld
>>> $> mvn clean test -Powb-tc
>>> $> mvn clean test -Pweld-tc
>>> $> mvn clean test -Pgf31
>>> $> mvn clean test -Pas7
>>> $> mvn clean test -Ptomee
>>>
>>> This might pretty much blow up our poms...
>>>
>>>
>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>> 1 module for each integrated container, e.g.
>>> deltaspike/test/weld-tc
>>> deltaspike/test/owb-tc
>>> deltaspike/test/weld-jetty
>>> deltaspike/test/owb-jetty
>>> deltaspike/test/tomee
>>> deltaspike/test/gf31
>>> deltaspike/test/geronimo
>>> deltaspike/test/websphere
>>> deltaspike/test/as7
>>> deltaspike/test/as6
>>> etc.
>>>
>>> The initial setup costs are higher, but it would be pretty easy to add new
>>> containers that way.
>>>
>>> Which route shall we take?
>>>
>>
>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>> We have regular unit tests (with mockito, stubs, etc) under
>> impl/src/test/... and then we have a separate module called testsuite which
>> contained all the integration tests. We used pom imports to get all the
>> containers setup correctly and then had different profiles which pulled in
>> the poms and also configured surefire for that server. Inside the code
>> structure we separated the servers by package: common, jbossas, glassfish,
>> etc. That way all of the tests were in one location, they were easy to
>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>> two downsides we had were 1) the extra config in the pom and 2) you can't
>> run the full integration suite in one maven execution. We spent quite a bit
>> of time examining options and trying some of them out. This was the best we
>> came up with, and also the one Aslak thought would work best as well.
>>
>>
>>> LieGrue,
>>> strub
>>>
>>>
>>
>>
>> --
>> Jason Porter
>> http://lightguard-jp.blogspot.com
>> http://twitter.com/lightguardjp
>>
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next Generation Java Exception Handling
>>
>> PGP key id: 926CCFF5
>> PGP key available at: keyserver.net, pgp.mit.edu
>
>
>
> --
> Christian Kaltepoth
> Blog: http://chkal.blogspot.com/
> Twitter: http://twitter.com/chkal
--
Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
Re: basic decisions: test setup
Posted by Christian Kaltepoth <ch...@kaltepoth.de>.
I would like to add some "non-binding" comments here regarding 3).
Actually I really like the concept Jason described and which is
currently used in Seam3. Having worked with it a bit I can confirm
that it runs fine. And somehow it's the best of both worlds (in regard
to 3a vs 3b).
That the full integration test suite with all containers cannot run in
a single Maven invocation is IMHO not a real problem. We could write a
simple script for this.
The only thing that could be optimized in this setup is the separation
of container dependent tests via package name. But perhaps this could
be solved by Arquillian? What I would really like to see is the
ability to configure a test so it is only executed in a specific
container (something similar to @TargetsContainer). This way tests
that require to be run in a full container could be executed only in
JBoss and Glassfish but would be skipped in embedded Weld.
Christian
2011/12/16 Jason Porter <li...@gmail.com>:
> Comments inline.
>
> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
>
>> Hi!
>>
>> Another round of discussion - this time about how to setup our unit
>> testing and integration testing.
>>
>> I think it's pretty much clear that we will gonna use Arquillian for all
>> our tests. That was also fundamental part of the incubator proposal.
>> And instead of inventing yet another abstraction layer, I'd favour to just
>> use Arquillian and contribute all things we need to this project.
>> So here a formal vote
>>
>> 1.)
>>
>> for using Arquillian as test integration framework.
>> +1 from me
>>
>
> +1 from me
>
>
>>
>> 2.)
>>
>> What do we like to use from the unit tests itself? TestNG or JUnit?
>> JUnit has better Arquillian integration and TestNG is better for 'real
>> world' projects, because it allows to define test dependencies.
>> So which one to take?
>> But we should definitely only use 1 of the two exclusively!
>>
>
> +1 for using one exclusively
>
> As for which, I'm also +0 I like both for different reasons, but JUnit is
> certainly the easier of the two to use with Arquillian.
>
> Also, are we open to having other languages for tests or using other
> testing structures such as spock (it works with Arquillian, BTW), RSpec,
> scala, etc.?
>
>
>> 3.)
>> 'Integrated' tests vs 'Integration Tests'
>> We have 2 options to test our projects
>>
>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>> and add a profile for each and every server (maybe we can trim this down
>> with pom imports?).
>> Then run the build with one after each other:
>>
>> $> mvn clean test -Powb
>> $> mvn clean test -Pweld
>> $> mvn clean test -Powb-tc
>> $> mvn clean test -Pweld-tc
>> $> mvn clean test -Pgf31
>> $> mvn clean test -Pas7
>> $> mvn clean test -Ptomee
>>
>> This might pretty much blow up our poms...
>>
>>
>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>> 1 module for each integrated container, e.g.
>> deltaspike/test/weld-tc
>> deltaspike/test/owb-tc
>> deltaspike/test/weld-jetty
>> deltaspike/test/owb-jetty
>> deltaspike/test/tomee
>> deltaspike/test/gf31
>> deltaspike/test/geronimo
>> deltaspike/test/websphere
>> deltaspike/test/as7
>> deltaspike/test/as6
>> etc.
>>
>> The initial setup costs are higher, but it would be pretty easy to add new
>> containers that way.
>>
>> Which route shall we take?
>>
>
> In Seam 3 we have an approach which is a little bit of a mixture of both.
> We have regular unit tests (with mockito, stubs, etc) under
> impl/src/test/... and then we have a separate module called testsuite which
> contained all the integration tests. We used pom imports to get all the
> containers setup correctly and then had different profiles which pulled in
> the poms and also configured surefire for that server. Inside the code
> structure we separated the servers by package: common, jbossas, glassfish,
> etc. That way all of the tests were in one location, they were easy to
> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
> two downsides we had were 1) the extra config in the pom and 2) you can't
> run the full integration suite in one maven execution. We spent quite a bit
> of time examining options and trying some of them out. This was the best we
> came up with, and also the one Aslak thought would work best as well.
>
>
>> LieGrue,
>> strub
>>
>>
>
>
> --
> Jason Porter
> http://lightguard-jp.blogspot.com
> http://twitter.com/lightguardjp
>
> Software Engineer
> Open Source Advocate
> Author of Seam Catch - Next Generation Java Exception Handling
>
> PGP key id: 926CCFF5
> PGP key available at: keyserver.net, pgp.mit.edu
--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by Jason Porter <li...@gmail.com>.
Comments inline.
On Fri, Dec 16, 2011 at 01:38, Mark Struberg <st...@yahoo.de> wrote:
> Hi!
>
> Another round of discussion - this time about how to setup our unit
> testing and integration testing.
>
> I think it's pretty much clear that we will gonna use Arquillian for all
> our tests. That was also fundamental part of the incubator proposal.
> And instead of inventing yet another abstraction layer, I'd favour to just
> use Arquillian and contribute all things we need to this project.
> So here a formal vote
>
> 1.)
>
> for using Arquillian as test integration framework.
> +1 from me
>
+1 from me
>
> 2.)
>
> What do we like to use from the unit tests itself? TestNG or JUnit?
> JUnit has better Arquillian integration and TestNG is better for 'real
> world' projects, because it allows to define test dependencies.
> So which one to take?
> But we should definitely only use 1 of the two exclusively!
>
+1 for using one exclusively
As for which, I'm also +0 I like both for different reasons, but JUnit is
certainly the easier of the two to use with Arquillian.
Also, are we open to having other languages for tests or using other
testing structures such as spock (it works with Arquillian, BTW), RSpec,
scala, etc.?
> 3.)
> 'Integrated' tests vs 'Integration Tests'
> We have 2 options to test our projects
>
> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
> and add a profile for each and every server (maybe we can trim this down
> with pom imports?).
> Then run the build with one after each other:
>
> $> mvn clean test -Powb
> $> mvn clean test -Pweld
> $> mvn clean test -Powb-tc
> $> mvn clean test -Pweld-tc
> $> mvn clean test -Pgf31
> $> mvn clean test -Pas7
> $> mvn clean test -Ptomee
>
> This might pretty much blow up our poms...
>
>
> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
> Then add a deltaspike/test/base/core with the very basic parent stuff and
> 1 module for each integrated container, e.g.
> deltaspike/test/weld-tc
> deltaspike/test/owb-tc
> deltaspike/test/weld-jetty
> deltaspike/test/owb-jetty
> deltaspike/test/tomee
> deltaspike/test/gf31
> deltaspike/test/geronimo
> deltaspike/test/websphere
> deltaspike/test/as7
> deltaspike/test/as6
> etc.
>
> The initial setup costs are higher, but it would be pretty easy to add new
> containers that way.
>
> Which route shall we take?
>
In Seam 3 we have an approach which is a little bit of a mixture of both.
We have regular unit tests (with mockito, stubs, etc) under
impl/src/test/... and then we have a separate module called testsuite which
contained all the integration tests. We used pom imports to get all the
containers setup correctly and then had different profiles which pulled in
the poms and also configured surefire for that server. Inside the code
structure we separated the servers by package: common, jbossas, glassfish,
etc. That way all of the tests were in one location, they were easy to
execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
two downsides we had were 1) the extra config in the pom and 2) you can't
run the full integration suite in one maven execution. We spent quite a bit
of time examining options and trying some of them out. This was the best we
came up with, and also the one Aslak thought would work best as well.
> LieGrue,
> strub
>
>
--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net, pgp.mit.edu
Re: basic decisions: test setup
Posted by Christian Kaltepoth <ch...@kaltepoth.de>.
I also prefer JUnit over TestNG. JUnit recently catched up with TestNG
in regard to features and JUnit integrates better with Arquillian. I
also like that JUnit is supported by Eclipse out of the box.
2011/12/16 José Rodolfo Freitas <jo...@gmail.com>:
> I've been using arquillian + junit, and it does the job nicely.
>
> On Fri, Dec 16, 2011 at 9:58 AM, Marek Schmidt <ma...@redhat.com> wrote:
>
>> On 12/16/2011 09:38 AM, Mark Struberg wrote:
>>
>>> Hi!
>>>
>>> Another round of discussion - this time about how to setup our unit
>>> testing and integration testing.
>>>
>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>> our tests. That was also fundamental part of the incubator proposal.
>>> And instead of inventing yet another abstraction layer, I'd favour to
>>> just use Arquillian and contribute all things we need to this project.
>>> So here a formal vote
>>>
>>> 1.)
>>>
>>> for using Arquillian as test integration framework.
>>> +1 from me
>>>
>>>
>>>
>>> 2.)
>>>
>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>> world' projects, because it allows to define test dependencies.
>>> So which one to take?
>>>
>>
>> There is now org.jboss.arquillian.junit.**InSequence annotation in
>> arquillian for use with junit, which I believe could be used for those use
>> cases where one would use TestNG test dependencies...
>>
>>
>> But we should definitely only use 1 of the two exclusively!
>>>
>>> 3.)
>>> 'Integrated' tests vs 'Integration Tests'
>>> We have 2 options to test our projects
>>>
>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add a profile for each and every server (maybe we can trim this down
>>> with pom imports?).
>>> Then run the build with one after each other:
>>>
>>> $> mvn clean test -Powb
>>> $> mvn clean test -Pweld
>>> $> mvn clean test -Powb-tc
>>> $> mvn clean test -Pweld-tc
>>> $> mvn clean test -Pgf31
>>> $> mvn clean test -Pas7
>>> $> mvn clean test -Ptomee
>>>
>>> This might pretty much blow up our poms...
>>>
>>>
>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>> 1 module for each integrated container, e.g.
>>> deltaspike/test/weld-tc
>>> deltaspike/test/owb-tc
>>> deltaspike/test/weld-jetty
>>> deltaspike/test/owb-jetty
>>> deltaspike/test/tomee
>>> deltaspike/test/gf31
>>> deltaspike/test/geronimo
>>> deltaspike/test/websphere
>>> deltaspike/test/as7
>>> deltaspike/test/as6
>>> etc.
>>>
>>> The initial setup costs are higher, but it would be pretty easy to add
>>> new containers that way.
>>>
>>> Which route shall we take?
>>>
>>>
>>> LieGrue,
>>> strub
>>>
>>>
>>
--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal
Re: basic decisions: test setup
Posted by José Rodolfo Freitas <jo...@gmail.com>.
I've been using arquillian + junit, and it does the job nicely.
On Fri, Dec 16, 2011 at 9:58 AM, Marek Schmidt <ma...@redhat.com> wrote:
> On 12/16/2011 09:38 AM, Mark Struberg wrote:
>
>> Hi!
>>
>> Another round of discussion - this time about how to setup our unit
>> testing and integration testing.
>>
>> I think it's pretty much clear that we will gonna use Arquillian for all
>> our tests. That was also fundamental part of the incubator proposal.
>> And instead of inventing yet another abstraction layer, I'd favour to
>> just use Arquillian and contribute all things we need to this project.
>> So here a formal vote
>>
>> 1.)
>>
>> for using Arquillian as test integration framework.
>> +1 from me
>>
>>
>>
>> 2.)
>>
>> What do we like to use from the unit tests itself? TestNG or JUnit?
>> JUnit has better Arquillian integration and TestNG is better for 'real
>> world' projects, because it allows to define test dependencies.
>> So which one to take?
>>
>
> There is now org.jboss.arquillian.junit.**InSequence annotation in
> arquillian for use with junit, which I believe could be used for those use
> cases where one would use TestNG test dependencies...
>
>
> But we should definitely only use 1 of the two exclusively!
>>
>> 3.)
>> 'Integrated' tests vs 'Integration Tests'
>> We have 2 options to test our projects
>>
>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>> and add a profile for each and every server (maybe we can trim this down
>> with pom imports?).
>> Then run the build with one after each other:
>>
>> $> mvn clean test -Powb
>> $> mvn clean test -Pweld
>> $> mvn clean test -Powb-tc
>> $> mvn clean test -Pweld-tc
>> $> mvn clean test -Pgf31
>> $> mvn clean test -Pas7
>> $> mvn clean test -Ptomee
>>
>> This might pretty much blow up our poms...
>>
>>
>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>> 1 module for each integrated container, e.g.
>> deltaspike/test/weld-tc
>> deltaspike/test/owb-tc
>> deltaspike/test/weld-jetty
>> deltaspike/test/owb-jetty
>> deltaspike/test/tomee
>> deltaspike/test/gf31
>> deltaspike/test/geronimo
>> deltaspike/test/websphere
>> deltaspike/test/as7
>> deltaspike/test/as6
>> etc.
>>
>> The initial setup costs are higher, but it would be pretty easy to add
>> new containers that way.
>>
>> Which route shall we take?
>>
>>
>> LieGrue,
>> strub
>>
>>
>
Re: basic decisions: test setup
Posted by Marek Schmidt <ma...@redhat.com>.
On 12/16/2011 09:38 AM, Mark Struberg wrote:
> Hi!
>
> Another round of discussion - this time about how to setup our unit testing and integration testing.
>
> I think it's pretty much clear that we will gonna use Arquillian for all our tests. That was also fundamental part of the incubator proposal.
> And instead of inventing yet another abstraction layer, I'd favour to just use Arquillian and contribute all things we need to this project.
> So here a formal vote
>
> 1.)
>
> for using Arquillian as test integration framework.
> +1 from me
>
>
>
> 2.)
>
> What do we like to use from the unit tests itself? TestNG or JUnit?
> JUnit has better Arquillian integration and TestNG is better for 'real world' projects, because it allows to define test dependencies.
> So which one to take?
There is now org.jboss.arquillian.junit.InSequence annotation in
arquillian for use with junit, which I believe could be used for those
use cases where one would use TestNG test dependencies...
> But we should definitely only use 1 of the two exclusively!
>
> 3.)
> 'Integrated' tests vs 'Integration Tests'
> We have 2 options to test our projects
>
> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl) and add a profile for each and every server (maybe we can trim this down with pom imports?).
> Then run the build with one after each other:
>
> $> mvn clean test -Powb
> $> mvn clean test -Pweld
> $> mvn clean test -Powb-tc
> $> mvn clean test -Pweld-tc
> $> mvn clean test -Pgf31
> $> mvn clean test -Pas7
> $> mvn clean test -Ptomee
>
> This might pretty much blow up our poms...
>
>
> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl) and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
> Then add a deltaspike/test/base/core with the very basic parent stuff and 1 module for each integrated container, e.g.
> deltaspike/test/weld-tc
> deltaspike/test/owb-tc
> deltaspike/test/weld-jetty
> deltaspike/test/owb-jetty
> deltaspike/test/tomee
> deltaspike/test/gf31
> deltaspike/test/geronimo
> deltaspike/test/websphere
> deltaspike/test/as7
> deltaspike/test/as6
> etc.
>
> The initial setup costs are higher, but it would be pretty easy to add new containers that way.
>
> Which route shall we take?
>
>
> LieGrue,
> strub
>