You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Prasad Kashyap <go...@gmail.com> on 2006/12/08 05:31:13 UTC
Convention on dropping tests under the test framework
Since there were some questions today on where to drop new tests, I'll
take a stab at creating a convention. Feel free to offer your
suggestions so that we can modify it as we go along.
We began by having 2 testsuites just as an example.
geronimo/testsuite/
console-testsuite
deployment-testsuite
But almost everything can fall under the category of the
deployment-testsuite since most tests do need some deployable
artifact. So I think we should use the deployment-testsuite purely for
testing the deploy tool. Especially, it should be used to test
features like hot-deploy, redeploy and undeploy which have had JIRAs
before.
We should categorize the tests so that they reflect the broad
functional areas of the server.
* web-testsuite (servlets, jsp, jstl)
* enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf etc)
* mgmt-testsuite (jee management, deployment)
* webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
* performance-testsuite (server startup time, server footprint etc)
* security-testsuite
* console-testsuite
* regression (compatibility)
If nobody has any objection to this top categorization, I shall go
ahead and create these testsuites over the weekend. Meanwhile you may
drop your tests in the existing testsuites for now. I shall move them
appropriately.
Lastly, how do we deal with super apps like daytrader that can span
across multiple testsuites ?
Cheers
Prasad
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/8/06, Jason Dillon <ja...@planet57.com> wrote:
> On Dec 8, 2006, at 6:33 AM, Prasad Kashyap wrote:
> >> What is a super app anyways? :-P
> >
> > Maybe I should have called it a uber app. Unlike other small apps that
> > we create and use to test a particular feature, DT can serve as the
> > test app to test features across many categories.
>
> Small apps to test a particular feature... a test-app? Don't need to
> go calling other normal applications something else :-P
So sue me :-)
>
> I still don't understand why you even brought daytrader up in this
> context though... still would like to know that.
Apart from the small itty bitty apps that we write as test-apps, I
think it would be a good idea to test the features on a real world
app, a super-duper-uber-app :-) like DayTrader.
>
> --jason
Cheers
Prasad
Re: Convention on dropping tests under the test framework
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 8, 2006, at 6:33 AM, Prasad Kashyap wrote:
>> What is a super app anyways? :-P
>
> Maybe I should have called it a uber app. Unlike other small apps that
> we create and use to test a particular feature, DT can serve as the
> test app to test features across many categories.
Small apps to test a particular feature... a test-app? Don't need to
go calling other normal applications something else :-P
I still don't understand why you even brought daytrader up in this
context though... still would like to know that.
--jason
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/8/06, Jason Dillon <ja...@planet57.com> wrote:
> too lazy to insert a :-) every 10 words... so... just assume that I
> have. :-P
>
> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
> > Lastly, how do we deal with super apps like daytrader that can span
> > across multiple testsuites ?
>
> I don't understand what you mean by this at all. Why does it matter?
>
> What is a super app anyways? :-P
Maybe I should have called it a uber app. Unlike other small apps that
we create and use to test a particular feature, DT can serve as the
test app to test features across many categories.
> --jason
Cheers
Prasad
Re: Convention on dropping tests under the test framework
Posted by Matt Hogstrom <ma...@hogstrom.org>.
On Dec 8, 2006, at 3:50 AM, Jason Dillon wrote:
> too lazy to insert a :-) every 10 words... so... just assume that I
> have. :-P
dude, you're killing me :-P
Matt Hogstrom
matt@hogstrom.org
Re: Convention on dropping tests under the test framework
Posted by Jason Dillon <ja...@planet57.com>.
too lazy to insert a :-) every 10 words... so... just assume that I
have. :-P
On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
> Since there were some questions today on where to drop new tests, I'll
> take a stab at creating a convention. Feel free to offer your
> suggestions so that we can modify it as we go along.
>
> We began by having 2 testsuites just as an example.
> geronimo/testsuite/
> console-testsuite
> deployment-testsuite
These were only basic, basic examples for how it _might_ work.
Probably needs more thought before its really usable... though
console-testsuite, which does not need to build custom apps or other
muck, might be fine as is. But other types of tests may need some
custom app modules built, which is currently not part of the
testsuite/* bits at all.
> But almost everything can fall under the category of the
> deployment-testsuite since most tests do need some deployable
> artifact. So I think we should use the deployment-testsuite purely for
> testing the deploy tool. Especially, it should be used to test
> features like hot-deploy, redeploy and undeploy which have had JIRAs
> before.
The deployment-testsuite was intended to test the deployment
functionality of Geronimo... not just any place to put tests which
have deployables.
And I just tossed that together to test module deployment, so maybe
it should be refactored... or removed, or who knows.
> We should categorize the tests so that they reflect the broad
> functional areas of the server.
>
> * web-testsuite (servlets, jsp, jstl)
> * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf etc)
> * mgmt-testsuite (jee management, deployment)
> * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
> * performance-testsuite (server startup time, server footprint etc)
> * security-testsuite
> * console-testsuite
> * regression (compatibility)
>
> If nobody has any objection to this top categorization, I shall go
> ahead and create these testsuites over the weekend. Meanwhile you may
> drop your tests in the existing testsuites for now. I shall move them
> appropriately.
Please wait on this until there is a real more complete set of tests
which show that the setup/layout will scale for most/all types of
tests... I am fairly sure that it will not scale asis.
> Lastly, how do we deal with super apps like daytrader that can span
> across multiple testsuites ?
I don't understand what you mean by this at all. Why does it matter?
What is a super app anyways? :-P
--jason
Re: Convention on dropping tests under the test framework
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 8, 2006, at 9:34 AM, David Jencks wrote:
> I was wondering why the SeleniumTestSupport and ExtendedSelenium
> classes were in the tests themselves rather than in a support jar.
> I'd think these would be used by just about all tests.
They were simply never moved to a common location.
Looks like they have just been moved though... they should have
probably made it into a "selenium" package in the testsupport module.
> FIxing the tests is likely to take me all day. If you'd like to
> demonstrate what you think is correct structure you could set
> something up with the test ear from GERONIMO-2125
>
> http://issues.apache.org/jira/secure/attachment/12336683/manifestcp-
> itest-v3.jar
>
> which has a maven project to build a test ear. With luck deploying
> it stlll works :-)
This is probably a good idea, get some more real tests put together
to learn what works and what does not work with the current testsuite
setup.
--jason
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/10/06, David Jencks <da...@yahoo.com> wrote:
>
> On Dec 8, 2006, at 1:29 PM, Prasad Kashyap wrote:
>
> > David,
> >
> > Check out this patch http://people.apache.org/~prasad/manifestcp.patch
> >
> > Apply it from the geronimo/testsuite/depoyment-testsuite directory.
> >
> > It will create 2 directories under it.
> > -- The manifestcp will create the ear for.
> > -- The test-manifestcp will deploy and undeploy it.
> >
> > Throw your testcases under test-manifestcp.
>
> When I apply the patch I get files with many copies of the content.
> Assuming this builds for you can you commit it?
>
I have committed this code. Please read the comments in
deployment-testsuite/manifestcp/pom.xml
Cheers
Prasad
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/10/06, David Jencks <da...@yahoo.com> wrote:
>
> On Dec 8, 2006, at 1:29 PM, Prasad Kashyap wrote:
>
> > David,
> >
> > Check out this patch http://people.apache.org/~prasad/manifestcp.patch
> >
> > Apply it from the geronimo/testsuite/depoyment-testsuite directory.
> >
> > It will create 2 directories under it.
> > -- The manifestcp will create the ear for.
> > -- The test-manifestcp will deploy and undeploy it.
> >
> > Throw your testcases under test-manifestcp.
>
> When I apply the patch I get files with many copies of the content.
> Assuming this builds for you can you commit it?
>
> IIRC there there aren't any tests needed for this, if it deploys it
> works. When it's committed I'll check.
Sure. let me work on commiting it for you.
>
> I think I'd prefer everything including the tests to be under the
> manifestcp dir, but we can try to rearrange this later. I think we
> need to study how to best set this up.
Usually, that shouldn't have been a problem but for the fact that it
might be left out of the entire reporting structure of the test
framework. Let me see how I can pull in the surefire-reports from the
deploy/undeploy of the manifestcp.ear if it gets embedded inside this
"pom"
>
> thanks
> david jencks
>
Cheers
Prasad
> >
> > Cheers
> > Prasad
> >
> > On 12/8/06, David Jencks <da...@yahoo.com> wrote:
> >>
> >> On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
> >>
> >> > On 12/8/06, David Jencks <da...@yahoo.com> wrote:
> >> >
> >> >> For instance it might be useful to have a maven archetype to
> >> set up
> >> >> everything except the app to test and the actual test cases.
> >> >>
> >> >> thanks
> >> >> david jencks
> >> >>
> >> >
> >> > I have alrady written an archetype plugin called
> >> > testsuite-archetype-plugin that will let you create a testsuite
> >> with
> >> > an empty testset project under it. Please see
> >> > http://cwiki.apache.org/GMOxDEV/integration-
> >> > testing.html#IntegrationTesting-Gettingstarted
> >> >
> >> > However, in your case, since you don't want us to be createing
> >> any moe
> >> > testsuites till we have colected enough tests, this is what you
> >> should
> >> > do :
> >> >
> >> > 1) just make a copy of test-deployment, (say call it cxf-
> >> deployment)
> >> > 2) use it's child profile to go thro the complete maven lifecycle -
> >> > compile, build and test your apps.
> >> > 3) it's parent (deployment-testsuite) will take care of the server
> >> > start/stop and reporting for you.
> >>
> >> I wasn't able to make anything work where the app being tested was
> >> under the directory that's a copy of *-deployment. Now I'm working
> >> with a structure where the *-deployment is a child of the app build
> >> directory, and it seems to work. Once I straighten out the tests so
> >> they work I'll check it in and we can see if it can be reorganized to
> >> be simpler.
> >>
> >> I was wondering why the SeleniumTestSupport and ExtendedSelenium
> >> classes were in the tests themselves rather than in a support jar.
> >> I'd think these would be used by just about all tests.
> >>
> >> FIxing the tests is likely to take me all day. If you'd like to
> >> demonstrate what you think is correct structure you could set
> >> something up with the test ear from GERONIMO-2125
> >>
> >> http://issues.apache.org/jira/secure/attachment/12336683/manifestcp-
> >> itest-v3.jar
> >>
> >> which has a maven project to build a test ear. With luck deploying
> >> it stlll works :-)
> >>
> >> thanks
> >> david jencks
> >>
> >> >
> >> > Cheers
> >> > Prasad
> >> >
> >> > Cheers
> >> > Prasad
> >> >
> >> >
> >> >
> >> >
> >> >> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
> >> >>
> >> >> > Since there were some questions today on where to drop new
> >> >> tests, I'll
> >> >> > take a stab at creating a convention. Feel free to offer your
> >> >> > suggestions so that we can modify it as we go along.
> >> >> >
> >> >> > We began by having 2 testsuites just as an example.
> >> >> > geronimo/testsuite/
> >> >> > console-testsuite
> >> >> > deployment-testsuite
> >> >> >
> >> >> > But almost everything can fall under the category of the
> >> >> > deployment-testsuite since most tests do need some deployable
> >> >> > artifact. So I think we should use the deployment-testsuite
> >> >> purely for
> >> >> > testing the deploy tool. Especially, it should be used to test
> >> >> > features like hot-deploy, redeploy and undeploy which have had
> >> >> JIRAs
> >> >> > before.
> >> >> >
> >> >> > We should categorize the tests so that they reflect the broad
> >> >> > functional areas of the server.
> >> >> >
> >> >> > * web-testsuite (servlets, jsp, jstl)
> >> >> > * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf
> >> >> etc)
> >> >> > * mgmt-testsuite (jee management, deployment)
> >> >> > * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata
> >> etc)
> >> >> > * performance-testsuite (server startup time, server
> >> footprint etc)
> >> >> > * security-testsuite
> >> >> > * console-testsuite
> >> >> > * regression (compatibility)
> >> >> >
> >> >> > If nobody has any objection to this top categorization, I
> >> shall go
> >> >> > ahead and create these testsuites over the weekend. Meanwhile
> >> >> you may
> >> >> > drop your tests in the existing testsuites for now. I shall move
> >> >> them
> >> >> > appropriately.
> >> >> >
> >> >> > Lastly, how do we deal with super apps like daytrader that
> >> can span
> >> >> > across multiple testsuites ?
> >> >> >
> >> >> > Cheers
> >> >> > Prasad
> >> >>
> >> >>
> >>
> >>
>
>
Re: Convention on dropping tests under the test framework
Posted by David Jencks <da...@yahoo.com>.
On Dec 8, 2006, at 1:29 PM, Prasad Kashyap wrote:
> David,
>
> Check out this patch http://people.apache.org/~prasad/manifestcp.patch
>
> Apply it from the geronimo/testsuite/depoyment-testsuite directory.
>
> It will create 2 directories under it.
> -- The manifestcp will create the ear for.
> -- The test-manifestcp will deploy and undeploy it.
>
> Throw your testcases under test-manifestcp.
When I apply the patch I get files with many copies of the content.
Assuming this builds for you can you commit it?
IIRC there there aren't any tests needed for this, if it deploys it
works. When it's committed I'll check.
I think I'd prefer everything including the tests to be under the
manifestcp dir, but we can try to rearrange this later. I think we
need to study how to best set this up.
thanks
david jencks
>
> Cheers
> Prasad
>
> On 12/8/06, David Jencks <da...@yahoo.com> wrote:
>>
>> On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
>>
>> > On 12/8/06, David Jencks <da...@yahoo.com> wrote:
>> >
>> >> For instance it might be useful to have a maven archetype to
>> set up
>> >> everything except the app to test and the actual test cases.
>> >>
>> >> thanks
>> >> david jencks
>> >>
>> >
>> > I have alrady written an archetype plugin called
>> > testsuite-archetype-plugin that will let you create a testsuite
>> with
>> > an empty testset project under it. Please see
>> > http://cwiki.apache.org/GMOxDEV/integration-
>> > testing.html#IntegrationTesting-Gettingstarted
>> >
>> > However, in your case, since you don't want us to be createing
>> any moe
>> > testsuites till we have colected enough tests, this is what you
>> should
>> > do :
>> >
>> > 1) just make a copy of test-deployment, (say call it cxf-
>> deployment)
>> > 2) use it's child profile to go thro the complete maven lifecycle -
>> > compile, build and test your apps.
>> > 3) it's parent (deployment-testsuite) will take care of the server
>> > start/stop and reporting for you.
>>
>> I wasn't able to make anything work where the app being tested was
>> under the directory that's a copy of *-deployment. Now I'm working
>> with a structure where the *-deployment is a child of the app build
>> directory, and it seems to work. Once I straighten out the tests so
>> they work I'll check it in and we can see if it can be reorganized to
>> be simpler.
>>
>> I was wondering why the SeleniumTestSupport and ExtendedSelenium
>> classes were in the tests themselves rather than in a support jar.
>> I'd think these would be used by just about all tests.
>>
>> FIxing the tests is likely to take me all day. If you'd like to
>> demonstrate what you think is correct structure you could set
>> something up with the test ear from GERONIMO-2125
>>
>> http://issues.apache.org/jira/secure/attachment/12336683/manifestcp-
>> itest-v3.jar
>>
>> which has a maven project to build a test ear. With luck deploying
>> it stlll works :-)
>>
>> thanks
>> david jencks
>>
>> >
>> > Cheers
>> > Prasad
>> >
>> > Cheers
>> > Prasad
>> >
>> >
>> >
>> >
>> >> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
>> >>
>> >> > Since there were some questions today on where to drop new
>> >> tests, I'll
>> >> > take a stab at creating a convention. Feel free to offer your
>> >> > suggestions so that we can modify it as we go along.
>> >> >
>> >> > We began by having 2 testsuites just as an example.
>> >> > geronimo/testsuite/
>> >> > console-testsuite
>> >> > deployment-testsuite
>> >> >
>> >> > But almost everything can fall under the category of the
>> >> > deployment-testsuite since most tests do need some deployable
>> >> > artifact. So I think we should use the deployment-testsuite
>> >> purely for
>> >> > testing the deploy tool. Especially, it should be used to test
>> >> > features like hot-deploy, redeploy and undeploy which have had
>> >> JIRAs
>> >> > before.
>> >> >
>> >> > We should categorize the tests so that they reflect the broad
>> >> > functional areas of the server.
>> >> >
>> >> > * web-testsuite (servlets, jsp, jstl)
>> >> > * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf
>> >> etc)
>> >> > * mgmt-testsuite (jee management, deployment)
>> >> > * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata
>> etc)
>> >> > * performance-testsuite (server startup time, server
>> footprint etc)
>> >> > * security-testsuite
>> >> > * console-testsuite
>> >> > * regression (compatibility)
>> >> >
>> >> > If nobody has any objection to this top categorization, I
>> shall go
>> >> > ahead and create these testsuites over the weekend. Meanwhile
>> >> you may
>> >> > drop your tests in the existing testsuites for now. I shall move
>> >> them
>> >> > appropriately.
>> >> >
>> >> > Lastly, how do we deal with super apps like daytrader that
>> can span
>> >> > across multiple testsuites ?
>> >> >
>> >> > Cheers
>> >> > Prasad
>> >>
>> >>
>>
>>
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
David,
Check out this patch http://people.apache.org/~prasad/manifestcp.patch
Apply it from the geronimo/testsuite/depoyment-testsuite directory.
It will create 2 directories under it.
-- The manifestcp will create the ear for.
-- The test-manifestcp will deploy and undeploy it.
Throw your testcases under test-manifestcp.
Cheers
Prasad
On 12/8/06, David Jencks <da...@yahoo.com> wrote:
>
> On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
>
> > On 12/8/06, David Jencks <da...@yahoo.com> wrote:
> >
> >> For instance it might be useful to have a maven archetype to set up
> >> everything except the app to test and the actual test cases.
> >>
> >> thanks
> >> david jencks
> >>
> >
> > I have alrady written an archetype plugin called
> > testsuite-archetype-plugin that will let you create a testsuite with
> > an empty testset project under it. Please see
> > http://cwiki.apache.org/GMOxDEV/integration-
> > testing.html#IntegrationTesting-Gettingstarted
> >
> > However, in your case, since you don't want us to be createing any moe
> > testsuites till we have colected enough tests, this is what you should
> > do :
> >
> > 1) just make a copy of test-deployment, (say call it cxf-deployment)
> > 2) use it's child profile to go thro the complete maven lifecycle -
> > compile, build and test your apps.
> > 3) it's parent (deployment-testsuite) will take care of the server
> > start/stop and reporting for you.
>
> I wasn't able to make anything work where the app being tested was
> under the directory that's a copy of *-deployment. Now I'm working
> with a structure where the *-deployment is a child of the app build
> directory, and it seems to work. Once I straighten out the tests so
> they work I'll check it in and we can see if it can be reorganized to
> be simpler.
>
> I was wondering why the SeleniumTestSupport and ExtendedSelenium
> classes were in the tests themselves rather than in a support jar.
> I'd think these would be used by just about all tests.
>
> FIxing the tests is likely to take me all day. If you'd like to
> demonstrate what you think is correct structure you could set
> something up with the test ear from GERONIMO-2125
>
> http://issues.apache.org/jira/secure/attachment/12336683/manifestcp-
> itest-v3.jar
>
> which has a maven project to build a test ear. With luck deploying
> it stlll works :-)
>
> thanks
> david jencks
>
> >
> > Cheers
> > Prasad
> >
> > Cheers
> > Prasad
> >
> >
> >
> >
> >> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
> >>
> >> > Since there were some questions today on where to drop new
> >> tests, I'll
> >> > take a stab at creating a convention. Feel free to offer your
> >> > suggestions so that we can modify it as we go along.
> >> >
> >> > We began by having 2 testsuites just as an example.
> >> > geronimo/testsuite/
> >> > console-testsuite
> >> > deployment-testsuite
> >> >
> >> > But almost everything can fall under the category of the
> >> > deployment-testsuite since most tests do need some deployable
> >> > artifact. So I think we should use the deployment-testsuite
> >> purely for
> >> > testing the deploy tool. Especially, it should be used to test
> >> > features like hot-deploy, redeploy and undeploy which have had
> >> JIRAs
> >> > before.
> >> >
> >> > We should categorize the tests so that they reflect the broad
> >> > functional areas of the server.
> >> >
> >> > * web-testsuite (servlets, jsp, jstl)
> >> > * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf
> >> etc)
> >> > * mgmt-testsuite (jee management, deployment)
> >> > * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
> >> > * performance-testsuite (server startup time, server footprint etc)
> >> > * security-testsuite
> >> > * console-testsuite
> >> > * regression (compatibility)
> >> >
> >> > If nobody has any objection to this top categorization, I shall go
> >> > ahead and create these testsuites over the weekend. Meanwhile
> >> you may
> >> > drop your tests in the existing testsuites for now. I shall move
> >> them
> >> > appropriately.
> >> >
> >> > Lastly, how do we deal with super apps like daytrader that can span
> >> > across multiple testsuites ?
> >> >
> >> > Cheers
> >> > Prasad
> >>
> >>
>
>
Re: Convention on dropping tests under the test framework
Posted by David Jencks <da...@yahoo.com>.
On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
> On 12/8/06, David Jencks <da...@yahoo.com> wrote:
>
>> For instance it might be useful to have a maven archetype to set up
>> everything except the app to test and the actual test cases.
>>
>> thanks
>> david jencks
>>
>
> I have alrady written an archetype plugin called
> testsuite-archetype-plugin that will let you create a testsuite with
> an empty testset project under it. Please see
> http://cwiki.apache.org/GMOxDEV/integration-
> testing.html#IntegrationTesting-Gettingstarted
>
> However, in your case, since you don't want us to be createing any moe
> testsuites till we have colected enough tests, this is what you should
> do :
>
> 1) just make a copy of test-deployment, (say call it cxf-deployment)
> 2) use it's child profile to go thro the complete maven lifecycle -
> compile, build and test your apps.
> 3) it's parent (deployment-testsuite) will take care of the server
> start/stop and reporting for you.
I wasn't able to make anything work where the app being tested was
under the directory that's a copy of *-deployment. Now I'm working
with a structure where the *-deployment is a child of the app build
directory, and it seems to work. Once I straighten out the tests so
they work I'll check it in and we can see if it can be reorganized to
be simpler.
I was wondering why the SeleniumTestSupport and ExtendedSelenium
classes were in the tests themselves rather than in a support jar.
I'd think these would be used by just about all tests.
FIxing the tests is likely to take me all day. If you'd like to
demonstrate what you think is correct structure you could set
something up with the test ear from GERONIMO-2125
http://issues.apache.org/jira/secure/attachment/12336683/manifestcp-
itest-v3.jar
which has a maven project to build a test ear. With luck deploying
it stlll works :-)
thanks
david jencks
>
> Cheers
> Prasad
>
> Cheers
> Prasad
>
>
>
>
>> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
>>
>> > Since there were some questions today on where to drop new
>> tests, I'll
>> > take a stab at creating a convention. Feel free to offer your
>> > suggestions so that we can modify it as we go along.
>> >
>> > We began by having 2 testsuites just as an example.
>> > geronimo/testsuite/
>> > console-testsuite
>> > deployment-testsuite
>> >
>> > But almost everything can fall under the category of the
>> > deployment-testsuite since most tests do need some deployable
>> > artifact. So I think we should use the deployment-testsuite
>> purely for
>> > testing the deploy tool. Especially, it should be used to test
>> > features like hot-deploy, redeploy and undeploy which have had
>> JIRAs
>> > before.
>> >
>> > We should categorize the tests so that they reflect the broad
>> > functional areas of the server.
>> >
>> > * web-testsuite (servlets, jsp, jstl)
>> > * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf
>> etc)
>> > * mgmt-testsuite (jee management, deployment)
>> > * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
>> > * performance-testsuite (server startup time, server footprint etc)
>> > * security-testsuite
>> > * console-testsuite
>> > * regression (compatibility)
>> >
>> > If nobody has any objection to this top categorization, I shall go
>> > ahead and create these testsuites over the weekend. Meanwhile
>> you may
>> > drop your tests in the existing testsuites for now. I shall move
>> them
>> > appropriately.
>> >
>> > Lastly, how do we deal with super apps like daytrader that can span
>> > across multiple testsuites ?
>> >
>> > Cheers
>> > Prasad
>>
>>
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
Starting and stopping the server for each app ? Wouldn't that be a tad too much.
Currently the start/stop is designed to happen for a suite (eg. web).
So a bunch of related tests (servlet tests, jsp tests, jsf tests etc)
can be performed in a suite with required apps getting deployed and
undeployed for the tests.
Cheers
Prasad
On 12/11/06, David Jencks <da...@yahoo.com> wrote:
> I thought about this a little more and have some ideas about what I'd
> like..... I dunno if this is remotely possible.
>
> - keep the test app and test code close together in svn. It's going
> to be a lot easier to work with a single test if these are close
> together, rather than like the testsupport stuff that is far from the
> tests that use it.
>
> - make it easy to run the tests on a single app... so there's an easy
> way to build the test app, start up a server, and run the tests for
> that app.
>
> - make it relatively quick to run all the tests.... which I think
> means starting a server, then for each app, deploying, running tests,
> and undeploying. My current understanding of what we have now is
> that the server is going to be started and stopped for each app if
> we've set things up for my second requirement :-)
>
> thanks
> david jencks
>
>
> On Dec 8, 2006, at 1:21 PM, Prasad Kashyap wrote:
>
> > I know what you mean. I just worked on the app that David was talking
> > about those do need a separate module by themselves. I didn't realise
> > his scenario until I saw the JIRA.
> >
> > I was talking about something like
> > testsuite/web-testsuite/test-servlet25. This example builds a war and
> > tests it in the same pom.
> >
> > Cheers
> > Prasad
> >
> > On 12/8/06, Jason Dillon <ja...@planet57.com> wrote:
> >> On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
> >> > 1) just make a copy of test-deployment, (say call it cxf-
> >> deployment)
> >> > 2) use it's child profile to go thro the complete maven lifecycle -
> >> > compile, build and test your apps.
> >> > 3) it's parent (deployment-testsuite) will take care of the server
> >> > start/stop and reporting for you.
> >>
> >> This won't actually work as easily as you may have imagined Prasad.
> >> To effectively build some apps, you actually need to have them live
> >> in their own module, so that they can be built using the associated
> >> maven plugins.
> >>
> >> I do not believe that the current testsuite setup takes this into
> >> account... and this is why I think it is premature to simply roll out
> >> the same config/layout with out having some more real tests, which
> >> build these test applications and allow for effective organization.
> >>
> >> I don't think we are quite there yet... close, but still a bit off.
> >>
> >> --jason
> >>
> >>
>
>
Re: Convention on dropping tests under the test framework
Posted by Jason Dillon <ja...@planet57.com>.
Eh... rebuild time for apps with no changes are negligible... it
takes longer for selenium to spin up firefox. I would not even worry
about the app rebuild time.
--jason
On Dec 12, 2006, at 7:23 AM, Prasad Kashyap wrote:
> On 12/11/06, David Jencks <da...@yahoo.com> wrote:
>> I thought about this a little more and have some ideas about what I'd
>> like..... I dunno if this is remotely possible.
>>
>> - keep the test app and test code close together in svn. It's going
>> to be a lot easier to work with a single test if these are close
>> together, rather than like the testsupport stuff that is far from the
>> tests that use it.
>>
>> - make it easy to run the tests on a single app... so there's an easy
>> way to build the test app, start up a server, and run the tests for
>> that app.
>>
>
> The downside to this is that you are building the app everytime you
> run the test. This is doubly wasteful since the tests are run once for
> each of the container-assemblies. Granted, this app builds very fast
> but still there is a delta time that it adds to the test process. Not
> every app may build this fast.
>
> IMO, The most ideal scenario is what we do for the
> enterprise-testsuite/test-ejbcontainer. There, both the ejb beans and
> the tests are compiled elsewhere and published. They are simply
> invoked in the framework.
>
> Cheers
> Prasad
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/11/06, David Jencks <da...@yahoo.com> wrote:
> I thought about this a little more and have some ideas about what I'd
> like..... I dunno if this is remotely possible.
>
> - keep the test app and test code close together in svn. It's going
> to be a lot easier to work with a single test if these are close
> together, rather than like the testsupport stuff that is far from the
> tests that use it.
>
> - make it easy to run the tests on a single app... so there's an easy
> way to build the test app, start up a server, and run the tests for
> that app.
>
The downside to this is that you are building the app everytime you
run the test. This is doubly wasteful since the tests are run once for
each of the container-assemblies. Granted, this app builds very fast
but still there is a delta time that it adds to the test process. Not
every app may build this fast.
IMO, The most ideal scenario is what we do for the
enterprise-testsuite/test-ejbcontainer. There, both the ejb beans and
the tests are compiled elsewhere and published. They are simply
invoked in the framework.
Cheers
Prasad
Re: Convention on dropping tests under the test framework
Posted by David Jencks <da...@yahoo.com>.
I thought about this a little more and have some ideas about what I'd
like..... I dunno if this is remotely possible.
- keep the test app and test code close together in svn. It's going
to be a lot easier to work with a single test if these are close
together, rather than like the testsupport stuff that is far from the
tests that use it.
- make it easy to run the tests on a single app... so there's an easy
way to build the test app, start up a server, and run the tests for
that app.
- make it relatively quick to run all the tests.... which I think
means starting a server, then for each app, deploying, running tests,
and undeploying. My current understanding of what we have now is
that the server is going to be started and stopped for each app if
we've set things up for my second requirement :-)
thanks
david jencks
On Dec 8, 2006, at 1:21 PM, Prasad Kashyap wrote:
> I know what you mean. I just worked on the app that David was talking
> about those do need a separate module by themselves. I didn't realise
> his scenario until I saw the JIRA.
>
> I was talking about something like
> testsuite/web-testsuite/test-servlet25. This example builds a war and
> tests it in the same pom.
>
> Cheers
> Prasad
>
> On 12/8/06, Jason Dillon <ja...@planet57.com> wrote:
>> On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
>> > 1) just make a copy of test-deployment, (say call it cxf-
>> deployment)
>> > 2) use it's child profile to go thro the complete maven lifecycle -
>> > compile, build and test your apps.
>> > 3) it's parent (deployment-testsuite) will take care of the server
>> > start/stop and reporting for you.
>>
>> This won't actually work as easily as you may have imagined Prasad.
>> To effectively build some apps, you actually need to have them live
>> in their own module, so that they can be built using the associated
>> maven plugins.
>>
>> I do not believe that the current testsuite setup takes this into
>> account... and this is why I think it is premature to simply roll out
>> the same config/layout with out having some more real tests, which
>> build these test applications and allow for effective organization.
>>
>> I don't think we are quite there yet... close, but still a bit off.
>>
>> --jason
>>
>>
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
I know what you mean. I just worked on the app that David was talking
about those do need a separate module by themselves. I didn't realise
his scenario until I saw the JIRA.
I was talking about something like
testsuite/web-testsuite/test-servlet25. This example builds a war and
tests it in the same pom.
Cheers
Prasad
On 12/8/06, Jason Dillon <ja...@planet57.com> wrote:
> On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
> > 1) just make a copy of test-deployment, (say call it cxf-deployment)
> > 2) use it's child profile to go thro the complete maven lifecycle -
> > compile, build and test your apps.
> > 3) it's parent (deployment-testsuite) will take care of the server
> > start/stop and reporting for you.
>
> This won't actually work as easily as you may have imagined Prasad.
> To effectively build some apps, you actually need to have them live
> in their own module, so that they can be built using the associated
> maven plugins.
>
> I do not believe that the current testsuite setup takes this into
> account... and this is why I think it is premature to simply roll out
> the same config/layout with out having some more real tests, which
> build these test applications and allow for effective organization.
>
> I don't think we are quite there yet... close, but still a bit off.
>
> --jason
>
>
Re: Convention on dropping tests under the test framework
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 8, 2006, at 6:30 AM, Prasad Kashyap wrote:
> 1) just make a copy of test-deployment, (say call it cxf-deployment)
> 2) use it's child profile to go thro the complete maven lifecycle -
> compile, build and test your apps.
> 3) it's parent (deployment-testsuite) will take care of the server
> start/stop and reporting for you.
This won't actually work as easily as you may have imagined Prasad.
To effectively build some apps, you actually need to have them live
in their own module, so that they can be built using the associated
maven plugins.
I do not believe that the current testsuite setup takes this into
account... and this is why I think it is premature to simply roll out
the same config/layout with out having some more real tests, which
build these test applications and allow for effective organization.
I don't think we are quite there yet... close, but still a bit off.
--jason
Re: Convention on dropping tests under the test framework
Posted by Prasad Kashyap <go...@gmail.com>.
On 12/8/06, David Jencks <da...@yahoo.com> wrote:
> For instance it might be useful to have a maven archetype to set up
> everything except the app to test and the actual test cases.
>
> thanks
> david jencks
>
I have alrady written an archetype plugin called
testsuite-archetype-plugin that will let you create a testsuite with
an empty testset project under it. Please see
http://cwiki.apache.org/GMOxDEV/integration-testing.html#IntegrationTesting-Gettingstarted
However, in your case, since you don't want us to be createing any moe
testsuites till we have colected enough tests, this is what you should
do :
1) just make a copy of test-deployment, (say call it cxf-deployment)
2) use it's child profile to go thro the complete maven lifecycle -
compile, build and test your apps.
3) it's parent (deployment-testsuite) will take care of the server
start/stop and reporting for you.
Cheers
Prasad
Cheers
Prasad
> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
>
> > Since there were some questions today on where to drop new tests, I'll
> > take a stab at creating a convention. Feel free to offer your
> > suggestions so that we can modify it as we go along.
> >
> > We began by having 2 testsuites just as an example.
> > geronimo/testsuite/
> > console-testsuite
> > deployment-testsuite
> >
> > But almost everything can fall under the category of the
> > deployment-testsuite since most tests do need some deployable
> > artifact. So I think we should use the deployment-testsuite purely for
> > testing the deploy tool. Especially, it should be used to test
> > features like hot-deploy, redeploy and undeploy which have had JIRAs
> > before.
> >
> > We should categorize the tests so that they reflect the broad
> > functional areas of the server.
> >
> > * web-testsuite (servlets, jsp, jstl)
> > * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf etc)
> > * mgmt-testsuite (jee management, deployment)
> > * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
> > * performance-testsuite (server startup time, server footprint etc)
> > * security-testsuite
> > * console-testsuite
> > * regression (compatibility)
> >
> > If nobody has any objection to this top categorization, I shall go
> > ahead and create these testsuites over the weekend. Meanwhile you may
> > drop your tests in the existing testsuites for now. I shall move them
> > appropriately.
> >
> > Lastly, how do we deal with super apps like daytrader that can span
> > across multiple testsuites ?
> >
> > Cheers
> > Prasad
>
>
Re: Convention on dropping tests under the test framework
Posted by Jason Dillon <ja...@planet57.com>.
I agree. We need some *real* tests first so that we can see if the
setup we have no will work, or will not. Looks like it might need
some more changes before we'd want to make a cookie cutter.
--jason
On Dec 8, 2006, at 12:35 AM, David Jencks wrote:
> I'm afraid I think it's way too early to try to categorize where
> tests should go. Based on my experience so far I think we need to
> work on simplifying how to set up a small app and have some simple
> tests for it first. Once we have enough of these to confuse
> ourselves about organization, and it's dead simple to write a new
> one, we can worry about categorizing them.
>
> For instance it might be useful to have a maven archetype to set up
> everything except the app to test and the actual test cases.
>
> thanks
> david jencks
>
> On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
>
>> Since there were some questions today on where to drop new tests,
>> I'll
>> take a stab at creating a convention. Feel free to offer your
>> suggestions so that we can modify it as we go along.
>>
>> We began by having 2 testsuites just as an example.
>> geronimo/testsuite/
>> console-testsuite
>> deployment-testsuite
>>
>> But almost everything can fall under the category of the
>> deployment-testsuite since most tests do need some deployable
>> artifact. So I think we should use the deployment-testsuite purely
>> for
>> testing the deploy tool. Especially, it should be used to test
>> features like hot-deploy, redeploy and undeploy which have had JIRAs
>> before.
>>
>> We should categorize the tests so that they reflect the broad
>> functional areas of the server.
>>
>> * web-testsuite (servlets, jsp, jstl)
>> * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf etc)
>> * mgmt-testsuite (jee management, deployment)
>> * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
>> * performance-testsuite (server startup time, server footprint etc)
>> * security-testsuite
>> * console-testsuite
>> * regression (compatibility)
>>
>> If nobody has any objection to this top categorization, I shall go
>> ahead and create these testsuites over the weekend. Meanwhile you may
>> drop your tests in the existing testsuites for now. I shall move them
>> appropriately.
>>
>> Lastly, how do we deal with super apps like daytrader that can span
>> across multiple testsuites ?
>>
>> Cheers
>> Prasad
>
Re: Convention on dropping tests under the test framework
Posted by David Jencks <da...@yahoo.com>.
I'm afraid I think it's way too early to try to categorize where
tests should go. Based on my experience so far I think we need to
work on simplifying how to set up a small app and have some simple
tests for it first. Once we have enough of these to confuse
ourselves about organization, and it's dead simple to write a new
one, we can worry about categorizing them.
For instance it might be useful to have a maven archetype to set up
everything except the app to test and the actual test cases.
thanks
david jencks
On Dec 7, 2006, at 8:31 PM, Prasad Kashyap wrote:
> Since there were some questions today on where to drop new tests, I'll
> take a stab at creating a convention. Feel free to offer your
> suggestions so that we can modify it as we go along.
>
> We began by having 2 testsuites just as an example.
> geronimo/testsuite/
> console-testsuite
> deployment-testsuite
>
> But almost everything can fall under the category of the
> deployment-testsuite since most tests do need some deployable
> artifact. So I think we should use the deployment-testsuite purely for
> testing the deploy tool. Especially, it should be used to test
> features like hot-deploy, redeploy and undeploy which have had JIRAs
> before.
>
> We should categorize the tests so that they reflect the broad
> functional areas of the server.
>
> * web-testsuite (servlets, jsp, jstl)
> * enterprise-testsuite (ejb, jpa, jms, jacc, jta, javamail, jaf etc)
> * mgmt-testsuite (jee management, deployment)
> * webservices-testsuite (wsee, jaxb, stax, saaj, ws-metadata etc)
> * performance-testsuite (server startup time, server footprint etc)
> * security-testsuite
> * console-testsuite
> * regression (compatibility)
>
> If nobody has any objection to this top categorization, I shall go
> ahead and create these testsuites over the weekend. Meanwhile you may
> drop your tests in the existing testsuites for now. I shall move them
> appropriately.
>
> Lastly, how do we deal with super apps like daytrader that can span
> across multiple testsuites ?
>
> Cheers
> Prasad