You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by Patrick Linskey <pl...@bea.com> on 2007/02/01 20:08:44 UTC
[DISCUSS] Move to TestNG?
Hi,
According to the discussion at
https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
TestNG's concept of test groups could let us easily create tests that
are expected to fail, and exclude them from test runs until the
corresponding behavior is fixed / feature is implemented. What do you
guys think about moving to TestNG? I believe that the work involved is
minimal; there are some conversion tools / compatibility modes or
something.
-Patrick
--
Patrick Linskey
BEA Systems, Inc.
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
Re: [DISCUSS] Move to TestNG?
Posted by Michael Dick <mi...@gmail.com>.
For what it's worth Marc is right. I've used testng via the surefire plugin
as a front end to running jUnit tests. In my case I just tried to get them
to run in parallel, but we should be able to use groups in the same manner.
On 2/1/07, Marc Prud'hommeaux <mp...@apache.org> wrote:
>
>
> Here's a good article comparing TestNG with JUnit:
>
> http://www-128.ibm.com/developerworks/java/library/j-cq08296/
>
> IMO, the groups feature is nice, but isn't necessarily sufficient to
> justify a lot of effort porting over tests (since we could always
> just check a system property before running expected failure tests or
> something). However, one feature that does sound very nice is their
> support for parametric testing. If I understand it correctly, this
> would allow us to automatically run tests against various data model
> implementations. This would be nice, because we could have a set of
> interfaces for data models and implementations that vary their
> mapping details (single-table mapping, table-per-class mapping,
> joined mapping, mappings with attributes in secondary tables, orm.xml
> mappings, etc), and transparently run the same tests against each of
> the data models. This would dramatically expand the completeness of
> our tests, since historically we have only done things like
> TestJustAFewSimpleOperationsAgaintCompositePrimaryKeyClasses,
> TestSomeOtherNuanceOfEmbeddedRelations, etc.
>
> Also, note that I don't think that using TestNG vs. JUnit is
> necessarily an xor proposition: I think that Maven's surefire harness
> can run against both frameworks simultaneously, and aggregate the
> test results. I'm not sure though.
>
> I'll play around with it a little bit today and let you all know the
> results.
>
>
>
> On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
>
> > Hi,
> >
> > According to the discussion at
> > https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> > TestNG's concept of test groups could let us easily create tests that
> > are expected to fail, and exclude them from test runs until the
> > corresponding behavior is fixed / feature is implemented. What do you
> > guys think about moving to TestNG? I believe that the work involved is
> > minimal; there are some conversion tools / compatibility modes or
> > something.
> >
> > -Patrick
> >
> > --
> > Patrick Linskey
> > BEA Systems, Inc.
> >
> > ______________________________________________________________________
> > _
> > Notice: This email message, together with any attachments, may
> > contain
> > information of BEA Systems, Inc., its subsidiaries and
> > affiliated
> > entities, that may be confidential, proprietary, copyrighted
> > and/or
> > legally privileged, and is intended solely for the use of the
> > individual
> > or entity named in this message. If you are not the intended
> > recipient,
> > and have received this message in error, please immediately return
> > this
> > by email and then delete it.
>
>
--
-Michael Dick
Re: [DISCUSS] Move to TestNG?
Posted by Marc Prud'hommeaux <mp...@apache.org>.
Here's a good article comparing TestNG with JUnit:
http://www-128.ibm.com/developerworks/java/library/j-cq08296/
IMO, the groups feature is nice, but isn't necessarily sufficient to
justify a lot of effort porting over tests (since we could always
just check a system property before running expected failure tests or
something). However, one feature that does sound very nice is their
support for parametric testing. If I understand it correctly, this
would allow us to automatically run tests against various data model
implementations. This would be nice, because we could have a set of
interfaces for data models and implementations that vary their
mapping details (single-table mapping, table-per-class mapping,
joined mapping, mappings with attributes in secondary tables, orm.xml
mappings, etc), and transparently run the same tests against each of
the data models. This would dramatically expand the completeness of
our tests, since historically we have only done things like
TestJustAFewSimpleOperationsAgaintCompositePrimaryKeyClasses,
TestSomeOtherNuanceOfEmbeddedRelations, etc.
Also, note that I don't think that using TestNG vs. JUnit is
necessarily an xor proposition: I think that Maven's surefire harness
can run against both frameworks simultaneously, and aggregate the
test results. I'm not sure though.
I'll play around with it a little bit today and let you all know the
results.
On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
> Hi,
>
> According to the discussion at
> https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> TestNG's concept of test groups could let us easily create tests that
> are expected to fail, and exclude them from test runs until the
> corresponding behavior is fixed / feature is implemented. What do you
> guys think about moving to TestNG? I believe that the work involved is
> minimal; there are some conversion tools / compatibility modes or
> something.
>
> -Patrick
>
> --
> Patrick Linskey
> BEA Systems, Inc.
>
> ______________________________________________________________________
> _
> Notice: This email message, together with any attachments, may
> contain
> information of BEA Systems, Inc., its subsidiaries and
> affiliated
> entities, that may be confidential, proprietary, copyrighted
> and/or
> legally privileged, and is intended solely for the use of the
> individual
> or entity named in this message. If you are not the intended
> recipient,
> and have received this message in error, please immediately return
> this
> by email and then delete it.
Re: [DISCUSS] Move to TestNG?
Posted by Dain Sundstrom <da...@iq80.com>.
On Feb 1, 2007, at 11:39 AM, Kevin Sutter wrote:
> I see that they have a plugin for Eclipse, but I haven't tried it
> yet. It
> doesn't look like it supports IntelliJ. On the surface, the
> conversion and
> usage looks doable and would satisfy the "failed testcase" problem.
We can exclude tests for both JUnit and TestNQ fairly easily in the pom.
-dain
RE: [DISCUSS] Move to TestNG?
Posted by Patrick Linskey <pl...@bea.com>.
Another benefit to TestNG would be the ability to separate out certain
tests for pre-checkin and other tests for continuous integration builds.
As our test suite gets bigger, this will start to become an issue.
-Patrick
--
Patrick Linskey
BEA Systems, Inc.
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
> -----Original Message-----
> From: Kevin Sutter [mailto:kwsutter@gmail.com]
> Sent: Thursday, February 01, 2007 11:39 AM
> To: open-jpa-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Move to TestNG?
>
> I see that they have a plugin for Eclipse, but I haven't
> tried it yet. It
> doesn't look like it supports IntelliJ. On the surface, the
> conversion and
> usage looks doable and would satisfy the "failed testcase" problem.
>
> My question is whether it's worth the effort. If we have a
> testcase that
> has failures, write appropriate JIRA reports, and comment out
> the failing
> variations until they get resolved. As part of the JIRA
> resolution, you
> uncomment the failing test variations.
>
> Even if we go with TestNG, it's a similar process. We
> discover a problem,
> write a JIRA report, and put the testcase in a special group
> that allows
> failures. When the JIRA report is resolved, we move the testcase to a
> regular group where it is not allowed to fail.
>
> Kevin
>
> On 2/1/07, Dain Sundstrom <da...@iq80.com> wrote:
> >
> > My only concern is IDE support and specifically the IDE I use
> > intellij. Does this work as a test case type in IntelliJ
> and Eclipse?
> >
> > If so, I don't really care which framework we use.
> >
> > -dain
> >
> > On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
> >
> > > Hi,
> > >
> > > According to the discussion at
> > > https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> > > TestNG's concept of test groups could let us easily
> create tests that
> > > are expected to fail, and exclude them from test runs until the
> > > corresponding behavior is fixed / feature is implemented.
> What do you
> > > guys think about moving to TestNG? I believe that the
> work involved is
> > > minimal; there are some conversion tools / compatibility modes or
> > > something.
> > >
> > > -Patrick
> > >
> > > --
> > > Patrick Linskey
> > > BEA Systems, Inc.
> > >
> > >
> ______________________________________________________________________
> > > _
> > > Notice: This email message, together with any attachments, may
> > > contain
> > > information of BEA Systems, Inc., its subsidiaries and
> > > affiliated
> > > entities, that may be confidential, proprietary, copyrighted
> > > and/or
> > > legally privileged, and is intended solely for the use of the
> > > individual
> > > or entity named in this message. If you are not the intended
> > > recipient,
> > > and have received this message in error, please immediately return
> > > this
> > > by email and then delete it.
> >
> >
>
RE: [DISCUSS] Move to TestNG?
Posted by Patrick Linskey <pl...@bea.com>.
Cedric alleges that the IntelliJ integration is on par with JUnit in the
current version, and Hani claims that upcoming IntelliJs will have even
better support out-of-the-box.
Personally, I run the tests via Maven.
-Patrick
--
Patrick Linskey
BEA Systems, Inc.
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
> -----Original Message-----
> From: Jim Cullison [mailto:jim.cullison@gmail.com]
> Sent: Thursday, February 01, 2007 12:21 PM
> To: open-jpa-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Move to TestNG?
>
> I am using TestNG with IntelliJ. See
> http://testng.org/doc/idea.html (I
> don't use OpenJPA yet, but I plan to on a personal project so
> I'm reading
> the list). My project uses Apache MINA and TestNG has been
> good for working
> with that backbone. One feature that comes to mind is the
> timeout property
> on the Test tag: @Test(timeOut = x).
>
> There is a tool (http://testng.org/doc/migrating.html) that automates
> converting JUnit tests to TestNG.
>
> Jim Cullison
>
> On 2/1/07, Kevin Sutter <kw...@gmail.com> wrote:
> >
> > I see that they have a plugin for Eclipse, but I haven't
> tried it yet. It
> > doesn't look like it supports IntelliJ. On the surface,
> the conversion
> > and
> > usage looks doable and would satisfy the "failed testcase" problem.
> >
> > My question is whether it's worth the effort. If we have a
> testcase that
> > has failures, write appropriate JIRA reports, and comment
> out the failing
> > variations until they get resolved. As part of the JIRA
> resolution, you
> > uncomment the failing test variations.
> >
> > Even if we go with TestNG, it's a similar process. We
> discover a problem,
> > write a JIRA report, and put the testcase in a special
> group that allows
> > failures. When the JIRA report is resolved, we move the
> testcase to a
> > regular group where it is not allowed to fail.
> >
> > Kevin
> >
> > On 2/1/07, Dain Sundstrom <da...@iq80.com> wrote:
> > >
> > > My only concern is IDE support and specifically the IDE I use
> > > intellij. Does this work as a test case type in IntelliJ
> and Eclipse?
> > >
> > > If so, I don't really care which framework we use.
> > >
> > > -dain
> > >
> > > On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
> > >
> > > > Hi,
> > > >
> > > > According to the discussion at
> > > > https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> > > > TestNG's concept of test groups could let us easily
> create tests that
> > > > are expected to fail, and exclude them from test runs until the
> > > > corresponding behavior is fixed / feature is
> implemented. What do you
> > > > guys think about moving to TestNG? I believe that the
> work involved is
> > > > minimal; there are some conversion tools /
> compatibility modes or
> > > > something.
> > > >
> > > > -Patrick
> > > >
> > > > --
> > > > Patrick Linskey
> > > > BEA Systems, Inc.
> > > >
> > > >
> ______________________________________________________________________
> > > > _
> > > > Notice: This email message, together with any attachments, may
> > > > contain
> > > > information of BEA Systems, Inc., its subsidiaries and
> > > > affiliated
> > > > entities, that may be confidential, proprietary, copyrighted
> > > > and/or
> > > > legally privileged, and is intended solely for the use of the
> > > > individual
> > > > or entity named in this message. If you are not the intended
> > > > recipient,
> > > > and have received this message in error, please
> immediately return
> > > > this
> > > > by email and then delete it.
> > >
> > >
> >
> >
>
Re: [DISCUSS] Move to TestNG?
Posted by Jim Cullison <ji...@gmail.com>.
I am using TestNG with IntelliJ. See http://testng.org/doc/idea.html (I
don't use OpenJPA yet, but I plan to on a personal project so I'm reading
the list). My project uses Apache MINA and TestNG has been good for working
with that backbone. One feature that comes to mind is the timeout property
on the Test tag: @Test(timeOut = x).
There is a tool (http://testng.org/doc/migrating.html) that automates
converting JUnit tests to TestNG.
Jim Cullison
On 2/1/07, Kevin Sutter <kw...@gmail.com> wrote:
>
> I see that they have a plugin for Eclipse, but I haven't tried it yet. It
> doesn't look like it supports IntelliJ. On the surface, the conversion
> and
> usage looks doable and would satisfy the "failed testcase" problem.
>
> My question is whether it's worth the effort. If we have a testcase that
> has failures, write appropriate JIRA reports, and comment out the failing
> variations until they get resolved. As part of the JIRA resolution, you
> uncomment the failing test variations.
>
> Even if we go with TestNG, it's a similar process. We discover a problem,
> write a JIRA report, and put the testcase in a special group that allows
> failures. When the JIRA report is resolved, we move the testcase to a
> regular group where it is not allowed to fail.
>
> Kevin
>
> On 2/1/07, Dain Sundstrom <da...@iq80.com> wrote:
> >
> > My only concern is IDE support and specifically the IDE I use
> > intellij. Does this work as a test case type in IntelliJ and Eclipse?
> >
> > If so, I don't really care which framework we use.
> >
> > -dain
> >
> > On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
> >
> > > Hi,
> > >
> > > According to the discussion at
> > > https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> > > TestNG's concept of test groups could let us easily create tests that
> > > are expected to fail, and exclude them from test runs until the
> > > corresponding behavior is fixed / feature is implemented. What do you
> > > guys think about moving to TestNG? I believe that the work involved is
> > > minimal; there are some conversion tools / compatibility modes or
> > > something.
> > >
> > > -Patrick
> > >
> > > --
> > > Patrick Linskey
> > > BEA Systems, Inc.
> > >
> > > ______________________________________________________________________
> > > _
> > > Notice: This email message, together with any attachments, may
> > > contain
> > > information of BEA Systems, Inc., its subsidiaries and
> > > affiliated
> > > entities, that may be confidential, proprietary, copyrighted
> > > and/or
> > > legally privileged, and is intended solely for the use of the
> > > individual
> > > or entity named in this message. If you are not the intended
> > > recipient,
> > > and have received this message in error, please immediately return
> > > this
> > > by email and then delete it.
> >
> >
>
>
Re: [DISCUSS] Move to TestNG?
Posted by Kevin Sutter <kw...@gmail.com>.
I see that they have a plugin for Eclipse, but I haven't tried it yet. It
doesn't look like it supports IntelliJ. On the surface, the conversion and
usage looks doable and would satisfy the "failed testcase" problem.
My question is whether it's worth the effort. If we have a testcase that
has failures, write appropriate JIRA reports, and comment out the failing
variations until they get resolved. As part of the JIRA resolution, you
uncomment the failing test variations.
Even if we go with TestNG, it's a similar process. We discover a problem,
write a JIRA report, and put the testcase in a special group that allows
failures. When the JIRA report is resolved, we move the testcase to a
regular group where it is not allowed to fail.
Kevin
On 2/1/07, Dain Sundstrom <da...@iq80.com> wrote:
>
> My only concern is IDE support and specifically the IDE I use
> intellij. Does this work as a test case type in IntelliJ and Eclipse?
>
> If so, I don't really care which framework we use.
>
> -dain
>
> On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
>
> > Hi,
> >
> > According to the discussion at
> > https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> > TestNG's concept of test groups could let us easily create tests that
> > are expected to fail, and exclude them from test runs until the
> > corresponding behavior is fixed / feature is implemented. What do you
> > guys think about moving to TestNG? I believe that the work involved is
> > minimal; there are some conversion tools / compatibility modes or
> > something.
> >
> > -Patrick
> >
> > --
> > Patrick Linskey
> > BEA Systems, Inc.
> >
> > ______________________________________________________________________
> > _
> > Notice: This email message, together with any attachments, may
> > contain
> > information of BEA Systems, Inc., its subsidiaries and
> > affiliated
> > entities, that may be confidential, proprietary, copyrighted
> > and/or
> > legally privileged, and is intended solely for the use of the
> > individual
> > or entity named in this message. If you are not the intended
> > recipient,
> > and have received this message in error, please immediately return
> > this
> > by email and then delete it.
>
>
Re: [DISCUSS] Move to TestNG?
Posted by Dain Sundstrom <da...@iq80.com>.
My only concern is IDE support and specifically the IDE I use
intellij. Does this work as a test case type in IntelliJ and Eclipse?
If so, I don't really care which framework we use.
-dain
On Feb 1, 2007, at 11:08 AM, Patrick Linskey wrote:
> Hi,
>
> According to the discussion at
> https://issues.apache.org/jira/browse/OPENJPA-123, it looks like
> TestNG's concept of test groups could let us easily create tests that
> are expected to fail, and exclude them from test runs until the
> corresponding behavior is fixed / feature is implemented. What do you
> guys think about moving to TestNG? I believe that the work involved is
> minimal; there are some conversion tools / compatibility modes or
> something.
>
> -Patrick
>
> --
> Patrick Linskey
> BEA Systems, Inc.
>
> ______________________________________________________________________
> _
> Notice: This email message, together with any attachments, may
> contain
> information of BEA Systems, Inc., its subsidiaries and
> affiliated
> entities, that may be confidential, proprietary, copyrighted
> and/or
> legally privileged, and is intended solely for the use of the
> individual
> or entity named in this message. If you are not the intended
> recipient,
> and have received this message in error, please immediately return
> this
> by email and then delete it.