TestNG groups and exclusions

[Chris Custine <ch...@gmail.com>]

Hi All,
Is anyone else interested in fixes for the testng configuration so that we can use the groups and excluded groups instead of the file name based approach from surefire?  The bug that caused this to not work has long been fixed in testng and I have a local branch where I have this working.  I just wonder if anyone else is interested.  The main benefits would be a little more flexibility in arranging and executing test groups, and it kind of opens up a bit more of the flexibility of testng composition.

I will open a PR with my code later today but I just wanted to throw this out there for comments.

Thanks,
Chris

-- 
Chris Custine

Re: TestNG groups and exclusions

[Chris Custine <ch...@gmail.com>]

Yeah, "long" tests and things like that are exactly what I was thinking. 

I should add that there are several nuances that will require a review of all downstream labs projects etc., that inherit from jclouds-parent. Testng is flexible with regard to these groups, but certain things like @BeforeXXX annotations in parent test classes also respect group names and you must specifically name the group or tell them to run every time or else the bare @BeforeXXX annotations won't run. I see evidence from early jclouds code that this was known, but some more recent work doesn't take that into account. 

I am also capturing test statistics to compare the old and new approach to make sure we get the same tests run :-)

I'll try to get this published in a PR soon so you guys can see the changes required and discuss options. 

Thanks, 
Chris 

--  
Chris Custine


On July 25, 2014 at 11:29:43 AM, Ignasi Barrera (nacx@apache.org) wrote:
> +1000 that would be highly appreciated.
>  
> There has been a recent discussion to include a new kind of test group
> ("long" tests) and this will make that config much cleaner.
> El 25/07/2014 19:05, "Andrew Phillips" escribió:
>  
> > I will open a PR with my code later today but I just wanted to throw this
> >> out there for comments.
> >>
> >
> > +1! I don't think there's any particular reason for us to stick with the
> > filename-based approach if this works now...
> >
> > Thanks!
> >
> > ap
> >
>  

Re: TestNG groups and exclusions

[Ignasi Barrera <na...@apache.org>]

Awesome. You rock! :)
El 25/07/2014 19:54, "Chris Custine" <ch...@gmail.com> escribió:

> Yeah, “long” tests and things like that are exactly what I was thinking.
>
> I should add that there are several nuances that will require a review of
> all downstream labs projects etc., that inherit from jclouds-parent. Testng
> is flexible with regard to these groups, but certain things like @BeforeXXX
> annotations in parent test classes also respect group names and you must
> specifically name the group or tell them to run every time or else the bare
> @BeforeXXX annotations won't run. I see evidence from early jclouds code
> that this was known, but some more recent work doesn't take that into
> account.
>
> I am also capturing test statistics to compare the old and new approach to
> make sure we get the same tests run :-)
>
> I'll try to get this published in a PR so you guys can see the changes
> required and discuss options.
>
> Thanks,
> Chris
> --
> Chris Custine
>
>
> On July 25, 2014 at 11:29:43 AM, Ignasi Barrera (nacx@apache.org) wrote:
>
> +1000 that would be highly appreciated.
>
> There has been a recent discussion to include a new kind of test group
> ("long" tests) and this will make that config much cleaner.
> El 25/07/2014 19:05, "Andrew Phillips" <ap...@qrmedia.com> escribió:
>
> > I will open a PR with my code later today but I just wanted to throw
> this
> >> out there for comments.
> >>
> >
> > +1! I don't think there's any particular reason for us to stick with the
> > filename-based approach if this works now...
> >
> > Thanks!
> >
> > ap
> >
>
>

Re: TestNG groups and exclusions

[Ignasi Barrera <na...@apache.org>]

+1000 that would be highly appreciated.

There has been a recent discussion to include a new kind of test group
("long" tests) and this will make that config much cleaner.
El 25/07/2014 19:05, "Andrew Phillips" <ap...@qrmedia.com> escribió:

> I will open a PR with my code later today but I just wanted to throw  this
>> out there for comments.
>>
>
> +1! I don't think there's any particular reason for us to stick with the
> filename-based approach if this works now...
>
> Thanks!
>
> ap
>

Re: TestNG groups and exclusions

[Andrew Phillips <ap...@qrmedia.com>]

> I will open a PR with my code later today but I just wanted to throw  
>  this out there for comments.

+1! I don't think there's any particular reason for us to stick with  
the filename-based approach if this works now...

Thanks!

ap