You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@groovy.apache.org by Sean LeBlanc <se...@icd-tech.com> on 2019/06/20 17:02:05 UTC
Code coverage tools?
What are people using currently to get accurate code coverage from new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as what this thread is talking about:
https://github.com/cobertura/cobertura/issues/184
Does anyone have good experiences with getting more accurate code coverage numbers with Groovy > 2.0.8? And what route did you take?
Re: Code coverage tools?
Posted by Sean Leblanc <se...@icd-tech.com>.
Seems I’m getting better numbers from Clover/Openclover for our projects (quite a bit better numbers!), however, it fails on one of our modules with an obscure message:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':icd-common:test'.
> java.lang.reflect.GenericSignatureFormatError: Signature Parse error: Failure to make progress!
Remaining input: .TradeTransaction:Ltradeticket/TradeTransaction;>(Ljava/util/Set<Ltradeticket/TradeTransaction;>;Ljava/lang/Integer;)Ljava/util/Set<Ltradeticket/TradeTransaction;>;
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
From: Sean Leblanc <se...@icd-tech.com>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>
Date: Friday, June 21, 2019 at 9:45 AM
To: "users@groovy.apache.org" <us...@groovy.apache.org>, Scott Hickey <js...@gmail.com>
Subject: Re: Code coverage tools?
What does the forcedModules do?
From: "Corum, Michael" <mc...@rgare.com>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>
Date: Friday, June 21, 2019 at 5:43 AM
To: "users@groovy.apache.org" <us...@groovy.apache.org>, Scott Hickey <js...@gmail.com>
Subject: Re: Code coverage tools?
Actually in my case, @CompileStatic helped improve accuracy of coverage for Cobertura. For Groovy/Dropwizard, I using this in my build.gradle:
Plugins {
id "net.saliman.cobertura" version "2.5.4"
}
And then later in my build.gradle, I also include:
configurations.all {
resolutionStrategy {
forcedModules = ['org.ow2.asm:asm:5.0.3', 'org.ow2.asm:asm-tree:5.0.3', 'org.ow2.asm:asm-commons:5.0.3', 'org.ow2.asm:asm-util:5.0.3', 'org.ow2.asm:asm-analysis:5.0.3']
}
}
I probably need to update the versions on those but that helped the accuracy for Cobertura quite a bit.
Michael Corum
VP, Technical Architecture Solutions
RGA Reinsurance Company
16600 Swingley Ridge Road
Chesterfield, Missouri 6301701706
T 636.736.7066
www.rgare.com
From: Søren Berg Glasius <so...@glasius.dk>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>
Date: Friday, June 21, 2019 at 1:04 AM
To: "users@groovy.apache.org" <us...@groovy.apache.org>, Scott Hickey <js...@gmail.com>
Subject: Re: Code coverage tools?
External e-mail. Use caution! / Courriel externe. Faites attention!
________________________________
@Scott Hickey<ma...@gmail.com> but can you make it work with @CompileStatic and the Elvis operator? We are having big struggles to get that working. (In a Grails 3.3.x application)
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
On Fri, 21 Jun 2019 at 00:31, Scott Hickey <js...@gmail.com>> wrote:
We are using Clover - it is open source now. We have found it works much better for us at Mutual of Omaha than JaCoCo or Cobertura.
Scott Hickey
Mutual of Omaha
On Thu, Jun 20, 2019 at 12:02 PM Sean LeBlanc <se...@icd-tech.com>> wrote:
What are people using currently to get accurate code coverage from new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as what this thread is talking about:
https://github.com/cobertura/cobertura/issues/184<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cobertura_cobertura_issues_184&d=DwMFaQ&c=5uPv0lijNz76uSeaN5P0Zw&r=rh3Qrw7azSI9xkZZ-a8EEw&m=p4nbyKOuvpOHHXv6u3ddQHdcrJqAeyq5xSfJhWp1BOc&s=VyqZbdEQzSqkh0b0fC9LKCqP6likq_eHkVjwWCHpmws&e=>
Does anyone have good experiences with getting more accurate code coverage numbers with Groovy > 2.0.8? And what route did you take?
** EXTERNAL **
________________________________
** EXTERNAL **
________________________________
Re: Code coverage tools?
Posted by Sean Leblanc <se...@icd-tech.com>.
What does the forcedModules do?
From: "Corum, Michael" <mc...@rgare.com>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>
Date: Friday, June 21, 2019 at 5:43 AM
To: "users@groovy.apache.org" <us...@groovy.apache.org>, Scott Hickey <js...@gmail.com>
Subject: Re: Code coverage tools?
Actually in my case, @CompileStatic helped improve accuracy of coverage for Cobertura. For Groovy/Dropwizard, I using this in my build.gradle:
Plugins {
id "net.saliman.cobertura" version "2.5.4"
}
And then later in my build.gradle, I also include:
configurations.all {
resolutionStrategy {
forcedModules = ['org.ow2.asm:asm:5.0.3', 'org.ow2.asm:asm-tree:5.0.3', 'org.ow2.asm:asm-commons:5.0.3', 'org.ow2.asm:asm-util:5.0.3', 'org.ow2.asm:asm-analysis:5.0.3']
}
}
I probably need to update the versions on those but that helped the accuracy for Cobertura quite a bit.
Michael Corum
VP, Technical Architecture Solutions
RGA Reinsurance Company
16600 Swingley Ridge Road
Chesterfield, Missouri 6301701706
T 636.736.7066
www.rgare.com
From: Søren Berg Glasius <so...@glasius.dk>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>
Date: Friday, June 21, 2019 at 1:04 AM
To: "users@groovy.apache.org" <us...@groovy.apache.org>, Scott Hickey <js...@gmail.com>
Subject: Re: Code coverage tools?
External e-mail. Use caution! / Courriel externe. Faites attention!
________________________________
@Scott Hickey<ma...@gmail.com> but can you make it work with @CompileStatic and the Elvis operator? We are having big struggles to get that working. (In a Grails 3.3.x application)
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
On Fri, 21 Jun 2019 at 00:31, Scott Hickey <js...@gmail.com>> wrote:
We are using Clover - it is open source now. We have found it works much better for us at Mutual of Omaha than JaCoCo or Cobertura.
Scott Hickey
Mutual of Omaha
On Thu, Jun 20, 2019 at 12:02 PM Sean LeBlanc <se...@icd-tech.com>> wrote:
What are people using currently to get accurate code coverage from new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as what this thread is talking about:
https://github.com/cobertura/cobertura/issues/184<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cobertura_cobertura_issues_184&d=DwMFaQ&c=5uPv0lijNz76uSeaN5P0Zw&r=rh3Qrw7azSI9xkZZ-a8EEw&m=p4nbyKOuvpOHHXv6u3ddQHdcrJqAeyq5xSfJhWp1BOc&s=VyqZbdEQzSqkh0b0fC9LKCqP6likq_eHkVjwWCHpmws&e=>
Does anyone have good experiences with getting more accurate code coverage numbers with Groovy > 2.0.8? And what route did you take?
** EXTERNAL **
________________________________
Re: Code coverage tools?
Posted by "Corum, Michael" <mc...@rgare.com>.
Actually in my case, @CompileStatic helped improve accuracy of coverage for Cobertura. For Groovy/Dropwizard, I using this in my build.gradle:
Plugins {
id "net.saliman.cobertura" version "2.5.4"
}
And then later in my build.gradle, I also include:
configurations.all {
resolutionStrategy {
forcedModules = ['org.ow2.asm:asm:5.0.3', 'org.ow2.asm:asm-tree:5.0.3', 'org.ow2.asm:asm-commons:5.0.3', 'org.ow2.asm:asm-util:5.0.3', 'org.ow2.asm:asm-analysis:5.0.3']
}
}
I probably need to update the versions on those but that helped the accuracy for Cobertura quite a bit.
Michael Corum
VP, Technical Architecture Solutions
RGA Reinsurance Company
16600 Swingley Ridge Road
Chesterfield, Missouri 6301701706
T 636.736.7066
www.rgare.com
From: Søren Berg Glasius <so...@glasius.dk>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>
Date: Friday, June 21, 2019 at 1:04 AM
To: "users@groovy.apache.org" <us...@groovy.apache.org>, Scott Hickey <js...@gmail.com>
Subject: Re: Code coverage tools?
External e-mail. Use caution! / Courriel externe. Faites attention!
________________________________
@Scott Hickey<ma...@gmail.com> but can you make it work with @CompileStatic and the Elvis operator? We are having big struggles to get that working. (In a Grails 3.3.x application)
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
On Fri, 21 Jun 2019 at 00:31, Scott Hickey <js...@gmail.com>> wrote:
We are using Clover - it is open source now. We have found it works much better for us at Mutual of Omaha than JaCoCo or Cobertura.
Scott Hickey
Mutual of Omaha
On Thu, Jun 20, 2019 at 12:02 PM Sean LeBlanc <se...@icd-tech.com>> wrote:
What are people using currently to get accurate code coverage from new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as what this thread is talking about:
https://github.com/cobertura/cobertura/issues/184<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cobertura_cobertura_issues_184&d=DwMFaQ&c=5uPv0lijNz76uSeaN5P0Zw&r=rh3Qrw7azSI9xkZZ-a8EEw&m=p4nbyKOuvpOHHXv6u3ddQHdcrJqAeyq5xSfJhWp1BOc&s=VyqZbdEQzSqkh0b0fC9LKCqP6likq_eHkVjwWCHpmws&e=>
Does anyone have good experiences with getting more accurate code coverage numbers with Groovy > 2.0.8? And what route did you take?
Re: Code coverage tools?
Posted by Søren Berg Glasius <so...@glasius.dk>.
@Scott Hickey <js...@gmail.com> but can you make it work
with @CompileStatic and the Elvis operator? We are having big struggles to
get that working. (In a Grails 3.3.x application)
Best regards / Med venlig hilsen,
Søren Berg Glasius
Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
On Fri, 21 Jun 2019 at 00:31, Scott Hickey <js...@gmail.com> wrote:
> We are using Clover - it is open source now. We have found it works much
> better for us at Mutual of Omaha than JaCoCo or Cobertura.
>
> Scott Hickey
> Mutual of Omaha
>
> On Thu, Jun 20, 2019 at 12:02 PM Sean LeBlanc <se...@icd-tech.com>
> wrote:
>
>> What are people using currently to get accurate code coverage from
>> new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as
>> what this thread is talking about:
>>
>>
>>
>> https://github.com/cobertura/cobertura/issues/184
>>
>>
>>
>>
>>
>>
>>
>> Does anyone have good experiences with getting more accurate code
>> coverage numbers with Groovy > 2.0.8? And what route did you take?
>>
>
Re: Code coverage tools?
Posted by Scott Hickey <js...@gmail.com>.
We are using Clover - it is open source now. We have found it works much
better for us at Mutual of Omaha than JaCoCo or Cobertura.
Scott Hickey
Mutual of Omaha
On Thu, Jun 20, 2019 at 12:02 PM Sean LeBlanc <se...@icd-tech.com>
wrote:
> What are people using currently to get accurate code coverage from
> new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as
> what this thread is talking about:
>
>
>
> https://github.com/cobertura/cobertura/issues/184
>
>
>
>
>
>
>
> Does anyone have good experiences with getting more accurate code coverage
> numbers with Groovy > 2.0.8? And what route did you take?
>
Re: Code coverage tools?
Posted by "Corum, Michael" <mc...@rgare.com>.
Typically, if I have something (a microservice) that is 100% covered both for lines and branches, neither JaCoCo or Cobertura will show it properly unless I do some things. Here are some examples of the type of things I have to do:
* Re-order methods in a class. This can make as much as 15% or more difference in the overall code coverage.
* Change to static typing on some returns from methods (but not all). This can make a large difference.
* Some refactors help coverage. In particular, turning “if’s” into types can help since the coverage tools usually show way more branch options than the expressions would indicate. For instance, if you had something like “if (x == y)” that might say there are something like 13 possible branch options due to odd generated code related to the “if.”
I have quite a few projects where I’ve gotten Cobertura to correctly show 100% line/branch coverage and many with it showing 99% so I set my build thresholds to around 97 or 98 to make sure I catch anything changing that but it takes some time to arrange things to be correctly measured.
Michael Corum
VP, Technical Architecture Solutions
RGA Reinsurance Company
16600 Swingley Ridge Road
Chesterfield, Missouri 6301701706
T 636.736.7066
www.rgare.com
From: Paul King <pa...@asert.com.au>
Reply-To: "users@groovy.apache.org" <us...@groovy.apache.org>, "paulk@asert.com.au" <pa...@asert.com.au>
Date: Thursday, June 20, 2019 at 4:15 PM
To: "users@groovy.apache.org" <us...@groovy.apache.org>
Subject: Re: Code coverage tools?
External e-mail. Use caution! / Courriel externe. Faites attention!
________________________________
Groovy has done a lot of work recently to mark generated methods with an @Generated annotation. We have other generated methods which are synthetic methods. I know JaCoCo has been making changes to ensure that such methods aren't included in coverage analysis. I am not sure whether Cobertura has done any updates along those lines. It did appear for a while that activity on Cobertura was decreasing but I notice there was some activity a few months ago. It is perhaps worth asking that community about whether they have or can consider making similar changes, otherwise perhaps switching to JaCoCo might be the way to go.
Cheers, Paul.
On Fri, Jun 21, 2019 at 3:02 AM Sean LeBlanc <se...@icd-tech.com>> wrote:
What are people using currently to get accurate code coverage from new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as what this thread is talking about:
https://github.com/cobertura/cobertura/issues/184<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cobertura_cobertura_issues_184&d=DwMFaQ&c=5uPv0lijNz76uSeaN5P0Zw&r=rh3Qrw7azSI9xkZZ-a8EEw&m=FjAAb8X6cAC-Y7QDFj98g3FXEKwft7b3XMPHfp053pk&s=ETpCgOjg6J5tjlse1mis1tpk5M2H1v2Hl4_xA0qPf4g&e=>
Does anyone have good experiences with getting more accurate code coverage numbers with Groovy > 2.0.8? And what route did you take?
Re: Code coverage tools?
Posted by Paul King <pa...@asert.com.au>.
Groovy has done a lot of work recently to mark generated methods with an
@Generated annotation. We have other generated methods which are synthetic
methods. I know JaCoCo has been making changes to ensure that such methods
aren't included in coverage analysis. I am not sure whether Cobertura has
done any updates along those lines. It did appear for a while that activity
on Cobertura was decreasing but I notice there was some activity a few
months ago. It is perhaps worth asking that community about whether they
have or can consider making similar changes, otherwise perhaps switching to
JaCoCo might be the way to go.
Cheers, Paul.
On Fri, Jun 21, 2019 at 3:02 AM Sean LeBlanc <se...@icd-tech.com>
wrote:
> What are people using currently to get accurate code coverage from
> new(ish) versions of Groovy? We are on 2.4.10 and I see similar issues as
> what this thread is talking about:
>
>
>
> https://github.com/cobertura/cobertura/issues/184
>
>
>
>
>
>
>
> Does anyone have good experiences with getting more accurate code coverage
> numbers with Groovy > 2.0.8? And what route did you take?
>