You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Duncan Grant <du...@cloudsoftcorp.com> on 2015/01/14 11:26:15 UTC
Code coverage with cobertura
I'm trying to get code coverage working for brooklyn.
As a first task I've upgraded the version of cobertura we are using and
fixed some issues with paths.
This means that you can run mvn install -P Coverage and it will produce
coverage reports for individual projects. I'll create a PR today.
I've tried swapping the ant-run approach that is currently used for just
the maven plugin but there were some errors that I couldn't resolve and I
don't think it's worth the effort.
I'd also like to have a single report for coverage across the project. I
think I just need a task that runs after all the projects have been built
but I'm not sure how to do that as the parent project seems to run first.
I haven't much experience with maven so I'm probably missing something
obvious.
Does anyone have an opinion on whether we should be producing xml reports
or html reports or both? The html report is a nice human readable report.
The xml report can be integrated (using a plugin) with jenkins to show
trends but I'm not sure how helpful that is.
I'm also a bit stuck getting the integration tests to run (due to a failed
dependancy over night). Also it seems to have been a while since they
complete successfully on http://brooklyn.builds.cloudsoftcorp.com/ . So I
don't have any idea what coverage will look like then.
Any thoughts would be appreciated
Duncan
Re: Code coverage with cobertura
Posted by Duncan Grant <du...@cloudsoftcorp.com>.
Aled,
Here's the top-line coverage for each project for unit tests
api/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="523" branches-covered="0" branches-valid="130"
complexity="1.14520202020202" version="1.9.4.1" timestamp="1421328275796">
camp/camp-base/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328287176">
camp/camp-server/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328298297">
core/target/site/cobertura/coverage.xml :
<coverage line-rate="0.640080146231721" branch-rate="0.5358428549068712"
lines-covered="18209" lines-valid="28448" branches-covered="6847"
branches-valid="12778" complexity="2.3311258278145695" version="1.9.4.1"
timestamp="1421328399007">
examples/global-web-fabric/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="1.0" lines-covered="0"
lines-valid="40" branches-covered="0" branches-valid="0" complexity="1.0"
version="1.9.4.1" timestamp="1421328859307">
examples/simple-messaging-pubsub/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="87" branches-covered="0" branches-valid="6"
complexity="1.8333333333333333" version="1.9.4.1" timestamp="1421328864312">
examples/simple-nosql-cluster/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="257" branches-covered="0" branches-valid="10"
complexity="1.3043478260869565" version="1.9.4.1" timestamp="1421328869077">
examples/simple-web-cluster/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="177" branches-covered="0" branches-valid="2"
complexity="1.1111111111111112" version="1.9.4.1" timestamp="1421328854477">
examples/webapps/hello-world-sql/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328847126">
examples/webapps/hello-world-webapp/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328845013">
locations/jclouds/target/site/cobertura/coverage.xml :
<coverage line-rate="0.21518987341772153" branch-rate="0.13455882352941176"
lines-covered="629" lines-valid="2923" branches-covered="183"
branches-valid="1360" complexity="2.8916155419222904" version="1.9.4.1"
timestamp="1421328479641">
policy/target/site/cobertura/coverage.xml :
<coverage line-rate="0.8294375456537618" branch-rate="0.6083202511773941"
lines-covered="2271" lines-valid="2738" branches-covered="775"
branches-valid="1274" complexity="2.4641350210970465" version="1.9.4.1"
timestamp="1421328452571">
software/base/target/site/cobertura/coverage.xml :
<coverage line-rate="0.45638406537282944" branch-rate="0.35834333733493395"
lines-covered="2234" lines-valid="4895" branches-covered="597"
branches-valid="1666" complexity="2.194888178913738" version="1.9.4.1"
timestamp="1421328526470">
software/database/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="721" branches-covered="0" branches-valid="138"
complexity="1.4970059880239521" version="1.9.4.1" timestamp="1421328585506">
software/messaging/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="1081" branches-covered="0" branches-valid="148"
complexity="1.2607449856733524" version="1.9.4.1" timestamp="1421328577885">
software/monitoring/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="99" branches-covered="0" branches-valid="12"
complexity="1.3043478260869565" version="1.9.4.1" timestamp="1421328608506">
software/network/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="265" branches-covered="0" branches-valid="52"
complexity="1.4166666666666667" version="1.9.4.1" timestamp="1421328534584">
software/nosql/target/site/cobertura/coverage.xml :
<coverage line-rate="0.16019818331957061" branch-rate="0.21768707482993196"
lines-covered="582" lines-valid="3633" branches-covered="256"
branches-valid="1176" complexity="2.0057887120115776" version="1.9.4.1"
timestamp="1421328600464">
software/osgi/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="181" branches-covered="0" branches-valid="34" complexity="1.6"
version="1.9.4.1" timestamp="1421328539923">
software/webapp/target/site/cobertura/coverage.xml :
<coverage line-rate="0.23116883116883116" branch-rate="0.21767241379310345"
lines-covered="623" lines-valid="2695" branches-covered="202"
branches-valid="928" complexity="2.0690298507462686" version="1.9.4.1"
timestamp="1421328568254">
storage/hazelcast/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="47" branches-covered="0" branches-valid="4"
complexity="1.2352941176470589" version="1.9.4.1" timestamp="1421328459030">
usage/all/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328811306">
usage/archetypes/quickstart/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328842718">
usage/camp/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328650960">
usage/cli/target/site/cobertura/coverage.xml :
<coverage line-rate="0.33574007220216606" branch-rate="0.22580645161290322"
lines-covered="279" lines-valid="831" branches-covered="70"
branches-valid="310" complexity="3.3636363636363638" version="1.9.4.1"
timestamp="1421328802377">
usage/dist/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328827806">
usage/jsgui/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328743328">
usage/launcher/target/site/cobertura/coverage.xml :
<coverage line-rate="0.5664516129032258" branch-rate="0.49158249158249157"
lines-covered="439" lines-valid="775" branches-covered="146"
branches-valid="297" complexity="2.7540983606557377" version="1.9.4.1"
timestamp="1421328788133">
usage/logback-includes/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="11" branches-covered="0" branches-valid="2" complexity="1.2"
version="1.9.4.1" timestamp="1421328251510">
usage/logback-xml/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328292376">
usage/qa/target/site/cobertura/coverage.xml :
<coverage line-rate="0.12598425196850394" branch-rate="0.13157894736842105"
lines-covered="80" lines-valid="635" branches-covered="25"
branches-valid="190" complexity="2.5301204819277108" version="1.9.4.1"
timestamp="1421328876342">
usage/rest-api/target/site/cobertura/coverage.xml :
<coverage line-rate="0.18016759776536312" branch-rate="0.16417910447761194"
lines-covered="129" lines-valid="716" branches-covered="66"
branches-valid="402" complexity="1.659025787965616" version="1.9.4.1"
timestamp="1421328656401">
usage/rest-client/target/site/cobertura/coverage.xml :
<coverage line-rate="0.14285714285714285" branch-rate="0.0"
lines-covered="4" lines-valid="28" branches-covered="0" branches-valid="4"
complexity="1.2105263157894737" version="1.9.4.1" timestamp="1421328722761">
usage/rest-server/target/site/cobertura/coverage.xml :
<coverage line-rate="0.6810602821718683" branch-rate="0.4912959381044487"
lines-covered="1593" lines-valid="2339" branches-covered="508"
branches-valid="1034" complexity="3.0214477211796247" version="1.9.4.1"
timestamp="1421328710813">
usage/test-support/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="614" branches-covered="0" branches-valid="306"
complexity="1.8611111111111112" version="1.9.4.1" timestamp="1421328282364">
utils/common/target/site/cobertura/coverage.xml :
<coverage line-rate="0.5037804076265615" branch-rate="0.4570335491465568"
lines-covered="3065" lines-valid="6084" branches-covered="1553"
branches-valid="3398" complexity="2.4800979791794244" version="1.9.4.1"
timestamp="1421328262980">
utils/groovy/target/site/cobertura/coverage.xml :
<coverage line-rate="0.3781094527363184" branch-rate="0.30434782608695654"
lines-covered="76" lines-valid="201" branches-covered="84"
branches-valid="276" complexity="1.0" version="1.9.4.1"
timestamp="1421328272060">
utils/jmx/jmxmp-ssl-agent/target/site/cobertura/coverage.xml :
<coverage line-rate="0.5338345864661654" branch-rate="0.38636363636363635"
lines-covered="71" lines-valid="133" branches-covered="17"
branches-valid="44" complexity="3.1052631578947367" version="1.9.4.1"
timestamp="1421328490860">
utils/jmx/jmxrmi-agent/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="56" branches-covered="0" branches-valid="14" complexity="2.4"
version="1.9.4.1" timestamp="1421328494007">
utils/rest-swagger/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="126" branches-covered="0" branches-valid="86" complexity="3.7"
version="1.9.4.1" timestamp="1421328290376">
utils/test-support/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="290" branches-covered="0" branches-valid="108"
complexity="2.204081632653061" version="1.9.4.1" timestamp="1421328248357">
And here are the partial numbers for integration tests:
api/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="523" branches-covered="0" branches-valid="130"
complexity="1.14520202020202" version="1.9.4.1" timestamp="1421255489048">
camp/camp-base/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421255496879">
camp/camp-server/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421255504839">
examples/global-web-fabric/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="1.0" lines-covered="0"
lines-valid="40" branches-covered="0" branches-valid="0" complexity="1.0"
version="1.9.4.1" timestamp="1421262294641">
examples/simple-messaging-pubsub/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="87" branches-covered="0" branches-valid="6"
complexity="1.8333333333333333" version="1.9.4.1" timestamp="1421262300045">
examples/simple-nosql-cluster/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="257" branches-covered="0" branches-valid="10"
complexity="1.3043478260869565" version="1.9.4.1" timestamp="1421262304724">
examples/simple-web-cluster/target/site/cobertura/coverage.xml :
<coverage line-rate="0.2937853107344633" branch-rate="0.5"
lines-covered="52" lines-valid="177" branches-covered="1"
branches-valid="2" complexity="1.1111111111111112" version="1.9.4.1"
timestamp="1421262290268">
examples/webapps/hello-world-sql/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262239392">
examples/webapps/hello-world-webapp/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262237740">
locations/jclouds/target/site/cobertura/coverage.xml :
<coverage line-rate="0.10708176530961341" branch-rate="0.03823529411764706"
lines-covered="313" lines-valid="2923" branches-covered="52"
branches-valid="1360" complexity="2.8916155419222904" version="1.9.4.1"
timestamp="1421256566766">
policy/target/site/cobertura/coverage.xml :
<coverage line-rate="0.5467494521548576" branch-rate="0.35949764521193095"
lines-covered="1497" lines-valid="2738" branches-covered="458"
branches-valid="1274" complexity="2.4641350210970465" version="1.9.4.1"
timestamp="1421256487045">
software/network/target/site/cobertura/coverage.xml :
<coverage line-rate="0.6679245283018868" branch-rate="0.6153846153846154"
lines-covered="177" lines-valid="265" branches-covered="32"
branches-valid="52" complexity="1.4166666666666667" version="1.9.4.1"
timestamp="1421257862576">
software/osgi/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="181" branches-covered="0" branches-valid="34" complexity="1.6"
version="1.9.4.1" timestamp="1421257867961">
storage/hazelcast/target/site/cobertura/coverage.xml :
<coverage line-rate="0.5319148936170213" branch-rate="0.25"
lines-covered="25" lines-valid="47" branches-covered="1" branches-valid="4"
complexity="1.2352941176470589" version="1.9.4.1" timestamp="1421256552341">
usage/archetypes/quickstart/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262235938">
usage/cli/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="831" branches-covered="0" branches-valid="310"
complexity="3.3636363636363638" version="1.9.4.1" timestamp="1421262219150">
usage/jsgui/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262131024">
usage/launcher/target/site/cobertura/coverage.xml :
<coverage line-rate="0.5690322580645162" branch-rate="0.43097643097643096"
lines-covered="441" lines-valid="775" branches-covered="128"
branches-valid="297" complexity="2.7540983606557377" version="1.9.4.1"
timestamp="1421262212380">
usage/logback-includes/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="11" branches-covered="0" branches-valid="2" complexity="1.2"
version="1.9.4.1" timestamp="1421255459645">
usage/logback-xml/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421255500917">
usage/qa/target/site/cobertura/coverage.xml :
<coverage line-rate="0.08346456692913386" branch-rate="0.07368421052631578"
lines-covered="53" lines-valid="635" branches-covered="14"
branches-valid="190" complexity="2.5301204819277108" version="1.9.4.1"
timestamp="1421262312249">
usage/rest-api/target/site/cobertura/coverage.xml :
<coverage line-rate="0.06284916201117319"
branch-rate="0.024875621890547265" lines-covered="45" lines-valid="716"
branches-covered="10" branches-valid="402" complexity="1.659025787965616"
version="1.9.4.1" timestamp="1421262082264">
usage/rest-client/target/site/cobertura/coverage.xml :
<coverage line-rate="0.25" branch-rate="0.0" lines-covered="7"
lines-valid="28" branches-covered="0" branches-valid="4"
complexity="1.2105263157894737" version="1.9.4.1" timestamp="1421262115976">
usage/rest-server/target/site/cobertura/coverage.xml :
<coverage line-rate="0.36896109448482256" branch-rate="0.23404255319148937"
lines-covered="863" lines-valid="2339" branches-covered="242"
branches-valid="1034" complexity="3.0214477211796247" version="1.9.4.1"
timestamp="1421262103160">
usage/test-support/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="614" branches-covered="0" branches-valid="306"
complexity="1.8611111111111112" version="1.9.4.1" timestamp="1421255492975">
utils/common/target/site/cobertura/coverage.xml :
<coverage line-rate="0.14896415652745806" branch-rate="0.08387286639199529"
lines-covered="906" lines-valid="6082" branches-covered="285"
branches-valid="3398" complexity="2.481912936848559" version="1.9.4.1"
timestamp="1421255479993">
utils/groovy/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="201" branches-covered="0" branches-valid="276"
complexity="1.0" version="1.9.4.1" timestamp="1421255486003">
utils/jmx/jmxmp-ssl-agent/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="133" branches-covered="0" branches-valid="44"
complexity="3.1052631578947367" version="1.9.4.1" timestamp="1421256573037">
utils/jmx/jmxrmi-agent/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="56" branches-covered="0" branches-valid="14" complexity="2.4"
version="1.9.4.1" timestamp="1421256575531">
utils/rest-swagger/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="126" branches-covered="0" branches-valid="86" complexity="3.7"
version="1.9.4.1" timestamp="1421255499592">
utils/test-support/target/site/cobertura/coverage.xml :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="290" branches-covered="0" branches-valid="108"
complexity="2.204081632653061" version="1.9.4.1" timestamp="1421255457257">
We can see that we expect the coverage to go up when we add integration
tests however these numbers represent code covered by integration tests in
the same project (I think) so aren't as high as they should be.
On 15 January 2015 at 12:17, Aled Sage <al...@gmail.com> wrote:
> Hi Duncan,
>
> Excellent, code coverage reports will be very useful moving forwards.
>
> ---
> Regarding generating a single report, here are some comments that were
> posted off-list:
>
> Could you accumulate the reports as you go? Rather than running one
> merge at the end do it for each project after the
> post-integration-test or package phase, perhaps. I can imagine how
> you would configure this, can't think how you'd ask for something to
> be run at the end of the build.
>
> Sam
>
> And:
>
> Have you tried the aggregate parameter -
> http://mojo.codehaus.org/cobertura-maven-plugin/
> cobertura-mojo.html#aggregate?
> Here’s an example that should aggregate it’s results:
> https://svn.codehaus.org/mojo/trunk/mojo/cobertura-maven-
> plugin/src/it/mcobertura-65/pom.xml
>
> Svet.
>
> ---
> For xml versus html, If it's easy to say both then might as well do that.
> Otherwise, the html report will be more useful in the short term.
>
> ---
> For integration tests, sounds like they need some tender loving care
> again. I'll try to find some time to look at any failures, but it might not
> be until next week. If anyone else picks up fixing any of these, then
> please do ping me if you want help.
>
> ---
> For what numbers to expect, I'm hoping for good test coverage in core,
> policies and utils. For the projects that contain mostly entities, I expect
> poor unit test coverage but reasonably good integration test coverage. The
> reason is that running the entities installs software etc, so in many
> places it is hard to unit test that beyond the not-particularly-useful what
> commands would it have wanted to execute.
>
> Aled
>
>
>
> On 14/01/2015 10:26, Duncan Grant wrote:
>
>> I'm trying to get code coverage working for brooklyn.
>>
>> As a first task I've upgraded the version of cobertura we are using and
>> fixed some issues with paths.
>> This means that you can run mvn install -P Coverage and it will produce
>> coverage reports for individual projects. I'll create a PR today.
>>
>> I've tried swapping the ant-run approach that is currently used for just
>> the maven plugin but there were some errors that I couldn't resolve and I
>> don't think it's worth the effort.
>>
>> I'd also like to have a single report for coverage across the project. I
>> think I just need a task that runs after all the projects have been built
>> but I'm not sure how to do that as the parent project seems to run first.
>> I haven't much experience with maven so I'm probably missing something
>> obvious.
>>
>> Does anyone have an opinion on whether we should be producing xml reports
>> or html reports or both? The html report is a nice human readable report.
>> The xml report can be integrated (using a plugin) with jenkins to show
>> trends but I'm not sure how helpful that is.
>>
>> I'm also a bit stuck getting the integration tests to run (due to a failed
>> dependancy over night). Also it seems to have been a while since they
>> complete successfully on http://brooklyn.builds.cloudsoftcorp.com/ . So
>> I
>> don't have any idea what coverage will look like then.
>>
>> Any thoughts would be appreciated
>>
>> Duncan
>>
>>
>
Re: Code coverage with cobertura
Posted by Aled Sage <al...@gmail.com>.
Hi Duncan,
Excellent, code coverage reports will be very useful moving forwards.
---
Regarding generating a single report, here are some comments that were
posted off-list:
Could you accumulate the reports as you go? Rather than running one
merge at the end do it for each project after the
post-integration-test or package phase, perhaps. I can imagine how
you would configure this, can't think how you'd ask for something to
be run at the end of the build.
Sam
And:
Have you tried the aggregate parameter -
http://mojo.codehaus.org/cobertura-maven-plugin/cobertura-mojo.html#aggregate?
Here’s an example that should aggregate it’s results:
https://svn.codehaus.org/mojo/trunk/mojo/cobertura-maven-plugin/src/it/mcobertura-65/pom.xml
Svet.
---
For xml versus html, If it's easy to say both then might as well do
that. Otherwise, the html report will be more useful in the short term.
---
For integration tests, sounds like they need some tender loving care
again. I'll try to find some time to look at any failures, but it might
not be until next week. If anyone else picks up fixing any of these,
then please do ping me if you want help.
---
For what numbers to expect, I'm hoping for good test coverage in core,
policies and utils. For the projects that contain mostly entities, I
expect poor unit test coverage but reasonably good integration test
coverage. The reason is that running the entities installs software etc,
so in many places it is hard to unit test that beyond the
not-particularly-useful what commands would it have wanted to execute.
Aled
On 14/01/2015 10:26, Duncan Grant wrote:
> I'm trying to get code coverage working for brooklyn.
>
> As a first task I've upgraded the version of cobertura we are using and
> fixed some issues with paths.
> This means that you can run mvn install -P Coverage and it will produce
> coverage reports for individual projects. I'll create a PR today.
>
> I've tried swapping the ant-run approach that is currently used for just
> the maven plugin but there were some errors that I couldn't resolve and I
> don't think it's worth the effort.
>
> I'd also like to have a single report for coverage across the project. I
> think I just need a task that runs after all the projects have been built
> but I'm not sure how to do that as the parent project seems to run first.
> I haven't much experience with maven so I'm probably missing something
> obvious.
>
> Does anyone have an opinion on whether we should be producing xml reports
> or html reports or both? The html report is a nice human readable report.
> The xml report can be integrated (using a plugin) with jenkins to show
> trends but I'm not sure how helpful that is.
>
> I'm also a bit stuck getting the integration tests to run (due to a failed
> dependancy over night). Also it seems to have been a while since they
> complete successfully on http://brooklyn.builds.cloudsoftcorp.com/ . So I
> don't have any idea what coverage will look like then.
>
> Any thoughts would be appreciated
>
> Duncan
>