Re: Measuring integration test code coverage for a Maven plugin (fwd)

[Dan Fabulich <da...@fabulich.com>]

Should have cc:ed surefire-dev also.

Here's the coverage report.

http://docs.codehaus.org/download/attachments/35422245/surefire-clover-2007-12-10.zip

-Dan

---------- Forwarded message ----------
Date: Mon, 10 Dec 2007 00:02:50 -0800 (Pacific Standard Time)
From: Dan Fabulich <da...@fabulich.com>
To: Maven Developers List <de...@maven.apache.org>
Subject: Re: Measuring integration test code coverage for a Maven plugin

Raphaël Piéroni wrote:

> My guess is that you should deploy your instrumented plugin in a test 
> repository.

That's basically what I wound up doing.

For the record, I started by specifying a special local repository in my 
setting directory.

Then I tried just using maven-clover-plugin 3.6, calling instrumentInternal 
directly (which you're not supposed to do because you risk getting cloverized 
stuff in your production output, though that's exactly what I wanted to do in 
my case).

That didn't quite work because instrumentInternal includes a function 
redirectArtifact() that changes the output jar to be blah-clover.jar; once I 
commented that out and added clover as a dependency to all of my code, I was 
able to run my integration tests and get a total view of my integration test 
coverage.

For the record, Surefire's unit tests covered 45% of Surefire; with the added 
integration tests we've covered 79.4% (87 classes, 3,323 / 4,187 elements).  It 
also highlighted some key areas where we could add more tests, which was 
exactly what I wanted.

-Dan

2007/12/7, Dan Fabulich <da...@fabulich.com>:
> 
> I recently added a bunch of integration tests for Surefire.  These
> integration tests automatically fork a separate Maven process to run real
> Maven builds, like the Maven core integration tests do.
> 
> This naturally led me to wonder: Does Surefire (now) have reasonable code
> coverage?  Specifically, which lines in Surefire were covered by unit
> tests, which by integration tests, and which weren't covered at all?
> 
> I know there's a variety of handy code coverage tools that work with
> Maven, allowing you to instrument classes for code coverage and run your
> unit tests against the instrumented classes.
> 
> The catch in this case is that I need to somehow convince Maven to use the
> instrumented version of my plugin, and not the regular "real" version of
> the plugin, when I go to run my integration tests.  The clover plugin, for
> example, doesn't seem to want to let me do that.
> 
> [On the other hand, maybe I should just use an instrumenting JVM
> instead...?  Java 1.5's new java.lang.instrument would probably do the
> trick, but I'm not aware of any code coverage tool that works with
> j.l.instrument, and anyway I'd have to fix SUREFIRE-179 just to get it to
> work... :-)]
> 
> Has anybody ever done this before?  More generally, I don't think I've
> ever seen an example of anyone using Maven to run multi-process
> integration tests (e.g. cargo tests) and also measuring code coverage on
> those integration tests.  Has anyone seen a good example of this that I
> could reuse?
> 
> -Dan
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
> 
>