You are viewing a plain text version of this content. The canonical link for it is here.
Posted to cactus-dev@jakarta.apache.org by Vincent Massol <vm...@octo.com> on 2002/06/09 21:36:49 UTC
Cactus now has test coverage reports !
Hi,
I have been working on generating Test Coverage reports for Cactus this
week end ! I have used Clover
(http://www.thecortex.net/clover/index.html) and the reports looks
really great I think.
You can check them out on
http://jakarta.apache.org/cactus/1.4/coverage.html
I'm also quite impressed by our test coverage ... It is quite good I
think, although it can always be improved ... :-)
What do you think ?
Thanks
-Vincent
_______________________
Vincent Massol
OCTO Technology UK Ltd
http://www.octo.com
vmassol@octo.com
Tel: (020) 8996 9540
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Cactus delivery philosophy (was RE: Cactus now has test coverage reports !)
Posted by Vincent Massol <vm...@octo.com>.
> -----Original Message-----
> From: Kaarle Kaila [mailto:kaarle.kaila@iki.fi]
> Sent: 10 June 2002 22:15
> To: Cactus Users List
> Subject: RE: Cactus now has test coverage reports !
>
> At 21:02 10.6.2002 +0100, you wrote:
>
> >Not sure what you mean here by "who might be lazy when the system
> >already seems to be working!" ... :-)
>
> When writing new code or modifications to existing one it happens that
you
> forget to write testcases
> to verify it. One reason for this is lazyness I guess!
>
yes, you're right ! The tool judges the overall quality of our tests and
yes, it would be nice if committers and contributors took the time to
write test cases with the code they submit (that includes myself !).
However, I think we should not be too strict here as we are definitely
eager for contributions and would rather have some with no test cases
that nothing at all ! :-)
That said, I was quite agreeably surprised when I ran Clover the first
time to see that we had > 50% of code coverage in our tests. That's
quite good ! You won't find lots of projects (open source or not) with
that level ... :-)
In addition to this, we are also running the tests on several servlet
engines, which provides us with an additional level of security !
But now that we have these nice reports, we can focus our energy on
writing some test cases for the code that we are not testing. I would
say reaching 70-80% should be our goal.
Some may wonder why all this ? What for ? Well, the goal is of course to
produce quality deliveries, including nightly builds ! It is meant to
replace the long process of code releases that go from alpha1 to alpha2
to alpha3 to beta1, beta2, beta3 and release !
As you may have noticed in the past, Cactus has always released directly
with no alpha or beta periods ... Quite dangerous you would say ? Quite
right, except that apart from some very minor glitches, it has worked
fine for us with no major bugs reporting and I believe this is due to
our extensive testing and strong build system.
What my goal has always been with Cactus is to have everything automated
as part of the build process (including all kinds of tests) so that we
are able to deliver at any time. And I think this has been achieved. Our
nightly builds should be of the same quality as the release is (apart
maybe from some missing or incomplete doco in nightly builds, but that
won't affect Cactus runtime).
This is all about CI (Continuous Integration) ...
What do you all think ?
Thanks
-Vincent
> regards
> Kaarle
>
>
> ---------------------------------------------
> Kaarle Kaila
> http://www.iki.fi/kaila
> mailto:kaarle.kaila@iki.fi
> tel: +358 50 3725844
>
>
>
> --
> To unsubscribe, e-mail: <mailto:cactus-user-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:cactus-user-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Cactus now has test coverage reports !
Posted by Kaarle Kaila <ka...@iki.fi>.
At 21:02 10.6.2002 +0100, you wrote:
>Not sure what you mean here by "who might be lazy when the system
>already seems to be working!" ... :-)
When writing new code or modifications to existing one it happens that you
forget to write testcases
to verify it. One reason for this is lazyness I guess!
regards
Kaarle
---------------------------------------------
Kaarle Kaila
http://www.iki.fi/kaila
mailto:kaarle.kaila@iki.fi
tel: +358 50 3725844
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Cactus now has test coverage reports !
Posted by Vincent Massol <vm...@octo.com>.
Hi Kaarle,
> -----Original Message-----
> From: Kaarle Kaila [mailto:kaarle.kaila@iki.fi]
> Sent: 10 June 2002 20:14
> To: Cactus Users List
> Cc: 'Cactus Developers List'
> Subject: Re: Cactus now has test coverage reports !
>
> At 20:36 9.6.2002 +0100, Vincent Massol wrote:
> >Hi,
>
> hi Vincent!
>
> It is remarkably where you find the time and energy for all this!
I try my best :-)
>
> If I understand after browsing clover pages a little it shows what
code of
> the
> software is used in general.
No, it shows what tests were covered by the tests.
> That would show that a remarkably section of
> many software packages are not used at all. Of Cactus code 54% is used
> and Xerces uses only 14.7 %. A lot of useless code!!
It simply shows that Cactus tests cover 54% of all the code, whereas
Xerces tests only tests 14.7% of its code ...
>
> I assume this means that clover only finds a part of the sections in
use
> by the application itself and especially those sections that are
> references
> by testcases can be found. Am I right here?
>
> If this can be used to get an idea of how well the testcases cover the
> application it could be very usefull to remind us who might be lazy
when
> the system already seems to be working! Perhaps this is what it shows
> but I did not find that after a short look at it!
Not sure what you mean here by "who might be lazy when the system
already seems to be working!" ... :-)
Regards
-Vincent
>
> regards
> Kaarle
>
>
> >I have been working on generating Test Coverage reports for Cactus
this
> >week end ! I have used Clover
> >(http://www.thecortex.net/clover/index.html) and the reports looks
> >really great I think.
> >
> >You can check them out on
> >http://jakarta.apache.org/cactus/1.4/coverage.html
> >
> >I'm also quite impressed by our test coverage ... It is quite good I
> >think, although it can always be improved ... :-)
> >
> >What do you think ?
> >
> >Thanks
> >-Vincent
> >
> >_______________________
> >Vincent Massol
> >OCTO Technology UK Ltd
> >http://www.octo.com
> >vmassol@octo.com
> >Tel: (020) 8996 9540
> >
> >
> >--
> >To unsubscribe, e-mail: <mailto:cactus-user-
> unsubscribe@jakarta.apache.org>
> >For additional commands, e-mail: <mailto:cactus-user-
> help@jakarta.apache.org>
>
> ---------------------------------------------
> Kaarle Kaila
> http://www.iki.fi/kaila
> mailto:kaarle.kaila@iki.fi
> tel: +358 50 3725844
>
>
>
> --
> To unsubscribe, e-mail: <mailto:cactus-user-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:cactus-user-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Cactus now has test coverage reports !
Posted by Kaarle Kaila <ka...@iki.fi>.
At 20:36 9.6.2002 +0100, Vincent Massol wrote:
>Hi,
hi Vincent!
It is remarkably where you find the time and energy for all this!
If I understand after browsing clover pages a little it shows what code of the
software is used in general. That would show that a remarkably section of
many software packages are not used at all. Of Cactus code 54% is used
and Xerces uses only 14.7 %. A lot of useless code!!
I assume this means that clover only finds a part of the sections in use
by the application itself and especially those sections that are references
by testcases can be found. Am I right here?
If this can be used to get an idea of how well the testcases cover the
application it could be very usefull to remind us who might be lazy when
the system already seems to be working! Perhaps this is what it shows
but I did not find that after a short look at it!
regards
Kaarle
>I have been working on generating Test Coverage reports for Cactus this
>week end ! I have used Clover
>(http://www.thecortex.net/clover/index.html) and the reports looks
>really great I think.
>
>You can check them out on
>http://jakarta.apache.org/cactus/1.4/coverage.html
>
>I'm also quite impressed by our test coverage ... It is quite good I
>think, although it can always be improved ... :-)
>
>What do you think ?
>
>Thanks
>-Vincent
>
>_______________________
>Vincent Massol
>OCTO Technology UK Ltd
>http://www.octo.com
>vmassol@octo.com
>Tel: (020) 8996 9540
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
---------------------------------------------
Kaarle Kaila
http://www.iki.fi/kaila
mailto:kaarle.kaila@iki.fi
tel: +358 50 3725844
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Cactus now has test coverage reports !
Posted by Kaarle Kaila <ka...@iki.fi>.
At 20:36 9.6.2002 +0100, Vincent Massol wrote:
>Hi,
hi Vincent!
It is remarkably where you find the time and energy for all this!
If I understand after browsing clover pages a little it shows what code of the
software is used in general. That would show that a remarkably section of
many software packages are not used at all. Of Cactus code 54% is used
and Xerces uses only 14.7 %. A lot of useless code!!
I assume this means that clover only finds a part of the sections in use
by the application itself and especially those sections that are references
by testcases can be found. Am I right here?
If this can be used to get an idea of how well the testcases cover the
application it could be very usefull to remind us who might be lazy when
the system already seems to be working! Perhaps this is what it shows
but I did not find that after a short look at it!
regards
Kaarle
>I have been working on generating Test Coverage reports for Cactus this
>week end ! I have used Clover
>(http://www.thecortex.net/clover/index.html) and the reports looks
>really great I think.
>
>You can check them out on
>http://jakarta.apache.org/cactus/1.4/coverage.html
>
>I'm also quite impressed by our test coverage ... It is quite good I
>think, although it can always be improved ... :-)
>
>What do you think ?
>
>Thanks
>-Vincent
>
>_______________________
>Vincent Massol
>OCTO Technology UK Ltd
>http://www.octo.com
>vmassol@octo.com
>Tel: (020) 8996 9540
>
>
>--
>To unsubscribe, e-mail: <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
---------------------------------------------
Kaarle Kaila
http://www.iki.fi/kaila
mailto:kaarle.kaila@iki.fi
tel: +358 50 3725844
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>