You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by "Mark R. Diggory" <md...@latte.harvard.edu> on 2003/11/22 21:16:05 UTC
[math] Clover Test Coverage
http://jakarta.apache.org/commons/math/clover/index.html
I have to say, I'm very impressed with the clover test coverage tool.
This report is very cool and shows us exactly where coverage is low.
However, I'm not convinced that test coverage = quality assurance. One
could easily write tests that cause test coverage to score high while
not being very informative in and of themselves.
-Mark
--
Mark Diggory
Software Developer
Harvard MIT Data Center
http://www.hmdc.harvard.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [math] Clover Test Coverage
Posted by Al Chou <ho...@yahoo.com>.
--- Phil Steitz <ph...@steitz.com> wrote:
> Al Chou wrote:
> > --- "Mark R. Diggory" <md...@latte.harvard.edu> wrote:
> >
> >>http://jakarta.apache.org/commons/math/clover/index.html
> >>
> >>I have to say, I'm very impressed with the clover test coverage tool.
> >>This report is very cool and shows us exactly where coverage is low.
> >>
> >>However, I'm not convinced that test coverage = quality assurance. One
> >>could easily write tests that cause test coverage to score high while
> >>not being very informative in and of themselves.
> >
> >
> > That's one of the overarching lessons in software testing. There are some
> > pre-existing tests for commons.lang.StringUtils.split(*) that pass
> regardless
> > of how I order two operations in code I submitted in a patch yesterday.So
> > while the tests seem to exercise those methods and thus provide test
> coverage,
> > only one out of the three relevant tests is sensitive to those changes,
> which
> > makes me nervous about both the tests and what the lack of sensitivity to
> > changes says about the code (both the pre-existing code and my patch),
> because
> > it seems like the logic should be different depending on the task ordering.
>
> If you can improve the data coverage of the tests, please submit
> patches. I agree that path converage is necessary but not sufficient
> for good tests. As bugs are identified, test cases should be added to
> cover the conditions leading to the failures.
I need to look at the code some more before I can figure out why the existing
tests are insensitive to what I thought was a necessary task sequence
reordering. I dearly hoped last night to be able to create a test that would
catch this change; I really don't yet understand why the existing tests don't.
Al
=====
Albert Davidson Chou
Get answers to Mac questions at http://www.Mac-Mgrs.org/ .
__________________________________
Do you Yahoo!?
Free Pop-Up Blocker - Get it now
http://companion.yahoo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [math] Clover Test Coverage
Posted by Phil Steitz <ph...@steitz.com>.
Al Chou wrote:
> --- "Mark R. Diggory" <md...@latte.harvard.edu> wrote:
>
>>http://jakarta.apache.org/commons/math/clover/index.html
>>
>>I have to say, I'm very impressed with the clover test coverage tool.
>>This report is very cool and shows us exactly where coverage is low.
>>
>>However, I'm not convinced that test coverage = quality assurance. One
>>could easily write tests that cause test coverage to score high while
>>not being very informative in and of themselves.
>
>
> That's one of the overarching lessons in software testing. There are some
> pre-existing tests for commons.lang.StringUtils.split(*) that pass regardless
> of how I order two operations in code I submitted in a patch yesterday.So
> while the tests seem to exercise those methods and thus provide test coverage,
> only one out of the three relevant tests is sensitive to those changes, which
> makes me nervous about both the tests and what the lack of sensitivity to
> changes says about the code (both the pre-existing code and my patch), because
> it seems like the logic should be different depending on the task ordering.
If you can improve the data coverage of the tests, please submit
patches. I agree that path converage is necessary but not sufficient
for good tests. As bugs are identified, test cases should be added to
cover the conditions leading to the failures.
>
> You may be interested in reading "How to Misuse Code Coverage" from Brian
> Marick's http://www.testing.com/writings.html .
>
>
> Al
>
> =====
> Albert Davidson Chou
>
> Get answers to Mac questions at http://www.Mac-Mgrs.org/ .
>
> __________________________________
> Do you Yahoo!?
> Free Pop-Up Blocker - Get it now
> http://companion.yahoo.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [math] Clover Test Coverage
Posted by Al Chou <ho...@yahoo.com>.
--- "Mark R. Diggory" <md...@latte.harvard.edu> wrote:
> http://jakarta.apache.org/commons/math/clover/index.html
>
> I have to say, I'm very impressed with the clover test coverage tool.
> This report is very cool and shows us exactly where coverage is low.
>
> However, I'm not convinced that test coverage = quality assurance. One
> could easily write tests that cause test coverage to score high while
> not being very informative in and of themselves.
That's one of the overarching lessons in software testing. There are some
pre-existing tests for commons.lang.StringUtils.split(*) that pass regardless
of how I order two operations in code I submitted in a patch yesterday. So
while the tests seem to exercise those methods and thus provide test coverage,
only one out of the three relevant tests is sensitive to those changes, which
makes me nervous about both the tests and what the lack of sensitivity to
changes says about the code (both the pre-existing code and my patch), because
it seems like the logic should be different depending on the task ordering.
You may be interested in reading "How to Misuse Code Coverage" from Brian
Marick's http://www.testing.com/writings.html .
Al
=====
Albert Davidson Chou
Get answers to Mac questions at http://www.Mac-Mgrs.org/ .
__________________________________
Do you Yahoo!?
Free Pop-Up Blocker - Get it now
http://companion.yahoo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org