You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by Paul Johnson <pa...@pjcj.net> on 2004/05/12 01:24:52 UTC
Re: gcov and mp2 tests coverage
On Mon, Mar 15, 2004 at 10:46:07AM -0800, Stas Bekman wrote:
> Philippe M. Chiasson wrote:
> >On Wed, 2004-03-10 at 15:42, Stas Bekman wrote:
> >
> >>rbb is back to the game and he brings gcov (test coverage) for the apr
> >>test suite:
> >>http://apr.apache.org/coverage/index.html
> >>I wonder if we could/should to do the same for mp2.
> >
> >
> >http://gozer.ectoplasm.org/mod_perl/coverage/report.html
>
> Thanks, Philippe! Looks like we have a pretty good coverage.
And here is another view on the data, this time with the Perl code
included.
http://pjcj.sytes.net/cover/mod_perl-1.99_13/cover_db-5.9.2/coverage.html
(That's on my local machine, so it may be a little unreliable.)
This was using apache-2.0.49, mod_perl-1.99_13, and bleadperl from a
couple of days ago. Oh, and gcc 3.3.3 and Devel::Cover 0.43 plus a
couple of patches to get gcov2perl working again.
I have included all the perl code in the report (normally the code under
t/ would not be reported on) because what strikes me most is the low
coverage of the perl code, and the number of perl modules which don't
ever appear to be use'd and hence don't even appear on the report.
Now it's quite possible that I have not run the tests correctly, but
inserting "die" near the top of a couple of random (apparently untested)
modules didn't appear to cause any test failures. I count 388 .pm files
in the mod_perl tree, and only 25 of the have any coverage at all.
The commands I used were:
$ perl5.9.2 Makefile.PL MP_APXS=/usr/local/pkg/apache-2.0.49/bin/apxs MODPERL_CC='gcc -fprofile-arcs -ftest-coverage'
$ HARNESS_PERL_SWITCHES=-I/home/pjcj/g/perl/dev/Devel-Cover/blib/lib\ -I/home/pjcj/g/perl/dev/Devel-Cover/blib/arch\ -MDevel::Cover=-ignore,Cover,-select,^t/ make test
$ for f (**/*.c)
for> (chdir `dirname $f`; gcov `basename $f`)
$ perl5.9.2 -I/home/pjcj/g/perl/dev/Devel-Cover/blib/lib -I/home/pjcj/g/perl/dev/Devel-Cover/blib/arch /home/pjcj/g/perl/dev/Devel-Cover/cover
(The -I's there are just to pick up my development copy of
Devel::Cover.)
Have I made some mistake here, is Devel::Cover wrong, or is the coverage
really that bad?
> For those entries which are reported to be not very well covered, it's not
> necessarily true, since 'make test' will exercise different parts of the
> code depending on the used Perl. I wonder if it's possible to run the test
> suite under several builds (5.6.1, 5.6.1-ithread, 5.8.3, 5.8.3-ithread) and
> then merge the results.
This is certainly possible using the Devel::Cover framework, but if I
understand correctly, a lot of the files are generated and will be
slightly different for each perl version. In this case current
Devel::Cover behaviour is to drop the results for the old version of the
file.
--
Paul Johnson - paul@pjcj.net
http://www.pjcj.net
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: gcov and mp2 tests coverage
Posted by Paul Johnson <pa...@pjcj.net>.
On Mon, May 17, 2004 at 03:13:03PM -0700, Philippe M. Chiasson wrote:
>
>
> Stas Bekman wrote:
> >Paul Johnson wrote:
> >[...]
> >
> >>And here is another view on the data, this time with the Perl code
> >>included.
> >>
> >> http://pjcj.sytes.net/cover/mod_perl-1.99_13/cover_db-5.9.2/coverage.html
> >>
> >>(That's on my local machine, so it may be a little unreliable.)
> >>
> >>This was using apache-2.0.49, mod_perl-1.99_13, and bleadperl from a
> >>couple of days ago. Oh, and gcc 3.3.3 and Devel::Cover 0.43 plus a
> >>couple of patches to get gcov2perl working again.
>
> Any change of getting a diff of those changes to gcov2perl ?
The patches lead to some refactoring and stuff, so I've uploaded
Devel::Cover 0.44 to CPAN. It's also available from my home page if you
want it sooner.
> I'd like to give it a spin myself as well ;-)
I'd be very pleased if you would, to make sure that I ran the tests
properly if nothing else. I've not yet had time to take a detailed look
at the results I got.
--
Paul Johnson - paul@pjcj.net
http://www.pjcj.net
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: gcov and mp2 tests coverage
Posted by "Philippe M. Chiasson" <go...@ectoplasm.org>.
Stas Bekman wrote:
> Paul Johnson wrote:
> [...]
>
>>And here is another view on the data, this time with the Perl code
>>included.
>>
>> http://pjcj.sytes.net/cover/mod_perl-1.99_13/cover_db-5.9.2/coverage.html
>>
>>(That's on my local machine, so it may be a little unreliable.)
>>
>>This was using apache-2.0.49, mod_perl-1.99_13, and bleadperl from a
>>couple of days ago. Oh, and gcc 3.3.3 and Devel::Cover 0.43 plus a
>>couple of patches to get gcov2perl working again.
Any change of getting a diff of those changes to gcov2perl ?
I'd like to give it a spin myself as well ;-)
>>I have included all the perl code in the report (normally the code under
>>t/ would not be reported on) because what strikes me most is the low
>>coverage of the perl code, and the number of perl modules which don't
>>ever appear to be use'd and hence don't even appear on the report.
>>
>>Now it's quite possible that I have not run the tests correctly, but
>>inserting "die" near the top of a couple of random (apparently untested)
>>modules didn't appear to cause any test failures. I count 388 .pm files
>>in the mod_perl tree, and only 25 of the have any coverage at all.
>>
>>The commands I used were:
>>
>> $ perl5.9.2 Makefile.PL MP_APXS=/usr/local/pkg/apache-2.0.49/bin/apxs MODPERL_CC='gcc -fprofile-arcs -ftest-coverage'
>> $ HARNESS_PERL_SWITCHES=-I/home/pjcj/g/perl/dev/Devel-Cover/blib/lib\ -I/home/pjcj/g/perl/dev/Devel-Cover/blib/arch\ -MDevel::Cover=-ignore,Cover,-select,^t/ make test
>> $ for f (**/*.c)
>> for> (chdir `dirname $f`; gcov `basename $f`)
>> $ perl5.9.2 -I/home/pjcj/g/perl/dev/Devel-Cover/blib/lib -I/home/pjcj/g/perl/dev/Devel-Cover/blib/arch /home/pjcj/g/perl/dev/Devel-Cover/cover
>>
>>(The -I's there are just to pick up my development copy of
>>Devel::Cover.)
>
>
> Nice!
>
>
>>Have I made some mistake here, is Devel::Cover wrong, or is the coverage
>>really that bad?
>
>
> Most module under modperl-2.0/lib and modperl-2.0/Apache-Test/lib (though in
> your report moved to blib) aren't used by mod_perl runtime. They are used
> during mod_perl build and test setup build. I'm sure if you exclude those the
> coverage will be more than 80%. That's why gozer's report's is much closer to
> reality (even though it has no Total numbers, it's easy to see that most of
> the API is well covered.
>
> Overall the coverage looks great to me and it'll be even better once we finish
> polishing the API, which will require having all of them tested. Of course new
> tests for those uncovered areas are more than welcome. That's one of the main
> reasons 2.0 is not released yet - some APIs aren't tested/polished yet.
>
>
>>>For those entries which are reported to be not very well covered, it's not
>>>necessarily true, since 'make test' will exercise different parts of the
>>>code depending on the used Perl. I wonder if it's possible to run the test
>>>suite under several builds (5.6.1, 5.6.1-ithread, 5.8.3, 5.8.3-ithread) and
>>>then merge the results.
>>
>>
>>This is certainly possible using the Devel::Cover framework, but if I
>>understand correctly, a lot of the files are generated and will be
>>slightly different for each perl version. In this case current
>>Devel::Cover behaviour is to drop the results for the old version of the
>>file.
>
>
> Well, I supposed what it was suggesting is that you will get raw input data
> across several builds and then average them or similar. The main point is that
> the report is never correct, since some parts of the API are not run under
> certains perls and they are covered by the test suite.
>
>
>
--
--------------------------------------------------------------------------------
Philippe M. Chiasson m/gozer\@(apache|cpan|ectoplasm)\.org/ GPG KeyID : 88C3A5A5
http://gozer.ectoplasm.org/ F9BF E0C2 480E 7680 1AE5 3631 CB32 A107 88C3A5A5
Re: gcov and mp2 tests coverage
Posted by Stas Bekman <st...@stason.org>.
Paul Johnson wrote:
[...]
> And here is another view on the data, this time with the Perl code
> included.
>
> http://pjcj.sytes.net/cover/mod_perl-1.99_13/cover_db-5.9.2/coverage.html
>
> (That's on my local machine, so it may be a little unreliable.)
>
> This was using apache-2.0.49, mod_perl-1.99_13, and bleadperl from a
> couple of days ago. Oh, and gcc 3.3.3 and Devel::Cover 0.43 plus a
> couple of patches to get gcov2perl working again.
>
> I have included all the perl code in the report (normally the code under
> t/ would not be reported on) because what strikes me most is the low
> coverage of the perl code, and the number of perl modules which don't
> ever appear to be use'd and hence don't even appear on the report.
>
> Now it's quite possible that I have not run the tests correctly, but
> inserting "die" near the top of a couple of random (apparently untested)
> modules didn't appear to cause any test failures. I count 388 .pm files
> in the mod_perl tree, and only 25 of the have any coverage at all.
>
> The commands I used were:
>
> $ perl5.9.2 Makefile.PL MP_APXS=/usr/local/pkg/apache-2.0.49/bin/apxs MODPERL_CC='gcc -fprofile-arcs -ftest-coverage'
> $ HARNESS_PERL_SWITCHES=-I/home/pjcj/g/perl/dev/Devel-Cover/blib/lib\ -I/home/pjcj/g/perl/dev/Devel-Cover/blib/arch\ -MDevel::Cover=-ignore,Cover,-select,^t/ make test
> $ for f (**/*.c)
> for> (chdir `dirname $f`; gcov `basename $f`)
> $ perl5.9.2 -I/home/pjcj/g/perl/dev/Devel-Cover/blib/lib -I/home/pjcj/g/perl/dev/Devel-Cover/blib/arch /home/pjcj/g/perl/dev/Devel-Cover/cover
>
> (The -I's there are just to pick up my development copy of
> Devel::Cover.)
Nice!
> Have I made some mistake here, is Devel::Cover wrong, or is the coverage
> really that bad?
Most module under modperl-2.0/lib and modperl-2.0/Apache-Test/lib (though in
your report moved to blib) aren't used by mod_perl runtime. They are used
during mod_perl build and test setup build. I'm sure if you exclude those the
coverage will be more than 80%. That's why gozer's report's is much closer to
reality (even though it has no Total numbers, it's easy to see that most of
the API is well covered.
Overall the coverage looks great to me and it'll be even better once we finish
polishing the API, which will require having all of them tested. Of course new
tests for those uncovered areas are more than welcome. That's one of the main
reasons 2.0 is not released yet - some APIs aren't tested/polished yet.
>>For those entries which are reported to be not very well covered, it's not
>>necessarily true, since 'make test' will exercise different parts of the
>>code depending on the used Perl. I wonder if it's possible to run the test
>>suite under several builds (5.6.1, 5.6.1-ithread, 5.8.3, 5.8.3-ithread) and
>>then merge the results.
>
>
> This is certainly possible using the Devel::Cover framework, but if I
> understand correctly, a lot of the files are generated and will be
> slightly different for each perl version. In this case current
> Devel::Cover behaviour is to drop the results for the old version of the
> file.
Well, I supposed what it was suggesting is that you will get raw input data
across several builds and then average them or similar. The main point is that
the report is never correct, since some parts of the API are not run under
certains perls and they are covered by the test suite.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org