You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael Busch (JIRA)" <ji...@apache.org> on 2006/11/21 08:01:02 UTC
[jira] Created: (LUCENE-721) Code coverage reports
Code coverage reports
---------------------
Key: LUCENE-721
URL: http://issues.apache.org/jira/browse/LUCENE-721
Project: Lucene - Java
Issue Type: New Feature
Components: Other
Reporter: Michael Busch
Priority: Minor
Attachments: code_coverage.patch
Hi all,
We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
- emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
- emma-instrument: instruments the compiled code
- generate-emma-report: generates an html code coverage report
The following steps are neccessary in order to generate a code coverage report:
- add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
- execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
- execute ant target 'test' to run the unit tests
- execute ant target 'generate-emma-report'
To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Created: (LUCENE-721) Code coverage reports
Posted by Doug Cutting <cu...@apache.org>.
Grant Ingersoll wrote:
> I would be more than happy to put 'em in once I get my zones account and
> we decide on which tool to use.
You now have a zones account.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Created: (LUCENE-721) Code coverage reports
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 21, 2006, at 2:01 AM, Michael Busch (JIRA) wrote:
>
> Furthermore people could take a look in the code coverage reports
> to figure out where work needs to be done, i. e. where additional
> testcases are neccessary. It would be nice if we could add a page
> to the Lucene website showing the report, generated by the nightly
> build. Maybe you could add that to your preview page (LUCENE-707),
> Grant?
>
I would be more than happy to put 'em in once I get my zones account
and we decide on which tool to use.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-721) Code coverage reports
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=all ]
Michael Busch updated LUCENE-721:
---------------------------------
Attachment: emma_report.zip
I attach a code coverage report of the current HEAD (revision: 477535), so people can check out how a report looks like.
The current code coverage looks not bad: 77% of the code lines are being tested by our unit tests. However, the coverage is not perfect: e. g. the PorterStemmer or IndexWriter.addIndexes(IndexReader[]) are not tested at all.
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-721) Code coverage reports
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=all ]
Grant Ingersoll reassigned LUCENE-721:
--------------------------------------
Assignee: Grant Ingersoll (was: Michael Busch)
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: clover.patch, code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-721) Code coverage reports
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=all ]
Michael Busch reassigned LUCENE-721:
------------------------------------
Assignee: Michael Busch
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Assigned To: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-721) Code coverage reports
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=all ]
Michael Busch updated LUCENE-721:
---------------------------------
Attachment: code_coverage.patch
The patch described above.
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12457084 ]
Grant Ingersoll commented on LUCENE-721:
----------------------------------------
OK, I committed this, with a minor change so that it is off by default (thanks Erik, for the ANT info).
To run, you need three things:
1. clover.jar in ANT lib
2. clover.license in ANT lib
3. On the command line or in the your ANT properties, if you want it on all the time, you must specify -Drun.clover=true
Thanks, Micheal.
I will incorporate this into the nightly build per http://issues.apache.org/jira/browse/LUCENE-708
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: clover.patch, code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-721) Code coverage reports
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=all ]
Grant Ingersoll resolved LUCENE-721.
------------------------------------
Resolution: Fixed
Committed the change. Linked to the reports from the Resources -> Developers page on the documentation. Current report was generated on 12/16/06. Once Lucene 708 is resolved, these reports will be updated nightly.
Thanks Michael!
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: clover.patch, code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12452553 ]
Michael Busch commented on LUCENE-721:
--------------------------------------
Sure Grant, I will update the patch soon to use clover instead of EMMA. Great that you will add it to the website!
Thanks,
Michael
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12451727 ]
Michael Busch commented on LUCENE-721:
--------------------------------------
Good idea, I could do that. But currently we don't fetch the JUnit jars either, just give a hint where to download them in case they cannot be found in the ant classpath. I did the same with the EMMA jars. Should we do it for both (JUnit and EMMA), just for EMMA or leave it like it is for now?
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-721) Code coverage reports
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Dec 9, 2006, at 10:16 AM, Grant Ingersoll (JIRA) wrote:
> I wanted to do something like adding a property named run.clover
> (off by default) that could then be overridden by -Drun.clover=true
> on the command line, but I admit I don't see in ANT how to AND
> together the clover.present attribute with this new property. The
> CONDITION and AVAILABLE tasks in ANT don't seem to allow for ANDing
> two existing properties, or perhaps I am missing something
<condition> can be used with <and> with nested <isset>s (or other
conditions here <http://ant.apache.org/manual/CoreTasks/
conditions.html>).
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12457080 ]
Grant Ingersoll commented on LUCENE-721:
----------------------------------------
I'm not familiar w/ clover, but here's what I did, when trying out your patch (which works):
1. applied the patch :-)
2. Put clover.jar and clover.license (retrieved from the committers private repository into my ANT lib)
3. Ran "ant clover" -- this passed and said clover is enabled
4. Ran "ant test"
5. ant generate-clover-reports
6. View the reports! They look good and I think will prove to be useful, especially for people who are looking for how they can help make Lucene better.
Is there anyway to turn off clover other than removing the jar? It seems like it takes quite a bit longer to run the tests and I was thinking I may just want to enable it when running nightly, but that most people wouldn't want it on by default even though they may have clover in there ANT lib.
I wanted to do something like adding a property named run.clover (off by default) that could then be overridden by -Drun.clover=true on the command line, but I admit I don't see in ANT how to AND together the clover.present attribute with this new property. The CONDITION and AVAILABLE tasks in ANT don't seem to allow for ANDing two existing properties, or perhaps I am missing something. I admit, though, my ANT knowledge isn't what it used to be since I moved to Maven a long time ago, so if someone can chime in here it would be appreciated.
Otherwise, the patch looks good and I will commit it once we figure out whether being able to turn it off is important to people and how to do it.
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: clover.patch, code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-721) Code coverage reports
Posted by Simon Willnauer <si...@googlemail.com>.
I agree with Michael, if there is a clover licence we should use it.
One major advantage of clover is the ide support for eclipse and
netbeans. Emma has no Eclipse support yet and a old not maintained
netbeans plugin.
There is a ASF donated clover license under:
https://svn.apache.org/repos/private/committers/donated-licenses/clover/
This is Clover 1.3.2 and the License file for use by the ASF.
It will work on the following Java packages:
org.apache
javax
org.xml
org.w3c
com.example
Although this licence can not be used with a IDE Plugin it would do
the job for the coverage reports!
best regards Simon
On 11/22/06, Michael Busch <bu...@gmail.com> wrote:
> Chris Hostetter wrote:
> > To throw another twist onto things, it would appear that the ASF has a
> > License for Clover 1.3.2 donated by Cenqua that Committers have access to
> > (see committers/donated-licenses/clover in SVN) ... it's not clear to me
> > if that License would allow for auto generated reports of nightly builds
> > on the public inter-web.
> >
> > An example of a Clover report for (an older version of) Lucene can
> > acctually be found online...
> >
> > http://developer.spikesource.com/spikewatch.logs/fedora-3-i386/2221/lucene/reports/clover/
> >
> >
> I proposed EMMA, because it is open source, but if we have a license for
> Clover we should use it, the reports look very good.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-721) Code coverage reports
Posted by Michael Busch <bu...@gmail.com>.
Chris Hostetter wrote:
> To throw another twist onto things, it would appear that the ASF has a
> License for Clover 1.3.2 donated by Cenqua that Committers have access to
> (see committers/donated-licenses/clover in SVN) ... it's not clear to me
> if that License would allow for auto generated reports of nightly builds
> on the public inter-web.
>
> An example of a Clover report for (an older version of) Lucene can
> acctually be found online...
>
> http://developer.spikesource.com/spikewatch.logs/fedora-3-i386/2221/lucene/reports/clover/
>
>
I proposed EMMA, because it is open source, but if we have a license for
Clover we should use it, the reports look very good.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-721) Code coverage reports
Posted by Chris Hostetter <ho...@fucit.org>.
: How about the nightly build, if we include this on the site? I'm not
: familiar with it enough to comment. Yonik and Doug will know if/how
: things can be set up there with EMMA jars, should we include this on the
: site.
To throw another twist onto things, it would appear that the ASF has a
License for Clover 1.3.2 donated by Cenqua that Committers have access to
(see committers/donated-licenses/clover in SVN) ... it's not clear to me
if that License would allow for auto generated reports of nightly builds
on the public inter-web.
An example of a Clover report for (an older version of) Lucene can
acctually be found online...
http://developer.spikesource.com/spikewatch.logs/fedora-3-i386/2221/lucene/reports/clover/
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12451806 ]
Otis Gospodnetic commented on LUCENE-721:
-----------------------------------------
Sounds like my previous comment was off - if we don't distribute EMMA jars and just require the person to have them, what you have will do. How about the nightly build, if we include this on the site? I'm not familiar with it enough to comment. Yonik and Doug will know if/how things can be set up there with EMMA jars, should we include this on the site.
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Updated: (LUCENE-721) Code coverage reports
Posted by Michael Busch <bu...@gmail.com>.
Chris Hostetter wrote:
> : Here it is, Grant. This new patch uses Clover to generate code coverage
> : reports. Simply add clover.jar to the ant classpath, do a "clean" and
> : run the target "test". During compiling Clover will automatically
> : instrument all classes under src/java.
>
> haven't had a chance to look at the patch, but i have two questions baout
> this:
>
> 1) is there any way to explicitly disable the instumentation (ie: with a
> system property set in the build.properties, or on the command line) in
> case people get into a situation where they are suspicious of hte
> instrumentation and what to run the test without it?
>
>
Good point, I can easily add a property to switch on/off
instrumentation. Currently it's always enabled if clover.jar can be
found in the classpath.
> 2) what is the beahvior of the report generatation after a test failure?
> DOes Clover know baout Ant failures? would the report reflect the fact
> that the tests failed in it's summary info?
>
>
>
No, clover has no information about failed tests. It shows the recorded
coverage since the last compile/instrumentation and if some tests failed
the coverage will probably be lower. We could show information about
failed tests in the title of the code coverage report. But I will read a
bit more in the clover manual to figure out if there are other
possibilities to show those information.
- Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Updated: (LUCENE-721) Code coverage reports
Posted by Chris Hostetter <ho...@fucit.org>.
: Here it is, Grant. This new patch uses Clover to generate code coverage
: reports. Simply add clover.jar to the ant classpath, do a "clean" and
: run the target "test". During compiling Clover will automatically
: instrument all classes under src/java.
haven't had a chance to look at the patch, but i have two questions baout
this:
1) is there any way to explicitly disable the instumentation (ie: with a
system property set in the build.properties, or on the command line) in
case people get into a situation where they are suspicious of hte
instrumentation and what to run the test without it?
2) what is the beahvior of the report generatation after a test failure?
DOes Clover know baout Ant failures? would the report reflect the fact
that the tests failed in it's summary info?
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-721) Code coverage reports
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=all ]
Michael Busch updated LUCENE-721:
---------------------------------
Attachment: clover.patch
Here it is, Grant. This new patch uses Clover to generate code coverage reports. Simply add clover.jar to the ant classpath, do a "clean" and run the target "test". During compiling Clover will automatically instrument all classes under src/java.
After the junit tests are done, execute the new target "generate-clover-reports". Clover will generate the html report in build/test/clover/reports.
Note: If clover.jar can not be found in the ant classpath then "test" will simply print an info message, saying that code coverage reports are disabled, and exeute the junit tests like it did before this patch.
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Assigned To: Michael Busch
> Priority: Minor
> Attachments: clover.patch, code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-721) Code coverage reports
Posted by Michael Busch <bu...@gmail.com>.
Chris Hostetter wrote:
> : Nice. I think we can't include EMMA jars int he repository, though, so
> : you'll want to add the ability to download the Jar on the fly, just like
> : Grant did it for the benchmark stuff.
>
> that's not strictly neccessary is it? ... coverage reports could just be
> an optional part of the build that only works if you've got EMMA installed
> -- much the way javacc is.
>
> I haven't really looked at EMMA or this patch, but if it involves an ant
> plugin then users pretty much have to install it themselves before
> executing ant right?
>
>
All users have to do is to download the EMMA jar files and add them to
their ant classpath. It's exactly the same as what they have to do to
get JUnit to work. If the jars cannot be found in the ant classpath the
users get an error message when they execute an EMMA target, saying to
download EMMA. All the other targets (like compile or test) work without
having the EMMA jars as they did before.
> It should also be noted, there are *many* code coverage tools ... some may
> have more compatible liscences if we do want to make them a *core* part of
> the build, but i think leaving it as an optional (but easy to use) part
> of the build.xml would be good enough ... in my experience, while code
> coverage test reports can be helpfull for spoting big gaps in your unit
> tests, they can lead to a false sense of security if you put too much
> stock in them...
> http://thediscoblog.com/?p=15
>
I agree. Good coverage does not mean that the quality of the tests is
high. My intention for this patch was just to find gaps in the current
testcases and to help in the future to quickly check if new patches do
not reduce the coverage. It certainly does not replace a thorough
review of patches and testcases.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-721) Code coverage reports
Posted by Chris Hostetter <ho...@fucit.org>.
: Nice. I think we can't include EMMA jars int he repository, though, so
: you'll want to add the ability to download the Jar on the fly, just like
: Grant did it for the benchmark stuff.
that's not strictly neccessary is it? ... coverage reports could just be
an optional part of the build that only works if you've got EMMA installed
-- much the way javacc is.
I haven't really looked at EMMA or this patch, but if it involves an ant
plugin then users pretty much have to install it themselves before
executing ant right?
It should also be noted, there are *many* code coverage tools ... some may
have more compatible liscences if we do want to make them a *core* part of
the build, but i think leaving it as an optional (but easy to use) part
of the build.xml would be good enough ... in my experience, while code
coverage test reports can be helpfull for spoting big gaps in your unit
tests, they can lead to a false sense of security if you put too much
stock in them...
http://thediscoblog.com/?p=15
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12451721 ]
Otis Gospodnetic commented on LUCENE-721:
-----------------------------------------
Nice. I think we can't include EMMA jars int he repository, though, so you'll want to add the ability to download the Jar on the fly, just like Grant did it for the benchmark stuff.
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-721) Code coverage reports
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-721?page=comments#action_12452544 ]
Grant Ingersoll commented on LUCENE-721:
----------------------------------------
Care to update this for Clover, Michael? Then I can hook it into the site and the nightly builds.
Thanks,
Grant
> Code coverage reports
> ---------------------
>
> Key: LUCENE-721
> URL: http://issues.apache.org/jira/browse/LUCENE-721
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Other
> Reporter: Michael Busch
> Priority: Minor
> Attachments: code_coverage.patch, emma_report.zip
>
>
> Hi all,
> We should be able to measure the code coverage of our unit testcases. I believe it would be very helpful for the committers, if they could verify before committing a patch if it does not reduce the coverage.
> Furthermore people could take a look in the code coverage reports to figure out where work needs to be done, i. e. where additional testcases are neccessary. It would be nice if we could add a page to the Lucene website showing the report, generated by the nightly build. Maybe you could add that to your preview page (LUCENE-707), Grant?
> I attach a patch here that uses the tool EMMA to generate the code coverage reports. EMMA is a very nice open-source tool released under the CPL (same license as junit). The patch adds three targets to common-build.xml:
> - emma-check: verifys if both emma.jar and emma_ant.jar are in the ant classpath
> - emma-instrument: instruments the compiled code
> - generate-emma-report: generates an html code coverage report
> The following steps are neccessary in order to generate a code coverage report:
> - add emma.jar and emma_ant.jar to your ant classpath (download emma from http://emma.sourceforge.net/)
> - execute ant target 'emma-instrument' (depends on compile-test, so it will compile all core and test classes)
> - execute ant target 'test' to run the unit tests
> - execute ant target 'generate-emma-report'
> To view the emma report open build/test/emma/index.html
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org