You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2009/07/30 19:20:14 UTC
[jira] Created: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
---------------------------------------------------------------------------------------
Key: LUCENE-1769
URL: https://issues.apache.org/jira/browse/LUCENE-1769
Project: Lucene - Java
Issue Type: Bug
Components: Build
Affects Versions: 2.9
Reporter: Uwe Schindler
This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
- the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
- the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739423#action_12739423 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
Hi Nick,
I will look into this at the end of the week, when you sent me the 2.6 snapshot build. My problems are currently:
- The tests are not directly instumented (neither in core nor contrib nor bw). Currenty the core classes and contrib classes are instrumented. The testcases are linked in clover-setup via <testsources>, if I leave this out, I get 0% coverage. The backwards tests do not appear in testsources. Because of that, the tests still call the instrumented classes, but the coverage report shows no coverage. You can easily see this e.g. for the old HitCollector backwards compatibility code. Also during running of the bw-tests, no additional files are created during tests runs in the db folder.
If I add the bw-test cases with testsources to the clover config, the clover report mixes the linkage of the tests together and you see sometimes 2 test with same name hitting the same method and so on.
Is there a possibility to tell clover, that you do not want to explicitely link the testcases of the backwards test and just measure coverage regardless of which class called the instrumented classes? I do not know, why the calls from "unknown" tests are not counted for the coverage.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791470#action_12791470 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
I verified, the latest clover version (2.6.3) with the attached license (the one attached by Nick) is working perfect.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739380#action_12739380 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
Hi Uwe,
Thanks for confirming that error. It was definitely a problem with Clover not correctly handling a unicode 4.1 character in the source file.
Clover was incorrectly asserting that if Character.isDefined(); returns false, then the character is illegal. That method only returns true for unicode Unicode 4.0 characters in java 1.5, and 1.6 I believe. I have changed the check to Character.isValidCodePoint http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Character.html#isValidCodePoint(int)
which fixes the problem.
So, I will get you a development build of Clover 2.6 with this fix and also with support for the org.apache site license by the end of this week.
I reviewed the patch, but didn't test it out. It looks good.
We could drop the encoding parameter on clover-setup. It was me trying to fix the above mentioned bug.
Cheers,
Nick
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791115#action_12791115 ]
Mark Miller commented on LUCENE-1769:
-------------------------------------
Would be cool to get this issue wrapped up ...
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1769) Fix wrong clover
analysis because of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739423#action_12739423 ]
Uwe Schindler edited comment on LUCENE-1769 at 8/5/09 3:46 AM:
---------------------------------------------------------------
Hi Nick,
I will look into this at the end of the week, when you sent me the 2.6 snapshot build. My problems are currently:
The tests are not directly instumented (neither in core nor contrib nor bw). Currenty the core classes and contrib classes are instrumented. The testcases are linked in clover-setup via <testsources>, if I leave this out, I get 0% coverage. The backwards tests do not appear in testsources. Because of that, the tests still call the instrumented classes, but the coverage report shows no coverage. You can easily see this e.g. for the old HitCollector backwards compatibility code. Also during running of the bw-tests, no additional files are created during tests runs in the db folder.
If I add the bw-test cases with testsources to the clover config, the clover report mixes the linkage of the tests together and you see sometimes 2 test with same name hitting the same method and so on.
Is there a possibility to tell clover, that you do not want to explicitely link the testcases of the backwards test and just measure coverage regardless of which class called the instrumented classes? I do not know, why the calls from "unknown" tests are not counted for the coverage.
was (Author: thetaphi):
Hi Nick,
I will look into this at the end of the week, when you sent me the 2.6 snapshot build. My problems are currently:
- The tests are not directly instumented (neither in core nor contrib nor bw). Currenty the core classes and contrib classes are instrumented. The testcases are linked in clover-setup via <testsources>, if I leave this out, I get 0% coverage. The backwards tests do not appear in testsources. Because of that, the tests still call the instrumented classes, but the coverage report shows no coverage. You can easily see this e.g. for the old HitCollector backwards compatibility code. Also during running of the bw-tests, no additional files are created during tests runs in the db folder.
If I add the bw-test cases with testsources to the clover config, the clover report mixes the linkage of the tests together and you see sometimes 2 test with same name hitting the same method and so on.
Is there a possibility to tell clover, that you do not want to explicitely link the testcases of the backwards test and just measure coverage regardless of which class called the instrumented classes? I do not know, why the calls from "unknown" tests are not counted for the coverage.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739379#action_12739379 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
Hi Nick,
I have another question about the instrumentation.
You may have noticed, that "ant test" runs all tests from core and crontrib, which are (now) correctly instrumented. After this, a special backwards branch is checked out from SVN containing all tests from Lucene 2.4 (with minor modifications when internal fields/methods were tested that changed), to test backwards compatibility. These tests are *compiled against the old 2.4 Lucene JAR* and then packaged as jar. After that these tests are *run against the trunk* (by that we test that drop-in-replacements of the Lucene JAR work with 2.9 and also that all backwards compatibility layers in Lucene core still work the same as before).
All tests in Lucene Core use only the new API, so to test the old APIs, the backwards tests do their job for the deprecated methods. The problem is now, that I cannot simply add this special build step to the clover analysis, because all bw test names are the same as in 2.9, so the linkage in the clover reports between tests and source code are completely broken. If I leave out the backwards tests (they are currently not in <testsources>, this is the current state of the latest patch), we have some "untested" code parts (because the methods are never called by core).
Is there any possibility to also fold in the coverage of the backwards tests, but without linking the test source code to it? Should we generate two clover reports (one for core, one for backwards)?
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1769) Fix wrong clover
analysis because of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737184#action_12737184 ]
Uwe Schindler edited comment on LUCENE-1769 at 7/30/09 1:16 PM:
----------------------------------------------------------------
This patch uses the features ov clover 2.0.
We can only commit this to trunk, when hudson is able to use clover 2.0. The JAR and license files are in the private contributor area [https://svn.apache.org/repos/private/committers/donated-licenses/clover/2.4.3] and can be installed on hudson. I am not sure if this can be done for Lucene Java alone or if the JAR files must be upgraded for all projects.
was (Author: thetaphi):
This patch uses the features ov clover 2.0.
We can only commit this to trunk, when hudson is able to use clover 2.0. The JAR and license files are in the private contributor are and can be installed on hudson. I am not sure if this can be done for Lucene Java alone or if the JAR files must be upgraded for all projects.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739854#action_12739854 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
I've provided a download of the jar, just for you [here|http://www.atlassian.com/software/clover/downloads/binary/clover-2.6.0-dev.jar]. Please note that this is a dev build.
This is a development build of Clover 2.6 . It has support for recognizing the org.apache site license attached to this issue.
Simply put the clover.jar and the attached clover.license in the same directory as each other, and point the taskdef task to use
the clover.jar for the classpath attribute.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791548#action_12791548 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
I committed the latest patch in revision: 891402
I keep this open because of further improvements with license and maybe ship it with lucene.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1769) Fix wrong clover
analysis because of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739423#action_12739423 ]
Uwe Schindler edited comment on LUCENE-1769 at 8/5/09 4:26 AM:
---------------------------------------------------------------
Hi Nick,
I will look into this at the end of the week, when you sent me the 2.6 snapshot build. My problems are currently:
The tests are not directly instumented (neither in core nor contrib nor bw). Currenty the core classes and contrib classes are instrumented. The testcases are linked in clover-setup via <testsources>, if I leave this out, I get 0% coverage. The backwards tests do not appear in testsources. Because of that, the tests still call the instrumented classes, but the coverage report shows no coverage. You can easily see this e.g. for the old HitCollector backwards compatibility code. Also during running of the bw-tests, no additional files are created during tests runs in the db folder.
If I add the bw-test cases with testsources to the clover config, the clover report mixes the linkage of the tests together and you see sometimes 2 test with same name hitting the same method and so on.
Is there a possibility to tell clover, that you do not want to explicitely link the testcases of the backwards test and just measure coverage regardless of which class called the instrumented classes? I do not know, why the calls from "unknown" tests are not counted for the coverage.
EDIT: If I run "ant test-tag" alone with a fresh created clover database, it seems to work. It shows the coverage in the methods called by the tests, but the tests itsself do not appear in the report (as it should be, because only the core tests are added to testsources). It seems to be only a problem, when the backwards and core tests are both instrumented to the same database. Maybe the equal testnames/classes lead to problems.
I will test this further on Friday, I have no time now, but I think, the problem seems to be solveable. Maybe I had some misunderstanding in the clover report. I will compare then the three different reports (core, backwards, core+backwards combined) for differences. Maybe all is good :)
was (Author: thetaphi):
Hi Nick,
I will look into this at the end of the week, when you sent me the 2.6 snapshot build. My problems are currently:
The tests are not directly instumented (neither in core nor contrib nor bw). Currenty the core classes and contrib classes are instrumented. The testcases are linked in clover-setup via <testsources>, if I leave this out, I get 0% coverage. The backwards tests do not appear in testsources. Because of that, the tests still call the instrumented classes, but the coverage report shows no coverage. You can easily see this e.g. for the old HitCollector backwards compatibility code. Also during running of the bw-tests, no additional files are created during tests runs in the db folder.
If I add the bw-test cases with testsources to the clover config, the clover report mixes the linkage of the tests together and you see sometimes 2 test with same name hitting the same method and so on.
Is there a possibility to tell clover, that you do not want to explicitely link the testcases of the backwards test and just measure coverage regardless of which class called the instrumented classes? I do not know, why the calls from "unknown" tests are not counted for the coverage.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: [jira] Commented: (LUCENE-1769) Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
Posted by Uwe Schindler <uw...@thetaphi.de>.
This is correct but still inefficient. The current workflow is:
- Checkout svn
- Build the source package with ant
- Build the maven artifacts (which compiles all classes)
- Then ant nightly is called, which builds the binary package and runs tests
-> this could be optimized to only build the binary package
- ant clean
- ant nightly with clover enabled -> This packages the binaries again, but
does not copy them. This is also not the best. This step should simple do
ant test with clover
- build the clover report
I only want to remove the call to the (I call it deprecated "nightly"
target, which is a relict from the time before Hudson and replace by
"package" in the first run and to "ant test" for the clover enabled version.
The build would run two times faster.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
> Sent: Friday, July 31, 2009 8:07 PM
> To: java-dev@lucene.apache.org
> Subject: Re: [jira] Commented: (LUCENE-1769) Fix wrong clover analysis
> because of backwards-tests, upgrade clover to 2.4.3 or better
>
>
> : I didn't realize the nightly build runs the tests twice (with & w/o
> : clover); I agree, running only with clover seems fine?
>
> i'm not caught up on this issue, but i happen to notice this comment in
> email.
>
> the reason the tests are run twice is because in between the two runs we
> package up the jars. clover instruments all the classes, so if we only
> ran hte tests once (w/clover), and then packaged the jars the nightly
> builds would include clover instrumented bytecode.
>
> if you look at the old Jira issues about clover this is discussed there.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by Chris Hostetter <ho...@fucit.org>.
: I didn't realize the nightly build runs the tests twice (with & w/o
: clover); I agree, running only with clover seems fine?
i'm not caught up on this issue, but i happen to notice this comment in
email.
the reason the tests are run twice is because in between the two runs we
package up the jars. clover instruments all the classes, so if we only
ran hte tests once (w/clover), and then packaged the jars the nightly
builds would include clover instrumented bytecode.
if you look at the old Jira issues about clover this is discussed there.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737566#action_12737566 ]
Michael McCandless commented on LUCENE-1769:
--------------------------------------------
bq. Maybe it is even not needed to run the tests without clover at all?
I didn't realize the nightly build runs the tests twice (with & w/o clover); I agree, running only with clover seems fine?
bq. Mike: You told me, that I should apply for a hudson account. On the Wiki page stands, that this is only possible for PMCs?
Urgh, I didn't realize that's only for PMCs. Hmm... I wonder why that is.
I have an account so I can make the necessary changes to Hudson once we work out what they are...
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739842#action_12739842 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
Hi Uwe,
Clover will still produce code coverage for your application sources, regardless of the tests being instrumented or not.
Did you try the using following fileset in clover-setup from my patch?:
{code}
<fileset dir="src">
<!-- see https://issues.apache.org/jira/browse/LUCENE-1772 -->
<include name="**/org/apache/**"/>
</fileset>
{code}
Clover will automatically detect any tests it finds as it instruments the source code.
Also, to remove the need to have the clover.jar on Ant's classpath, it is best to use the classpath on the taskdef for cloverlib.xml like so:
{code}
<property name="clover.jar" value="path/to/clover.jar"/>
<taskdef resource="cloverlib.xml" classpath="${clover.jar}"/>
{code}
This makes upgrading the version of clover being used very easy.
I will upload a dev build of Clover 2.6 for you to test with the new org.apache site license.
Cheers,
Nick
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: (was: clover.license)
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769-2.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Affects Version/s: (was: 2.9)
3.1
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: clover.license
Updated license file from Atlassian. Thanks Nicholas Muldoon! (without ASF grant attached)
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, clover.license, LUCENE-1769-2.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737481#action_12737481 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
bq. Thanks for all your help!
And thanks for all yours by making Lucene!
re: <clover-setup><testsources/>..
OK - I was not aware of the backwards-branch tests.
As long as the right tests are being instrumented Clover, and correctly detected as tests, then all should be good.
bq. The problem with that is, that we then have clover.jar inside the source distribution, which adds 9 MB of extra stuff.
I see. I didn't realise that the lib dir gets included into the source distribution. Is there another directory in svn, other than lib, that the clover.jar could live and not get included into the source distro? Or is the source distro just an archive of the entire trunk directory?
bq. And from the legal standpoint, I am not sure, if it is ok to bundle a non-free product together with Apache licensed software
Yep - not a good idea to bundle with Lucene, but it is fine to have it checked into your version control.
bq. I would like to have the clover.jar outside of the Lucene source distrib as before
Whatever you think will work best - I was just thinking that the clover.jar was not checked into svn within lucene because of the old Ant Lib requirement of clover1.
bq. What tests are failing for you? Normally all should pass at the moment (see build logs at Apache's Hudson).
One of the XML tests was failing. I was only running :
{code}
ant clover test -Drun.clover
{code}
so thought this may have been an environment flag or target I was missing.
Anyway, please let me know if you have any more questions regarding Clover2.
Cheers,
Nick
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Summary: Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better (was: Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better)
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738754#action_12738754 ]
Hoss Man commented on LUCENE-1769:
----------------------------------
bq. I only want to remove the call to the (I call it deprecated "nightly" target, which is a relict from the time before Hudson and replace by "package" in the first run and to "ant test" for the clover enabled version. The build would run two times faster.
But then we would be "publishing" artifacts before testing them. that was the whole point behind the double run when clover was first introduced: run the tests to ensure there *should* be a build, then produce the build artifacts, then run reports against the build.
i don't disagree with your goal: if we can eliminate the double test run that would be great, i'm just explaining why it is the way it is, and why removing that middle step will result in a process which isn't equivalent.
bq. We will be generating a new site license for org.apache that can also be committed to your public svn server, so anyone wanting to develop on Lucene can just use Clover out of the box, on their desktop.
Nick: if i'm understanding this comment correctly you are implying...
* that you represent Atlassian and are offering an updated license to use clover 2.x
* that Atlassian will allow us to make this license available for anyone, anywhere in the world, to use when building/developing Lucene (regardless of what their affiliation is with the ASF)
FYI: in addition to 1.3.2, Apache already has a clover.license file for 2.0.3 and 2.4.3 (Lucene just never got around to using them in our build system) and these are checked into the "private" svn repository (ie: only accessible to committers). The fact that these licenses are committed into the private repository suggests that when they were granted to the ASF, it was with some instructions that it only be made available to people who were directly associated with the AF, and not made publicly available (The README files associated with the licenses don't contain any specific instructions either way, so the location in "private/committers" suggests it's access restriction). I believe that has been the general understanding of many ASF projects, and is the basis for the current hacks in the Lucene build system -- it was never an issue of where to put clover.jar, it was an issue of making sure anyone could use the build.xml file even without access to a clover.license.
If we truly can make both the license publicly available then we could simplify a lot of things by commiting clover.license into the source tree (and including in source releases), and having the build file download the clover.jar as needed if people set the build property to run the tests in clover.
If i am in fact understanding this all correctly, then the best way to proceed is if you (or someone else with an @atlassian.com address) could send an "official" email to java-dev@lucene spelling out exactly how the license can be used/disseminated. then one of the lucene committers can commit the new license into both the private/committers repository along with the full email so that there won't be any confusion for other developers in other projects about how it can be shared. .... then we copy the license to the lucene repository and commit the simplifications to the lucene build system.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738893#action_12738893 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
{quote}
bq. I only want to remove the call to the (I call it deprecated "nightly" target, which is a relict from the time before Hudson and replace by "package" in the first run and to "ant test" for the clover enabled version. The build would run two times faster.
But then we would be "publishing" artifacts before testing them. that was the whole point behind the double run when clover was first introduced: run the tests to ensure there *should* be a build, then produce the build artifacts, then run reports against the build.
i don't disagree with your goal: if we can eliminate the double test run that would be great, i'm just explaining why it is the way it is, and why removing that middle step will result in a process which isn't equivalent.
{quote}
I agree, the solution for this would be to reorder the build steps. As far as I know (I have seen this from the previous hudson builds), the build process will stop, if of two separate ANT runs the first one failes (I assume, hudson checks the exit code). If this is the case, the order could be changed to:
- Checkout svn
- Build the source package with ant
- "ant test" with clover enabled
- build the clover report
- "ant clean"
- Build the maven artifacts (which compiles all classes)
- "ant package" is called, which builds the binary package
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-1769.
-----------------------------------
Resolution: Fixed
Assignee: Uwe Schindler
It seems to work, so closing this issue.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Attachments: clover.license, clover.license, LUCENE-1769-2.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739388#action_12739388 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
bq. we have some "untested" code parts (because the methods are never called by core).
Are these untested parts in the test cases? To generate the code coverage of an application, the test cases don't need to be instrumented. Instrumentation of tests cases is needed only for "per-test coverage", which you may not need for your regression tests
bq. so the linkage in the clover reports between tests and source code are completely broken.
In what way is it broken? Are the test method names missing, or do you have duplicate test methods? There should be one testMethod listed for each test run that occured.
bq. Is there any possibility to also fold in the coverage of the backwards tests, but without linking the test source code to it?
I'm guessing that not instrumenting your old tests should do what you want.
bq. Should we generate two clover reports (one for core, one for backwards)?
This is also an option. Clover2 will link the reports together if you generate them in the same <clover-report/> block.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: LUCENE-1769.patch
Here updated patch for current trunk.
I would like to commit this, but it would break hudson. Somebody with hudson access (command line + config) should do the following:
- download [http://www.atlassian.com/software/clover/downloads/binary/clover-ant-2.5.1.zip] and extract the clover.jar
- download [https://svn.apache.org/repos/private/committers/donated-licenses/clover/2.4.3/clover.license]
- put both files into a directory that is persistent on hudson
- reconfigure hudson to let the ANT_LIB path point to this dir
- I will commit my patch
...and then we have fresh and complete reports after the next run.
I verified: All runs are instrumented and linkage with tests work. Also test-tag is instrumented, but lnk to tests is missing (this is wanted), as the trunk tests count.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791567#action_12791567 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
Howto to install locally:
- download [http://repo2.maven.org/maven2/com/cenqua/clover/clover/2.6.3/clover-2.6.3.jar]
- fetch the attached license [https://issues.apache.org/jira/secure/attachment/12415461/clover.license]
put both in your ANT lib path - have fun.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739859#action_12739859 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
When this is integrated and committed to svn, someone from Atlassian will send the following email to the lucene mailing list confirming the usage and terms of the site license.
{quote}
This Clover license can be used for any code that is under an org.apache package. Further, this license can be used by any developer on their machine in conjunction with our Eclipse or IntelliJ plugins for development on an org.apache project.
{quote}
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: LUCENE-1769-2.patch
Improvements, will commit soon:
- separate core/contrib and tag tests completely. Currently the xml files for test report generation are mixed together, so tag tests overwrite core results. This was by the way the reason for the defect in clover 1.x - only recorded test runs were instrumented. Also contrib was missing. The junit report now generates a separate report for test-tag
- the test results are now also used in clover analysis. This helps when real JUnit4 tests are used.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769-2.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Fix Version/s: 3.1
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 3.1
>
> Attachments: clover.license, clover.license, LUCENE-1769-2.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737366#action_12737366 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
HI Uwe,
Great work getting Clover upgraded. I've also done an upgrade, although have some test failures on my local machine. I am pretty sure these are not caused by Clover, however would love for you to double check.
Clover 2 does not require the clover.jar to be installed into an Ant Lib, so I suggest checking it in trunk/lib/ , right next to junit.jar .
We will be generating a new site license for org.apache that can also be committed to your public svn server, so anyone wanting to develop on Lucene can just use Clover out of the box, on their desktop.
Some other things I did was to change the fileset in the clover-setup task to:
{code}
<fileset dir="src">
<!-- see https://issues.apache.org/jira/browse/LUCENE-1772 -->
<include name="**/org/apache/**"/>
<include name="**/lucli/**"/>
<include name="**/com/example/**"/>
<exclude name="**/org/apache/lucene/analysis/TestASCIIFoldingFilter.java"/> <!-- Illegal char on line 190-->
</fileset>
{code}
This means that all of the src and test java files in contrib will also be instrumented by clover, if that is what you want?
I also removed the clover.enabled property, since clover should always be available from now on.
I added the <testsources/> element to both the HTML and the XML report tasks too. This ensures that the coverage of your test cases themselves are reported in a seperate section of each report.
Other than that, I've tried to leave things as they were.
Please let me know if you have any questions, or problems!
The 2.4.3 license will work - we will have a site license available next week.
Cheers,
Nick
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1769) Fix wrong clover
analysis because of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737479#action_12737479 ]
Uwe Schindler edited comment on LUCENE-1769 at 7/31/09 3:05 AM:
----------------------------------------------------------------
Here an updated patch with merged changes.
There is currently a bug in the hudson build script:
The new clover version creates very large additional class files when instrumenting the source, which also go into the TAR file.
Hudson builds this tar file without clover and publishes it, which is ok. But later hudson runs all tests a second time, with clover enabled. Hudson should do this only with "ant -Drun.clover=true clean test generate-clover-reports" (and not "nightly" instead of "test"). Because of this the TAR files are regenerated with instrumented classes.
EDIT: This is not a problem anymore (the TAR file is copied before to the artifacts dir), nevertheless the target "nightly" should not be used. Instead Hudson should run "test". With/without clover. Maybe it is even not needed to run the tests without clover at all? Why two times?
Mike: You told me, that I should apply for a hudson account. On the Wiki page stands, that this is only possible for PMCs?
was (Author: thetaphi):
Here an updated patch with merged changes.
There is currently a bug in the hudson build script:
The new clover version creates very large additional class files when instrumenting the source, which also go into the TAR file.
Hudson builds this tar file without clover and publishes it, which is ok. But later hudson runs all tests a second time, with clover enabled. Hudson should do this only with "ant -Drun.clover=true clean test generate-clover-reports" (and not "nightly" instead of "test"). Because of this the TAR files are regenerated with instrumented classes.
Mike: You told me, that I should apply for a hudson account. On the Wiki page stands, that this is only possible for PMCs?
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: LUCENE-1769.patch
Further optimization to instrument all files, also test. Also add the test for ASCII folding back (was bug in 2.5.1).
When hudson is reconfigured, this should be ready for commit.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1769) Fix wrong clover
analysis because of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737366#action_12737366 ]
Nick Pellow edited comment on LUCENE-1769 at 7/30/09 4:59 PM:
--------------------------------------------------------------
HI Uwe,
Great work getting Clover upgraded. I've also done an upgrade, although have some test failures on my local machine. I am pretty sure these are not caused by Clover, however would love for you to double check.
Clover 2 does not require the clover.jar to be installed into an Ant Lib, so I suggest checking it in trunk/lib/ , right next to junit.jar .
We will be generating a new site license for org.apache that can also be committed to your public svn server, so anyone wanting to develop on Lucene can just use Clover out of the box, on their desktop.
Some other things I did was to change the fileset in the clover-setup task to:
{code}
<fileset dir="src">
<!-- see https://issues.apache.org/jira/browse/LUCENE-1772 -->
<include name="**/org/apache/**"/>
<include name="**/lucli/**"/>
<include name="**/com/example/**"/>
<exclude name="**/org/apache/lucene/analysis/TestASCIIFoldingFilter.java"/> <!-- Illegal char on line 190-->
</fileset>
{code}
This means that all of the src and test java files in contrib will also be instrumented by clover, if that is what you want?
I also removed the clover.enabled property, since clover should always be available from now on.
I added the <testsources/> element to both the HTML and the XML report tasks too. This ensures that the coverage of your test cases themselves are reported in a seperate section of each report.
I would also suggest we remove the clover.run property. Simply calling the 'clover' target at the start of the build will be enough to have clover swtiched on for the build.
Other than that, I've tried to leave things as they were.
Please let me know if you have any questions, or problems!
The 2.4.3 license will work - we will have a site license available next week.
Cheers,
Nick
was (Author: npellow):
HI Uwe,
Great work getting Clover upgraded. I've also done an upgrade, although have some test failures on my local machine. I am pretty sure these are not caused by Clover, however would love for you to double check.
Clover 2 does not require the clover.jar to be installed into an Ant Lib, so I suggest checking it in trunk/lib/ , right next to junit.jar .
We will be generating a new site license for org.apache that can also be committed to your public svn server, so anyone wanting to develop on Lucene can just use Clover out of the box, on their desktop.
Some other things I did was to change the fileset in the clover-setup task to:
{code}
<fileset dir="src">
<!-- see https://issues.apache.org/jira/browse/LUCENE-1772 -->
<include name="**/org/apache/**"/>
<include name="**/lucli/**"/>
<include name="**/com/example/**"/>
<exclude name="**/org/apache/lucene/analysis/TestASCIIFoldingFilter.java"/> <!-- Illegal char on line 190-->
</fileset>
{code}
This means that all of the src and test java files in contrib will also be instrumented by clover, if that is what you want?
I also removed the clover.enabled property, since clover should always be available from now on.
I added the <testsources/> element to both the HTML and the XML report tasks too. This ensures that the coverage of your test cases themselves are reported in a seperate section of each report.
Other than that, I've tried to leave things as they were.
Please let me know if you have any questions, or problems!
The 2.4.3 license will work - we will have a site license available next week.
Cheers,
Nick
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739922#action_12739922 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
I will look into this later. Thanks for the JAR and license!
bq. Clover will automatically detect any tests it finds as it instruments the source code.
But they must not be tests nor available as source code? I say this, because the backwards tests are not available as source code, they are only packaged in a JAR and then run against the trunk Lucene JAR (instrumented). So there is nothing that could be detected as test from source code.
I will try out and verify as I noted in the EDIT of my last comment. It seems that the code was instrumented without any linkage to test source code (it was enough that the instrumented code was ran).
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: LUCENE-1769.patch
Here an updated patch with merged changes.
There is currently a bug in the hudson build script:
The new clover version creates very large additional class files when instrumenting the source, which also go into the TAR file.
Hudson builds this tar file without clover and publishes it, which is ok. But later hudson runs all tests a second time, with clover enabled. Hudson should do this only with "ant -Drun.clover=true clean test generate-clover-reports" (and not "nightly" instead of "test"). Because of this the TAR files are regenerated with instrumented classes.
Mike: You told me, that I should apply for a hudson account. On the Wiki page stands, that this is only possible for PMCs?
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737671#action_12737671 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
This discussion was on java-dev, I copy it here for completeness:
> : I didn't realize the nightly build runs the tests twice (with & w/o
> : clover); I agree, running only with clover seems fine?
>
> i'm not caught up on this issue, but i happen to notice this comment in
> email.
>
> the reason the tests are run twice is because in between the two runs we
> package up the jars. clover instruments all the classes, so if we only
> ran hte tests once (w/clover), and then packaged the jars the nightly
> builds would include clover instrumented bytecode.
>
> if you look at the old Jira issues about clover this is discussed there.
>
> -Hoss
This is correct but still inefficient. The current workflow is:
- Checkout svn
- Build the source package with ant
- Build the maven artifacts (which compiles all classes)
- Then ant nightly is called, which builds the binary package and runs tests -> this could be optimized to only build the binary package
- ant clean
- ant nightly with clover enabled -> This packages the binaries again, but does not copy them. This is also not the best. This step should simple do ant test with clover
- build the clover report
I only want to remove the call to the (I call it deprecated "nightly" target, which is a relict from the time before Hudson and replace by "package" in the first run and to "ant test" for the clover enabled version. The build would run two times faster.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: LUCENE-1769.patch
This patch uses the features ov clover 2.0.
We can only commit this to trunk, when hudson is able to use clover 2.0. The JAR and license files are in the private contributor are and can be installed on hudson. I am not sure if this can be done for Lucene Java alone or if the JAR files must be upgraded for all projects.
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737444#action_12737444 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
Thanks for all your help!
bq. This means that all of the src and test java files in contrib will also be instrumented by clover, if that is what you want?
Not really, my intention was to only instrument the source files using the tests (I can see this, that during compilation it tells me that XXX files are instrumented). This is how it was before.
The <testsources> tag in the clover setup was added, because the backwards-branch tests (ant test-tag) completely broke the clover run (as clover was not able to automatically figure out, which are the real test files that should be run to instrument the source - maybe because the backwards branch is checked out on the fly using svn). Because of this I added explicitely the source files to <testsources> in the init task - see my patch. Since I did this, it works as exspected. I used the already defined variables for that.
Without this <testsources> you cannot see any instrumentation data generated during run of tests (see above), it seems, that clover only instruments during the backwards-branch tests. With the explicit tests in <testsources> it works as exspected.
bq. I added the <testsources/> element to both the HTML and the XML report tasks too. This ensures that the coverage of your test cases themselves are reported in a seperate section of each report.
This is good, because without that clover-report does not see the contrib tests, only the core tests (during the compilation and instrumentation of contrib it works, as each contrib is run in a subant. When building the main report, no subants are involved and the report is only generated at top-level, where the tests are not available.
bq. Clover 2 does not require the clover.jar to be installed into an Ant Lib, so I suggest checking it in trunk/lib/ , right next to junit.jar. We will be generating a new site license for org.apache that can also be committed to your public svn server, so anyone wanting to develop on Lucene can just use Clover out of the box, on their desktop.
The problem with that is, that we then have clover.jar inside the source distribution, which adds 9 MB of extra stuff. Also I am not sure, if it is good, to also add the clover license to that. And from the legal standpoint, I am not sure, if it is ok to bundle a non-free product together with Apache licensed software?
I would like to have the clover.jar outside of the Lucene source distrib as before and e.g. only enable clover on our hudson build servers. If somebody do not want to put his clover.jar into the ant build dir, we could add a property that can be added to the local build.properties file. Or the user can run with "ant -lib ...". Alternatively we could add a ant task, that download the clover license and jar from Apache's private Committer SVN. We do this with other JAR files, too. I will think about it.
I will prepare a patch and merge your's and mine changes and restore the testsources in the clover-setup task.
bq. although have some test failures on my local machine. I am pretty sure these are not caused by Clover, however would love for you to double check.
What tests are failing for you? Normally all should pass at the moment (see build logs at Apache's Hudson).
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739375#action_12739375 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
> Hi Uwe,
> I noticed that in your patch to integrate Clover, you also excluded
> the following from instrumentation:
> <exclude name="org/apache/lucene/analysis/
> TestASCIIFoldingFilter.java" /><!-- Class too large for clover -->
> Do you remember the error you were getting when Clover tried to
> instrument this file?
>
> I was getting:
> [clover] /Users/niick/work/hudson/plugins/clover/work/jobs/Lucene/
> workspace/trunk/src/test/org/apache/lucene/analysis/
> TestASCIIFoldingFilter.java:190:10:expecting '"', found '?'
Hi Nick,
this was the same error I got. I thought it may have to do with charset encoding (how does Clover detect the encoding? from the javac parameters like -source 1.4 and so on?). I have seen you added an encoding parameter to clover-setup, but I was not sure, which encoding was meant (encoding of source files, encoding of html report,... - as there was no documentation about it in clover's docs).
In the last patch on the JIRA issue, I already included the large ASCIIFoldingFilter to instrumentation (in the first patch, I only preserved all config from the Clover 1.x analysis to be sure to have a working FileSet). I then noticed, that Clover 2.4 was able to instrument this file and removed the exclusion.
If the charset bug is fixed in 2.6, I will then remove the ASCIIFolding Test exclusion from the patch, too.
Did you review and tried out my last patch?
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.6.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1769:
----------------------------------
Attachment: clover.license
Updated license file from Atlassian. Thanks Nicholas Muldoon!
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.6.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769-2.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1769) Fix wrong clover analysis because of
backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nick Pellow updated LUCENE-1769:
--------------------------------
Attachment: nicks-LUCENE-1769.patch
a patch to upgrade to clover 2.5.1 .
Clover 2.5.1 can be download directly from: http://www.atlassian.com/software/clover/downloads/binary/clover-ant-2.5.1.zip
This should be stored in trunk/lib/clover.jar .
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791241#action_12791241 ]
Uwe Schindler commented on LUCENE-1769:
---------------------------------------
For fixing the clover analysis on hudson, the patch must be applied and the license and JAR file copied to hudson. I will try out again during the day and verify everything.
Currently we should think about the new clover license and upgrade to the latest version. Our "official" one in SVN's committers area 2.4.3 also works for this (I will try this out, too).
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: clover.license, LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1769) Fix wrong clover analysis because
of backwards-tests, upgrade clover to 2.4.3 or better
Posted by "Nick Pellow (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738840#action_12738840 ]
Nick Pellow commented on LUCENE-1769:
-------------------------------------
Hi Hoss,
{quote}
Nick: if i'm understanding this comment correctly you are implying...
* that you represent Atlassian and are offering an updated license to use clover 2.x
* that Atlassian will allow us to make this license available for anyone, anywhere in the world, to use when building/developing Lucene (regardless of what their affiliation is with the ASF)
{quote}
Correct on both counts, except the updated license will only work with the upcoming release of Clover 2.6, and onwards.
bq. The README files associated with the licenses don't contain any specific instructions either way, so the location in "private/committers" suggests it's access restrictions
I can't comment on that clover license. It may well need to stay in the private repo.
{quote}
If we truly can make both the license publicly available then we could simplify a lot of things by commiting clover.license into the source tree
{quote}
That is exactly the idea. The license will also work inside the Clover IDEA and Eclipse plugins for anyone, anywhere developing on org.apache projects.
If you don't wish to check the clover.jar into subversion, the clover.jar can always be downloaded directly from the maven central repository: http://repo2.maven.org/maven2/com/cenqua/clover/clover/ (when version 2.6.0 arrives ;) .
So, to move forward:
* I will have an email sent to java-dev@lucene spelling out exactly how the license can be used/disseminated
* I will get you an EAP of Clover 2.6 with the upgraded licensing code
* I will also attach a copy of the new Clover Apache Site license to this ticket for someone to commit to your repository.
Please let me know if there is anything we can do, or if you have any further questions/suggestions!
Cheers,
Nick
> Fix wrong clover analysis because of backwards-tests, upgrade clover to 2.4.3 or better
> ---------------------------------------------------------------------------------------
>
> Key: LUCENE-1769
> URL: https://issues.apache.org/jira/browse/LUCENE-1769
> Project: Lucene - Java
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1769.patch, LUCENE-1769.patch, nicks-LUCENE-1769.patch
>
>
> This is a followup for [http://www.lucidimagination.com/search/document/6248d6eafbe10ef4/build_failed_in_hudson_lucene_trunk_902]
> The problem with clover running on hudson is, that it does not instrument all tests ran. The autodetection of clover 1.x is not able to find out which files are the correct tests and only instruments the backwards test. Because of this, the current coverage report is only from the backwards tests running against the current Lucene JAR.
> You can see this, if you install clover and start the tests. During test-core no clover data is added to the db, only when backwards-tests begin, new files are created in the clover db folder.
> Clover 2.x supports a new ant task, <testsources> that can be used to specify the files, that are the tests. It works here locally with clover 2.4.3 and produces a really nice coverage report, also linking with test files work, it tells which tests failed and so on.
> I will attach a patch, that changes common-build.xml to the new clover version (other initialization resource) and tells clover where to find the tests (using the test folder include/exclude properties).
> One problem with the current patch: It does *not* instrument the backwards branch, so you see only coverage of the core/contrib tests. Getting the coverage also from the backwards tests is not easy possible because of two things:
> - the tag test dir is not easy to find out and add to <testsources> element (there may be only one of them)
> - the test names in BW branch are identical to the trunk tests. This completely corrupts the linkage between tests and code in the coverage report.
> In principle the best would be to generate a second coverage report for the backwards branch with a separate clover DB. The attached patch does not instrument the bw branch, it only does trunk tests.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org