You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by TestSuiteReductionResearch <gi...@git.apache.org> on 2016/07/15 17:32:50 UTC
[GitHub] commons-lang pull request #171: Removing test redundant for org.apache.commo...
GitHub user TestSuiteReductionResearch opened a pull request:
https://github.com/apache/commons-lang/pull/171
Removing test redundant for org.apache.commons.lang3.ClassUtilsTest.testWrapperToPrimitiveNoWrapper
We are researchers working on identifying redundant tests in a test suite. Our analysis of finding redundant tests involve each test's dynamic code coverage and their potential fault-detection capability.
Through our analysis, we found that the tests org.apache.commons.lang3.ClassUtilsTest.testWrapperToPrimitiveNoWrapper, org.apache.commons.lang3.ClassUtilsTest.testWrapperToPrimitiveNull are redundant with respect to one another. In this pull request, we are proposing to keep only the test org.apache.commons.lang3.ClassUtilsTest.testWrapperToPrimitiveNoWrapper while adding @Ignore annotations to the remaining test. However, as we believe these tests are identical, any one of them can be kept while skipping the remaining test.
If you do not believe any of these tests should be ignored, we would greatly appreciate it if you could follow up on this pull request and let us know your reasons.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/TestSuiteReductionResearch/commons-lang redundant0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/commons-lang/pull/171.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #171
----
commit ed27ea4132fbe36d0aec804f21b960c9505f80b0
Author: TestSuiteReductionResearch <te...@gmail.com>
Date: 2016-07-15T17:31:14Z
Ignoring test that is redundant for org.apache.commons.lang3.ClassUtilsTest.testWrapperToPrimitiveNoWrapper
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by august782 <gi...@git.apache.org>.
Github user august782 commented on the issue:
https://github.com/apache/commons-lang/pull/171
Thank you very much for the feedback! It does make sense that you want to keep tests for different scenarios even if they do have the same coverage.
Besides these two tests, our analysis actually did discover other tests that are redundant with one another in the test suite. Would you be open to us sending some more pull requests concerning removing redundant tests?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by TestSuiteReductionResearch <gi...@git.apache.org>.
Github user TestSuiteReductionResearch commented on the issue:
https://github.com/apache/commons-lang/pull/171
Thank you again for providing more feedback. I believe our analysis did find some other tests that do seem to be covering the same scenarios, and we will be sending them as separate pull requests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by TestSuiteReductionResearch <gi...@git.apache.org>.
Github user TestSuiteReductionResearch commented on the issue:
https://github.com/apache/commons-lang/pull/171
Thank you very much for the feedback! It does make sense that you want to keep tests for different scenarios even if they do have the same coverage.
Besides these two tests, our analysis actually did discover other tests that are redundant with one another in the test suite. Would you be open to us sending some more pull requests concerning removing redundant tests?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/commons-lang/pull/171
I agree, -1 to remove the test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/commons-lang/pull/171
[](https://coveralls.io/builds/7027718)
Coverage increased (+0.03%) to 93.468% when pulling **ed27ea4132fbe36d0aec804f21b960c9505f80b0 on TestSuiteReductionResearch:redundant0** into **1d1883f0e20215720eb583fd43eee088808c616b on apache:master**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang pull request #171: Removing test redundant for org.apache.commo...
Posted by TestSuiteReductionResearch <gi...@git.apache.org>.
Github user TestSuiteReductionResearch closed the pull request at:
https://github.com/apache/commons-lang/pull/171
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by kinow <gi...@git.apache.org>.
Github user kinow commented on the issue:
https://github.com/apache/commons-lang/pull/171
Both tests use ClassUtils#wrapperToPrimitive. One passes a String.class, and the other a null value. Then the method looks at a map, created and filled in a static constructor.
Since String.class is not in the map, it returns a null value.
Then trying to get a value for the key null, also returns a null value.
I agree that, considering code coverage, both tests are redundant. But they are assessing two different things, as defined in the method names. One assessing that asking for null will return a null. And the other that requesting an element that is not in the map also returns null.
Even though they may be redundant now, something may change in the future, and having tests covering scenarios, regardless of coverage, is a good thing too.
>If you do not believe any of these tests should be ignored, we would greatly appreciate it if you could follow up on this pull request and let us know your reasons.
So for this, I'm -1 for removing the test based only on coverage. They may be redundant, looking from a test coverage perspective, but as a developer using that method, I expect to be able to quickly understand what's going to happen in each scenario I use it. Tests and Javadoc with some examples, covering these scenarios, can be extremely useful.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] commons-lang issue #171: Removing test redundant for org.apache.commons.lang...
Posted by kinow <gi...@git.apache.org>.
Github user kinow commented on the issue:
https://github.com/apache/commons-lang/pull/171
@august782, thanks for being understanding. I think finding redundant tests, and simplifying it is a good idea, just not in the case of tests covering many different scenarios.
I think many tests in Apache Commons components are written with scenarios in mind, and method-oriented. As normally the components are libraries to be included in other systems, the tests aim at asserting that methods work as expected, under different conditions (i.e. with a diverse range of possible arguments).
Having good code coverage is important as well, but tests are not being written in Apache Commons components simply to have a high score in the reports - though normally covering well your scenarios, you end up with a good branch and line test coverage as well (citation needed :) ).
So if you have found redundant tests, where the same scenario is being covered in multiple tests, and you believe that reducing it will i) benefit users and developers in understanding the code, and ii) still cover the possible use cases for the library, or iii) make it easier to assert the same thing, then I think these pull requests would be useful.
For other pull requests that disable or remove tests based only on line and or branch coverage, I would still argue against removing them, and in favor to keep them; for the reasons I stated before.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---