You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by David Martín Nieto <da...@gmail.com> on 2011/03/24 15:10:16 UTC

Derby Testing Problems

Hello,

I've been launching the test of the Derby DB and I can't achieve 100% 
success in this testing. I put on some errors if anyone could comment 
any way to solve them.

I think the right thing is to obtain a 100% success rate in tests 
without Failures or Errors, so as to properly test a modification of the 
code that I have developed and see if after this modification in the 
source maintains the 100% success with the modification proving its 
correct. Am right?

At first launching the 
org.apache.derbyTesting.functionTests.tests.lang._Suite I obtains 158 
failures that I con group in types because they are similars:

    * |junit.framework.AssertionFailedError
      at
      org.apache.derbyTesting.functionTests.tests.lang.DynamicLikeOptimizationTest.testDynamicLikeOptimization(DynamicLikeOptimizationTest.java:548)
      at
      org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)|
    * |junit.framework.AssertionFailedError
      at
      org.apache.derbyTesting.functionTests.tests.lang.DynamicLikeOptimizationTest.testDynamicLikeOptimization(DynamicLikeOptimizationTest.java:548)
      at
      org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)|
    * |junit.framework.AssertionFailedError
      at
      org.apache.derbyTesting.functionTests.tests.lang.InbetweenTest.testInBetween(InbetweenTest.java:1660)
      at
      org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)|

And a lot more of this type in diferent classes of the tests.

Also in the same tests I obtains 14 Errors, like:

    * |java.lang.AssertionError: Sequence not found in statistics
      at
      org.apache.derbyTesting.junit.RuntimeStatisticsParser.assertSequence(RuntimeStatisticsParser.java:497)
      at org.apache.derbyTesting.junit.JDBC.checkPlan(JDBC.java:1565)
      at
      org.apache.derbyTesting.functionTests.tests.lang.LojReorderTest.test_LojReorderTest_a(LojReorderTest.java:724)
      at
      org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)|
    * |java.lang.AssertionError: Sequence not found in statistics
      at
      org.apache.derbyTesting.junit.RuntimeStatisticsParser.assertSequence(RuntimeStatisticsParser.java:497)
      at org.apache.derbyTesting.junit.JDBC.checkPlan(JDBC.java:1565)
      at
      org.apache.derbyTesting.functionTests.tests.lang.LojReorderTest.test_LojReorderTest_b(LojReorderTest.java:2179)
      at
      org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)|
    * |java.lang.AssertionError: Sequence not found in statistics
      at
      org.apache.derbyTesting.junit.RuntimeStatisticsParser.assertSequence(RuntimeStatisticsParser.java:497)
      at org.apache.derbyTesting.junit.JDBC.checkPlan(JDBC.java:1565)
      at
      org.apache.derbyTesting.functionTests.tests.lang.OuterJoinTest.testDerby_4471d(OuterJoinTest.java:2850)
      at
      org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at
      org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)|

Thanks.

David Martin

Re: Derby Testing Problems

Posted by Myrna van Lunteren <m....@gmail.com>.
On Thu, Mar 24, 2011 at 7:10 AM, David Martín Nieto
<da...@gmail.com> wrote:
> Hello,
>
> I've been launching the test of the Derby DB and I can't achieve 100%
> success in this testing. I put on some errors if anyone could comment any
> way to solve them.
>
> I think the right thing is to obtain a 100% success rate in tests without
> Failures or Errors, so as to properly test a modification of the code that I
> have developed and see if after this modification in the source maintains
> the 100% success with the modification proving its correct. Am right?
>
> At first launching the
> org.apache.derbyTesting.functionTests.tests.lang._Suite I obtains 158
> failures that I con group in types because they are similars:
>
> junit.framework.AssertionFailedError
> at
> org.apache.derbyTesting.functionTests.tests.lang.DynamicLikeOptimizationTest.testDynamicLikeOptimization(DynamicLikeOptimizationTest.java:548)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> junit.framework.AssertionFailedError
> at
> org.apache.derbyTesting.functionTests.tests.lang.DynamicLikeOptimizationTest.testDynamicLikeOptimization(DynamicLikeOptimizationTest.java:548)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> junit.framework.AssertionFailedError
> at
> org.apache.derbyTesting.functionTests.tests.lang.InbetweenTest.testInBetween(InbetweenTest.java:1660)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
>
> And a lot more of this type in diferent classes of the tests.
>
> Also in the same tests I obtains 14 Errors, like:
>
> java.lang.AssertionError: Sequence not found in statistics
> at
> org.apache.derbyTesting.junit.RuntimeStatisticsParser.assertSequence(RuntimeStatisticsParser.java:497)
> at org.apache.derbyTesting.junit.JDBC.checkPlan(JDBC.java:1565)
> at
> org.apache.derbyTesting.functionTests.tests.lang.LojReorderTest.test_LojReorderTest_a(LojReorderTest.java:724)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> java.lang.AssertionError: Sequence not found in statistics
> at
> org.apache.derbyTesting.junit.RuntimeStatisticsParser.assertSequence(RuntimeStatisticsParser.java:497)
> at org.apache.derbyTesting.junit.JDBC.checkPlan(JDBC.java:1565)
> at
> org.apache.derbyTesting.functionTests.tests.lang.LojReorderTest.test_LojReorderTest_b(LojReorderTest.java:2179)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> java.lang.AssertionError: Sequence not found in statistics
> at
> org.apache.derbyTesting.junit.RuntimeStatisticsParser.assertSequence(RuntimeStatisticsParser.java:497)
> at org.apache.derbyTesting.junit.JDBC.checkPlan(JDBC.java:1565)
> at
> org.apache.derbyTesting.functionTests.tests.lang.OuterJoinTest.testDerby_4471d(OuterJoinTest.java:2850)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
> at junit.extensions.TestSetup.run(TestSetup.java:27)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
>
> Thanks.
>
> David Martin
>
>
>

Hi David,

Yes, you're right, we expect a 100% success rate in the tests. There
are a few 'known' intermittent test failures that we've not gotten our
hands/heads around, but for each of those there should be an existing
entry in JIRA.

These failures do not look like any of the intermittent failures I
recognize. I'm assuming you're running off trunk...Recently there have
been changes in the automatic index statistics functionality, so
perhaps your tree has gotten in some in between state.

I have the following suggestions:
- after making a patch of your current changes (to save anything if
something goes wrong; save svn diff to a file, or a comparable action
if you're using an IDE) get to a completely clean build environment
followed by svn update. So, first do ant clobber, and delete (if
they're left) the jars and classes and generated directories. Then svn
update to get to the most recent state, then redo the build.
- run the tests in a clean directory, not in one where you've run tests before.

Hope this helps...
Myrna