You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Knut Anders Hatlen (JIRA)" <ji...@apache.org> on 2011/03/08 13:58:59 UTC
[jira] Created: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
ProcedureInTriggerTest depends on implicit ordering of test cases
-----------------------------------------------------------------
Key: DERBY-5103
URL: https://issues.apache.org/jira/browse/DERBY-5103
Project: Derby
Issue Type: Bug
Components: Test
Affects Versions: 10.8.0.0
Reporter: Knut Anders Hatlen
If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
at junit.extensions.TestSetup.run(TestSetup.java:25)
at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-5103:
--------------------------------------
Attachment: derby-5103-2a.diff
There's still an intermittent failure in this test seen in the nightly runs: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/sol/1080669-suitesAll_diff.txt
When testTriggerNegative runs early, it may alter the ordering of rows returned in some other test cases because it creates an index on T1.
The attached patch makes testTriggerNegative drop the index in the cleanup section.
Committed revision 1081081.
> ProcedureInTriggerTest depends on implicit ordering of test cases
> -----------------------------------------------------------------
>
> Key: DERBY-5103
> URL: https://issues.apache.org/jira/browse/DERBY-5103
> Project: Derby
> Issue Type: Bug
> Components: Test
> Affects Versions: 10.8.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Fix For: 10.8.0.0
>
> Attachments: derby-5103-1a.diff, derby-5103-2a.diff, rjall.out
>
>
> If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
> 1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
> at junit.extensions.TestSetup.run(TestSetup.java:25)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
> Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen resolved DERBY-5103.
---------------------------------------
Resolution: Fixed
Fix Version/s: 10.8.0.0
Assignee: Knut Anders Hatlen
Committed revision 1079805.
> ProcedureInTriggerTest depends on implicit ordering of test cases
> -----------------------------------------------------------------
>
> Key: DERBY-5103
> URL: https://issues.apache.org/jira/browse/DERBY-5103
> Project: Derby
> Issue Type: Bug
> Components: Test
> Affects Versions: 10.8.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Fix For: 10.8.0.0
>
> Attachments: derby-5103-1a.diff, rjall.out
>
>
> If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
> 1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
> at junit.extensions.TestSetup.run(TestSetup.java:25)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
> Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
Posted by "Rick Hillegas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004047#comment-13004047 ]
Rick Hillegas commented on DERBY-5103:
--------------------------------------
I ran this test on linux on Java 7 build 132. I could not reproduce the unusual ordering of test cases which is turning up in the Java 7 test runs on Solaris. I ran the test 10 times without errors. I also tried running some other tests which show instability on Solaris on Java 7. Again, I am not able to reproduce these instabilities on linux on Java 7 build 132.
> ProcedureInTriggerTest depends on implicit ordering of test cases
> -----------------------------------------------------------------
>
> Key: DERBY-5103
> URL: https://issues.apache.org/jira/browse/DERBY-5103
> Project: Derby
> Issue Type: Bug
> Components: Test
> Affects Versions: 10.8.0.0
> Reporter: Knut Anders Hatlen
>
> If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
> 1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
> at junit.extensions.TestSetup.run(TestSetup.java:25)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
> Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
Posted by "Lily Wei (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lily Wei updated DERBY-5103:
----------------------------
Attachment: rjall.out
I ran this test on Windows 7 on Java 7 build 132. I saw this same stack trace on my machine:
4) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.Proce
dureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<4>
at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTe
st.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTe
st.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:
112)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:23)
at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57
)
I am attaching my output rjall.out here.
> ProcedureInTriggerTest depends on implicit ordering of test cases
> -----------------------------------------------------------------
>
> Key: DERBY-5103
> URL: https://issues.apache.org/jira/browse/DERBY-5103
> Project: Derby
> Issue Type: Bug
> Components: Test
> Affects Versions: 10.8.0.0
> Reporter: Knut Anders Hatlen
> Attachments: rjall.out
>
>
> If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
> 1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
> at junit.extensions.TestSetup.run(TestSetup.java:25)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
> Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004542#comment-13004542 ]
Knut Anders Hatlen commented on DERBY-5103:
-------------------------------------------
Thanks, Lily and Rick, for running the tests.
I think there are multiple issues with this test that causes problems when the test cases are reordered:
- some tests leave rows in tables that are shared between the test cases
- some tests don't drop the triggers they create
- static counters are not reset between the test cases
> ProcedureInTriggerTest depends on implicit ordering of test cases
> -----------------------------------------------------------------
>
> Key: DERBY-5103
> URL: https://issues.apache.org/jira/browse/DERBY-5103
> Project: Derby
> Issue Type: Bug
> Components: Test
> Affects Versions: 10.8.0.0
> Reporter: Knut Anders Hatlen
> Attachments: rjall.out
>
>
> If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
> 1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
> at junit.extensions.TestSetup.run(TestSetup.java:25)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
> Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-5103) ProcedureInTriggerTest depends on
implicit ordering of test cases
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-5103:
--------------------------------------
Attachment: derby-5103-1a.diff
Attaching a patch that attempts to loosen up the dependency on a particular ordering. The patch does the following:
1) Adds a tearDown() method that truncates the test tables between each run and resets the static counters.
2) Move an insert statement from testTriggerNoSql to testTriggerContainsSql. testTriggerNoSql doesn't use that row (inserting it is the last thing it does), so it's better that it's inserted in the test case that actually needs it.
3) Added an insert statement in testTriggerNegative to compensate for the rows that are removed by tearDown(). It used to depend on these rows getting inserted by testModifiesSql, but now it should work even if testModifiesSql hasn't been run yet.
4) Drop the triggers created in testTriggerNegative.
With these changes, the test runs cleanly in an environment where the test failed almost consistently before.
> ProcedureInTriggerTest depends on implicit ordering of test cases
> -----------------------------------------------------------------
>
> Key: DERBY-5103
> URL: https://issues.apache.org/jira/browse/DERBY-5103
> Project: Derby
> Issue Type: Bug
> Components: Test
> Affects Versions: 10.8.0.0
> Reporter: Knut Anders Hatlen
> Attachments: derby-5103-1a.diff, rjall.out
>
>
> If for example the test case testTriggerContainsSql is executed as the first test case, it will fail with the following error:
> 1) testTriggerContainsSql(org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest)junit.framework.AssertionFailedError: expected:<2> but was:<0>
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.checkAndResetGetConnectionProcCount(ProcedureInTriggerTest.java:514)
> at org.apache.derbyTesting.functionTests.tests.lang.ProcedureInTriggerTest.testTriggerContainsSql(ProcedureInTriggerTest.java:120)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
> at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
> at junit.extensions.TestSetup.run(TestSetup.java:25)
> at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> The test should either make the order of the test cases explicit or make the test cases independent of the order in which they run.
> Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1076682-suitesAll_diff.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira