You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Sean Qiu <se...@gmail.com> on 2006/10/18 07:52:17 UTC
[testing]-run a test in different mode leads to different result
After running the suite of derbyall, i want to recur each single test from
the derbyall_fail.txt
But when i run it in different test mode, like
spesifying the -Dframework=DerbynetClient, it will produce different
results.
It may fail in one mode while pass in another mode.
So my question is how could i konw which mode is the request one?
Run this test in an embedded environment, it will pass:
<<<<<<<<<<<<<<<<<
root$ java -Dverbose=true
org.apache.derbyTesting.functionTests.harness.RunTestlang/triggerGeneral.sql
derby.infolog.append=true
derby.locks.waitTimeout=4
console.encoding:UTF-8 file.encoding:UTF-8 derby.ui.codeset: null
*** Start: triggerGeneral jdk1.4.2 subset 2006-10-18 13:38:18 ***
Appending to derby_tests.policy
About to execute: diff.exec( outName =
/home/qbit/harmony/workspace/derby/result/test/triggerGeneral.out,outDir =
/home/qbit/harmony/workspace/derby/result/test,pwDiff =
java.io.PrintWriter@7e607e60,testOutName = triggerGeneral,frameworkMaster =
,jvmName = jdk15,iminor = 4,useprocess = false,systemdiff = false,canondir =
null,canonpath = null)
MasterFileName = master/triggerGeneral.out
*** End: triggerGeneral jdk1.4.2 subset 2006-10-18 13:38:55 ***
>>>>>>>>>>>>>>>>>
Run this test in a server environment, it will fail:
<<<<<<<<<<<<<<<<<
java -Dverbose=true -Dframework=DerbyNetClient
org.apache.derbyTesting.functionTests.harness.RunTestlang/triggerGeneral.sql
derby.infolog.append=true
derby.locks.waitTimeout=4
console.encoding:UTF-8 file.encoding:UTF-8 derby.ui.codeset: null
*** Start: triggerGeneral jdk1.4.2 subset DerbyNetClient 2006-10-18 13:40:29
***
Appending to derby_tests.policy
Initialize for framework: DerbyNetClient
java -Duser.language=en -Duser.country=US -
Dderby.system.home=/home/qbit/harmony/workspace/derby/result/test/DerbyNetClient/triggerGeneral-
Djava.security.manager -
Djava.security.policy=/home/qbit/harmony/workspace/derby/result/test/derby_tests.policy-
DderbyTesting.codeclasses=file:/home/qbit/harmony/workspace/derby/classes/ -
DderbyTesting.codedir=/home/qbit/harmony/workspace/derby/classes -
DderbyTesting.serverhost=localhost -DderbyTesting.clienthost=localhost
org.apache.derby.drda.NetworkServerControl start
Appending to derby_tests.policy
Attempt to shutdown framework: DerbyNetClient
About to execute: diff.exec( outName =
/home/qbit/harmony/workspace/derby/result/test/DerbyNetClient/triggerGeneral.out,outDir
= /home/qbit/harmony/workspace/derby/result/test/DerbyNetClient,pwDiff =
java.io.PrintWriter@28c828c8,testOutName = triggerGeneral,frameworkMaster =
DerbyNetClient,jvmName = jdk15,iminor = 4,useprocess = false,systemdiff =
false,canondir = null,canonpath = null)
MasterFileName = master/triggerGeneral.out
0a1
> ERROR 08001: java.net.ConnectException : Error connecting to server
xxxFILTERED_HOSTNAMExxx on port 1527 with message
xxxFILTERED_HOSTNAMExxx/127.0.0.1:1527 - Connection refused.
6 del
< 0 rows inserted/updated/deleted
6a7
> IJ ERROR: Unable to establish connection
9 del
< 0 rows inserted/updated/deleted
9a10
> IJ ERROR: Unable to establish connection
11 del
< ERROR 42Y55: 'DROP TABLE' cannot be performed on 'X' because it does not
exist.
11a12
..
..
..
< ij(USER1)> drop schema ippo restrict;
1305 del
< 0 rows inserted/updated/deleted
1306 del
< ij(USER1)>
1306 add
> ij>
Test Failed.
*** End: triggerGeneral jdk1.4.2 subset DerbyNetClient 2006-10-18 13:40:40
***
>>>>>>>>>>>>>>>>>>>>>>>>
BTW, i sitll confuse with the diff result. Could anyone do me a favor?
--
Sean Qiu
Re: [testing]-run a test in different mode leads to different result
Posted by Myrna van Lunteren <m....@gmail.com>.
On 10/18/06, John Embretsen <Jo...@sun.com> wrote:
> Sean Qiu wrote:
> > After running the suite of derbyall, i want to recur each single test
> > from the derbyall_fail.txt
> > But when i run it in different test mode, like
> > spesifying the -Dframework=DerbynetClient, it will produce different
> > results.
> > It may fail in one mode while pass in another mode.
> >
> > So my question is how could i konw which mode is the request one?
>
> It is a bit tricky to figure out in which frameworks a particular test will or
> should run. There is a wiki page with some further details, please take a look:
>
> http://wiki.apache.org/db-derby/TestingFrameworks
>
Just to add, a task was added in Jira suggesting that derbyall suite
should be run with DerbyNetClient and the results analyzed to see
whether any of the probable test failures are due to test design
issues or actual bugs:
https://issues.apache.org/jira/browse/DERBY-1342.
Myrna
Re: [testing]-run a test in different mode leads to different result
Posted by John Embretsen <Jo...@Sun.COM>.
Sean Qiu wrote:
> I wonder whether there exits any tests that need testing in both mode?
Yes, there are. For example, there are tests listed in
suites/derbynetmats.runall that are also listed in suites/derbylang.runall.
Hence, these tests will run in all three frameworks during a derbyall run
(unless they are listed in some .exclude file).
Currently I don't think we expect tests to run in other frameworks/modes than
what is used during a derbyall run (again, JUnit tests are different).
Generally speaking, we would like to be able to run tests in as many frameworks
as possible, but that's not the case with the current non-JUnit tests. However,
the recent efforts made on JUnit-based testing seem to go in that direction.
There is another wiki page describing which tests would run (standalone) in
which frameworks at some point in time (2006-03-24), on a specific platform.
This may give you a hint on which tests should (not) fail in a specific framework:
http://wiki.apache.org/db-derby/StandaloneStatus
> 2006/10/18, John Embretsen <John.Embretsen@sun.com
> <ma...@sun.com> >:
>
> Sean Qiu wrote:
> > After running the suite of derbyall, i want to recur each single test
> > from the derbyall_fail.txt
> > But when i run it in different test mode, like
> > spesifying the -Dframework=DerbynetClient, it will produce different
> > results.
> > It may fail in one mode while pass in another mode.
> >
> > So my question is how could i konw which mode is the request one?
>
> It is a bit tricky to figure out in which frameworks a particular
> test will or
> should run. There is a wiki page with some further details, please
> take a look:
>
> http://wiki.apache.org/db-derby/TestingFrameworks
[...]
--
John
Re: [testing]-run a test in different mode leads to different result
Posted by Sean Qiu <se...@gmail.com>.
I wonder whether there exits any tests that need testing in both mode?
Thanks.
2006/10/18, John Embretsen <Jo...@sun.com>:
>
> Sean Qiu wrote:
> > After running the suite of derbyall, i want to recur each single test
> > from the derbyall_fail.txt
> > But when i run it in different test mode, like
> > spesifying the -Dframework=DerbynetClient, it will produce different
> > results.
> > It may fail in one mode while pass in another mode.
> >
> > So my question is how could i konw which mode is the request one?
>
> It is a bit tricky to figure out in which frameworks a particular test
> will or
> should run. There is a wiki page with some further details, please take a
> look:
>
> http://wiki.apache.org/db-derby/TestingFrameworks
>
>
> For example, if your derbyall_fail.txt contains something like
>
> derbyall/derbylang/derbylang.fail:lang/triggerGeneral.sql
>
> it means that the test was run as part of the derbylang suite.
>
> The derbylang suite is only run in the Embedded framework by default.
> We "know" this because
>
> 1) there is no derbylang.properties file in the functionTests/suites/
> directory
> specifying a "framework" property
> 2) the derbylang suite is not part of a larger (super)suite that is run in
> a
> non-default framework (no <suite>.properties file includes derbylang in
> its list
> of suites while it also includes a non-default framework setting)
> 3) the default test framework is the Embedded framework
>
> Also, there is no "framework" property specified in the
> tests/lang/triggerGeneral_*.properties files. Thus we may conclude that
> the test
> is normally only run in the Embedded (default) framework.
>
> The JUnit tests work a bit differently, though...
>
> > Run this test in an embedded environment, it will pass:
>
> [snip output from test command]
>
> > Run this test in a server environment, it will fail:
> > <<<<<<<<<<<<<<<<<
> > java -Dverbose=true -Dframework=DerbyNetClient
> > org.apache.derbyTesting.functionTests.harness.RunTest
> > lang/triggerGeneral.sql
>
> [snip output from test command and diff]
>
> > BTW, i sitll confuse with the diff result. Could anyone do me a favor?
>
> I don't think this test is designed to run in client/server frameworks,
> which
> probably is the reason why it fails.
>
>
> --
> John
>
>
>
--
Sean Qiu
Re: [testing]-run a test in different mode leads to different result
Posted by John Embretsen <Jo...@Sun.COM>.
Sean Qiu wrote:
> After running the suite of derbyall, i want to recur each single test
> from the derbyall_fail.txt
> But when i run it in different test mode, like
> spesifying the -Dframework=DerbynetClient, it will produce different
> results.
> It may fail in one mode while pass in another mode.
>
> So my question is how could i konw which mode is the request one?
It is a bit tricky to figure out in which frameworks a particular test will or
should run. There is a wiki page with some further details, please take a look:
http://wiki.apache.org/db-derby/TestingFrameworks
For example, if your derbyall_fail.txt contains something like
derbyall/derbylang/derbylang.fail:lang/triggerGeneral.sql
it means that the test was run as part of the derbylang suite.
The derbylang suite is only run in the Embedded framework by default.
We "know" this because
1) there is no derbylang.properties file in the functionTests/suites/ directory
specifying a "framework" property
2) the derbylang suite is not part of a larger (super)suite that is run in a
non-default framework (no <suite>.properties file includes derbylang in its list
of suites while it also includes a non-default framework setting)
3) the default test framework is the Embedded framework
Also, there is no "framework" property specified in the
tests/lang/triggerGeneral_*.properties files. Thus we may conclude that the test
is normally only run in the Embedded (default) framework.
The JUnit tests work a bit differently, though...
> Run this test in an embedded environment, it will pass:
[snip output from test command]
> Run this test in a server environment, it will fail:
> <<<<<<<<<<<<<<<<<
> java -Dverbose=true -Dframework=DerbyNetClient
> org.apache.derbyTesting.functionTests.harness.RunTest
> lang/triggerGeneral.sql
[snip output from test command and diff]
> BTW, i sitll confuse with the diff result. Could anyone do me a favor?
I don't think this test is designed to run in client/server frameworks, which
probably is the reason why it fails.
--
John