You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Daniel John Debrunner <dj...@apache.org> on 2006/10/05 20:50:16 UTC

[junit] Making progess faster towards a pure Junit world

I'd like to make progress faster towards a pure JUnit set of tests for 
Derby but I believe the fact we need to setup the tests and the JUnit 
utility classes to run under the harness slows down progress as well as 
adding complications. An example of how it slows down is that any change 
to a test or JUnit utility class requires that the contributor runs the 
tests multiple ways to ensure no regressions.

Assuming we had a top-level JUnit suite that ran all the Junit tests 
successfully (with the same configuration coverage as they are run today 
in derbyall) then could we switch to a dual testing environment until 
all the tests were converted to JUnit? This would include removing all 
the JUnit tests from the old harness suite files.

That is if one wanted to run all the tests one would have to run:

   derbyall with the old harness
   suites.All using JUnit test runners directly.

Is this an issue for people?

What about the folks that do nightly testing? Has any JUnit related work 
been done on the scripts used to report nightly tests. Are those scripts 
going to be added to Derby's svn so that others can help make progress 
on this issue?

Thanks,
Dan.

Re: Workaround for running JUnit tests with the GUI runners

Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Kristian Waagan <Kr...@Sun.COM> writes:

> Hi Dyre,
>
> I think you can create a Jira for this, if you haven't already.
>
> The problem is that the GUI runners are using a custom classload to
> allow for dynamic reloading of the test classes.

I'm seeing this message in derby.log when I'm running with (any)
TestRunner:

DEBUG RAFContainerFactory OUTPUT: Caught exception when setting up
rafContainerConstructor

The tests don't fail because RAFContainerFactory falls back to the old
RAFContainer implementation. The exception seems to be a
ClassNotFoundException (cannot find RAFContainer4). Could it also be
caused by the custom classloader?

-- 
Knut Anders

Re: Workaround for running JUnit tests with the GUI runners (was: Re: [junit] Making progess faster towards a pure Junit world)

Posted by Dy...@Sun.COM.
Kristian Waagan <Kr...@Sun.COM> writes:

[snip]

>> Should I create a JIRA for this?
>
> Hi Dyre,
>
> I think you can create a Jira for this, if you haven't already.

Thank you for an excellent analysis, Kristian! 

Logged as 
https://issues.apache.org/jira/browse/DERBY-1977

> The problem is that the GUI runners are using a custom classload to
> allow for dynamic reloading of the test classes. It defeats the getURL
> method in SecurityManagerSetup, seemingly by returning an empty/null
> (the object itself isn't null) ProtectionDomain object. This finally
> leads to the URL object extracted being null, which causes the NPE.
>
> A workaround is to specify the -noloading option for the runner, like this:
> java -classpath X junit.swingui.TestRunner -noloading
> org.apache.derbyTesting...
>
> If you do this, you must restart the GUI if you recompile the test classes.
> We should investigate this and see if we can get it working without
> specifying the -noloading argument.

-- 
dt

Workaround for running JUnit tests with the GUI runners (was: Re: [junit] Making progess faster towards a pure Junit world)

Posted by Kristian Waagan <Kr...@Sun.COM>.
Dyre.Tjeldvoll@Sun.COM wrote:
> Kristian Waagan <Kr...@Sun.COM> writes:
> 
>> Daniel John Debrunner wrote:

[ snip - progress towards a pure JUnit world ]

> 
> Should it be possible to run junit tests with ALL the junit runners,
> or just with junit.textui.TestRunner? 
> 
> With the current trunk I able to
> run
> 
> org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite
> 
> successfully with junit.textui.TestRunner (if I set
> derby.system.home), but with 
> junit.awtui.TestRunner or junit.swingui.TestRunner the first test
> fails with 
> 
> java.lang.ExceptionInInitializerError
> 	at org.apache.derbyTesting.junit.TestConfiguration.defaultSecurityManagerSetup(TestConfiguration.java:667)
> 	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:73)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> Caused by: java.lang.NullPointerException
> 	at org.apache.derbyTesting.junit.SecurityManagerSetup.determineClasspath(SecurityManagerSetup.java:218)
> 	at org.apache.derbyTesting.junit.SecurityManagerSetup.<clinit>(SecurityManagerSetup.java:56)
> 	... 19 more
> 
> and the following tests fail with
> 
> java.lang.NoClassDefFoundError: Could not initialize class org.apache.derbyTesting.junit.SecurityManagerSetup
> 	at org.apache.derbyTesting.junit.TestConfiguration.defaultSecurityManagerSetup(TestConfiguration.java:667)
> 	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:73)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 
> Should I create a JIRA for this?

Hi Dyre,

I think you can create a Jira for this, if you haven't already.

The problem is that the GUI runners are using a custom classload to 
allow for dynamic reloading of the test classes. It defeats the getURL 
method in SecurityManagerSetup, seemingly by returning an empty/null 
(the object itself isn't null) ProtectionDomain object. This finally 
leads to the URL object extracted being null, which causes the NPE.

A workaround is to specify the -noloading option for the runner, like this:
java -classpath X junit.swingui.TestRunner -noloading 
org.apache.derbyTesting...

If you do this, you must restart the GUI if you recompile the test classes.
We should investigate this and see if we can get it working without 
specifying the -noloading argument.



-- 
Kristian

>

Re: [junit] Making progess faster towards a pure Junit world

Posted by Dy...@Sun.COM.
Kristian Waagan <Kr...@Sun.COM> writes:

> Daniel John Debrunner wrote:
>> I'd like to make progress faster towards a pure JUnit set of tests
>> for Derby but I believe the fact we need to setup the tests and the
>> JUnit utility classes to run under the harness slows down progress
>> as well as adding complications. An example of how it slows down is
>> that any change to a test or JUnit utility class requires that the
>> contributor runs the tests multiple ways to ensure no regressions.
>> Assuming we had a top-level JUnit suite that ran all the Junit tests
>> successfully (with the same configuration coverage as they are run
>> today in derbyall) then could we switch to a dual testing
>> environment until all the tests were converted to JUnit? This would
>> include removing all the JUnit tests from the old harness suite
>> files.
>> That is if one wanted to run all the tests one would have to run:
>>   derbyall with the old harness
>>   suites.All using JUnit test runners directly.
>> Is this an issue for people?
>
> Hi,
>
> For me this is not an issue. I might even be able to run the two in
> parallel by changing the port number for the JUnit run.
>
> I'm happy to see that there is activity on converting tests to JUnit.
> We still have a long way to go though, so keep up the good work!
>
>
>
> (BTW: I have some problems with the security manager, and if people
> have any ideas, I'd like to hear them;
> http://issues.apache.org/jira/browse/DERBY-1001)

Should it be possible to run junit tests with ALL the junit runners,
or just with junit.textui.TestRunner? 

With the current trunk I able to
run

org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite

successfully with junit.textui.TestRunner (if I set
derby.system.home), but with 
junit.awtui.TestRunner or junit.swingui.TestRunner the first test
fails with 

java.lang.ExceptionInInitializerError
	at org.apache.derbyTesting.junit.TestConfiguration.defaultSecurityManagerSetup(TestConfiguration.java:667)
	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:73)
	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
	at junit.extensions.TestSetup.run(TestSetup.java:23)
	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
	at junit.extensions.TestSetup.run(TestSetup.java:23)
Caused by: java.lang.NullPointerException
	at org.apache.derbyTesting.junit.SecurityManagerSetup.determineClasspath(SecurityManagerSetup.java:218)
	at org.apache.derbyTesting.junit.SecurityManagerSetup.<clinit>(SecurityManagerSetup.java:56)
	... 19 more

and the following tests fail with

java.lang.NoClassDefFoundError: Could not initialize class org.apache.derbyTesting.junit.SecurityManagerSetup
	at org.apache.derbyTesting.junit.TestConfiguration.defaultSecurityManagerSetup(TestConfiguration.java:667)
	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:73)
	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
	at junit.extensions.TestSetup.run(TestSetup.java:23)
	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
	at junit.extensions.TestSetup.run(TestSetup.java:23)

Should I create a JIRA for this?

-- 
dt

Re: [junit] Making progess faster towards a pure Junit world

Posted by Kristian Waagan <Kr...@Sun.COM>.
Daniel John Debrunner wrote:
> I'd like to make progress faster towards a pure JUnit set of tests for 
> Derby but I believe the fact we need to setup the tests and the JUnit 
> utility classes to run under the harness slows down progress as well as 
> adding complications. An example of how it slows down is that any change 
> to a test or JUnit utility class requires that the contributor runs the 
> tests multiple ways to ensure no regressions.
> 
> Assuming we had a top-level JUnit suite that ran all the Junit tests 
> successfully (with the same configuration coverage as they are run today 
> in derbyall) then could we switch to a dual testing environment until 
> all the tests were converted to JUnit? This would include removing all 
> the JUnit tests from the old harness suite files.
> 
> That is if one wanted to run all the tests one would have to run:
> 
>   derbyall with the old harness
>   suites.All using JUnit test runners directly.
> 
> Is this an issue for people?

Hi,

For me this is not an issue. I might even be able to run the two in 
parallel by changing the port number for the JUnit run.

I'm happy to see that there is activity on converting tests to JUnit.
We still have a long way to go though, so keep up the good work!



(BTW: I have some problems with the security manager, and if people have 
any ideas, I'd like to hear them; 
http://issues.apache.org/jira/browse/DERBY-1001)


-- 
Kristian


> 
> What about the folks that do nightly testing? Has any JUnit related work 
> been done on the scripts used to report nightly tests. Are those scripts 
> going to be added to Derby's svn so that others can help make progress 
> on this issue?
> 
> Thanks,
> Dan.
>