[junit] Move JUnit base/utility classes???

[Daniel John Debrunner <dj...@apache.org>]

Currently the JUnit base and utility classes are in this package:

org.apache.derbyTesting.functionTests.util

(See http://wiki.apache.org/db-derby/DerbyJUnitTesting)

I was wondering if they should be moved, for two reasons:

  1) That package is cluttered up with other stuff, it's more or less a
dumping ground. Utilites, JUnit base classes, "user level" classes for
procedures and vtis, etc.

  2) JUnit tests can be much more than funcional tests, e.g. having
system tests as JUnit tests would make them easy for everyone to run.

I was thinking of the following package:

org.apache.derbyTesting.junit

The package would be limited to base classes for JUnit tests and JUnit
related utilities such as the JDBC class. Classes for specific tests, or
those that implement Java procedures for tests etc. would not be allowed.

The functional tests would continue to live in their current location,
just that the super-class BaseJDBCTestCase would be in the new package.

Thoughts?
Dan.

Re: [junit] Move JUnit base/utility classes???

[Kristian Waagan <Kr...@Sun.COM>]

Daniel John Debrunner wrote:
> Kristian Waagan wrote:
> 
>> Andreas Korneliussen wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Kristian Waagan wrote:
>>> <>
>>>
>>>> Hi,
>>>>
>>>> I still have a question regarding the placement of JUnit tests. I have
>>>> brought this up before, but got very little response (I did get some,
>>>> see below).
>>>>
>>>> Do we want to support [unit] testing of package private classes?
>>>
>>> Maybe it is sufficient to support testing the public methods of public
>>> classes ?
>>
>> Hi Andreas,
>>
>> In my opinion, it is not.
>> There are some quite complex classes which are not part of the public
>> API. Testing these indirectly through the public API is a lot of work,
>> and it might be hard to construct all variations/codepaths.
> 
> I think Andreas meant testing a class through its public methods and
> interfaces, not Derby's public API. E.g. a module implementation can be
> tested through its api from org.apache.derby.iapi. That should be pretty
> direct testing that is able to cover all of the code in the module.
> 
> 
>> I recently experienced this problem. I "solved" it by writing my own
>> unit test and placing it in the appropriate package, which was not
>> contributed to the community. But maybe this is more of an exception
>> than the general case. The existing code base does not have this kind of
>> tests.
> 
> I must have missed this test being discussed on the list, can you
> provide a link to the archives so I can see the kind of issues you were
> hitting.
> 
>> If the consensus is not to support unit tests of this kind, I will
>> either write indirect tests instead or test less of the code. My worry
>> is the latter approach...
> 
> One argument could be that if you can't test a code through its public
> api then there is something wrong with the code, but maybe that's
> optimistic. That's why I'd like to see the example you hit.

Sorry for this short answer Dan.

I hope to write a better one tomorrow. Just to show that what I propose 
is not a new idea, I recommend people to read this entry in the JUnit FAQ:
http://junit.sourceforge.net/doc/faq/faq.htm#organize_1

I guess maybe I have formulated myself a little poorly.



Regards,
-- 
Kristian


> 
> Thanks,
> Dan.
> 

Re: [junit] Move JUnit base/utility classes???

[Daniel John Debrunner <dj...@apache.org>]

Kristian Waagan wrote:

> Andreas Korneliussen wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Kristian Waagan wrote:
>> <>
>>
>>> Hi,
>>>
>>> I still have a question regarding the placement of JUnit tests. I have
>>> brought this up before, but got very little response (I did get some,
>>> see below).
>>>
>>> Do we want to support [unit] testing of package private classes?
>>
>>
>> Maybe it is sufficient to support testing the public methods of public
>> classes ?
> 
> 
> Hi Andreas,
> 
> In my opinion, it is not.
> There are some quite complex classes which are not part of the public
> API. Testing these indirectly through the public API is a lot of work,
> and it might be hard to construct all variations/codepaths.

I think Andreas meant testing a class through its public methods and
interfaces, not Derby's public API. E.g. a module implementation can be
tested through its api from org.apache.derby.iapi. That should be pretty
direct testing that is able to cover all of the code in the module.


> I recently experienced this problem. I "solved" it by writing my own
> unit test and placing it in the appropriate package, which was not
> contributed to the community. But maybe this is more of an exception
> than the general case. The existing code base does not have this kind of
> tests.

I must have missed this test being discussed on the list, can you
provide a link to the archives so I can see the kind of issues you were
hitting.

> If the consensus is not to support unit tests of this kind, I will
> either write indirect tests instead or test less of the code. My worry
> is the latter approach...

One argument could be that if you can't test a code through its public
api then there is something wrong with the code, but maybe that's
optimistic. That's why I'd like to see the example you hit.

Thanks,
Dan.

Re: [junit] Move JUnit base/utility classes???

[Kristian Waagan <Kr...@Sun.COM>]

Andreas Korneliussen wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Kristian Waagan wrote:
> <>
> 
>> Hi,
>>
>> I still have a question regarding the placement of JUnit tests. I have
>> brought this up before, but got very little response (I did get some,
>> see below).
>>
>> Do we want to support [unit] testing of package private classes?
> 
> Maybe it is sufficient to support testing the public methods of public
> classes ?

Hi Andreas,

In my opinion, it is not.
There are some quite complex classes which are not part of the public 
API. Testing these indirectly through the public API is a lot of work, 
and it might be hard to construct all variations/codepaths.

I recently experienced this problem. I "solved" it by writing my own 
unit test and placing it in the appropriate package, which was not 
contributed to the community. But maybe this is more of an exception 
than the general case. The existing code base does not have this kind of 
tests.

If the consensus is not to support unit tests of this kind, I will 
either write indirect tests instead or test less of the code. My worry 
is the latter approach...



-- 
Kristian

> 
>> The easiest solution to achieve this is to keep a mirrored/separate
>> source tree, where the tests are put into the corresponding Derby
>> production code package (for instance 'org.apache.derby.iapi.types').
>>
> 
> I think there are some testclasses which are in the same source tree as
> the derby source, however the .class files are packaged into
> dergbyTesting.jar.  I.e:  org/apache/derby/impl/drda/TestProto.class
> 
> 
>> Andrew brought up the concern wrt building and distribution. We do not
>> want to include the testing classes in the releases. Further, JAR
>> sealing is causing headaches (unable to distribute the test classes in a
>> separate JAR).
>>
> 
> Yes, and that is probably why derbynet's manifest has:
> 
> Name: org/apache/derby/impl/drda/
> Sealed: false
> 
> 
> Andreas
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.4 (SunOS)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFE6vbY36DpyYjYNyIRArhHAJsE/FmJpRui8eGLjCGZR2RCj4DLIACeOmtf
> Vvr1uNT1BlAslq+Oahti3HI=
> =nXDM
> -----END PGP SIGNATURE-----

Re: [junit] Move JUnit base/utility classes???

[Daniel John Debrunner <dj...@apache.org>]

Andreas Korneliussen wrote:

> I think there are some testclasses which are in the same source tree as
> the derby source, however the .class files are packaged into
> dergbyTesting.jar.  I.e:  org/apache/derby/impl/drda/TestProto.class

I think that class should be moved into the derbyTesting area so that
the drda code can be sealed. I don't think it should be uses as a
precedence for adding more tests outside of derbyTesting.

Dan.

Re: [junit] Move JUnit base/utility classes???

[Andreas Korneliussen <An...@Sun.COM>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kristian Waagan wrote:
<>

> Hi,
> 
> I still have a question regarding the placement of JUnit tests. I have
> brought this up before, but got very little response (I did get some,
> see below).
> 
> Do we want to support [unit] testing of package private classes?

Maybe it is sufficient to support testing the public methods of public
classes ?

> The easiest solution to achieve this is to keep a mirrored/separate
> source tree, where the tests are put into the corresponding Derby
> production code package (for instance 'org.apache.derby.iapi.types').
> 

I think there are some testclasses which are in the same source tree as
the derby source, however the .class files are packaged into
dergbyTesting.jar.  I.e:  org/apache/derby/impl/drda/TestProto.class


> Andrew brought up the concern wrt building and distribution. We do not
> want to include the testing classes in the releases. Further, JAR
> sealing is causing headaches (unable to distribute the test classes in a
> separate JAR).
> 

Yes, and that is probably why derbynet's manifest has:

Name: org/apache/derby/impl/drda/
Sealed: false


Andreas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.4 (SunOS)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFE6vbY36DpyYjYNyIRArhHAJsE/FmJpRui8eGLjCGZR2RCj4DLIACeOmtf
Vvr1uNT1BlAslq+Oahti3HI=
=nXDM
-----END PGP SIGNATURE-----

Re: [junit] Move JUnit base/utility classes???

[Kristian Waagan <Kr...@Sun.COM>]

Daniel John Debrunner wrote:
> 
>>> I don't see any value in this, the existing layput looks fine to me.
>>>
>>> org.apache.derbyTesting.junit.functionalTests.tests.jdbcapi
>>> org.apache.derbyTesting.junit.unitTests.lang
>>> org.apache.derbyTesting.junit.systemTests.app1 (future)
>>>
>> (Note: existing layout does not have .junit.)
> (sorry copy & paste error)
> 
>> The value would have been to separate out the junit tests from the other
>> tests, however I understand it is not prioritized and possibly not worth
>> the effort.
> 
> once they are all JUnit though there is no benefit :-)

Hi,

I still have a question regarding the placement of JUnit tests. I have 
brought this up before, but got very little response (I did get some, 
see below).

Do we want to support [unit] testing of package private classes?
The easiest solution to achieve this is to keep a mirrored/separate 
source tree, where the tests are put into the corresponding Derby 
production code package (for instance 'org.apache.derby.iapi.types').

Andrew brought up the concern wrt building and distribution. We do not 
want to include the testing classes in the releases. Further, JAR 
sealing is causing headaches (unable to distribute the test classes in a 
separate JAR).

If people have more opinions at this time, I would like to hear them. If 
not, I'll nag the list again at a later time :)



-- 
Kristian

> 
> Dan.
> 

Re: [junit] Move JUnit base/utility classes???

[Daniel John Debrunner <dj...@apache.org>]

>> I don't see any value in this, the existing layput looks fine to me.
>>
>> org.apache.derbyTesting.junit.functionalTests.tests.jdbcapi
>> org.apache.derbyTesting.junit.unitTests.lang
>> org.apache.derbyTesting.junit.systemTests.app1 (future)
>>
> (Note: existing layout does not have .junit.)
(sorry copy & paste error)

> 
> The value would have been to separate out the junit tests from the other
> tests, however I understand it is not prioritized and possibly not worth
> the effort.

once they are all JUnit though there is no benefit :-)

Dan.

Re: [junit] Move JUnit base/utility classes???

[Andreas Korneliussen <An...@Sun.COM>]

>>>> The functional tests would continue to live in their current location,
>>>> just that the super-class BaseJDBCTestCase would be in the new package.
>>>>
>>>> Thoughts?
>>
>> I support moving the utility classes to a new package.
>>
>> Below are some thoughts on how Junit tests could be structured, in case
>> we would like to also move tests:
>>
>> Package for derby-specific utility classes:
>>
>> org.apache.derbyTesting.junit.util
>>  -> TestConfiguration, JDBCClient, JDBC,..
>>
>>
>> Package for common testcase/testsetup classes:
>>
>> org.apache.derbyTesting.junit.common
>>  -> BaseTestCase, BaseJDBCTestCase, BaseJDBCTestSetup
> 
> I'm not sure the split between util and common here is worth it. The
> classes in common have a very close relationship with the classes in
> util, to my mind it seems they should be together.
> 

Yes, that is correct.

>> Packages for tests:
>>  org.apache.derbyTesting.junit.tests
>>  org.apache.derbyTesting.junit.tests.jdbcapi
>>  org.apache.derbyTesting.junit.tests.lang
>>  org.apache.derbyTesting.junit.tests.store
>>  org.apache.derbyTesting.junit.tests. . .
> 
> 
> I don't see any value in this, the existing layput looks fine to me.
> 
> org.apache.derbyTesting.junit.functionalTests.tests.jdbcapi
> org.apache.derbyTesting.junit.unitTests.lang
> org.apache.derbyTesting.junit.systemTests.app1 (future)
>
(Note: existing layout does not have .junit.)

The value would have been to separate out the junit tests from the other 
tests, however I understand it is not prioritized and possibly not worth 
the effort.

Regards
Andreas

Re: [junit] Move JUnit base/utility classes???

[Daniel John Debrunner <dj...@apache.org>]

Andreas Korneliussen wrote:

> David Van Couvering wrote:
> 
>> Sounds good to me...
>>
>> David
>>
>> Daniel John Debrunner wrote:
>>
>>> Currently the JUnit base and utility classes are in this package:
>>>
>>> org.apache.derbyTesting.functionTests.util
>>>
>>> (See http://wiki.apache.org/db-derby/DerbyJUnitTesting)
>>>
>>> I was wondering if they should be moved, for two reasons:
>>>
>>>   1) That package is cluttered up with other stuff, it's more or less a
>>> dumping ground. Utilites, JUnit base classes, "user level" classes for
>>> procedures and vtis, etc.
>>>
>>>   2) JUnit tests can be much more than funcional tests, e.g. having
>>> system tests as JUnit tests would make them easy for everyone to run.
>>>
>>> I was thinking of the following package:
>>>
>>> org.apache.derbyTesting.junit
>>>
>>> The package would be limited to base classes for JUnit tests and JUnit
>>> related utilities such as the JDBC class. Classes for specific tests, or
>>> those that implement Java procedures for tests etc. would not be
>>> allowed.
>>>
>>> The functional tests would continue to live in their current location,
>>> just that the super-class BaseJDBCTestCase would be in the new package.
>>>
>>> Thoughts?
> 
> 
> I support moving the utility classes to a new package.
> 
> Below are some thoughts on how Junit tests could be structured, in case
> we would like to also move tests:
> 
> Package for derby-specific utility classes:
> 
> org.apache.derbyTesting.junit.util
>  -> TestConfiguration, JDBCClient, JDBC,..
> 
> 
> Package for common testcase/testsetup classes:
> 
> org.apache.derbyTesting.junit.common
>  -> BaseTestCase, BaseJDBCTestCase, BaseJDBCTestSetup

I'm not sure the split between util and common here is worth it. The
classes in common have a very close relationship with the classes in
util, to my mind it seems they should be together.

> Packages for tests:
>  org.apache.derbyTesting.junit.tests
>  org.apache.derbyTesting.junit.tests.jdbcapi
>  org.apache.derbyTesting.junit.tests.lang
>  org.apache.derbyTesting.junit.tests.store
>  org.apache.derbyTesting.junit.tests. . .


I don't see any value in this, the existing layput looks fine to me.

org.apache.derbyTesting.junit.functionalTests.tests.jdbcapi
org.apache.derbyTesting.junit.unitTests.lang
org.apache.derbyTesting.junit.systemTests.app1 (future)

Though maybe one day some cleanup could happen and some naming
guidlines, e.g.

org.apache.derbyTesting.junit.functionalTests.tests.derbyNet.testProperties

having test in the name four times seems excessive.

My point was just to get the Junit generic stuff out of the
functionalTest layer since it can apply to so much more. I'm not willing
to spend time moving the existing tests around.

Dan.

Re: [junit] Move JUnit base/utility classes???

[Andreas Korneliussen <An...@Sun.COM>]

David Van Couvering wrote:
> Sounds good to me...
> 
> David
> 
> Daniel John Debrunner wrote:
>> Currently the JUnit base and utility classes are in this package:
>>
>> org.apache.derbyTesting.functionTests.util
>>
>> (See http://wiki.apache.org/db-derby/DerbyJUnitTesting)
>>
>> I was wondering if they should be moved, for two reasons:
>>
>>   1) That package is cluttered up with other stuff, it's more or less a
>> dumping ground. Utilites, JUnit base classes, "user level" classes for
>> procedures and vtis, etc.
>>
>>   2) JUnit tests can be much more than funcional tests, e.g. having
>> system tests as JUnit tests would make them easy for everyone to run.
>>
>> I was thinking of the following package:
>>
>> org.apache.derbyTesting.junit
>>
>> The package would be limited to base classes for JUnit tests and JUnit
>> related utilities such as the JDBC class. Classes for specific tests, or
>> those that implement Java procedures for tests etc. would not be allowed.
>>
>> The functional tests would continue to live in their current location,
>> just that the super-class BaseJDBCTestCase would be in the new package.
>>
>> Thoughts?

I support moving the utility classes to a new package.

Below are some thoughts on how Junit tests could be structured, in case 
we would like to also move tests:

Package for derby-specific utility classes:

org.apache.derbyTesting.junit.util
  -> TestConfiguration, JDBCClient, JDBC,..


Package for common testcase/testsetup classes:

org.apache.derbyTesting.junit.common
  -> BaseTestCase, BaseJDBCTestCase, BaseJDBCTestSetup


Packages for tests:
  org.apache.derbyTesting.junit.tests
  org.apache.derbyTesting.junit.tests.jdbcapi
  org.apache.derbyTesting.junit.tests.lang
  org.apache.derbyTesting.junit.tests.store
  org.apache.derbyTesting.junit.tests. . .


Andreas

Re: [junit] Move JUnit base/utility classes???

[David Van Couvering <Da...@Sun.COM>]

Sounds good to me...

David

Daniel John Debrunner wrote:
> Currently the JUnit base and utility classes are in this package:
> 
> org.apache.derbyTesting.functionTests.util
> 
> (See http://wiki.apache.org/db-derby/DerbyJUnitTesting)
> 
> I was wondering if they should be moved, for two reasons:
> 
>   1) That package is cluttered up with other stuff, it's more or less a
> dumping ground. Utilites, JUnit base classes, "user level" classes for
> procedures and vtis, etc.
> 
>   2) JUnit tests can be much more than funcional tests, e.g. having
> system tests as JUnit tests would make them easy for everyone to run.
> 
> I was thinking of the following package:
> 
> org.apache.derbyTesting.junit
> 
> The package would be limited to base classes for JUnit tests and JUnit
> related utilities such as the JDBC class. Classes for specific tests, or
> those that implement Java procedures for tests etc. would not be allowed.
> 
> The functional tests would continue to live in their current location,
> just that the super-class BaseJDBCTestCase would be in the new package.
> 
> Thoughts?
> Dan.
>