Contribution of System tests for Derby

[Manjula G Kutty <ma...@gmail.com>]

Hello,

As a continued effort towards improving Derby quality, I wish to 
contribute some long running system tests for Derby.  Here is
a brief description of each:

- Single Table -- This test run with a maximum of 250 connections and 
the database has only one table with all supported data types and 
corresponding indices.
Duration of the test is 2 weeks and is usable for checking issues 
regarding memory, locking and concurrency

- Scenario Test -  This test is a mail client application where derby is 
used  as the client database. It is a multi threaded test with 4 threads 
that  mimics doing tasks like
user activity, purging, backup and refresh(syncing). Duration of each 
transaction is calculated and can be run in both embedded and n/w server 
mode. The run time is configurable.

- Network Server Test - similar to Single table using Network 
Server/Client and all possible data types in a single table and IUD 
transactions
 along with  triggers,  online back-up/restore, re-encryption. The run 
time is configurable

- Optimizer tests - An assortment of test scenarios using multi-level 
views, aggregated views, complex joins, etc that verifies the
 recent DERBY-805/DERBY-1205 optimizer fixes for push-down predicates. 
Typical runtime is  approximately 2 days (depends upon the derby 
performance)

 I don't think these tests can be run as functional tests on a nightly 
basis. Some of these could be  run for weeks and would be
useful to test release candidates. I suggest  
org.apache.derbyTesting.system.tests.<name of the test>  as a  location 
for these tests.

I would also suggest a new ant target to build these classes. So that 
the simple ant command do not have to
compile these test classes everytime. If any one wants them, they could 
run it  as ant <target name>.

Also, I was considering opening a JIRA issue for contributing each of of 
them, but am open to suggestions.

Do let me know if there are any other comments.

Thanks,
Manjula.

Contributing code to Apache (was Re: Contribution of System tests for Derby)

["Jean T. Anderson" <jt...@bristowhill.com>]

I'm deliberately top posting to Manjula's original post (included in its
entirety down below).

Manjula and I briefly chatted about this contribution off line. The
summary of what we discussed for the list is: these tests were
originally developed for Cloudscape before IBM contributed code to
Apache for the Derby project.

The ASF has a code contribution process we should follow that is
outlined at :

   http://incubator.apache.org/ip-clearance/index.html

I've been watching how other ASF projects bring in code and I believe
the steps we need to follow are:

1) Upload the code to a new Jira issue.
2) Fax the software grant form [1] to the ASF secretary.
3) Call a vote on derby-dev to accept the contribution.
4) Fill in the IP clearance template [2] and add it to the Incubator
repo. (I have karma to the Incubator repo and will offer to do this.)
5) Announce the contribution to the Incubator PMC. If no issues are
raised in 72 hours, then the code can be checked into the derby repo. A
recent example of Jakarta's announce to the Incubator is at [3].

Manjula mentioned this in her post:

> I will open individual jira  entries  for each test to contribute them
> one by one.

If it makes sense to split up the contribution, then do so. But if any
pieces are reasonable to combine, then combining them would cut down on
the process required to bring them in.

 -jean


[1] Software grant form: http://www.apache.org/licenses/software-grant.txt
[2] IP clearance template:
http://incubator.apache.org/ip-clearance/ip-clearance-template.html
[3]
http://mail-archives.apache.org/mod_mbox/incubator-general/200611.mbox/%3c8a81b4af0611121246of332969ubca1380d8359253d@mail.gmail.com%3e


-------- Original Message --------
Subject: Re: Contribution of System tests for Derby
Date: Fri, 10 Nov 2006 11:36:42 -0800
From: Manjula G Kutty <ma...@gmail.com>
Reply-To: derby-dev@db.apache.org
To: derby-dev@db.apache.org
References: <45...@gmail.com> <45...@apache.org>

Daniel John Debrunner wrote:

> Manjula G Kutty wrote:
>
>> Hello,
>>
>> As a continued effort towards improving Derby quality, I wish to
>> contribute some long running system tests for Derby.  Here is
>> a brief description of each:
>
>
> [snip - four test descriptions]
>
> Sounds good, look forward to have this additional testing ability.
>
>>
>> I don't think these tests can be run as functional tests on a nightly
>> basis. Some of these could be  run for weeks and would be
>> useful to test release candidates.
>
>
> It might be useful to run some form as functional tests, this at least
> ensures the test is not broken by some change. That of course could be
> done subsequent to the contribution. Possibly it's useful to think of
> these as test toolkits and not just their potential use to a long
> running test. For example with the OE I imagine there is a functional
> test that could be written to ensure the schema scripts work, data can
> be loaded etc.
>
>
>> I suggest  org.apache.derbyTesting.system.tests.<name of the test>
>> as a  location for these tests.
>
>
> I put the OE test, which is similar in concept to what you describe,
> directly under org.apache.derbyTesting.system. I don't think there's
> any benefit to that extra 'tests'. So I would propose:
>
> org.apache.derbyTesting.system.<name of the test>
>
> to match oe.
>
>> I would also suggest a new ant target to build these classes. So that
>> the simple ant command do not have to
>> compile these test classes everytime. If any one wants them, they
>> could run it  as ant <target name>.
>
>
> Probably should be included in the all target, probably depends on how
> much extra time is involved.
>
> Thanks,
> Dan.
>
>
Thanks Dan for your valuable suggestions.

Thanks Dan for your valuable suggestions.

I will open individual jira  entries  for each test to contribute them
one by one. I agree the package name can be
org.apache.derbyTesting.system.<testname>, similar
to the order entry tests.

The time taken to compile them is very minimal hence we can include
these tests in the  ant all target.  A readme file that tells how
to run and check the results will be included in each of the tests. Also
note that  these tests can be run for few hours since
there is no explicit exit conditions, so anyone can choose to run them
for the desired duration by using some sort of utility wrappers
around them

Please let me know if you have any further questions.

Thanks
Manjula

Re: Contribution of System tests for Derby

[Manjula G Kutty <ma...@gmail.com>]

Daniel John Debrunner wrote:

> Manjula G Kutty wrote:
>
>> Hello,
>>
>> As a continued effort towards improving Derby quality, I wish to 
>> contribute some long running system tests for Derby.  Here is
>> a brief description of each:
>
>
> [snip - four test descriptions]
>
> Sounds good, look forward to have this additional testing ability.
>
>>
>> I don't think these tests can be run as functional tests on a nightly 
>> basis. Some of these could be  run for weeks and would be
>> useful to test release candidates.
>
>
> It might be useful to run some form as functional tests, this at least 
> ensures the test is not broken by some change. That of course could be 
> done subsequent to the contribution. Possibly it's useful to think of 
> these as test toolkits and not just their potential use to a long 
> running test. For example with the OE I imagine there is a functional 
> test that could be written to ensure the schema scripts work, data can 
> be loaded etc.
>
>
>> I suggest  org.apache.derbyTesting.system.tests.<name of the test>  
>> as a  location for these tests.
>
>
> I put the OE test, which is similar in concept to what you describe, 
> directly under org.apache.derbyTesting.system. I don't think there's 
> any benefit to that extra 'tests'. So I would propose:
>
> org.apache.derbyTesting.system.<name of the test>
>
> to match oe.
>
>> I would also suggest a new ant target to build these classes. So that 
>> the simple ant command do not have to
>> compile these test classes everytime. If any one wants them, they 
>> could run it  as ant <target name>.
>
>
> Probably should be included in the all target, probably depends on how 
> much extra time is involved.
>
> Thanks,
> Dan.
>
>
Thanks Dan for your valuable suggestions.

 Thanks Dan for your valuable suggestions.

 I will open individual jira  entries  for each test to contribute them 
one by one. I agree the package name can be 
org.apache.derbyTesting.system.<testname>, similar
to the order entry tests.

The time taken to compile them is very minimal hence we can include 
these tests in the  ant all target.  A readme file that tells how
to run and check the results will be included in each of the tests. Also 
note that  these tests can be run for few hours since
there is no explicit exit conditions, so anyone can choose to run them 
for the desired duration by using some sort of utility wrappers
around them

Please let me know if you have any further questions.

Thanks
Manjula

Re: Contribution of System tests for Derby

[Daniel John Debrunner <dj...@apache.org>]

Manjula G Kutty wrote:
> Hello,
> 
> As a continued effort towards improving Derby quality, I wish to 
> contribute some long running system tests for Derby.  Here is
> a brief description of each:

[snip - four test descriptions]

Sounds good, look forward to have this additional testing ability.

> 
> I don't think these tests can be run as functional tests on a nightly 
> basis. Some of these could be  run for weeks and would be
> useful to test release candidates.

It might be useful to run some form as functional tests, this at least 
ensures the test is not broken by some change. That of course could be 
done subsequent to the contribution. Possibly it's useful to think of 
these as test toolkits and not just their potential use to a long 
running test. For example with the OE I imagine there is a functional 
test that could be written to ensure the schema scripts work, data can 
be loaded etc.


> I suggest  
> org.apache.derbyTesting.system.tests.<name of the test>  as a  location 
> for these tests.

I put the OE test, which is similar in concept to what you describe, 
directly under org.apache.derbyTesting.system. I don't think there's any 
benefit to that extra 'tests'. So I would propose:

org.apache.derbyTesting.system.<name of the test>

to match oe.

> I would also suggest a new ant target to build these classes. So that 
> the simple ant command do not have to
> compile these test classes everytime. If any one wants them, they could 
> run it  as ant <target name>.

Probably should be included in the all target, probably depends on how 
much extra time is involved.

Thanks,
Dan.