discussion re DERBY-1746 - upgrade tests

[Myrna van Lunteren <m....@gmail.com>]

Hi,

I've found myself in a quandary with respect to the upgrade tests.
DERBY-1746 describes a problem with sealed jars - which was resolved
by Dan by converting the test to a junit test using a different
approach. I want to close this issue.

However, we now have a svn:externals setting that's unnecessary, and a
number of  releases with all jars in svn intended to be used by the
upgrade test, whereas the upgrade test only needs derby.jar.
svn:externals can only map directories, though.

While I don't think we can automatically force everyone to run the
junit upgrade tests with all older releases, we should make it as easy
as possible.

I like having all older releases' jar files readily available in svn -
however, I don't want to duplicate them, I don't want all files in my
trunk co, and I want the svn:externals to be as simple as possible and
not require maintenance for future older point releases. And I don't
actually access them that much and they're always available in the
offiical distribution files.

I propose the following:
- adjust the current old version files in
https://svn.apache.org/repos/asf/db/derby/jars to have a lib directory
and only contain derby.jar. remove all other jars.
- modify svn:externals for <trunk>/tools/testing to:
derby https://svn.apache.org/repos/asf/db/derby/jars

The upgrade tests would then be run with suites.All like so:
java -DderbyTesting.oldReleasePath=<trunk>/tools/testing/derby
<junitrunner> org.apache.derbyTesting.functionTests.suites.All

This would also make it easy to copy the old jars around for platform
testing using only jars.

If I don't hear opposing comments, I'll implement this tomorrow (a.m. PDT).

Myrna

Re: discussion re DERBY-1746 - upgrade tests

[Mike Matrigali <mi...@sbcglobal.net>]

Myrna van Lunteren wrote:
> Hi,
> 
> I've found myself in a quandary with respect to the upgrade tests.
> DERBY-1746 describes a problem with sealed jars - which was resolved
> by Dan by converting the test to a junit test using a different
> approach. I want to close this issue.
> 
> However, we now have a svn:externals setting that's unnecessary, and a
> number of  releases with all jars in svn intended to be used by the
> upgrade test, whereas the upgrade test only needs derby.jar.
> svn:externals can only map directories, though.
> 
> While I don't think we can automatically force everyone to run the
> junit upgrade tests with all older releases, we should make it as easy
> as possible.
> 
> I like having all older releases' jar files readily available in svn -
> however, I don't want to duplicate them, I don't want all files in my
> trunk co, and I want the svn:externals to be as simple as possible and
> not require maintenance for future older point releases. And I don't
> actually access them that much and they're always available in the
> offiical distribution files.
> 
> I propose the following:
> - adjust the current old version files in
> https://svn.apache.org/repos/asf/db/derby/jars to have a lib directory
> and only contain derby.jar. remove all other jars.
> - modify svn:externals for <trunk>/tools/testing to:
> derby https://svn.apache.org/repos/asf/db/derby/jars
> 
> The upgrade tests would then be run with suites.All like so:
> java -DderbyTesting.oldReleasePath=<trunk>/tools/testing/derby
> <junitrunner> org.apache.derbyTesting.functionTests.suites.All
Is there anything we can do in the suites.All code to set a default
value for derbyTesting.oldReleasePath if one is not set already.  The
default being wherever the svn:externals defaults to.
> 
> This would also make it easy to copy the old jars around for platform
> testing using only jars.
> 
> If I don't hear opposing comments, I'll implement this tomorrow (a.m. PDT).
> 
> Myrna
> 
> 

Re: discussion re DERBY-1746 - upgrade tests

[Myrna van Lunteren <m....@gmail.com>]

andrew wrote:

> If you're removing all the other jars, why have the extra lib
> directory? Why not just put derby.jar in the directory with the
> version label?

Well, the current upgrade tests use 'lib', to mimic the 'real'
distribution pattern.
It's no problem to remove the lib, though - except for those who
maintain some kind of nightly testing environment and have the
-DderbyTesting.oldReleasePath already set. I can do this tomorrow if
it makes sense to everyone. just means I'd also modify
upgradeRun.java.

>
> > - modify svn:externals for <trunk>/tools/testing to:
> > derby https://svn.apache.org/repos/asf/db/derby/jars
>
> It might make more sense to just map in the directories that
> correspond to the set of jars appropriate for upgrade testing. For
> trunk, this is all of the above, but that won't continue to be true
> for every branch as branches age and releases accumulate that are
> newer than the branch.
>

Wouldn't branches have their own svn:externals setting at some point?
That's how I was thinking of this.
But, sure, I actually was thinking of doing the mapping like that for
a while, currently my svn:externals looks like this:
derby/10.0.2.1/lib https://svn.apache.org/repos/asf/db/derby/jars/10.0.2.1
derby/10.1.1.0/lib https://svn.apache.org/repos/asf/db/derby/jars/10.1.1.0
derby/10.1.2.1/lib https://svn.apache.org/repos/asf/db/derby/jars/10.1.2.1
derby/10.1.3.1/lib https://svn.apache.org/repos/asf/db/derby/jars/10.1.3.1
derby/10.2.1.6/lib https://svn.apache.org/repos/asf/db/derby/jars/10.2.1.6
derby/10.2.2.0/lib https://svn.apache.org/repos/asf/db/derby/jars/10.2.2.0
I can remove the /lib.

> > This would also make it easy to copy the old jars around for platform
> > testing using only jars.
>
> I'm not sure I understand this comment. If you want to run the
> functional tests for a specific version, wouldn't you want the full
> set of jars for that version including the derbyTesting.jar?
>

I meant, for upgrade tests with the latest version.  That is, to run
suites.All including upgrade tests for trunk on a machine on which I
have not installed ant or svn, I can copy all jars of the release
under test, plus the entire derby/testing/derby dir - i.e. holding the
derby.jar of all older versions that the upgrade tests need to be run
against.

Myrna

Re: discussion re DERBY-1746 - upgrade tests

[Andrew McIntyre <mc...@gmail.com>]

On 5/16/07, Myrna van Lunteren <m....@gmail.com> wrote:
>
> The way it will work now, is:
> - if the derbyTesting.oldReleasePath property is not set, the test
> will attempt to access the jars from
> https://svn.apache.org/repos/asf/db/derby/jars. If svn.apache.org is
> not accessible (either because there is no network, or there is some
> other problem with connecting to svn.apache.org), the tests will run
> but fail.
> - if the property is set, the test will attempt to access the jars
> directly underneath
> subdirectories with the version strings. If the jars are not found,
> the tests will get skipped.

Thank you for providing an alternative way to locate the jars. Those
who are running automated nightly tests should be nice to Apache's
infrastructure and point this property to a local checkout of the jars
whenever possible.

andrew

Re: discussion re DERBY-1746 - upgrade tests

[Myrna van Lunteren <m....@gmail.com>]

On 5/15/07, Daniel John Debrunner <dj...@apache.org> wrote:
> Andrew McIntyre wrote:
> > On 5/15/07, Myrna van Lunteren <m....@gmail.com> wrote:
> >>
> >> While I don't think we can automatically force everyone to run the
> >> junit upgrade tests with all older releases, we should make it as easy
> >> as possible.
> >>
> >> I like having all older releases' jar files readily available in svn -
> >> however, I don't want to duplicate them, I don't want all files in my
> >> trunk co, and I want the svn:externals to be as simple as possible and
> >> not require maintenance for future older point releases. And I don't
> >> actually access them that much and they're always available in the
> >> offiical distribution files.
>
> I think the upgrade tests could use the jar files directly from
> https://svn.apache.org/repos/asf/db/derby/jars
> and thus not need to have them copied into a client using svn:externals.
> Of course one would need to be on-line to run the tests ...
>
> Dan.
>
>
>
Thanks for that suggestion.
I've implemented it and will commit shortly.
I'll leave the checked in derby*.jar files as is.
I am removing the 'lib' subdirectory from the upgrade tests' needs.
I am removing the svn:externals property from java/tools/testing.

The way it will work now, is:
- if the derbyTesting.oldReleasePath property is not set, the test
will attempt to access the jars from
https://svn.apache.org/repos/asf/db/derby/jars. If svn.apache.org is
not accessible (either because there is no network, or there is some
other problem with connecting to svn.apache.org), the tests will run
but fail.
- if the property is set, the test will attempt to access the jars
directly underneath
subdirectories with the version strings. If the jars are not found,
the tests will get skipped.

So, after my change the following command:
  > java -DderbyTesting.oldReleasePath=/home/myrna/oldderbyreleases
junit.textui.TestRunner
org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite
will look for the following files:
  /home/myrna/oldderbyreleases/10.0.2.1/derby.jar
  /home/myrna/oldderbyreleases/10.1.1.0/derby.jar
  /home/myrna/oldderbyreleases/10.1.2.1/derby.jar
  /home/myrna/oldderbyreleases/10.1.3.1/derby.jar
  /home/myrna/oldderbyreleases/10.2.1.6/derby.jar
  /home/myrna/oldderbyreleases/10.2.2.0/derby.jar

This also means, that if you have been running with the property set
and dirs with the 'lib' in the path (for example:
/home/myrna/oldderbyreleases/10.0.2.1/lib/derby.jar) you will now not
get any tests run for that old release.

The only way upgrade tests will not run with suites.All is if the
property has been set to an incorrect/non-existing directory.

Regards,
Myrna

Re: discussion re DERBY-1746 - upgrade tests

[Daniel John Debrunner <dj...@apache.org>]

Andrew McIntyre wrote:
> On 5/15/07, Myrna van Lunteren <m....@gmail.com> wrote:
>>
>> While I don't think we can automatically force everyone to run the
>> junit upgrade tests with all older releases, we should make it as easy
>> as possible.
>>
>> I like having all older releases' jar files readily available in svn -
>> however, I don't want to duplicate them, I don't want all files in my
>> trunk co, and I want the svn:externals to be as simple as possible and
>> not require maintenance for future older point releases. And I don't
>> actually access them that much and they're always available in the
>> offiical distribution files.

I think the upgrade tests could use the jar files directly from
https://svn.apache.org/repos/asf/db/derby/jars
and thus not need to have them copied into a client using svn:externals.
Of course one would need to be on-line to run the tests ...

Dan.

Re: discussion re DERBY-1746 - upgrade tests

[Andrew McIntyre <mc...@gmail.com>]

On 5/15/07, Myrna van Lunteren <m....@gmail.com> wrote:
>
> While I don't think we can automatically force everyone to run the
> junit upgrade tests with all older releases, we should make it as easy
> as possible.
>
> I like having all older releases' jar files readily available in svn -
> however, I don't want to duplicate them, I don't want all files in my
> trunk co, and I want the svn:externals to be as simple as possible and
> not require maintenance for future older point releases. And I don't
> actually access them that much and they're always available in the
> offiical distribution files.
>
> I propose the following:
> - adjust the current old version files in
> https://svn.apache.org/repos/asf/db/derby/jars to have a lib directory
> and only contain derby.jar. remove all other jars.

If you're removing all the other jars, why have the extra lib
directory? Why not just put derby.jar in the directory with the
version label?

> - modify svn:externals for <trunk>/tools/testing to:
> derby https://svn.apache.org/repos/asf/db/derby/jars

It might make more sense to just map in the directories that
correspond to the set of jars appropriate for upgrade testing. For
trunk, this is all of the above, but that won't continue to be true
for every branch as branches age and releases accumulate that are
newer than the branch.

> This would also make it easy to copy the old jars around for platform
> testing using only jars.

I'm not sure I understand this comment. If you want to run the
functional tests for a specific version, wouldn't you want the full
set of jars for that version including the derbyTesting.jar?

andrew