Refactoring ant junit target to automatically set classpath

[Kristian Waagan <kr...@oracle.com>]

Hi all,

I just uploaded patch with a possible solution for the issue DERBY-4089 
[1], where it is suggested that it should be possible to run the junit 
targets without manually setting the classpath after 'ant all' has been 
run. This suggestion got some pushback by a claim that it would be more 
reasonable to run against the production jars instead of the 
classes-directory.

Since the patch may affect some people using the junit targets, I'd like 
to get some feedback before I make any changes.


--> CHANGE
The classpath for the junit targets will be set automatically, with the 
following preferences:
  o if derby.junit.classpath is specified by the user, don't do anything
  o if derby.junit.classpath is unspecified
     o look for insane jars
     o look for sane jars
     o if either of them are found, set derby.junit.classpath to the 
empty string
  o if no jars are located [2], set derby.junit.classpath to the 
classes-directory
  o additionally, ant will append the user's CLASSPATH environment 
variable to the junit classpath


--> WHO WILL BE AFFECTED
Users who have set the CLASSPATH environment variable when running the 
junit-tasks.
With the changes mentioned, it is likely that what's on the CLASSPATH 
variable will be shadowed by either the jars or the classes-directory.
To get the same behavior as earlier, you would have to set 
derby.junit.classpath="" when invoking ant.


--> PROBLEM?
ant clobber doesn't clean the jars directory ([2])
If a developer has built the jars, then runs 'ant clobber', modifies the 
code, runs 'ant all', followed by for instance 'ant junit-all', the bits 
tested will be the now outdated jar files.

I did find this from Dan:
"I think some of it is history with the product. The Cloudscape jars,
when it was a closed source product, took a long time to build and the
process needed a machine with a lot of memeory. This was due to the
obfuscation process. Thus developers typically did not build the jars,
only the classes, and the build scripts were set up to reflect that."
Are there other reasons why 'ant clobber' doesn't delete the jars?
Can we make 'ant clobber' delete the jars?


Finally, if the patch is committed, the following targets are basically 
rendered redundant:
junit-all-codeline-jars
junit-system-codeline-jars
junit-single-codeline-jars

Is anyone using these?
If not, I'd say we remove them.


-- 
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-4089
[2] Strictly speaking, the script only checks for the existence of the 
directory in which the jars are supposed to be located.
[3] Seems Bryan was bit by this a while ago: 
http://bryanpendleton.blogspot.com/2009/05/derbys-clean-targets-are-quite-complex.html

Re: Refactoring ant junit target to automatically set classpath

[Dave Brosius <db...@apache.org>]

+1 thats great

On 06/09/2011 08:30 AM, Kristian Waagan wrote:
> Hi all,
>
> I just uploaded patch with a possible solution for the issue 
> DERBY-4089 [1], where it is suggested that it should be possible to 
> run the junit targets without manually setting the classpath after 
> 'ant all' has been run. This suggestion got some pushback by a claim 
> that it would be more reasonable to run against the production jars 
> instead of the classes-directory.
>
> Since the patch may affect some people using the junit targets, I'd 
> like to get some feedback before I make any changes. 

Re: Refactoring ant junit target to automatically set classpath

[Kristian Waagan <kr...@oracle.com>]

On 09.06.11 14:30, Kristian Waagan wrote:
> Hi all,
>
> I just uploaded patch with a possible solution for the issue DERBY-4089
> [1], where it is suggested that it should be possible to run the junit
> targets without manually setting the classpath after 'ant all' has been
> run. This suggestion got some pushback by a claim that it would be more
> reasonable to run against the production jars instead of the
> classes-directory.
>
> Since the patch may affect some people using the junit targets, I'd like
> to get some feedback before I make any changes.
>
>

I committed the change on trunk with revision 1136011.


-- 
Kristian

> --> CHANGE
> The classpath for the junit targets will be set automatically, with the
> following preferences:
> o if derby.junit.classpath is specified by the user, don't do anything
> o if derby.junit.classpath is unspecified
> o look for insane jars
> o look for sane jars
> o if either of them are found, set derby.junit.classpath to the empty
> string
> o if no jars are located [2], set derby.junit.classpath to the
> classes-directory
> o additionally, ant will append the user's CLASSPATH environment
> variable to the junit classpath
>
>
> --> WHO WILL BE AFFECTED
> Users who have set the CLASSPATH environment variable when running the
> junit-tasks.
> With the changes mentioned, it is likely that what's on the CLASSPATH
> variable will be shadowed by either the jars or the classes-directory.
> To get the same behavior as earlier, you would have to set
> derby.junit.classpath="" when invoking ant.
>
>
> --> PROBLEM?
> ant clobber doesn't clean the jars directory ([2])
> If a developer has built the jars, then runs 'ant clobber', modifies the
> code, runs 'ant all', followed by for instance 'ant junit-all', the bits
> tested will be the now outdated jar files.
>
> I did find this from Dan:
> "I think some of it is history with the product. The Cloudscape jars,
> when it was a closed source product, took a long time to build and the
> process needed a machine with a lot of memeory. This was due to the
> obfuscation process. Thus developers typically did not build the jars,
> only the classes, and the build scripts were set up to reflect that."
> Are there other reasons why 'ant clobber' doesn't delete the jars?
> Can we make 'ant clobber' delete the jars?
>
>
> Finally, if the patch is committed, the following targets are basically
> rendered redundant:
> junit-all-codeline-jars
> junit-system-codeline-jars
> junit-single-codeline-jars
>
> Is anyone using these?
> If not, I'd say we remove them.
>
>

Re: Refactoring ant junit target to automatically set classpath

[Kristian Waagan <kr...@oracle.com>]

On 09.06.11 16:50, Rick Hillegas wrote:
> Thanks for fixing this stuff, Kristian. I think that your proposed
> changes to the JUnit framework sound reasonable and won't disrupt the
> way I run the tests, so +1 from me.
>
> I agree that the clobber target should delete the jar artifacts.
> However, the release-generation target "prepareforrelease" relies on the
> current odd behavior. That target builds both the sane and insane
> versions of the jar files; when that target exits, both versions are
> ready to be stuffed into release distributions. If you make clobber do
> the right thing, then please fix prepareforrelease so that it will still
> produce both the sane and insane versions side-by-side.

Thanks, Rick.

I've identified that the following targets rely on clobber:
   cibuild
   release (via prepareforrelease)
   snapshot (invoked from prepareforrelease)
   buildworld

In addition we have cleanjars and cleanalljars; the former will delete 
jars/$sanity and the second will delete jars/*. To make it worthwhile, I 
think clobber should invoke cleanalljars.

I'm hoping the following changes will suffice, but that remains to be seen:
  1) Rename clobber to clobber-keep-jars
  2) Redefine clobber to invoke:
      o clobber-keep-jars
      o cleanalljars
  3) Identify usages of clobber that must be replaced with clobber-keep-jars

I'll soon file a JIRA to track this work, but I'd like to wait a while 
longer (i.e. trying to avoid that the changes break the work-flow for ppl).


Regards,
-- 
Kristian

>
> Thanks,
> -Rick
>
> On 6/9/11 5:30 AM, Kristian Waagan wrote:
>> Hi all,
>>
>> I just uploaded patch with a possible solution for the issue
>> DERBY-4089 [1], where it is suggested that it should be possible to
>> run the junit targets without manually setting the classpath after
>> 'ant all' has been run. This suggestion got some pushback by a claim
>> that it would be more reasonable to run against the production jars
>> instead of the classes-directory.
>>
>> Since the patch may affect some people using the junit targets, I'd
>> like to get some feedback before I make any changes.
>>
>>
>> --> CHANGE
>> The classpath for the junit targets will be set automatically, with
>> the following preferences:
>> o if derby.junit.classpath is specified by the user, don't do anything
>> o if derby.junit.classpath is unspecified
>> o look for insane jars
>> o look for sane jars
>> o if either of them are found, set derby.junit.classpath to the empty
>> string
>> o if no jars are located [2], set derby.junit.classpath to the
>> classes-directory
>> o additionally, ant will append the user's CLASSPATH environment
>> variable to the junit classpath
>>
>>
>> --> WHO WILL BE AFFECTED
>> Users who have set the CLASSPATH environment variable when running the
>> junit-tasks.
>> With the changes mentioned, it is likely that what's on the CLASSPATH
>> variable will be shadowed by either the jars or the classes-directory.
>> To get the same behavior as earlier, you would have to set
>> derby.junit.classpath="" when invoking ant.
>>
>>
>> --> PROBLEM?
>> ant clobber doesn't clean the jars directory ([2])
>> If a developer has built the jars, then runs 'ant clobber', modifies
>> the code, runs 'ant all', followed by for instance 'ant junit-all',
>> the bits tested will be the now outdated jar files.
>>
>> I did find this from Dan:
>> "I think some of it is history with the product. The Cloudscape jars,
>> when it was a closed source product, took a long time to build and the
>> process needed a machine with a lot of memeory. This was due to the
>> obfuscation process. Thus developers typically did not build the jars,
>> only the classes, and the build scripts were set up to reflect that."
>> Are there other reasons why 'ant clobber' doesn't delete the jars?
>> Can we make 'ant clobber' delete the jars?
>>
>>
>> Finally, if the patch is committed, the following targets are
>> basically rendered redundant:
>> junit-all-codeline-jars
>> junit-system-codeline-jars
>> junit-single-codeline-jars
>>
>> Is anyone using these?
>> If not, I'd say we remove them.
>>
>>
>

Re: Refactoring ant junit target to automatically set classpath

[Rick Hillegas <ri...@oracle.com>]

Thanks for fixing this stuff, Kristian. I think that your proposed 
changes to the JUnit framework sound reasonable and won't disrupt the 
way I run the tests, so +1 from me.

I agree that the clobber target should delete the jar artifacts. 
However, the release-generation target "prepareforrelease" relies on the 
current odd behavior. That target builds both the sane and insane 
versions of the jar files; when that target exits, both versions are 
ready to be stuffed into release distributions. If you make clobber do 
the right thing, then please fix prepareforrelease so that it will still 
produce both the sane and insane versions side-by-side.

Thanks,
-Rick

On 6/9/11 5:30 AM, Kristian Waagan wrote:
> Hi all,
>
> I just uploaded patch with a possible solution for the issue 
> DERBY-4089 [1], where it is suggested that it should be possible to 
> run the junit targets without manually setting the classpath after 
> 'ant all' has been run. This suggestion got some pushback by a claim 
> that it would be more reasonable to run against the production jars 
> instead of the classes-directory.
>
> Since the patch may affect some people using the junit targets, I'd 
> like to get some feedback before I make any changes.
>
>
> --> CHANGE
> The classpath for the junit targets will be set automatically, with 
> the following preferences:
>  o if derby.junit.classpath is specified by the user, don't do anything
>  o if derby.junit.classpath is unspecified
>     o look for insane jars
>     o look for sane jars
>     o if either of them are found, set derby.junit.classpath to the 
> empty string
>  o if no jars are located [2], set derby.junit.classpath to the 
> classes-directory
>  o additionally, ant will append the user's CLASSPATH environment 
> variable to the junit classpath
>
>
> --> WHO WILL BE AFFECTED
> Users who have set the CLASSPATH environment variable when running the 
> junit-tasks.
> With the changes mentioned, it is likely that what's on the CLASSPATH 
> variable will be shadowed by either the jars or the classes-directory.
> To get the same behavior as earlier, you would have to set 
> derby.junit.classpath="" when invoking ant.
>
>
> --> PROBLEM?
> ant clobber doesn't clean the jars directory ([2])
> If a developer has built the jars, then runs 'ant clobber', modifies 
> the code, runs 'ant all', followed by for instance 'ant junit-all', 
> the bits tested will be the now outdated jar files.
>
> I did find this from Dan:
> "I think some of it is history with the product. The Cloudscape jars,
> when it was a closed source product, took a long time to build and the
> process needed a machine with a lot of memeory. This was due to the
> obfuscation process. Thus developers typically did not build the jars,
> only the classes, and the build scripts were set up to reflect that."
> Are there other reasons why 'ant clobber' doesn't delete the jars?
> Can we make 'ant clobber' delete the jars?
>
>
> Finally, if the patch is committed, the following targets are 
> basically rendered redundant:
> junit-all-codeline-jars
> junit-system-codeline-jars
> junit-single-codeline-jars
>
> Is anyone using these?
> If not, I'd say we remove them.
>
>