You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "David W. Van Couvering" <Da...@Sun.COM> on 2005/09/02 00:34:44 UTC

Sharing code

Hi, all.  Yes, it's your favorite topic.  :)

I've been thinking about this further, and I would like to say that it's 
time to bite the bullet and address this.  I am planning on piloting 
this with the i18n work so that I can reuse the message.properties file 
rather than duplicate error messages. I am sure other uses will quickly 
follow.

I talked with Craig Russell today, and he had some very helpful advice 
for an approach to common code.  It is based on employing engineering 
discipline and policies around shared code. 

The basic principle is you have full backward compatibility.  Each new 
revision of shared code has behavior and interfaces that are fully 
backward-compatible with older revisions.  Here is my proposal on how 
this is implemented.  After some discussion, I'd like to put this up for 
a vote.

- An "interface" is defined as anything externally visible that is or 
may be depended upon by subsystems outside of the common packages.  This 
is not just a Java interface, but any method, constant, variable, class 
name, package name, etc., that is externally accessible.
- All common interfaces are guaranteed to work as defined in all 
subsequent releases of Derby.  This means you can't for instance keep 
the same method signature but rewrite its behavior. 
- If you want to introduce a new revision of an interface (e.g. new 
arguments or new behavior), you do not modify or remove the old 
interface.  You instead create a completely new revision.  For example, 
if you have a method doIt(int a), then the new version would be 
doIt_2(int a) or doIt(int a, String b)
- If an interface needs to be deprecated, it is documented as deprecated 
and is removed in the next major release (e.g if it's deprecated in 
10.2, it can be removed in 11.0).  This should be avoided if at all possible

The common classes will be placed into both derby.jar and 
derbyclient.jar.  When you have a classpath with a network client at one 
revision and the embedded driver at another revision, the jar with the 
highest revision should always go first, e.g 
"/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This 
ensures that the newer code that depends on new interfaces (e.g. a new 
method for a class) will be able to function properly.

Thanks,

David




Re: Sharing code

Posted by Francois Orsini <fr...@gmail.com>.
I think it would be pretty heavy to have to check a version on every 
interface (in this model, an interface is not just a java interface as 
described previously) - The CommonVersion class might work but you still 
need to ensure it gets loaded before _other_ actual (common) interfaces; 
otherwise you could still get into a situation where version check would not 
happen, leading to unexpected (interface) errors as some of the interfaces 
could be accessed very early on during start-up. Not a blocker I think, just 
pointing that out...

--francois

Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
Rick, I like your idea, it definitely simplifies things.  Since we know 
that all classes in the common package are at the same version level, we 
can just have a single class (called CommonVersion) that has a single 
static method (called something like checkVersion(expectedVersion) to do 
what you're describing.

David

Rick Hillegas wrote:

> I like the idea of a version number in the interface. We could provide 
> a utility method so that, at startup, the client and server could 
> complain if the interface wasn't capable enough. It could work like this:
>
> o The client/server startup code would call this utility method, 
> passing in the actual and expected version numbers of the interface.
>
> o If the actual version number was less than the expected number, then 
> the utility could throw an exception.
>
> o The exception would complain that the classpath was mis-wired and 
> explain how to wire it correctly.
>
> In any event, we should add test cases to our (upcoming) compatibility 
> test suite.
>
> -Rick
>
> Daniel John Debrunner wrote:
>
>> David W. Van Couvering wrote:
>>
>>  
>>
>>> Hi, all.  Yes, it's your favorite topic.  :)
>>>
>>>   
>>
>>
>> [snipped - good shared code proposal]
>>
>>  
>>
>>> The common classes will be placed into both derby.jar and
>>> derbyclient.jar.  When you have a classpath with a network client at 
>>> one
>>> revision and the embedded driver at another revision, the jar with the
>>> highest revision should always go first,
>>>   
>>
>>
>> The issue is how to enforce that. I think it's impossible to enforce
>> that or guarantee any class path ordering. Remember class path problems
>> are by far the leading cause of technical support calls related to Java
>> programs.
>>
>> What it means is that the interface needs to have a version number that
>> can be obtained easily, and thus the caller must only use the interface
>> to the level offered. If a caller was written against version 3, but
>> only version 2 is available then it can only use version 2 
>> functionality.
>>
>> Dan.
>>
>>  
>>
>

Re: Sharing code

Posted by Rick Hillegas <Ri...@Sun.COM>.
I like the idea of a version number in the interface. We could provide a 
utility method so that, at startup, the client and server could complain 
if the interface wasn't capable enough. It could work like this:

o The client/server startup code would call this utility method, passing 
in the actual and expected version numbers of the interface.

o If the actual version number was less than the expected number, then 
the utility could throw an exception.

o The exception would complain that the classpath was mis-wired and 
explain how to wire it correctly.

In any event, we should add test cases to our (upcoming) compatibility 
test suite.

-Rick

Daniel John Debrunner wrote:

>David W. Van Couvering wrote:
>
>  
>
>>Hi, all.  Yes, it's your favorite topic.  :)
>>
>>    
>>
>
>[snipped - good shared code proposal]
>
>  
>
>>The common classes will be placed into both derby.jar and
>>derbyclient.jar.  When you have a classpath with a network client at one
>>revision and the embedded driver at another revision, the jar with the
>>highest revision should always go first,
>>    
>>
>
>The issue is how to enforce that. I think it's impossible to enforce
>that or guarantee any class path ordering. Remember class path problems
>are by far the leading cause of technical support calls related to Java
>programs.
>
>What it means is that the interface needs to have a version number that
>can be obtained easily, and thus the caller must only use the interface
>to the level offered. If a caller was written against version 3, but
>only version 2 is available then it can only use version 2 functionality.
>
>Dan.
>
>  
>


Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
So, I've thought about this further, and here is what I like to 
propose.  Our guidelines and any framework we build for writing and 
using common code should allow for a consumer of a common component to 
fall back to reduced functionality if it so chooses, but it should also 
allow a consumer to raise an exception if it encounters a revision of 
the service that it can not work with.  We should recommend that all 
consumers work with at least one minor version prior to the current version.

Thoughts?

Thanks,

David

David W. Van Couvering wrote:

> Given that the edge case we are talking about is where the user 
> intentionally wants to run the network client at a different rev than 
> the embedded client, it seems to me the user must know how to affect 
> the way the VM loads classes, otherwise they wouldn't be able to 
> accomplish this goal in the first place.
> That aside, I really do see this as a laudable goal, but technically I 
> can not get my head around how a consumer can manage the logic of 
> working in a degraded mode.  What if, for example, there are versions 
> 10.1, 10.2, 10.3, and 10.4 of a service, each one with increasingly 
> extended capabilities.  Does every consumer of this service need to 
> know how to work with each of these versions?  It seems to me that you 
> would need pluggable versions of the consumer, one for each version of 
> common code. So you would have this cascading effect of reduced 
> functionality services stacked on top of each other, and ultimately 
> wouldn't you end up with the entire network driver or engine running 
> at an old rev?  Doesn't this defeat the purpose of having mixed 
> versions running in the same VM?
>
> Thanks,
>
> David
>
> Daniel John Debrunner wrote:
>
>> David W. Van Couvering wrote:
>>
>>
>>  
>>
>>> If
>>> the service manager says "I don't have that" then we can throw an
>>> exception saying something like "Unable to load version 10.2 of the
>>> Message Service.  It is probable that your classpath has older versions
>>> of Derby jar files listed first; please check your classpath and make
>>> sure that all Derby jar files are listed ahead of older versions."
>>>   
>>
>>
>> Also Rick said:
>>
>>  
>>
>>> o The exception would complain that the classpath was mis-wired and 
>>> explain how to wire it correctly.   
>>
>>
>> Just remember that classes get loaded into the JVM in more ways than the
>> class path variable, thus there may not be an easy way to describe how
>> to wire it correctly.
>>
>> And what happens if there is no way for the user to reverse the order
>> the jars are searched? This is why a defensive approach is better, if I
>> don't have the functionality available then I continue to work but with
>> reduced functionality.
>>
>> Dan.
>>
>>  
>>

Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
Given that the edge case we are talking about is where the user 
intentionally wants to run the network client at a different rev than 
the embedded client, it seems to me the user must know how to affect the 
way the VM loads classes, otherwise they wouldn't be able to accomplish 
this goal in the first place. 

That aside, I really do see this as a laudable goal, but technically I 
can not get my head around how a consumer can manage the logic of 
working in a degraded mode.  What if, for example, there are versions 
10.1, 10.2, 10.3, and 10.4 of a service, each one with increasingly 
extended capabilities.  Does every consumer of this service need to know 
how to work with each of these versions?  It seems to me that you would 
need pluggable versions of the consumer, one for each version of common 
code. So you would have this cascading effect of reduced functionality 
services stacked on top of each other, and ultimately wouldn't you end 
up with the entire network driver or engine running at an old rev?  
Doesn't this defeat the purpose of having mixed versions running in the 
same VM?

Thanks,

David

Daniel John Debrunner wrote:

>David W. Van Couvering wrote:
>
>
>  
>
>> If
>>the service manager says "I don't have that" then we can throw an
>>exception saying something like "Unable to load version 10.2 of the
>>Message Service.  It is probable that your classpath has older versions
>>of Derby jar files listed first; please check your classpath and make
>>sure that all Derby jar files are listed ahead of older versions."
>>    
>>
>
>Also Rick said:
>
>  
>
>>o The exception would complain that the classpath was mis-wired and explain how to wire it correctly. 
>>    
>>
>
>Just remember that classes get loaded into the JVM in more ways than the
>class path variable, thus there may not be an easy way to describe how
>to wire it correctly.
>
>And what happens if there is no way for the user to reverse the order
>the jars are searched? This is why a defensive approach is better, if I
>don't have the functionality available then I continue to work but with
>reduced functionality.
>
>Dan.
>
>  
>

Re: Sharing code

Posted by Daniel John Debrunner <dj...@debrunners.com>.
David W. Van Couvering wrote:


>  If
> the service manager says "I don't have that" then we can throw an
> exception saying something like "Unable to load version 10.2 of the
> Message Service.  It is probable that your classpath has older versions
> of Derby jar files listed first; please check your classpath and make
> sure that all Derby jar files are listed ahead of older versions."

Also Rick said:

> o The exception would complain that the classpath was mis-wired and explain how to wire it correctly. 

Just remember that classes get loaded into the JVM in more ways than the
class path variable, thus there may not be an easy way to describe how
to wire it correctly.

And what happens if there is no way for the user to reverse the order
the jars are searched? This is why a defensive approach is better, if I
don't have the functionality available then I continue to work but with
reduced functionality.

Dan.


Re: Sharing code

Posted by Mike Matrigali <mi...@sbcglobal.net>.

David W. Van Couvering wrote:
> Yes it's true that we can't control the ordering of a classpath, and we 
> do need to address this.  However, most version policies I have worked 
> with let an old version of a consumer work with a new version of 
> producer, but if a new version of consumer tries to use an old version 
> of producer it gets an exception.  Even today if a new version of Derby 
> finds an old version of a database file, it performs some form of 
> upgrade before it will work with it (right?) rather than continuing to 
> work with the old format (right?).
> 
With soft upgrade no upgrade is done, the code must work with the old
format, so that a previous version of the server can still access the
database.  With respect to database files, we never expect an old
server to be able to access a "new" format - so once you allow
hard upgrade "upgrade=true", you can't go back.

> Maybe you have an idea of how to make it easy to accomplish, but as far 
> as I can see, it's a big burden on the developers of consumer classes to 
> know how to "work with" older versions of common services, especially 
> when this type of configuration (with mixed versions of jar files) is 
> such an edge case.
> 
> That said, I think it's not right for the user to get some strange 
> NoSuchMethodException when the new consumer tries to use a method that 
> doesn't exist in the old producer.  So it seems to me we need some type 
> of service manager (I'm not committing to which one yet, I need to do 
> some investigation) that allows you to say something like "give me an 
> implementation of this interface at this version level or higher".  If 
> the service manager says "I don't have that" then we can throw an 
> exception saying something like "Unable to load version 10.2 of the 
> Message Service.  It is probable that your classpath has older versions 
> of Derby jar files listed first; please check your classpath and make 
> sure that all Derby jar files are listed ahead of older versions."
> 
> David
> 
> Daniel John Debrunner wrote:
> 
>> David W. Van Couvering wrote:
>>
>>  
>>
>>> Hi, all.  Yes, it's your favorite topic.  :)
>>>
>>>   
>>
>>
>> [snipped - good shared code proposal]
>>
>>  
>>
>>> The common classes will be placed into both derby.jar and
>>> derbyclient.jar.  When you have a classpath with a network client at one
>>> revision and the embedded driver at another revision, the jar with the
>>> highest revision should always go first,
>>>   
>>
>>
>> The issue is how to enforce that. I think it's impossible to enforce
>> that or guarantee any class path ordering. Remember class path problems
>> are by far the leading cause of technical support calls related to Java
>> programs.
>>
>> What it means is that the interface needs to have a version number that
>> can be obtained easily, and thus the caller must only use the interface
>> to the level offered. If a caller was written against version 3, but
>> only version 2 is available then it can only use version 2 functionality.
>>
>> Dan.
>>
>>  
>>


Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
Yes it's true that we can't control the ordering of a classpath, and we 
do need to address this.  However, most version policies I have worked 
with let an old version of a consumer work with a new version of 
producer, but if a new version of consumer tries to use an old version 
of producer it gets an exception.  Even today if a new version of Derby 
finds an old version of a database file, it performs some form of 
upgrade before it will work with it (right?) rather than continuing to 
work with the old format (right?).

Maybe you have an idea of how to make it easy to accomplish, but as far 
as I can see, it's a big burden on the developers of consumer classes to 
know how to "work with" older versions of common services, especially 
when this type of configuration (with mixed versions of jar files) is 
such an edge case.

That said, I think it's not right for the user to get some strange 
NoSuchMethodException when the new consumer tries to use a method that 
doesn't exist in the old producer.  So it seems to me we need some type 
of service manager (I'm not committing to which one yet, I need to do 
some investigation) that allows you to say something like "give me an 
implementation of this interface at this version level or higher".  If 
the service manager says "I don't have that" then we can throw an 
exception saying something like "Unable to load version 10.2 of the 
Message Service.  It is probable that your classpath has older versions 
of Derby jar files listed first; please check your classpath and make 
sure that all Derby jar files are listed ahead of older versions."

David

Daniel John Debrunner wrote:

>David W. Van Couvering wrote:
>
>  
>
>>Hi, all.  Yes, it's your favorite topic.  :)
>>
>>    
>>
>
>[snipped - good shared code proposal]
>
>  
>
>>The common classes will be placed into both derby.jar and
>>derbyclient.jar.  When you have a classpath with a network client at one
>>revision and the embedded driver at another revision, the jar with the
>>highest revision should always go first,
>>    
>>
>
>The issue is how to enforce that. I think it's impossible to enforce
>that or guarantee any class path ordering. Remember class path problems
>are by far the leading cause of technical support calls related to Java
>programs.
>
>What it means is that the interface needs to have a version number that
>can be obtained easily, and thus the caller must only use the interface
>to the level offered. If a caller was written against version 3, but
>only version 2 is available then it can only use version 2 functionality.
>
>Dan.
>
>  
>

Re: Sharing code

Posted by Daniel John Debrunner <dj...@debrunners.com>.
David W. Van Couvering wrote:

> Hi, all.  Yes, it's your favorite topic.  :)
> 

[snipped - good shared code proposal]

> The common classes will be placed into both derby.jar and
> derbyclient.jar.  When you have a classpath with a network client at one
> revision and the embedded driver at another revision, the jar with the
> highest revision should always go first,

The issue is how to enforce that. I think it's impossible to enforce
that or guarantee any class path ordering. Remember class path problems
are by far the leading cause of technical support calls related to Java
programs.

What it means is that the interface needs to have a version number that
can be obtained easily, and thus the caller must only use the interface
to the level offered. If a caller was written against version 3, but
only version 2 is available then it can only use version 2 functionality.

Dan.


Re: Sharing code

Posted by Rick Hillegas <Ri...@Sun.COM>.
I suspect this is adequate for the problem user Dan has identified: the 
novice Java developer who doesn't understand classpaths. I suspect that 
we will not come up with a bullet-proof solution for advanced users who 
experiment with tricky techniques for faulting in classes. I would err 
on the side of keeping it simple.

-Rick

Francois Orsini wrote:

> This issue reminds me of the database deadlocks issue - you can't 
> completely get rid of them but you can do a good job at 
> preventing/reducing them...
>
> I think it has been proposed before but can't we have the very first 
> class of the network Client driver and Server check for a 
> legitimate/compatible version of the common classes (package) via a 
> Version class and a static method being passed the caller's version 
> number  - this can be done with a static block early on on the 
> ClientDriver class for instance (for the client part) - The common 
> Version class could have a compatibilty table (matrix) if we need more 
> advanced compatibility check computations...If the common package is 
> compatible (based on the compatibility table) then all is fine, 
> otherwise a (runtime) exception is raised in ClientDriver. I do _not_  
> think we need to check for all interfaces as long as we preliminary 
> checked on Version.checkIfCompatible(versionNumber) early on via the 
> static block - I think we could assume that if you get the proper 
> Version class from the common package and it is compatible, then the 
> rest of the common interfaces should be compatible as well since they 
> would get loaded in the same order as the common Version class which 
> has been identified as (version) compatible. Sure people could come up 
> with twisted scenarios shooting themselves in the foot, but at the end 
> I think we want to be able to address the majority of the 
> cases/situations...just some more thoughts...
>
> --francois



Re: Sharing code

Posted by Francois Orsini <fr...@gmail.com>.
This issue reminds me of the database deadlocks issue - you can't completely 
get rid of them but you can do a good job at preventing/reducing them...

I think it has been proposed before but can't we have the very first class 
of the network Client driver and Server check for a legitimate/compatible 
version of the common classes (package) via a Version class and a static 
method being passed the caller's version number - this can be done with a 
static block early on on the ClientDriver class for instance (for the client 
part) - The common Version class could have a compatibilty table (matrix) if 
we need more advanced compatibility check computations...If the common 
package is compatible (based on the compatibility table) then all is fine, 
otherwise a (runtime) exception is raised in ClientDriver. I do _not_ think 
we need to check for all interfaces as long as we preliminary checked on 
Version.checkIfCompatible(versionNumber) early on via the static block - I 
think we could assume that if you get the proper Version class from the 
common package and it is compatible, then the rest of the common interfaces 
should be compatible as well since they would get loaded in the same order 
as the common Version class which has been identified as (version) 
compatible. Sure people could come up with twisted scenarios shooting 
themselves in the foot, but at the end I think we want to be able to address 
the majority of the cases/situations...just some more thoughts...

--francois

Re: Sharing code

Posted by Kathey Marsden <km...@sbcglobal.net>.
Daniel John Debrunner wrote:

>
>
>You or I might not, but I'm assuming the worst case where the end-user
>has no idea about classpath, or maybe even Java. They have just
>installed two applications, one which uses Derby client at version X and
>one which uses Derby embedded at version Y.
>
>  
>
I  recently spoke with a customer regarding this issue and found that in
fact to be the case.  He said ...

"We have two kind of providers, NetworkServer and embedded.  in the same
JVM, there will be either or both and we can't guarantee that users
don't use a jar file other than the one shipped with Derby. "

This is in an app server environment.

Kathey




Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
I agree with you that managing another JAR file is a pain, and if we can 
get away with not having another JAR file, I would be happy.

Jeremy, you had some good points about what a separate JAR file was 
needed to allow for more flexibility, but then you and Dan had an 
ongoing discussion and I wasn't sure where we ended up there.  It seems 
to me that if an incompatibility is detected using this classpath (where 
"classpath" is defined as any type of ordering for classloading)

/home/derby/10.1/derby.jar:/home/derby/10.2/derby-client.jar

this can be resolved by reordering, e.g.

/home/derby/10.2/derby-client.jar:/home/derby/10.1/derby.jar

(given that we're guaranteed backward-compatibility).

Is that correct or am I missing some subtleties here?

Thanks,

David

Daniel John Debrunner wrote:

>Another argument against putting the common code in a separate jar (e.g.
>derbycommon.jar) is that both the embedded driver and client driver will
>be spread across two jar files. This is the case for the IBM DB2
>Universal driver and while it was the only client for the Derby network
>server, we had several complaints about the two jar files. This was
>because several JDBC tools expect a driver to be in a single jar, e.g.
>they pop up a dialog to allow you to specify the location for a single
>jar. Folks worked around this by re-jarring the IBM driver into a single
>jar, I do not think this is a good approach for Derby.
>
>Dan.
>
>  
>

Re: Sharing code

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Another argument against putting the common code in a separate jar (e.g.
derbycommon.jar) is that both the embedded driver and client driver will
be spread across two jar files. This is the case for the IBM DB2
Universal driver and while it was the only client for the Derby network
server, we had several complaints about the two jar files. This was
because several JDBC tools expect a driver to be in a single jar, e.g.
they pop up a dialog to allow you to specify the location for a single
jar. Folks worked around this by re-jarring the IBM driver into a single
jar, I do not think this is a good approach for Derby.

Dan.


Re: Sharing code

Posted by Jeremy Boynes <jb...@apache.org>.
Daniel John Debrunner wrote:
> 
> You or I might not, but I'm assuming the worst case where the end-user
> has no idea about classpath, or maybe even Java. They have just
> installed two applications, one which uses Derby client at version X and
> one which uses Derby embedded at version Y.
> 

I would hope for that kind of environment the installer for each 
application took care of separating the classpaths so the user was never 
exposed to them. After all one could be using embedded X and the other 
embedded Y with all sorts of unexpected consequences.

That, and nasty platform issues like the 256 char length, are why most 
applications have evolved away from using the environment to solutions 
like extension directories.

In many cases, applications /want/ to be able to load different versions 
  concurrently, not just application servers and frameworks like Spring 
or OSGI but also build tools like Ant or Maven and development tools 
like Eclipse. None of these are able to use the traditional classpath alone.

Clarity in which jar is providing which class (once and only once) 
greatly simplifies things. Users can use the capabilities of the 
application (such as classloader hierarchies and ordering) to ensure 
that the correct version is used for each component without risk of 
cross-contamination between functions.

--
Jeremy

Re: Sharing code

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Jeremy Boynes wrote:

> Daniel John Debrunner wrote:
> 
>> Jeremy Boynes wrote:
>>
>>
>>> David W. Van Couvering wrote:
>>>
>>>
>>>> The common classes will be placed into both derby.jar and
>>>> derbyclient.jar.  When you have a classpath with a network client at
>>>> one revision and the embedded driver at another revision, the jar with
>>>> the highest revision should always go first, e.g
>>>> "/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This
>>>> ensures that the newer code that depends on new interfaces (e.g. a new
>>>> method for a class) will be able to function properly.
>>>
>>>
>>>
>>> Others have pointed out the problems here with classpath ordering.
>>>
>>> The root of this is that the same classes are being included in multiple
>>> jars leading to potential duplication. This can be avoided by placing
>>> them in a separate jar so that each is only present once e.g.
>>
>>
>>
>> Not true, the problem exists if the classes are in a single jar or
>> multiple jars.
>>
> 
> Only if you include two different versions of the same jar in the path,
> which you wouldn't.

You or I might not, but I'm assuming the worst case where the end-user
has no idea about classpath, or maybe even Java. They have just
installed two applications, one which uses Derby client at version X and
one which uses Derby embedded at version Y.

Dan.


Re: Sharing code

Posted by Jeremy Boynes <jb...@apache.org>.
Daniel John Debrunner wrote:
> Jeremy Boynes wrote:
> 
> 
>>David W. Van Couvering wrote:
>>
>>
>>>The common classes will be placed into both derby.jar and
>>>derbyclient.jar.  When you have a classpath with a network client at
>>>one revision and the embedded driver at another revision, the jar with
>>>the highest revision should always go first, e.g
>>>"/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This
>>>ensures that the newer code that depends on new interfaces (e.g. a new
>>>method for a class) will be able to function properly.
>>
>>
>>Others have pointed out the problems here with classpath ordering.
>>
>>The root of this is that the same classes are being included in multiple
>>jars leading to potential duplication. This can be avoided by placing
>>them in a separate jar so that each is only present once e.g.
> 
> 
> Not true, the problem exists if the classes are in a single jar or
> multiple jars.
> 

Only if you include two different versions of the same jar in the path, 
which you wouldn't.

Because there is no overlap between jars the user is able to clearly 
specify the version they want of every class by including the 
appropriate jar in the path irrespective of order.

--
Jeremy

Re: Sharing code

Posted by Daniel John Debrunner <dj...@debrunners.com>.
Jeremy Boynes wrote:

> David W. Van Couvering wrote:
> 
>>
>> The common classes will be placed into both derby.jar and
>> derbyclient.jar.  When you have a classpath with a network client at
>> one revision and the embedded driver at another revision, the jar with
>> the highest revision should always go first, e.g
>> "/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This
>> ensures that the newer code that depends on new interfaces (e.g. a new
>> method for a class) will be able to function properly.
> 
> 
> Others have pointed out the problems here with classpath ordering.
> 
> The root of this is that the same classes are being included in multiple
> jars leading to potential duplication. This can be avoided by placing
> them in a separate jar so that each is only present once e.g.

Not true, the problem exists if the classes are in a single jar or
multiple jars.

Dan.


Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
Thanks, Jeremy.  Interesting point.  If we can assume/require full 
backward-compatibility as I am proposing, we can have the following 
classpath:

derby-10.1.jar;derbyclient-10.2.jar:derbycommon-10.2.jar

and the 10.1 code should work just fine.

Is it possible that you may still end up with a classpath where the 
derbycommon-10.1.jar shows up ahead of the 10.2 jar file and the user is 
unable to remove it?

Thanks,

David


Jeremy Boynes wrote:

> David W. Van Couvering wrote:
>
>>
>> The common classes will be placed into both derby.jar and 
>> derbyclient.jar.  When you have a classpath with a network client at 
>> one revision and the embedded driver at another revision, the jar 
>> with the highest revision should always go first, e.g 
>> "/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This 
>> ensures that the newer code that depends on new interfaces (e.g. a 
>> new method for a class) will be able to function properly.
>
>
> Others have pointed out the problems here with classpath ordering.
>
> The root of this is that the same classes are being included in 
> multiple jars leading to potential duplication. This can be avoided by 
> placing them in a separate jar so that each is only present once e.g.
>
> derby-10.2.jar:derbyclient-10.2.jar:derbycommon-10.2.jar
>
> The downside of this is that multiple jars need to be included to form 
> a complete solution. Concern has been expressed in the past about 
> exceeding the length of the CLASSPATH environment variable. However, 
> most frameworks today, including J2EE, do not use that mechanism to 
> include libraries relying instead on features like the manifest 
> classpath, extensions, WEB-INF/lib, uber-jar or others provided by a 
> container environment.
>
> IMO the benefits of a more modular structure and a clear lack of 
> duplication exceed the benefits of a bundled distribution and I would 
> prefer to see the common code distributed separately.
>
> -- 
> Jeremy


Re: Sharing code

Posted by Jeremy Boynes <jb...@apache.org>.
David W. Van Couvering wrote:
> 
> The common classes will be placed into both derby.jar and 
> derbyclient.jar.  When you have a classpath with a network client at one 
> revision and the embedded driver at another revision, the jar with the 
> highest revision should always go first, e.g 
> "/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This 
> ensures that the newer code that depends on new interfaces (e.g. a new 
> method for a class) will be able to function properly.

Others have pointed out the problems here with classpath ordering.

The root of this is that the same classes are being included in multiple 
jars leading to potential duplication. This can be avoided by placing 
them in a separate jar so that each is only present once e.g.

derby-10.2.jar:derbyclient-10.2.jar:derbycommon-10.2.jar

The downside of this is that multiple jars need to be included to form a 
complete solution. Concern has been expressed in the past about 
exceeding the length of the CLASSPATH environment variable. However, 
most frameworks today, including J2EE, do not use that mechanism to 
include libraries relying instead on features like the manifest 
classpath, extensions, WEB-INF/lib, uber-jar or others provided by a 
container environment.

IMO the benefits of a more modular structure and a clear lack of 
duplication exceed the benefits of a bundled distribution and I would 
prefer to see the common code distributed separately.

--
Jeremy

Re: Sharing code

Posted by "David W. Van Couvering" <Da...@Sun.COM>.
Sorry, Lance, I am assuming you mean I can't enforce classpath order.  
Yes, Dan has also made that point, thanks.

David

Lance J. Andersen wrote:

>
>
> David W. Van Couvering wrote:
>
>> Hi, all.  Yes, it's your favorite topic.  :)
>>
>> I've been thinking about this further, and I would like to say that 
>> it's time to bite the bullet and address this.  I am planning on 
>> piloting this with the i18n work so that I can reuse the 
>> message.properties file rather than duplicate error messages. I am 
>> sure other uses will quickly follow.
>>
>> I talked with Craig Russell today, and he had some very helpful 
>> advice for an approach to common code.  It is based on employing 
>> engineering discipline and policies around shared code.
>> The basic principle is you have full backward compatibility.  Each 
>> new revision of shared code has behavior and interfaces that are 
>> fully backward-compatible with older revisions.  Here is my proposal 
>> on how this is implemented.  After some discussion, I'd like to put 
>> this up for a vote.
>>
>> - An "interface" is defined as anything externally visible that is or 
>> may be depended upon by subsystems outside of the common packages.  
>> This is not just a Java interface, but any method, constant, 
>> variable, class name, package name, etc., that is externally accessible.
>> - All common interfaces are guaranteed to work as defined in all 
>> subsequent releases of Derby.  This means you can't for instance keep 
>> the same method signature but rewrite its behavior. - If you want to 
>> introduce a new revision of an interface (e.g. new arguments or new 
>> behavior), you do not modify or remove the old interface.  You 
>> instead create a completely new revision.  For example, if you have a 
>> method doIt(int a), then the new version would be doIt_2(int a) or 
>> doIt(int a, String b)
>> - If an interface needs to be deprecated, it is documented as 
>> deprecated and is removed in the next major release (e.g if it's 
>> deprecated in 10.2, it can be removed in 11.0).  This should be 
>> avoided if at all possible
>>
>> The common classes will be placed into both derby.jar and 
>> derbyclient.jar.  When you have a classpath with a network client at 
>> one revision and the embedded driver at another revision, the jar 
>> with the highest revision should always go first, e.g 
>> "/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This 
>> ensures that the newer code that depends on new interfaces (e.g. a 
>> new method for a class) will be able to function properly.
>
>
> Hi David,
>
> I do not believe you can enforce this if you are you the lib ext dir 
> model which would make this problematic for a J2EE environment.
>
> Regards
> Lance
>
>>
>> Thanks,
>>
>> David
>>
>>
>>

Re: Sharing code

Posted by "Lance J. Andersen" <La...@Sun.COM>.

David W. Van Couvering wrote:

> Hi, all.  Yes, it's your favorite topic.  :)
>
> I've been thinking about this further, and I would like to say that 
> it's time to bite the bullet and address this.  I am planning on 
> piloting this with the i18n work so that I can reuse the 
> message.properties file rather than duplicate error messages. I am 
> sure other uses will quickly follow.
>
> I talked with Craig Russell today, and he had some very helpful advice 
> for an approach to common code.  It is based on employing engineering 
> discipline and policies around shared code.
> The basic principle is you have full backward compatibility.  Each new 
> revision of shared code has behavior and interfaces that are fully 
> backward-compatible with older revisions.  Here is my proposal on how 
> this is implemented.  After some discussion, I'd like to put this up 
> for a vote.
>
> - An "interface" is defined as anything externally visible that is or 
> may be depended upon by subsystems outside of the common packages.  
> This is not just a Java interface, but any method, constant, variable, 
> class name, package name, etc., that is externally accessible.
> - All common interfaces are guaranteed to work as defined in all 
> subsequent releases of Derby.  This means you can't for instance keep 
> the same method signature but rewrite its behavior. - If you want to 
> introduce a new revision of an interface (e.g. new arguments or new 
> behavior), you do not modify or remove the old interface.  You instead 
> create a completely new revision.  For example, if you have a method 
> doIt(int a), then the new version would be doIt_2(int a) or doIt(int 
> a, String b)
> - If an interface needs to be deprecated, it is documented as 
> deprecated and is removed in the next major release (e.g if it's 
> deprecated in 10.2, it can be removed in 11.0).  This should be 
> avoided if at all possible
>
> The common classes will be placed into both derby.jar and 
> derbyclient.jar.  When you have a classpath with a network client at 
> one revision and the embedded driver at another revision, the jar with 
> the highest revision should always go first, e.g 
> "/home/derby/10.2/derbyclient.jar:/home/derby/10.1/derby.jar".  This 
> ensures that the newer code that depends on new interfaces (e.g. a new 
> method for a class) will be able to function properly.

Hi David,

I do not believe you can enforce this if you are you the lib ext dir 
model which would make this problematic for a J2EE environment.

Regards
Lance

>
> Thanks,
>
> David
>
>
>