You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Simon Kitching <si...@ecnetwork.co.nz> on 2004/05/04 00:18:29 UTC
Re: [general] library management
On Tue, 2004-05-04 at 04:53, Craig McClanahan wrote:
> robert burrell donkin wrote:
>
> > On 26 Apr 2004, at 00:26, Simon Kitching wrote:
> >
> >> On Sun, 2004-04-25 at 15:00, Gary Gregory wrote:
> >>
> >>> Or:
> >>>
> >>> You release commons-all.jar with a pruner. This pruner would run just
> >>> like base it's input on Clover output or manual input to create a
> >>> smaller what-my-app-needs-out-of-commons.jar.
> >>
> >>
> >> Like some other posters, I understand that people have some problems
> >> with commons being composed of a dozen separate projects. However I
> >> don't see the "one commons jar" approach as being feasable. I can't
> >> imagine how releases would be synchronized, how unit tests would be
> >> applied to the combined code, etc.
> >
> >
> > there's nothing technically unfeasible about this. (gump does this and
> > more :)
> >
> > one of the good rules we have is that each commons release should only
> > depend on previously released code so it should just be a case of
> > re-rolling the big jar and creating a new big-commons release each
> > time any component was released.
> >
> If you want to play with exactly this concept, check out the build.xml
> script in [combo]. It was designed to make a "pick the latest published
> release of all the constituent libraries" policy very easy to accomplish
> -- all you need is to update the CVS tag to pull as new releases are
> made. In addition, someone who wanted a different set of versions could
> easily override the CVS tags themselves.
>
> There are some wrinkles yet to be worked out, and I don't have time at
> the moment to focus on it, but there's a starting point for someone
> interested. If for no other reason, consolidated Javadocs for all of
> commons (one of the outputs of this script) is very useful.
>
> > IMHO the main obstacle is organizational. people have enough
> > difficulty with the current release process without adding to it.
> > unless someone's willing to step forward and volunteer to manage the
> > organizational side of the big commons jar (including release
> > management) then it's not really worth wasting time on. (craig started
> > something similar i'd guess over a year ago now but no one was really
> > interested enough in the idea to push it forward.) i'd be happy to
> > give a hand on the integration side but i don't cut jakarta releases
> > any more (so i'm not willing to take this one on).
> >
> I think the mechanical aspects of building a combination JAR can be
> dealt with. What I don't think we have in place at all is any sort of
> unit or system tests to ensure that version X of library FOO works with
> version Y of library BAR. Therefore, I'd be cautious about declaring
> anything about a release of [combo] other than "here is a particular
> combination of Commons libraries, conveniently packaged in one JAR for you."
That's my concern. If collections releases 3.0, we can't just roll a new
"commons-all" jar without testing that projects like beanutils or
digester work with this new collections release. But how on earth do we
do that? It seems to me that the unit tests for every project that
depends upon the newly released project would need to be run - at the
very least - in order to provide any guarantees that the "commons-all"
release is stable. Does the "combo" project do this? If not, is it
really the kind of release that Jakarta wants to put its name to?
Gary Gregory's "pruner" suggestion seems like a possible way forward to
me. The pruner could takes a configuration file that specifies a set of
classes. It would then run against a set of jars and extract the
specified classes plus all their dependencies into a new jar file.
Jakarta could publish a set of configuration files for various purposes.
Running the tool against a single jar, or against a project's jars + its
officially-supported dependencies would guarantee a functioning result.
Running the tool against the complete set of latest releases of all
projects might not result in a jar file that is valid (eg collections
3.0 + classes that depend on collections-2.x). The one issue it does
*not* address is generation of javadoc for the resulting jar. However
such a tool should be simple for users to use. We *could* potentially
automate its use to build a set of customised releases, but I would
certainly not be in favour of this for the "cross-project" bundles, due
to the inability to run the necessary unit tests as described above.
What do people think about the "pruner" suggestion?
Regards,
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [general] library management
Posted by Henri Yandell <ba...@generationjava.com>.
[not really a direct answer of your email as I'm -0 to Commons creating
systems to create jar slices]
Tools to do such things to jars exist already I think from the Applet
days. I know I've written something that seemed to work for binaries but
had trouble with source [due to final variable inlining].
Personally I think that if project A needs Collection 2.0 and project B
needs Collection 3.0, then project A needs to get a release out.
This would be an advantage of some larger, managed by Commons, jar. It
would lag behind the releases of the minor components, much as SuSE ES and
Red Hat AS lag behind their desktop versions.
I'd expect: Commons-J2SE.jar and Commons-J2EE.jar, the latter of which
allows things like commons-email and commons-modeler which [assuming] have
only dependencies on Commons and J2EE.
Hen
On Tue, 4 May 2004, Mario Ivankovits wrote:
> *) It should be possible to strip out only the needed classfiles out of
> the commons-project to create a new custom all-in-one.jar - right?
> *) If not - at least it should be possible to pack all needet jar files
> together.
> *) If a commons project has as dependency e.g. collections-2.0 in the
> project.xml while collections-3.0 is already out - there is (should be)
> no chance to bundle collections-3.0. Maybe the user could overrule this
> (and run into erros).
>
> But here i see a problem - what if one project (A) states
> collections-2.0 and another one (B) states collections-3.0 as their
> dependency. The user might have no chance to use both together.
> So beside something like pruner - a custom classloader might also be needet.
> Based on an configuration - it should know, if a class must be loaded
> from e.g collections-2.0.jar or collections-3.0.jar. So if project-A
> requested a collection it should load it from collections-2.0.jar else
> for project-B it uses collections-3.0.jar.
>
> I came to this if i think to upgrade to the new net-1.2 in vfs - what if
> one would like to use vfs and net-1.1 together. For sure - it might be
> no great action to upgrade to net-1.2 but for some reason it is not
> possible.
> It would be nice to be able to use net-1.2 with vfs ONLY! (there are
> some other dependencies which could conflict on the users machine -
> webdav??)
>
> For sure - this might only work as long as there is no reference about
> such a "sealed" class exported to the userspace. I am not sure, but this
> might often be the case.
>
> So i see two problems:
> 1) to create the smallest possible jar
> 2) to use multiple versions of the same class in one project
>
> -- Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [general] library management
Posted by Mario Ivankovits <im...@apache.org>.
I would like to bring something in line about this discussion - just to
be sure - i know what you talking about ;-)
*) It should be possible to strip out only the needed classfiles out of
the commons-project to create a new custom all-in-one.jar - right?
*) If not - at least it should be possible to pack all needet jar files
together.
*) If a commons project has as dependency e.g. collections-2.0 in the
project.xml while collections-3.0 is already out - there is (should be)
no chance to bundle collections-3.0. Maybe the user could overrule this
(and run into erros).
But here i see a problem - what if one project (A) states
collections-2.0 and another one (B) states collections-3.0 as their
dependency. The user might have no chance to use both together.
So beside something like pruner - a custom classloader might also be needet.
Based on an configuration - it should know, if a class must be loaded
from e.g collections-2.0.jar or collections-3.0.jar. So if project-A
requested a collection it should load it from collections-2.0.jar else
for project-B it uses collections-3.0.jar.
I came to this if i think to upgrade to the new net-1.2 in vfs - what if
one would like to use vfs and net-1.1 together. For sure - it might be
no great action to upgrade to net-1.2 but for some reason it is not
possible.
It would be nice to be able to use net-1.2 with vfs ONLY! (there are
some other dependencies which could conflict on the users machine -
webdav??)
For sure - this might only work as long as there is no reference about
such a "sealed" class exported to the userspace. I am not sure, but this
might often be the case.
So i see two problems:
1) to create the smallest possible jar
2) to use multiple versions of the same class in one project
-- Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [general] library management
Posted by Craig McClanahan <cr...@apache.org>.
Simon Kitching wrote:
>On Tue, 2004-05-04 at 10:45, Craig McClanahan wrote:
>
>
>>Simon Kitching wrote:
>>
>>
>>>That's my concern. If collections releases 3.0, we can't just roll a new
>>>"commons-all" jar without testing that projects like beanutils or
>>>digester work with this new collections release. But how on earth do we
>>>do that? It seems to me that the unit tests for every project that
>>>depends upon the newly released project would need to be run - at the
>>>very least - in order to provide any guarantees that the "commons-all"
>>>release is stable. Does the "combo" project do this? If not, is it
>>>really the kind of release that Jakarta wants to put its name to?
>>>
>>>
>>>
>>>
>>>
>>I'd worry about this more if I believed that projects that ship multiple
>>commons JARs today (like Tomcat or Struts) do any interoperability
>>testing beyond making sure that their own uses of the underlying
>>libraries was successful. It doesn't seem unreasonable to offer a
>>convenience package whose contents is a specifically listed set of
>>commons JARs ... the functional result of using this JAR should match
>>the functional result of using all the individual JAR files themselves.
>>
>>If we're still concerned about compatibility testing, another way to
>>look at [combo] is a tool for people to build their own custom
>>combinations, rather than something we use to ship a combined package
>>ourselves.
>>
>>
>
>If combo is just something that people working on tomcat, geronimo, etc
>check out from CVS then use to build a custom jar that sounds fine to
>me.
>
>If combo is used to generate a bunch of custom jars, eg
>"commons-all.jar", that are directly downloadable via a link on the
>jakarta commons page, then I am concerned that it would damage Jakarta's
>image. Publishing binaries that may crash when run with some kind of
>"incompatible classes" error message doesn't look good, no matter what
>disclaimers are on the link. And if the disclaimers are too severe, then
>who will use these jars anyway?
>
>
>
I wouldn't see us publishing anything more than just one combo JAR that
was always the latest released version of every commons proper package
(i.e. not sandbox). Anyone who wanted to subset, or pick different
version combinations, can do it with properties files and a rebuild.
>>>Gary Gregory's "pruner" suggestion seems like a possible way forward to
>>>me. The pruner could takes a configuration file that specifies a set of
>>>classes. It would then run against a set of jars and extract the
>>>specified classes plus all their dependencies into a new jar file.
>>>Jakarta could publish a set of configuration files for various purposes.
>>>Running the tool against a single jar, or against a project's jars + its
>>>officially-supported dependencies would guarantee a functioning result.
>>>Running the tool against the complete set of latest releases of all
>>>projects might not result in a jar file that is valid (eg collections
>>>3.0 + classes that depend on collections-2.x). The one issue it does
>>>*not* address is generation of javadoc for the resulting jar. However
>>>such a tool should be simple for users to use. We *could* potentially
>>>automate its use to build a set of customised releases, but I would
>>>certainly not be in favour of this for the "cross-project" bundles, due
>>>to the inability to run the necessary unit tests as described above.
>>>
>>>What do people think about the "pruner" suggestion?
>>>
>>>
>>>
>>>
>>>
>>I can see how pruner might be useful for static dependencies (i.e.
>>import statements). What do you do about dynamic dependencies (for
>>example, a class loaded by ClassLoader.loadClass() whose name is only
>>known because it's in a configuration file)? And how does this do any
>>better at dealing with inter-library compatibility testing?
>>Compiles-clean isn't really good enough -- and Gump can already tell us
>>that.
>>
>>
>
>Dynamic dependencies just need to be specified in the configuration
>file. As an example, in Digester the PluginCreateRule is not directly
>referenced by the Digester class. So if the config file just specifies
>"org.apache.commons.Digester" then the PluginCreateRule will not be in
>the resulting jar. If the config file specifies PluginCreateRule as
>well, then not only that Rule but all the necessary supporting classes
>(most of the rest of the plugins module) gets pulled in too. The person
>writing the config file needs to decide what they do and do not want
>included - which is the point, really.
>
>Re inter-library compatibility:
>
>If you gather together digester.jar + all its official dependencies
>(beanutils 1.6.1, collections 2.1) then run the pruner you are
>guaranteed a good resulting jar. We could even automate this process (eg
>via a maven plugin), because Jakarta would be publishing a proper subset
>of libraries that have been tested together.
>
>If you gather together digester.jar + the *latest* releases of
>beanutils, collections, etc. and run the pruner then you are *not*
>guaranteed a good resulting jar. I would recommend that Jakarta never
>publish such jars pre-built. But I see no problems with publishing the
>pruner configuration files that allow people to build such custom jars
>very easily.
>
>In the case of Collections, which has no dependencies, things are even
>easier. Rather than mess around with the ant buildfiles to generate a
>bunch of different distribution targets, just create a set of pruner
>config files which can be run against the complete collections jar
>(automatically or manually) to generate the simpler kind of
>distributions that people have been asking for. [except that propably
>also want matching pruned javadoc, which I don't have an answer for at
>the moment].
>
>NB: I'm not heavily pushing the pruner concept. It just seemed an
>interesting idea I thought might be worth raising for discussion..
>
>
>
That makes sense -- thanks.
>Regards,
>
>Simon
>
>
>
Craig
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [general] library management
Posted by Simon Kitching <si...@ecnetwork.co.nz>.
On Tue, 2004-05-04 at 10:45, Craig McClanahan wrote:
> Simon Kitching wrote:
> >
> >That's my concern. If collections releases 3.0, we can't just roll a new
> >"commons-all" jar without testing that projects like beanutils or
> >digester work with this new collections release. But how on earth do we
> >do that? It seems to me that the unit tests for every project that
> >depends upon the newly released project would need to be run - at the
> >very least - in order to provide any guarantees that the "commons-all"
> >release is stable. Does the "combo" project do this? If not, is it
> >really the kind of release that Jakarta wants to put its name to?
> >
> >
> >
> I'd worry about this more if I believed that projects that ship multiple
> commons JARs today (like Tomcat or Struts) do any interoperability
> testing beyond making sure that their own uses of the underlying
> libraries was successful. It doesn't seem unreasonable to offer a
> convenience package whose contents is a specifically listed set of
> commons JARs ... the functional result of using this JAR should match
> the functional result of using all the individual JAR files themselves.
>
> If we're still concerned about compatibility testing, another way to
> look at [combo] is a tool for people to build their own custom
> combinations, rather than something we use to ship a combined package
> ourselves.
If combo is just something that people working on tomcat, geronimo, etc
check out from CVS then use to build a custom jar that sounds fine to
me.
If combo is used to generate a bunch of custom jars, eg
"commons-all.jar", that are directly downloadable via a link on the
jakarta commons page, then I am concerned that it would damage Jakarta's
image. Publishing binaries that may crash when run with some kind of
"incompatible classes" error message doesn't look good, no matter what
disclaimers are on the link. And if the disclaimers are too severe, then
who will use these jars anyway?
>
> >Gary Gregory's "pruner" suggestion seems like a possible way forward to
> >me. The pruner could takes a configuration file that specifies a set of
> >classes. It would then run against a set of jars and extract the
> >specified classes plus all their dependencies into a new jar file.
> >Jakarta could publish a set of configuration files for various purposes.
> >Running the tool against a single jar, or against a project's jars + its
> >officially-supported dependencies would guarantee a functioning result.
> >Running the tool against the complete set of latest releases of all
> >projects might not result in a jar file that is valid (eg collections
> >3.0 + classes that depend on collections-2.x). The one issue it does
> >*not* address is generation of javadoc for the resulting jar. However
> >such a tool should be simple for users to use. We *could* potentially
> >automate its use to build a set of customised releases, but I would
> >certainly not be in favour of this for the "cross-project" bundles, due
> >to the inability to run the necessary unit tests as described above.
> >
> >What do people think about the "pruner" suggestion?
> >
> >
> >
> I can see how pruner might be useful for static dependencies (i.e.
> import statements). What do you do about dynamic dependencies (for
> example, a class loaded by ClassLoader.loadClass() whose name is only
> known because it's in a configuration file)? And how does this do any
> better at dealing with inter-library compatibility testing?
> Compiles-clean isn't really good enough -- and Gump can already tell us
> that.
Dynamic dependencies just need to be specified in the configuration
file. As an example, in Digester the PluginCreateRule is not directly
referenced by the Digester class. So if the config file just specifies
"org.apache.commons.Digester" then the PluginCreateRule will not be in
the resulting jar. If the config file specifies PluginCreateRule as
well, then not only that Rule but all the necessary supporting classes
(most of the rest of the plugins module) gets pulled in too. The person
writing the config file needs to decide what they do and do not want
included - which is the point, really.
Re inter-library compatibility:
If you gather together digester.jar + all its official dependencies
(beanutils 1.6.1, collections 2.1) then run the pruner you are
guaranteed a good resulting jar. We could even automate this process (eg
via a maven plugin), because Jakarta would be publishing a proper subset
of libraries that have been tested together.
If you gather together digester.jar + the *latest* releases of
beanutils, collections, etc. and run the pruner then you are *not*
guaranteed a good resulting jar. I would recommend that Jakarta never
publish such jars pre-built. But I see no problems with publishing the
pruner configuration files that allow people to build such custom jars
very easily.
In the case of Collections, which has no dependencies, things are even
easier. Rather than mess around with the ant buildfiles to generate a
bunch of different distribution targets, just create a set of pruner
config files which can be run against the complete collections jar
(automatically or manually) to generate the simpler kind of
distributions that people have been asking for. [except that propably
also want matching pruned javadoc, which I don't have an answer for at
the moment].
NB: I'm not heavily pushing the pruner concept. It just seemed an
interesting idea I thought might be worth raising for discussion..
Regards,
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [general] library management
Posted by Craig McClanahan <cr...@apache.org>.
Simon Kitching wrote:
>On Tue, 2004-05-04 at 04:53, Craig McClanahan wrote:
>
>
>>robert burrell donkin wrote:
>>
>>
>>
>>>On 26 Apr 2004, at 00:26, Simon Kitching wrote:
>>>
>>>
>>>
>>>>On Sun, 2004-04-25 at 15:00, Gary Gregory wrote:
>>>>
>>>>
>>>>
>>>>>Or:
>>>>>
>>>>>You release commons-all.jar with a pruner. This pruner would run just
>>>>>like base it's input on Clover output or manual input to create a
>>>>>smaller what-my-app-needs-out-of-commons.jar.
>>>>>
>>>>>
>>>>Like some other posters, I understand that people have some problems
>>>>with commons being composed of a dozen separate projects. However I
>>>>don't see the "one commons jar" approach as being feasable. I can't
>>>>imagine how releases would be synchronized, how unit tests would be
>>>>applied to the combined code, etc.
>>>>
>>>>
>>>there's nothing technically unfeasible about this. (gump does this and
>>>more :)
>>>
>>>one of the good rules we have is that each commons release should only
>>>depend on previously released code so it should just be a case of
>>>re-rolling the big jar and creating a new big-commons release each
>>>time any component was released.
>>>
>>>
>>>
>>If you want to play with exactly this concept, check out the build.xml
>>script in [combo]. It was designed to make a "pick the latest published
>>release of all the constituent libraries" policy very easy to accomplish
>>-- all you need is to update the CVS tag to pull as new releases are
>>made. In addition, someone who wanted a different set of versions could
>>easily override the CVS tags themselves.
>>
>>There are some wrinkles yet to be worked out, and I don't have time at
>>the moment to focus on it, but there's a starting point for someone
>>interested. If for no other reason, consolidated Javadocs for all of
>>commons (one of the outputs of this script) is very useful.
>>
>>
>>
>>>IMHO the main obstacle is organizational. people have enough
>>>difficulty with the current release process without adding to it.
>>>unless someone's willing to step forward and volunteer to manage the
>>>organizational side of the big commons jar (including release
>>>management) then it's not really worth wasting time on. (craig started
>>>something similar i'd guess over a year ago now but no one was really
>>>interested enough in the idea to push it forward.) i'd be happy to
>>>give a hand on the integration side but i don't cut jakarta releases
>>>any more (so i'm not willing to take this one on).
>>>
>>>
>>>
>>I think the mechanical aspects of building a combination JAR can be
>>dealt with. What I don't think we have in place at all is any sort of
>>unit or system tests to ensure that version X of library FOO works with
>>version Y of library BAR. Therefore, I'd be cautious about declaring
>>anything about a release of [combo] other than "here is a particular
>>combination of Commons libraries, conveniently packaged in one JAR for you."
>>
>>
>
>That's my concern. If collections releases 3.0, we can't just roll a new
>"commons-all" jar without testing that projects like beanutils or
>digester work with this new collections release. But how on earth do we
>do that? It seems to me that the unit tests for every project that
>depends upon the newly released project would need to be run - at the
>very least - in order to provide any guarantees that the "commons-all"
>release is stable. Does the "combo" project do this? If not, is it
>really the kind of release that Jakarta wants to put its name to?
>
>
>
I'd worry about this more if I believed that projects that ship multiple
commons JARs today (like Tomcat or Struts) do any interoperability
testing beyond making sure that their own uses of the underlying
libraries was successful. It doesn't seem unreasonable to offer a
convenience package whose contents is a specifically listed set of
commons JARs ... the functional result of using this JAR should match
the functional result of using all the individual JAR files themselves.
If we're still concerned about compatibility testing, another way to
look at [combo] is a tool for people to build their own custom
combinations, rather than something we use to ship a combined package
ourselves.
>Gary Gregory's "pruner" suggestion seems like a possible way forward to
>me. The pruner could takes a configuration file that specifies a set of
>classes. It would then run against a set of jars and extract the
>specified classes plus all their dependencies into a new jar file.
>Jakarta could publish a set of configuration files for various purposes.
>Running the tool against a single jar, or against a project's jars + its
>officially-supported dependencies would guarantee a functioning result.
>Running the tool against the complete set of latest releases of all
>projects might not result in a jar file that is valid (eg collections
>3.0 + classes that depend on collections-2.x). The one issue it does
>*not* address is generation of javadoc for the resulting jar. However
>such a tool should be simple for users to use. We *could* potentially
>automate its use to build a set of customised releases, but I would
>certainly not be in favour of this for the "cross-project" bundles, due
>to the inability to run the necessary unit tests as described above.
>
>What do people think about the "pruner" suggestion?
>
>
>
I can see how pruner might be useful for static dependencies (i.e.
import statements). What do you do about dynamic dependencies (for
example, a class loaded by ClassLoader.loadClass() whose name is only
known because it's in a configuration file)? And how does this do any
better at dealing with inter-library compatibility testing?
Compiles-clean isn't really good enough -- and Gump can already tell us
that.
>Regards,
>
>Simon
>
>
>
Craig
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org