You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Jon Stevens <jo...@latchkey.com> on 2000/11/25 02:30:52 UTC
[PROPOSAL] jar distribution system (was: Re: Jars in CVS ??)
on 11/24/2000 5:02 PM, "Conor MacNeill" <co...@ebinteractive.com.au> wrote:
>
> I do not support including jars for the optional components. By their nature
> optional components are not required to develop ant nor to use the core
> features of ant.
Uh, the optional stuff is part of Ant though. So, if someone wants to work
on an optional portion of Ant, why not make it easy for them to do so?
> As the number of optional tasks grows, trying to include
> and manage all the supporting jars and their versions will become a
> headache. The incidence of jar version conflict also will increase.
> Duplicating jars from a host of other projects doesn't really make sense to
> me.
The person who is responsible for the optional portion that is in Ant should
simply also be responsible for their .jar file. If their code doesn't get
updated, then it won't matter if the .jar file doesn't either.
> Currently, if I install a number of different projects, I end up with a
> number of versions of ant on my machine. We still get people wondering why
> ant doesn't compile their files with JDK 1.3 because of the ant bundled with
> Tomcat.
Right, so those people who are using that version of Ant to try to compile
there code should expect it to not work. It should be clearly documented
that the version of Ant with Tomcat is X version. Then people can come to
the Ant website and find out that Ant version X+1 handles compiling with JDK
1.3.
> I think the documentation should state the dependencies and provide a URL
> from which a user can download the appropriate components. I don't think
> that would be too difficult.
I agree that should be handled as well.
> I do agree that the issue is not black and white though. I agree versioning
> is an issue and ant itself is a culprit here. I just don't feel that
> including jars in CVS is the solution.
Ok.
> BTW, I agree that building Tomcat 4.0 was not easy. The real issue is
> getting JMX from Sun. It worked once for me, but when I tried to get a
> second copy, it stated that it couldn't export that software to my address.
> If the required bits were simple downloads, there would be no real issue.
Exactly. But, they aren't.
What I think would be cool to do is to have a system on jakarta.apache.org
where we allow people to download the .jar files.
I envision something like this:
<http://jakarta.apache.org/jars/allfiles.xml>
Then, in that file is a list of URL's to the .jar files...it could look
something like this (not well thought out):
<product name="Ant" permissionToDistribute="true">
<version="1.2" latest="true">
<file url="http://jakarta.apache.org/jars/files/ant-1.2.jar">
<depends product="Xerces" version="1.2.1"/>
</file>
</version>
<version="1.1" latest="false">
<file url="http://jakarta.apache.org/jars/files/ant-1.1.jar">
<depends product="Xerces" version="1.1"/>
</file>
</version>
</product>
<product name="Xerces" permissionToDistribute="true">
<version="1.2.1" latest="true">
<file url="http://jakarta.apache.org/jars/files/xerces-1.2.1.jar"/>
</version>
<version="1.1" latest="false">
<file url="http://jakarta.apache.org/jars/files/xerces-1.1.jar"/>
</version>
</product>
<product name="JMX" permissionToDistribute="false">
<permission>
Sorry, we do not have permission to distribute this product. You can
download it from here: <insert url> instead.
</permission>
</product>
Then, we could write a simple Ant Task that would grab the file and then
download the appropriate .jar files as needed.
<jpan url="http://jakarta.apache.org/jars/allfiles.xml" product="Ant"
version="latest">
It would then download ant-1.2.jar and xerces-1.2.1.jar.
We could put whatever .jar files we got permission to distribute up there.
If we didn't have permission, then we could have a permission error message
that would tell people where they could download the file from instead.
Note that Ant is a bad example above since there is a catch-22. You can't
download Ant with <jpan> (!CPAN) unless you already have Ant. :-)
So, who volunteers to build such a system? It really shouldn't be that hard
and I think it would be immensely useful.
-jon
--
twice of not very much is still a lot more than not very much
Re: [PROPOSAL] jar distribution system
Posted by Stefan Bodewig <bo...@apache.org>.
Conor MacNeill <co...@m64.com> wrote:
> I don't think the "available" approach is going to work. It relies
> on the classes being in the classpath, but version conflict is one
> of the problems cjan seeks to address. I think it is going to need a
> repository in which it tracks information about what products and
> (potentially conflicting) versions have been installed.
Well, the "available" approach could use a custom classloader that
contains those jars.
I'm still not sure that having a repository will work out that well,
you'd need two repositories at least, one for things installed for the
whole system and one for the things a user has installed in his/her
private workspace. I'm less concerned with companies - here the system
administrators should be willing to install everything the developers
need - but with universities and similiar organizations.
And then you'd need a way to tell CJAN to add something to the
repository that has allready been there before CJAN came into the
game.
> 1. To what type of user is cjan aimed? Is it
> a) the CVS developer,
> b) the source distribution user,
> c) the binary distribution user
guess it's all of them - unless you want to end up holding required
JARs within CVS or ship them with distributions.
> 2. How pervasive/invasive should cjan become? Can it be a
> requirement for a product to launch?
Too launch or to build? I think Jose Alberto's distinction is
important. But I can imagine you'd require it for both, just not from
the start, let CJAN grow.
> 3. What are the goals - simple automated downloads of support jars
> for a single product - no attempt to worry about managing the
> mismatch of versions across a system. Or should it be a package
> management system which reports, or even tries to resolve, version
> conflicts?
My take would be: all of them, one after the other.
Stefan
RE: [PROPOSAL] jar distribution system
Posted by Conor MacNeill <co...@m64.com>.
> From: Stefan Bodewig [mailto:bodewig@apache.org]
> Sent: Thursday, 30 November 2000 21:47
> To: ant-dev@jakarta.apache.org
> Subject: Re: [PROPOSAL] jar distribution system
>
>
> Jon Stevens <jo...@latchkey.com> wrote:
>
> > I have placed the start of cjan.xml up on the jakarta.apache.org
> > site
> >
> > <http://jakarta.apache.org/cjan/cjan.xml>
> >
>
> Just a quick thought:
>
> CJAN (the client side application) needs some way to know whether a
> given dependency is fulfilled on the local system or not. I see two
> options here:
>
> (1) keep track on all JARs that have ever been loaded and installed in
> a CJAN registry.
>
> This forces the user to install every component which might be
> available to CJAN through CJAN - I don't think you can expect this on
> a multi user system, even for single use boxes this seems too strong
> therefore I'd prefer
>
I wonder how extensive this cjan system might be. Should it be a system
for simply downloading the support jars needed for a particular product.
That doesn't really manage the problem, though. Where does it put these
jars? How does it "connect" the product in question to the jars. Can
this be standardized or is it the user's responsibility?
If I think beyond simply downloading the support jars to some form of
package/product management, I think it needs to become more "invasive".
Consider the problem of trying to have Tomcat 4.0 and Ant on a system
(This is just an example). One requires Jaxp 1.0 and the other Jaxp 1.1.
(I have run into some problems using ant with Jaxp1.1 - sealing
exception - whatever that is). The approach to this sort of problem
today is to have a whole load of *HOME environment variables which
"locate" the approriate jars. The build scripts then copy the
approrpiate jars into the lib area where the product's launcher script
dynamically builds the appropriate classpath.
A simplistic approach would be to standardize this system of lib
directories across products and have cjan copy the necessary support
jars into the product's lib directory. A central repository on the local
system of product jars would need to be used, I guess.
Such a simplistic approach seems rather poor, however. Why duplicate
support jars in each product's lib directory? I can concieve of a cjan
product-launcher which would manage a product's classpath. Some form of
classloader which dynamically constructs the classpath using the
installed jars that satisfy the correct product dependencies, and then
launches the Main class? Is that going to be feasible?
I don't think the "available" approach is going to work. It relies on
the classes being in the classpath, but version conflict is one of the
problems cjan seeks to address. I think it is going to need a repository
in which it tracks information about what products and (potentially
conflicting) versions have been installed.
I have some questions which I think it may be interesting to think
about.
1. To what type of user is cjan aimed? Is it a) the CVS developer, b)
the source distribution user, c) the binary distribution user or some
combination of these. I think it make a difference in terms of what we
want cjan to do and what we include in distributions?
2. How pervasive/invasive should cjan become? Can it be a requirement
for a product to launch?
3. What are the goals - simple automated downloads of support jars for a
single product - no attempt to worry about managing the mismatch of
versions across a system. Or should it be a package management system
which reports, or even tries to resolve, version conflicts?
Maybe I am "overthinking" the whole issue - thoughts?
Conor
Re: [PROPOSAL] jar distribution system
Posted by Stefan Bodewig <bo...@apache.org>.
Jon Stevens <jo...@latchkey.com> wrote:
> I have placed the start of cjan.xml up on the jakarta.apache.org
> site
>
> <http://jakarta.apache.org/cjan/cjan.xml>
>
Just a quick thought:
CJAN (the client side application) needs some way to know whether a
given dependency is fulfilled on the local system or not. I see two
options here:
(1) keep track on all JARs that have ever been loaded and installed in
a CJAN registry.
This forces the user to install every component which might be
available to CJAN through CJAN - I don't think you can expect this on
a multi user system, even for single use boxes this seems too strong
therefore I'd prefer
(2) Add some <available> like functionality. The section describing a
product as well as a service (I like Conor's concept) should hold
something like
<product name="Ant" permissionToDistribute="true">
<version label="1.2" latest="true">
<provides classname="org.apache.tools.ant.types.FileSet" />
<depends service="jaxp" version="1.0" required="true"/>
...
</version>
<version label="1.1" latest="false">
<provides classname="org.apache.tools.ant.Main" />
...
</version>
</product>
<service name="jaxp">
<version label="1.0" latest="false">
<provides classname="javax.xml.parsers.SAXParserFactory" />
<provider name="xerces" version="1.2"/>
<provider name="jaxp_reference" version="1.0"/>
</version>
</service>
Stefan
Re: [PROPOSAL] jar distribution system
Posted by Jon Stevens <jo...@latchkey.com>.
I have placed the start of cjan.xml up on the jakarta.apache.org site
<http://jakarta.apache.org/cjan/cjan.xml>
It of course isn't complete or final in any way, but it is a start. Now,
someone want to speak up and start to write an Ant Task to deal with it?
I will get you access to the box if you don't already have it and you can
edit the cjan.xml file and put files up in the cjan/files directory.
It is up to all of you to help make this a reality. I'm not going to be the
only one doing all this work.
thanks,
-jon
--
twice of not very much is still a lot more than not very much
Re: [PROPOSAL] jar distribution system (was: Re: Jars in CVS ??)
Posted by Jon Stevens <jo...@latchkey.com>.
on 11/25/2000 7:25 AM, "Conor MacNeill" <co...@m64.com> wrote:
> OK, I like this idea. Rather than duplicate jars into every project, it
> allows each project to go and fetch the jars where necessary, from where
> they live. More like a web.
Well, more like CPAN.
> We wouldn't need to redistribute anyway, since such a task could
> conceivably go to the actual home of each jar. For example, why
> duplicate junit.jar on jakarta.apache.org when it is available from
> junit.org. I think that where a jar is not available as a "direct"
> download, such as JMX, taht is whenwe would need to display the
> "permission" message.
The issue is that you simply need just the .jar. Nearly all of the sites I
know of don't provide a link to just the .jar. You would then need to encode
information to be able to extract the .jar from a .zip or .tar.gz and then
deal with it. That could be a serious pain in the ass.
> But ant could bootstrap in the components needed for the optional tasks,
> such as junit.
Right, but not Ant itself. :-)
> I'm certainly interested. I see this as a sort of distributed package
> manager. Rather than an allfiles.xml, I would see each "product" having
> its own file. When the <jpan> task processes that description, it would
> download the files of the other products upon which it depends.
Right, but then you would have to hit the server multiple times for all of
the dependencies. I'm trying to lower that amount as this could become HUGE
and you don't want scalability issues. :-)
One thing we could also do is .gz the allfiles.xml so that it is even
compressed. Text compresses very nicely. :-) We could even add a caching
routine so that it would simply do a HEAD request on the HTTP server to find
out the last modified version of the file and keep a cache of the file
locally instead of having to download it each time.
> jpan may have to be interactive. Just kicking around a few ideas :-)
Exactly. :-)
-jon
--
twice of not very much is still a lot more than not very much
RE: [PROPOSAL] jar distribution system (was: Re: Jars in CVS ??)
Posted by Conor MacNeill <co...@m64.com>.
Jon,
> From: Jon Stevens [mailto:jon@latchkey.com]
> Sent: Saturday, 25 November 2000 12:31
> To: ant-dev@jakarta.apache.org
> Subject: [PROPOSAL] jar distribution system (was: Re: Jars in CVS ??)
>
>
> I envision something like this:
>
> <http://jakarta.apache.org/jars/allfiles.xml>
>
> Then, in that file is a list of URL's to the .jar files...it
> could look
> something like this (not well thought out):
>
> <product name="Ant" permissionToDistribute="true">
> <version="1.2" latest="true">
> <file url="http://jakarta.apache.org/jars/files/ant-1.2.jar">
> <depends product="Xerces" version="1.2.1"/>
> </file>
> </version>
> <version="1.1" latest="false">
> <file url="http://jakarta.apache.org/jars/files/ant-1.1.jar">
> <depends product="Xerces" version="1.1"/>
> </file>
> </version>
> </product>
>
OK, I like this idea. Rather than duplicate jars into every project, it
allows each project to go and fetch the jars where necessary, from where
they live. More like a web.
> Then, we could write a simple Ant Task that would grab the
> file and then
> download the appropriate .jar files as needed.
>
> <jpan url="http://jakarta.apache.org/jars/allfiles.xml" product="Ant"
> version="latest">
>
> It would then download ant-1.2.jar and xerces-1.2.1.jar.
>
> We could put whatever .jar files we got permission to
> distribute up there.
> If we didn't have permission, then we could have a permission
> error message
> that would tell people where they could download the file
> from instead.
We wouldn't need to redistribute anyway, since such a task could
conceivably go to the actual home of each jar. For example, why
duplicate junit.jar on jakarta.apache.org when it is available from
junit.org. I think that where a jar is not available as a "direct"
download, such as JMX, taht is whenwe would need to display the
"permission" message.
>
> Note that Ant is a bad example above since there is a
> catch-22. You can't
> download Ant with <jpan> (!CPAN) unless you already have Ant. :-)
>
But ant could bootstrap in the components needed for the optional tasks,
such as junit.
> So, who volunteers to build such a system? It really
> shouldn't be that hard
> and I think it would be immensely useful.
>
I'm certainly interested. I see this as a sort of distributed package
manager. Rather than an allfiles.xml, I would see each "product" having
its own file. When the <jpan> task processes that description, it would
download the files of the other products upon which it depends.
<product name="Ant">
<version="1.2" latest="true">
<file url="http://jakarta.apache.org/jars/files/ant-1.2.jar"/>
<require product="jaxp" version="1.0"
jpanurl="http://jakarta.apache.org/products/jaxp.xml"/>
<optional product="junit" version="3.2"
jpanurl="http://jakarta.apache.org/products/junit.xml"/>
</version>
</product>
<product name="jaxp">
<version="1.0" latest="true">
<choice>
<product name="jaxp_reference" version="1.01" jpanurl="..."/>
<product name="Xerces" version="1.2.1" jpanurl="..."/>
</choice>
</version>
</product>
jpan may have to be interactive. Just kicking around a few ideas :-)
Conor