You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Sylvain Wallez <sy...@apache.org> on 2005/01/14 11:06:09 UTC
Importing libraries from ibiblio
Hi all,
I just found (via TSS) Ivy [1], a BSD-licensed Ant task to download
dependencies from artifact repositories, including the Maven repo at
ibiblio.
I've already seen a number of Ant extensions that aim to provide this
feature, but that one looked particuarily interesting to me as this is
the first one where dependencies are not describes in the Ant file itself.
What that means is that we can create the list of jars to download from
our jars.xml or the gump descriptor, and have them automatically
downloaded from the ibiblio repo. Very few modifications on the current
build are needed.
That could allow to really reduce the size of both our distro and our
SVN repository.
WDYT?
Sylvain
[1] http://www.jayasoft.fr/org/modules/ivy/
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: Importing libraries from ibiblio
Posted by Joerg Heinicke <jo...@gmx.de>.
On 14.01.2005 23:51, Sylvain Wallez wrote:
>>> What that means is that we can create the list of jars to download
>>> from our jars.xml or the gump descriptor, and have them automatically
>>> downloaded from the ibiblio repo. Very few modifications on the
>>> current build are needed.
>>>
>>> That could allow to really reduce the size of both our distro and our
>>> SVN repository.
>>>
>>> WDYT?
>>
>> But have in mind that the download computer is not always the install
>> computer. Or the other way around: the install computer(s) has/have
>> not necessarily access to the internet. I don't want to download all
>> the dependent libraries one by one.
>
> My idea about this wasn't deployment (although this could be useful with
> a repository on the local network) but svn and distros.
Ah, sorry, my "install computer" was not very clear. My idea was neither
about deployment, but the machine, where you develop and build your
Cocoon. I meant, if you have no access to the internet on this machine
and have to fetch the libraries from the repository on build time ...
> With dependencies fetched from a repository, we can provide 3 different
> distros:
> - source, with no dependent libraries included. This is basically a
> snapshot of the svn tree
> - small binary, with no dependent libraries included. It's up to the
> user to download the libraries that are needed for the blocks he chooses
> to use (can be automated with blocks.properties), if they aren't already
> present in his local cache.
> - full binary, with all dependent libraries included.
But providing the without-dependencies-version additionally, not
substitutingly, dispels my fears.
Joerg
Re: Importing libraries from ibiblio
Posted by Vadim Gritsenko <va...@reverycodes.com>.
Sylvain Wallez wrote:
> With dependencies fetched from a repository, we can provide 3 different
> distros:
> - source, with no dependent libraries included. This is basically a
> snapshot of the svn tree
- full source (jdk1.3 for 2.1 and jdk1.4 for 2.2)
> - small binary, with no dependent libraries included. It's up to the
> user to download the libraries that are needed for the blocks he chooses
> to use (can be automated with blocks.properties), if they aren't already
> present in his local cache.
> - full binary, with all dependent libraries included.
(jdk1.3 for 2.1 and jdk1.4 for 2.2)
+1
Vadim
> There are a lot of projects that do this, e.g. Spring [1]. This reduces
> download size, especially when there is a local repository used by a
> number of different projects.
>
> Sylvain
>
> [1] http://sourceforge.net/project/showfiles.php?group_id=73357
Re: Importing libraries from ibiblio
Posted by Torsten Curdt <tc...@apache.org>.
>>> WDYT?
>>
>>
>>
>> But have in mind that the download computer is not always the install
>> computer. Or the other way around: the install computer(s) has/have
>> not necessarily access to the internet. I don't want to download all
>> the dependent libraries one by one.
>
>
>
> My idea about this wasn't deployment (although this could be useful with
> a repository on the local network) but svn and distros.
>
> With dependencies fetched from a repository, we can provide 3 different
> distros:
> - source, with no dependent libraries included. This is basically a
> snapshot of the svn tree
> - small binary, with no dependent libraries included. It's up to the
> user to download the libraries that are needed for the blocks he chooses
> to use (can be automated with blocks.properties), if they aren't already
> present in his local cache.
> - full binary, with all dependent libraries included.
>
> There are a lot of projects that do this, e.g. Spring [1]. This reduces
> download size, especially when there is a local repository used by a
> number of different projects.
I think this is a great idea! big +1
We could also directly talk to the
ibiblio folks if things really don't
work out. I am sure we can work this
out.
cheers
--
Torsten
Re: Importing libraries from ibiblio
Posted by Sylvain Wallez <sy...@apache.org>.
Joerg Heinicke wrote:
> On 14.01.2005 11:06, Sylvain Wallez wrote:
>
>> What that means is that we can create the list of jars to download
>> from our jars.xml or the gump descriptor, and have them automatically
>> downloaded from the ibiblio repo. Very few modifications on the
>> current build are needed.
>>
>> That could allow to really reduce the size of both our distro and our
>> SVN repository.
>>
>> WDYT?
>
>
> But have in mind that the download computer is not always the install
> computer. Or the other way around: the install computer(s) has/have
> not necessarily access to the internet. I don't want to download all
> the dependent libraries one by one.
My idea about this wasn't deployment (although this could be useful with
a repository on the local network) but svn and distros.
With dependencies fetched from a repository, we can provide 3 different
distros:
- source, with no dependent libraries included. This is basically a
snapshot of the svn tree
- small binary, with no dependent libraries included. It's up to the
user to download the libraries that are needed for the blocks he chooses
to use (can be automated with blocks.properties), if they aren't already
present in his local cache.
- full binary, with all dependent libraries included.
There are a lot of projects that do this, e.g. Spring [1]. This reduces
download size, especially when there is a local repository used by a
number of different projects.
Sylvain
[1] http://sourceforge.net/project/showfiles.php?group_id=73357
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: Importing libraries from ibiblio
Posted by Ralph Goers <Ra...@dslextreme.com>.
Joerg Heinicke said:
>
> But have in mind that the download computer is not always the install
> computer. Or the other way around: the install computer(s) has/have not
> necessarily access to the internet. I don't want to download all the
> dependent libraries one by one.
>
> Joerg
The install computer doesn't necessarily need access to the internet. It
needs access to a repository. You can actually run your build in such a
way that it populates a local repository that all other computers then use
as a distribution point.
Ralph
RE: Importing libraries from ibiblio
Posted by Mark Lundquist <ml...@wrinkledog.com>.
> From: Joerg Heinicke [mailto:joerg.heinicke@gmx.de]
>
>
> But have in mind that the download computer is not always the install
> computer. Or the other way around: the install computer(s) has/have not
> necessarily access to the internet. I don't want to download all the
> dependent libraries one by one.
Good point.
So maybe there would be an build.properties option to locate the local
repository cache within the Cocoon distribution directory structure? Then
you can build (which will fetch all the dependent jars), and you can re-zip
the whole thing and redistribute it.
-ml-
Re: Importing libraries from ibiblio
Posted by Joerg Heinicke <jo...@gmx.de>.
On 14.01.2005 11:06, Sylvain Wallez wrote:
> What that means is that we can create the list of jars to download from
> our jars.xml or the gump descriptor, and have them automatically
> downloaded from the ibiblio repo. Very few modifications on the current
> build are needed.
>
> That could allow to really reduce the size of both our distro and our
> SVN repository.
>
> WDYT?
But have in mind that the download computer is not always the install
computer. Or the other way around: the install computer(s) has/have not
necessarily access to the internet. I don't want to download all the
dependent libraries one by one.
Joerg
Re: Importing libraries from ibiblio
Posted by Sylvain Wallez <sy...@apache.org>.
Antonio Gallardo wrote:
>>>Currently our update procedure is a lot easier. I am not against having
>>>repositories, but this will add extra work if we want to have up to date
>>>all our libraries. As we have now. :-D
>>>
>>>
For some strange reason, I havent received your initial answer, but I've
read it in the archives.
If you look at Ivy, you will see that it is very unintrusive. It "just"
allows you to download artifacts from a repository in a directory that
you specify. That's all. From there on, its just standard Ant, where you
declare your classpath as usual. So it doesn't change much compared to
what we have today.
Furthermore, you can speficy a list of repositories (either local or
remote) that are searched in order for artifacts. That means you can
have your local repo (i.e. local cache), the Cocoon repo, then Apache
repo and then ibiblio. A lot of places where to store the needed
libraries and ensure they don't get lost.
>>>Another problem is with what version are compiled the jars. We need to
>>>make sure that for 2.1.x branch they need to run in java 1.3 and 1.4 for
>>>2.2.0 branch. And this is not the case with some jars you find in the
>>>repositories.
>>>
>>>
That is a valid point. However, we can store jdk-specific libraries in
our own repo whenever that situation arises.
>>>I am not against repositories, I am just let know the pitfalls that I
>>>met using them.
>>>
>>>
>>I think we've discussed this before, but it should be possible for Cocoon
>>to have its own repository with only the jars in it that need to be there.
>>It can use ibiblio as a secondary repository for the ones that can't be
>>located in Cocoon's.
>>
>>I'm not sure about this ant task, but one nice thing about maven is that
>>the jars end up in your local repository, so if you get a new version of
>>Cocoon only jars that have changed will be downloaded.
>>
>>
>
>Are we planning to release binaries again?
>
>
The include feature in 2.2 can allow this, because adding/removing a
block will just be a matter or moving around a jar and a xconf, rather
than using xpatch.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: Importing libraries from ibiblio
Posted by Ralph Goers <Ra...@dslextreme.com>.
Antonio Gallardo said:
> On Vie, 14 de Enero de 2005, 9:56, Ralph Goers dijo:
>> I think we've discussed this before, but it should be possible for
>> Cocoon
>> to have its own repository with only the jars in it that need to be
>> there.
>> It can use ibiblio as a secondary repository for the ones that can't be
>> located in Cocoon's.
>>
>> I'm not sure about this ant task, but one nice thing about maven is that
>> the jars end up in your local repository, so if you get a new version of
>> Cocoon only jars that have changed will be downloaded.
>
> Are we planning to release binaries again?
>
Well, eventually, but that isn't what I meant. Sorry I wasn't clear. In
the case you mentioned we might need a version of a dependent jar to be
compiled with JDK 1.3 for our customers who require it. In that case, the
jar would be stored in Cocoon's repository as it may not be explicitly
supported by that project. So our user's would download it from our
repository. However, if the end user is compiling Cocoon with JDK 1.4,
the build might be able to use a different dependency list and then
download the 1.4 version from ibiblio.
Ralph
Re: Importing libraries from ibiblio
Posted by Antonio Gallardo <ag...@agssa.net>.
On Vie, 14 de Enero de 2005, 9:56, Ralph Goers dijo:
>
>
>
> Antonio Gallardo said:
>> Hi Sylvain:
>>
> <snip>
>> All this was done because I wanted to compile and test apache jcs with
>> this util.concurrent version before we released cocoon 2.1.5! (jcs use
>> maven).
>
> Why didn't you just put the util.concurrent version in your local
> repository then?
Because we want to be good neighbors or not? ;-) JCS needed it too. This
is why I ended in a maven mail-list for few days. But, this is OT here:
If I am going to update cocoon jars, I cannot only update my local
repository. :-(
>
>>
>> Currently our update procedure is a lot easier. I am not against having
>> repositories, but this will add extra work if we want to have up to date
>> all our libraries. As we have now. :-D
>>
>> Another problem is with what version are compiled the jars. We need to
>> make sure that for 2.1.x branch they need to run in java 1.3 and 1.4 for
>> 2.2.0 branch. And this is not the case with some jars you find in the
>> repositories.
>>
>> I am not against repositories, I am just let know the pitfalls that I
>> met
>> using them.
>
> I think we've discussed this before, but it should be possible for Cocoon
> to have its own repository with only the jars in it that need to be there.
> It can use ibiblio as a secondary repository for the ones that can't be
> located in Cocoon's.
>
> I'm not sure about this ant task, but one nice thing about maven is that
> the jars end up in your local repository, so if you get a new version of
> Cocoon only jars that have changed will be downloaded.
Are we planning to release binaries again?
Best Regards,
Antonio Gallardo
Re: Importing libraries from ibiblio
Posted by Ralph Goers <Ra...@dslextreme.com>.
Antonio Gallardo said:
> Hi Sylvain:
>
<snip>
> All this was done because I wanted to compile and test apache jcs with
> this util.concurrent version before we released cocoon 2.1.5! (jcs use
> maven).
Why didn't you just put the util.concurrent version in your local
repository then?
>
> Currently our update procedure is a lot easier. I am not against having
> repositories, but this will add extra work if we want to have up to date
> all our libraries. As we have now. :-D
>
> Another problem is with what version are compiled the jars. We need to
> make sure that for 2.1.x branch they need to run in java 1.3 and 1.4 for
> 2.2.0 branch. And this is not the case with some jars you find in the
> repositories.
>
> I am not against repositories, I am just let know the pitfalls that I met
> using them.
I think we've discussed this before, but it should be possible for Cocoon
to have its own repository with only the jars in it that need to be there.
It can use ibiblio as a secondary repository for the ones that can't be
located in Cocoon's.
I'm not sure about this ant task, but one nice thing about maven is that
the jars end up in your local repository, so if you get a new version of
Cocoon only jars that have changed will be downloaded.
Ralph
Re: Importing libraries from ibiblio
Posted by Torsten Curdt <tc...@apache.org>.
> I had really bad experiences with this repositories. They are often
> outdated with current releases. ie: few days ago, in the repositories i
> was not able to find log4j version 1.2.9. I am not sure if now the 1.2.9
> is there. I bet is still missing.... Until some charitative soul will take
> care of that. Is supposed that Apache jars are automatically in the repos,
> but this is not true currently. The situation is worse with non-apache
> libraries.
Well, I also needed to get the eclipse compiler updated (javaflow
uses maven). And I've put the library into jira. To be honest it took
a few days. ...I got some things wrong and stuff. But it wasn't *that*
bad. I really like the repository idea.
BTW: there is an apache repository where you can copy apache jars
into. It will be synched automagically!
> Another problem is with what version are compiled the jars. We need to
> make sure that for 2.1.x branch they need to run in java 1.3 and 1.4 for
> 2.2.0 branch. And this is not the case with some jars you find in the
> repositories.
Well, this is of course a good point :-/
cheers
--
Torsten
Re: Importing libraries from ibiblio
Posted by Antonio Gallardo <ag...@agssa.net>.
Hi Bertrand:
Here is the maven output few seconds ago showing that log4j 1.2.9 is still
not there:
****************************************************************
~/bin/maven/bin/maven
__ __
| \/ |__ _Apache__ ___
| |\/| / _` \ V / -_) ' \ ~ intelligent projects ~
|_| |_\__,_|\_/\___|_||_| v. 1.0.2
Intentando descargar log4j-1.2.9.jar.
¡ATENCION!: ¡Imposible descargar log4j-1.2.9.jar!
¡El proceso no puede seguir! Falta la siguiente dependencia:
log4j-1.2.9.jar
Total time: 8 seconds
Finished at: Fri Jan 14 10:12:28 CST 2005
***************************************************************
We are using this version since 2004-12-08
[run: svn log lib/core/log4j-1.2.9.jar].
This time I was not a good neighboor.
Best Regards,
Antonio Gallardo.
On Vie, 14 de Enero de 2005, 10:00, Bertrand Delacretaz dijo:
> Le 14 janv. 05, à 16:38, Antonio Gallardo a écrit :
>
>> ...I had really bad experiences with this repositories. They are often
>> outdated with current releases. ie: few days ago, in the repositories i
>> was not able to find log4j version 1.2.9. I am not sure if now the
>> 1.2.9
>> is there. I bet is still missing....
>
> Excuse my ignorance, but aren't jars supposed to be available with
> unique and permanent URLs on repositories like ibiblio?
I theory yes. but....... things happens.
> If that's the case, and if there's a quick way for some of us to add
> missing jars there, it shouldn't be a problem (but feel free to point
> me to resources where I can learn more about how this works ;-)
Well, I learned from maven:
http://maven.apache.org/repository-upload.html
> I very much like Sylvain's proposal, although I haven't used ibiblio
> enough to be aware of the potential problems.
Please note, I am not against the repos. I am just trying to share with
the community what else needs to be done. I think the idea is good after
all. :-D
Best Regards,
Antonio Gallardo
Re: Importing libraries from ibiblio
Posted by Bertrand Delacretaz <bd...@apache.org>.
Le 14 janv. 05, à 16:38, Antonio Gallardo a écrit :
> ...I had really bad experiences with this repositories. They are often
> outdated with current releases. ie: few days ago, in the repositories i
> was not able to find log4j version 1.2.9. I am not sure if now the
> 1.2.9
> is there. I bet is still missing....
Excuse my ignorance, but aren't jars supposed to be available with
unique and permanent URLs on repositories like ibiblio?
If that's the case, and if there's a quick way for some of us to add
missing jars there, it shouldn't be a problem (but feel free to point
me to resources where I can learn more about how this works ;-)
I very much like Sylvain's proposal, although I haven't used ibiblio
enough to be aware of the potential problems.
-Bertrand
Re: Importing libraries from ibiblio
Posted by Antonio Gallardo <ag...@agssa.net>.
Hi Sylvain:
I had really bad experiences with this repositories. They are often
outdated with current releases. ie: few days ago, in the repositories i
was not able to find log4j version 1.2.9. I am not sure if now the 1.2.9
is there. I bet is still missing.... Until some charitative soul will take
care of that. Is supposed that Apache jars are automatically in the repos,
but this is not true currently. The situation is worse with non-apache
libraries.
An example of the extra works and what means to work with the nice
repositores:
For util.concurrent 1.3.4, I needed to create a maven bundle and publish
it in the web, then create an issue in maven jira. Then subscribe on the
maven mail-list and push on the mail list to include it. Mein God!
http://www.mail-archive.com/dev@maven.apache.org/msg14465.html
All this was done because I wanted to compile and test apache jcs with
this util.concurrent version before we released cocoon 2.1.5! (jcs use
maven).
Currently our update procedure is a lot easier. I am not against having
repositories, but this will add extra work if we want to have up to date
all our libraries. As we have now. :-D
Another problem is with what version are compiled the jars. We need to
make sure that for 2.1.x branch they need to run in java 1.3 and 1.4 for
2.2.0 branch. And this is not the case with some jars you find in the
repositories.
I am not against repositories, I am just let know the pitfalls that I met
using them.
Best Regards,
Antonio Gallardo.
On Vie, 14 de Enero de 2005, 4:06, Sylvain Wallez dijo:
> Hi all,
>
> I just found (via TSS) Ivy [1], a BSD-licensed Ant task to download
> dependencies from artifact repositories, including the Maven repo at
> ibiblio.
>
> I've already seen a number of Ant extensions that aim to provide this
> feature, but that one looked particuarily interesting to me as this is
> the first one where dependencies are not describes in the Ant file itself.
>
> What that means is that we can create the list of jars to download from
> our jars.xml or the gump descriptor, and have them automatically
> downloaded from the ibiblio repo. Very few modifications on the current
> build are needed.
>
> That could allow to really reduce the size of both our distro and our
> SVN repository.
>
> WDYT?
>
> Sylvain
>
> [1] http://www.jayasoft.fr/org/modules/ivy/
>
> --
> Sylvain Wallez Anyware Technologies
> http://www.apache.org/~sylvain http://www.anyware-tech.com
> { XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
>
Re: Importing libraries from ibiblio
Posted by Ralph Goers <Ra...@dslextreme.com>.
Sylvain Wallez wrote:
> Hi all,
>
> I just found (via TSS) Ivy [1], a BSD-licensed Ant task to download
> dependencies from artifact repositories, including the Maven repo at
> ibiblio.
>
> I've already seen a number of Ant extensions that aim to provide this
> feature, but that one looked particuarily interesting to me as this is
> the first one where dependencies are not describes in the Ant file
> itself.
>
> What that means is that we can create the list of jars to download
> from our jars.xml or the gump descriptor, and have them automatically
> downloaded from the ibiblio repo. Very few modifications on the
> current build are needed.
>
> That could allow to really reduce the size of both our distro and our
> SVN repository.
>
> WDYT?
>
> Sylvain
+1
Re: Importing libraries from ibiblio
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Sylvain Wallez wrote:
> Nicola Ken Barozzi wrote:
...
>> +1 to using Maven Wagon to download the artifacts
>
> Didn't knew about this. Interesting, but is it visible somewhere else
> than in the CVS [1]?
No.
We could prod them though :-)
> I couldn't find it on the maven site. Also, is
> there an Ant task using Wagon?
There was, they deleted it, now should add it back IIUC.
...
>> If we want to use the Maven repository system, it's better to stick
>> with the Maven implementation, that is supported. ATM it doesn't have
>> an Ant task anymore, but over at repository@apache.org I have
>> understood that they agreed to add it back.
>
> I'm all for a Maven-supported solution, but do we want to wait until Ant
> and Maven agree on something common when we have today a lightweight
> tool that can do the job?
There is already agreement (@see repository@apache.org), but we don't
need it, as Wagon is all we need ATM... I think we should help Maven do
a point release and use that.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: Importing libraries from ibiblio
Posted by Sylvain Wallez <sy...@apache.org>.
Nicola Ken Barozzi wrote:
> Sylvain Wallez wrote:
> ...
>
>> What that means is that we can create the list of jars to download
>> from our jars.xml or the gump descriptor, and have them automatically
>> downloaded from the ibiblio repo. Very few modifications on the
>> current build are needed.
>
>
> +1 to use a jar repository
>
> +1 to use a Maven repository, so that users can reuse their local
> Maven-downloaded jars reducing bandwidth usage and download times
>
> +1 to using Maven Wagon to download the artifacts
Didn't knew about this. Interesting, but is it visible somewhere else
than in the CVS [1]? I couldn't find it on the maven site. Also, is
there an Ant task using Wagon?
> -1 to using any download mechanism different from Maven Wagon or
> pure http get
Well, then anything is compliant, as it always go down to a pure http
get ;-)
(just joking)
> If we want to use the Maven repository system, it's better to stick
> with the Maven implementation, that is supported. ATM it doesn't have
> an Ant task anymore, but over at repository@apache.org I have
> understood that they agreed to add it back.
I'm all for a Maven-supported solution, but do we want to wait until Ant
and Maven agree on something common when we have today a lightweight
tool that can do the job?
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: Importing libraries from ibiblio
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Sylvain Wallez wrote:
...
> What that means is that we can create the list of jars to download from
> our jars.xml or the gump descriptor, and have them automatically
> downloaded from the ibiblio repo. Very few modifications on the current
> build are needed.
+1 to use a jar repository
+1 to use a Maven repository, so that users can reuse their local
Maven-downloaded jars reducing bandwidth usage and download times
+1 to using Maven Wagon to download the artifacts
-1 to using any download mechanism different from Maven Wagon or
pure http get
If we want to use the Maven repository system, it's better to stick with
the Maven implementation, that is supported. ATM it doesn't have an Ant
task anymore, but over at repository@apache.org I have understood that
they agreed to add it back.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------