You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@forrest.apache.org by Ross Gardler <rg...@apache.org> on 2007/02/07 10:51:43 UTC
[PROPOSAL] Switch dependency managment to IVY
It appears my CCLA is now recorded so I can start moving some stuff from
an internal project to Forrest.
Recently I've been mentioning IVY [1] for dependency management.
Others have mentioned Maven2 [2]. From the little discussion we have had
so far I get the feeling that we are happy to go with Ivy.
The advantages include:
- central repository (eventually, see below)
- dependencies need not be in SVN (eventually, see below)
- transitive dependency management (i.e. we say we need X, if X depends
on Y it will come automatically)
- conflict management between dependencies (easier upgrading of
dependencies)
- much easier integration testing. We can have different build
configurations for release, development and test, release using fully
tested jars, development using latest releases and/or milestones
- easier continuous integration testing
- full dependency reports (i.e. better developer docs)
The disadvantage is that, as far as I am aware, no developers here other
than myself know ivy (and I'm still learning). However, Ivy docs are
great [2]
Here is my plan:
1 - I create a Forrest only repo in SVN (as I have started for
Forrest2). although wherever possible we will use the IVY repo and the
Maven2 repo
2 - I modify our build files to use IVY (I have a whole set of generic
build files that are intended to be extended by project specific build
files. I plan to give them to the IVY folk if they think it they are a
good idea, if not we'll take what we need from them)
3 - We make sure it works
4 - We move the entries in our repo to the IVY repo (I need to enter
into a dialogue with IVY devs to see what is happening with regards an
official Apache repo)
The end result will be a build system that uses three repos:
The forrest one for any jars we have customised but are not yet
available in the main project releases (i.e. we have something in the
patch queue)
The official IVY repo (whatever that turns out to be)
The official ASF Maven repo
So, any questions? Objections?
If nobody raises an objection in the next seven days (I'm busy until
then) then I'll go ahead. Otherwise we can discuss and I'll call a vote
if necessary next week.
Ross
Ross
[1] http://incubator.apache.org/ivy/
[2] http://maven.apache.org/
[3] http://incubator.apache.org/ivy/doc.html
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
> So, any questions? Objections?
>
> If nobody raises an objection in the next seven days (I'm busy until
> then) then I'll go ahead. Otherwise we can discuss and I'll call a vote
> if necessary next week.
Wow, thank you. That is such a professional proposal
and way of doing things. The least fuss approach.
I know a little Ivy and happy to learn more.
-David
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Ross Gardler wrote:
> David Crossley wrote:
...
>> I gather that we can have Ivy descriptors in our local
>> repository, but no jars, and then point to other repositories
>> for the jars, e.g. maven repo.
>
> We don't even need the descriptors. We just need the ivy.xml file in our
> trunk. This tells Ivy what Forrest depends on. It goes off to the
> defined repositories (defined in tools/ivy/ivyconf.xml) and finds the
> relevant ivy.xml files for that dependency. This identifies any jars
> that are required by that dependency, which allows Ivy to go looking for
> them, and so on.
>
> So, we are aiming at only needing to maintain Forrests ivy.xml file here
> in Forrest.
I missed your point here. My statements above are still relevant but I
now suspect your point was that ivy repositories need not contain the
actual jar file, they can, in fact, just contain ivy.xml files that
point to a download location for the jar.
This is correct.
However, I chose not to go down that route because I've set ivy up so
that if the jar is available in any of the ivy repos or some of the
common maven repos it will be pulled from there and we need not define
an ivy.xml.
As I mentioned in my previous mail I thought I had set it up to use the
Cocoon maven repo (which contains almost all of what we need in trunk).
However, in my testing it appears I got this config wrong and have yet
to sek user list guidance from ivy.
I'm learning as I go ;-)
Ross
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@apache.org]
> Sent: Monday, 26 February 2007 10:53 PM
> To: dev@forrest.apache.org
> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>
> Gav.... wrote:
> >
...
> > I thought I'd do a quick comparison of this branch and trunk.
> >
> > (The comparison includes .svn directories and files so is not an
> > Indicator of true size)
> >
> > Seems ivyBuild uses up 140Mb compared to Trunk's 256MB
> > ivyBuild has 23,565 files compared with Trunks 41,886.
> >
> > Heading in the right direction I think.
>
> I have no idea how that can be the case, thre are only a few jars ripped
> out so far.
I'll do a proper one later with no .svn , I would say that it is due to
The trunk checkout having a lot more history in the .svn that the
Difference is so big.
Thanks for fielding all the questions so far, really does make things
Much clearer.
Gav...
...
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Gav.... wrote:
>
>> -----Original Message-----
>> From: Ross Gardler [mailto:rgardler@apache.org]
>> Sent: Monday, 26 February 2007 8:13 PM
>> To: dev@forrest.apache.org
>> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>>
>> Gav.... wrote:
>>>> -----Original Message-----
>>>> From: Ross Gardler [mailto:rgardler@apache.org]
>>>> Sent: Thursday, 22 February 2007 10:31 PM
>>>> To: dev@forrest.apache.org
>>>> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>>>>
>>>> Ross Gardler wrote:
>>>>
>>>>> * Note that I am using an Eclipse plugin to resolve the dependencies
>> for
>>>>> me. Since you want to help I will make it a priority to create an ANT
>>>>> task to resolve them for us - I should be able to do this later today
>>>> (GMT)
>>>>
>>>> I've added a basic build target for checking that jars resolve (ivy
>>>> branch only of course). Do:
>>>>
>>>> cd [FORREST_HOME]/main
>>>> ant -buildfile ivy-build.xml
>>>>
>>>> Note that you must be using the ant in our tools/ant directory or you
>>>> must place the ivy jar file (found in tools/ivy) into your ant
>>>> installations lib folder.
>>> Ok I give up, I'm assuming you added ivy-1.4.1.jar to /tools/ant/lib/
>>> And you added a ivy-build.xml to /main/ , but I can't find them
>>> Anywhere.
>> Are you sure you are using the ivyBuild branch?
>>
>> http://svn.apache.org/viewvc/forrest/branches/ivyBuild/main/ivy-
>> build.xml?view=log
>
> Erm, nope sorry, I was still looking at Ivy in Forrest 2, oops.
> I must have missed/skimmed a conversation somewhere. Are you
> Now concentrating on converting current 0.8-dev to use Ivy?
Yes.
> Downloaded the branch now, been a while since I downloaded a complete
> Version of Forrest, this took over an hour, the checkout gave up half
> Way through and I had to svn up the rest of it.
Once the move to ivy is complete it will be much smaller.
> I thought I'd do a quick comparison of this branch and trunk.
>
> (The comparison includes .svn directories and files so is not an
> Indicator of true size)
>
> Seems ivyBuild uses up 140Mb compared to Trunk's 256MB
> ivyBuild has 23,565 files compared with Trunks 41,886.
>
> Heading in the right direction I think.
I have no idea how that can be the case, thre are only a few jars ripped
out so far.
> When Ivy is in use properly I guess to start with only a handful
> Of these are downloaded initially and the 30MB in /lib/ wont
> Be downloaded at all, a great saving.
> When Forrest is run for The first time and Ivy is invoked then the 24MB of
> files will Then be downloaded into Ivy Cache, is this right.
That is correct.
> If so then
> A bit of a wait is in store for users. Will there be an (option
> Of an) initial Download of all these files packaged as part of the release
> and Then Ivy takes over from there for updates?
Source downloaders will see the build system grab the jars not already
present in the local ivy cache from an ivy repo on the first build.
Binary downloaders will get all required jars in their initial download.
No need for Ivy to get involved as they have the compiled version.
This begs the question, what happens when someone downloads a binary
version but then wants to run build on the source files? This would
require Ivy to download all the jars, but they will already be present
as part of the binary download, just not in the ivy cache.
I think the best thing to do would be to not include the source with the
binary download, but to provide a separate release and snapshot source
download.
Ross
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Thorsten Scherler <th...@apache.org>.
On Tue, 2007-02-27 at 09:49 +1100, David Crossley wrote:
> Thorsten Scherler wrote:
> >
> > Why have you not used
> > svn switch https://svn.apache.org/repos/asf/forrest/branches/ivyBuild
> > on your trunk?
>
> I have been working with a separate checkout of the branch.
> It seems useful that way, i do spurts of development in both.
>
> I suppose with 'switch' there would be quite a bit of data transfer
> with the difference in jar organisation.
>
Not that much. It was quite quick.
> There are probably advantages to each technique.
Agree.
salu2
--
Thorsten Scherler thorsten.at.apache.org
Open Source Java & XML consulting, training and solutions
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by David Crossley <cr...@apache.org>.
Thorsten Scherler wrote:
>
> Why have you not used
> svn switch https://svn.apache.org/repos/asf/forrest/branches/ivyBuild
> on your trunk?
I have been working with a separate checkout of the branch.
It seems useful that way, i do spurts of development in both.
I suppose with 'switch' there would be quite a bit of data transfer
with the difference in jar organisation.
There are probably advantages to each technique.
-David
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
> -----Original Message-----
> From: Thorsten Scherler [mailto:thorsten@apache.org]
> Sent: Monday, 26 February 2007 10:11 PM
> To: dev@forrest.apache.org
> Subject: RE: [PROPOSAL] Switch dependency managment to IVY
>
> On Mon, 2007-02-26 at 22:01 +0900, Gav.... wrote:
> ...
> > Downloaded the branch now, been a while since I downloaded a complete
> > Version of Forrest, this took over an hour, the checkout gave up half
> > Way through and I had to svn up the rest of it.
>
> Why have you not used
> svn switch https://svn.apache.org/repos/asf/forrest/branches/ivyBuild
> on your trunk?
Oh b*****, didn't occur to me, never used a branch before.
Thanks for the info, will do next time.
Gav...
>
> salu2
> --
> Thorsten Scherler thorsten.at.apache.org
> Open Source Java & XML consulting, training and solutions
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by Thorsten Scherler <th...@apache.org>.
On Mon, 2007-02-26 at 22:01 +0900, Gav.... wrote:
...
> Downloaded the branch now, been a while since I downloaded a complete
> Version of Forrest, this took over an hour, the checkout gave up half
> Way through and I had to svn up the rest of it.
Why have you not used
svn switch https://svn.apache.org/repos/asf/forrest/branches/ivyBuild
on your trunk?
salu2
--
Thorsten Scherler thorsten.at.apache.org
Open Source Java & XML consulting, training and solutions
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@apache.org]
> Sent: Monday, 26 February 2007 8:13 PM
> To: dev@forrest.apache.org
> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>
> Gav.... wrote:
> >
> >> -----Original Message-----
> >> From: Ross Gardler [mailto:rgardler@apache.org]
> >> Sent: Thursday, 22 February 2007 10:31 PM
> >> To: dev@forrest.apache.org
> >> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
> >>
> >> Ross Gardler wrote:
> >>
> >>> * Note that I am using an Eclipse plugin to resolve the dependencies
> for
> >>> me. Since you want to help I will make it a priority to create an ANT
> >>> task to resolve them for us - I should be able to do this later today
> >> (GMT)
> >>
> >> I've added a basic build target for checking that jars resolve (ivy
> >> branch only of course). Do:
> >>
> >> cd [FORREST_HOME]/main
> >> ant -buildfile ivy-build.xml
> >>
> >> Note that you must be using the ant in our tools/ant directory or you
> >> must place the ivy jar file (found in tools/ivy) into your ant
> >> installations lib folder.
> >
> > Ok I give up, I'm assuming you added ivy-1.4.1.jar to /tools/ant/lib/
> > And you added a ivy-build.xml to /main/ , but I can't find them
> > Anywhere.
>
> Are you sure you are using the ivyBuild branch?
>
> http://svn.apache.org/viewvc/forrest/branches/ivyBuild/main/ivy-
> build.xml?view=log
Erm, nope sorry, I was still looking at Ivy in Forrest 2, oops.
I must have missed/skimmed a conversation somewhere. Are you
Now concentrating on converting current 0.8-dev to use Ivy?
Downloaded the branch now, been a while since I downloaded a complete
Version of Forrest, this took over an hour, the checkout gave up half
Way through and I had to svn up the rest of it.
I thought I'd do a quick comparison of this branch and trunk.
(The comparison includes .svn directories and files so is not an
Indicator of true size)
Seems ivyBuild uses up 140Mb compared to Trunk's 256MB
ivyBuild has 23,565 files compared with Trunks 41,886.
Heading in the right direction I think.
The repository used up 24Mb and contains 1030 files (inc .svn)
When Ivy is in use properly I guess to start with only a handful
Of these are downloaded initially and the 30MB in /lib/ wont
Be downloaded at all, a great saving.
When Forrest is run for The first time and Ivy is invoked then the 24MB of
files will Then be downloaded into Ivy Cache, is this right. If so then
A bit of a wait is in store for users. Will there be an (option
Of an) initial Download of all these files packaged as part of the release
and Then Ivy takes over from there for updates?
Gav...
>
> Ross
>
>
> --
> Internal Virus Database is out-of-date.
> Checked by AVG Free Edition.
> Version: 7.1.412 / Virus Database: 268.18.2/692 - Release Date: 2/18/2007
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Gav.... wrote:
>
>> -----Original Message-----
>> From: Ross Gardler [mailto:rgardler@apache.org]
>> Sent: Thursday, 22 February 2007 10:31 PM
>> To: dev@forrest.apache.org
>> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>>
>> Ross Gardler wrote:
>>
>>> * Note that I am using an Eclipse plugin to resolve the dependencies for
>>> me. Since you want to help I will make it a priority to create an ANT
>>> task to resolve them for us - I should be able to do this later today
>> (GMT)
>>
>> I've added a basic build target for checking that jars resolve (ivy
>> branch only of course). Do:
>>
>> cd [FORREST_HOME]/main
>> ant -buildfile ivy-build.xml
>>
>> Note that you must be using the ant in our tools/ant directory or you
>> must place the ivy jar file (found in tools/ivy) into your ant
>> installations lib folder.
>
> Ok I give up, I'm assuming you added ivy-1.4.1.jar to /tools/ant/lib/
> And you added a ivy-build.xml to /main/ , but I can't find them
> Anywhere.
Are you sure you are using the ivyBuild branch?
http://svn.apache.org/viewvc/forrest/branches/ivyBuild/main/ivy-build.xml?view=log
Ross
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@apache.org]
> Sent: Thursday, 22 February 2007 10:31 PM
> To: dev@forrest.apache.org
> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>
> Ross Gardler wrote:
>
> > * Note that I am using an Eclipse plugin to resolve the dependencies for
> > me. Since you want to help I will make it a priority to create an ANT
> > task to resolve them for us - I should be able to do this later today
> (GMT)
>
> I've added a basic build target for checking that jars resolve (ivy
> branch only of course). Do:
>
> cd [FORREST_HOME]/main
> ant -buildfile ivy-build.xml
>
> Note that you must be using the ant in our tools/ant directory or you
> must place the ivy jar file (found in tools/ivy) into your ant
> installations lib folder.
Ok I give up, I'm assuming you added ivy-1.4.1.jar to /tools/ant/lib/
And you added a ivy-build.xml to /main/ , but I can't find them
Anywhere.
Gav...
>
> Note that you cannot yet build using this build file as I've only moved
> a handful of the jars to ivy. Devs should not waste time on trying to
> make it build (I have generic scripts ready to kick this off, if you
> really want to peek ahead look inside the tools/ivy/build-scripts dir).
>
> You may also fin the output of the report target useful (look for the
> html in the build directory after you run it)
>
> Ross
>
>
>
>
> --
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.412 / Virus Database: 268.18.2/692 - Release Date: 2/18/2007
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Ross Gardler wrote:
> * Note that I am using an Eclipse plugin to resolve the dependencies for
> me. Since you want to help I will make it a priority to create an ANT
> task to resolve them for us - I should be able to do this later today (GMT)
I've added a basic build target for checking that jars resolve (ivy
branch only of course). Do:
cd [FORREST_HOME]/main
ant -buildfile ivy-build.xml
Note that you must be using the ant in our tools/ant directory or you
must place the ivy jar file (found in tools/ivy) into your ant
installations lib folder.
Note that you cannot yet build using this build file as I've only moved
a handful of the jars to ivy. Devs should not waste time on trying to
make it build (I have generic scripts ready to kick this off, if you
really want to peek ahead look inside the tools/ivy/build-scripts dir).
You may also fin the output of the report target useful (look for the
html in the build directory after you run it)
Ross
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Gav.... wrote:
> Below is some great information, I am beginning to understand
> This better now, so thanks for this, very useful. Can it be
> Doc'ed anywhere do you think as is or do you think the docs
> You have done say the same thing ? (Either way this says it
> Better IMHO)
I'm quite certain the Ivy project would welcome a "case study" doc on
how Forrest managed this migration. If you want to start such a doc in
the tools/ivy/docs directory that would be cool.
I don't think we (Forrest) should host any docs that are specific to
Ivy, it'll get a wider audience over there.
Ross
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
Below is some great information, I am beginning to understand
This better now, so thanks for this, very useful. Can it be
Doc'ed anywhere do you think as is or do you think the docs
You have done say the same thing ? (Either way this says it
Better IMHO)
Cheers
Gav...
> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@apache.org]
> Sent: Thursday, 22 February 2007 7:30 PM
> To: dev@forrest.apache.org
> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>
> David Crossley wrote:
> > Ross Gardler wrote:
> >> Ross Gardler wrote:
> >>> Here is my plan:
> >>>
> >>> 1 - I create a Forrest only repo in SVN (as I have started for
> >>> Forrest2). although wherever possible we will use the IVY repo and the
> >>> Maven2 repo
> >>>
> >>> 2 - I modify our build files to use IVY (I have a whole set of generic
> >>> build files that are intended to be extended by project specific build
> >>> files. I plan to give them to the IVY folk if they think it they are a
> >>> good idea, if not we'll take what we need from them)
> >>>
> >>> 3 - We make sure it works
> >>>
> >>> 4 - We move the entries in our repo to the IVY repo (I need to enter
> >>> into a dialogue with IVY devs to see what is happening with regards an
> >>> official Apache repo)
> >>>
>
> ...
>
> >
> > I want to assist. Some initial questions ...
>
> Cool. As you have seen my work on this has frozen due to other
> commitments. Should be getting back to it soon.
>
> > I see that you have Ivy module descriptors, e.g.
> > tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
> >
> > Are these descriptors something that the Forrest
> > project maintains, or do we copy them from somewhere
> > else and thus need to keep them synchronised? I want
> > to clean them up to be consistent w.r.t. license stuff
> > etc. but not if they will come from some other project.
> > It seems to me that it is better to keep our own descriptors.
>
> The whole tools/ivy/repository section dir really go to an ivy community
> managed repo. The Ivy project has two, a scratchpad one where things may
> not be perfectly described in terms of their dependencies etc. and a
> main one where there are tight controls on what goes in there and so it
> should be complete and accurate (i.e. stable).
>
> So, my intention is that whatever we end up with in our local repo will
> eventually be moved to the ivy scratchpad where the community can clean
> it up an eventually move it into the main ivy repo.
>
> Why not do it straight into the ivy scratchpad? Because I want Forrest
> devs to see how the ivy files are put together so that there is a
> history in our archives for people to manage the scratchpad/main reop
> over in Ivy.
>
> > Why xerces-2.5.0 when we currently use xerces-2.8.0?
>
> I can only guess when I did the Forrest2 ivy repo I started it off by
> pulling the latest available from a Maven repo. So I ended up with
> xerces-2.5.0. I've not yet done the one in Forrest trunk so not yet
> upgraded to 2.8.0.
>
> > Is your plan to move everything that we use out of lib/*
> > into this local tools/ivy/repository, then gradually
> > remove/tune stuff as we find them in other official repositories?
>
> Yes, see above.
>
> > I gather that we can have Ivy descriptors in our local
> > repository, but no jars, and then point to other repositories
> > for the jars, e.g. maven repo.
>
> We don't even need the descriptors. We just need the ivy.xml file in our
> trunk. This tells Ivy what Forrest depends on. It goes off to the
> defined repositories (defined in tools/ivy/ivyconf.xml) and finds the
> relevant ivy.xml files for that dependency. This identifies any jars
> that are required by that dependency, which allows Ivy to go looking for
> them, and so on.
>
> So, we are aiming at only needing to maintain Forrests ivy.xml file here
> in Forrest.
>
> ----
>
> Here's the process I'm using at present:
>
> 1. remove a jar from the lib directory
>
> 2. add a dependency to the forrest ivy.xml file
>
> 3. try resolving the dependencies* - since the jar is not in our local
> repo it will go to the ivy and maven repos - it's supposed to go to the
> Cocoon repos too, but I haven't got that working yet. I joined the ivy
> user list to ask for advice but have not got around to it yet
>
> 4. if it resolves OK commit the changes, and go back to 1
>
> 5. if it fails to resolve we need to manually inspect the ivy repo to
> look to see if it is available with a different organisation or jar name
> (I don't bother doing this in the maven repos though
>
> 6. if it doesn't exist in the ivy repos then add the jar and ivy.xml to
> our tools/ivy/repository
>
> 7. try resolving dependencies, it should now work, so go back to 1.
>
> * Note that I am using an Eclipse plugin to resolve the dependencies for
> me. Since you want to help I will make it a priority to create an ANT
> task to resolve them for us - I should be able to do this later today
> (GMT)
>
> Ross
>
>
> --
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.412 / Virus Database: 268.18.2/692 - Release Date: 2/18/2007
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by David Crossley <cr...@apache.org>.
An old reply that i forgot to send ...
Ross Gardler wrote:
> Gav.... wrote:
> > Ross Gardler wrote:
> >>David Crossley wrote:
> >>
> >>>I see that you have Ivy module descriptors, e.g.
> >>>tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
> >>>
> >>>Are these descriptors something that the Forrest
> >>>project maintains, or do we copy them from somewhere
> >>>else and thus need to keep them synchronised? I want
> >>>to clean them up to be consistent w.r.t. license stuff
> >>>etc. but not if they will come from some other project.
> >>>It seems to me that it is better to keep our own descriptors.
> >
> >I think the aim is that Ivy will update the license stuff in their
> >Public repo, they are now part of Apache and so I don't see why a
> >Transfer of license shouldn't take place. ...
I don't understand what you mean by "transfer of license".
These descriptors declare the license that is used by each
particular third-party product.
> ... This would negate the
> >Need for us to update the license info I would have thought.
> >(Will check on that more thoroughly)
>
> No, the aim is for us to pass a set of conf files to the ivy scratchpad
> repository that work an are complete. It's our itch, we should scratch it.
Ah, that is what i wanted to know.
So we should tidy up these files here.
-David
> Besides, it is our PMC's responsibility to ensure our licence
> information is valid.
>
> >>Why not do it straight into the ivy scratchpad? Because I want Forrest
> >>devs to see how the ivy files are put together so that there is a
> >>history in our archives for people to manage the scratchpad/main reop
> >>over in Ivy.
> >
> >Don't most jars/dependencies come from Ivy/ibiblio anyway ?
>
> Some do, whether it will be most remains to be seen. Certainly, some
> will not. Those that do not come from Ivy/Ibiblio need to be in our own
> repo.
[ snip ]
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Thorsten Scherler <th...@apache.org>.
On Mon, 2007-02-26 at 14:05 +0000, Ross Gardler wrote:
> Gav.... wrote:
...
> >> Right now the repo is in forrest/tools/ivy and is therefore checked out
> >> with Forrest svn checkout. This is a temporary thing. It makes it easier
> >> for other forrest devs to get involved with the migration to Ivy (a
> >> single checkout. However, you are right, we should, eventually, make
> >> this repo a completely separate SVN tree.
> >
> > Ok, not sure how including the repo in the checkout makes things easier
> > For devs, may as well have it set up how its supposed to be, this way
> > Devs will get first hand usage of it straight away.
>
> It may just be the way I am working locally. But I find it useful to
> have everything in a single checkout whilst I move things from one part
> of the tree to another. It makes the SVN commands considerably shorter
> an therefore less error prone.
>
> Like I say, this may be a personal issue. I've set the config files up
> with a property to define the location of the shared ivy repo so it is
> really simple to move it once the migration is complete.
Yes, I think the result of the migration should be a completely separate
SVN tree. This makes moving it to Ivyrep (where IMO all this in the end
belongs) much easier. For now I see the benefit from using
forrest/tools/ivy.
salu2
--
Thorsten Scherler thorsten.at.apache.org
Open Source Java & XML consulting, training and solutions
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Gav.... wrote:
>>> I think the aim is that Ivy will update the license stuff in their
>>> Public repo, they are now part of Apache and so I don't see why a
>>> Transfer of license shouldn't take place. This would negate the
>>> Need for us to update the license info I would have thought.
>>> (Will check on that more thoroughly)
>> No, the aim is for us to pass a set of conf files to the ivy scratchpad
>> repository that work an are complete. It's our itch, we should scratch it.
>>
>> Besides, it is our PMC's responsibility to ensure our licence
>> information is valid.
>
> I get the PMC responsibility bit, what I don't get is us updating files that
> Come from the ivyrep. I thought these are files that many projects make use
> Of. Or are you saying that 'one' of the projects that makes use of these
> Files will ensure they are compliant.
I'm saying that by the time we hand them off to Ivy they should be
complete and useful files. Should we be the first to notice a future
change with respect to licenses, version numbers or whatever, we should
provide a patch to the Ivy team.
Just because it is not in our SVN does not mean we cannot continue to
help maintain it.
Of course, the idea is that other projects use the same repo and conf
files, thus we may find that some other project does updates before us.
Then we have saved ourselves some time.
>> When building a binary release they will, of course, be copied into the
>> release. Ivy has tools for doing this.
>
> Ok, this answers my other post I guess, your intending to include all
> Jars as part of the release. So are we only utilising Ivy for our
> Own purposes / ease of use and not using it to ensure that when a
> Release version gets downloaded by someone, Ivy could get the latest of
> Everything at the time. This would make for current release patches of
> Required files easier.
We are only using during the build process. A release should have a
collection of libs that have been fully tested and, to the best of our
knowledge work. Allowing Ivy to update dependencies in a release means
that we have no longer tested that configuration.
>> Right now the repo is in forrest/tools/ivy and is therefore checked out
>> with Forrest svn checkout. This is a temporary thing. It makes it easier
>> for other forrest devs to get involved with the migration to Ivy (a
>> single checkout. However, you are right, we should, eventually, make
>> this repo a completely separate SVN tree.
>
> Ok, not sure how including the repo in the checkout makes things easier
> For devs, may as well have it set up how its supposed to be, this way
> Devs will get first hand usage of it straight away.
It may just be the way I am working locally. But I find it useful to
have everything in a single checkout whilst I move things from one part
of the tree to another. It makes the SVN commands considerably shorter
an therefore less error prone.
Like I say, this may be a personal issue. I've set the config files up
with a property to define the location of the shared ivy repo so it is
really simple to move it once the migration is complete.
>
>
>>>>> Why xerces-2.5.0 when we currently use xerces-2.8.0?
>>>> I can only guess when I did the Forrest2 ivy repo I started it off by
>>>> pulling the latest available from a Maven repo. So I ended up with
>>>> xerces-2.5.0. I've not yet done the one in Forrest trunk so not yet
>>>> upgraded to 2.8.0.
>>> I guess this would be a prime example of using Forrests own Shared
>>> Repo, if we don't like what version Ivy is sending us, we switch to
>>> Our repo for the version we do want.
>> No, we update the public repo so that others get the benefit of our
>> update.
>
> I'm now thinking along the lines of our Xalan issue and also in a way
> Cocoons upgrade to 2.1 could be an example. If we upgrade the public
> Repo version of xerces to 2.8 from 2.5 is there a possibility that
> We will break other peoples builds because of it?
No. That's the whole point of Ivy. You say what version you want, 2.5,
2.8. 2.x. latest release, latest milestone or whatever. Ivy will get the
closest fit to the version you want (as well as trying to resolve
conflicts in the dependency tree).
That is, the public repo can have version Xalan 2.5 and 2.8 at the same
time. It is the local config that defines which one we retrieve. This is
similar to, but more robust than, the plugin resolution mechanism we have.
>> Note in this case, since I have now made things work from the apache
>> maven repo (rather than ibiblio) we should find the later version of
>> xerces is available to us because someone else will have updated the
>> apache maven repo (I would have thought)
>
> Ok, if not then we should do it?
yes.
Ross
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@apache.org]
> Sent: Monday, 26 February 2007 8:10 PM
> To: dev@forrest.apache.org
> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>
> Gav.... wrote:
> >
> >> -----Original Message-----
> >> From: Ross Gardler [mailto:rgardler@apache.org]
> >> Sent: Thursday, 22 February 2007 7:30 PM
> >> To: dev@forrest.apache.org
> >> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
> >>
> >> David Crossley wrote:
> >>> Ross Gardler wrote:
> >>>> Ross Gardler wrote:
> >>>>> Here is my plan:
> >>>>>
> >>>>> 1 - I create a Forrest only repo in SVN (as I have started for
> >>>>> Forrest2). although wherever possible we will use the IVY repo and
> the
> >>>>> Maven2 repo
> >>>>>
> >>>>> 2 - I modify our build files to use IVY (I have a whole set of
> generic
> >>>>> build files that are intended to be extended by project specific
> build
> >>>>> files. I plan to give them to the IVY folk if they think it they are
> a
> >>>>> good idea, if not we'll take what we need from them)
> >>>>>
> >>>>> 3 - We make sure it works
> >>>>>
> >>>>> 4 - We move the entries in our repo to the IVY repo (I need to enter
> >>>>> into a dialogue with IVY devs to see what is happening with regards
> an
> >>>>> official Apache repo)
> >>>>>
> >> ...
> >>
> >>> I want to assist. Some initial questions ...
> >> Cool. As you have seen my work on this has frozen due to other
> >> commitments. Should be getting back to it soon.
> >>
> >>> I see that you have Ivy module descriptors, e.g.
> >>> tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
> >>>
> >>> Are these descriptors something that the Forrest
> >>> project maintains, or do we copy them from somewhere
> >>> else and thus need to keep them synchronised? I want
> >>> to clean them up to be consistent w.r.t. license stuff
> >>> etc. but not if they will come from some other project.
> >>> It seems to me that it is better to keep our own descriptors.
> >
> > I think the aim is that Ivy will update the license stuff in their
> > Public repo, they are now part of Apache and so I don't see why a
> > Transfer of license shouldn't take place. This would negate the
> > Need for us to update the license info I would have thought.
> > (Will check on that more thoroughly)
>
> No, the aim is for us to pass a set of conf files to the ivy scratchpad
> repository that work an are complete. It's our itch, we should scratch it.
>
> Besides, it is our PMC's responsibility to ensure our licence
> information is valid.
I get the PMC responsibility bit, what I don't get is us updating files that
Come from the ivyrep. I thought these are files that many projects make use
Of. Or are you saying that 'one' of the projects that makes use of these
Files will ensure they are compliant.
>
> >> Why not do it straight into the ivy scratchpad? Because I want Forrest
> >> devs to see how the ivy files are put together so that there is a
> >> history in our archives for people to manage the scratchpad/main reop
> >> over in Ivy.
> >
> > Don't most jars/dependencies come from Ivy/ibiblio anyway ?
>
> Some do, whether it will be most remains to be seen. Certainly, some
> will not. Those that do not come from Ivy/Ibiblio need to be in our own
> repo.
>
> > During my
> > Limited time taking a look at Ivy over the last few days I understood
> > That required jars are downloaded if not already present, and look for
> > Updated versions and download them automatically if a new iteration
> > Exists.
>
> Whether updates are downloaded automatically depends on our own ivy.xml.
> We can say "only version x.y.z", "any release", "any milestone release",
> "any minor version in the 1.2.y tree" etc.
>
> > These jars would then be stored in our /lib/*/ directory.
>
> They can be downloaded to a /lib/* directory, but there is little point
> in that.
>
> You see ivy downloads all jars to a central cache on the hard disk, by
> default this is ~/.ivy/cache. All applications you have that use Ivy
> will use the jar from that location (i.e. only one version of each jar
> on your hard disk).
Ok, yep got that.
>
> When building a binary release they will, of course, be copied into the
> release. Ivy has tools for doing this.
Ok, this answers my other post I guess, your intending to include all
Jars as part of the release. So are we only utilising Ivy for our
Own purposes / ease of use and not using it to ensure that when a
Release version gets downloaded by someone, Ivy could get the latest of
Everything at the time. This would make for current release patches of
Required files easier.
>
> > Any 'Forrest' derived jars, are you intending that these get uploaded
> > To Ivy Scratchpad for approval and then moved to the main Ivy repo?
>
> I'm not sure what you mean by "'Forrest' derived jars".
>
> If you mean should binary releases of Forrest be in the ivy repo - then
> probably no, since the jars of Forrest1 are not really useful in an
> embedded environment. Forrest2 is a different story, but it is also far
> too early to b e worrying about that.
Ok.
>
> If you mean the snapshot builds of Cocoon we use, I doubt they will be
> useful in a public Ivy repo, so we will probably keep them in a Forrest
> managed repo.
>
> > You mention 'local' repo, just thinking out loud here, my understanding
> > Of the 'local' repo was for testing/using in a localised environment
> such
> > As your computer, whereas using the 'Shared' repo would be a better fit
> > For Forrest as the Shared repo would be stored in our SVN. Even this, in
> > The future may not be needed at all if Forrest specific dependencies
> were
> > Added to the Ivy public repo. That's my interpretation, could be wrong
> of
> > Course.
>
> You are right. A shared repo would make more sense, and that is exactly
> what we are moving towards. I've been mixing my terminology, by local I
> meant local to Forrest (i.e. what ivy calls a shared repo).
>
> Right now the repo is in forrest/tools/ivy and is therefore checked out
> with Forrest svn checkout. This is a temporary thing. It makes it easier
> for other forrest devs to get involved with the migration to Ivy (a
> single checkout. However, you are right, we should, eventually, make
> this repo a completely separate SVN tree.
Ok, not sure how including the repo in the checkout makes things easier
For devs, may as well have it set up how its supposed to be, this way
Devs will get first hand usage of it straight away.
>
> >>> Why xerces-2.5.0 when we currently use xerces-2.8.0?
> >> I can only guess when I did the Forrest2 ivy repo I started it off by
> >> pulling the latest available from a Maven repo. So I ended up with
> >> xerces-2.5.0. I've not yet done the one in Forrest trunk so not yet
> >> upgraded to 2.8.0.
> >
> > I guess this would be a prime example of using Forrests own Shared
> > Repo, if we don't like what version Ivy is sending us, we switch to
> > Our repo for the version we do want.
>
> No, we update the public repo so that others get the benefit of our
> update.
I'm now thinking along the lines of our Xalan issue and also in a way
Cocoons upgrade to 2.1 could be an example. If we upgrade the public
Repo version of xerces to 2.8 from 2.5 is there a possibility that
We will break other peoples builds because of it? Then again, Ivyrep
Would never get updated if everyone thought like that (!?!) - I
Suppose it would be up to the other projects to ensure they're Ivy was
Configured correctly and so would not download it until they are
Ready.
>
> Note in this case, since I have now made things work from the apache
> maven repo (rather than ibiblio) we should find the later version of
> xerces is available to us because someone else will have updated the
> apache maven repo (I would have thought)
Ok, if not then we should do it?
>
> >>> Is your plan to move everything that we use out of lib/*
> >>> into this local tools/ivy/repository, then gradually
> >>> remove/tune stuff as we find them in other official repositories?
> >> Yes, see above.
> >
> > I thought the idea was we use Ivy for our dependencies, Ivy manages
> > This and downloads the appropriate jars/versions - they would still
> > End up in the /lib/ directory , or not ? (They would come from either
> > Ivy Public repo or Forrest Shared Repo but still end up in /lib/)
>
> See above.
>
> >>> I gather that we can have Ivy descriptors in our local
> >>> repository, but no jars, and then point to other repositories
> >>> for the jars, e.g. maven repo.
> >> We don't even need the descriptors. We just need the ivy.xml file in
> our
> >> trunk. This tells Ivy what Forrest depends on. It goes off to the
> >> defined repositories (defined in tools/ivy/ivyconf.xml) and finds the
> >> relevant ivy.xml files for that dependency. This identifies any jars
> >> that are required by that dependency, which allows Ivy to go looking
> for
> >> them, and so on.
> >>
> >> So, we are aiming at only needing to maintain Forrests ivy.xml file
> here
> >> in Forrest.
> >
> > And the ivyconf.xml
>
> Yes.
>
> >> 3. try resolving the dependencies* - since the jar is not in our local
> >> repo it will go to the ivy and maven repos - it's supposed to go to the
> >> Cocoon repos too, but I haven't got that working yet. I joined the ivy
> >> user list to ask for advice but have not got around to it yet
>
> [ADDENDUM - I've fixed it now so it does go to the Apache Maven
> repositories, as used by Cocoon]
>
> > Hmm, I'm confused and not for the first time. I thought a repo holds all
> > Possible jars for all situations. I didn't think that is where you are
> > Supposed to actually use them from. You use the ivy.xml to pull needed
> > Jars into /lib/ and use them from there. This is what I am assuming from
> > The Ivy Docs and downloaded tutorials.
>
> See above clarification regarding /lib/* - is it clearer now?
Yep, much, thanks.
Gav...
>
> > Hope I'm helping not hindering, looks like the answer to many a prayer
> > Does Ivy so trying to understand it a little bit - as I can see we will
> > Be using it one way or another.
>
> As ever, I am learning as we go so having to answer questions and
> justify any actions I am taking will highlight anything I have
> misunderstood from the Ivy docs. Keep it up!
>
> Ross
>
>
> --
> Internal Virus Database is out-of-date.
> Checked by AVG Free Edition.
> Version: 7.1.412 / Virus Database: 268.18.2/692 - Release Date: 2/18/2007
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Gav.... wrote:
>
>> -----Original Message-----
>> From: Ross Gardler [mailto:rgardler@apache.org]
>> Sent: Thursday, 22 February 2007 7:30 PM
>> To: dev@forrest.apache.org
>> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>>
>> David Crossley wrote:
>>> Ross Gardler wrote:
>>>> Ross Gardler wrote:
>>>>> Here is my plan:
>>>>>
>>>>> 1 - I create a Forrest only repo in SVN (as I have started for
>>>>> Forrest2). although wherever possible we will use the IVY repo and the
>>>>> Maven2 repo
>>>>>
>>>>> 2 - I modify our build files to use IVY (I have a whole set of generic
>>>>> build files that are intended to be extended by project specific build
>>>>> files. I plan to give them to the IVY folk if they think it they are a
>>>>> good idea, if not we'll take what we need from them)
>>>>>
>>>>> 3 - We make sure it works
>>>>>
>>>>> 4 - We move the entries in our repo to the IVY repo (I need to enter
>>>>> into a dialogue with IVY devs to see what is happening with regards an
>>>>> official Apache repo)
>>>>>
>> ...
>>
>>> I want to assist. Some initial questions ...
>> Cool. As you have seen my work on this has frozen due to other
>> commitments. Should be getting back to it soon.
>>
>>> I see that you have Ivy module descriptors, e.g.
>>> tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
>>>
>>> Are these descriptors something that the Forrest
>>> project maintains, or do we copy them from somewhere
>>> else and thus need to keep them synchronised? I want
>>> to clean them up to be consistent w.r.t. license stuff
>>> etc. but not if they will come from some other project.
>>> It seems to me that it is better to keep our own descriptors.
>
> I think the aim is that Ivy will update the license stuff in their
> Public repo, they are now part of Apache and so I don't see why a
> Transfer of license shouldn't take place. This would negate the
> Need for us to update the license info I would have thought.
> (Will check on that more thoroughly)
No, the aim is for us to pass a set of conf files to the ivy scratchpad
repository that work an are complete. It's our itch, we should scratch it.
Besides, it is our PMC's responsibility to ensure our licence
information is valid.
>> Why not do it straight into the ivy scratchpad? Because I want Forrest
>> devs to see how the ivy files are put together so that there is a
>> history in our archives for people to manage the scratchpad/main reop
>> over in Ivy.
>
> Don't most jars/dependencies come from Ivy/ibiblio anyway ?
Some do, whether it will be most remains to be seen. Certainly, some
will not. Those that do not come from Ivy/Ibiblio need to be in our own
repo.
> During my
> Limited time taking a look at Ivy over the last few days I understood
> That required jars are downloaded if not already present, and look for
> Updated versions and download them automatically if a new iteration
> Exists.
Whether updates are downloaded automatically depends on our own ivy.xml.
We can say "only version x.y.z", "any release", "any milestone release",
"any minor version in the 1.2.y tree" etc.
> These jars would then be stored in our /lib/*/ directory.
They can be downloaded to a /lib/* directory, but there is little point
in that.
You see ivy downloads all jars to a central cache on the hard disk, by
default this is ~/.ivy/cache. All applications you have that use Ivy
will use the jar from that location (i.e. only one version of each jar
on your hard disk).
When building a binary release they will, of course, be copied into the
release. Ivy has tools for doing this.
> Any 'Forrest' derived jars, are you intending that these get uploaded
> To Ivy Scratchpad for approval and then moved to the main Ivy repo?
I'm not sure what you mean by "'Forrest' derived jars".
If you mean should binary releases of Forrest be in the ivy repo - then
probably no, since the jars of Forrest1 are not really useful in an
embedded environment. Forrest2 is a different story, but it is also far
too early to b e worrying about that.
If you mean the snapshot builds of Cocoon we use, I doubt they will be
useful in a public Ivy repo, so we will probably keep them in a Forrest
managed repo.
> You mention 'local' repo, just thinking out loud here, my understanding
> Of the 'local' repo was for testing/using in a localised environment such
> As your computer, whereas using the 'Shared' repo would be a better fit
> For Forrest as the Shared repo would be stored in our SVN. Even this, in
> The future may not be needed at all if Forrest specific dependencies were
> Added to the Ivy public repo. That's my interpretation, could be wrong of
> Course.
You are right. A shared repo would make more sense, and that is exactly
what we are moving towards. I've been mixing my terminology, by local I
meant local to Forrest (i.e. what ivy calls a shared repo).
Right now the repo is in forrest/tools/ivy and is therefore checked out
with Forrest svn checkout. This is a temporary thing. It makes it easier
for other forrest devs to get involved with the migration to Ivy (a
single checkout. However, you are right, we should, eventually, make
this repo a completely separate SVN tree.
>>> Why xerces-2.5.0 when we currently use xerces-2.8.0?
>> I can only guess when I did the Forrest2 ivy repo I started it off by
>> pulling the latest available from a Maven repo. So I ended up with
>> xerces-2.5.0. I've not yet done the one in Forrest trunk so not yet
>> upgraded to 2.8.0.
>
> I guess this would be a prime example of using Forrests own Shared
> Repo, if we don't like what version Ivy is sending us, we switch to
> Our repo for the version we do want.
No, we update the public repo so that others get the benefit of our update.
Note in this case, since I have now made things work from the apache
maven repo (rather than ibiblio) we should find the later version of
xerces is available to us because someone else will have updated the
apache maven repo (I would have thought)
>>> Is your plan to move everything that we use out of lib/*
>>> into this local tools/ivy/repository, then gradually
>>> remove/tune stuff as we find them in other official repositories?
>> Yes, see above.
>
> I thought the idea was we use Ivy for our dependencies, Ivy manages
> This and downloads the appropriate jars/versions - they would still
> End up in the /lib/ directory , or not ? (They would come from either
> Ivy Public repo or Forrest Shared Repo but still end up in /lib/)
See above.
>>> I gather that we can have Ivy descriptors in our local
>>> repository, but no jars, and then point to other repositories
>>> for the jars, e.g. maven repo.
>> We don't even need the descriptors. We just need the ivy.xml file in our
>> trunk. This tells Ivy what Forrest depends on. It goes off to the
>> defined repositories (defined in tools/ivy/ivyconf.xml) and finds the
>> relevant ivy.xml files for that dependency. This identifies any jars
>> that are required by that dependency, which allows Ivy to go looking for
>> them, and so on.
>>
>> So, we are aiming at only needing to maintain Forrests ivy.xml file here
>> in Forrest.
>
> And the ivyconf.xml
Yes.
>> 3. try resolving the dependencies* - since the jar is not in our local
>> repo it will go to the ivy and maven repos - it's supposed to go to the
>> Cocoon repos too, but I haven't got that working yet. I joined the ivy
>> user list to ask for advice but have not got around to it yet
[ADDENDUM - I've fixed it now so it does go to the Apache Maven
repositories, as used by Cocoon]
> Hmm, I'm confused and not for the first time. I thought a repo holds all
> Possible jars for all situations. I didn't think that is where you are
> Supposed to actually use them from. You use the ivy.xml to pull needed
> Jars into /lib/ and use them from there. This is what I am assuming from
> The Ivy Docs and downloaded tutorials.
See above clarification regarding /lib/* - is it clearer now?
> Hope I'm helping not hindering, looks like the answer to many a prayer
> Does Ivy so trying to understand it a little bit - as I can see we will
> Be using it one way or another.
As ever, I am learning as we go so having to answer questions and
justify any actions I am taking will highlight anything I have
misunderstood from the Ivy docs. Keep it up!
Ross
RE: [PROPOSAL] Switch dependency managment to IVY
Posted by "Gav...." <br...@brightontown.com.au>.
> -----Original Message-----
> From: Ross Gardler [mailto:rgardler@apache.org]
> Sent: Thursday, 22 February 2007 7:30 PM
> To: dev@forrest.apache.org
> Subject: Re: [PROPOSAL] Switch dependency managment to IVY
>
> David Crossley wrote:
> > Ross Gardler wrote:
> >> Ross Gardler wrote:
> >>> Here is my plan:
> >>>
> >>> 1 - I create a Forrest only repo in SVN (as I have started for
> >>> Forrest2). although wherever possible we will use the IVY repo and the
> >>> Maven2 repo
> >>>
> >>> 2 - I modify our build files to use IVY (I have a whole set of generic
> >>> build files that are intended to be extended by project specific build
> >>> files. I plan to give them to the IVY folk if they think it they are a
> >>> good idea, if not we'll take what we need from them)
> >>>
> >>> 3 - We make sure it works
> >>>
> >>> 4 - We move the entries in our repo to the IVY repo (I need to enter
> >>> into a dialogue with IVY devs to see what is happening with regards an
> >>> official Apache repo)
> >>>
>
> ...
>
> >
> > I want to assist. Some initial questions ...
>
> Cool. As you have seen my work on this has frozen due to other
> commitments. Should be getting back to it soon.
>
> > I see that you have Ivy module descriptors, e.g.
> > tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
> >
> > Are these descriptors something that the Forrest
> > project maintains, or do we copy them from somewhere
> > else and thus need to keep them synchronised? I want
> > to clean them up to be consistent w.r.t. license stuff
> > etc. but not if they will come from some other project.
> > It seems to me that it is better to keep our own descriptors.
I think the aim is that Ivy will update the license stuff in their
Public repo, they are now part of Apache and so I don't see why a
Transfer of license shouldn't take place. This would negate the
Need for us to update the license info I would have thought.
(Will check on that more thoroughly)
>
> The whole tools/ivy/repository section dir really go to an ivy community
> managed repo. The Ivy project has two, a scratchpad one where things may
> not be perfectly described in terms of their dependencies etc. and a
> main one where there are tight controls on what goes in there and so it
> should be complete and accurate (i.e. stable).
>
> So, my intention is that whatever we end up with in our local repo will
> eventually be moved to the ivy scratchpad where the community can clean
> it up an eventually move it into the main ivy repo.
>
> Why not do it straight into the ivy scratchpad? Because I want Forrest
> devs to see how the ivy files are put together so that there is a
> history in our archives for people to manage the scratchpad/main reop
> over in Ivy.
Don't most jars/dependencies come from Ivy/ibiblio anyway ? During my
Limited time taking a look at Ivy over the last few days I understood
That required jars are downloaded if not already present, and look for
Updated versions and download them automatically if a new iteration
Exists.
These jars would then be stored in our /lib/*/ directory.
Any 'Forrest' derived jars, are you intending that these get uploaded
To Ivy Scratchpad for approval and then moved to the main Ivy repo?
You mention 'local' repo, just thinking out loud here, my understanding
Of the 'local' repo was for testing/using in a localised environment such
As your computer, whereas using the 'Shared' repo would be a better fit
For Forrest as the Shared repo would be stored in our SVN. Even this, in
The future may not be needed at all if Forrest specific dependencies were
Added to the Ivy public repo. That's my interpretation, could be wrong of
Course.
>
> > Why xerces-2.5.0 when we currently use xerces-2.8.0?
>
> I can only guess when I did the Forrest2 ivy repo I started it off by
> pulling the latest available from a Maven repo. So I ended up with
> xerces-2.5.0. I've not yet done the one in Forrest trunk so not yet
> upgraded to 2.8.0.
I guess this would be a prime example of using Forrests own Shared
Repo, if we don't like what version Ivy is sending us, we switch to
Our repo for the version we do want.
>
> > Is your plan to move everything that we use out of lib/*
> > into this local tools/ivy/repository, then gradually
> > remove/tune stuff as we find them in other official repositories?
>
> Yes, see above.
I thought the idea was we use Ivy for our dependencies, Ivy manages
This and downloads the appropriate jars/versions - they would still
End up in the /lib/ directory , or not ? (They would come from either
Ivy Public repo or Forrest Shared Repo but still end up in /lib/)
>
> > I gather that we can have Ivy descriptors in our local
> > repository, but no jars, and then point to other repositories
> > for the jars, e.g. maven repo.
>
> We don't even need the descriptors. We just need the ivy.xml file in our
> trunk. This tells Ivy what Forrest depends on. It goes off to the
> defined repositories (defined in tools/ivy/ivyconf.xml) and finds the
> relevant ivy.xml files for that dependency. This identifies any jars
> that are required by that dependency, which allows Ivy to go looking for
> them, and so on.
>
> So, we are aiming at only needing to maintain Forrests ivy.xml file here
> in Forrest.
And the ivyconf.xml
>
> ----
>
> Here's the process I'm using at present:
>
> 1. remove a jar from the lib directory
>
> 2. add a dependency to the forrest ivy.xml file
>
> 3. try resolving the dependencies* - since the jar is not in our local
> repo it will go to the ivy and maven repos - it's supposed to go to the
> Cocoon repos too, but I haven't got that working yet. I joined the ivy
> user list to ask for advice but have not got around to it yet
Hmm, I'm confused and not for the first time. I thought a repo holds all
Possible jars for all situations. I didn't think that is where you are
Supposed to actually use them from. You use the ivy.xml to pull needed
Jars into /lib/ and use them from there. This is what I am assuming from
The Ivy Docs and downloaded tutorials.
I think you got an answer on the list.
Hope I'm helping not hindering, looks like the answer to many a prayer
Does Ivy so trying to understand it a little bit - as I can see we will
Be using it one way or another.
Gav...
>
> 4. if it resolves OK commit the changes, and go back to 1
>
> 5. if it fails to resolve we need to manually inspect the ivy repo to
> look to see if it is available with a different organisation or jar name
> (I don't bother doing this in the maven repos though
>
> 6. if it doesn't exist in the ivy repos then add the jar and ivy.xml to
> our tools/ivy/repository
>
> 7. try resolving dependencies, it should now work, so go back to 1.
>
> * Note that I am using an Eclipse plugin to resolve the dependencies for
> me. Since you want to help I will make it a priority to create an ANT
> task to resolve them for us - I should be able to do this later today
> (GMT)
>
> Ross
>
>
> --
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.412 / Virus Database: 268.18.2/692 - Release Date: 2/18/2007
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
David Crossley wrote:
> Ross Gardler wrote:
>> Ross Gardler wrote:
>>> Here is my plan:
>>>
>>> 1 - I create a Forrest only repo in SVN (as I have started for
>>> Forrest2). although wherever possible we will use the IVY repo and the
>>> Maven2 repo
>>>
>>> 2 - I modify our build files to use IVY (I have a whole set of generic
>>> build files that are intended to be extended by project specific build
>>> files. I plan to give them to the IVY folk if they think it they are a
>>> good idea, if not we'll take what we need from them)
>>>
>>> 3 - We make sure it works
>>>
>>> 4 - We move the entries in our repo to the IVY repo (I need to enter
>>> into a dialogue with IVY devs to see what is happening with regards an
>>> official Apache repo)
>>>
...
>
> I want to assist. Some initial questions ...
Cool. As you have seen my work on this has frozen due to other
commitments. Should be getting back to it soon.
> I see that you have Ivy module descriptors, e.g.
> tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
>
> Are these descriptors something that the Forrest
> project maintains, or do we copy them from somewhere
> else and thus need to keep them synchronised? I want
> to clean them up to be consistent w.r.t. license stuff
> etc. but not if they will come from some other project.
> It seems to me that it is better to keep our own descriptors.
The whole tools/ivy/repository section dir really go to an ivy community
managed repo. The Ivy project has two, a scratchpad one where things may
not be perfectly described in terms of their dependencies etc. and a
main one where there are tight controls on what goes in there and so it
should be complete and accurate (i.e. stable).
So, my intention is that whatever we end up with in our local repo will
eventually be moved to the ivy scratchpad where the community can clean
it up an eventually move it into the main ivy repo.
Why not do it straight into the ivy scratchpad? Because I want Forrest
devs to see how the ivy files are put together so that there is a
history in our archives for people to manage the scratchpad/main reop
over in Ivy.
> Why xerces-2.5.0 when we currently use xerces-2.8.0?
I can only guess when I did the Forrest2 ivy repo I started it off by
pulling the latest available from a Maven repo. So I ended up with
xerces-2.5.0. I've not yet done the one in Forrest trunk so not yet
upgraded to 2.8.0.
> Is your plan to move everything that we use out of lib/*
> into this local tools/ivy/repository, then gradually
> remove/tune stuff as we find them in other official repositories?
Yes, see above.
> I gather that we can have Ivy descriptors in our local
> repository, but no jars, and then point to other repositories
> for the jars, e.g. maven repo.
We don't even need the descriptors. We just need the ivy.xml file in our
trunk. This tells Ivy what Forrest depends on. It goes off to the
defined repositories (defined in tools/ivy/ivyconf.xml) and finds the
relevant ivy.xml files for that dependency. This identifies any jars
that are required by that dependency, which allows Ivy to go looking for
them, and so on.
So, we are aiming at only needing to maintain Forrests ivy.xml file here
in Forrest.
----
Here's the process I'm using at present:
1. remove a jar from the lib directory
2. add a dependency to the forrest ivy.xml file
3. try resolving the dependencies* - since the jar is not in our local
repo it will go to the ivy and maven repos - it's supposed to go to the
Cocoon repos too, but I haven't got that working yet. I joined the ivy
user list to ask for advice but have not got around to it yet
4. if it resolves OK commit the changes, and go back to 1
5. if it fails to resolve we need to manually inspect the ivy repo to
look to see if it is available with a different organisation or jar name
(I don't bother doing this in the maven repos though
6. if it doesn't exist in the ivy repos then add the jar and ivy.xml to
our tools/ivy/repository
7. try resolving dependencies, it should now work, so go back to 1.
* Note that I am using an Eclipse plugin to resolve the dependencies for
me. Since you want to help I will make it a priority to create an ANT
task to resolve them for us - I should be able to do this later today (GMT)
Ross
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
> Ross Gardler wrote:
> >Here is my plan:
> >
> >1 - I create a Forrest only repo in SVN (as I have started for
> >Forrest2). although wherever possible we will use the IVY repo and the
> >Maven2 repo
> >
> >2 - I modify our build files to use IVY (I have a whole set of generic
> >build files that are intended to be extended by project specific build
> >files. I plan to give them to the IVY folk if they think it they are a
> >good idea, if not we'll take what we need from them)
> >
> >3 - We make sure it works
> >
> >4 - We move the entries in our repo to the IVY repo (I need to enter
> >into a dialogue with IVY devs to see what is happening with regards an
> >official Apache repo)
> >
> >The end result will be a build system that uses three repos:
> >
> >The forrest one for any jars we have customised but are not yet
> >available in the main project releases (i.e. we have something in the
> >patch queue)
> >
> >The official IVY repo (whatever that turns out to be)
> >
> >The official ASF Maven repo
>
> ...
>
> >If nobody raises an objection in the next seven days (I'm busy until
> >then) then I'll go ahead. Otherwise we can discuss and I'll call a vote
> >if necessary next week.
>
> I'm starting to do this work as of now. I don't have much time at
> present so I'll do it in a branch (called ivyBuild) since I don't want
> to stop others working if I get called away. Of course this also means
> that others have the opportunity to play before we move it to trunk.
I want to assist. Some initial questions ...
I see that you have Ivy module descriptors, e.g.
tools/ivy/repository/apache/xerces/ivy-2.5.0.xml
Are these descriptors something that the Forrest
project maintains, or do we copy them from somewhere
else and thus need to keep them synchronised? I want
to clean them up to be consistent w.r.t. license stuff
etc. but not if they will come from some other project.
It seems to me that it is better to keep our own descriptors.
Why xerces-2.5.0 when we currently use xerces-2.8.0?
Is your plan to move everything that we use out of lib/*
into this local tools/ivy/repository, then gradually
remove/tune stuff as we find them in other official repositories?
I gather that we can have Ivy descriptors in our local
repository, but no jars, and then point to other repositories
for the jars, e.g. maven repo.
-David
Re: [PROPOSAL] Switch dependency managment to IVY
Posted by Ross Gardler <rg...@apache.org>.
Ross Gardler wrote:
> Here is my plan:
>
> 1 - I create a Forrest only repo in SVN (as I have started for
> Forrest2). although wherever possible we will use the IVY repo and the
> Maven2 repo
>
> 2 - I modify our build files to use IVY (I have a whole set of generic
> build files that are intended to be extended by project specific build
> files. I plan to give them to the IVY folk if they think it they are a
> good idea, if not we'll take what we need from them)
>
> 3 - We make sure it works
>
> 4 - We move the entries in our repo to the IVY repo (I need to enter
> into a dialogue with IVY devs to see what is happening with regards an
> official Apache repo)
...
> If nobody raises an objection in the next seven days (I'm busy until
> then) then I'll go ahead. Otherwise we can discuss and I'll call a vote
> if necessary next week.
I'm starting to do this work as of now. I don't have much time at
present so I'll do it in a branch (called ivyBuild) since I don't want
to stop others working if I get called away. Of course this also means
that others have the opportunity to play before we move it to trunk.
It would be really helpful if commits can be made to this branch as
well, but this is certainly not a requirement unless they affect the
build system.
Ross