You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Willem Verstraeten <wi...@gmail.com> on 2009/08/28 16:30:35 UTC
How to set the resolve ID for an inline resolve
Hello,
How can you set a resolveId that depends on the properties of the
module to be resolved ?
Ie. I am trying to do something like this:
<ivy:retrieve inline="true"
organisation="com.mycorp"
module="common"
type="buildscript"
conf="build"
resolveid="[org]-[module]-[branch]-${ant.project.name}"
pattern="${common.dir}/[artifact].[ext]"/>
The resolveId is not treated as a pattern though, so the resolveId is
the literal [org]-[module]-[branch]-project_name, iso
com.mycorp-common-development-project_name.
Now for the [org] and [module] parts I can obviously fill them in as a
literal, because I specify them as literals anyway. However, the
branch is determined by the settings file, so I'd like to avoid
duplicating that logic here.
Also, when I would not do an inline resolve (ie. resolve based on an
ivy file) I guess I would need this functionality as well, as I don't
have the literal information available then.
All this would be necessary to support concurrent building of several
modules which all start with this piece. Ie. I have a multi-module
setup, and the build scripts all use the above retrieve task to get
the common parts of the build script. If two modules are being built
at the same time (which is quite likely when you have a continuous
integration server), you get the possibility that two build scripts
are resolving the same module at the same time with the same
resolveId, leading to concurrency problems.
Re: How to set the resolve ID for an inline resolve
Posted by Willem Verstraeten <wi...@gmail.com>.
Hi Joshua,
Thanks for the feedback. This seems to be a workable alternative indeed.
I also figured out in the meanwhile that you can use the <ivy:info/>
task to load all the properties I needed to construct a unique
resolveId, so that solved my problem as well.
It would still make sense that the resolveId is treated as a pattern
and that the default resolveId includes the branch name, so I'm
leaving IVY-1116 open for the moment.
Thanks again,
Willem
2009/9/1 Joshua Tharp <jo...@alumni.calpoly.edu>:
> In case you hadn't already figured this out I added this to the advanced
> settings for each ant build step (that requires Ivy at some point) in
> Hudson:
> ivy.default.ivy.user.dir=${WORKSPACE}/ivy
>
> I've built up a repository inside our corporate firewall, so it isn't that
> expensive for my build server to have to go out and re-acquire dependencies.
>
> Josh
>
> On Mon, Aug 31, 2009 at 12:26 PM, Joshua Tharp <
> joshua-tharp@alumni.calpoly.edu> wrote:
>
>> The way I solved this (and a host of other issues) is by specifying
>> that the ivy directory be in the Hudson project workspace (I actually
>> started the practice with CruiseControl). It is a less efficient at
>> build time because the cache is isolated for each build, but the trade
>> off is that I don't have to worry about any concuurency issues in the
>> cache, nor any side effects between builds. I can't get at my Hudson
>> server right at the moment, but if you need to know what variable I
>> set, please ask.
>>
>> Josh
>>
>> On Monday, August 31, 2009, Willem Verstraeten
>> <wi...@gmail.com> wrote:
>> > I'm using the Hudson CI server.
>> >
>> > The thing is that, if I don't set the resolveId, I can have two
>> > projects which are performing the exact same resolve.
>> >
>> > Ie. project A is retrieving, inline, the '[com.mycorp]-[common]'
>> > module, and project B as well, possibly at the same time. Now the
>> > resolveId for both these resolves is the [com.mycorp]-[common-caller],
>> > as they are both inline resolves. Since they have the same resolveId,
>> > they also use the same files in the ~/.ivy2/cache directory, which
>> > leads to errors when they both run at the same time.
>> >
>> > So I'd like to set a resolveId that is dependent on the
>> > http://ant.project.name. I'd also like the resolveId to depend on the
>> ivy
>> > branch, as, again, it is possible that the same project is built for
>> > two branches simultaneously, resulting in the same resolveId again if
>> > the branch is not part of the resolveId.
>> >
>> > So, in short, is there a way to specify in the ant file a resolveId
>> > for an inline retrieve that depends on the dynamic Ivy properties such
>> > as org, rev and branch ?
>> >
>> > If not, is there an alternative way in which I can prevent the
>> > concurrency issues described above ?
>> >
>> > I hope this makes my problem a bit clearer.
>> >
>> >> Joshua Tarp wrote:
>> >>
>> >> What CI build server are you using? I've never heard of cross-Ant
>> >> concurrency issues that didn't involve the filesystem. I thought most
>> >> if not all forked Ant.
>> >
>> >> On Friday, August 28, 2009, Willem Verstraeten
>> >> <wi...@gmail.com> wrote:
>> >> > Hello,
>> >> >
>> >> > How can you set a resolveId that depends on the properties of the
>> >> > module to be resolved ?
>> >> >
>> >> > Ie. I am trying to do something like this:
>> >> >
>> >> >
>> >> > <ivy:retrieve inline="true"
>> >> > organisation="com.mycorp"
>> >> > module="common"
>> >> > type="buildscript"
>> >> > conf="build"
>> >> > resolveid="[org]-[module]-[branch]-${
>> http://ant.project.name}";
>> >> > pattern="${common.dir}/[artifact].[ext]"/>
>> >> >
>> >> > The resolveId is not treated as a pattern though, so the resolveId is
>> >> > the literal [org]-[module]-[branch]-project_name, iso
>> >> > com.mycorp-common-development-project_name.
>> >> >
>> >> > Now for the [org] and [module] parts I can obviously fill them in as a
>> >> > literal, because I specify them as literals anyway. However, the
>> >> > branch is determined by the settings file, so I'd like to avoid
>> >> > duplicating that logic here.
>> >> >
>> >> > Also, when I would not do an inline resolve (ie. resolve based on an
>> >> > ivy file) I guess I would need this functionality as well, as I don't
>> >> > have the literal information available then.
>> >> >
>> >> > All this would be necessary to support concurrent building of several
>> >> > modules which all start with this piece. Ie. I have a multi-module
>> >> > setup, and the build scripts all use the above retrieve task to get
>> >> > the common parts of the build script. If two modules are being built
>> >> > at the same time (which is quite likely when you have a continuous
>> >> > integration server), you get the possibility that two build scripts
>> >> > are resolving the same module at the same time with the same
>> >> > resolveId, leading to concurrency problems.
>> >>
>> >
>>
>
Re: How to set the resolve ID for an inline resolve
Posted by Joshua Tharp <jo...@alumni.calpoly.edu>.
In case you hadn't already figured this out I added this to the advanced
settings for each ant build step (that requires Ivy at some point) in
Hudson:
ivy.default.ivy.user.dir=${WORKSPACE}/ivy
I've built up a repository inside our corporate firewall, so it isn't that
expensive for my build server to have to go out and re-acquire dependencies.
Josh
On Mon, Aug 31, 2009 at 12:26 PM, Joshua Tharp <
joshua-tharp@alumni.calpoly.edu> wrote:
> The way I solved this (and a host of other issues) is by specifying
> that the ivy directory be in the Hudson project workspace (I actually
> started the practice with CruiseControl). It is a less efficient at
> build time because the cache is isolated for each build, but the trade
> off is that I don't have to worry about any concuurency issues in the
> cache, nor any side effects between builds. I can't get at my Hudson
> server right at the moment, but if you need to know what variable I
> set, please ask.
>
> Josh
>
> On Monday, August 31, 2009, Willem Verstraeten
> <wi...@gmail.com> wrote:
> > I'm using the Hudson CI server.
> >
> > The thing is that, if I don't set the resolveId, I can have two
> > projects which are performing the exact same resolve.
> >
> > Ie. project A is retrieving, inline, the '[com.mycorp]-[common]'
> > module, and project B as well, possibly at the same time. Now the
> > resolveId for both these resolves is the [com.mycorp]-[common-caller],
> > as they are both inline resolves. Since they have the same resolveId,
> > they also use the same files in the ~/.ivy2/cache directory, which
> > leads to errors when they both run at the same time.
> >
> > So I'd like to set a resolveId that is dependent on the
> > http://ant.project.name. I'd also like the resolveId to depend on the
> ivy
> > branch, as, again, it is possible that the same project is built for
> > two branches simultaneously, resulting in the same resolveId again if
> > the branch is not part of the resolveId.
> >
> > So, in short, is there a way to specify in the ant file a resolveId
> > for an inline retrieve that depends on the dynamic Ivy properties such
> > as org, rev and branch ?
> >
> > If not, is there an alternative way in which I can prevent the
> > concurrency issues described above ?
> >
> > I hope this makes my problem a bit clearer.
> >
> >> Joshua Tarp wrote:
> >>
> >> What CI build server are you using? I've never heard of cross-Ant
> >> concurrency issues that didn't involve the filesystem. I thought most
> >> if not all forked Ant.
> >
> >> On Friday, August 28, 2009, Willem Verstraeten
> >> <wi...@gmail.com> wrote:
> >> > Hello,
> >> >
> >> > How can you set a resolveId that depends on the properties of the
> >> > module to be resolved ?
> >> >
> >> > Ie. I am trying to do something like this:
> >> >
> >> >
> >> > <ivy:retrieve inline="true"
> >> > organisation="com.mycorp"
> >> > module="common"
> >> > type="buildscript"
> >> > conf="build"
> >> > resolveid="[org]-[module]-[branch]-${
> http://ant.project.name}";
> >> > pattern="${common.dir}/[artifact].[ext]"/>
> >> >
> >> > The resolveId is not treated as a pattern though, so the resolveId is
> >> > the literal [org]-[module]-[branch]-project_name, iso
> >> > com.mycorp-common-development-project_name.
> >> >
> >> > Now for the [org] and [module] parts I can obviously fill them in as a
> >> > literal, because I specify them as literals anyway. However, the
> >> > branch is determined by the settings file, so I'd like to avoid
> >> > duplicating that logic here.
> >> >
> >> > Also, when I would not do an inline resolve (ie. resolve based on an
> >> > ivy file) I guess I would need this functionality as well, as I don't
> >> > have the literal information available then.
> >> >
> >> > All this would be necessary to support concurrent building of several
> >> > modules which all start with this piece. Ie. I have a multi-module
> >> > setup, and the build scripts all use the above retrieve task to get
> >> > the common parts of the build script. If two modules are being built
> >> > at the same time (which is quite likely when you have a continuous
> >> > integration server), you get the possibility that two build scripts
> >> > are resolving the same module at the same time with the same
> >> > resolveId, leading to concurrency problems.
> >>
> >
>
Re: How to set the resolve ID for an inline resolve
Posted by Joshua Tharp <jo...@alumni.calpoly.edu>.
The way I solved this (and a host of other issues) is by specifying
that the ivy directory be in the Hudson project workspace (I actually
started the practice with CruiseControl). It is a less efficient at
build time because the cache is isolated for each build, but the trade
off is that I don't have to worry about any concuurency issues in the
cache, nor any side effects between builds. I can't get at my Hudson
server right at the moment, but if you need to know what variable I
set, please ask.
Josh
On Monday, August 31, 2009, Willem Verstraeten
<wi...@gmail.com> wrote:
> I'm using the Hudson CI server.
>
> The thing is that, if I don't set the resolveId, I can have two
> projects which are performing the exact same resolve.
>
> Ie. project A is retrieving, inline, the '[com.mycorp]-[common]'
> module, and project B as well, possibly at the same time. Now the
> resolveId for both these resolves is the [com.mycorp]-[common-caller],
> as they are both inline resolves. Since they have the same resolveId,
> they also use the same files in the ~/.ivy2/cache directory, which
> leads to errors when they both run at the same time.
>
> So I'd like to set a resolveId that is dependent on the
> http://ant.project.name. I'd also like the resolveId to depend on the ivy
> branch, as, again, it is possible that the same project is built for
> two branches simultaneously, resulting in the same resolveId again if
> the branch is not part of the resolveId.
>
> So, in short, is there a way to specify in the ant file a resolveId
> for an inline retrieve that depends on the dynamic Ivy properties such
> as org, rev and branch ?
>
> If not, is there an alternative way in which I can prevent the
> concurrency issues described above ?
>
> I hope this makes my problem a bit clearer.
>
>> Joshua Tarp wrote:
>>
>> What CI build server are you using? I've never heard of cross-Ant
>> concurrency issues that didn't involve the filesystem. I thought most
>> if not all forked Ant.
>
>> On Friday, August 28, 2009, Willem Verstraeten
>> <wi...@gmail.com> wrote:
>> > Hello,
>> >
>> > How can you set a resolveId that depends on the properties of the
>> > module to be resolved ?
>> >
>> > Ie. I am trying to do something like this:
>> >
>> >
>> > <ivy:retrieve inline="true"
>> > organisation="com.mycorp"
>> > module="common"
>> > type="buildscript"
>> > conf="build"
>> > resolveid="[org]-[module]-[branch]-${http://ant.project.name}";
>> > pattern="${common.dir}/[artifact].[ext]"/>
>> >
>> > The resolveId is not treated as a pattern though, so the resolveId is
>> > the literal [org]-[module]-[branch]-project_name, iso
>> > com.mycorp-common-development-project_name.
>> >
>> > Now for the [org] and [module] parts I can obviously fill them in as a
>> > literal, because I specify them as literals anyway. However, the
>> > branch is determined by the settings file, so I'd like to avoid
>> > duplicating that logic here.
>> >
>> > Also, when I would not do an inline resolve (ie. resolve based on an
>> > ivy file) I guess I would need this functionality as well, as I don't
>> > have the literal information available then.
>> >
>> > All this would be necessary to support concurrent building of several
>> > modules which all start with this piece. Ie. I have a multi-module
>> > setup, and the build scripts all use the above retrieve task to get
>> > the common parts of the build script. If two modules are being built
>> > at the same time (which is quite likely when you have a continuous
>> > integration server), you get the possibility that two build scripts
>> > are resolving the same module at the same time with the same
>> > resolveId, leading to concurrency problems.
>>
>
Re: How to set the resolve ID for an inline resolve
Posted by Willem Verstraeten <wi...@gmail.com>.
I'm using the Hudson CI server.
The thing is that, if I don't set the resolveId, I can have two
projects which are performing the exact same resolve.
Ie. project A is retrieving, inline, the '[com.mycorp]-[common]'
module, and project B as well, possibly at the same time. Now the
resolveId for both these resolves is the [com.mycorp]-[common-caller],
as they are both inline resolves. Since they have the same resolveId,
they also use the same files in the ~/.ivy2/cache directory, which
leads to errors when they both run at the same time.
So I'd like to set a resolveId that is dependent on the
ant.project.name. I'd also like the resolveId to depend on the ivy
branch, as, again, it is possible that the same project is built for
two branches simultaneously, resulting in the same resolveId again if
the branch is not part of the resolveId.
So, in short, is there a way to specify in the ant file a resolveId
for an inline retrieve that depends on the dynamic Ivy properties such
as org, rev and branch ?
If not, is there an alternative way in which I can prevent the
concurrency issues described above ?
I hope this makes my problem a bit clearer.
> Joshua Tarp wrote:
>
> What CI build server are you using? I've never heard of cross-Ant
> concurrency issues that didn't involve the filesystem. I thought most
> if not all forked Ant.
> On Friday, August 28, 2009, Willem Verstraeten
> <wi...@gmail.com> wrote:
> > Hello,
> >
> > How can you set a resolveId that depends on the properties of the
> > module to be resolved ?
> >
> > Ie. I am trying to do something like this:
> >
> >
> > <ivy:retrieve inline="true"
> > organisation="com.mycorp"
> > module="common"
> > type="buildscript"
> > conf="build"
> > resolveid="[org]-[module]-[branch]-${http://ant.project.name}";
> > pattern="${common.dir}/[artifact].[ext]"/>
> >
> > The resolveId is not treated as a pattern though, so the resolveId is
> > the literal [org]-[module]-[branch]-project_name, iso
> > com.mycorp-common-development-project_name.
> >
> > Now for the [org] and [module] parts I can obviously fill them in as a
> > literal, because I specify them as literals anyway. However, the
> > branch is determined by the settings file, so I'd like to avoid
> > duplicating that logic here.
> >
> > Also, when I would not do an inline resolve (ie. resolve based on an
> > ivy file) I guess I would need this functionality as well, as I don't
> > have the literal information available then.
> >
> > All this would be necessary to support concurrent building of several
> > modules which all start with this piece. Ie. I have a multi-module
> > setup, and the build scripts all use the above retrieve task to get
> > the common parts of the build script. If two modules are being built
> > at the same time (which is quite likely when you have a continuous
> > integration server), you get the possibility that two build scripts
> > are resolving the same module at the same time with the same
> > resolveId, leading to concurrency problems.
>
Re: How to set the resolve ID for an inline resolve
Posted by Joshua Tharp <jo...@alumni.calpoly.edu>.
What CI build server are you using? I've never heard of cross-Ant
concurrency issues that didn't involve the filesystem. I thought most
if not all forked Ant.
On Friday, August 28, 2009, Willem Verstraeten
<wi...@gmail.com> wrote:
> Hello,
>
> How can you set a resolveId that depends on the properties of the
> module to be resolved ?
>
> Ie. I am trying to do something like this:
>
>
> <ivy:retrieve inline="true"
> organisation="com.mycorp"
> module="common"
> type="buildscript"
> conf="build"
> resolveid="[org]-[module]-[branch]-${http://ant.project.name}"
> pattern="${common.dir}/[artifact].[ext]"/>
>
> The resolveId is not treated as a pattern though, so the resolveId is
> the literal [org]-[module]-[branch]-project_name, iso
> com.mycorp-common-development-project_name.
>
> Now for the [org] and [module] parts I can obviously fill them in as a
> literal, because I specify them as literals anyway. However, the
> branch is determined by the settings file, so I'd like to avoid
> duplicating that logic here.
>
> Also, when I would not do an inline resolve (ie. resolve based on an
> ivy file) I guess I would need this functionality as well, as I don't
> have the literal information available then.
>
> All this would be necessary to support concurrent building of several
> modules which all start with this piece. Ie. I have a multi-module
> setup, and the build scripts all use the above retrieve task to get
> the common parts of the build script. If two modules are being built
> at the same time (which is quite likely when you have a continuous
> integration server), you get the possibility that two build scripts
> are resolving the same module at the same time with the same
> resolveId, leading to concurrency problems.
>