You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Guillaume Polet <gu...@gmail.com> on 2012/03/07 09:02:21 UTC
Maven concurrent builds
Dear all,
I was having some doubts regarding Maven's ability to handle properly
concurrent builds. Lately, I had several builds of our project that
failed and it seemed that it was due to interferences between concurrent
builds of the same version of the same project.
I have a jenkins server which have 6 different builds:
* 1 for continuous build (which runs tests but does not deploy the
artifacts)
* 1 for nightly build
* 1 for releases (or almost release, ie, the pom is still in SNAPSHOT
version but could become a release)
All this is multiplied by two, because I have two versions (on 2
different branches).
So I had 2 questions:
1. Can/Does Maven support concurrent builds of the same version of the
same project? If yes, do I need to do something to achieve this? If not,
is it something that Maven developers are considering?
2. Can/Does Maven support concurrent builds of different versions of the
same project?
Cheers,
Guillaume
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven concurrent builds
Posted by Guillaume Polet <gu...@gmail.com>.
Le 7/03/2012 13:52, Jörg Schaible a écrit :
> Guillaume Polet wrote:
>
>> OK, good to know. Somehow, I always assumed that Maven was actually
>> handling concurrency safely.
>> Too bad that we have to waste extra space to handle this, but at least
>> there are easy workarounds.
> If you use a repository manager in your intranet, you may simple wipe the
> local repo every time.
Yes but then my builds takes longer to perform because it needs to
download again all dependencies (and there are quite a few) plus all the
required plugins.
Thanks for all your comments, I learned something today?
Cheers,
Guillaume
>
> - Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven concurrent builds
Posted by Jörg Schaible <Jo...@scalaris.com>.
Guillaume Polet wrote:
> OK, good to know. Somehow, I always assumed that Maven was actually
> handling concurrency safely.
> Too bad that we have to waste extra space to handle this, but at least
> there are easy workarounds.
If you use a repository manager in your intranet, you may simple wipe the
local repo every time.
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven concurrent builds
Posted by Guillaume Polet <gu...@gmail.com>.
OK, good to know. Somehow, I always assumed that Maven was actually
handling concurrency safely.
Too bad that we have to waste extra space to handle this, but at least
there are easy workarounds.
Cheers
Guillaume
Le 7/03/2012 11:41, Srinath C a écrit :
> Always use a different local repositories for different projects in Jenkins.
> You can do this by setting the system property
> -Dmaven.local.repo={project_workspace_dir}
>
> Regards,
> Srinath.
>
>
> On Wed, Mar 7, 2012 at 1:59 PM, Jörg Schaible
> <Jo...@scalaris.com>wrote:
>
>> Hi Guillaume,
>>
>> Guillaume Polet wrote:
>>
>>> Dear all,
>>>
>>> I was having some doubts regarding Maven's ability to handle properly
>>> concurrent builds. Lately, I had several builds of our project that
>>> failed and it seemed that it was due to interferences between concurrent
>>> builds of the same version of the same project.
>>>
>>> I have a jenkins server which have 6 different builds:
>>> * 1 for continuous build (which runs tests but does not deploy the
>>> artifacts)
>>> * 1 for nightly build
>>> * 1 for releases (or almost release, ie, the pom is still in SNAPSHOT
>>> version but could become a release)
>>> All this is multiplied by two, because I have two versions (on 2
>>> different branches).
>>>
>>> So I had 2 questions:
>>> 1. Can/Does Maven support concurrent builds of the same version of the
>>> same project?
>> Not out of the box. Since by default both builds will use the same local
>> repository creating the same artifacts at same time, the artifacts in the
>> repository are likely to be corrupted.
>>
>>> If yes, do I need to do something to achieve this?
>> Use separate local repositories, it can be defined using separate
>> settings.xml files or by call with a system property. Actually I don't know
>> if Jenkins already supports something like this, because it's a general
>> problem, see below.
>>
>>> If not,
>>> is it something that Maven developers are considering?
>> It's not the first time this problem has been risen.
>>
>>> 2. Can/Does Maven support concurrent builds of different versions of the
>>> same project?
>> They may still corrupt the metadata.xml files, but it is less likely and
>> has
>> AFAICS less consequences.
>>
>> However, every concurrent build may corrupt a shared local repository,
>> because both builds may download the same artifacts at same time. A
>> scenario
>> where this might happen is when you upgrade versions (e.g. of plugins) in a
>> common master-pom and start two parallel builds then. Actually I am not
>> sure
>> if you're safe when running one Maven build in parallel mode. A repository
>> manager like Nexus might prevent this scenario by detecting the same
>> request, but again I am not sure if it does.
>>
>> BTW: This is also the reason why you should never share a local repository
>> with a network mount by multiple users.
>>
>>
>>> Cheers,
>>> Guillaume
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven concurrent builds
Posted by Guillaume Polet <gu...@gmail.com>.
Yeah, I just found out about this but it is in an advanced hidden by
default panel. I checked it for all my builds that could interfere with
each other. Seems kind of strange that this is not more obvious as not
checking it results unpredictable behaviour.
Sorry to the ML-users as this is getting of Maven and more on Jenkins. I
won't continue on the subject.
Thanks to everyone who answered, it really solved my issues.
Cheers,
Guillaume
Le 7/03/2012 16:24, Michael Haefele a écrit :
> Jenkins also has a setting "Use private Maven repository" in the build
> section of the job configuration which takes care of this for you.
>
> Mike
>
> On Wed, Mar 7, 2012 at 5:41 AM, Srinath C<sr...@gmail.com> wrote:
>> Always use a different local repositories for different projects in Jenkins.
>> You can do this by setting the system property
>> -Dmaven.local.repo={project_workspace_dir}
>>
>> Regards,
>> Srinath.
>>
>>
>> On Wed, Mar 7, 2012 at 1:59 PM, Jörg Schaible
>> <Jo...@scalaris.com>wrote:
>>
>>> Hi Guillaume,
>>>
>>> Guillaume Polet wrote:
>>>
>>>> Dear all,
>>>>
>>>> I was having some doubts regarding Maven's ability to handle properly
>>>> concurrent builds. Lately, I had several builds of our project that
>>>> failed and it seemed that it was due to interferences between concurrent
>>>> builds of the same version of the same project.
>>>>
>>>> I have a jenkins server which have 6 different builds:
>>>> * 1 for continuous build (which runs tests but does not deploy the
>>>> artifacts)
>>>> * 1 for nightly build
>>>> * 1 for releases (or almost release, ie, the pom is still in SNAPSHOT
>>>> version but could become a release)
>>>> All this is multiplied by two, because I have two versions (on 2
>>>> different branches).
>>>>
>>>> So I had 2 questions:
>>>> 1. Can/Does Maven support concurrent builds of the same version of the
>>>> same project?
>>> Not out of the box. Since by default both builds will use the same local
>>> repository creating the same artifacts at same time, the artifacts in the
>>> repository are likely to be corrupted.
>>>
>>>> If yes, do I need to do something to achieve this?
>>> Use separate local repositories, it can be defined using separate
>>> settings.xml files or by call with a system property. Actually I don't know
>>> if Jenkins already supports something like this, because it's a general
>>> problem, see below.
>>>
>>>> If not,
>>>> is it something that Maven developers are considering?
>>> It's not the first time this problem has been risen.
>>>
>>>> 2. Can/Does Maven support concurrent builds of different versions of the
>>>> same project?
>>> They may still corrupt the metadata.xml files, but it is less likely and
>>> has
>>> AFAICS less consequences.
>>>
>>> However, every concurrent build may corrupt a shared local repository,
>>> because both builds may download the same artifacts at same time. A
>>> scenario
>>> where this might happen is when you upgrade versions (e.g. of plugins) in a
>>> common master-pom and start two parallel builds then. Actually I am not
>>> sure
>>> if you're safe when running one Maven build in parallel mode. A repository
>>> manager like Nexus might prevent this scenario by detecting the same
>>> request, but again I am not sure if it does.
>>>
>>> BTW: This is also the reason why you should never share a local repository
>>> with a network mount by multiple users.
>>>
>>>
>>>> Cheers,
>>>> Guillaume
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven concurrent builds
Posted by Michael Haefele <mi...@parityenergy.com>.
Jenkins also has a setting "Use private Maven repository" in the build
section of the job configuration which takes care of this for you.
Mike
On Wed, Mar 7, 2012 at 5:41 AM, Srinath C <sr...@gmail.com> wrote:
> Always use a different local repositories for different projects in Jenkins.
> You can do this by setting the system property
> -Dmaven.local.repo={project_workspace_dir}
>
> Regards,
> Srinath.
>
>
> On Wed, Mar 7, 2012 at 1:59 PM, Jörg Schaible
> <Jo...@scalaris.com>wrote:
>
>> Hi Guillaume,
>>
>> Guillaume Polet wrote:
>>
>> > Dear all,
>> >
>> > I was having some doubts regarding Maven's ability to handle properly
>> > concurrent builds. Lately, I had several builds of our project that
>> > failed and it seemed that it was due to interferences between concurrent
>> > builds of the same version of the same project.
>> >
>> > I have a jenkins server which have 6 different builds:
>> > * 1 for continuous build (which runs tests but does not deploy the
>> > artifacts)
>> > * 1 for nightly build
>> > * 1 for releases (or almost release, ie, the pom is still in SNAPSHOT
>> > version but could become a release)
>> > All this is multiplied by two, because I have two versions (on 2
>> > different branches).
>> >
>> > So I had 2 questions:
>> > 1. Can/Does Maven support concurrent builds of the same version of the
>> > same project?
>>
>> Not out of the box. Since by default both builds will use the same local
>> repository creating the same artifacts at same time, the artifacts in the
>> repository are likely to be corrupted.
>>
>> > If yes, do I need to do something to achieve this?
>>
>> Use separate local repositories, it can be defined using separate
>> settings.xml files or by call with a system property. Actually I don't know
>> if Jenkins already supports something like this, because it's a general
>> problem, see below.
>>
>> > If not,
>> > is it something that Maven developers are considering?
>>
>> It's not the first time this problem has been risen.
>>
>> > 2. Can/Does Maven support concurrent builds of different versions of the
>> > same project?
>>
>> They may still corrupt the metadata.xml files, but it is less likely and
>> has
>> AFAICS less consequences.
>>
>> However, every concurrent build may corrupt a shared local repository,
>> because both builds may download the same artifacts at same time. A
>> scenario
>> where this might happen is when you upgrade versions (e.g. of plugins) in a
>> common master-pom and start two parallel builds then. Actually I am not
>> sure
>> if you're safe when running one Maven build in parallel mode. A repository
>> manager like Nexus might prevent this scenario by detecting the same
>> request, but again I am not sure if it does.
>>
>> BTW: This is also the reason why you should never share a local repository
>> with a network mount by multiple users.
>>
>>
>> >
>> > Cheers,
>> > Guillaume
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven concurrent builds
Posted by Srinath C <sr...@gmail.com>.
Always use a different local repositories for different projects in Jenkins.
You can do this by setting the system property
-Dmaven.local.repo={project_workspace_dir}
Regards,
Srinath.
On Wed, Mar 7, 2012 at 1:59 PM, Jörg Schaible
<Jo...@scalaris.com>wrote:
> Hi Guillaume,
>
> Guillaume Polet wrote:
>
> > Dear all,
> >
> > I was having some doubts regarding Maven's ability to handle properly
> > concurrent builds. Lately, I had several builds of our project that
> > failed and it seemed that it was due to interferences between concurrent
> > builds of the same version of the same project.
> >
> > I have a jenkins server which have 6 different builds:
> > * 1 for continuous build (which runs tests but does not deploy the
> > artifacts)
> > * 1 for nightly build
> > * 1 for releases (or almost release, ie, the pom is still in SNAPSHOT
> > version but could become a release)
> > All this is multiplied by two, because I have two versions (on 2
> > different branches).
> >
> > So I had 2 questions:
> > 1. Can/Does Maven support concurrent builds of the same version of the
> > same project?
>
> Not out of the box. Since by default both builds will use the same local
> repository creating the same artifacts at same time, the artifacts in the
> repository are likely to be corrupted.
>
> > If yes, do I need to do something to achieve this?
>
> Use separate local repositories, it can be defined using separate
> settings.xml files or by call with a system property. Actually I don't know
> if Jenkins already supports something like this, because it's a general
> problem, see below.
>
> > If not,
> > is it something that Maven developers are considering?
>
> It's not the first time this problem has been risen.
>
> > 2. Can/Does Maven support concurrent builds of different versions of the
> > same project?
>
> They may still corrupt the metadata.xml files, but it is less likely and
> has
> AFAICS less consequences.
>
> However, every concurrent build may corrupt a shared local repository,
> because both builds may download the same artifacts at same time. A
> scenario
> where this might happen is when you upgrade versions (e.g. of plugins) in a
> common master-pom and start two parallel builds then. Actually I am not
> sure
> if you're safe when running one Maven build in parallel mode. A repository
> manager like Nexus might prevent this scenario by detecting the same
> request, but again I am not sure if it does.
>
> BTW: This is also the reason why you should never share a local repository
> with a network mount by multiple users.
>
>
> >
> > Cheers,
> > Guillaume
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: Maven concurrent builds
Posted by Jörg Schaible <Jo...@scalaris.com>.
Hi Guillaume,
Guillaume Polet wrote:
> Dear all,
>
> I was having some doubts regarding Maven's ability to handle properly
> concurrent builds. Lately, I had several builds of our project that
> failed and it seemed that it was due to interferences between concurrent
> builds of the same version of the same project.
>
> I have a jenkins server which have 6 different builds:
> * 1 for continuous build (which runs tests but does not deploy the
> artifacts)
> * 1 for nightly build
> * 1 for releases (or almost release, ie, the pom is still in SNAPSHOT
> version but could become a release)
> All this is multiplied by two, because I have two versions (on 2
> different branches).
>
> So I had 2 questions:
> 1. Can/Does Maven support concurrent builds of the same version of the
> same project?
Not out of the box. Since by default both builds will use the same local
repository creating the same artifacts at same time, the artifacts in the
repository are likely to be corrupted.
> If yes, do I need to do something to achieve this?
Use separate local repositories, it can be defined using separate
settings.xml files or by call with a system property. Actually I don't know
if Jenkins already supports something like this, because it's a general
problem, see below.
> If not,
> is it something that Maven developers are considering?
It's not the first time this problem has been risen.
> 2. Can/Does Maven support concurrent builds of different versions of the
> same project?
They may still corrupt the metadata.xml files, but it is less likely and has
AFAICS less consequences.
However, every concurrent build may corrupt a shared local repository,
because both builds may download the same artifacts at same time. A scenario
where this might happen is when you upgrade versions (e.g. of plugins) in a
common master-pom and start two parallel builds then. Actually I am not sure
if you're safe when running one Maven build in parallel mode. A repository
manager like Nexus might prevent this scenario by detecting the same
request, but again I am not sure if it does.
BTW: This is also the reason why you should never share a local repository
with a network mount by multiple users.
>
> Cheers,
> Guillaume
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org