You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Konrad Windszus <kw...@apache.org> on 2022/06/21 07:50:50 UTC
Named Lock implementation for Maven with Jenkins
Hi,
By default Jenkins uses a global Maven repository which is accessed in parallel by multiple jobs running in dedicated VMs. To prevent race conditions one should probably configure one of the distributed named locks outlined in https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
Is there any recommendation which one to pick and also some concrete hints how to set that up with Jenkins (Pipelines, leveraging https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
Is ASF Jenkins using named locks already?
Thanks for any pointers
Konrad
Re: Named Lock implementation for Maven with Jenkins
Posted by "Oliver B. Fischer" <o....@swe-blog.net>.
Hi,
Am 21.06.22 um 22:31 schrieb mfriedenhagen@gmx.de:
> I have made no experience. We do our Maven builds in Docker containers, the .m2 directory is prefilled in the image will all dependencies managed in our company pom, which is used by most of our applications because we ran into problems with shared repositories. Before we used the sharded approach suggested by Lasse. Hard disks are not that expensive nowadays.
I really recommand you, dependening on the type of agents you use,
always also to have a kind of a clean-up policy to delete or discard you
local repository. One reason is that even if disk space is cheep it is
finite. I have seen many CI systems failing becaus of missing
diskspace. Resources are finite.
Oliver
--
N Oliver B. Fischer
A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
P +49 30 44793251
M +49 178 7903538
E o.b.fischer@swe-blog.net
S oliver.b.fischer
J oliver.b.fischer@jabber.org
X http://xing.to/obf
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Re: Named Lock implementation for Maven with Jenkins
Posted by mf...@gmx.de.
Hi Konrad,
I have made no experience. We do our Maven builds in Docker containers, the .m2 directory is prefilled in the image will all dependencies managed in our company pom, which is used by most of our applications because we ran into problems with shared repositories. Before we used the sharded approach suggested by Lasse. Hard disks are not that expensive nowadays.
Best Regards
Mirko Friedenhagen
—
Sent from my mobile
Am 21.06.22 um 12:01 schrieb Konrad Windszus
> Hi Mirko,
> Right, but isn’t “file-lock” relying internally on https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html which says: "This file-locking API is intended to map directly to the native locking facility of the underlying operating system. Thus the locks held on a file should be visible to all programs that have access to the file, regardless of the language in which those programs are written.”
>
> Therefore I assume “file-lock” should work also cross VMs or is that assumption not correct?
>
> Konrad
>
> > On 21. Jun 2022, at 11:57, mfriedenhagen@gmx.de wrote:
> >
> > Hi Konrad,
> >
> > citing the linked web site: "Local named locks are only suited within one JVM with a multithreaded build. Sharing a local repository between multiple Maven processes (i.e., on a busy CI server) requires a distributed named lock!"
> >
> > Best Regards
> > Mirko Friedenhagen
> > —
> > Sent from my mobile
> >
> > Am 21.06.22 um 10:20 schrieb Konrad Windszus
> >
> >> Thanks for that suggestion. I know about that workaround. But to speed up builds and to reduce disk space usage I would really like to use a shared repo.
> >>
> >> I was wondering if there is any experience with the simple "file-lock” [1]. That has the advantage that it doesn’t need additional libraries (IIUC). Configuration should be as easy as setting “aether.syncContext.named.factory” to “file-lock” [2]. Unfortunately even Maven 3.8.6 still ships with Resolver 1.6.3 [3] and it seems the suggested branch https://github.com/apache/maven/commits/maven-3.8.x-resolver-1.7.x is a bit outdated now…
> >>
> >> Konrad
> >>
> >> [1] - https://github.com/apache/maven-resolver/blob/2af7bff2a5238bf2597cd7a46b60a12779ee31a3/maven-resolver-named-locks/src/main/java/org/eclipse/aether/named/providers/FileLockNamedLockFactory.java
> >> [2] - https://maven.apache.org/resolver/configuration.html
> >> [3] - https://maven.apache.org/components/resolver/maven-3.8.x.html
> >>
> >>
> >>> On 21. Jun 2022, at 10:01, Lasse Lindqvist <la...@gmail.com> wrote:
> >>>
> >>> Hi. If you want to have pipeline build safely, you also have an
> >>> alternative. With the withMaven clause (Pipeline Maven Integration Plugin)
> >>> you can use something like
> >>>
> >>>
> >>>
> >>> *withMaven(maven: 'Maven123', mavenLocalRepo:
> >>> '$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
> >>> commands here*
> >>> * }*
> >>> It will use the workspace folder, travel out of it to a parent location and
> >>> make a sibling .m2 folder and unique subfolder there based on the executor
> >>> number. This way the folders used are executor specific and there will be
> >>> no concurrency problems. The downside is that it will use more disk space
> >>> because many artifacts are stored multiple times.
> >>>
> >>> ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org) kirjoitti:
> >>>
> >>>> Hi,
> >>>> By default Jenkins uses a global Maven repository which is accessed in
> >>>> parallel by multiple jobs running in dedicated VMs. To prevent race
> >>>> conditions one should probably configure one of the distributed named locks
> >>>> outlined in
> >>>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
> >>>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
> >>>> Is there any recommendation which one to pick and also some concrete hints
> >>>> how to set that up with Jenkins (Pipelines, leveraging
> >>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
> >>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
> >>>>
> >>>> Is ASF Jenkins using named locks already?
> >>>> Thanks for any pointers
> >>>>
> >>>> Konrad
> >>>>
> >>>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: users-help@maven.apache.org
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Named Lock implementation for Maven with Jenkins
Posted by Tamás Cservenák <ta...@cservenak.net>.
Actually, maven daemon since 0.8.0 uses file-lock to protect shared local
repository from multiple daemon accesses.
T
On Tue, Jun 21, 2022 at 12:36 PM Tamás Cservenák <ta...@cservenak.net>
wrote:
> Correct, file-lock does work cross-VM,
>
> T
>
> On Tue, Jun 21, 2022 at 12:01 PM Konrad Windszus <ko...@gmx.de> wrote:
>
>> Hi Mirko,
>> Right, but isn’t “file-lock” relying internally on
>> https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html
>> which says: "This file-locking API is intended to map directly to the
>> native locking facility of the underlying operating system. Thus the locks
>> held on a file should be visible to all programs that have access to the
>> file, regardless of the language in which those programs are written.”
>>
>> Therefore I assume “file-lock” should work also cross VMs or is that
>> assumption not correct?
>>
>> Konrad
>>
>> > On 21. Jun 2022, at 11:57, mfriedenhagen@gmx.de wrote:
>> >
>> > Hi Konrad,
>> >
>> > citing the linked web site: "Local named locks are only suited within
>> one JVM with a multithreaded build. Sharing a local repository between
>> multiple Maven processes (i.e., on a busy CI server) requires a distributed
>> named lock!"
>> >
>> > Best Regards
>> > Mirko Friedenhagen
>> > —
>> > Sent from my mobile
>> >
>> > Am 21.06.22 um 10:20 schrieb Konrad Windszus
>> >
>> >> Thanks for that suggestion. I know about that workaround. But to speed
>> up builds and to reduce disk space usage I would really like to use a
>> shared repo.
>> >>
>> >> I was wondering if there is any experience with the simple "file-lock”
>> [1]. That has the advantage that it doesn’t need additional libraries
>> (IIUC). Configuration should be as easy as setting
>> “aether.syncContext.named.factory” to “file-lock” [2]. Unfortunately even
>> Maven 3.8.6 still ships with Resolver 1.6.3 [3] and it seems the suggested
>> branch https://github.com/apache/maven/commits/maven-3.8.x-resolver-1.7.x
>> is a bit outdated now…
>> >>
>> >> Konrad
>> >>
>> >> [1] -
>> https://github.com/apache/maven-resolver/blob/2af7bff2a5238bf2597cd7a46b60a12779ee31a3/maven-resolver-named-locks/src/main/java/org/eclipse/aether/named/providers/FileLockNamedLockFactory.java
>> >> [2] - https://maven.apache.org/resolver/configuration.html
>> >> [3] - https://maven.apache.org/components/resolver/maven-3.8.x.html
>> >>
>> >>
>> >>> On 21. Jun 2022, at 10:01, Lasse Lindqvist <
>> lasse.k.lindqvist@gmail.com> wrote:
>> >>>
>> >>> Hi. If you want to have pipeline build safely, you also have an
>> >>> alternative. With the withMaven clause (Pipeline Maven Integration
>> Plugin)
>> >>> you can use something like
>> >>>
>> >>>
>> >>>
>> >>> *withMaven(maven: 'Maven123', mavenLocalRepo:
>> >>> '$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
>> >>> commands here*
>> >>> * }*
>> >>> It will use the workspace folder, travel out of it to a parent
>> location and
>> >>> make a sibling .m2 folder and unique subfolder there based on the
>> executor
>> >>> number. This way the folders used are executor specific and there
>> will be
>> >>> no concurrency problems. The downside is that it will use more disk
>> space
>> >>> because many artifacts are stored multiple times.
>> >>>
>> >>> ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org)
>> kirjoitti:
>> >>>
>> >>>> Hi,
>> >>>> By default Jenkins uses a global Maven repository which is accessed
>> in
>> >>>> parallel by multiple jobs running in dedicated VMs. To prevent race
>> >>>> conditions one should probably configure one of the distributed
>> named locks
>> >>>> outlined in
>> >>>>
>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
>> >>>>
>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
>> >>>> Is there any recommendation which one to pick and also some concrete
>> hints
>> >>>> how to set that up with Jenkins (Pipelines, leveraging
>> >>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
>> >>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
>> >>>>
>> >>>> Is ASF Jenkins using named locks already?
>> >>>> Thanks for any pointers
>> >>>>
>> >>>> Konrad
>> >>>>
>> >>>>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> >> For additional commands, e-mail: users-help@maven.apache.org
>> >>
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> > For additional commands, e-mail: users-help@maven.apache.org
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
Re: Named Lock implementation for Maven with Jenkins
Posted by Tamás Cservenák <ta...@cservenak.net>.
Correct, file-lock does work cross-VM,
T
On Tue, Jun 21, 2022 at 12:01 PM Konrad Windszus <ko...@gmx.de> wrote:
> Hi Mirko,
> Right, but isn’t “file-lock” relying internally on
> https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html
> which says: "This file-locking API is intended to map directly to the
> native locking facility of the underlying operating system. Thus the locks
> held on a file should be visible to all programs that have access to the
> file, regardless of the language in which those programs are written.”
>
> Therefore I assume “file-lock” should work also cross VMs or is that
> assumption not correct?
>
> Konrad
>
> > On 21. Jun 2022, at 11:57, mfriedenhagen@gmx.de wrote:
> >
> > Hi Konrad,
> >
> > citing the linked web site: "Local named locks are only suited within
> one JVM with a multithreaded build. Sharing a local repository between
> multiple Maven processes (i.e., on a busy CI server) requires a distributed
> named lock!"
> >
> > Best Regards
> > Mirko Friedenhagen
> > —
> > Sent from my mobile
> >
> > Am 21.06.22 um 10:20 schrieb Konrad Windszus
> >
> >> Thanks for that suggestion. I know about that workaround. But to speed
> up builds and to reduce disk space usage I would really like to use a
> shared repo.
> >>
> >> I was wondering if there is any experience with the simple "file-lock”
> [1]. That has the advantage that it doesn’t need additional libraries
> (IIUC). Configuration should be as easy as setting
> “aether.syncContext.named.factory” to “file-lock” [2]. Unfortunately even
> Maven 3.8.6 still ships with Resolver 1.6.3 [3] and it seems the suggested
> branch https://github.com/apache/maven/commits/maven-3.8.x-resolver-1.7.x
> is a bit outdated now…
> >>
> >> Konrad
> >>
> >> [1] -
> https://github.com/apache/maven-resolver/blob/2af7bff2a5238bf2597cd7a46b60a12779ee31a3/maven-resolver-named-locks/src/main/java/org/eclipse/aether/named/providers/FileLockNamedLockFactory.java
> >> [2] - https://maven.apache.org/resolver/configuration.html
> >> [3] - https://maven.apache.org/components/resolver/maven-3.8.x.html
> >>
> >>
> >>> On 21. Jun 2022, at 10:01, Lasse Lindqvist <
> lasse.k.lindqvist@gmail.com> wrote:
> >>>
> >>> Hi. If you want to have pipeline build safely, you also have an
> >>> alternative. With the withMaven clause (Pipeline Maven Integration
> Plugin)
> >>> you can use something like
> >>>
> >>>
> >>>
> >>> *withMaven(maven: 'Maven123', mavenLocalRepo:
> >>> '$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
> >>> commands here*
> >>> * }*
> >>> It will use the workspace folder, travel out of it to a parent
> location and
> >>> make a sibling .m2 folder and unique subfolder there based on the
> executor
> >>> number. This way the folders used are executor specific and there will
> be
> >>> no concurrency problems. The downside is that it will use more disk
> space
> >>> because many artifacts are stored multiple times.
> >>>
> >>> ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org)
> kirjoitti:
> >>>
> >>>> Hi,
> >>>> By default Jenkins uses a global Maven repository which is accessed in
> >>>> parallel by multiple jobs running in dedicated VMs. To prevent race
> >>>> conditions one should probably configure one of the distributed named
> locks
> >>>> outlined in
> >>>>
> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
> >>>>
> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
> >>>> Is there any recommendation which one to pick and also some concrete
> hints
> >>>> how to set that up with Jenkins (Pipelines, leveraging
> >>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
> >>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
> >>>>
> >>>> Is ASF Jenkins using named locks already?
> >>>> Thanks for any pointers
> >>>>
> >>>> Konrad
> >>>>
> >>>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: users-help@maven.apache.org
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: Named Lock implementation for Maven with Jenkins
Posted by Konrad Windszus <ko...@gmx.de>.
Hi Mirko,
Right, but isn’t “file-lock” relying internally on https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html which says: "This file-locking API is intended to map directly to the native locking facility of the underlying operating system. Thus the locks held on a file should be visible to all programs that have access to the file, regardless of the language in which those programs are written.”
Therefore I assume “file-lock” should work also cross VMs or is that assumption not correct?
Konrad
> On 21. Jun 2022, at 11:57, mfriedenhagen@gmx.de wrote:
>
> Hi Konrad,
>
> citing the linked web site: "Local named locks are only suited within one JVM with a multithreaded build. Sharing a local repository between multiple Maven processes (i.e., on a busy CI server) requires a distributed named lock!"
>
> Best Regards
> Mirko Friedenhagen
> —
> Sent from my mobile
>
> Am 21.06.22 um 10:20 schrieb Konrad Windszus
>
>> Thanks for that suggestion. I know about that workaround. But to speed up builds and to reduce disk space usage I would really like to use a shared repo.
>>
>> I was wondering if there is any experience with the simple "file-lock” [1]. That has the advantage that it doesn’t need additional libraries (IIUC). Configuration should be as easy as setting “aether.syncContext.named.factory” to “file-lock” [2]. Unfortunately even Maven 3.8.6 still ships with Resolver 1.6.3 [3] and it seems the suggested branch https://github.com/apache/maven/commits/maven-3.8.x-resolver-1.7.x is a bit outdated now…
>>
>> Konrad
>>
>> [1] - https://github.com/apache/maven-resolver/blob/2af7bff2a5238bf2597cd7a46b60a12779ee31a3/maven-resolver-named-locks/src/main/java/org/eclipse/aether/named/providers/FileLockNamedLockFactory.java
>> [2] - https://maven.apache.org/resolver/configuration.html
>> [3] - https://maven.apache.org/components/resolver/maven-3.8.x.html
>>
>>
>>> On 21. Jun 2022, at 10:01, Lasse Lindqvist <la...@gmail.com> wrote:
>>>
>>> Hi. If you want to have pipeline build safely, you also have an
>>> alternative. With the withMaven clause (Pipeline Maven Integration Plugin)
>>> you can use something like
>>>
>>>
>>>
>>> *withMaven(maven: 'Maven123', mavenLocalRepo:
>>> '$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
>>> commands here*
>>> * }*
>>> It will use the workspace folder, travel out of it to a parent location and
>>> make a sibling .m2 folder and unique subfolder there based on the executor
>>> number. This way the folders used are executor specific and there will be
>>> no concurrency problems. The downside is that it will use more disk space
>>> because many artifacts are stored multiple times.
>>>
>>> ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org) kirjoitti:
>>>
>>>> Hi,
>>>> By default Jenkins uses a global Maven repository which is accessed in
>>>> parallel by multiple jobs running in dedicated VMs. To prevent race
>>>> conditions one should probably configure one of the distributed named locks
>>>> outlined in
>>>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
>>>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
>>>> Is there any recommendation which one to pick and also some concrete hints
>>>> how to set that up with Jenkins (Pipelines, leveraging
>>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
>>>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
>>>>
>>>> Is ASF Jenkins using named locks already?
>>>> Thanks for any pointers
>>>>
>>>> Konrad
>>>>
>>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Re: Named Lock implementation for Maven with Jenkins
Posted by mf...@gmx.de.
Hi Konrad,
citing the linked web site: "Local named locks are only suited within one JVM with a multithreaded build. Sharing a local repository between multiple Maven processes (i.e., on a busy CI server) requires a distributed named lock!"
Best Regards
Mirko Friedenhagen
—
Sent from my mobile
Am 21.06.22 um 10:20 schrieb Konrad Windszus
> Thanks for that suggestion. I know about that workaround. But to speed up builds and to reduce disk space usage I would really like to use a shared repo.
>
> I was wondering if there is any experience with the simple "file-lock” [1]. That has the advantage that it doesn’t need additional libraries (IIUC). Configuration should be as easy as setting “aether.syncContext.named.factory” to “file-lock” [2]. Unfortunately even Maven 3.8.6 still ships with Resolver 1.6.3 [3] and it seems the suggested branch https://github.com/apache/maven/commits/maven-3.8.x-resolver-1.7.x is a bit outdated now…
>
> Konrad
>
> [1] - https://github.com/apache/maven-resolver/blob/2af7bff2a5238bf2597cd7a46b60a12779ee31a3/maven-resolver-named-locks/src/main/java/org/eclipse/aether/named/providers/FileLockNamedLockFactory.java
> [2] - https://maven.apache.org/resolver/configuration.html
> [3] - https://maven.apache.org/components/resolver/maven-3.8.x.html
>
>
> > On 21. Jun 2022, at 10:01, Lasse Lindqvist <la...@gmail.com> wrote:
> >
> > Hi. If you want to have pipeline build safely, you also have an
> > alternative. With the withMaven clause (Pipeline Maven Integration Plugin)
> > you can use something like
> >
> >
> >
> > *withMaven(maven: 'Maven123', mavenLocalRepo:
> > '$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
> > commands here*
> > * }*
> > It will use the workspace folder, travel out of it to a parent location and
> > make a sibling .m2 folder and unique subfolder there based on the executor
> > number. This way the folders used are executor specific and there will be
> > no concurrency problems. The downside is that it will use more disk space
> > because many artifacts are stored multiple times.
> >
> > ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org) kirjoitti:
> >
> >> Hi,
> >> By default Jenkins uses a global Maven repository which is accessed in
> >> parallel by multiple jobs running in dedicated VMs. To prevent race
> >> conditions one should probably configure one of the distributed named locks
> >> outlined in
> >> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
> >> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
> >> Is there any recommendation which one to pick and also some concrete hints
> >> how to set that up with Jenkins (Pipelines, leveraging
> >> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
> >> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
> >>
> >> Is ASF Jenkins using named locks already?
> >> Thanks for any pointers
> >>
> >> Konrad
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Named Lock implementation for Maven with Jenkins
Posted by Konrad Windszus <ko...@gmx.de>.
Thanks for that suggestion. I know about that workaround. But to speed up builds and to reduce disk space usage I would really like to use a shared repo.
I was wondering if there is any experience with the simple "file-lock” [1]. That has the advantage that it doesn’t need additional libraries (IIUC). Configuration should be as easy as setting “aether.syncContext.named.factory” to “file-lock” [2]. Unfortunately even Maven 3.8.6 still ships with Resolver 1.6.3 [3] and it seems the suggested branch https://github.com/apache/maven/commits/maven-3.8.x-resolver-1.7.x is a bit outdated now…
Konrad
[1] - https://github.com/apache/maven-resolver/blob/2af7bff2a5238bf2597cd7a46b60a12779ee31a3/maven-resolver-named-locks/src/main/java/org/eclipse/aether/named/providers/FileLockNamedLockFactory.java
[2] - https://maven.apache.org/resolver/configuration.html
[3] - https://maven.apache.org/components/resolver/maven-3.8.x.html
> On 21. Jun 2022, at 10:01, Lasse Lindqvist <la...@gmail.com> wrote:
>
> Hi. If you want to have pipeline build safely, you also have an
> alternative. With the withMaven clause (Pipeline Maven Integration Plugin)
> you can use something like
>
>
>
> *withMaven(maven: 'Maven123', mavenLocalRepo:
> '$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
> commands here*
> * }*
> It will use the workspace folder, travel out of it to a parent location and
> make a sibling .m2 folder and unique subfolder there based on the executor
> number. This way the folders used are executor specific and there will be
> no concurrency problems. The downside is that it will use more disk space
> because many artifacts are stored multiple times.
>
> ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org) kirjoitti:
>
>> Hi,
>> By default Jenkins uses a global Maven repository which is accessed in
>> parallel by multiple jobs running in dedicated VMs. To prevent race
>> conditions one should probably configure one of the distributed named locks
>> outlined in
>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
>> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
>> Is there any recommendation which one to pick and also some concrete hints
>> how to set that up with Jenkins (Pipelines, leveraging
>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
>> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
>>
>> Is ASF Jenkins using named locks already?
>> Thanks for any pointers
>>
>> Konrad
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Named Lock implementation for Maven with Jenkins
Posted by Lasse Lindqvist <la...@gmail.com>.
Hi. If you want to have pipeline build safely, you also have an
alternative. With the withMaven clause (Pipeline Maven Integration Plugin)
you can use something like
*withMaven(maven: 'Maven123', mavenLocalRepo:
'$WORKSPACE/../../.m2/$EXECUTOR_NUMBER/repository') { // Run my Maven
commands here*
* }*
It will use the workspace folder, travel out of it to a parent location and
make a sibling .m2 folder and unique subfolder there based on the executor
number. This way the folders used are executor specific and there will be
no concurrency problems. The downside is that it will use more disk space
because many artifacts are stored multiple times.
ti 21. kesäk. 2022 klo 10.51 Konrad Windszus (kwin@apache.org) kirjoitti:
> Hi,
> By default Jenkins uses a global Maven repository which is accessed in
> parallel by multiple jobs running in dedicated VMs. To prevent race
> conditions one should probably configure one of the distributed named locks
> outlined in
> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html <
> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html>.
> Is there any recommendation which one to pick and also some concrete hints
> how to set that up with Jenkins (Pipelines, leveraging
> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/ <
> https://www.jenkins.io/doc/pipeline/steps/pipeline-maven/>)
>
> Is ASF Jenkins using named locks already?
> Thanks for any pointers
>
> Konrad
>
>