You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Greg Parker <gm...@gmail.com> on 2017/05/19 17:12:20 UTC
Jenkins builds
I’m looking for a way to isolate builds from one another on our Jenkins server. What I would like to do is have a common .m2 folder for external dependencies. For my own artifacts I’d like to install/retrieve them to/from a separate local repository. I suspect that this capability does not exist, but I thought I would check.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Jenkins builds
Posted by Karl Heinz Marbaise <kh...@gmx.de>.
Hi Greg,
On 19/05/17 23:55, Greg Parker wrote:
> Thanks for the reply ,
>
....
>
Although having a separate .m2 folder is quite possible and would
address my issue,
> it’s something I’m trying to avoid for performance reasons.
> Pulling down all the external dependencies from Archiva,
> or central repository, for each project takes too long
> and takes up too much space.
Archiva should proxy central repository and you should never directly
pick dependencies from Maven Central only via a proxies repositores...
The time is an issue but only for the first time ...you can use updates
in version control so it's not necessary to download the deps for each
build (I don't know how long your builds take? We have build about max
15 minutes release build and ci build takes 6 minutes)
Disk space ? Is that really a question? disk space is cheap...currently
each of our builds takes 15 GiB of disk space for CI and our release
build take 54 GiB...currently having 5 branches in parallel having some
other builds as well..in total we are talking about approx 1.5 TiB...
It's important to check how long the downloads really take it might be a
task to improve your infrastructure...
>
Our builds have a customer implementation
> component which means we have many
> builds for many customers.
what does "many " builds mean? 10, 100, 1000 ? different builds?
>
What I would like to have is a common .m2 folder for external dependencies,
> > and a build specific .m2 folder with just the build output.
In Maven only a single location for the cache exists..
Apart from that I really don't understand your concerns having your own
artifacts separated from those others? They are separated by
groupId/artifactId/version etc.
What kind of problem are you trying to solve?
Furthermore in the minute you are using a single cache part for several
build all those builds are coupled via this cache...
Another thing is important here. This means you are coupling those
builds to a particular build node if this cache does not exist your node
can't run the build there...That's why the build cache in the workspace
is the best option...
Kind regards
Karl Heinz Marbaise
Something like:
>
> ~/.m2/repository
> /log4j
> /commons-io
> /…
>
> $WORKSPACE1/.m2/repository
> /customer1-lib
> /build-lib
> /...
>
> $WORKSPACE2/.m2/repository
> /customer2-lib
> /build-lib
> /...
>
> That way I don’t have to pull down all the common stuff into a fresh .m2 for each build.
>
>
>> On May 19, 2017, at 5:36 PM, Karl Heinz Marbaise <kh...@gmx.de> wrote:
>>
>> Hi Greg,
>>
>> On 19/05/17 19:12, Greg Parker wrote:
>>> I’m looking for a way to isolate builds from one another on our Jenkins server.
>>> What I would like to do is have a common .m2 folder
>>> for external dependencies.
>>
>>> For my own artifacts I’d like to install/retrieve
>>> them to/from a separate local repository.
>>
>>> I suspect that this capability does not exist, but I thought I would check.
>>
>> The first thing what you need to understand is that the $HOME/.m2/repository is a cache location where the artifacts which are downloaded only cached to prevent repeated downloads from remote repositories..
>>
>> The first step is to separate build jobs in Jenkins means that each job has it's own local cache directory which means having the local cache in the workspace ($WORKSPACE/.repository)...
>>
>> The second step is having a repository manager (like Nexus, Artifactory or Archiva) which handles the separation between the artifacts which are created by your own and the ones you are consuming from outside (for example from Maven Central)..
>>
>> There I would suggest to make this difference for your own artifacts and the artifacts which you call "external"...
>>
>>
>> Kind regards
>> Karl Heinz Marbaise
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Jenkins builds
Posted by Greg Parker <gm...@gmail.com>.
Thanks for the reply ,
I understand the concept of the .m2 folder and generally how maven works. We run an Archiva server where we cache dependencies and publish our released artifacts etc etc. Although having a separate .m2 folder is quite possible and would address my issue, it’s something I’m trying to avoid for performance reasons. Pulling down all the external dependencies from Archiva, or central repository, for each project takes too long and takes up too much space. Our builds have a customer implementation component which means we have many builds for many customers. What I would like to have is a common .m2 folder for external dependencies, and a build specific .m2 folder with just the build output. Something like:
~/.m2/repository
/log4j
/commons-io
/…
$WORKSPACE1/.m2/repository
/customer1-lib
/build-lib
/...
$WORKSPACE2/.m2/repository
/customer2-lib
/build-lib
/...
That way I don’t have to pull down all the common stuff into a fresh .m2 for each build.
> On May 19, 2017, at 5:36 PM, Karl Heinz Marbaise <kh...@gmx.de> wrote:
>
> Hi Greg,
>
> On 19/05/17 19:12, Greg Parker wrote:
>> I’m looking for a way to isolate builds from one another on our Jenkins server.
> > What I would like to do is have a common .m2 folder
> > for external dependencies.
>
> > For my own artifacts I’d like to install/retrieve
> > them to/from a separate local repository.
>
> > I suspect that this capability does not exist, but I thought I would check.
>
> The first thing what you need to understand is that the $HOME/.m2/repository is a cache location where the artifacts which are downloaded only cached to prevent repeated downloads from remote repositories..
>
> The first step is to separate build jobs in Jenkins means that each job has it's own local cache directory which means having the local cache in the workspace ($WORKSPACE/.repository)...
>
> The second step is having a repository manager (like Nexus, Artifactory or Archiva) which handles the separation between the artifacts which are created by your own and the ones you are consuming from outside (for example from Maven Central)..
>
> There I would suggest to make this difference for your own artifacts and the artifacts which you call "external"...
>
>
> Kind regards
> Karl Heinz Marbaise
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Jenkins builds
Posted by Karl Heinz Marbaise <kh...@gmx.de>.
Hi Greg,
On 19/05/17 19:12, Greg Parker wrote:
> I’m looking for a way to isolate builds from one another on our Jenkins server.
> What I would like to do is have a common .m2 folder
> for external dependencies.
> For my own artifacts I’d like to install/retrieve
> them to/from a separate local repository.
> I suspect that this capability does not exist, but I thought I would
check.
The first thing what you need to understand is that the
$HOME/.m2/repository is a cache location where the artifacts which are
downloaded only cached to prevent repeated downloads from remote
repositories..
The first step is to separate build jobs in Jenkins means that each job
has it's own local cache directory which means having the local cache in
the workspace ($WORKSPACE/.repository)...
The second step is having a repository manager (like Nexus, Artifactory
or Archiva) which handles the separation between the artifacts which are
created by your own and the ones you are consuming from outside (for
example from Maven Central)..
There I would suggest to make this difference for your own artifacts and
the artifacts which you call "external"...
Kind regards
Karl Heinz Marbaise
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org