You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by David Hoffer <dh...@gmail.com> on 2015/07/30 16:45:47 UTC
Parent snapshot pom questions
I've got a case where we get random CI build failures. The error is:
[FATAL] Non-resolvable parent POM: Could not find artifact
com.issinc.jms.jms-thirdparty.installers:installer-parent:pom:11.8.0-SNAPSHOT
in nexus (https://mavenserver/nexus/content/groups/public) and
'parent.relativePath' points at wrong local POM @ line 43, column 13
Here is the pom and build layout.
- pom.xml (parent) This pom has level1 as module but does not have the
others
- level1 module
-- pom.xml
- level2 module (Has top level pom as parent)
-- pom.xml
- level3 module (Has top level pom as parent)
-- pom.xml
- level4 module (Has top level pom as parent)
-- pom.xml
Now for our CI builds they build this as 4 separate builds, here are what
they checkout and build.
- pom.xml (parent)
- level1
-- pom.xml
- level2
-- pom.xml
- level3
-- pom.xml
- level4
-- pom.xml
The first build, builds pointing to the parent and deploys the parent pom
to Nexus, this build always works. The level2 to level4 builds don't
checkout the parent pom as that is not part of the reactor of the build,
yet it is still the parent. What we expected to happen is that it would
pull the parent pom from Nexus. We have verified that it does exist in
Nexus yet it can't find it.
This error is random sometimes it works and sometimes it does not. We have
several build agents (14 or so). The error might be agent specific, e.g.
if the same agent did the parent/level1 build then it works but not if a
different agent did it (not sure if that is the case yet).
What I'm wondering is what is the Maven rules for finding parent (snapshot)
poms that are not part of the current reactor's build? We assumed it would
follow the normal search order, e.g. local repo first, then Nexus but that
does not seem to be happening. The error says it is looking in Nexus but
it doesn't appear to be doing that rather it seems to be looking on the
local hard drive for the pom via 'parent.relativePath' but we don't even
use relativePath at all in any of these poms.
Any idea what is going on?
.
-Dave
Re: Parent snapshot pom questions
Posted by Laurence Bordowitz <lb...@yahoo-inc.com.INVALID>.
Wow that got mangled. Let's try that link again:
http://takari.io/book/30-team-maven.html
-- Larry Bordowitz
On Thursday, July 30, 2015 9:58 AM, Laurence Bordowitz <lb...@yahoo-inc.com.INVALID> wrote:
Another solution is to try the Takari.io extensions. One of them has filesystem locks on the local repo for exactly this scenario. I haven't tried these out on our CI pipeline yet, though.
Takari | Team-maven
| |
| | | | | |
| Takari | Team-mavenTakari is focused on continuous delivery for the Maven ecosystem and offers Apache Maven training classes |
| |
| View on takari.io | Preview by Yahoo |
| |
| |
-- Larry Bordowitz
On Thursday, July 30, 2015 8:06 AM, Curtis Rueden <ct...@wisc.edu> wrote:
Hi Dave,
Maven is not safe for concurrent builds from the same local repository
cache. Do these CI jobs share the same cache? If so that would explain the
nondeterminism you are seeing.
It is best practice if each job has its own local repo cache, as expensive
as that is disk-wise.
Or you can use only a single executor on each node doing Maven builds. But
of course that doesn't scale particularly well.
Also: is there some reason you don't just have a single job that builds the
whole multi-module project? What is the point of having a multi-module
project then? Or: if I misunderstood and these projects are each
single-module, why not use release couplings always (i.e., a release
version of the parent POM)? Then you won't have issues with SNAPSHOTs
getting nondeterministically overwritten etc.
Regards,
Curtis
On Thu, Jul 30, 2015 at 9:45 AM, David Hoffer <dh...@gmail.com> wrote:
> I've got a case where we get random CI build failures. The error is:
>
> [FATAL] Non-resolvable parent POM: Could not find artifact
>
> com.issinc.jms.jms-thirdparty.installers:installer-parent:pom:11.8.0-SNAPSHOT
> in nexus (https://mavenserver/nexus/content/groups/public) and
> 'parent.relativePath' points at wrong local POM @ line 43, column 13
>
> Here is the pom and build layout.
>
> - pom.xml (parent) This pom has level1 as module but does not have the
> others
> - level1 module
> -- pom.xml
> - level2 module (Has top level pom as parent)
> -- pom.xml
> - level3 module (Has top level pom as parent)
> -- pom.xml
> - level4 module (Has top level pom as parent)
> -- pom.xml
>
> Now for our CI builds they build this as 4 separate builds, here are what
> they checkout and build.
>
> - pom.xml (parent)
> - level1
> -- pom.xml
>
> - level2
> -- pom.xml
>
> - level3
> -- pom.xml
>
> - level4
> -- pom.xml
>
> The first build, builds pointing to the parent and deploys the parent pom
> to Nexus, this build always works. The level2 to level4 builds don't
> checkout the parent pom as that is not part of the reactor of the build,
> yet it is still the parent. What we expected to happen is that it would
> pull the parent pom from Nexus. We have verified that it does exist in
> Nexus yet it can't find it.
>
> This error is random sometimes it works and sometimes it does not. We have
> several build agents (14 or so). The error might be agent specific, e.g.
> if the same agent did the parent/level1 build then it works but not if a
> different agent did it (not sure if that is the case yet).
>
> What I'm wondering is what is the Maven rules for finding parent (snapshot)
> poms that are not part of the current reactor's build? We assumed it would
> follow the normal search order, e.g. local repo first, then Nexus but that
> does not seem to be happening. The error says it is looking in Nexus but
> it doesn't appear to be doing that rather it seems to be looking on the
> local hard drive for the pom via 'parent.relativePath' but we don't even
> use relativePath at all in any of these poms.
>
> Any idea what is going on?
> .
> -Dave
>
Re: Parent snapshot pom questions
Posted by Laurence Bordowitz <lb...@yahoo-inc.com.INVALID>.
Another solution is to try the Takari.io extensions. One of them has filesystem locks on the local repo for exactly this scenario. I haven't tried these out on our CI pipeline yet, though.
Takari | Team-maven
| |
| | | | | |
| Takari | Team-mavenTakari is focused on continuous delivery for the Maven ecosystem and offers Apache Maven training classes |
| |
| View on takari.io | Preview by Yahoo |
| |
| |
-- Larry Bordowitz
On Thursday, July 30, 2015 8:06 AM, Curtis Rueden <ct...@wisc.edu> wrote:
Hi Dave,
Maven is not safe for concurrent builds from the same local repository
cache. Do these CI jobs share the same cache? If so that would explain the
nondeterminism you are seeing.
It is best practice if each job has its own local repo cache, as expensive
as that is disk-wise.
Or you can use only a single executor on each node doing Maven builds. But
of course that doesn't scale particularly well.
Also: is there some reason you don't just have a single job that builds the
whole multi-module project? What is the point of having a multi-module
project then? Or: if I misunderstood and these projects are each
single-module, why not use release couplings always (i.e., a release
version of the parent POM)? Then you won't have issues with SNAPSHOTs
getting nondeterministically overwritten etc.
Regards,
Curtis
On Thu, Jul 30, 2015 at 9:45 AM, David Hoffer <dh...@gmail.com> wrote:
> I've got a case where we get random CI build failures. The error is:
>
> [FATAL] Non-resolvable parent POM: Could not find artifact
>
> com.issinc.jms.jms-thirdparty.installers:installer-parent:pom:11.8.0-SNAPSHOT
> in nexus (https://mavenserver/nexus/content/groups/public) and
> 'parent.relativePath' points at wrong local POM @ line 43, column 13
>
> Here is the pom and build layout.
>
> - pom.xml (parent) This pom has level1 as module but does not have the
> others
> - level1 module
> -- pom.xml
> - level2 module (Has top level pom as parent)
> -- pom.xml
> - level3 module (Has top level pom as parent)
> -- pom.xml
> - level4 module (Has top level pom as parent)
> -- pom.xml
>
> Now for our CI builds they build this as 4 separate builds, here are what
> they checkout and build.
>
> - pom.xml (parent)
> - level1
> -- pom.xml
>
> - level2
> -- pom.xml
>
> - level3
> -- pom.xml
>
> - level4
> -- pom.xml
>
> The first build, builds pointing to the parent and deploys the parent pom
> to Nexus, this build always works. The level2 to level4 builds don't
> checkout the parent pom as that is not part of the reactor of the build,
> yet it is still the parent. What we expected to happen is that it would
> pull the parent pom from Nexus. We have verified that it does exist in
> Nexus yet it can't find it.
>
> This error is random sometimes it works and sometimes it does not. We have
> several build agents (14 or so). The error might be agent specific, e.g.
> if the same agent did the parent/level1 build then it works but not if a
> different agent did it (not sure if that is the case yet).
>
> What I'm wondering is what is the Maven rules for finding parent (snapshot)
> poms that are not part of the current reactor's build? We assumed it would
> follow the normal search order, e.g. local repo first, then Nexus but that
> does not seem to be happening. The error says it is looking in Nexus but
> it doesn't appear to be doing that rather it seems to be looking on the
> local hard drive for the pom via 'parent.relativePath' but we don't even
> use relativePath at all in any of these poms.
>
> Any idea what is going on?
> .
> -Dave
>
Re: Parent snapshot pom questions
Posted by David Hoffer <dh...@gmail.com>.
Hi Curtis,
To answer your first question...we have lots of build agents...each agent
will run one build at a time. That agent will share a local repo with
other builds that happen to run on the same agent but it's sequential
nothing concurrent ever happens in the CI build.
Regarding why this isn't a single multi-module build...it's because it was
a conversion from a legacy ANT build to Maven but they didn't refactor the
contents of the build. The simple answer is they are huge. Each build
probably has a 10GB minimum SVN checkout. If they were combined it's
probably over 50GB. I think it was either a checkout speed or disk space
issue, or perhaps both, as to why they were separated. I can try to
combine again and just live with it but it seems it should work separate
too.
-Dave
On Thu, Jul 30, 2015 at 9:05 AM, Curtis Rueden <ct...@wisc.edu> wrote:
> Hi Dave,
>
> Maven is not safe for concurrent builds from the same local repository
> cache. Do these CI jobs share the same cache? If so that would explain the
> nondeterminism you are seeing.
>
> It is best practice if each job has its own local repo cache, as expensive
> as that is disk-wise.
>
> Or you can use only a single executor on each node doing Maven builds. But
> of course that doesn't scale particularly well.
>
> Also: is there some reason you don't just have a single job that builds the
> whole multi-module project? What is the point of having a multi-module
> project then? Or: if I misunderstood and these projects are each
> single-module, why not use release couplings always (i.e., a release
> version of the parent POM)? Then you won't have issues with SNAPSHOTs
> getting nondeterministically overwritten etc.
>
> Regards,
> Curtis
>
> On Thu, Jul 30, 2015 at 9:45 AM, David Hoffer <dh...@gmail.com> wrote:
>
> > I've got a case where we get random CI build failures. The error is:
> >
> > [FATAL] Non-resolvable parent POM: Could not find artifact
> >
> >
> com.issinc.jms.jms-thirdparty.installers:installer-parent:pom:11.8.0-SNAPSHOT
> > in nexus (https://mavenserver/nexus/content/groups/public) and
> > 'parent.relativePath' points at wrong local POM @ line 43, column 13
> >
> > Here is the pom and build layout.
> >
> > - pom.xml (parent) This pom has level1 as module but does not have the
> > others
> > - level1 module
> > -- pom.xml
> > - level2 module (Has top level pom as parent)
> > -- pom.xml
> > - level3 module (Has top level pom as parent)
> > -- pom.xml
> > - level4 module (Has top level pom as parent)
> > -- pom.xml
> >
> > Now for our CI builds they build this as 4 separate builds, here are what
> > they checkout and build.
> >
> > - pom.xml (parent)
> > - level1
> > -- pom.xml
> >
> > - level2
> > -- pom.xml
> >
> > - level3
> > -- pom.xml
> >
> > - level4
> > -- pom.xml
> >
> > The first build, builds pointing to the parent and deploys the parent pom
> > to Nexus, this build always works. The level2 to level4 builds don't
> > checkout the parent pom as that is not part of the reactor of the build,
> > yet it is still the parent. What we expected to happen is that it would
> > pull the parent pom from Nexus. We have verified that it does exist in
> > Nexus yet it can't find it.
> >
> > This error is random sometimes it works and sometimes it does not. We
> have
> > several build agents (14 or so). The error might be agent specific, e.g.
> > if the same agent did the parent/level1 build then it works but not if a
> > different agent did it (not sure if that is the case yet).
> >
> > What I'm wondering is what is the Maven rules for finding parent
> (snapshot)
> > poms that are not part of the current reactor's build? We assumed it
> would
> > follow the normal search order, e.g. local repo first, then Nexus but
> that
> > does not seem to be happening. The error says it is looking in Nexus but
> > it doesn't appear to be doing that rather it seems to be looking on the
> > local hard drive for the pom via 'parent.relativePath' but we don't even
> > use relativePath at all in any of these poms.
> >
> > Any idea what is going on?
> > .
> > -Dave
> >
>
Re: Parent snapshot pom questions
Posted by Curtis Rueden <ct...@wisc.edu>.
Hi Dave,
Maven is not safe for concurrent builds from the same local repository
cache. Do these CI jobs share the same cache? If so that would explain the
nondeterminism you are seeing.
It is best practice if each job has its own local repo cache, as expensive
as that is disk-wise.
Or you can use only a single executor on each node doing Maven builds. But
of course that doesn't scale particularly well.
Also: is there some reason you don't just have a single job that builds the
whole multi-module project? What is the point of having a multi-module
project then? Or: if I misunderstood and these projects are each
single-module, why not use release couplings always (i.e., a release
version of the parent POM)? Then you won't have issues with SNAPSHOTs
getting nondeterministically overwritten etc.
Regards,
Curtis
On Thu, Jul 30, 2015 at 9:45 AM, David Hoffer <dh...@gmail.com> wrote:
> I've got a case where we get random CI build failures. The error is:
>
> [FATAL] Non-resolvable parent POM: Could not find artifact
>
> com.issinc.jms.jms-thirdparty.installers:installer-parent:pom:11.8.0-SNAPSHOT
> in nexus (https://mavenserver/nexus/content/groups/public) and
> 'parent.relativePath' points at wrong local POM @ line 43, column 13
>
> Here is the pom and build layout.
>
> - pom.xml (parent) This pom has level1 as module but does not have the
> others
> - level1 module
> -- pom.xml
> - level2 module (Has top level pom as parent)
> -- pom.xml
> - level3 module (Has top level pom as parent)
> -- pom.xml
> - level4 module (Has top level pom as parent)
> -- pom.xml
>
> Now for our CI builds they build this as 4 separate builds, here are what
> they checkout and build.
>
> - pom.xml (parent)
> - level1
> -- pom.xml
>
> - level2
> -- pom.xml
>
> - level3
> -- pom.xml
>
> - level4
> -- pom.xml
>
> The first build, builds pointing to the parent and deploys the parent pom
> to Nexus, this build always works. The level2 to level4 builds don't
> checkout the parent pom as that is not part of the reactor of the build,
> yet it is still the parent. What we expected to happen is that it would
> pull the parent pom from Nexus. We have verified that it does exist in
> Nexus yet it can't find it.
>
> This error is random sometimes it works and sometimes it does not. We have
> several build agents (14 or so). The error might be agent specific, e.g.
> if the same agent did the parent/level1 build then it works but not if a
> different agent did it (not sure if that is the case yet).
>
> What I'm wondering is what is the Maven rules for finding parent (snapshot)
> poms that are not part of the current reactor's build? We assumed it would
> follow the normal search order, e.g. local repo first, then Nexus but that
> does not seem to be happening. The error says it is looking in Nexus but
> it doesn't appear to be doing that rather it seems to be looking on the
> local hard drive for the pom via 'parent.relativePath' but we don't even
> use relativePath at all in any of these poms.
>
> Any idea what is going on?
> .
> -Dave
>