You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Dan Tran <da...@gmail.com> on 2016/04/19 21:08:26 UTC
Hickup in multi thread build
Hi
My 150+ modules with multi-thread build randomly error out with something
like this
java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
Have you encountered similar issue? how do you troubleshoot?
I am using maven 3.3.9 and latest compiler plugin
Thanks
-Dan
Re: Hickup in multi thread build
Posted by Dan Tran <da...@gmail.com>.
I am able to pinpoint an internal aggregated plugin that may trigger this
issue. my builds are very happy for the last couple of days
Thanks
-Dan
On Mon, May 2, 2016 at 12:55 AM, Dan Tran <da...@gmail.com> wrote:
> Hi Fabian,
>
> I am going inspect all the plugins
>
> Thanks for the great suggestion
>
> -Dan
>
> On Sun, May 1, 2016 at 11:55 PM, Fabian van der Veen <
> F.van.der.Veen@topdesk.com> wrote:
>
>> Hey Dan,
>>
>> I think I've seen my extension still fail a few times as well, but it
>> wasn't reproducible enough to debug for me. It is still a concurrency issue
>> amongst the other problems; so it could simply be something I assumed
>> thread-safe API actually isn't thread-safe at all. (Which opens another
>> interesting dimension of problems, should that be true...)
>> On the other hand, I decided to tackle the core problem in our build
>> instead: removing aggregating plugin executions from anything but the root
>> pom.
>>
>> It might be a lot of work, but given how these plugins resolve their
>> dependencies, it might be for the best. (Besides the fact that, if I recall
>> correctly, these kinds of plugins are described as aggregating data from
>> the entire project.)
>> For example, in our case the javadoc plugin (amongst others) had the
>> aggregate goals declared in the root pom, but - unfortunately - these goals
>> simply get inherited to all child poms as well (causing the mentioned
>> problems).
>>
>> What we did was simply run the help:effective-pom target on the projects
>> (randomly) failing in a concurrent build; and check for all the resulting
>> plugin executions whether or not that goal was an aggregate goal or not.
>> With the list of offending plugins ready, we removed those that were
>> configured wrongly by developers. The ones left, we configured with
>> "<inherited>false</inherited>" in the plugin definition in the root pom;
>> which makes the plugin only configured and run on the parent.
>>
>> Actually working through the poms and fixing the offenders has made our
>> concurrent build 100% stable again. So while the extension does - in our
>> case, at least - certainly improve stability; it's still (unfortunately)
>> not a complete (easy) workaround.
>>
>> Hopefully this'll help you get your build stable again as well.
>> Otherwise, it'll be a waiting game until the issues I mentioned are picked
>> up.
>>
>> - Fabian
>>
>> On vr, 2016-04-29 at 17:04 -0700, Dan Tran wrote:
>>
>> I dont see much improvement. I am seeing 2-3 fail build a day
>>
>> -D
>>
>> On Wed, Apr 20, 2016 at 12:40 AM, Dan Tran <dantran@gmail.com<mailto:
>> dantran@gmail.com>> wrote:
>>
>>
>>
>> Hi Fabian
>>
>> I will give your extension a try
>>
>> Thanks
>>
>> -Dan
>>
>> On Wed, Apr 20, 2016 at 12:26 AM, Fabian van der Veen <
>> F.van.der.Veen@topdesk.com<ma...@topdesk.com>> wrote:
>>
>>
>>
>> Hey,
>>
>> The way you describe it; this sounds like a problem I also encountered in
>> our (similar sized) project here.
>> I reported this here: https://issues.apache.org/jira/browse/MNG-5960,
>> which has been linked as related to
>> https://issues.apache.org/jira/browse/MNG-5750.
>>
>> Troubleshooting boiled down to running maven in debug mode; which might
>> be something you don't want to do.
>> In any case, I found that plugins running aggregate goals (e.g.
>> javadoc:aggregate) will resolve (compile) dependencies for _all_ projects
>> in the reactor. Running such a plugin in anything else than your root pom
>> might cause the random failures you have. (And having the plugin in your
>> root pom might also effectively include its configuration and execution in
>> child poms; which is "debuggable" using help:effective-pom.)
>>
>> Hopefully this helps you.
>>
>> - Fabian
>>
>> On di, 2016-04-19 at 12:08 -0700, Dan Tran wrote:
>>
>> Hi
>>
>> My 150+ modules with multi-thread build randomly error out with something
>> like this
>>
>> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
>>
>>
>> Have you encountered similar issue? how do you troubleshoot?
>>
>>
>> I am using maven 3.3.9 and latest compiler plugin
>>
>>
>> Thanks
>>
>>
>> -Dan
>>
>>
>>
>>
>>
>>
>>
>
Re: Hickup in multi thread build
Posted by Dan Tran <da...@gmail.com>.
Hi Fabian,
I am going inspect all the plugins
Thanks for the great suggestion
-Dan
On Sun, May 1, 2016 at 11:55 PM, Fabian van der Veen <
F.van.der.Veen@topdesk.com> wrote:
> Hey Dan,
>
> I think I've seen my extension still fail a few times as well, but it
> wasn't reproducible enough to debug for me. It is still a concurrency issue
> amongst the other problems; so it could simply be something I assumed
> thread-safe API actually isn't thread-safe at all. (Which opens another
> interesting dimension of problems, should that be true...)
> On the other hand, I decided to tackle the core problem in our build
> instead: removing aggregating plugin executions from anything but the root
> pom.
>
> It might be a lot of work, but given how these plugins resolve their
> dependencies, it might be for the best. (Besides the fact that, if I recall
> correctly, these kinds of plugins are described as aggregating data from
> the entire project.)
> For example, in our case the javadoc plugin (amongst others) had the
> aggregate goals declared in the root pom, but - unfortunately - these goals
> simply get inherited to all child poms as well (causing the mentioned
> problems).
>
> What we did was simply run the help:effective-pom target on the projects
> (randomly) failing in a concurrent build; and check for all the resulting
> plugin executions whether or not that goal was an aggregate goal or not.
> With the list of offending plugins ready, we removed those that were
> configured wrongly by developers. The ones left, we configured with
> "<inherited>false</inherited>" in the plugin definition in the root pom;
> which makes the plugin only configured and run on the parent.
>
> Actually working through the poms and fixing the offenders has made our
> concurrent build 100% stable again. So while the extension does - in our
> case, at least - certainly improve stability; it's still (unfortunately)
> not a complete (easy) workaround.
>
> Hopefully this'll help you get your build stable again as well. Otherwise,
> it'll be a waiting game until the issues I mentioned are picked up.
>
> - Fabian
>
> On vr, 2016-04-29 at 17:04 -0700, Dan Tran wrote:
>
> I dont see much improvement. I am seeing 2-3 fail build a day
>
> -D
>
> On Wed, Apr 20, 2016 at 12:40 AM, Dan Tran <dantran@gmail.com<mailto:
> dantran@gmail.com>> wrote:
>
>
>
> Hi Fabian
>
> I will give your extension a try
>
> Thanks
>
> -Dan
>
> On Wed, Apr 20, 2016 at 12:26 AM, Fabian van der Veen <
> F.van.der.Veen@topdesk.com<ma...@topdesk.com>> wrote:
>
>
>
> Hey,
>
> The way you describe it; this sounds like a problem I also encountered in
> our (similar sized) project here.
> I reported this here: https://issues.apache.org/jira/browse/MNG-5960,
> which has been linked as related to
> https://issues.apache.org/jira/browse/MNG-5750.
>
> Troubleshooting boiled down to running maven in debug mode; which might
> be something you don't want to do.
> In any case, I found that plugins running aggregate goals (e.g.
> javadoc:aggregate) will resolve (compile) dependencies for _all_ projects
> in the reactor. Running such a plugin in anything else than your root pom
> might cause the random failures you have. (And having the plugin in your
> root pom might also effectively include its configuration and execution in
> child poms; which is "debuggable" using help:effective-pom.)
>
> Hopefully this helps you.
>
> - Fabian
>
> On di, 2016-04-19 at 12:08 -0700, Dan Tran wrote:
>
> Hi
>
> My 150+ modules with multi-thread build randomly error out with something
> like this
>
> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
>
>
> Have you encountered similar issue? how do you troubleshoot?
>
>
> I am using maven 3.3.9 and latest compiler plugin
>
>
> Thanks
>
>
> -Dan
>
>
>
>
>
>
>
Re: Hickup in multi thread build
Posted by Fabian van der Veen <F....@topdesk.com>.
Hey Dan,
I think I've seen my extension still fail a few times as well, but it wasn't reproducible enough to debug for me. It is still a concurrency issue amongst the other problems; so it could simply be something I assumed thread-safe API actually isn't thread-safe at all. (Which opens another interesting dimension of problems, should that be true...)
On the other hand, I decided to tackle the core problem in our build instead: removing aggregating plugin executions from anything but the root pom.
It might be a lot of work, but given how these plugins resolve their dependencies, it might be for the best. (Besides the fact that, if I recall correctly, these kinds of plugins are described as aggregating data from the entire project.)
For example, in our case the javadoc plugin (amongst others) had the aggregate goals declared in the root pom, but - unfortunately - these goals simply get inherited to all child poms as well (causing the mentioned problems).
What we did was simply run the help:effective-pom target on the projects (randomly) failing in a concurrent build; and check for all the resulting plugin executions whether or not that goal was an aggregate goal or not. With the list of offending plugins ready, we removed those that were configured wrongly by developers. The ones left, we configured with "<inherited>false</inherited>" in the plugin definition in the root pom; which makes the plugin only configured and run on the parent.
Actually working through the poms and fixing the offenders has made our concurrent build 100% stable again. So while the extension does - in our case, at least - certainly improve stability; it's still (unfortunately) not a complete (easy) workaround.
Hopefully this'll help you get your build stable again as well. Otherwise, it'll be a waiting game until the issues I mentioned are picked up.
- Fabian
On vr, 2016-04-29 at 17:04 -0700, Dan Tran wrote:
I dont see much improvement. I am seeing 2-3 fail build a day
-D
On Wed, Apr 20, 2016 at 12:40 AM, Dan Tran <da...@gmail.com>> wrote:
Hi Fabian
I will give your extension a try
Thanks
-Dan
On Wed, Apr 20, 2016 at 12:26 AM, Fabian van der Veen <
F.van.der.Veen@topdesk.com<ma...@topdesk.com>> wrote:
Hey,
The way you describe it; this sounds like a problem I also encountered in
our (similar sized) project here.
I reported this here: https://issues.apache.org/jira/browse/MNG-5960,
which has been linked as related to
https://issues.apache.org/jira/browse/MNG-5750.
Troubleshooting boiled down to running maven in debug mode; which might
be something you don't want to do.
In any case, I found that plugins running aggregate goals (e.g.
javadoc:aggregate) will resolve (compile) dependencies for _all_ projects
in the reactor. Running such a plugin in anything else than your root pom
might cause the random failures you have. (And having the plugin in your
root pom might also effectively include its configuration and execution in
child poms; which is "debuggable" using help:effective-pom.)
Hopefully this helps you.
- Fabian
On di, 2016-04-19 at 12:08 -0700, Dan Tran wrote:
Hi
My 150+ modules with multi-thread build randomly error out with something
like this
java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
Have you encountered similar issue? how do you troubleshoot?
I am using maven 3.3.9 and latest compiler plugin
Thanks
-Dan
Re: Hickup in multi thread build
Posted by Dan Tran <da...@gmail.com>.
I dont see much improvement. I am seeing 2-3 fail build a day
-D
On Wed, Apr 20, 2016 at 12:40 AM, Dan Tran <da...@gmail.com> wrote:
> Hi Fabian
>
> I will give your extension a try
>
> Thanks
>
> -Dan
>
> On Wed, Apr 20, 2016 at 12:26 AM, Fabian van der Veen <
> F.van.der.Veen@topdesk.com> wrote:
>
>> Hey,
>>
>> The way you describe it; this sounds like a problem I also encountered in
>> our (similar sized) project here.
>> I reported this here: https://issues.apache.org/jira/browse/MNG-5960,
>> which has been linked as related to
>> https://issues.apache.org/jira/browse/MNG-5750.
>>
>> Troubleshooting boiled down to running maven in debug mode; which might
>> be something you don't want to do.
>> In any case, I found that plugins running aggregate goals (e.g.
>> javadoc:aggregate) will resolve (compile) dependencies for _all_ projects
>> in the reactor. Running such a plugin in anything else than your root pom
>> might cause the random failures you have. (And having the plugin in your
>> root pom might also effectively include its configuration and execution in
>> child poms; which is "debuggable" using help:effective-pom.)
>>
>> Hopefully this helps you.
>>
>> - Fabian
>>
>> On di, 2016-04-19 at 12:08 -0700, Dan Tran wrote:
>>
>> Hi
>>
>> My 150+ modules with multi-thread build randomly error out with something
>> like this
>>
>> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
>>
>>
>> Have you encountered similar issue? how do you troubleshoot?
>>
>>
>> I am using maven 3.3.9 and latest compiler plugin
>>
>>
>> Thanks
>>
>>
>> -Dan
>>
>>
>
Re: Hickup in multi thread build
Posted by Dan Tran <da...@gmail.com>.
Hi Fabian
I will give your extension a try
Thanks
-Dan
On Wed, Apr 20, 2016 at 12:26 AM, Fabian van der Veen <
F.van.der.Veen@topdesk.com> wrote:
> Hey,
>
> The way you describe it; this sounds like a problem I also encountered in
> our (similar sized) project here.
> I reported this here: https://issues.apache.org/jira/browse/MNG-5960,
> which has been linked as related to
> https://issues.apache.org/jira/browse/MNG-5750.
>
> Troubleshooting boiled down to running maven in debug mode; which might be
> something you don't want to do.
> In any case, I found that plugins running aggregate goals (e.g.
> javadoc:aggregate) will resolve (compile) dependencies for _all_ projects
> in the reactor. Running such a plugin in anything else than your root pom
> might cause the random failures you have. (And having the plugin in your
> root pom might also effectively include its configuration and execution in
> child poms; which is "debuggable" using help:effective-pom.)
>
> Hopefully this helps you.
>
> - Fabian
>
> On di, 2016-04-19 at 12:08 -0700, Dan Tran wrote:
>
> Hi
>
> My 150+ modules with multi-thread build randomly error out with something
> like this
>
> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
>
>
> Have you encountered similar issue? how do you troubleshoot?
>
>
> I am using maven 3.3.9 and latest compiler plugin
>
>
> Thanks
>
>
> -Dan
>
>
Re: Hickup in multi thread build
Posted by Fabian van der Veen <F....@topdesk.com>.
Hey,
The way you describe it; this sounds like a problem I also encountered in our (similar sized) project here.
I reported this here: https://issues.apache.org/jira/browse/MNG-5960, which has been linked as related to https://issues.apache.org/jira/browse/MNG-5750.
Troubleshooting boiled down to running maven in debug mode; which might be something you don't want to do.
In any case, I found that plugins running aggregate goals (e.g. javadoc:aggregate) will resolve (compile) dependencies for _all_ projects in the reactor. Running such a plugin in anything else than your root pom might cause the random failures you have. (And having the plugin in your root pom might also effectively include its configuration and execution in child poms; which is "debuggable" using help:effective-pom.)
Hopefully this helps you.
- Fabian
On di, 2016-04-19 at 12:08 -0700, Dan Tran wrote:
Hi
My 150+ modules with multi-thread build randomly error out with something
like this
java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
Have you encountered similar issue? how do you troubleshoot?
I am using maven 3.3.9 and latest compiler plugin
Thanks
-Dan