You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Oleg Estekhin <ol...@gmail.com> on 2012/07/11 11:42:35 UTC
problem: Maven creates the second plugin classloader during parallel build and uses two classloader hierachies simultaneously
Hi,
I am trying to solve the mystery behind
http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16
with the help of the reporter of the original issue and the developer of
SqlJet library.
So far we have found that at some point there appears second classloader,
which initializes some singleton classes second time, and after that the
parallel build continues with two separate singletons. One set of build
tasks still uses the first singleton, other set of build tasks uses the
second singleton, and that, just as expected, causes a lot of pain later.
What is worse, I can't reproduce this behavior on my system (Windows 7,
Maven 3.0.4, JDK 1.7u5) with a test project, but it is reliably reproduced
on the system of one of the plugin' users (OpenSUSE 11.1, maven 3.0.4, JDK
1.7u4) with his own project, by my test project builds just fine there too.
Means I am missing something in the test project, but so far I do not know
what I am missing.
Questions, questions:
What could cause the Maven to create the second classloader during the
parallel build while the first classloader and singleton are definitely
still in use?
What kind of even more logging could be added to the plugin or to the SqlJet
library that can provide some additional insight of what is going on?
What kind of debugging could be used on Maven parallel build?
What should I have asked besides the questions above?
Best regards,
Oleg.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Posted by Kristian Rosenvold <kr...@zenior.no>.
Thanks for a precise problem description. Having studied the source a bit,
I found a race condition that would make this happen.
I'm still analyzing the implications a bit, but I suspect this one is the
underlying source of a few of the jiras on the parallel build, your report
was the first that was clear enough to get through my dense head ;)
The reason only a few people see this is because it depends on the shape of
your reactor; the case I'm seeing will only happen if the first-time
invocation of a plugin happens in a parallel part of the reactor (and
additionally your plugin must actually mind about this; most dont).
Does this fit your problem? Did you create a jira ?
Kristian
Den 23. juli 2012 kl. 22:08 skrev Dennis Lundberg <de...@apache.org>:
On 2012-07-11 11:42, Oleg Estekhin wrote:
Hi,
I am trying to solve the mystery behind
http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16
with the help of the reporter of the original issue and the developer of
SqlJet library.
So far we have found that at some point there appears second
classloader, which initializes some singleton classes second time, and
after that the parallel build continues with two separate singletons.
One set of build tasks still uses the first singleton, other set of
build tasks uses the second singleton, and that, just as expected,
causes a lot of pain later.
What is worse, I can't reproduce this behavior on my system (Windows 7,
Maven 3.0.4, JDK 1.7u5) with a test project, but it is reliably
reproduced on the system of one of the plugin' users (OpenSUSE 11.1,
maven 3.0.4, JDK 1.7u4) with his own project, by my test project builds
just fine there too. Means I am missing something in the test project,
but so far I do not know what I am missing.
Questions, questions:
What could cause the Maven to create the second classloader during the
parallel build while the first classloader and singleton are definitely
still in use?
What kind of even more logging could be added to the plugin or to the
SqlJet library that can provide some additional insight of what is going
on?
What kind of debugging could be used on Maven parallel build?
What should I have asked besides the questions above?
Best regards,
Oleg.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
test
--
Dennis Lundberg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
mandag 23. juli 2012 skrev Dennis Lundberg følgende:
> On 2012-07-11 11:42, Oleg Estekhin wrote:
> > Hi,
> >
> > I am trying to solve the mystery behind
> >
> http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16
> > with the help of the reporter of the original issue and the developer of
> > SqlJet library.
> >
> > So far we have found that at some point there appears second
> > classloader, which initializes some singleton classes second time, and
> > after that the parallel build continues with two separate singletons.
> > One set of build tasks still uses the first singleton, other set of
> > build tasks uses the second singleton, and that, just as expected,
> > causes a lot of pain later.
> >
> > What is worse, I can't reproduce this behavior on my system (Windows 7,
> > Maven 3.0.4, JDK 1.7u5) with a test project, but it is reliably
> > reproduced on the system of one of the plugin' users (OpenSUSE 11.1,
> > maven 3.0.4, JDK 1.7u4) with his own project, by my test project builds
> > just fine there too. Means I am missing something in the test project,
> > but so far I do not know what I am missing.
> >
> > Questions, questions:
> > What could cause the Maven to create the second classloader during the
> > parallel build while the first classloader and singleton are definitely
> > still in use?
> > What kind of even more logging could be added to the plugin or to the
> > SqlJet library that can provide some additional insight of what is going
> > on?
> > What kind of debugging could be used on Maven parallel build?
> > What should I have asked besides the questions above?
> >
> > Best regards,
> > Oleg.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org<javascript:;>
> > For additional commands, e-mail: users-help@maven.apache.org<javascript:;>
>
> test
>
> --
> Dennis Lundberg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org <javascript:;>
> For additional commands, e-mail: users-help@maven.apache.org<javascript:;>
>
>
Re: problem: Maven creates the second plugin classloader during parallel
build and uses two classloader hierachies simultaneously
Posted by Dennis Lundberg <de...@apache.org>.
On 2012-07-11 11:42, Oleg Estekhin wrote:
> Hi,
>
> I am trying to solve the mystery behind
> http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16
> with the help of the reporter of the original issue and the developer of
> SqlJet library.
>
> So far we have found that at some point there appears second
> classloader, which initializes some singleton classes second time, and
> after that the parallel build continues with two separate singletons.
> One set of build tasks still uses the first singleton, other set of
> build tasks uses the second singleton, and that, just as expected,
> causes a lot of pain later.
>
> What is worse, I can't reproduce this behavior on my system (Windows 7,
> Maven 3.0.4, JDK 1.7u5) with a test project, but it is reliably
> reproduced on the system of one of the plugin' users (OpenSUSE 11.1,
> maven 3.0.4, JDK 1.7u4) with his own project, by my test project builds
> just fine there too. Means I am missing something in the test project,
> but so far I do not know what I am missing.
>
> Questions, questions:
> What could cause the Maven to create the second classloader during the
> parallel build while the first classloader and singleton are definitely
> still in use?
> What kind of even more logging could be added to the plugin or to the
> SqlJet library that can provide some additional insight of what is going
> on?
> What kind of debugging could be used on Maven parallel build?
> What should I have asked besides the questions above?
>
> Best regards,
> Oleg.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
test
--
Dennis Lundberg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: problem: Maven creates the second plugin classloader during parallel build and uses two classloader hierachies simultaneously
Posted by Oleg Estekhin <ol...@gmail.com>.
All parties use Maven 3.0.4.
-----Исходное сообщение-----
From: Stephen Connolly
Sent: Thursday, July 12, 2012 2:13 PM
To: Maven Users List
Subject: Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Are you testing with Maven 2.x and they building with Maven 3.x?
Maven 2.x uses one plugin classloader for the reactor which causes tones of
issues.
Maven 3.x fixes this by instantiating a new classloader for every invokation
On 12 July 2012 11:10, Oleg Estekhin <ol...@gmail.com> wrote:
> The very special test project with 7000 modules total and 4000 of them
> using the plugin works fine - single classloader, single singleton inside
> SqlJet library. The actual real-life project that has this issue has 100+
> modules with almost each module using the plugin and during the build of
> this project there seems to be two classloaders, not 100+. So what do you
> mean by "each plugin invocation"?
>
> In both cases the plugin gav is configured only once in the top-level
> parent pom.
>
> It is not multiple classloaders per se that are problem, the problem is
> that the second class loader does not see dependency/library classes that
> are already initialized by the first class loader, and so it reloads and
> initializes them for the second time.
>
> -----Исходное сообщение----- From: Stephen Connolly
> Sent: Thursday, July 12, 2012 12:42 PM
>
> To: Maven Users List
> Subject: Re: problem: Maven creates the second plugin classloader during
> parallel build and uses two classloader hierachies simultaneously
>
> Each plugin invokation gets its own classloader as this is to allow for
> different versions of the same plugin being used in a multi-module
> project,
> also to allow for injecting different dependencies into a plugin.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail:
> users-unsubscribe@maven.**apache.org<us...@maven.apache.org>
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Posted by Stephen Connolly <st...@gmail.com>.
Are you testing with Maven 2.x and they building with Maven 3.x?
Maven 2.x uses one plugin classloader for the reactor which causes tones of
issues.
Maven 3.x fixes this by instantiating a new classloader for every invokation
On 12 July 2012 11:10, Oleg Estekhin <ol...@gmail.com> wrote:
> The very special test project with 7000 modules total and 4000 of them
> using the plugin works fine - single classloader, single singleton inside
> SqlJet library. The actual real-life project that has this issue has 100+
> modules with almost each module using the plugin and during the build of
> this project there seems to be two classloaders, not 100+. So what do you
> mean by "each plugin invocation"?
>
> In both cases the plugin gav is configured only once in the top-level
> parent pom.
>
> It is not multiple classloaders per se that are problem, the problem is
> that the second class loader does not see dependency/library classes that
> are already initialized by the first class loader, and so it reloads and
> initializes them for the second time.
>
> -----Исходное сообщение----- From: Stephen Connolly
> Sent: Thursday, July 12, 2012 12:42 PM
>
> To: Maven Users List
> Subject: Re: problem: Maven creates the second plugin classloader during
> parallel build and uses two classloader hierachies simultaneously
>
> Each plugin invokation gets its own classloader as this is to allow for
> different versions of the same plugin being used in a multi-module project,
> also to allow for injecting different dependencies into a plugin.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@maven.**apache.org<us...@maven.apache.org>
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: problem: Maven creates the second plugin classloader during parallel build and uses two classloader hierachies simultaneously
Posted by Oleg Estekhin <ol...@gmail.com>.
The very special test project with 7000 modules total and 4000 of them using
the plugin works fine - single classloader, single singleton inside SqlJet
library. The actual real-life project that has this issue has 100+ modules
with almost each module using the plugin and during the build of this
project there seems to be two classloaders, not 100+. So what do you mean by
"each plugin invocation"?
In both cases the plugin gav is configured only once in the top-level parent
pom.
It is not multiple classloaders per se that are problem, the problem is that
the second class loader does not see dependency/library classes that are
already initialized by the first class loader, and so it reloads and
initializes them for the second time.
-----Исходное сообщение-----
From: Stephen Connolly
Sent: Thursday, July 12, 2012 12:42 PM
To: Maven Users List
Subject: Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Each plugin invokation gets its own classloader as this is to allow for
different versions of the same plugin being used in a multi-module project,
also to allow for injecting different dependencies into a plugin.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Posted by Stephen Connolly <st...@gmail.com>.
Each plugin invokation gets its own classloader as this is to allow for
different versions of the same plugin being used in a multi-module project,
also to allow for injecting different dependencies into a plugin.
On 12 July 2012 06:55, Oleg Estekhin <ol...@gmail.com> wrote:
> HI, thanks for response.
>
> By the "build tasks" I meant invocations of plugin:goal. In our case we
> narrowed the issue down to the single phase and single plugin, so during
> parallel build Maven just runs the same plugin:goal
> (svn-revision-number-maven-**plugin:revision) for different modules. The
> instance of the plugin mojo is, of course, different for each invocation,
> but at some point there appears one more classloader, the plugin mojo class
> and library classes that it uses are initialized for the second time, and
> after that some build tasks use the "old" plugin.class hierarchy, and some
> build tasks use the "new" plugin.class hierarchy.
>
> For example, the log contains the following log messages produced by the
> plugin:
> [INFO] [pool-1-thread-1] identities: thread=0x4FC90D2C, this=0x67CC4E4D,
> this.class=0x64375C2A, this.class.loader=0x3BAF655E,
> SqlJetFileLockManager.class=**0x58D3E2
> Plugin logs these identities each time it is invoked, so there are o lot
> of such messages with different "thread" and "this", but the same
> "this.class", "this.class.loader" and "SqlJetFileLockManager.class".
>
> And then the following log messages start to show:
> [INFO] [pool-1-thread-14] identities: thread=0x7429D88, this=0x3345FC7A,
> this.class=0x12329908, this.class.loader=0x5742691B,
> SqlJetFileLockManager.class=**0x3EE53DAE
>
>
> Regards,
> Oleg
>
> -----Исходное сообщение----- From: Olivier Lamy
> Sent: Thursday, July 12, 2012 2:49 AM
> To: Maven Users List
> Subject: Re: problem: Maven creates the second plugin classloader during
> parallel build and uses two classloader hierachies simultaneously
>
>
> Hi,
>
> 2012/7/11 Oleg Estekhin <ol...@gmail.com>:
>
>> Hi,
>>
>> I am trying to solve the mystery behind
>> http://code.google.com/p/**maven-svn-revision-number-**
>> plugin/issues/detail?id=16<http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16>
>> with the help of the reporter of the original issue and the developer of
>> SqlJet library.
>>
>> So far we have found that at some point there appears second classloader,
>> which initializes some singleton classes second time, and after that the
>> parallel build continues with two separate singletons. One set of build
>> tasks still uses the first singleton, other set of build tasks uses the
>> second singleton, and that, just as expected, causes a lot of pain later.
>>
> What do you mean with "build tasks" ? A maven plugin ?
> If yes that's normal each plugin have his own classloader.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@maven.**apache.org<us...@maven.apache.org>
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: problem: Maven creates the second plugin classloader during parallel build and uses two classloader hierachies simultaneously
Posted by Oleg Estekhin <ol...@gmail.com>.
HI, thanks for response.
By the "build tasks" I meant invocations of plugin:goal. In our case we
narrowed the issue down to the single phase and single plugin, so during
parallel build Maven just runs the same plugin:goal
(svn-revision-number-maven-plugin:revision) for different modules. The
instance of the plugin mojo is, of course, different for each invocation,
but at some point there appears one more classloader, the plugin mojo class
and library classes that it uses are initialized for the second time, and
after that some build tasks use the "old" plugin.class hierarchy, and some
build tasks use the "new" plugin.class hierarchy.
For example, the log contains the following log messages produced by the
plugin:
[INFO] [pool-1-thread-1] identities: thread=0x4FC90D2C, this=0x67CC4E4D,
this.class=0x64375C2A, this.class.loader=0x3BAF655E,
SqlJetFileLockManager.class=0x58D3E2
Plugin logs these identities each time it is invoked, so there are o lot of
such messages with different "thread" and "this", but the same "this.class",
"this.class.loader" and "SqlJetFileLockManager.class".
And then the following log messages start to show:
[INFO] [pool-1-thread-14] identities: thread=0x7429D88, this=0x3345FC7A,
this.class=0x12329908, this.class.loader=0x5742691B,
SqlJetFileLockManager.class=0x3EE53DAE
Regards,
Oleg
-----Исходное сообщение-----
From: Olivier Lamy
Sent: Thursday, July 12, 2012 2:49 AM
To: Maven Users List
Subject: Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Hi,
2012/7/11 Oleg Estekhin <ol...@gmail.com>:
> Hi,
>
> I am trying to solve the mystery behind
> http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16
> with the help of the reporter of the original issue and the developer of
> SqlJet library.
>
> So far we have found that at some point there appears second classloader,
> which initializes some singleton classes second time, and after that the
> parallel build continues with two separate singletons. One set of build
> tasks still uses the first singleton, other set of build tasks uses the
> second singleton, and that, just as expected, causes a lot of pain later.
What do you mean with "build tasks" ? A maven plugin ?
If yes that's normal each plugin have his own classloader.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: problem: Maven creates the second plugin classloader during
parallel build and uses two classloader hierachies simultaneously
Posted by Olivier Lamy <ol...@apache.org>.
Hi,
2012/7/11 Oleg Estekhin <ol...@gmail.com>:
> Hi,
>
> I am trying to solve the mystery behind
> http://code.google.com/p/maven-svn-revision-number-plugin/issues/detail?id=16
> with the help of the reporter of the original issue and the developer of
> SqlJet library.
>
> So far we have found that at some point there appears second classloader,
> which initializes some singleton classes second time, and after that the
> parallel build continues with two separate singletons. One set of build
> tasks still uses the first singleton, other set of build tasks uses the
> second singleton, and that, just as expected, causes a lot of pain later.
What do you mean with "build tasks" ? A maven plugin ?
If yes that's normal each plugin have his own classloader.
>
> What is worse, I can't reproduce this behavior on my system (Windows 7,
> Maven 3.0.4, JDK 1.7u5) with a test project, but it is reliably reproduced
> on the system of one of the plugin' users (OpenSUSE 11.1, maven 3.0.4, JDK
> 1.7u4) with his own project, by my test project builds just fine there too.
> Means I am missing something in the test project, but so far I do not know
> what I am missing.
>
> Questions, questions:
> What could cause the Maven to create the second classloader during the
> parallel build while the first classloader and singleton are definitely
> still in use?
> What kind of even more logging could be added to the plugin or to the SqlJet
> library that can provide some additional insight of what is going on?
> What kind of debugging could be used on Maven parallel build?
> What should I have asked besides the questions above?
>
> Best regards,
> Oleg.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org