You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@felix.apache.org by Cédric Casenove <cc...@axway.com> on 2011/03/29 13:36:33 UTC
Deadlock in ModuleClassLoader
Hello,
I found a deadlock when using felix framework 3.0.9.
I built 2 bundles that depend on each other. Bundle A dynamically imports a
package from bundle B, and bundle B imports a package from bundle A.
When the bundle A starts, an activator launches a thread that waits for a
file "trigger" to be created. When the file appears, the thread tries to
load a class from bundle B with Class.forName().
When the bundle B starts, an activator launches a thread that creates the
file "trigger" and then tries to instantiate a class from bundle A.
Here is the standard output during the test execution and the threads stack
trace showing the deadlock.
g! lb
START LEVEL 1
ID|State |Level|Name
0|Active | 0|System Bundle (3.0.9)
1|Active | 1|Apache Felix Bundle Repository (1.6.2)
2|Active | 1|Apache Felix Gogo Command (0.8.0)
3|Active | 1|Apache Felix Gogo Runtime (0.8.0)
4|Active | 1|Apache Felix Gogo Shell (0.8.0)
5|Installed | 1|Test - BundleA (0.0.1.SNAPSHOT)
6|Installed | 1|Test - BundleB (0.0.1.SNAPSHOT)
g! start 5
Start bundle A
g! bundle A waits trigger file
start 6
Start bundle B
g! bundle A found trigger file
2011-03-29 12:59:50
Full thread dump Java HotSpot(TM) Client VM (19.1-b02 mixed mode, sharing):
"BThread" daemon prio=10 tid=0x09dbb400 nid=0x8bd waiting for monitor entry
[0xb4359000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
- waiting to lock <0x7f4869f0> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at com.axway.testb.BThread.run(BThread.java:32)
"MessageExecutor" daemon prio=10 tid=0x09db6c00 nid=0x8bc waiting for
monitor entry [0xb43aa000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
- waiting to lock <0x7f50ba18> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1427)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at com.axway.testa.MessageExecutor.run(MessageExecutor.java:40)
"Gogo shell" daemon prio=10 tid=0x09daa400 nid=0x8b6 runnable [0xb43fb000]
java.lang.Thread.State: RUNNABLE
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:199)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
- locked <0x846f37d0> (a java.io.BufferedInputStream)
at
org.apache.felix.gogo.runtime.threadio.ThreadInputStream.read(ThreadInputStream.java:77)
at org.apache.felix.gogo.shell.Console.getLine(Console.java:117)
at org.apache.felix.gogo.shell.Console.run(Console.java:53)
at org.apache.felix.gogo.shell.Shell.console(Shell.java:203)
at org.apache.felix.gogo.shell.Shell.gosh(Shell.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.felix.gogo.runtime.Reflective.method(Reflective.java:136)
at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:469)
at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:395)
at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)
at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)
at
org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:89)
at org.apache.felix.gogo.shell.Activator.run(Activator.java:75)
at java.lang.Thread.run(Thread.java:662)
"FelixPackageAdmin" daemon prio=10 tid=0x09d41c00 nid=0x8b5 in Object.wait()
[0xb4454000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x846f0110> (a org.apache.felix.framework.PackageAdminImpl)
at java.lang.Object.wait(Object.java:485)
at
org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:348)
- locked <0x846f0110> (a org.apache.felix.framework.PackageAdminImpl)
at java.lang.Thread.run(Thread.java:662)
"FelixStartLevel" daemon prio=10 tid=0x09d30000 nid=0x8b4 in Object.wait()
[0xb44a5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x846f0198> (a java.util.ArrayList)
at java.lang.Object.wait(Object.java:485)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:242)
- locked <0x846f0198> (a java.util.ArrayList)
at java.lang.Thread.run(Thread.java:662)
"FelixDispatchQueue" prio=10 tid=0x09d42c00 nid=0x8b3 in Object.wait()
[0xb451a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x846f0238> (a java.util.ArrayList)
at java.lang.Object.wait(Object.java:485)
at
org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:924)
- locked <0x846f0238> (a java.util.ArrayList)
at
org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
at
org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:106)
at java.lang.Thread.run(Thread.java:662)
"Low Memory Detector" daemon prio=10 tid=0x09c8ec00 nid=0x8b1 runnable
[0x00000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=10 tid=0x09c8d000 nid=0x8b0 waiting on
condition [0x00000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x09c8b400 nid=0x8af waiting on
condition [0x00000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x09c83400 nid=0x8ae in Object.wait()
[0xb4761000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x846f0418> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x846f0418> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x09c81c00 nid=0x8ad in Object.wait()
[0xb47b2000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x846f0098> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x846f0098> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x09c54000 nid=0x8ab in Object.wait() [0xb6c79000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x846f04a8> (a org.apache.felix.framework.util.ThreadGate)
at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79)
- locked <0x846f04a8> (a org.apache.felix.framework.util.ThreadGate)
at org.apache.felix.framework.Felix.waitForStop(Felix.java:906)
at org.apache.felix.main.Main.main(Main.java:295)
"VM Thread" prio=10 tid=0x09c77800 nid=0x8ac runnable
"VM Periodic Task Thread" prio=10 tid=0x09c9ac00 nid=0x8b2 waiting on
condition
JNI global references: 1217
Found one Java-level deadlock:
=============================
"BThread":
waiting to lock monitor 0x09d5cbd0 (object 0x7f4869f0, a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
which is held by "MessageExecutor"
"MessageExecutor":
waiting to lock monitor 0x09d5bf50 (object 0x7f50ba18, a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
which is held by "BThread"
Java stack information for the threads listed above:
===================================================
"BThread":
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
- waiting to lock <0x7f4869f0> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at com.axway.testb.BThread.run(BThread.java:32)
"MessageExecutor":
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
- waiting to lock <0x7f50ba18> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1427)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at com.axway.testa.MessageExecutor.run(MessageExecutor.java:40)
Found 1 deadlock.
http://old.nabble.com/file/p31266713/test.zip test.zip
--
View this message in context: http://old.nabble.com/Deadlock-in-ModuleClassLoader-tp31266713p31266713.html
Sent from the Apache Felix - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Deadlock in ModuleClassLoader
Posted by Cédric Casenove <cc...@axway.com>.
Richard S. Hall wrote:
>
> I am fairly certain that you are experiencing the long-standing issue
> where the JVM is too aggressive in its locking of class loaders.
>
> The Felix framework doesn't hold class loader locks while class loader.
> The point where the threads are blocking are just simple checks to see
> if the class has already been loaded.
>
> You should look into this to see if it resolves your issue:
>
>
> http://underlap.blogspot.com/2006/11/experimental-fix-for-sunbug-4670071.html
>
> -> richard
>
Thanks for your reply, the page you point me to seems actually to be related
to my problem. I'm going to test the workaround and see if the deadlock
disappears.
Regards
--
View this message in context: http://old.nabble.com/Deadlock-in-ModuleClassLoader-tp31266713p31274765.html
Sent from the Apache Felix - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Deadlock in ModuleClassLoader
Posted by metatech <me...@gmail.com>.
Hi,
On second inspection, the problem seems to be slightly different.
Here are more details about the locked being held :
Lock 0x95506608 : org.ops4j.pax.web.service.WebContainer
Lock 0x9552b2d8 : org.osgi.service.http.HttpService
[ 127] [Active ] [ ] [ ] [ 60] OPS4J Pax Web - API
(0.7.4)
[ 129] [Active ] [ ] [ ] [ 60] OPS4J Pax Web -
Runtime (0.7.4)
[ 133] [Starting ] [ ] [ ] [ 60] OPS4J Pax Web -
Extender - Whiteboard (0.7.4)
metatech wrote:
>
> For the record, here is another occurence of the problem on ServiceMix
> 4.3.
>
--
View this message in context: http://old.nabble.com/Deadlock-in-ModuleClassLoader-tp31266713p33127356.html
Sent from the Apache Felix - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Deadlock in ModuleClassLoader
Posted by "Richard S. Hall" <he...@ungoverned.org>.
This appears to be an older version of the Felix framework, right? Makes
it difficult to know if the issue still effects the latest release since
there were definitely changes in this area. If you could verify that, it
would be good. If it is related to JVM internal class loader locking,
then there still isn't much we can do.
-> richard
On 1/12/12 05:57 , metatech wrote:
>
> Richard S. Hall wrote:
>> http://underlap.blogspot.com/2006/11/experimental-fix-for-sunbug-4670071.html
>>
> For the record, here is another occurence of the problem on ServiceMix 4.3.
>
> [ 129] [Active ] [ ] [ ] [ 60] OPS4J Pax Web -
> Runtime (0.7.4)
> [ 133] [Starting ] [ ] [ ] [ 60] OPS4J Pax Web -
> Extender - Whiteboard (0.7.4)
>
> Both bundles are trying to load class "org.osgi.service.http.HttpService".
>
> Found one Java-level deadlock:
> =============================
> "Thread-12":
> waiting to lock monitor 0x080eea70 (object 0x95506608, a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
> which is held by "FelixStartLevel"
> "FelixStartLevel":
> waiting to lock monitor 0x081b9768 (object 0x9552b2d8, a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
> which is held by "Thread-12"
>
> Java stack information for the threads listed above:
> ===================================================
> "Thread-12":
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.findClass(ModuleImpl.java:1830)
> - waiting to lock<0x95506608> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:727)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.ops4j.pax.web.service.internal.Activator.createHttpServiceFactory(Activator.java:98)
> at
> org.ops4j.pax.web.service.internal.Activator.access$200(Activator.java:47)
> at org.ops4j.pax.web.service.internal.Activator$1.run(Activator.java:76)
> at java.lang.Thread.run(Thread.java:662)
> "FelixStartLevel":
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1759)
> - waiting to lock<0x9552b2d8> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.findClass(ModuleImpl.java:1907)
> - locked<0x95506608> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:727)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.ops4j.pax.web.extender.whiteboard.internal.util.WebContainerUtils.webContainerAvailable(WebContainerUtils.java:46)
> at
> org.ops4j.pax.web.extender.whiteboard.internal.util.WebContainerUtils.<clinit>(WebContainerUtils.java:35)
> at
> org.ops4j.pax.web.extender.whiteboard.internal.Activator.start(Activator.java:78)
> at
> org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
> at org.apache.felix.framework.Felix.activateBundle(Felix.java:1842)
> at org.apache.felix.framework.Felix.startBundle(Felix.java:1759)
> at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1163)
> at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
> at java.lang.Thread.run(Thread.java:662)
>
> Found 1 deadlock.
>
> The problem sometimes happens on very slow machines and with full DEBUG log
> enabled, so I cannot really test that the JVM flags
> "XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass" work reliably as a
> workaround.
>
> metatech
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Deadlock in ModuleClassLoader
Posted by metatech <me...@gmail.com>.
Richard S. Hall wrote:
>
> http://underlap.blogspot.com/2006/11/experimental-fix-for-sunbug-4670071.html
>
For the record, here is another occurence of the problem on ServiceMix 4.3.
[ 129] [Active ] [ ] [ ] [ 60] OPS4J Pax Web -
Runtime (0.7.4)
[ 133] [Starting ] [ ] [ ] [ 60] OPS4J Pax Web -
Extender - Whiteboard (0.7.4)
Both bundles are trying to load class "org.osgi.service.http.HttpService".
Found one Java-level deadlock:
=============================
"Thread-12":
waiting to lock monitor 0x080eea70 (object 0x95506608, a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
which is held by "FelixStartLevel"
"FelixStartLevel":
waiting to lock monitor 0x081b9768 (object 0x9552b2d8, a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
which is held by "Thread-12"
Java stack information for the threads listed above:
===================================================
"Thread-12":
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.findClass(ModuleImpl.java:1830)
- waiting to lock <0x95506608> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:727)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.ops4j.pax.web.service.internal.Activator.createHttpServiceFactory(Activator.java:98)
at
org.ops4j.pax.web.service.internal.Activator.access$200(Activator.java:47)
at org.ops4j.pax.web.service.internal.Activator$1.run(Activator.java:76)
at java.lang.Thread.run(Thread.java:662)
"FelixStartLevel":
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1759)
- waiting to lock <0x9552b2d8> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.findClass(ModuleImpl.java:1907)
- locked <0x95506608> (a
org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:727)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
at
org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
at
org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.ops4j.pax.web.extender.whiteboard.internal.util.WebContainerUtils.webContainerAvailable(WebContainerUtils.java:46)
at
org.ops4j.pax.web.extender.whiteboard.internal.util.WebContainerUtils.<clinit>(WebContainerUtils.java:35)
at
org.ops4j.pax.web.extender.whiteboard.internal.Activator.start(Activator.java:78)
at
org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1842)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1759)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1163)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
at java.lang.Thread.run(Thread.java:662)
Found 1 deadlock.
The problem sometimes happens on very slow machines and with full DEBUG log
enabled, so I cannot really test that the JVM flags
"XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass" work reliably as a
workaround.
metatech
--
View this message in context: http://old.nabble.com/Deadlock-in-ModuleClassLoader-tp31266713p33127027.html
Sent from the Apache Felix - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Deadlock in ModuleClassLoader
Posted by "Richard S. Hall" <he...@ungoverned.org>.
I am fairly certain that you are experiencing the long-standing issue
where the JVM is too aggressive in its locking of class loaders.
The Felix framework doesn't hold class loader locks while class loader.
The point where the threads are blocking are just simple checks to see
if the class has already been loaded.
You should look into this to see if it resolves your issue:
http://underlap.blogspot.com/2006/11/experimental-fix-for-sunbug-4670071.html
-> richard
On 3/29/11 7:36, Cédric Casenove wrote:
> Hello,
>
> I found a deadlock when using felix framework 3.0.9.
> I built 2 bundles that depend on each other. Bundle A dynamically imports a
> package from bundle B, and bundle B imports a package from bundle A.
>
> When the bundle A starts, an activator launches a thread that waits for a
> file "trigger" to be created. When the file appears, the thread tries to
> load a class from bundle B with Class.forName().
>
> When the bundle B starts, an activator launches a thread that creates the
> file "trigger" and then tries to instantiate a class from bundle A.
>
> Here is the standard output during the test execution and the threads stack
> trace showing the deadlock.
>
> g! lb
> START LEVEL 1
> ID|State |Level|Name
> 0|Active | 0|System Bundle (3.0.9)
> 1|Active | 1|Apache Felix Bundle Repository (1.6.2)
> 2|Active | 1|Apache Felix Gogo Command (0.8.0)
> 3|Active | 1|Apache Felix Gogo Runtime (0.8.0)
> 4|Active | 1|Apache Felix Gogo Shell (0.8.0)
> 5|Installed | 1|Test - BundleA (0.0.1.SNAPSHOT)
> 6|Installed | 1|Test - BundleB (0.0.1.SNAPSHOT)
> g! start 5
> Start bundle A
> g! bundle A waits trigger file
> start 6
> Start bundle B
> g! bundle A found trigger file
> 2011-03-29 12:59:50
> Full thread dump Java HotSpot(TM) Client VM (19.1-b02 mixed mode, sharing):
>
> "BThread" daemon prio=10 tid=0x09dbb400 nid=0x8bd waiting for monitor entry
> [0xb4359000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
> - waiting to lock<0x7f4869f0> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at com.axway.testb.BThread.run(BThread.java:32)
>
> "MessageExecutor" daemon prio=10 tid=0x09db6c00 nid=0x8bc waiting for
> monitor entry [0xb43aa000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
> - waiting to lock<0x7f50ba18> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1427)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at com.axway.testa.MessageExecutor.run(MessageExecutor.java:40)
>
> "Gogo shell" daemon prio=10 tid=0x09daa400 nid=0x8b6 runnable [0xb43fb000]
> java.lang.Thread.State: RUNNABLE
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:199)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
> - locked<0x846f37d0> (a java.io.BufferedInputStream)
> at
> org.apache.felix.gogo.runtime.threadio.ThreadInputStream.read(ThreadInputStream.java:77)
> at org.apache.felix.gogo.shell.Console.getLine(Console.java:117)
> at org.apache.felix.gogo.shell.Console.run(Console.java:53)
> at org.apache.felix.gogo.shell.Shell.console(Shell.java:203)
> at org.apache.felix.gogo.shell.Shell.gosh(Shell.java:128)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.felix.gogo.runtime.Reflective.method(Reflective.java:136)
> at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
> at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:469)
> at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:395)
> at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
> at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)
> at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)
> at
> org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:89)
> at org.apache.felix.gogo.shell.Activator.run(Activator.java:75)
> at java.lang.Thread.run(Thread.java:662)
>
> "FelixPackageAdmin" daemon prio=10 tid=0x09d41c00 nid=0x8b5 in Object.wait()
> [0xb4454000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x846f0110> (a org.apache.felix.framework.PackageAdminImpl)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:348)
> - locked<0x846f0110> (a org.apache.felix.framework.PackageAdminImpl)
> at java.lang.Thread.run(Thread.java:662)
>
> "FelixStartLevel" daemon prio=10 tid=0x09d30000 nid=0x8b4 in Object.wait()
> [0xb44a5000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x846f0198> (a java.util.ArrayList)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:242)
> - locked<0x846f0198> (a java.util.ArrayList)
> at java.lang.Thread.run(Thread.java:662)
>
> "FelixDispatchQueue" prio=10 tid=0x09d42c00 nid=0x8b3 in Object.wait()
> [0xb451a000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x846f0238> (a java.util.ArrayList)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:924)
> - locked<0x846f0238> (a java.util.ArrayList)
> at
> org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
> at
> org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:662)
>
> "Low Memory Detector" daemon prio=10 tid=0x09c8ec00 nid=0x8b1 runnable
> [0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread0" daemon prio=10 tid=0x09c8d000 nid=0x8b0 waiting on
> condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> "Signal Dispatcher" daemon prio=10 tid=0x09c8b400 nid=0x8af waiting on
> condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> "Finalizer" daemon prio=10 tid=0x09c83400 nid=0x8ae in Object.wait()
> [0xb4761000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x846f0418> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
> - locked<0x846f0418> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
> "Reference Handler" daemon prio=10 tid=0x09c81c00 nid=0x8ad in Object.wait()
> [0xb47b2000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x846f0098> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked<0x846f0098> (a java.lang.ref.Reference$Lock)
>
> "main" prio=10 tid=0x09c54000 nid=0x8ab in Object.wait() [0xb6c79000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x846f04a8> (a org.apache.felix.framework.util.ThreadGate)
> at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79)
> - locked<0x846f04a8> (a org.apache.felix.framework.util.ThreadGate)
> at org.apache.felix.framework.Felix.waitForStop(Felix.java:906)
> at org.apache.felix.main.Main.main(Main.java:295)
>
> "VM Thread" prio=10 tid=0x09c77800 nid=0x8ac runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x09c9ac00 nid=0x8b2 waiting on
> condition
>
> JNI global references: 1217
>
>
> Found one Java-level deadlock:
> =============================
> "BThread":
> waiting to lock monitor 0x09d5cbd0 (object 0x7f4869f0, a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
> which is held by "MessageExecutor"
> "MessageExecutor":
> waiting to lock monitor 0x09d5bf50 (object 0x7f50ba18, a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5),
> which is held by "BThread"
>
> Java stack information for the threads listed above:
> ===================================================
> "BThread":
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
> - waiting to lock<0x7f4869f0> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchImports(ModuleImpl.java:1390)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:722)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at com.axway.testb.BThread.run(BThread.java:32)
> "MessageExecutor":
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1761)
> - waiting to lock<0x7f50ba18> (a
> org.apache.felix.framework.ModuleImpl$ModuleClassLoaderJava5)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at
> org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645)
> at org.apache.felix.framework.resolver.WireImpl.getClass(WireImpl.java:99)
> at
> org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1427)
> at
> org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734)
> at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71)
> at
> org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at com.axway.testa.MessageExecutor.run(MessageExecutor.java:40)
>
> Found 1 deadlock.
>
>
>
>
> http://old.nabble.com/file/p31266713/test.zip test.zip
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org