You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Shawn Jiang <ge...@gmail.com> on 2011/01/29 04:19:36 UTC

deadlocks found in tomcat-web m2 release when running tck.

See my comments to the tck results in tck list and the log in GERONIMO-5782.

-- 
Shawn

Re: deadlocks found in tomcat-web m2 release when running tck.

Posted by David Jencks <da...@yahoo.com>.
Hi Ivan,

Here's my current thinking on this.  I could easily be wrong about many aspects of this :-) and I'm not done...

I think there are two kinds of problems to avoid here.

The easy one is to synchronize access to our configuration tracking objects.  I think we can do this by synchronizing on configurations for all access to configurations, configurationModel, and reloadingConfiguration.  This is really similar to your commit, but without the new object and (I think) synchronizing access to configurationModel.

The harder part is that we don't want two threads trying to change the state of a configuration at the same time.  So I think we need an Artifact to lock object map and when we are changing the state of a configuration we secure the appropriate lock object and synchronize on it while the state change is happening.  Possibly the ConfigurationStatus in the configurationModel can be the lock object, I'm not sure if the lifecycle is correct for the status object to be the lock object.

I think we also need to check before each state change (after getting the lock) what state the configuration is actually in. 

Finally I don't think a single reloadingConfiguration will work any more.  I'm not sure if the map of lock objects will work for this too or if we need something else.

Many thanks for fixing at least the initial problem so quickly!
david jencks

On Jan 29, 2011, at 5:59 PM, Ivan wrote:

> For this single issue, another lock on the loadingConfiguration is required, too. I attached a proposed patch on GERONIMO-5782 
> But that is only for this one, if no objection, I would commit it soon. As many of jsf/ejb cases are blocked by this. Personally, I always get nervous while introducing more locks in one classes :-) And I guess that we might check some other bundle extender codes. I am still working on another dead lock on the WAB deployment, it uses a new thread for the deployment, too.
> Thanks.
> 
> 2011/1/29 David Jencks <da...@yahoo.com>
> We might have to make the SimpleConfigurationManager less synchronized or better synchronized.  I think the first rule of avoiding deadlocks is to not call unknown code from within a synchronized block.  We are definitely calling such code here, we're starting the entire configuration from inside a synchronized block.
> 
> Basically I think we need to review the SimpleConfigurationManager code and make the locking finer grained and avoid locking when calling out.  For instance we could synchronize getConfiguration on the artifact-configuration map rather than the configuration manager object.
> 
> I'll try to look at this in the next couple days.  Any other analysis or suggestions would be more than welcome.
> 
> thanks
> david jencks
> 
> On Jan 28, 2011, at 7:45 PM, Ivan wrote:
> 
>> The related stack should be, it is caused by the new thread used by Tomcat for listener initialization, MyFaces listener called some functions in the SimpleConfigurationManager
>> I do not have a good idea so far, maybe we should try to disable the function in Tomcat ? Or any invocation on the configuraiton manager in any listener/servlet init will cause the dead lock.
>> 
>> --->
>> "Thread-58" daemon prio=10 tid=0x0000000040ac5000 nid=0x3ff7 waiting for monitor entry [0x00007f6cdd948000]
>>   java.lang.Thread.State: BLOCKED (on object monitor)
>>        at org.apache.geronimo.kernel.config.SimpleConfigurationManager.getConfiguration(SimpleConfigurationManager.java:264)
>>        - waiting to lock <0x00007f6cf84b1570> (a org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>>        at org.apache.geronimo.naming.reference.ConfigurationAwareReference.getConfiguration(ConfigurationAwareReference.java:61)
>>        at org.apache.geronimo.naming.reference.ConfigurationAwareReference.resolveTargetName(ConfigurationAwareReference.java:79)
>>        at org.apache.geronimo.naming.reference.GBeanReference.getContent(GBeanReference.java:46)
>>        at org.apache.xbean.naming.context.ContextUtil.resolve(ContextUtil.java:61)
>>        at org.apache.xbean.naming.context.ContextUtil$ReadOnlyBinding.getObject(ContextUtil.java:199)
>>        at org.apache.xbean.naming.context.ContextFederation.getFederatedBindings(ContextFederation.java:119)
>>        at org.apache.xbean.naming.context.AbstractFederatedContext.getBindings(AbstractFederatedContext.java:99)
>>        at org.apache.xbean.naming.context.AbstractFederatedContext.getBinding(AbstractFederatedContext.java:86)
>>        at org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:133)
>>        at org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:605)
>>        at org.apache.geronimo.naming.java.RootContext.lookup(RootContext.java:55)
>>        at org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:148)
>>        at javax.naming.InitialContext.lookup(InitialContext.java:392)
>>        at org.apache.myfaces.application.ApplicationImpl.getProjectStage(ApplicationImpl.java:713)
>>        at org.apache.webbeans.jsf.OwbApplication.getProjectStage(OwbApplication.java:443)
>>        at javax.faces.context.FacesContext.isProjectStage(FacesContext.java:411)
>>        at org.apache.myfaces.config.FacesConfigurator.configureLifecycle(FacesConfigurator.java:1011)
>>        at org.apache.myfaces.config.FacesConfigurator.configure(FacesConfigurator.java:465)
>>        at org.apache.myfaces.webapp.AbstractFacesInitializer.buildConfiguration(AbstractFacesInitializer.java:303)
>>        at org.apache.myfaces.webapp.Jsp21FacesInitializer.initContainerIntegration(Jsp21FacesInitializer.java:73)
>>        at org.apache.myfaces.webapp.AbstractFacesInitializer.initFaces(AbstractFacesInitializer.java:126)
>>        at org.apache.myfaces.webapp.StartupServletContextListener.contextInitialized(StartupServletContextListener.java:110)
>>        at org.apache.geronimo.myfaces.webapp.GeronimoStartupServletContextListener.contextInitialized(GeronimoStartupServletContextListener.java:55)
>>        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4521)
>>        at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5004)
>>        at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:4999)
>>        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>        at java.lang.Thread.run(Thread.java:619)
>> 
>> "RMI TCP Connection(6)-172.16.21.3" daemon prio=10 tid=0x0000000040c8e800 nid=0x3ff2 waiting on condition [0x00007f6cdf189000]
>>   java.lang.Thread.State: WAITING (parking)
>>        at sun.misc.Unsafe.park(Native Method)
>>        - parking to wait for  <0x00007f6cfbc53170> (a java.util.concurrent.FutureTask$Sync)
>>        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>>        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>        at org.apache.tomcat.util.threads.DedicatedThreadExecutor.execute(DedicatedThreadExecutor.java:56)
>>        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4998)
>>        - locked <0x00007f6cfbc53250> (a org.apache.geronimo.tomcat.GeronimoStandardContext)
>>        at org.apache.geronimo.tomcat.GeronimoStandardContext.access$101(GeronimoStandardContext.java:105)
>>        at org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java:728)
>>        at org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke(GeronimoBeforeAfterValve.java:47)
>>        at org.apache.geronimo.tomcat.valve.ProtectedTargetValve.invoke(ProtectedTargetValve.java:53)
>>        at org.apache.geronimo.tomcat.GeronimoStandardContext.startInternal(GeronimoStandardContext.java:463)
>>        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
>>        - locked <0x00007f6cfbc53250> (a org.apache.geronimo.tomcat.GeronimoStandardContext)
>>        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:812)
>>        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:787)
>>        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
>>        at org.apache.geronimo.tomcat.TomcatContainer.addContext(TomcatContainer.java:310)
>>        at org.apache.geronimo.tomcat.TomcatWebAppContext.doStart(TomcatWebAppContext.java:568)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:975)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:271)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>>        at org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>>        - locked <0x00007f6cfbae5210> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>>        - locked <0x00007f6cfbae5210> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>>        at org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>>        - locked <0x00007f6cfbadedf0> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>>        - locked <0x00007f6cfbadedf0> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:127)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:560)
>>        at org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:386)
>>        at org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:460)
>>        at org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:226)
>>        at org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:698)
>>        - locked <0x00007f6cf84b1570> (a org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>>        at org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:677)
>>        - locked <0x00007f6cf84b1570> (a org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>>        at org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>>        at org.apache.geronimo.kernel.KernelGBean.invoke(KernelGBean.java:344)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>>        at org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>>        at org.apache.geronimo.system.jmx.MBeanGBeanBridge.invoke(MBeanGBeanBridge.java:172)
>>        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>>        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>>        at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>>        at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>>        at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1367)
>>        at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>>        at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>>        at sun.rmi.transport.Transport$1.run(Transport.java:159)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>>        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>>        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>>        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>        at java.lang.Thread.run(Thread.java:619)
>> <--
>> 
>> 2011/1/29 Shawn Jiang <ge...@gmail.com>
>> See my comments to the tck results in tck list and the log in GERONIMO-5782.
>> 
>> -- 
>> Shawn
>> 
>> 
>> 
>> -- 
>> Ivan
> 
> 
> 
> 
> -- 
> Ivan


Re: deadlocks found in tomcat-web m2 release when running tck.

Posted by Ivan <xh...@gmail.com>.
I went ahead to commit the patch to trunk at revision: 106518, and I could
deploy the jsf application now.
If any problem, please feel free to update it, and I might not connect to
the Internet as I will back to my hometown.
thanks.

2011/1/30 Ivan <xh...@gmail.com>

> For this single issue, another lock on the loadingConfiguration is
> required, too. I attached a proposed patch on *GERONIMO-5782<https://issues.apache.org/jira/browse/GERONIMO-5782>
> *But that is only for this one, if no objection, I would commit it soon.
> As many of jsf/ejb cases are blocked by this. Personally, I always get
> nervous while introducing more locks in one classes :-) And I guess that we
> might check some other bundle extender codes. I am still working on another
> dead lock on the WAB deployment, it uses a new thread for the deployment,
> too.
> Thanks.
>
> 2011/1/29 David Jencks <da...@yahoo.com>
>
> We might have to make the SimpleConfigurationManager less synchronized or
>> better synchronized.  I think the first rule of avoiding deadlocks is to not
>> call unknown code from within a synchronized block.  We are definitely
>> calling such code here, we're starting the entire configuration from inside
>> a synchronized block.
>>
>> Basically I think we need to review the SimpleConfigurationManager code
>> and make the locking finer grained and avoid locking when calling out.  For
>> instance we could synchronize getConfiguration on the artifact-configuration
>> map rather than the configuration manager object.
>>
>> I'll try to look at this in the next couple days.  Any other analysis or
>> suggestions would be more than welcome.
>>
>> thanks
>> david jencks
>>
>> On Jan 28, 2011, at 7:45 PM, Ivan wrote:
>>
>> The related stack should be, it is caused by the new thread used by Tomcat
>> for listener initialization, MyFaces listener called some functions in the
>> SimpleConfigurationManager
>> I do not have a good idea so far, maybe we should try to disable the
>> function in Tomcat ? Or any invocation on the configuraiton manager in any
>> listener/servlet init will cause the dead lock.
>>
>> --->
>> "Thread-58" daemon prio=10 tid=0x0000000040ac5000 nid=0x3ff7 waiting for
>> monitor entry [0x00007f6cdd948000]
>>   java.lang.Thread.State: BLOCKED (on object monitor)
>>        at
>> org.apache.geronimo.kernel.config.SimpleConfigurationManager.getConfiguration(SimpleConfigurationManager.java:264)
>>        - waiting to lock <0x00007f6cf84b1570> (a
>> org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>>        at
>> org.apache.geronimo.naming.reference.ConfigurationAwareReference.getConfiguration(ConfigurationAwareReference.java:61)
>>        at
>> org.apache.geronimo.naming.reference.ConfigurationAwareReference.resolveTargetName(ConfigurationAwareReference.java:79)
>>        at
>> org.apache.geronimo.naming.reference.GBeanReference.getContent(GBeanReference.java:46)
>>        at
>> org.apache.xbean.naming.context.ContextUtil.resolve(ContextUtil.java:61)
>>        at
>> org.apache.xbean.naming.context.ContextUtil$ReadOnlyBinding.getObject(ContextUtil.java:199)
>>        at
>> org.apache.xbean.naming.context.ContextFederation.getFederatedBindings(ContextFederation.java:119)
>>        at
>> org.apache.xbean.naming.context.AbstractFederatedContext.getBindings(AbstractFederatedContext.java:99)
>>        at
>> org.apache.xbean.naming.context.AbstractFederatedContext.getBinding(AbstractFederatedContext.java:86)
>>        at
>> org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:133)
>>        at
>> org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:605)
>>        at
>> org.apache.geronimo.naming.java.RootContext.lookup(RootContext.java:55)
>>        at
>> org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:148)
>>        at javax.naming.InitialContext.lookup(InitialContext.java:392)
>>        at
>> org.apache.myfaces.application.ApplicationImpl.getProjectStage(ApplicationImpl.java:713)
>>        at
>> org.apache.webbeans.jsf.OwbApplication.getProjectStage(OwbApplication.java:443)
>>        at
>> javax.faces.context.FacesContext.isProjectStage(FacesContext.java:411)
>>        at
>> org.apache.myfaces.config.FacesConfigurator.configureLifecycle(FacesConfigurator.java:1011)
>>        at
>> org.apache.myfaces.config.FacesConfigurator.configure(FacesConfigurator.java:465)
>>        at
>> org.apache.myfaces.webapp.AbstractFacesInitializer.buildConfiguration(AbstractFacesInitializer.java:303)
>>        at
>> org.apache.myfaces.webapp.Jsp21FacesInitializer.initContainerIntegration(Jsp21FacesInitializer.java:73)
>>        at
>> org.apache.myfaces.webapp.AbstractFacesInitializer.initFaces(AbstractFacesInitializer.java:126)
>>        at
>> org.apache.myfaces.webapp.StartupServletContextListener.contextInitialized(StartupServletContextListener.java:110)
>>        at
>> org.apache.geronimo.myfaces.webapp.GeronimoStartupServletContextListener.contextInitialized(GeronimoStartupServletContextListener.java:55)
>>        at
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4521)
>>        at
>> org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5004)
>>        at
>> org.apache.catalina.core.StandardContext$1.call(StandardContext.java:4999)
>>        at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>        at java.lang.Thread.run(Thread.java:619)
>>
>> "RMI TCP Connection(6)-172.16.21.3" daemon prio=10 tid=0x0000000040c8e800
>> nid=0x3ff2 waiting on condition [0x00007f6cdf189000]
>>   java.lang.Thread.State: WAITING (parking)
>>        at sun.misc.Unsafe.park(Native Method)
>>        - parking to wait for  <0x00007f6cfbc53170> (a
>> java.util.concurrent.FutureTask$Sync)
>>        at
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>>        at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>>        at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>>        at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>>        at
>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>        at
>> org.apache.tomcat.util.threads.DedicatedThreadExecutor.execute(DedicatedThreadExecutor.java:56)
>>        at
>> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4998)
>>        - locked <0x00007f6cfbc53250> (a
>> org.apache.geronimo.tomcat.GeronimoStandardContext)
>>        at
>> org.apache.geronimo.tomcat.GeronimoStandardContext.access$101(GeronimoStandardContext.java:105)
>>        at
>> org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java:728)
>>        at
>> org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke(GeronimoBeforeAfterValve.java:47)
>>        at
>> org.apache.geronimo.tomcat.valve.ProtectedTargetValve.invoke(ProtectedTargetValve.java:53)
>>        at
>> org.apache.geronimo.tomcat.GeronimoStandardContext.startInternal(GeronimoStandardContext.java:463)
>>        at
>> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
>>        - locked <0x00007f6cfbc53250> (a
>> org.apache.geronimo.tomcat.GeronimoStandardContext)
>>        at
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:812)
>>        at
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:787)
>>        at
>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
>>        at
>> org.apache.geronimo.tomcat.TomcatContainer.addContext(TomcatContainer.java:310)
>>        at
>> org.apache.geronimo.tomcat.TomcatWebAppContext.doStart(TomcatWebAppContext.java:568)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:975)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:271)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>>        - locked <0x00007f6cfbae5210> (a
>> org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>>        - locked <0x00007f6cfbae5210> (a
>> org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>>        at
>> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>>        at
>> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>>        at
>> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>>        - locked <0x00007f6cfbadedf0> (a
>> org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>>        - locked <0x00007f6cfbadedf0> (a
>> org.apache.geronimo.gbean.runtime.GBeanDependency)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>>        at
>> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>>        at
>> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>>        at
>> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:127)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:560)
>>        at
>> org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:386)
>>        at
>> org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:460)
>>        at
>> org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:226)
>>        at
>> org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:698)
>>        - locked <0x00007f6cf84b1570> (a
>> org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>>        at
>> org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:677)
>>        - locked <0x00007f6cf84b1570> (a
>> org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>>        at
>> org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>>        at
>> org.apache.geronimo.kernel.KernelGBean.invoke(KernelGBean.java:344)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>>        at
>> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>>        at
>> org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>>        at
>> org.apache.geronimo.system.jmx.MBeanGBeanBridge.invoke(MBeanGBeanBridge.java:172)
>>        at
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>>        at
>> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1367)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>>        at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>>        at sun.rmi.transport.Transport$1.run(Transport.java:159)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>>        at
>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>>        at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>>        at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>        at java.lang.Thread.run(Thread.java:619)
>> <--
>>
>> 2011/1/29 Shawn Jiang <ge...@gmail.com>
>>
>>> See my comments to the tck results in tck list and the log
>>> in GERONIMO-5782.
>>>
>>> --
>>> Shawn
>>>
>>
>>
>>
>> --
>> Ivan
>>
>>
>>
>
>
> --
> Ivan
>



-- 
Ivan

Re: deadlocks found in tomcat-web m2 release when running tck.

Posted by Ivan <xh...@gmail.com>.
For this single issue, another lock on the loadingConfiguration is required,
too. I attached a proposed patch on
*GERONIMO-5782<https://issues.apache.org/jira/browse/GERONIMO-5782>
*But that is only for this one, if no objection, I would commit it soon. As
many of jsf/ejb cases are blocked by this. Personally, I always get nervous
while introducing more locks in one classes :-) And I guess that we might
check some other bundle extender codes. I am still working on another dead
lock on the WAB deployment, it uses a new thread for the deployment, too.
Thanks.

2011/1/29 David Jencks <da...@yahoo.com>

> We might have to make the SimpleConfigurationManager less synchronized or
> better synchronized.  I think the first rule of avoiding deadlocks is to not
> call unknown code from within a synchronized block.  We are definitely
> calling such code here, we're starting the entire configuration from inside
> a synchronized block.
>
> Basically I think we need to review the SimpleConfigurationManager code and
> make the locking finer grained and avoid locking when calling out.  For
> instance we could synchronize getConfiguration on the artifact-configuration
> map rather than the configuration manager object.
>
> I'll try to look at this in the next couple days.  Any other analysis or
> suggestions would be more than welcome.
>
> thanks
> david jencks
>
> On Jan 28, 2011, at 7:45 PM, Ivan wrote:
>
> The related stack should be, it is caused by the new thread used by Tomcat
> for listener initialization, MyFaces listener called some functions in the
> SimpleConfigurationManager
> I do not have a good idea so far, maybe we should try to disable the
> function in Tomcat ? Or any invocation on the configuraiton manager in any
> listener/servlet init will cause the dead lock.
>
> --->
> "Thread-58" daemon prio=10 tid=0x0000000040ac5000 nid=0x3ff7 waiting for
> monitor entry [0x00007f6cdd948000]
>   java.lang.Thread.State: BLOCKED (on object monitor)
>        at
> org.apache.geronimo.kernel.config.SimpleConfigurationManager.getConfiguration(SimpleConfigurationManager.java:264)
>        - waiting to lock <0x00007f6cf84b1570> (a
> org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>        at
> org.apache.geronimo.naming.reference.ConfigurationAwareReference.getConfiguration(ConfigurationAwareReference.java:61)
>        at
> org.apache.geronimo.naming.reference.ConfigurationAwareReference.resolveTargetName(ConfigurationAwareReference.java:79)
>        at
> org.apache.geronimo.naming.reference.GBeanReference.getContent(GBeanReference.java:46)
>        at
> org.apache.xbean.naming.context.ContextUtil.resolve(ContextUtil.java:61)
>        at
> org.apache.xbean.naming.context.ContextUtil$ReadOnlyBinding.getObject(ContextUtil.java:199)
>        at
> org.apache.xbean.naming.context.ContextFederation.getFederatedBindings(ContextFederation.java:119)
>        at
> org.apache.xbean.naming.context.AbstractFederatedContext.getBindings(AbstractFederatedContext.java:99)
>        at
> org.apache.xbean.naming.context.AbstractFederatedContext.getBinding(AbstractFederatedContext.java:86)
>        at
> org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:133)
>        at
> org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:605)
>        at
> org.apache.geronimo.naming.java.RootContext.lookup(RootContext.java:55)
>        at
> org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:148)
>        at javax.naming.InitialContext.lookup(InitialContext.java:392)
>        at
> org.apache.myfaces.application.ApplicationImpl.getProjectStage(ApplicationImpl.java:713)
>        at
> org.apache.webbeans.jsf.OwbApplication.getProjectStage(OwbApplication.java:443)
>        at
> javax.faces.context.FacesContext.isProjectStage(FacesContext.java:411)
>        at
> org.apache.myfaces.config.FacesConfigurator.configureLifecycle(FacesConfigurator.java:1011)
>        at
> org.apache.myfaces.config.FacesConfigurator.configure(FacesConfigurator.java:465)
>        at
> org.apache.myfaces.webapp.AbstractFacesInitializer.buildConfiguration(AbstractFacesInitializer.java:303)
>        at
> org.apache.myfaces.webapp.Jsp21FacesInitializer.initContainerIntegration(Jsp21FacesInitializer.java:73)
>        at
> org.apache.myfaces.webapp.AbstractFacesInitializer.initFaces(AbstractFacesInitializer.java:126)
>        at
> org.apache.myfaces.webapp.StartupServletContextListener.contextInitialized(StartupServletContextListener.java:110)
>        at
> org.apache.geronimo.myfaces.webapp.GeronimoStartupServletContextListener.contextInitialized(GeronimoStartupServletContextListener.java:55)
>        at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4521)
>        at
> org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5004)
>        at
> org.apache.catalina.core.StandardContext$1.call(StandardContext.java:4999)
>        at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
>
> "RMI TCP Connection(6)-172.16.21.3" daemon prio=10 tid=0x0000000040c8e800
> nid=0x3ff2 waiting on condition [0x00007f6cdf189000]
>   java.lang.Thread.State: WAITING (parking)
>        at sun.misc.Unsafe.park(Native Method)
>        - parking to wait for  <0x00007f6cfbc53170> (a
> java.util.concurrent.FutureTask$Sync)
>        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>        at
> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>        at
> org.apache.tomcat.util.threads.DedicatedThreadExecutor.execute(DedicatedThreadExecutor.java:56)
>        at
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4998)
>        - locked <0x00007f6cfbc53250> (a
> org.apache.geronimo.tomcat.GeronimoStandardContext)
>        at
> org.apache.geronimo.tomcat.GeronimoStandardContext.access$101(GeronimoStandardContext.java:105)
>        at
> org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java:728)
>        at
> org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke(GeronimoBeforeAfterValve.java:47)
>        at
> org.apache.geronimo.tomcat.valve.ProtectedTargetValve.invoke(ProtectedTargetValve.java:53)
>        at
> org.apache.geronimo.tomcat.GeronimoStandardContext.startInternal(GeronimoStandardContext.java:463)
>        at
> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
>        - locked <0x00007f6cfbc53250> (a
> org.apache.geronimo.tomcat.GeronimoStandardContext)
>        at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:812)
>        at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:787)
>        at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
>        at
> org.apache.geronimo.tomcat.TomcatContainer.addContext(TomcatContainer.java:310)
>        at
> org.apache.geronimo.tomcat.TomcatWebAppContext.doStart(TomcatWebAppContext.java:568)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:975)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:271)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>        at
> org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>        - locked <0x00007f6cfbae5210> (a
> org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at
> org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>        - locked <0x00007f6cfbae5210> (a
> org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at
> org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>        at
> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>        at
> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>        at
> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>        at
> org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>        - locked <0x00007f6cfbadedf0> (a
> org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at
> org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>        - locked <0x00007f6cfbadedf0> (a
> org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at
> org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>        at
> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>        at
> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>        at
> org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:127)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:560)
>        at
> org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:386)
>        at
> org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:460)
>        at
> org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:226)
>        at
> org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:698)
>        - locked <0x00007f6cf84b1570> (a
> org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>        at
> org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:677)
>        - locked <0x00007f6cf84b1570> (a
> org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at
> org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>        at
> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>        at
> org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>        at
> org.apache.geronimo.kernel.KernelGBean.invoke(KernelGBean.java:344)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at
> org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>        at
> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>        at
> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>        at
> org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>        at
> org.apache.geronimo.system.jmx.MBeanGBeanBridge.invoke(MBeanGBeanBridge.java:172)
>        at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>        at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>        at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>        at
> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>        at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1367)
>        at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>        at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at
> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>        at sun.rmi.transport.Transport$1.run(Transport.java:159)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>        at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>        at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>        at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> <--
>
> 2011/1/29 Shawn Jiang <ge...@gmail.com>
>
>> See my comments to the tck results in tck list and the log
>> in GERONIMO-5782.
>>
>> --
>> Shawn
>>
>
>
>
> --
> Ivan
>
>
>


-- 
Ivan

Re: deadlocks found in tomcat-web m2 release when running tck.

Posted by David Jencks <da...@yahoo.com>.
We might have to make the SimpleConfigurationManager less synchronized or better synchronized.  I think the first rule of avoiding deadlocks is to not call unknown code from within a synchronized block.  We are definitely calling such code here, we're starting the entire configuration from inside a synchronized block.

Basically I think we need to review the SimpleConfigurationManager code and make the locking finer grained and avoid locking when calling out.  For instance we could synchronize getConfiguration on the artifact-configuration map rather than the configuration manager object.

I'll try to look at this in the next couple days.  Any other analysis or suggestions would be more than welcome.

thanks
david jencks

On Jan 28, 2011, at 7:45 PM, Ivan wrote:

> The related stack should be, it is caused by the new thread used by Tomcat for listener initialization, MyFaces listener called some functions in the SimpleConfigurationManager
> I do not have a good idea so far, maybe we should try to disable the function in Tomcat ? Or any invocation on the configuraiton manager in any listener/servlet init will cause the dead lock.
> 
> --->
> "Thread-58" daemon prio=10 tid=0x0000000040ac5000 nid=0x3ff7 waiting for monitor entry [0x00007f6cdd948000]
>   java.lang.Thread.State: BLOCKED (on object monitor)
>        at org.apache.geronimo.kernel.config.SimpleConfigurationManager.getConfiguration(SimpleConfigurationManager.java:264)
>        - waiting to lock <0x00007f6cf84b1570> (a org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>        at org.apache.geronimo.naming.reference.ConfigurationAwareReference.getConfiguration(ConfigurationAwareReference.java:61)
>        at org.apache.geronimo.naming.reference.ConfigurationAwareReference.resolveTargetName(ConfigurationAwareReference.java:79)
>        at org.apache.geronimo.naming.reference.GBeanReference.getContent(GBeanReference.java:46)
>        at org.apache.xbean.naming.context.ContextUtil.resolve(ContextUtil.java:61)
>        at org.apache.xbean.naming.context.ContextUtil$ReadOnlyBinding.getObject(ContextUtil.java:199)
>        at org.apache.xbean.naming.context.ContextFederation.getFederatedBindings(ContextFederation.java:119)
>        at org.apache.xbean.naming.context.AbstractFederatedContext.getBindings(AbstractFederatedContext.java:99)
>        at org.apache.xbean.naming.context.AbstractFederatedContext.getBinding(AbstractFederatedContext.java:86)
>        at org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:133)
>        at org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:605)
>        at org.apache.geronimo.naming.java.RootContext.lookup(RootContext.java:55)
>        at org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:148)
>        at javax.naming.InitialContext.lookup(InitialContext.java:392)
>        at org.apache.myfaces.application.ApplicationImpl.getProjectStage(ApplicationImpl.java:713)
>        at org.apache.webbeans.jsf.OwbApplication.getProjectStage(OwbApplication.java:443)
>        at javax.faces.context.FacesContext.isProjectStage(FacesContext.java:411)
>        at org.apache.myfaces.config.FacesConfigurator.configureLifecycle(FacesConfigurator.java:1011)
>        at org.apache.myfaces.config.FacesConfigurator.configure(FacesConfigurator.java:465)
>        at org.apache.myfaces.webapp.AbstractFacesInitializer.buildConfiguration(AbstractFacesInitializer.java:303)
>        at org.apache.myfaces.webapp.Jsp21FacesInitializer.initContainerIntegration(Jsp21FacesInitializer.java:73)
>        at org.apache.myfaces.webapp.AbstractFacesInitializer.initFaces(AbstractFacesInitializer.java:126)
>        at org.apache.myfaces.webapp.StartupServletContextListener.contextInitialized(StartupServletContextListener.java:110)
>        at org.apache.geronimo.myfaces.webapp.GeronimoStartupServletContextListener.contextInitialized(GeronimoStartupServletContextListener.java:55)
>        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4521)
>        at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5004)
>        at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:4999)
>        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> 
> "RMI TCP Connection(6)-172.16.21.3" daemon prio=10 tid=0x0000000040c8e800 nid=0x3ff2 waiting on condition [0x00007f6cdf189000]
>   java.lang.Thread.State: WAITING (parking)
>        at sun.misc.Unsafe.park(Native Method)
>        - parking to wait for  <0x00007f6cfbc53170> (a java.util.concurrent.FutureTask$Sync)
>        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>        at org.apache.tomcat.util.threads.DedicatedThreadExecutor.execute(DedicatedThreadExecutor.java:56)
>        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4998)
>        - locked <0x00007f6cfbc53250> (a org.apache.geronimo.tomcat.GeronimoStandardContext)
>        at org.apache.geronimo.tomcat.GeronimoStandardContext.access$101(GeronimoStandardContext.java:105)
>        at org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java:728)
>        at org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke(GeronimoBeforeAfterValve.java:47)
>        at org.apache.geronimo.tomcat.valve.ProtectedTargetValve.invoke(ProtectedTargetValve.java:53)
>        at org.apache.geronimo.tomcat.GeronimoStandardContext.startInternal(GeronimoStandardContext.java:463)
>        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
>        - locked <0x00007f6cfbc53250> (a org.apache.geronimo.tomcat.GeronimoStandardContext)
>        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:812)
>        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:787)
>        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
>        at org.apache.geronimo.tomcat.TomcatContainer.addContext(TomcatContainer.java:310)
>        at org.apache.geronimo.tomcat.TomcatWebAppContext.doStart(TomcatWebAppContext.java:568)
>        at org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:975)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:271)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>        at org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>        at org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>        - locked <0x00007f6cfbae5210> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>        - locked <0x00007f6cfbae5210> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>        at org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
>        at org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
>        - locked <0x00007f6cfbadedf0> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
>        - locked <0x00007f6cfbadedf0> (a org.apache.geronimo.gbean.runtime.GBeanDependency)
>        at org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
>        at org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
>        at org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:127)
>        at org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:560)
>        at org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:386)
>        at org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:460)
>        at org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:226)
>        at org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:698)
>        - locked <0x00007f6cf84b1570> (a org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>        at org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:677)
>        - locked <0x00007f6cf84b1570> (a org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>        at org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>        at org.apache.geronimo.kernel.KernelGBean.invoke(KernelGBean.java:344)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
>        at org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
>        at org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
>        at org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
>        at org.apache.geronimo.system.jmx.MBeanGBeanBridge.invoke(MBeanGBeanBridge.java:172)
>        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>        at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>        at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>        at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1367)
>        at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>        at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>        at sun.rmi.transport.Transport$1.run(Transport.java:159)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> <--
> 
> 2011/1/29 Shawn Jiang <ge...@gmail.com>
> See my comments to the tck results in tck list and the log in GERONIMO-5782.
> 
> -- 
> Shawn
> 
> 
> 
> -- 
> Ivan


Re: deadlocks found in tomcat-web m2 release when running tck.

Posted by Ivan <xh...@gmail.com>.
The related stack should be, it is caused by the new thread used by Tomcat
for listener initialization, MyFaces listener called some functions in the
SimpleConfigurationManager
I do not have a good idea so far, maybe we should try to disable the
function in Tomcat ? Or any invocation on the configuraiton manager in any
listener/servlet init will cause the dead lock.

--->
"Thread-58" daemon prio=10 tid=0x0000000040ac5000 nid=0x3ff7 waiting for
monitor entry [0x00007f6cdd948000]
  java.lang.Thread.State: BLOCKED (on object monitor)
       at
org.apache.geronimo.kernel.config.SimpleConfigurationManager.getConfiguration(SimpleConfigurationManager.java:264)
       - waiting to lock <0x00007f6cf84b1570> (a
org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
       at
org.apache.geronimo.naming.reference.ConfigurationAwareReference.getConfiguration(ConfigurationAwareReference.java:61)
       at
org.apache.geronimo.naming.reference.ConfigurationAwareReference.resolveTargetName(ConfigurationAwareReference.java:79)
       at
org.apache.geronimo.naming.reference.GBeanReference.getContent(GBeanReference.java:46)
       at
org.apache.xbean.naming.context.ContextUtil.resolve(ContextUtil.java:61)
       at
org.apache.xbean.naming.context.ContextUtil$ReadOnlyBinding.getObject(ContextUtil.java:199)
       at
org.apache.xbean.naming.context.ContextFederation.getFederatedBindings(ContextFederation.java:119)
       at
org.apache.xbean.naming.context.AbstractFederatedContext.getBindings(AbstractFederatedContext.java:99)
       at
org.apache.xbean.naming.context.AbstractFederatedContext.getBinding(AbstractFederatedContext.java:86)
       at
org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:133)
       at
org.apache.xbean.naming.context.AbstractContext.lookup(AbstractContext.java:605)
       at
org.apache.geronimo.naming.java.RootContext.lookup(RootContext.java:55)
       at
org.apache.aries.jndi.DelegateContext.lookup(DelegateContext.java:148)
       at javax.naming.InitialContext.lookup(InitialContext.java:392)
       at
org.apache.myfaces.application.ApplicationImpl.getProjectStage(ApplicationImpl.java:713)
       at
org.apache.webbeans.jsf.OwbApplication.getProjectStage(OwbApplication.java:443)
       at
javax.faces.context.FacesContext.isProjectStage(FacesContext.java:411)
       at
org.apache.myfaces.config.FacesConfigurator.configureLifecycle(FacesConfigurator.java:1011)
       at
org.apache.myfaces.config.FacesConfigurator.configure(FacesConfigurator.java:465)
       at
org.apache.myfaces.webapp.AbstractFacesInitializer.buildConfiguration(AbstractFacesInitializer.java:303)
       at
org.apache.myfaces.webapp.Jsp21FacesInitializer.initContainerIntegration(Jsp21FacesInitializer.java:73)
       at
org.apache.myfaces.webapp.AbstractFacesInitializer.initFaces(AbstractFacesInitializer.java:126)
       at
org.apache.myfaces.webapp.StartupServletContextListener.contextInitialized(StartupServletContextListener.java:110)
       at
org.apache.geronimo.myfaces.webapp.GeronimoStartupServletContextListener.contextInitialized(GeronimoStartupServletContextListener.java:55)
       at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4521)
       at
org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5004)
       at
org.apache.catalina.core.StandardContext$1.call(StandardContext.java:4999)
       at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
       at java.util.concurrent.FutureTask.run(FutureTask.java:138)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:619)

"RMI TCP Connection(6)-172.16.21.3" daemon prio=10 tid=0x0000000040c8e800
nid=0x3ff2 waiting on condition [0x00007f6cdf189000]
  java.lang.Thread.State: WAITING (parking)
       at sun.misc.Unsafe.park(Native Method)
       - parking to wait for  <0x00007f6cfbc53170> (a
java.util.concurrent.FutureTask$Sync)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
       at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
       at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
       at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
       at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
       at java.util.concurrent.FutureTask.get(FutureTask.java:83)
       at
org.apache.tomcat.util.threads.DedicatedThreadExecutor.execute(DedicatedThreadExecutor.java:56)
       at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4998)
       - locked <0x00007f6cfbc53250> (a
org.apache.geronimo.tomcat.GeronimoStandardContext)
       at
org.apache.geronimo.tomcat.GeronimoStandardContext.access$101(GeronimoStandardContext.java:105)
       at
org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java:728)
       at
org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke(GeronimoBeforeAfterValve.java:47)
       at
org.apache.geronimo.tomcat.valve.ProtectedTargetValve.invoke(ProtectedTargetValve.java:53)
       at
org.apache.geronimo.tomcat.GeronimoStandardContext.startInternal(GeronimoStandardContext.java:463)
       at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
       - locked <0x00007f6cfbc53250> (a
org.apache.geronimo.tomcat.GeronimoStandardContext)
       at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:812)
       at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:787)
       at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
       at
org.apache.geronimo.tomcat.TomcatContainer.addContext(TomcatContainer.java:310)
       at
org.apache.geronimo.tomcat.TomcatWebAppContext.doStart(TomcatWebAppContext.java:568)
       at
org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:975)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:271)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
       at
org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
       at
org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
       - locked <0x00007f6cfbae5210> (a
org.apache.geronimo.gbean.runtime.GBeanDependency)
       at
org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
       - locked <0x00007f6cfbae5210> (a
org.apache.geronimo.gbean.runtime.GBeanDependency)
       at
org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
       at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
       at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
       at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
       at
org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:546)
       at
org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:110)
       - locked <0x00007f6cfbadedf0> (a
org.apache.geronimo.gbean.runtime.GBeanDependency)
       at
org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:145)
       - locked <0x00007f6cfbadedf0> (a
org.apache.geronimo.gbean.runtime.GBeanDependency)
       at
org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:119)
       at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:175)
       at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
       at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:253)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:301)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:105)
       at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:127)
       at
org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:560)
       at
org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:386)
       at
org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:460)
       at
org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:226)
       at
org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:698)
       - locked <0x00007f6cf84b1570> (a
org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
       at
org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:677)
       - locked <0x00007f6cf84b1570> (a
org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at
org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
       at
org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
       at
org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
       at
org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
       at
org.apache.geronimo.kernel.KernelGBean.invoke(KernelGBean.java:344)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at
org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
       at
org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:131)
       at
org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:872)
       at
org.apache.geronimo.kernel.basic.BasicKernel.invoke(BasicKernel.java:245)
       at
org.apache.geronimo.system.jmx.MBeanGBeanBridge.invoke(MBeanGBeanBridge.java:172)
       at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
       at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
       at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
       at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
       at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
       at java.security.AccessController.doPrivileged(Native Method)
       at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1367)
       at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
       at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
       at sun.rmi.transport.Transport$1.run(Transport.java:159)
       at java.security.AccessController.doPrivileged(Native Method)
       at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
       at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
       at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
       at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:619)
<--

2011/1/29 Shawn Jiang <ge...@gmail.com>

> See my comments to the tck results in tck list and the log
> in GERONIMO-5782.
>
> --
> Shawn
>



-- 
Ivan