You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by "Martinez, Antonio (Antonio)" <an...@alcatel-lucent.com> on 2010/02/27 00:39:27 UTC
Jackrabbit 1.6.0 deadlock
Hello,
I've been struggling for some time with a deadlock in Jackrabbit 1.4 (see https://issues.apache.org/jira/browse/JCR-2426).
Based on some recommendation in ticket I have moved to Jackrabbit 1.6.0 and used NIOFS (I'm using JVM version 1.6.0_06-b02 for Solaris SPARC)
With Jackrabbit 1.6.0, after some time of intensive load in the system I hit a jackrabbit deadlock
I see these bugs fixed in 1.6.1. I was wondering if any of these would fix the deadlock I'm seeing or if this is something new
Thanks,
Antonio
JCR-2443 AbstractSession should not synchronize on the session instance
JCR-2250 Base64 bug - last buffer not flushed
JCR-2356 Session holds LockToken after removeLockToken in XA Environment
JCR-2367 RepositoryCopier does not copy open-scoped Locks
JCR-2421 Unable to create repository using jackrabbit-webapp because a directory called "jackrabbit" already exists
JCR-769 Unable to login with two different Credentials to same workspace in one Transaction
JCR-2332 Unable to delete a non session-scoped locked node in XA Environment
JCR-2323 InputStream.read return value is ignored.
JCR-2364 NullPointerException when accessing the about.jsp page because of missing /META-INF/NOTICE.TXT
JCR-2297 Registering multiple node types with the same name in a single file must fail
JCR-2299 Bad check for sv:name attribute presence in system view import
JCR-2369 Problem importing node with binary property in a repository configured with datastore
---------------------------------------------------
Found one Java-level deadlock:
=============================
"jmssecondaryApplnJobExecutor-8":
waiting to lock monitor 0x054bdf88 (object 0xa0950f08, a org.apache.jackrabbit.core.state.NodeState),
which is held by "jmssecondaryApplnJobExecutor-7"
"jmssecondaryApplnJobExecutor-7":
waiting to lock monitor 0x067ee410 (object 0xa4ee3148, a java.lang.Object),
which is held by "jmssecondaryApplnJobExecutor-8"
Java stack information for the threads listed above:
===================================================
"jmssecondaryApplnJobExecutor-8":
at org.apache.jackrabbit.core.state.NodeState.getChildNodeEntry(NodeState.java:300)
- waiting to lock <0xa0950f08> (a org.apache.jackrabbit.core.state.NodeState)
at org.apache.jackrabbit.core.CachingHierarchyManager.nodeModified(CachingHierarchyManager.java:316)
- locked <0xa4ee3148> (a java.lang.Object)
at org.apache.jackrabbit.core.CachingHierarchyManager.stateModified(CachingHierarchyManager.java:293)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.SessionItemStateManager.stateModified(SessionItemStateManager.java:889)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.LocalItemStateManager.stateModified(LocalItemStateManager.java:452)
at org.apache.jackrabbit.core.state.XAItemStateManager.stateModified(XAItemStateManager.java:602)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.SharedItemStateManager.stateModified(SharedItemStateManager.java:400)
at org.apache.jackrabbit.core.state.ItemState.notifyStateUpdated(ItemState.java:244)
at org.apache.jackrabbit.core.state.ChangeLog.persisted(ChangeLog.java:297)
at org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:749)
at org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1115)
at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
at org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:354)
at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
at org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:325)
at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1111)
- locked <0x9c151520> (a org.apache.jackrabbit.core.XASessionImpl)
at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:915)
at org.apache.jackrabbit.jca.JCASessionHandle.save(JCASessionHandle.java:180)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.collectionBegin(InventoryNeListServiceImpl.java:153)
at com.alcatel.axs.app.idm.InventoryDiscoveryServiceImpl.doDiscoveryNow(InventoryDiscoveryServiceImpl.java:219)
at sun.reflect.GeneratedMethodAccessor1067.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:36)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:243)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955)
at org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:88)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.modelmbean.RequiredModelMBeanInvoker.invoke(RequiredModelMBeanInvoker.java:127)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.server.jmx.LazyMBeanServer.invoke(LazyMBeanServer.java:291)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
at $Proxy692.doDiscoveryNow(Unknown Source)
at com.alcatel.axs.app.idm.InventoryJob.execute(InventoryJob.java:41)
at com.alcatel.axs.container.jobmanager.ManagedJob.run(ManagedJob.java:55)
at com.alcatel.axs.container.jobmanager.JMSJobExecutor.onMessage(JMSJobExecutor.java:50)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:619)
"jmssecondaryApplnJobExecutor-7":
at org.apache.jackrabbit.core.CachingHierarchyManager.nodeAdded(CachingHierarchyManager.java:362)
- waiting to lock <0xa4ee3148> (a java.lang.Object)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(StateChangeDispatcher.java:159)
at org.apache.jackrabbit.core.state.SessionItemStateManager.nodeAdded(SessionItemStateManager.java:947)
at org.apache.jackrabbit.core.state.NodeState.notifyNodeAdded(NodeState.java:882)
at org.apache.jackrabbit.core.state.NodeState.addChildNodeEntry(NodeState.java:351)
- locked <0xa0950f08> (a org.apache.jackrabbit.core.state.NodeState)
at org.apache.jackrabbit.core.NodeImpl.createChildNode(NodeImpl.java:541)
- locked <0xa3e63c58> (a org.apache.jackrabbit.core.NodeImpl)
at org.apache.jackrabbit.core.NodeImpl.internalAddChildNode(NodeImpl.java:802)
at org.apache.jackrabbit.core.NodeImpl.internalAddNode(NodeImpl.java:735)
at org.apache.jackrabbit.core.NodeImpl.addNodeWithUuid(NodeImpl.java:2200)
- locked <0xa3e63ca8> (a org.apache.jackrabbit.core.NodeImpl)
at org.apache.jackrabbit.core.NodeImpl.addNode(NodeImpl.java:2133)
- locked <0xa3e63ca8> (a org.apache.jackrabbit.core.NodeImpl)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.createNeNode(InventoryNeListServiceImpl.java:641)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.collectionBegin(InventoryNeListServiceImpl.java:142)
at com.alcatel.axs.app.idm.InventoryDiscoveryServiceImpl.doDiscoveryNow(InventoryDiscoveryServiceImpl.java:219)
at sun.reflect.GeneratedMethodAccessor1067.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:36)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:243)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955)
at org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:88)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.modelmbean.RequiredModelMBeanInvoker.invoke(RequiredModelMBeanInvoker.java:127)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.server.jmx.LazyMBeanServer.invoke(LazyMBeanServer.java:291)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
at $Proxy692.doDiscoveryNow(Unknown Source)
at com.alcatel.axs.app.idm.InventoryJob.execute(InventoryJob.java:41)
at com.alcatel.axs.container.jobmanager.ManagedJob.run(ManagedJob.java:55)
at com.alcatel.axs.container.jobmanager.JMSJobExecutor.onMessage(JMSJobExecutor.java:50)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:619)
Found 1 deadlock.
RE: Jackrabbit 1.6.0 deadlock
Posted by "Martinez, Antonio (Antonio)" <an...@alcatel-lucent.com>.
Hi,
Any feedback on this ?
I would not like to create a bug if it is known issue.
Thank you,
Antonio
-----Original Message-----
From: Martinez, Antonio (Antonio) [mailto:antonio.martinez@alcatel-lucent.com]
Sent: Saturday, February 27, 2010 12:46 AM
To: users@jackrabbit.apache.org
Subject: RE: Jackrabbit 1.6.0 deadlock
Hello,
Unfortunately I got the same deadlock in jackrabbit 1.6.1.
These two threads are creating a node each under the same parent node. One of them is saving and the other is still adding it. Then the deadlock happened.
Seams basic, but is that not supported by jackrabbit for some reason? I did not hit this with jackrabbit 1.4.
Is this a jackrabbit 1.6.0/1.6.1 bug ?
Thanks,
Antonio
-----Original Message-----
From: Martinez, Antonio (Antonio) [mailto:antonio.martinez@alcatel-lucent.com]
Sent: Friday, February 26, 2010 3:39 PM
To: users@jackrabbit.apache.org
Subject: Jackrabbit 1.6.0 deadlock
Importance: High
Hello,
I've been struggling for some time with a deadlock in Jackrabbit 1.4 (see https://issues.apache.org/jira/browse/JCR-2426).
Based on some recommendation in ticket I have moved to Jackrabbit 1.6.0 and used NIOFS (I'm using JVM version 1.6.0_06-b02 for Solaris SPARC)
With Jackrabbit 1.6.0, after some time of intensive load in the system I hit a jackrabbit deadlock
I see these bugs fixed in 1.6.1. I was wondering if any of these would fix the deadlock I'm seeing or if this is something new
Thanks,
Antonio
JCR-2443 AbstractSession should not synchronize on the session instance
JCR-2250 Base64 bug - last buffer not flushed
JCR-2356 Session holds LockToken after removeLockToken in XA Environment
JCR-2367 RepositoryCopier does not copy open-scoped Locks
JCR-2421 Unable to create repository using jackrabbit-webapp because a directory called "jackrabbit" already exists
JCR-769 Unable to login with two different Credentials to same workspace in one Transaction
JCR-2332 Unable to delete a non session-scoped locked node in XA Environment
JCR-2323 InputStream.read return value is ignored.
JCR-2364 NullPointerException when accessing the about.jsp page because of missing /META-INF/NOTICE.TXT
JCR-2297 Registering multiple node types with the same name in a single file must fail
JCR-2299 Bad check for sv:name attribute presence in system view import
JCR-2369 Problem importing node with binary property in a repository configured with datastore
---------------------------------------------------
Found one Java-level deadlock:
=============================
"jmssecondaryApplnJobExecutor-8":
waiting to lock monitor 0x054bdf88 (object 0xa0950f08, a org.apache.jackrabbit.core.state.NodeState),
which is held by "jmssecondaryApplnJobExecutor-7"
"jmssecondaryApplnJobExecutor-7":
waiting to lock monitor 0x067ee410 (object 0xa4ee3148, a java.lang.Object),
which is held by "jmssecondaryApplnJobExecutor-8"
Java stack information for the threads listed above:
===================================================
"jmssecondaryApplnJobExecutor-8":
at org.apache.jackrabbit.core.state.NodeState.getChildNodeEntry(NodeState.java:300)
- waiting to lock <0xa0950f08> (a org.apache.jackrabbit.core.state.NodeState)
at org.apache.jackrabbit.core.CachingHierarchyManager.nodeModified(CachingHierarchyManager.java:316)
- locked <0xa4ee3148> (a java.lang.Object)
at org.apache.jackrabbit.core.CachingHierarchyManager.stateModified(CachingHierarchyManager.java:293)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.SessionItemStateManager.stateModified(SessionItemStateManager.java:889)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.LocalItemStateManager.stateModified(LocalItemStateManager.java:452)
at org.apache.jackrabbit.core.state.XAItemStateManager.stateModified(XAItemStateManager.java:602)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.SharedItemStateManager.stateModified(SharedItemStateManager.java:400)
at org.apache.jackrabbit.core.state.ItemState.notifyStateUpdated(ItemState.java:244)
at org.apache.jackrabbit.core.state.ChangeLog.persisted(ChangeLog.java:297)
at org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:749)
at org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1115)
at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
at org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:354)
at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
at org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:325)
at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1111)
- locked <0x9c151520> (a org.apache.jackrabbit.core.XASessionImpl)
at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:915)
at org.apache.jackrabbit.jca.JCASessionHandle.save(JCASessionHandle.java:180)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.collectionBegin(InventoryNeListServiceImpl.java:153)
at com.alcatel.axs.app.idm.InventoryDiscoveryServiceImpl.doDiscoveryNow(InventoryDiscoveryServiceImpl.java:219)
at sun.reflect.GeneratedMethodAccessor1067.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:36)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:243)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955)
at org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:88)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.modelmbean.RequiredModelMBeanInvoker.invoke(RequiredModelMBeanInvoker.java:127)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.server.jmx.LazyMBeanServer.invoke(LazyMBeanServer.java:291)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
at $Proxy692.doDiscoveryNow(Unknown Source)
at com.alcatel.axs.app.idm.InventoryJob.execute(InventoryJob.java:41)
at com.alcatel.axs.container.jobmanager.ManagedJob.run(ManagedJob.java:55)
at com.alcatel.axs.container.jobmanager.JMSJobExecutor.onMessage(JMSJobExecutor.java:50)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:619)
"jmssecondaryApplnJobExecutor-7":
at org.apache.jackrabbit.core.CachingHierarchyManager.nodeAdded(CachingHierarchyManager.java:362)
- waiting to lock <0xa4ee3148> (a java.lang.Object)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(StateChangeDispatcher.java:159)
at org.apache.jackrabbit.core.state.SessionItemStateManager.nodeAdded(SessionItemStateManager.java:947)
at org.apache.jackrabbit.core.state.NodeState.notifyNodeAdded(NodeState.java:882)
at org.apache.jackrabbit.core.state.NodeState.addChildNodeEntry(NodeState.java:351)
- locked <0xa0950f08> (a org.apache.jackrabbit.core.state.NodeState)
at org.apache.jackrabbit.core.NodeImpl.createChildNode(NodeImpl.java:541)
- locked <0xa3e63c58> (a org.apache.jackrabbit.core.NodeImpl)
at org.apache.jackrabbit.core.NodeImpl.internalAddChildNode(NodeImpl.java:802)
at org.apache.jackrabbit.core.NodeImpl.internalAddNode(NodeImpl.java:735)
at org.apache.jackrabbit.core.NodeImpl.addNodeWithUuid(NodeImpl.java:2200)
- locked <0xa3e63ca8> (a org.apache.jackrabbit.core.NodeImpl)
at org.apache.jackrabbit.core.NodeImpl.addNode(NodeImpl.java:2133)
- locked <0xa3e63ca8> (a org.apache.jackrabbit.core.NodeImpl)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.createNeNode(InventoryNeListServiceImpl.java:641)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.collectionBegin(InventoryNeListServiceImpl.java:142)
at com.alcatel.axs.app.idm.InventoryDiscoveryServiceImpl.doDiscoveryNow(InventoryDiscoveryServiceImpl.java:219)
at sun.reflect.GeneratedMethodAccessor1067.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:36)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:243)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955)
at org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:88)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.modelmbean.RequiredModelMBeanInvoker.invoke(RequiredModelMBeanInvoker.java:127)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.server.jmx.LazyMBeanServer.invoke(LazyMBeanServer.java:291)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
at $Proxy692.doDiscoveryNow(Unknown Source)
at com.alcatel.axs.app.idm.InventoryJob.execute(InventoryJob.java:41)
at com.alcatel.axs.container.jobmanager.ManagedJob.run(ManagedJob.java:55)
at com.alcatel.axs.container.jobmanager.JMSJobExecutor.onMessage(JMSJobExecutor.java:50)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:619)
Found 1 deadlock.
RE: Jackrabbit 1.6.0 deadlock
Posted by "Martinez, Antonio (Antonio)" <an...@alcatel-lucent.com>.
Hello,
Unfortunately I got the same deadlock in jackrabbit 1.6.1.
These two threads are creating a node each under the same parent node. One of them is saving and the other is still adding it. Then the deadlock happened.
Seams basic, but is that not supported by jackrabbit for some reason? I did not hit this with jackrabbit 1.4.
Is this a jackrabbit 1.6.0/1.6.1 bug ?
Thanks,
Antonio
-----Original Message-----
From: Martinez, Antonio (Antonio) [mailto:antonio.martinez@alcatel-lucent.com]
Sent: Friday, February 26, 2010 3:39 PM
To: users@jackrabbit.apache.org
Subject: Jackrabbit 1.6.0 deadlock
Importance: High
Hello,
I've been struggling for some time with a deadlock in Jackrabbit 1.4 (see https://issues.apache.org/jira/browse/JCR-2426).
Based on some recommendation in ticket I have moved to Jackrabbit 1.6.0 and used NIOFS (I'm using JVM version 1.6.0_06-b02 for Solaris SPARC)
With Jackrabbit 1.6.0, after some time of intensive load in the system I hit a jackrabbit deadlock
I see these bugs fixed in 1.6.1. I was wondering if any of these would fix the deadlock I'm seeing or if this is something new
Thanks,
Antonio
JCR-2443 AbstractSession should not synchronize on the session instance
JCR-2250 Base64 bug - last buffer not flushed
JCR-2356 Session holds LockToken after removeLockToken in XA Environment
JCR-2367 RepositoryCopier does not copy open-scoped Locks
JCR-2421 Unable to create repository using jackrabbit-webapp because a directory called "jackrabbit" already exists
JCR-769 Unable to login with two different Credentials to same workspace in one Transaction
JCR-2332 Unable to delete a non session-scoped locked node in XA Environment
JCR-2323 InputStream.read return value is ignored.
JCR-2364 NullPointerException when accessing the about.jsp page because of missing /META-INF/NOTICE.TXT
JCR-2297 Registering multiple node types with the same name in a single file must fail
JCR-2299 Bad check for sv:name attribute presence in system view import
JCR-2369 Problem importing node with binary property in a repository configured with datastore
---------------------------------------------------
Found one Java-level deadlock:
=============================
"jmssecondaryApplnJobExecutor-8":
waiting to lock monitor 0x054bdf88 (object 0xa0950f08, a org.apache.jackrabbit.core.state.NodeState),
which is held by "jmssecondaryApplnJobExecutor-7"
"jmssecondaryApplnJobExecutor-7":
waiting to lock monitor 0x067ee410 (object 0xa4ee3148, a java.lang.Object),
which is held by "jmssecondaryApplnJobExecutor-8"
Java stack information for the threads listed above:
===================================================
"jmssecondaryApplnJobExecutor-8":
at org.apache.jackrabbit.core.state.NodeState.getChildNodeEntry(NodeState.java:300)
- waiting to lock <0xa0950f08> (a org.apache.jackrabbit.core.state.NodeState)
at org.apache.jackrabbit.core.CachingHierarchyManager.nodeModified(CachingHierarchyManager.java:316)
- locked <0xa4ee3148> (a java.lang.Object)
at org.apache.jackrabbit.core.CachingHierarchyManager.stateModified(CachingHierarchyManager.java:293)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.SessionItemStateManager.stateModified(SessionItemStateManager.java:889)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.LocalItemStateManager.stateModified(LocalItemStateManager.java:452)
at org.apache.jackrabbit.core.state.XAItemStateManager.stateModified(XAItemStateManager.java:602)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(StateChangeDispatcher.java:111)
at org.apache.jackrabbit.core.state.SharedItemStateManager.stateModified(SharedItemStateManager.java:400)
at org.apache.jackrabbit.core.state.ItemState.notifyStateUpdated(ItemState.java:244)
at org.apache.jackrabbit.core.state.ChangeLog.persisted(ChangeLog.java:297)
at org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:749)
at org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1115)
at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
at org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:354)
at org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
at org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:325)
at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1111)
- locked <0x9c151520> (a org.apache.jackrabbit.core.XASessionImpl)
at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:915)
at org.apache.jackrabbit.jca.JCASessionHandle.save(JCASessionHandle.java:180)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.collectionBegin(InventoryNeListServiceImpl.java:153)
at com.alcatel.axs.app.idm.InventoryDiscoveryServiceImpl.doDiscoveryNow(InventoryDiscoveryServiceImpl.java:219)
at sun.reflect.GeneratedMethodAccessor1067.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:36)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:243)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955)
at org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:88)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.modelmbean.RequiredModelMBeanInvoker.invoke(RequiredModelMBeanInvoker.java:127)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.server.jmx.LazyMBeanServer.invoke(LazyMBeanServer.java:291)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
at $Proxy692.doDiscoveryNow(Unknown Source)
at com.alcatel.axs.app.idm.InventoryJob.execute(InventoryJob.java:41)
at com.alcatel.axs.container.jobmanager.ManagedJob.run(ManagedJob.java:55)
at com.alcatel.axs.container.jobmanager.JMSJobExecutor.onMessage(JMSJobExecutor.java:50)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:619)
"jmssecondaryApplnJobExecutor-7":
at org.apache.jackrabbit.core.CachingHierarchyManager.nodeAdded(CachingHierarchyManager.java:362)
- waiting to lock <0xa4ee3148> (a java.lang.Object)
at org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(StateChangeDispatcher.java:159)
at org.apache.jackrabbit.core.state.SessionItemStateManager.nodeAdded(SessionItemStateManager.java:947)
at org.apache.jackrabbit.core.state.NodeState.notifyNodeAdded(NodeState.java:882)
at org.apache.jackrabbit.core.state.NodeState.addChildNodeEntry(NodeState.java:351)
- locked <0xa0950f08> (a org.apache.jackrabbit.core.state.NodeState)
at org.apache.jackrabbit.core.NodeImpl.createChildNode(NodeImpl.java:541)
- locked <0xa3e63c58> (a org.apache.jackrabbit.core.NodeImpl)
at org.apache.jackrabbit.core.NodeImpl.internalAddChildNode(NodeImpl.java:802)
at org.apache.jackrabbit.core.NodeImpl.internalAddNode(NodeImpl.java:735)
at org.apache.jackrabbit.core.NodeImpl.addNodeWithUuid(NodeImpl.java:2200)
- locked <0xa3e63ca8> (a org.apache.jackrabbit.core.NodeImpl)
at org.apache.jackrabbit.core.NodeImpl.addNode(NodeImpl.java:2133)
- locked <0xa3e63ca8> (a org.apache.jackrabbit.core.NodeImpl)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.createNeNode(InventoryNeListServiceImpl.java:641)
at com.alcatel.axs.app.idm.nelist.InventoryNeListServiceImpl.collectionBegin(InventoryNeListServiceImpl.java:142)
at com.alcatel.axs.app.idm.InventoryDiscoveryServiceImpl.doDiscoveryNow(InventoryDiscoveryServiceImpl.java:219)
at sun.reflect.GeneratedMethodAccessor1067.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:36)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:243)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955)
at org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:88)
at org.jboss.mx.server.RawDynamicInvoker.invoke(RawDynamicInvoker.java:164)
at org.jboss.mx.modelmbean.RequiredModelMBeanInvoker.invoke(RequiredModelMBeanInvoker.java:127)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.server.jmx.LazyMBeanServer.invoke(LazyMBeanServer.java:291)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:288)
at $Proxy692.doDiscoveryNow(Unknown Source)
at com.alcatel.axs.app.idm.InventoryJob.execute(InventoryJob.java:41)
at com.alcatel.axs.container.jobmanager.ManagedJob.run(ManagedJob.java:55)
at com.alcatel.axs.container.jobmanager.JMSJobExecutor.onMessage(JMSJobExecutor.java:50)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:531)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:466)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:322)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:260)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:944)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:868)
at java.lang.Thread.run(Thread.java:619)
Found 1 deadlock.