[jira] Commented: (JCR-2618) Almost all application server threads were waiting for ReaderLock or WriterLock

["Sergei (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JCR-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864690#action_12864690 ] 

Sergei commented on JCR-2618:
-----------------------------

Hi everyone!

Thank you for your feedbacks!

We have upgraded our system up to 1.5.7 version (we avoid upgrade to 1.6.2 or 2.1.*  because of backward compability of these releases with a legacy data). Nevertheless we have had this issue once again and have to upgrade to the next stable version. I guess it will be 1.6.2

@ Thomas: Yes, you are right - it's not a deadlock. But the BLOCKED thread (please find attached ClusterNodeBlockStackttrace.txt) was waiting in this code so long that another threads were waiting it (were blocked by it)(please find attached LockStackTrace.txt). After some hours almost all HTTP threads on the server were busy (=waiting BLOCKED thread) and a new http connection with the server could not be set and we had to reset it. 

We have analyzed logs and found the next trace (I will post a piece of it):

Caused by: javax.jcr.ItemNotFoundException: failed to build path of bb6be0d8-f2c9-469b-b87c-a6fedda04722: ce739ec1-75b6-4a20-b96a-8e970d263564 has no child entry for bb6be0d8-f2c9-469b-b87c-a6fedda04722
at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:289)


As I can see the item was not found by id. But how it can happen and why?

Thanks in advice.


> Almost all application server threads were waiting for ReaderLock or WriterLock 
> --------------------------------------------------------------------------------
>
>                 Key: JCR-2618
>                 URL: https://issues.apache.org/jira/browse/JCR-2618
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>    Affects Versions: 1.5.0, 1.5.2
>         Environment: Sun OS, Oracle DB, SAP AS
>            Reporter: Sergei
>         Attachments: ClusterNodeBlockStackttrace.txt, LockStackTrace.txt
>
>
> Hello Community!
> We need to advice on  the server failure cases which related to Jackrabbit Content repository functionality.
> Almost all application server threads were waiting for ReaderLock or WriterLock (please find attached LockStackTrace.txt). In addition to the Read-/WriteLock there was blocked thread related to Jackrabbit cluster nodes functionality (please find attached ClusterNodeBlockStackttrace.txt).
> We have clustered environment and using the version of the Jackrabbit library as follows:
> concurrent-1.3.4.jar
> derby-10.2.1.6.jar
> jackrabbit-1.5.2-src.jar
> jackrabbit-api-1.5.0.jar
> jackrabbit-core-1.5.2.jar
> jackrabbit-jca-1.5.2.jar
> jackrabbit-jcr-commons-1.5.2.jar
> jackrabbit-spi-1.5.0.jar
> jackrabbit-spi-commons-1.5.0.jar
> jackrabbit-text-extractors-1.5.0.jar
> jcr-1.0.jar
> lucene-core-2.3.2.jar  
> The advice that we need:
> 1)	What could cause appearance of a Lock which prevent getting Reader-/WriterLock ?
> 2)	Can it be related to the blocked thread from ClusterNodeBlockStackttrace.txt ?(please see attachments)
> We found that an update of GLOBAL_REVISION  table can take up to 30 min ("GLOBAL_REVISION" - internal table name in Jackrabbit).  
> Is it  possible that this issue is related to the Read-\WriteLocks?  
> Thanks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.