You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Peter Harrison <ch...@gmail.com> on 2017/03/06 21:57:14 UTC

DocumentStoreException

After updating Oak a little while ago I began seeing the following error on
startup. A restart is required to resolve the issue, but it can recur, thus
needing a number of restarts.

This is making deployment and operations unreliable.

Once it occurs it appears to enter a endless loop.

This was not happening in a previous earlier version of Oak.


2017-03-06 21:37:31,274 ERROR:
org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo
- This oak instance failed to update the lease in time and can therefore no
longer access this DocumentNodeStore.
2017-03-06 21:37:31,274 WARN :
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
- Background operation failed: org.apache.jackrabbit.oak.plug
ins.document.DocumentStoreException: This oak instance failed to update the
lease in time and can therefore no longer access this DocumentNodeStore.
org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: This oak
instance failed to update the lease in time and can therefore no longer
access this DocumentNodeStore.
        at org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo.
leaseExpired(ClusterNodeInfo.java:1154)
        at org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo.p
erformLeaseCheck(ClusterNodeInfo.java:662)
        at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDo
cumentStoreWrapper.performLeaseCheck(LeaseCheckDocumentStoreWrapper.java:59)
        at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDo
cumentStoreWrapper.find(LeaseCheckDocumentStoreWrapper.java:72)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
.backgroundRead(DocumentNodeStore.java:1990)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
.internalRunBackgroundReadOperations(DocumentNodeStore.java:1911)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
.runBackgroundReadOperations(DocumentNodeStore.java:1895)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
$BackgroundReadOperation.execute(DocumentNodeStore.java:2872)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
$NodeStoreTask.run(DocumentNodeStore.java:2825)
        at java.lang.Thread.run(Thread.java:745)

Re: DocumentStoreException

Posted by Julian Reschke <ju...@gmx.de>.
a) Oak version? (before/after)

b) Persistence Mongo?

c) Try with DEBUG logging on 
org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo

Best regards, Julian


On 2017-03-06 22:57, Peter Harrison wrote:
> After updating Oak a little while ago I began seeing the following error on
> startup. A restart is required to resolve the issue, but it can recur, thus
> needing a number of restarts.
>
> This is making deployment and operations unreliable.
>
> Once it occurs it appears to enter a endless loop.
>
> This was not happening in a previous earlier version of Oak.
>
>
> 2017-03-06 21:37:31,274 ERROR:
> org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo
> - This oak instance failed to update the lease in time and can therefore no
> longer access this DocumentNodeStore.
> 2017-03-06 21:37:31,274 WARN :
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> - Background operation failed: org.apache.jackrabbit.oak.plug
> ins.document.DocumentStoreException: This oak instance failed to update the
> lease in time and can therefore no longer access this DocumentNodeStore.
> org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: This oak
> instance failed to update the lease in time and can therefore no longer
> access this DocumentNodeStore.
>         at org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo.
> leaseExpired(ClusterNodeInfo.java:1154)
>         at org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo.p
> erformLeaseCheck(ClusterNodeInfo.java:662)
>         at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDo
> cumentStoreWrapper.performLeaseCheck(LeaseCheckDocumentStoreWrapper.java:59)
>         at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDo
> cumentStoreWrapper.find(LeaseCheckDocumentStoreWrapper.java:72)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> .backgroundRead(DocumentNodeStore.java:1990)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> .internalRunBackgroundReadOperations(DocumentNodeStore.java:1911)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> .runBackgroundReadOperations(DocumentNodeStore.java:1895)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> $BackgroundReadOperation.execute(DocumentNodeStore.java:2872)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> $NodeStoreTask.run(DocumentNodeStore.java:2825)
>         at java.lang.Thread.run(Thread.java:745)
>


Re: DocumentStoreException

Posted by Clay Ferguson <wc...@gmail.com>.
Peter,
I know you're already aware of meta64, but at the very bottom of the
following file is a shutdown procedure for gracefully terminating that I
think works. (Various dispose, close, and shutdown methods are required.)

https://github.com/Clay-Ferguson/meta64/blob/master/src/main/java/com/meta64/mobile/repo/OakRepository.java

I've noticed if i terminate my server, without that code executing, it will
complain about lease stuff at the next restart, and will have to wait like
60seconds in a countdown type thing to start next time. Also be sure you
have all your code that deinitializes stuff in finally blocks also so that
you can be sure none of your processes are leaving things hanging around,
in memory unterminated.. Only good finally blocks can fix that.

or of course it may be an actual bug in JCR. :)


Best regards,
Clay Ferguson
wclayf@gmail.com


On Mon, Mar 6, 2017 at 3:57 PM, Peter Harrison <ch...@gmail.com> wrote:

> After updating Oak a little while ago I began seeing the following error on
> startup. A restart is required to resolve the issue, but it can recur, thus
> needing a number of restarts.
>
> This is making deployment and operations unreliable.
>
> Once it occurs it appears to enter a endless loop.
>
> This was not happening in a previous earlier version of Oak.
>
>
> 2017-03-06 21:37:31,274 ERROR:
> org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo
> - This oak instance failed to update the lease in time and can therefore no
> longer access this DocumentNodeStore.
> 2017-03-06 21:37:31,274 WARN :
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> - Background operation failed: org.apache.jackrabbit.oak.plug
> ins.document.DocumentStoreException: This oak instance failed to update
> the
> lease in time and can therefore no longer access this DocumentNodeStore.
> org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: This
> oak
> instance failed to update the lease in time and can therefore no longer
> access this DocumentNodeStore.
>         at org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo.
> leaseExpired(ClusterNodeInfo.java:1154)
>         at org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo.p
> erformLeaseCheck(ClusterNodeInfo.java:662)
>         at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDo
> cumentStoreWrapper.performLeaseCheck(LeaseCheckDocumentStoreWrapper
> .java:59)
>         at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDo
> cumentStoreWrapper.find(LeaseCheckDocumentStoreWrapper.java:72)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> .backgroundRead(DocumentNodeStore.java:1990)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> .internalRunBackgroundReadOperations(DocumentNodeStore.java:1911)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> .runBackgroundReadOperations(DocumentNodeStore.java:1895)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> $BackgroundReadOperation.execute(DocumentNodeStore.java:2872)
>         at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> $NodeStoreTask.run(DocumentNodeStore.java:2825)
>         at java.lang.Thread.run(Thread.java:745)
>