[jira] [Created] (OAK-5601) documentMk backgroundRead should handle missing journal entries

["Stefan Egli (JIRA)" <ji...@apache.org>]

Stefan Egli created OAK-5601:
--------------------------------

             Summary: documentMk backgroundRead should handle missing journal entries
                 Key: OAK-5601
                 URL: https://issues.apache.org/jira/browse/OAK-5601
             Project: Jackrabbit Oak
          Issue Type: Bug
          Components: core
    Affects Versions: 1.4.1
            Reporter: Stefan Egli
            Assignee: Stefan Egli


The following exception has been encountered:
{noformat}03.02.2017 02:39:08.068 *WARN* [DocumentNodeStore background read thread (1)] org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore Background operation failed: java.lang.IllegalStateException: M
issing external change for branch revision: b3-00000159f839b963-00000000
java.lang.IllegalStateException: Missing external change for branch revision: b3-00000159f839b963-00000000
        at org.apache.jackrabbit.oak.plugins.document.JournalEntry$3$1.computeNext(JournalEntry.java:321)
        at org.apache.jackrabbit.oak.plugins.document.JournalEntry$3$1.computeNext(JournalEntry.java:309)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.jackrabbit.oak.plugins.document.JournalEntry.addTo(JournalEntry.java:287)
        at org.apache.jackrabbit.oak.plugins.document.JournalEntry.fillExternalChanges(JournalEntry.java:210)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.backgroundRead(DocumentNodeStore.java:1813)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.internalRunBackgroundReadOperations(DocumentNodeStore.java:1702)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.runBackgroundReadOperations(DocumentNodeStore.java:1687)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$BackgroundReadOperation.execute(DocumentNodeStore.java:2633)
        at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$NodeStoreTask.run(DocumentNodeStore.java:2592)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

That revision b3-00000159f839b963-00000000 corresponds to 1485928249699ns == 2017-02-01T05:50:49+00:00, which means the revision that the backgroundRead is trying to read here is 21 hours old.

The default journal GC maxAge is 6 hours though.

When it tries to read an old revision that is already deleted it [throws an IllegalStateException|https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.4.1/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/JournalEntry.java#L321]. That one is not caught properly, except in [NodeStoreTask.run|https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.4.1/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java#L2593]. 

This means the backgroundRead has failed, with a warn, and will be retried 1 second later. Just that nothing has changed, the journal entry can still not be found, so the same warning is issued. 

Resulting in the situation that the backgroundRead will never recover.

That {{IllegalStateException}} should be caught [when calling fillExternalChanges|https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.4.1/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java#L1814], resulting in a fall-back to not reading from the journal but going the old route.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)