You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Mohit Kataria (Jira)" <ji...@apache.org> on 2022/03/28 04:28:00 UTC

[jira] [Commented] (OAK-9721) Add state information while indeximporting

    [ https://issues.apache.org/jira/browse/OAK-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513130#comment-17513130 ] 

Mohit Kataria commented on OAK-9721:
------------------------------------

The importindex method consists of following steps:
 # switchLanes : This step changes names of indexing lanes so that current indexing  don't alter this index. Assuming current indexing lane is "async ", property changes from "async": "async" to "async":"temp-async" and "async-previous":"async". 
 # importIndexData: In this step, index data is imported and indexes are marked for disabling if index has supersedes property. At this stage we have a new index committed with switchedlanes.
 # bringIndexUpToDate: With this step we stop current indexing lane, find diff between latest checkpoint and checkpoint we are using to import index.
 # releaseCheckpoint: Release the old checkpoint we are using for importing.

 

As these states are persisted ones, we can't come back to exactly same condition of nodestore in case the process halts/exits at any state. To mitigate this to some extent, in case of failure, we revert back lane names (there will be a window for which data is not indexed). But as we are saving import state, we can figure out whether we should run import process again or not. 

> Add state information while indeximporting
> ------------------------------------------
>
>                 Key: OAK-9721
>                 URL: https://issues.apache.org/jira/browse/OAK-9721
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: indexing
>            Reporter: Mohit Kataria
>            Assignee: Mohit Kataria
>            Priority: Major
>
> While importing index (IndexImporter), there are majorly 4 steps. In case of failure the whole index becomes unusable. 
> So to handle this, we are introducing a property to manage state of indeximport and add retry logic on each step.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)