You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Mohit Kataria (Jira)" <ji...@apache.org> on 2022/03/28 04:28:00 UTC
[jira] [Commented] (OAK-9721) Add state information while indeximporting
[ https://issues.apache.org/jira/browse/OAK-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513130#comment-17513130 ]
Mohit Kataria commented on OAK-9721:
------------------------------------
The importindex method consists of following steps:
# switchLanes : This step changes names of indexing lanes so that current indexing don't alter this index. Assuming current indexing lane is "async ", property changes from "async": "async" to "async":"temp-async" and "async-previous":"async".
# importIndexData: In this step, index data is imported and indexes are marked for disabling if index has supersedes property. At this stage we have a new index committed with switchedlanes.
# bringIndexUpToDate: With this step we stop current indexing lane, find diff between latest checkpoint and checkpoint we are using to import index.
# releaseCheckpoint: Release the old checkpoint we are using for importing.
As these states are persisted ones, we can't come back to exactly same condition of nodestore in case the process halts/exits at any state. To mitigate this to some extent, in case of failure, we revert back lane names (there will be a window for which data is not indexed). But as we are saving import state, we can figure out whether we should run import process again or not.
> Add state information while indeximporting
> ------------------------------------------
>
> Key: OAK-9721
> URL: https://issues.apache.org/jira/browse/OAK-9721
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing
> Reporter: Mohit Kataria
> Assignee: Mohit Kataria
> Priority: Major
>
> While importing index (IndexImporter), there are majorly 4 steps. In case of failure the whole index becomes unusable.
> So to handle this, we are introducing a property to manage state of indeximport and add retry logic on each step.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)