[jira] [Commented] (OAK-4400) Correlate index with the index definition used to build it

["Chetan Mehrotra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15722275#comment-15722275 ] 

Chetan Mehrotra commented on OAK-4400:
--------------------------------------

[~alex.parvulescu] [~catholicon] I think I would need this feature for OAK-5218 so looking for solution. Few approach I can think of

h3. A - IndexDefinition based on Checkpoint 

# Have AsyncIndexUpdate communicate the after checkpoint as part of CommitInfo. Bit like approach being discussed in OAK-3606
# Index implementation like LuceneIndexEditor would save the checkpoint as part of close say under {{:status}} node
# At time of open in {{IndexNode}} code would retrieve the NodeState for index definition as per checkpoint saved in {{:status}} node. If not null then that would be used otherwise it fallbacks to current NodeState

This would ensure that {{IndexDefinition}} is created from same NodeState as it was at time of last incremental indexing. This would allow us to handle that case where index definition was changed and reindex flag was set. 

h3. B - Squeeze the NodeState as time of reindex

In this mode when reindexing is done then current NodeState related to index definition would be squeezed out and copied as hidden tree (leaving out the non visible part). And when IndexNode opens it would always construct the IndexDefinition based on this stored NodeState

This would be more resilient as any change done in index definition would not reflect in query 

Thoughts?

Thoughts?


> Correlate index with the index definition used to build it
> ----------------------------------------------------------
>
>                 Key: OAK-4400
>                 URL: https://issues.apache.org/jira/browse/OAK-4400
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: lucene, query
>    Affects Versions: 1.4
>            Reporter: Valentin Olteanu
>            Assignee: Chetan Mehrotra
>             Fix For: 1.8
>
>
> Currently, if the definition of an index is changed without reindexing, it will get in an "inconsistent" state. 
> Of course, the reindexing is usually necessary, but it would be useful to know with which definition the index was built. This could increase the visibility of the indexing state and help debugging issues related to it.
> Some questions this improvement should respond to:
> # What is the definition of the index when the (re)indexing was triggered?
> # Are there any changes in the definition since the trigger? Which?
> I can imagine a solution built by "versioning" the definition nodes (oak:QueryIndexDefinition). When the reindex is triggered, a new version of the node is created and the indexer stores a reference to it.
> This would also allow the indexer to keep using the same definition until a new reindex, even if changes are made meanwhile (i.e. use a fixed version instead of the latest definition).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)