[jira] [Commented] (OAK-4939) Isolate corrupted index and make async indexer more resilient

["Chetan Mehrotra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686259#comment-15686259 ] 

Chetan Mehrotra commented on OAK-4939:
--------------------------------------

Implemented the support in [feature branch|https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-4939?expand=1]. Also attached as [patch|^OAK-4939-v1.diff]

With this AsyncIndexerService would now mark any index which fails to update for 30 mins (configurable) as {{corrupt}} and ignore such indexes from further indexing. 

When any index is marked as corrupt following log entry would be made
{noformat}
2016-11-22 12:52:35,484 INFO  NA [async-index-update-fulltext-async] o.a.j.o.p.i.AsyncIndexUpdate - Marking [/oak:index/lucene] as corrupt. The index is failing since Tue Nov 22 12:51:25 IST 2016 ,1 indexing cycles, failed 7 times, skipped 0 time 
{noformat}

Post this when any new content gets indexed and any such corrupt index is skipped then following warn entry would be made
{noformat}
2016-11-22 12:52:35,485 WARN  NA [async-index-update-fulltext-async] o.a.j.o.p.index.IndexUpdate - Ignoring corrupt index [/oak:index/lucene] which has been marked as corrupt since [2016-11-22T12:51:25.492+05:30]. This index MUST be reindexed for indexing to work properly 
{noformat}

This info would also be see in MBean

!corrupt-index-mbean.png!

Later once the index is reindexed following log entry would be made
{noformat}
2016-11-22 12:56:25,486 INFO  NA [async-index-update-fulltext-async] o.a.j.o.p.index.IndexUpdate - Removing corrupt flag from index [/oak:index/lucene] which has been marked as corrupt since [corrupt = 2016-11-22T12:51:25.492+05:30] 
{noformat}

> Isolate corrupted index and make async indexer more resilient
> -------------------------------------------------------------
>
>                 Key: OAK-4939
>                 URL: https://issues.apache.org/jira/browse/OAK-4939
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: lucene, query
>            Reporter: Chetan Mehrotra
>            Assignee: Chetan Mehrotra
>             Fix For: 1.6
>
>         Attachments: OAK-4939-v1.diff, corrupt-index-mbean.png
>
>
> Currently if any one of the async index gets corrupted it brings down the whole async indexer and no other index gets updated untill system administrator reindexes the problamatic async index. 
> Instead of fail all we should isolate such corrupted index and mark them as corrupted. And still let async indexer progress for other working indexes. 
> This would ensure that one corrupted index does not affect the whole system and allow the application to work partially. 
> Feature branch - https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-4939?expand=1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)