[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable

["Chetan Mehrotra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15492487#comment-15492487 ] 

Chetan Mehrotra commented on OAK-4805:
--------------------------------------

Current patch would address this specific issue but I would like to take it one step further. Per current design we can have bad index in following ways

# Index got corrupted (for some reason) - In this case for each query {{IndexTracker}} would try to acquire {{IndexNode}} which would fail while getting constructed i.e. creation of IndexSearcher would lead to read of index resulting in an exception. This index would then be ignored due to catch block in {{IndexTracker#findIndexNode}}
# Index content is valid but some later change caused the IndexDefinition to have invalid configuration - In this case IndexNode would get created but exception from it would break down the query engine (current issue)

In both cases once we have a bad index then it would continue to participate in subsequent query evaluation and 
# flood the logs with failure message
# Slow down the processing as for Case #1 above IndexTracker would still try to initialize it

So to address it properly we should do following
# Have some state in {{IndexTracker}} around failed index. If some failure is seen either due to #1 and #2 then we mark such index paths as failed and log the exception
# Upon lookup for index for path we check if its failed - If yes then ignore
# The state would should be cleared once any index gets updated i.e. in {{IndexTracker#diffAndUpdate}}

 As chance of index resuming back to normal is only after any index gets update it should be fine to clear the state only upon index update. With this we should see such warning every 5 sec at minimum. If require we can make such logging more time bound

This would ensure that a fail index should not affect broader working of system.

Probably all this can be done in another issue and current issue gets fixed with proposed patch as its much more critical. In that case would suggest to catch {{Exception}} instead of {{IllegalArgumentException}} to guard against other possible exceptions

> Misconfigured lucene index definition can render the whole system unusable
> --------------------------------------------------------------------------
>
>                 Key: OAK-4805
>                 URL: https://issues.apache.org/jira/browse/OAK-4805
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: lucene
>            Reporter: Vikas Saurabh
>            Assignee: Vikas Saurabh
>              Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
>             Fix For: 1.6
>
>         Attachments: OAK-4805.patch
>
>
> Mis-configured index definition can throw an exception while collecting plans. This causes any query (even unrelated ones) to not work as cost calculation logic would consult a badly constructed index def. Overall a mis-configured index definition can practically grind the whole system to halt as the whole query framework stops working.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)