Cleanup Callback to IndexEditor in case of exception in processing

[Chetan Mehrotra <ch...@gmail.com>]

Currently while performing indexing if there is any exception thrown
by one of the IndexEditor then the whole commit is aborted. However
other IndexEditors do not get a notification that there is a failure
and hence cannot perform any cleanup like releasing external
resources.

This problem does not effect editors which only deal with NodeStore.
However if a editor is dealing with external resources like
LuceneIndexEditor which might be using FSDirectory and has taken locks
on file system then it cause problem as seen in OAK-3053.

Would it be possible to modify current design to allow some callback
to IndexEditor if they are implementing Closeable?

Thoughts?

Chetan Mehrotra

Re: Cleanup Callback to IndexEditor in case of exception in processing

[Alex Parvulescu <al...@gmail.com>]

Hi,

> For now just a callback for failure is missing. The editors current
> anyway perform required cleanup when leaving the root node which kind
> of act like a success callback. If we can get both then much better!

I'm all in favor of adding a failure callback to allow for resource
cleanup. But I would not want to start fixing problems we don't yet have,
ie. for the successful finish event we have the #leave method which works
quite well already.

alex




On Thu, Jul 23, 2015 at 1:50 PM, Chetan Mehrotra <ch...@gmail.com>
wrote:

> On Thu, Jul 23, 2015 at 4:19 PM, Davide Giannella <da...@apache.org>
> wrote:
> > So to have a callback that is always invoked either on
> > success or failure.
>
> For now just a callback for failure is missing. The editors current
> anyway perform required cleanup when leaving the root node which kind
> of act like a success callback. If we can get both then much better!
>
> Chetan Mehrotra
>

Re: Cleanup Callback to IndexEditor in case of exception in processing

[Chetan Mehrotra <ch...@gmail.com>]

On Thu, Jul 23, 2015 at 4:19 PM, Davide Giannella <da...@apache.org> wrote:
> So to have a callback that is always invoked either on
> success or failure.

For now just a callback for failure is missing. The editors current
anyway perform required cleanup when leaving the root node which kind
of act like a success callback. If we can get both then much better!

Chetan Mehrotra

Re: Cleanup Callback to IndexEditor in case of exception in processing

[Davide Giannella <da...@apache.org>]

On 23/07/2015 05:25, Chetan Mehrotra wrote:
> Currently while performing indexing if there is any exception thrown
> by one of the IndexEditor then the whole commit is aborted. However
> other IndexEditors do not get a notification that there is a failure
> and hence cannot perform any cleanup like releasing external
> resources.
>
> This problem does not effect editors which only deal with NodeStore.
> However if a editor is dealing with external resources like
> LuceneIndexEditor which might be using FSDirectory and has taken locks
> on file system then it cause problem as seen in OAK-3053.
>
> Would it be possible to modify current design to allow some callback
> to IndexEditor if they are implementing Closeable?
>
> Thoughts?
>
I like the idea and I think we could extend it with a concept of
success/failure. So to have a callback that is always invoked either on
success or failure. When failing it could provide additional info as
well so that the implementers could act accordingly.

Davide