[jira] [Commented] (OAK-2809) Save Lucene directory listing as array property

["Thomas Mueller (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516558#comment-14516558 ] 

Thomas Mueller commented on OAK-2809:
-------------------------------------

I see this more as a workaround. It would be much better if we can solve the problem in the DocumentNodeStore, for example run GC more often.

It is also a risk, because in some cases using an array property might actually give us _more_ problems (for example: large list -> out of memory). If the property is frequently updated and / or if there are many entries, the node would become very large. Performance of adding n entries in n transactions is O(n^2), while without the property it should be O(n\).

But I'm not against doing that. What I would do is add a simple feature flag (system property) where this can be enabled / disabled, in case it turns out to be a problem, and in case we want to run benchmarks with this feature enabled against this feature disabled.


> Save Lucene directory listing as array property
> -----------------------------------------------
>
>                 Key: OAK-2809
>                 URL: https://issues.apache.org/jira/browse/OAK-2809
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: lucene
>            Reporter: Chetan Mehrotra
>             Fix For: 1.3.0
>
>
> OakDirectory has to at times perform directory listing specially at the time of opening of index. With DocumentNodeStore such listing of child nodes "might" be slow if there are lots more deleted nodes and GC has not cleared them so far (due to OAK-1557). 
> As seen in OAK-2808 Lucene might be creating and deleting lot more files. To speed up such lookup one OakDirectory can save the listing of child nodes as an array property once the writer is closed. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)