[jira] [Commented] (OAK-7285) Reindexing using --doc-traversal-mode can OOM while aggregation in some cases

["Vikas Saurabh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377067#comment-16377067 ] 

Vikas Saurabh commented on OAK-7285:
------------------------------------

Following test in {{FaltFileStoreIteratorTest}} shows the behavior (not committing ignored test, as we'd probably need to restructure {{FlatFileStoreIteartor}} ctor to take in preferred list instead of just size:
{noformat}
    @Test
    public void getChildNodeForNonPreferred() {
        // have more than 1 preferred names
        Set<String> preferred = ImmutableSet.of("j:c", "md");

        CountingIterable<NodeStateEntry> citr = createList(preferred, asList("/a", "/a/b", "/a/c"));

        FlatFileStoreIterator fitr = new FlatFileStoreIterator(citr.iterator(), preferred.size());

        NodeStateEntry a = fitr.next();
        assertEquals("/a", a.getPath());

        NodeState aNS = a.getNodeState();
        aNS.getChildNode("j:c");

        // Don't read whole tree to conclude that "j:c" doesn't exist (reading /a/b should imply that it doesn't exist)
        assertEquals(1, fitr.getBufferSize());
    }
{noformat}

> Reindexing using --doc-traversal-mode can OOM while aggregation in some cases
> -----------------------------------------------------------------------------
>
>                 Key: OAK-7285
>                 URL: https://issues.apache.org/jira/browse/OAK-7285
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: lucene, mongomk
>    Affects Versions: 1.8.0
>            Reporter: Vikas Saurabh
>            Assignee: Vikas Saurabh
>            Priority: Major
>              Labels: candidate_oak_1_8
>             Fix For: 1.10
>
>
> {{--doc-traversal-mode}} works on the notion of {{preferred}} children which is computed using path fragments that form aggregate rules.
> The idea is reading through aggregated paths should avoid keeping non useful nodes (for path being currently indexed) in memory.
> But, currently, in case, say when there multiple preferred children - {{jcr:content}}, {{metadata}}, then an index defn indexing parent of a very deep tree root would try to read in the whole tree before concluding that it doesn't have preferred children
> e.g. with preferred list - {{j:c}} and {{md} and index looking for {{j:c}} indexing following structure
> {noformat}
> + /path/being/indexed
>    + very
>       + very
>       + very
>            + deep
>            + tree
> + /some-sibling
> {noformat}
> Currently, while looking for {{j:c}}, the code concludes that it doesn't exist only after reaching {{/some-sibling}} (or if number of children read of {{/path/being/indexed}} is >= num_preferred_children).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)