[jira] [Comment Edited] (OAK-1322) Reduce calls to MongoDB

["Vikas Saurabh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-1322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357218#comment-15357218 ] 

Vikas Saurabh edited comment on OAK-1322 at 6/30/16 3:03 PM:
-------------------------------------------------------------

In an offline discussion with [~mreutegg], it seemed that we can have lots of redundant entries. Here are a few persistent cache stats from a couple of setups I could get my hands on:
1) Single instance backed by mongo
Total size of children cache entries -> 4200783
Sum of redundant entries -> 15192395

2) 2 Node cluster backed by RDB
2.a) Node1
Total size of children cache entries -> 30662155
Sum of redundant entries -> 61976127
2.b) Node2
Total size of children cache entries -> 14489964
Sum of redundant entries -> 286490527

Some details:
- redundant entries mean that key(without revision), value are same across revisions
- only node and children cache entries were considered (diff, local_diff, prev_docs) were ignored
- the reason to collect the stats ... we can probably store interesting children for a change in revision along with child cache entries. This can be utilized to read nodes/children on sub-tree at better (potentially actual rev) and thus reduce getting redundant cached entries.



was (Author: catholicon):
In an offline discussion with [~mreutegg], it seemed that we can have lots of redundant entries. Here are a few persistent cache stats from a couple of setups I could get my hands on:
1) Single instance backed by mongo
Total size of children cache entries -> 4200783
Sum of redundant entries -> 15192395

2) 2 Node cluster backed by RDB
2.a) Node1
Total size of children cache entries -> 30662155
Sum of redundant entries -> 61976127
2.a) Node1
Total size of children cache entries -> 14489964
Sum of redundant entries -> 286490527

Some details:
- redundant entries mean that key(without revision), value are same across revisions
- only node and children cache entries were considered (diff, local_diff, prev_docs) were ignored
- the reason to collect the stats ... we can probably store interesting children for a change in revision along with child cache entries. This can be utilized to read nodes/children on sub-tree at better (potentially actual rev) and thus reduce getting redundant cached entries.


> Reduce calls to MongoDB
> -----------------------
>
>                 Key: OAK-1322
>                 URL: https://issues.apache.org/jira/browse/OAK-1322
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: core, mongomk
>            Reporter: Marcel Reutegger
>            Assignee: Marcel Reutegger
>              Labels: performance
>         Attachments: OAK-1322-mreutegg.patch
>
>
> As discussed with Chetan offline we'd like to reduce the number of calls to MongoDB when content is added to the repository with a filevault package import.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)