[jira] [Updated] (OAK-6538) Investigate cold standby memory consumption

["Andrei Dulceanu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OAK-6538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrei Dulceanu updated OAK-6538:
---------------------------------
    Attachment: OAK-6538.patch

After fixing OAK-6565, I enabled again {{DataStoreTestBase.testSyncBigBlob}} and adjusted the blob and chunk size and also the timeout. Successfully syncing a blob of {{1 GB}} in the same JVM with a max heap size of {{512 MB}} proves the point that this issue is now gone.

Moreover, reducing the blob size to {{1 GB}} reduced also the time for running the test from ~ {{180 s}} to {{68 s}}.

[~frm], could you take a look at the patch, please?

> Investigate cold standby memory consumption 
> --------------------------------------------
>
>                 Key: OAK-6538
>                 URL: https://issues.apache.org/jira/browse/OAK-6538
>             Project: Jackrabbit Oak
>          Issue Type: Task
>          Components: segment-tar
>    Affects Versions: 1.6.1
>            Reporter: Andrei Dulceanu
>            Assignee: Andrei Dulceanu
>            Priority: Minor
>             Fix For: 1.8, 1.7.6
>
>         Attachments: OAK-6538.patch
>
>
> In an investigation from some time ago, 4GB of heap were needed for transferring 1GB blob and 6GB for 2GB blob. This was in part due to using {{addTestContent}} [0] in the investigation, which allocates a huge {{byte[]}} on the heap. 
> OAK-5902 introduced chunking for transferring blobs between primary and standby. This way, the memory needed for syncing a big blob should be around the chunk size used. Solving the way test data is created, it should be possible to transfer a big blob (e.g. 2.5 GB) with less memory.
> [0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/standby/DataStoreTestBase.java#L96



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)