[jira] [Comment Edited] (OAK-4966) Re-introduce a blocker for compaction based on available heap

["Alex Parvulescu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15632258#comment-15632258 ] 

Alex Parvulescu edited comment on OAK-4966 at 11/3/16 12:50 PM:
----------------------------------------------------------------

attaching new patch based on the jmx notification mechanism. includes all config options (jmx, osgi). I set the default at {{15%}}, not sure if this is enough.

Left a TODO related to introducing a minimal value of available memory. previous analysis showed that all caches with the current defaults can go over {{2GB}}, so arguably it doesn't even make sense to run compaction if you don't have at least that, otherwise you'd be racing agains the OOME notification, which might not be a perfect way to prevent a crash.

The Felix issue is still pending, did not look into patching that yet.

[~mduerig] feedback appreciated!

[edit] found a bug related to resetting the {{sufficientMemory}} flag when you disable the check by setting the percentage to {{0}}. I'm going to wait for feedback before attaching a new version of the patch.


was (Author: alex.parvulescu):
attaching new patch based on the jmx notification mechanism. includes all config options (jmx, osgi). I set the default at {{15%}}, not sure if this is enough.

Left a TODO related to introducing a minimal value of available memory. previous analysis showed that all caches with the current defaults can go over {{2GB}}, so arguably it doesn't even make sense to run compaction if you don't have at least that, otherwise you'd be racing agains the OOME notification, which might not be a perfect way to prevent a crash.

The Felix issue is still pending, did not look into patching that yet.

[~mduerig] feedback appreciated!

> Re-introduce a blocker for compaction based on available heap
> -------------------------------------------------------------
>
>                 Key: OAK-4966
>                 URL: https://issues.apache.org/jira/browse/OAK-4966
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segment-tar
>            Reporter: Alex Parvulescu
>            Assignee: Alex Parvulescu
>             Fix For: 1.6, 1.5.13
>
>         Attachments: OAK-4966-jmx-notification.patch, OAK-4966.patch
>
>
> As seen in a local test, running compaction on a tight heap can lead to OOMEs. There used to be a best effort barrier against this situation 'not enough heap for compaction', but we removed it with the compaction maps.
> I think it makes sense to add it again based on the max size of some of the caches: segment cache {{256MB}} by default [0] and some writer caches which can go up to {{2GB}} all combined [1] and probably others I missed.
> [0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentCache.java#L48
> [1] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/WriterCacheManager.java#L50



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)