[jira] [Comment Edited] (OAK-2045) Long running JCR session prevent live cleanup in Segment FileStore

["Chetan Mehrotra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14108416#comment-14108416 ] 

Chetan Mehrotra edited comment on OAK-2045 at 8/24/14 4:10 PM:
---------------------------------------------------------------

[patch|^OAK-2045-segment-fault-approach.patch] based on Segment fault approach which relies on logical linking of SegmentNodeState based on path they refer to post cleanup.

Each segmentId would records its in memory creationTime. Further each {{SegmentNodeState}} would maintain a path property if the state is part of repository tree. While performing cleanup those segmentId would be marked stale which are considered _old_. Due to this when a SegmentNodeState which refers to an old revision tries to access any property then it would result in a {{SegmentStaleException}}. Upon detecting such an exception aka _SegmentFault_ the state would locate the latest nodeState for the given path and update its segmentId.

SegementNodeState maintains an in memory state of templateId. Once the backing segment turn stale this state also need to be purged. To manage that each SegmentNodeState registers itself with the SegementId. Once any SegmentId is marked stale then each linked SegmentNodeState is refreshed such that they let go of in memory state.

This should enable JCR session which hold reference to old NodeState upon startup be linked to latest NodeState and allow collection of revisions from those states. 

Points to consider
* cleanup and lastCreationTime - The time argument passed to cleanup call should be older than any pending session operation. This can be based on lastSave from SessionMbeans. 

*Cons*
* Need to make the segmentId and offset volatile in Record which does not look good and might have some effect on access performance
* SegmentId maintains a wekreference list of the SegmentNodeState - This might need bit more care

[~alexparvulescu] [~jukkaz] [~mduerig] [~tmueller] [~mreutegg] Kindly review on the approach taken. Tracking the SegmentNodeState so as notify them of templateId change is proving to be costly ... so looking into that.


was (Author: chetanm):
[patch|^OAK-2045-segment-fault-approach.patch] based on Segment fault approach which relies on logical linking of SegmentNodeState based on path they refer to post cleanup.

Each segmentId would records its in memory creationTime. Further each {{SegmentNodeState}} would maintain a path property if the state is part of repository tree. While performing cleanup those segmentId would be marked stale which are considered _old_. Due to this when a SegmentNodeState which refers to an old revision tries to access any property then it would result in a {{SegmentStaleException}}. Upon detecting such an exception aka _SegmentFault_ the state would locate the latest nodeState for the given path and update its segmentId.

SegementNodeState maintains an in memory state of templateId. Once the backing segment turn stale this state also need to be purged. To manage that each SegmentNodeState registers itself with the SegementId. Once any SegmentId is marked stale then each linked SegmentNodeState is refreshed such that they let go of in memory state.

This should enable JCR session which hold reference to old NodeState upon startup be linked to latest NodeState and allow collection of revisions from those states. 

Points to consider
* cleanup and lastCreationTime - The time argument passed to cleanup call should be older than any pending session operation. This can be based on lastSave from SessionMbeans. 

*Cons*
* Need to make the segmentId and offset volatile in Record which does not look good and might have some effect on access performance
* SegmentId maintains a wekreference list of the SegmentNodeState - This might need bit more care

[~alexparvulescu] [~jukkaz] [~mduerig] [~tmueller] [~mreutegg] Kindly review

> Long running JCR session prevent live cleanup in Segment FileStore
> ------------------------------------------------------------------
>
>                 Key: OAK-2045
>                 URL: https://issues.apache.org/jira/browse/OAK-2045
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: segmentmk
>            Reporter: Chetan Mehrotra
>            Assignee: Chetan Mehrotra
>             Fix For: 1.1
>
>         Attachments: OAK-2045-segment-fault-approach.patch, OAK-2045-test-fix.patch
>
>
> Cleanup operation in SegmentNodeStore detects the un referenced garbage and clean it up. To determine the reference validity it starts with an initial set of SegmentId which have a live java reference. 
> This works fine for simple setup but when Oak repository is used in an application (like Sling) where application code can create long running session (for observation) then such session are bound to old NodeState at time of startup. Such references prevent the cleanup logic to remove older revisions while system is running. Such revisions can only be removed via an offline compaction-> cleanup.
> Need to find out a way where we can _migrate_ such old NodeState references to newer revisions



--
This message was sent by Atlassian JIRA
(v6.2#6252)