[jira] [Commented] (RATIS-243) Add log purge function after taking snapshot

["Tsz Wo Nicholas Sze (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RATIS-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815940#comment-16815940 ] 

Tsz Wo Nicholas Sze commented on RATIS-243:
-------------------------------------------

Thanks [~andywu], the pull request https://github.com/apache/incubator-ratis/pull/17/commits/3d20aefd6e9f21019368fc3843ad3d5dd6cded26 looks good.  Some minor comments on LogSegmentList.purge
- It should acquire the write lock.
- Let's add an else-clause to throw IllegalStateException.
- The return can be simplified using "? :".

{code}
//LogSegmentList
    TruncationSegments purge(long index) {
      try(AutoCloseableLock writeLock = writeLock()) {
        final int segmentIndex = binarySearch(index);
        final List<SegmentFileInfo> list = new ArrayList<>();

        if (segmentIndex == -segments.size() - 1) {
          for (LogSegment ls : segments) {
            list.add(new SegmentFileInfo(ls.getStartIndex(), ls.getEndIndex(), false, 0, 0));
          }
          segments.clear();
        } else if (segmentIndex >= 0) {
          // we start to purge the closedSegments which do not overlap with index.
          for (int i = segmentIndex - 1; i >= 0; i--) {
            LogSegment ls = segments.get(i);
            list.add(new SegmentFileInfo(ls.getStartIndex(), ls.getEndIndex(), false, 0, 0));
            segments.remove(i);
          }
        } else {
          throw new IllegalStateException("Unexpected gap in segments: binarySearch(" + index + ") returns "
              + segmentIndex + ", segments=" + segments);
        }
        return list.isEmpty()? null: new TruncationSegments(null, list);
      }
    }
{code}


> Add log purge function after taking snapshot
> --------------------------------------------
>
>                 Key: RATIS-243
>                 URL: https://issues.apache.org/jira/browse/RATIS-243
>             Project: Ratis
>          Issue Type: Improvement
>          Components: server
>            Reporter: Andy Wu
>            Assignee: Andy Wu
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> After snapshotting the state machine, we can safely purge logs in the cache and disk.
> Based on the lastAppliedIndex, we can find out which segment the index lands, we can purge all previous segments if leader has no pending RPC on it. We will leave the segment where index lands alone, so we do not need to deal with partial file deletion logic. 
> Also if we only have snapshots, make sure we can install snapshots to the followers. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)