You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2024/06/12 06:51:21 UTC

[PR] [RATIS-2111] Reinitialize should load the latest snapshot [ratis]

AngersZhuuuu opened a new pull request, #1111:
URL: https://github.com/apache/ratis/pull/1111

   ## What changes were proposed in this pull request?
   When we setting `INSTALL_SNAPSHOT_ENABLED_KEY` as true and when master sync snapshot to follower,
   the status will be PAUSED and won't take snapshot, also follower StateMachine will call `reinitialize()`.
   Then we need to load the the latest snapshot.
   ```
     /**
      * Re-initializes the State Machine in PAUSED state. The
      * state machine is responsible reading the latest snapshot from the file system (if any) and
      * initialize itself with the latest term and index there including all the edits.
      */
     void reinitialize() throws IOException;
   ```
   
   
   But current `SimpleStateMachineStorage` when init(), `latestSnapshot` was set. But the master's snapshot won't be taken so the `latestSnapshot` won't be updated to master's snapshot.
   
   So here we should load the latest snapshot and update it before do reinitialize.
   Similar thing was done in alluxio https://github.com/Alluxio/alluxio/blob/efed93c95e6f1cf3b2131066208f03cfd7821b58/dora/core/server/common/src/main/java/alluxio/master/journal/raft/JournalStateMachine.java#L209-L214
   
   and celeborn also meet such issue. https://github.com/apache/celeborn/pull/2547
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/RATIS-2111
   
   ## How was this patch tested?
   
   (Please explain how this patch was tested. Ex: unit tests, manual tests)
   (If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [RATIS-2111] Reinitialize should load the latest snapshot [ratis]

Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu commented on code in PR #1111:
URL: https://github.com/apache/ratis/pull/1111#discussion_r1635957841


##########
ratis-server/src/main/java/org/apache/ratis/statemachine/impl/SimpleStateMachineStorage.java:
##########
@@ -228,6 +228,17 @@ public SingleFileSnapshotInfo getLatestSnapshot() {
     }
   }
 
+  public void loadLatestSnapshot() {

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [RATIS-2111] Reinitialize should load the latest snapshot [ratis]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo merged PR #1111:
URL: https://github.com/apache/ratis/pull/1111


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [RATIS-2111] Reinitialize should load the latest snapshot [ratis]

Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu commented on PR #1111:
URL: https://github.com/apache/ratis/pull/1111#issuecomment-2165078941

   @szetszwo GA passed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [RATIS-2111] Reinitialize should load the latest snapshot [ratis]

Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu commented on PR #1111:
URL: https://github.com/apache/ratis/pull/1111#issuecomment-2162241993

   ping @szetszwo 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [RATIS-2111] Reinitialize should load the latest snapshot [ratis]

Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #1111:
URL: https://github.com/apache/ratis/pull/1111#discussion_r1635951987


##########
ratis-server/src/main/java/org/apache/ratis/statemachine/impl/SimpleStateMachineStorage.java:
##########
@@ -228,6 +228,17 @@ public SingleFileSnapshotInfo getLatestSnapshot() {
     }
   }
 
+  public void loadLatestSnapshot() {

Review Comment:
   Let's make it return `SingleFileSnapshotInfo` and then reuse it in `getLatestSnapshot()`.
   ```java
   diff --git a/ratis-server/src/main/java/org/apache/ratis/statemachine/impl/SimpleStateMachineStorage.java b/ratis-server/src/main/java/org/apache/ratis/statemachine/impl/SimpleStateMachineStorage.java
   index 88cc57dab5..7e8afbaa85 100644
   --- a/ratis-server/src/main/java/org/apache/ratis/statemachine/impl/SimpleStateMachineStorage.java
   +++ b/ratis-server/src/main/java/org/apache/ratis/statemachine/impl/SimpleStateMachineStorage.java
   @@ -217,6 +217,10 @@ public class SimpleStateMachineStorage implements StateMachineStorage {
        if (s != null) {
          return s;
        }
   +    return loadLatestSnapshot();
   +  }
   +
   +  public SingleFileSnapshotInfo loadLatestSnapshot() {
        final File dir = stateMachineDir;
        if (dir == null) {
          return null;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org