[jira] [Created] (IOTDB-998) [Distributed] Split the snapshot when sending to follower

["Houliang Qi (Jira)" <ji...@apache.org>]

Houliang Qi created IOTDB-998:
---------------------------------

             Summary: [Distributed] Split the snapshot when sending to follower 
                 Key: IOTDB-998
                 URL: https://issues.apache.org/jira/browse/IOTDB-998
             Project: Apache IoTDB
          Issue Type: Improvement
          Components: Core/Cluster
            Reporter: Houliang Qi


Now, when the follower is much behind the leader, the leader will send a snapshot, but when the snapshot size is greater than thrift's maxFrameSize, the sending will fail, so that the follower will never catch up with the leader.

I think we need to split the snapshot request and then send it to the follower. But this also brings a problem when we use an asynchronous server. How to ensure the order after the split request is sent to the follower. Two options are possible as follows:
1. Mark the segment number in the header of the request sent by the leader, and splice the request on the follower side.
2. Like the follower downloading TsFile process, the snapshot is downloaded in segments on the follower side, but the snapshot data needs to be persisted on the leader side.

I would like the first one, any ideas about this?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)