[jira] [Created] (RATIS-1717) Reduce data copy when serialize LogEntry to RaftLog

["Song Ziyang (Jira)" <ji...@apache.org>]

Song Ziyang created RATIS-1717:
----------------------------------

             Summary: Reduce data copy when serialize LogEntry to RaftLog
                 Key: RATIS-1717
                 URL: https://issues.apache.org/jira/browse/RATIS-1717
             Project: Ratis
          Issue Type: Improvement
          Components: performance
            Reporter: Song Ziyang


In current implementation, before a log entry is serialized to RaftLog, it first needs to be serialized to a byte array[1], and then be written to FileChannel using directByteBuffer[2]. It requires two data copies, one from LogEntryProto to byte array, one from byte array to direct ByteBuffer.

We can reduce the data copy by *serializing LogEntry to directByteBuffer, removing the byte array:*
 # We need a directByteBuffer large enough to hold a single log (maxBufferSize).
 # When we need to serialize a log entry to RaftLog, first we check whether there's enough space remained in directByteBuffer. If not, we flush the directByteBuffer to FileChannel. Otherwise we serialize the data directly into directByteBuffer.
 # When data buffered in directByteBuffer reaches the threshold (say 64KB), we flush the directByteBuffer, the same as current implementation.

[1] [https://github.com/apache/ratis/blob/289db3ae64e8cf620eba882468dda661af0439bc/ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/SegmentedRaftLogOutputStream.java#L96]

[2] [https://github.com/apache/ratis/blob/289db3ae64e8cf620eba882468dda661af0439bc/ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/BufferedWriteChannel.java#L65]

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)