[jira] [Updated] (RATIS-1717) Reduce data copy when serialize LogEntry to RaftLog

["Song Ziyang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RATIS-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Song Ziyang updated RATIS-1717:
-------------------------------
    Priority: Major  (was: Minor)

> Reduce data copy when serialize LogEntry to RaftLog
> ---------------------------------------------------
>
>                 Key: RATIS-1717
>                 URL: https://issues.apache.org/jira/browse/RATIS-1717
>             Project: Ratis
>          Issue Type: Improvement
>          Components: performance
>            Reporter: Song Ziyang
>            Priority: Major
>
> In current implementation, before a log entry is serialized to RaftLog, it first needs to be serialized to a byte array[1], and then be written to FileChannel using directByteBuffer[2]. It requires two data copies, one from LogEntryProto to byte array, one from byte array to direct ByteBuffer.
> We can reduce the data copy by *serializing LogEntry to directByteBuffer, removing the byte array:*
>  # We need a directByteBuffer large enough to hold a single log (maxBufferSize).
>  # When we need to serialize a log entry to RaftLog, first we check whether there's enough space remained in directByteBuffer. If not, we flush the directByteBuffer to FileChannel. Otherwise we serialize the data directly into directByteBuffer.
>  # When data buffered in directByteBuffer reaches the threshold (say 64KB), we flush the directByteBuffer, the same as current implementation.
> [1] [https://github.com/apache/ratis/blob/289db3ae64e8cf620eba882468dda661af0439bc/ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/SegmentedRaftLogOutputStream.java#L96]
> [2] [https://github.com/apache/ratis/blob/289db3ae64e8cf620eba882468dda661af0439bc/ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/BufferedWriteChannel.java#L65]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)