[jira] [Created] (IOTDB-1856) Improve raft log size check

["Tian Jiang (Jira)" <ji...@apache.org>]

Tian Jiang created IOTDB-1856:
---------------------------------

             Summary: Improve raft log size check 
                 Key: IOTDB-1856
                 URL: https://issues.apache.org/jira/browse/IOTDB-1856
             Project: Apache IoTDB
          Issue Type: Improvement
          Components: Core/Cluster
            Reporter: Tian Jiang
         Attachments: image-2021-10-18-09-51-15-583.png

To avoid log-append failures caused by oversized log entries, the size of each entry is checked to ensure that it will fit the flush buffer before it is appended to the local log. However, the current implementation has two drawbacks:
# The size check falls in a synchronized block, which means concurrent requests cannot be checked in parallel, and other requests will be significantly blocked if the request is rather large.
# The size check directly serializes an entry into a ByteBuffer and uses it to get the size of the entry, but the buffer is discarded immediately after the check, so both CPU computation and memory allocation are wasted during the serialization are wasted.

 !image-2021-10-18-09-51-15-583.png|thumbnail! 

To resolve the two aforementioned problems, we need:
# Move the size check out of the synchronized block.
# Use a serialization-free way to get the size of an entry, or save the serialization result and reuse it during log persistency and log replication. If the second one is adopted, the serialization result should be freed in time to avoid a large memory footprint.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)