[jira] [Resolved] (RATIS-1979) Allow StateMachine.read to return a ReferentCountedObject

["Tsz-wo Sze (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/RATIS-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz-wo Sze resolved RATIS-1979.
-------------------------------
    Fix Version/s: 3.1.0
       Resolution: Fixed

The pull request is now merged. Thanks, [~duongnguyen]!

> Allow StateMachine.read to return a ReferentCountedObject
> ---------------------------------------------------------
>
>                 Key: RATIS-1979
>                 URL: https://issues.apache.org/jira/browse/RATIS-1979
>             Project: Ratis
>          Issue Type: Sub-task
>          Components: StateMachine
>            Reporter: Duong
>            Assignee: Duong
>            Priority: Major
>             Fix For: 3.1.0
>
>         Attachments: 1062_review.patch
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> With zero-copy (https://github.com/apache/ratis/pull/990), we rely on RaftLog cache eviction to release the zero-copy input streams.
> There's a problems, with stateMachineCachingEnabled, the cache-size of LogEntryproto is calculated with StateMachineData trimmed. That leave the actual cached size if a log entry is too small comparing to the amount of direct memory backed for the StateMachineData. 
>  
> {code:java}
> static long getEntrySize(LogEntryProto entry, Op op) {
>   LogEntryProto e = entry;
>   if (op == Op.CHECK_SEGMENT_FILE_FULL) {
>     e = LogProtoUtils.removeStateMachineData(entry);
>   } else if (op == Op.LOAD_SEGMENT_FILE || op == Op.WRITE_CACHE_WITH_STATE_MACHINE_CACHE) {
>     Preconditions.assertTrue(entry == LogProtoUtils.removeStateMachineData(entry),
>         () -> "Unexpected LogEntryProto with StateMachine data: op=" + op + ", entry=" + entry);
>   } else {
>     Preconditions.assertTrue(op == Op.WRITE_CACHE_WITHOUT_STATE_MACHINE_CACHE || op == Op.REMOVE_CACHE,
>         () -> "Unexpected op " + op + ", entry=" + entry);
>   }
>   final int serialized = e.getSerializedSize();
>   return serialized + CodedOutputStream.computeUInt32SizeNoTag(serialized) + 4L;
> } {code}
>  
> With the default 200MB limit for raft log cache, cache eviction is likely never happens until the server run out of direct memory for zero-copy.
>  
> Maybe for cache calculation, we should take the StateMachineData size into account.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)