[jira] [Commented] (IGNITE-28980) BinaryMemoryAllocator$ThreadLocalAllocator$Chunk does not reset maxMsgSize after shrink check

["Evgeny Stanilovsky (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18105259#comment-18105259 ] 

Evgeny Stanilovsky commented on IGNITE-28980:
---------------------------------------------

[~mroeppis] thank you, fill some comments, plz check them 

> BinaryMemoryAllocator$ThreadLocalAllocator$Chunk does not reset maxMsgSize after shrink check
> ---------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-28980
>                 URL: https://issues.apache.org/jira/browse/IGNITE-28980
>             Project: Ignite
>          Issue Type: Bug
>          Components: general
>    Affects Versions: 2.18
>         Environment: * Apache Ignite 2.18.0
> * {{BinaryMemoryAllocator$ThreadLocalAllocator$Chunk}}
> * Java 25.0.3+9
> * Linux 6.8.0-100-generic amd64
>            Reporter: Mathias Roeppischer
>            Assignee: Mathias Roeppischer
>            Priority: Major
>             Fix For: 2.19
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> {{BinaryMemoryAllocator$ThreadLocalAllocator$Chunk}} keeps track of the maximum message size detected between checks in {{maxMsgSize}}.
> The value is updated as follows:
> {code:java}
> if (maxMsgSize > this.maxMsgSize)
>     this.maxMsgSize = maxMsgSize;
> {code}
> The shrink logic periodically checks this value based on {{CHECK_FREQ}}:
> {code:java}
> if (CommonUtils.nanosToMillis(nowNanos - lastCheckNanos) >= CHECK_FREQ) {
>     int halfSize = data.length >> 1;
>     if (this.maxMsgSize < halfSize)
>         this.data = new byte[halfSize];
>     lastCheckNanos = nowNanos;
> }
> {code}
> However, {{this.maxMsgSize}} is never reset after the check.
> As a result, the value effectively represents the maximum message size seen during the entire lifetime of the {{Chunk}}, rather than the maximum message size detected between checks as documented.
> h3. Impact
> A single large serialization can permanently prevent the chunk from shrinking again.
> For example, if a chunk grows to 128 MB due to a large message and {{maxMsgSize}} reaches 100 MB, subsequent small messages cannot lower {{maxMsgSize}} below 100 MB. Consequently, the shrink condition
> {code:java}
> this.maxMsgSize < halfSize
> {code}
> may never become true, even if the chunk is subsequently used only for much smaller messages.
> This can cause large byte arrays to remain retained by thread-local allocators. In applications using thread pools, a number of worker threads can therefore retain large buffers after processing occasional large messages, resulting in unnecessarily high heap usage.
> h3. Expected behavior
> {{maxMsgSize}} should represent the maximum message size observed during the current check interval.
> After the shrink check has been performed, the recorded maximum should be reset so that the next interval starts with a fresh measurement.
> h3. Proposed fix
> Reset {{maxMsgSize}} after each {{CHECK_FREQ}} check:
> {code:java}
> if (CommonUtils.nanosToMillis(nowNanos - lastCheckNanos) >= CHECK_FREQ) {
>     int halfSize = data.length >> 1;
>     if (this.maxMsgSize < halfSize)
>         this.data = new byte[halfSize];
>     this.maxMsgSize = 0;
>     lastCheckNanos = nowNanos;
> }
> {code}
> This preserves the existing design of collecting the maximum message size between checks while allowing the chunk to shrink when subsequent serialization workloads are smaller.
> h3. Additional Information
> Discussion on the Ignite User Mailing list:
> https://lists.apache.org/thread/cr2rvdpyh1njn2wghvg1mq7gr66wgjls



--
This message was sent by Atlassian Jira
(v8.20.10#820010)