[jira] [Comment Edited] (CASSANDRA-18212) Reduce memory allocations of calls to ByteBufer.duplicate() made in org.apache.cassandra.transport.CBUtil#writeValue

["David Capwell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686130#comment-17686130 ] 

David Capwell edited comment on CASSANDRA-18212 at 2/9/23 12:06 AM:
--------------------------------------------------------------------

bq. I thought we had discussed either using the byte[] or casting to BufferPoolAllocator.Wrapped on slack?

That is what the patch does.  If the BB is a heap based one, then we use the array (due to a CMS bug, Nate will update the code explaining that issue), if the BB is direct, then it uses the same "duplicate" logic used in other logic and calls MemoryUtil.duplicateDirectByteBuffer...  If a new type of BB is added (not heap and not direct) then we fall back to duplicate.

{code}
if (src.hasArray())
{
  byte[] array = src.array();
  dest.writeBytes(array, src.arrayOffset() + src.position(), length);
}
else if (src.isDirect())
{
  ByteBuffer local = getLocalDirectBuffer();
  MemoryUtil.duplicateDirectByteBuffer(src, local);
  dest.writeBytes(local);
}
else
{
  dest.writeBytes(src.duplicate());
}
{code}


was (Author: dcapwell):
bq. I thought we had discussed either using the byte[] or casting to BufferPoolAllocator.Wrapped on slack?

That is what the patch does.  If the BB is a heap based one, then we use the array (due to a CMS bug, Nate will update the code explaining that issue), if the BB is direct, then it uses the same "duplicate" logic used in other logic and calls MemoryUtil.duplicateDirectByteBuffer...  If a new type of BB is added (not heap and not direct) then we fall back to duplicate.

> Reduce memory allocations of calls to ByteBufer.duplicate() made in org.apache.cassandra.transport.CBUtil#writeValue
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-18212
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-18212
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Legacy/Local Write-Read Paths
>            Reporter: Natnael Adere
>            Assignee: Natnael Adere
>            Priority: Normal
>         Attachments: DiskBased.png, MemoryBased.png, allocations-after.html, allocations-before.html
>
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Currently, org.apache.cassandra.transport.CBUtil#writeValue(java.nio.ByteBuffer, io.netty.buffer.ByteBuf) calls ByteBufer.duplicate() and is found to be 20% of memory allocations in production. No changes have been made to reduce this but there is discussion related to the issue in CASSANDRA-13741. Attached below are the performance result for disk based queries and mostly memory based queries. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org