You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Benedict Elliott Smith (Jira)" <ji...@apache.org> on 2023/03/02 18:56:00 UTC
[jira] [Comment Edited] (CASSANDRA-18212) Reduce memory allocations of calls to ByteBufer.duplicate() made in org.apache.cassandra.transport.CBUtil#writeValue
[ https://issues.apache.org/jira/browse/CASSANDRA-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695863#comment-17695863 ]
Benedict Elliott Smith edited comment on CASSANDRA-18212 at 3/2/23 6:55 PM:
----------------------------------------------------------------------------
bq. I am not against changing the allocator and special casing that as well, but I don't have a good intuition on the testing needed to validate it, so it comes off to me as higher risk than just handling the 2 cases we have today.
I think you misunderstood me here, I was referring to using the cast to {{Wrapped}} approach without changing the allocator for legacy clients, which avoids the extra work of duplicating the buffer for everyone using the latest client - and for legacy clients with direct buffers as sources we could probably have just left behaviour as before (or fixed using this approach).
But either way, the patch is definitely an improvement. +1.
was (Author: benedict):
bq. I am not against changing the allocator and special casing that as well, but I don't have a good intuition on the testing needed to validate it, so it comes off to me as higher risk than just handling the 2 cases we have today.
I think you misunderstood me here, I was referring to using the cast to {{Wrapped}} approach without changing the allocator for legacy clients, which avoids the extra work of duplicating the buffer for everyone using the latest client - and for legacy clients using off heap memtables we could probably have just left behaviour as before (or fixed using this approach).
But either way, the patch is definitely an improvement. +1.
> Reduce memory allocations of calls to ByteBufer.duplicate() made in org.apache.cassandra.transport.CBUtil#writeValue
> --------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-18212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18212
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
> Reporter: Natnael Adere
> Assignee: Natnael Adere
> Priority: Normal
> Attachments: DiskBased.png, allocations-after.html, allocations-before.html
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Currently, org.apache.cassandra.transport.CBUtil#writeValue(java.nio.ByteBuffer, io.netty.buffer.ByteBuf) calls ByteBufer.duplicate() and is found to be 20% of memory allocations in production. No changes have been made to reduce this but there is discussion related to the issue in CASSANDRA-13741. Attached below are the performance result for disk based queries and mostly memory based queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org