[jira] [Comment Edited] (CASSANDRA-16201) Cassandra 4.0 b2 - OOM / memory pressure due to object array pre-allocations in BatchUpdatesCollector

["Thomas Steinmaurer (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210238#comment-17210238 ] 

Thomas Steinmaurer edited comment on CASSANDRA-16201 at 10/8/20, 2:14 PM:
--------------------------------------------------------------------------

[~marcuse], yes I think so. :-) TRUNK, locally checked out, calling hierarchy from {{BatchUpdatesCollector.getPartitionUpdateBuilder}} up to {{PartitionUpdate.Builder.rowBuilder}}

 !screenshot-2.png! 

Thanks again.


was (Author: tsteinmaurer):
[~marcuse], yes I think so. :-) Locally checked out, calling hierarchy from {{BatchUpdatesCollector.getPartitionUpdateBuilder}} up to {{PartitionUpdate.Builder.rowBuilder}}

 !screenshot-2.png! 

Thanks again.

> Cassandra 4.0 b2 - OOM / memory pressure due to object array pre-allocations in BatchUpdatesCollector
> -----------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-16201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-16201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local/Other
>            Reporter: Thomas Steinmaurer
>            Priority: Normal
>         Attachments: screenshot-1.png, screenshot-2.png
>
>
> In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, we see 4.0b2 going OOM from time to time. According to a heap dump, we have multiple NTR threads in a 3-digit MB range.
> This is likely related to object array pre-allocations at the size of {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always only 1 {{BTreeRow}} in the {{BTree}}.
>  !screenshot-1.png! 
> So it seems we have many, many 20K elemnts pre-allocated object arrays resulting in a shallow heap of 80K each, although there is only one element in the array.
> This sort of pre-allocation is causing a lot of memory pressure.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org