You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Michael Semb Wever (Jira)" <ji...@apache.org> on 2020/11/03 16:27:00 UTC
[jira] [Commented] (CASSANDRA-16201) Reduce amount of allocations
during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225523#comment-17225523 ]
Michael Semb Wever commented on CASSANDRA-16201:
------------------------------------------------
Re-doing the screenshots as done in CASSANDRA-15430 for comparison.
h4. 2.1.18
Skipping these results as its based on the same C* code as in 15430, and I've confirmed the allocation counts and object sizes proportionally match.
h4. 3.0.20
Allocations
* BatchMessage.execute - 1926411
** BatchStatement.getMutations => 1008170 == 52% ( previously 60%)
** BatchStatement.executeWithoutConditions => 692322 == 36% ( previously 30%)
!16201_jfr_3023_alloc.png!
Sizes by object under {{BatchStatement.getMutations}}
!16201_jfr_3023_obj.png!
With the invocation count only a 1% decrease, the {{Object[]}} size has gone from 29.8GB down to 18.7GB, or 37% reduction.
h4. 3.11.8
Allocations
* BatchMessage.execute - 1210873
** BatchStatement.getMutations => 396645 == 33% ( previously 62%)
** BatchStatement.executeWithoutConditions => 664650 == 55% ( previously 30%)
!16201_jfr_3118_alloc.png!
Sizes by object under {{BatchStatement.getMutations}}
!16201_jfr_3118_obj.png!
The {{Object[]}} size has gone from 116GB down to 8.14GB. With the invocation count 40% that from CASSANDRA-15430, proportionally this is a 82% reduction.
h4. 4.0-beta2
Allocations
* BatchMessage.execute - 969189
** BatchStatement.getMutations => 410739 == 42% ( previously 70%)
** BatchStatement.executeWithoutConditions => 425337 == 44% ( previously 23%)
!16201_jfr_40b3_alloc.png!
Sizes by object under {{BatchStatement.getMutations}}
!16201_jfr_40b3_obj.png!
The {{Object[]}} size has gone from 129GB down to 8.52GB. With the invocation count 30% that from CASSANDRA-15430, proportionally this is a 78% reduction.
> Reduce amount of allocations during batch statement execution
> -------------------------------------------------------------
>
> Key: CASSANDRA-16201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16201
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
> Reporter: Thomas Steinmaurer
> Assignee: Marcus Eriksson
> Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, we see 4.0b2 going OOM from time to time. According to a heap dump, we have multiple NTR threads in a 3-digit MB range.
> This is likely related to object array pre-allocations at the size of {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always only 1 {{BTreeRow}} in the {{BTree}}.
> !screenshot-1.png|width=100%!
> So it seems we have many, many 20K elemnts pre-allocated object arrays resulting in a shallow heap of 80K each, although there is only one element in the array.
> This sort of pre-allocation is causing a lot of memory pressure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org