You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Yifan Cai (Jira)" <ji...@apache.org> on 2021/01/06 00:38:00 UTC
[jira] [Comment Edited] (CASSANDRA-16339) LCS steady state load of
table with vs. w/o GC performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259315#comment-17259315 ]
Yifan Cai edited comment on CASSANDRA-16339 at 1/6/21, 12:37 AM:
-----------------------------------------------------------------
Result report link: [https://github.com/yifan-c/CASSANDRA-15581-COMPACTION-TEST/blob/main/CASSANDRA-16339/7019-Test:%20Perf%20Comparison%20%5BLCS%20-%20provide_overlapping_tombstones%5D.pdf]
Seen from the result charts, we have those observations after altering 'provided_overlapping_tombstones' == 'row' for the table.
* The read latency drops initially but it becomes more unstable. There are larger spikes of the tail latencies, p95 and p99. The avg. latencies also gets higher.
* The write latency is about the same.
* The number of L0 sstables builds up quickly and it further affects the compaction speed. Since almost all L0 sstables can be used as the shadow sources for GarbageSkipper.
!flamegraph_grabageskipper.png|width=1831,height=920!
The flame graph (attached, flamegraph_garbageskipper.png) confirms that GarbageSkipper occupies the majority of the cpu time.
Garbage skipping is a feature that utilizes the *spare* IO capacity to produce more compacted SSTables.
We may want to avoid doing the garbage skipping, when the system does not have IO to spare.
In the case of LCS, it is when the number of L0 sstables is building up.
was (Author: yifanc):
Result report link: https://github.com/yifan-c/CASSANDRA-15581-COMPACTION-TEST/blob/main/CASSANDRA-16339/7019-Test:%20Perf%20Comparison%20%5BLCS%20-%20provide_overlapping_tombstones%5D.pdf
Seen from the result charts, we have those observations after altering 'provided_overlapping_tombstones' == 'row' for the table.
* The read latency drops initially but it becomes more unstable. There are larger spikes of the tail latencies, p95 and p99. The avg. latencies also gets higher.
* The write latency is about the same.
* The number of L0 sstables builds up quickly and it further affects the compaction speed. Since almost all L0 sstables can be used as the shadow sources for GarbageSkipper.
The flame graph (attached, flamegraph_garbageskipper.png) confirms that GarbageSkipper occupies the majority of the cpu time.
Garbage skipping is a feature that utilizes the *spare* IO capacity to produce more compacted SSTables.
We may want to avoid doing the garbage skipping, when the system does not have IO to spare.
In the case of LCS, it is when the number of L0 sstables is building up.
> LCS steady state load of table with vs. w/o GC performance test
> ---------------------------------------------------------------
>
> Key: CASSANDRA-16339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16339
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
> Reporter: Yifan Cai
> Assignee: Yifan Cai
> Priority: Normal
> Attachments: flamegraph_grabageskipper.png
>
>
> The testing cluster should be pre-populated with ~200GB data in each node. The baseline cluster has the table created with {{provide_overlapping_tombstones}} disabled. The other cluster has the table with {{provide_overlapping_tombstones == row}}. Compare the read, write and compaction performance between those 2 clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org