You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Yifan Cai (Jira)" <ji...@apache.org> on 2021/01/22 04:55:00 UTC
[jira] [Commented] (CASSANDRA-16339) LCS steady state load of table
with vs. w/o GC performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17269790#comment-17269790 ]
Yifan Cai commented on CASSANDRA-16339:
---------------------------------------
I ran several experiments that avoid running the garbage skipping step when there are over N shadow source candidates for the compaction task. With a small threshold value, flame graph shows the cpu time % of GarbageSkipper is lowered. (Below graph is obtained with threshold == 10)
!flamegraph_garbageskipper_with_threshold.png|width=684,height=350!
The full report can be found at [https://github.com/yifan-c/CASSANDRA-15581-COMPACTION-TEST/blob/main/CASSANDRA-16339/7019-Test:%20Perf%20Comparison%20%5BLCS%20-%20provide_overlapping_tombstones%20%2B%20experiments%5D.pdf]
The smaller the threshold, the read latency gets more stable. However, they are not as stable as in the steady state. In steady state, we can consider the threshold is 0.
We can observe some latency reduction in different time spans. Now I think they are coming from the slower compaction. According to the charts of the report, when the compaction throughput is low, the read latency goes low too. The garbage skipping step is CPU heavy. When the step is active, it greatly lower the I/O ops from the compaction. So in the meantime, the system can serve read queries faster.
The gain we are expecting from the step is to reduce the file size and the disk usage. However, in all the runs, there is no noticeable difference after enabling for the workload (read : write : delete = 5 : 4 : 1).
Since the reward is too little comparing to the price paid. I think it is probably better to advice not using this feature, unless there is a suitable use case that have the a lot of deletes and few reads.
> LCS steady state load of table with vs. w/o GC performance test
> ---------------------------------------------------------------
>
> Key: CASSANDRA-16339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16339
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
> Reporter: Yifan Cai
> Assignee: Yifan Cai
> Priority: Normal
> Attachments: flamegraph_garbageskipper_with_threshold.png, flamegraph_grabageskipper.png
>
>
> The testing cluster should be pre-populated with ~200GB data in each node. The baseline cluster has the table created with {{provide_overlapping_tombstones}} disabled. The other cluster has the table with {{provide_overlapping_tombstones == row}}. Compare the read, write and compaction performance between those 2 clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org