You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefan Miklosovic (Jira)" <ji...@apache.org> on 2024/12/14 22:46:00 UTC

[jira] [Commented] (CASSANDRA-13338) JMX: EstimatedPartitionCount / SnapshotSize are expensive

    [ https://issues.apache.org/jira/browse/CASSANDRA-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905763#comment-17905763 ] 

Stefan Miklosovic commented on CASSANDRA-13338:
-----------------------------------------------

By CASSANDRA-18111, we improved the speed a lot.

I have done some perf tests with before / after CASSANDRA-1811, I generated 10 tables for 5 keyspaces each and 10 snapshots per table (500 snapshots in total) and each snapshot had 10 sstables, then I measured listing of snapshots for run 1 (new stuff) and run 2 (old stuff). I run a bench test from 2 to 16 threads which would just list all over again for 1 minute. Average time vs throughput shows we are basically listing three times faster across all used threads, Latency Percentiles vs Threads shows that p99 on 8 threads (8 clients listing) takes 800ms per listing request while it took around 250ms with refactored stuff .... Lastly, Throughput vs number of threads shows that up to 8 threads (I have 8 threds CPU), we can process around 2 times more listing requests then before. This all should make latencies for listings way faster.

There is also improvement when getting "true snapshot size" this ticket is about, the speedup is basically 10x.

All the results are attached to this ticket.

I will rename this ticket to deal with "EstimatedPartitionCount" as the latter one is not a problem anymore.

> JMX: EstimatedPartitionCount / SnapshotSize are expensive
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-13338
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13338
>             Project: Apache Cassandra
>          Issue Type: Improvement
>          Components: Legacy/Observability
>            Reporter: Corentin Chary
>            Assignee: Stefan Miklosovic
>            Priority: Normal
>              Labels: lhf
>         Attachments: Average_Time_vs_Threads_Combined_snapshot_listing.png, Average_Time_vs_Threads_Combined_true_snapshot_size.png, Percentiles_vs_Threads_Combined_snapshot_listing.png, Percentiles_vs_Threads_Combined_true_snapshot_size.png, Throughput_vs_Threads_Combined _true_snapshot_size.png, Throughput_vs_Threads_Combined_snapshot_listing.png
>
>
> EstimatedPartitionCount / EstimatedRowCount / SnapshotSize seem particularly expensive. For example on our system org.apache.cassandra.metrics:type=ColumnFamily,name=SnapshotsSize can take as much as half a second.
> All this cumulated means that export stats for all your tables (with metrics-graphite or jmx_exporter) is going to take quite some time.
> We should certainly try to find the most expensive end points and see if there is a way to cache some of the values.
> cc: [~rgerard]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org