[jira] [Comment Edited] (CASSANDRA-15511) Utilising BTree Improvements

["Benedict Elliott Smith (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019381#comment-17019381 ] 

Benedict Elliott Smith edited comment on CASSANDRA-15511 at 1/20/20 10:34 AM:
------------------------------------------------------------------------------

I have uploaded the results of a several days of performance comparisons, and for posterity the scripts I used to generate and parse them into a spreadsheet, on the effect of this change on inserting into a partition.

The results are pretty significant; some highlights:
Collection types are universally and dramatically improved:
* >99% memory consumption reduction
* ~25x throughput

Concurrent operations on non-collections:
* vs 3.0 with locking enabled _instantly_ (i.e. upgrade to locking after first competition)
** With 4 competing threads, across the board lower memory utilisation; near-equivalent with 16-threads
** With 4 or 16 competing threads, ~200% throughput improvement
* vs 3.0 with default locking
** 100-200% throughput improvement
** 30-60% reduction in memory consumption

~25% improvement in throughput and 50% reduction in memory allocated for uncontended successful operations.

I recommend taking a look at the attached (zipped) Excel spreadsheet, which has some nice visual formatting to see the full breakdown of results.  I've also attached as {{ods}} format for compatibility, but this loses the formatting.


was (Author: benedict):
I have uploaded the results of a several days of performance comparisons, and for posterity the scripts I used to generate and parse them into a spreadsheet, on the effect of this change on inserting into a partition.

The results are pretty significant; some highlights:
Collection types are universally and dramatically improved:
* >99% memory consumption reduction
* ~25x throughput

Concurrent operations on non-collections:
* vs 3.0 with locking enabled _instantly_ (i.e. first upgrade to locking after first competition)
** With 4 competing threads, across the board lower memory utilisation; near-equivalent with 16-threads
** With 4 or 16 competing threads, ~200% throughput improvement
* vs 3.0 with default locking
** 100-200% throughput improvement
** 30-60% reduction in memory consumption

~25% improvement in throughput and 50% reduction in memory allocated for uncontended successful operations.

I recommend taking a look at the attached (zipped) Excel spreadsheet, which has some nice visual formatting to see the full breakdown of results.  I've also attached as {{ods}} format for compatibility, but this loses the formatting.

> Utilising BTree Improvements
> ----------------------------
>
>                 Key: CASSANDRA-15511
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15511
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Local/Other
>            Reporter: Benedict Elliott Smith
>            Assignee: Benedict Elliott Smith
>            Priority: Normal
>         Attachments: atomicbtreepartition.ods, atomicbtreepartition.xlsx.zip, perfsh.tar.gz
>
>
> This patch utilises CASSANDRA-15510 to improve throughput and reduce garbage produced by a number of common operations, by employing {{transformAndFilter}}, {{transform}} and {{FastBuilder}}
> * {{Row}}, {{Cell}} and {{ComplexColumnData}} cloning are implemented with {{BTree.transform}}, so no special builders are necessary; 
> ** {{Rows.copy}} removed
> * {{Rows.merge}} implemented using {{BTree.update}} and a {{ColumnData}} reconciler
> ** Zero-allocations if result of merge is same as {{existing}}
> ** Fewer comparisons
> * {{ColumnData}} reconciler implemented in same manner
> ** {{Cells.reconcileComplex}} is retired
> ** {{ComplexColumnData}} reconciliation now
> *** Garbage-free if the merge has no effect
> *** Always fewer allocations
> *** Fewer comparisons
> * {{FastBuilder}} employed widely:
> ** {{ClusteringIndexNamesFilter}} deserialization
> ** {{Columns}} deserialization
> ** {{PartitionUpdate}} deserialization
> ** {{AbstractBTreePartition}} construction
> ** Misc others
> The upshot of this work when combined with the proposed patch for CASSANDRA-15367 has a dramatic impact on operations over collection types - under contention, as much as 100x improved throughput, and hundreds of megabytes of reduced allocations.  For all operations, allocations under contention and no contention are significantly reduced and throughput improved.
> I am still awaiting the final results of the comprehensive performance comparison for {{AtomicBTreePartition}}, after which I will establish what impact there might be on compaction and normal reads, both of which should be affected by this patch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org