[jira] [Comment Edited] (CASSANDRA-15229) BufferPool Regression

["ZhaoYang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065411#comment-17065411 ] 

ZhaoYang edited comment on CASSANDRA-15229 at 3/24/20, 8:07 AM:
----------------------------------------------------------------

I am planning to implement the multiple buffer pool approach on top of CASSANDRA-15358:
* Permanent Pool - used by chunk cache, hints and other disk access - bounded by file_cache_size_in_mb
* Ephemeral Pool - used by internode connection, client-server native transport - bounded by network_cache_size_in_mb, default min(128mb, system_memory/8)

bq. With these improvements we can also safely increase the BufferPool chunk size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce the amount of global coordination and per-allocation overhead. We don’t need 1KiB granularity for allocations, nor 16 byte granularity for tiny allocations.

I am not sure if it's still necessary to increase chunk-size and min-allocation size, given they are already 128kb and 2kb..


was (Author: jasonstack):
I am planning to implement the multiple buffer pool approach on top of CASSANDRA-15358:
* Permanent Pool - used by chunk cache, hints and other disk access - bounded by file_cache_size_in_mb
* Ephemeral Pool - used by internode connection, client-server native transport - bounded by network_cache_size_in_mb, default min(128mb, system_memory/8)

> BufferPool Regression
> ---------------------
>
>                 Key: CASSANDRA-15229
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local/Caching
>            Reporter: Benedict Elliott Smith
>            Assignee: ZhaoYang
>            Priority: Normal
>             Fix For: 4.0, 4.0-beta
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we need to either change our behaviour to handle uncorrelated lifetimes or use something else.  This is particularly important with the default chunk size for compressed sstables being reduced.  If we address the problem, we should also utilise the BufferPool for native transport connections like we do for internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used for things with uncorrelated lifetimes, which essentially boils down to tracking those chunks that have not been freed and re-circulating them when we run out of completely free blocks.  We should probably also permit instantiating separate {{BufferPool}}, so that we can insulate internode messaging from the {{ChunkCache}}, or at least have separate memory bounds for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce the amount of global coordination and per-allocation overhead.  We don’t need 1KiB granularity for allocations, nor 16 byte granularity for tiny allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org