[jira] [Resolved] (HBASE-27365) Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time

["Rajeshbabu Chintaguntla (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajeshbabu Chintaguntla resolved HBASE-27365.
---------------------------------------------
    Resolution: Fixed

Thanks for reviews [~wchevreuil] for reviews. Committed to master and branch-2, 2.4 and 2.5.

> Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-27365
>                 URL: https://issues.apache.org/jira/browse/HBASE-27365
>             Project: HBase
>          Issue Type: Improvement
>          Components: BucketCache
>    Affects Versions: 3.0.0-alpha-3
>            Reporter: Rajeshbabu Chintaguntla
>            Assignee: Rajeshbabu Chintaguntla
>            Priority: Major
>             Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> Currently in bucket cache asynchronous caching mechanism introduced where initially the blocks to be cached will be added to queue and writer threads consume the blocks from the queue and write to bucket cache. In case if block writing to bucket cache is slow then there is a chance that  queue of writer threads become full  and following block additions will be failed. In case of slower storages like s3 might introduce latencies even if we enable bigger sizes of bucket cache using ephemeral storages. So we can allow configurable wait time while adding blocks to queue so that chances of queue free up is possible during the wait time and block addition failures can be minimised. To avoid the performance impact of wait time in regular read paths we can use the wait time mainly during background operations like compactions, flushes or prefetches etc.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)