[jira] [Work started] (HBASE-29398) Server side scan metrics for bytes read from FS vs Block cache vs memstore

["Sanjeet Malhotra (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-29398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on HBASE-29398 started by Sanjeet Malhotra.
------------------------------------------------
> Server side scan metrics for bytes read from FS vs Block cache vs memstore
> --------------------------------------------------------------------------
>
>                 Key: HBASE-29398
>                 URL: https://issues.apache.org/jira/browse/HBASE-29398
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Sanjeet Malhotra
>            Assignee: Sanjeet Malhotra
>            Priority: Major
>
> Currently, HBase doesn't have a metric on the server side which counts how many bytes were read from FS vs block cache vs memstore. Reading cells from in-memory like block cache or memstore vs from FS can make latencies vary drastically.
> Separate metrics for bytes scanned from block cache vs memstore are beneficial for use cases which immediately read (like within 5 sec) after writing the data. There the expectation would be that bytes scanned from FS or block cache should be negligible unless a flush happened (which can be checked from logs) or cache on write is enabled. 
> Currently, HBase has a server side scan metric `countOfBlockBytesScanned` which aims to capture the block bytes scanned by read request. But it doesn't capture:
>  * Block bytes scanned as part of KeyValueHeap#pollRealKV(). So, we could end up scanning a lot of excess HFiles in case when scanner at the top of heap hasn't done a real seek and the row might get returned from memstore. In this case the block I/O done in scanning excess HFile won't be accounted in any metrics.
>  * Index block bytes scanned, bloom filter bytes scanned.
>  * Difference between bytes scanned from block cache vs FS.
> The proposal is to add 3 new server side scan metrics, one each for: bytes scanned from FS, bytes scanned from block cache and bytes scanned from memstore. 
>  
> I intend to cherry-pick this change to HBase 3 and HBase 2 (till HBase 2.5).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)