[jira] [Comment Edited] (HBASE-23938) Replicate slow/large RPC calls to HDFS

["Viraj Jasani (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097334#comment-17097334 ] 

Viraj Jasani edited comment on HBASE-23938 at 5/6/20, 7:17 AM:
---------------------------------------------------------------

Oh yes, these query patterns are definitely useful. Since there would not be any Get query and user would want to use Scan with filters (op type, user, client etc), we don't have specifics about rowkeys. Maybe *TooSlowLog.hashcode() (protobuf generated)* as byte[] could be used as rawkey and single CF with all qualifiers (region, op type, client etc) so that client can use Scan with single/multiple ColumnValueFilter to get relevant data.

Edit: Since it would be better to scan table and get data in ascending order of timestamps for users, and even if we use ("epoch timestamp" + "hashcode of object") as rowkey, it should be fine given that we are not expecting too much of hotspotting for new region at any point in time i.e. (system IO/RPC calls can not be slow forever).


was (Author: vjasani):
Oh yes, these query patterns are definitely useful. Since there would not be any Get query and user would want to use Scan with filters (op type, user, client etc), we don't have specifics about rowkeys. Maybe *TooSlowLog.hashcode() (protobuf generated)* as byte[] could be used as rawkey and single CF with all qualifiers (region, op type, client etc) so that client can use Scan with single/multiple ColumnValueFilter to get relevant data.

> Replicate slow/large RPC calls to HDFS
> --------------------------------------
>
>                 Key: HBASE-23938
>                 URL: https://issues.apache.org/jira/browse/HBASE-23938
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0
>            Reporter: Viraj Jasani
>            Assignee: Viraj Jasani
>            Priority: Major
>
> We should provide capability to replicate complete slow and large RPC logs to HDFS or create new system table in addition to Ring Buffer. This way we don't lose any of slow logs and operator can retrieve all the slow/large logs. Replicating logs to HDFS / creating new system table should be configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)