[jira] [Commented] (HBASE-15180) Reduce garbage created while reading Cells from Codec Decoder

["Jingcheng Du (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125780#comment-15125780 ] 

Jingcheng Du commented on HBASE-15180:
--------------------------------------

bq. I believe Jingcheng Du done some YCSB test with writes (G1GC we have) and he said it was giving bad results.
Thanks [~anoop.hbase].
I did some YCSB tests with HBase 1.0.0 release and G1GC (JDK is 1.8.0_51), and I used 64GB for the JVM heap in each RS, and test the cases with/without MSLAB.
I ran YCSB with the command "ycsb load hbase-10 -P 1T_workload -threads 200 -p columnfamily=family -p clientbuffering=true -s > 1T_workload.dat". I observed the performance (throughput, latency and gc pause time), the performance of the case when enabling MSLAB with chunk pool is better than the case when enabling MSLAB without chunk pool, and the case without MSLAB is worst.
The performance might be different with G1GC in different jre versions and different heap sizes. According to my tests, MSLAB with chunk was best.

> Reduce garbage created while reading Cells from Codec Decoder
> -------------------------------------------------------------
>
>                 Key: HBASE-15180
>                 URL: https://issues.apache.org/jira/browse/HBASE-15180
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15180.patch, HBASE-15180_V2.patch
>
>
> In KeyValueDecoder#parseCell (Default Codec decoder) we use KeyValueUtil#iscreate to read cells from the InputStream. Here we 1st create a byte[] of length 4 and read the cell length and then an array of Cell's length and read in cell bytes into it and create a KV.
> Actually in server we read the reqs into a byte[] and CellScanner is created on top of a ByteArrayInputStream on top of this. By default in write path, we have MSLAB usage ON. So while adding Cells to memstore, we will copy the Cell bytes to MSLAB memory chunks (default 2 MB size) and recreate Cells over that bytes.  So there is no issue if we create Cells over the RPC read byte[] directly here in Decoder.  No need for 2 byte[] creation and copy for every Cell in request.
> My plan is to make a Cell aware ByteArrayInputStream which can read Cells directly from it.  
> Same Codec path is used in client side also. There better we can avoid this direct Cell create and continue to do the copy to smaller byte[]s path.  Plan to introduce some thing like a CodecContext associated with every Codec instance which can say the server/client context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)