[jira] [Updated] (HBASE-15204) Try to estimate the cell count for adding into WALEdit

["ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ramkrishna.s.vasudevan updated HBASE-15204:
-------------------------------------------
    Attachment: HBASE-15204.patch
                WAlEdit_add_allocation_after_patch.jpg
                WAlEdit_add_allocation.jpg

Just attaching a first cut version of the patch to get feedback. We can also estimate the WALEdit size based on the number of Cells added to WALEdits by the CP. That is also covered here. But in order to pass the cellcount either we need to change the signature of batchMutate in Region or need to directly call the new batchMutate API with CellCount in HRegion directly. 
Attached jpgs shows the allocation size and count just for this Arraylist to grow. 

> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>
>                 Key: HBASE-15204
>                 URL: https://issues.apache.org/jira/browse/HBASE-15204
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15204.patch, WAlEdit_add_allocation.jpg, WAlEdit_add_allocation_after_patch.jpg
>
>
> The write path profiling shows that when we try to add Cells to WALEdits we try to do a lot of Array copy inorder to grow the Arraylist backing the WALEdits. In a simple one min profiling of the write path with 50 YCSB threads shows around 261MB of allocation done for the Array copy to happen. We can try to avoid that. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)