[jira] [Comment Edited] (HBASE-24754) Bulk load performance is degraded in HBase 2

["Y. SREENIVASULU REDDY (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170687#comment-17170687 ] 

Y. SREENIVASULU REDDY edited comment on HBASE-24754 at 8/4/20, 9:58 AM:
------------------------------------------------------------------------

Verified the mapper task operations, and data writing operation done by HFileOutputFormat2. There didn't observed any time taking operations.

But Time differences observed in the PutSortReducer class for processing the "Put" objects.  

For the same executed the tests and posted the results here, please find the attached sample code to reproduce the issue, for between the Branch-2 and Branch-1.3
 In Reduce operation to process the "PUT" objects observed the difference ~30% reduced.

1. Verified the test with 100000 rows.
 2. Each row size is ~1K.
 3. Each row have single column-family and 300 qualifiers
 4. Tested with java version (JDK1.8.0_232)
 5. Test Results
||Rows processing Time||Branch 1.3 Time (ms)||Branch 2 Time (ms)||%Difference||
|Test 1|12545|18955|-33.8|
|Test 2|12693|18840|-32.6|
|Test 3|12694|18939|-32.9|


was (Author: sreenivasulureddy):
Attached the sample code to reproduce the issue, for between the Branch-2 and Branch-1.3
In Reduce operation to process the "PUT" objects observed the difference ~30% reduced.

1. Verified the test with 100000 rows.
2. Each row size is ~1K.
3. Each row have single column-family and 300 qualifiers
4. Tested with java version (JDK1.8.0_232)
5. Test Results
||Rows processing Time||Branch 1.3 Time (ms)||Branch 2 Time (ms)||%Difference||
|Test 1|12545|18955|-33.8|
|Test 2|12693|18840|-32.6|
|Test 3|12694|18939|-32.9|

> Bulk load performance is degraded in HBase 2 
> ---------------------------------------------
>
>                 Key: HBASE-24754
>                 URL: https://issues.apache.org/jira/browse/HBASE-24754
>             Project: HBase
>          Issue Type: Bug
>          Components: Performance
>    Affects Versions: 2.2.3
>            Reporter: Ajeet Rai
>            Priority: Major
>         Attachments: Branch1.3_putSortReducer_sampleCode.patch, Branch2_putSortReducer_sampleCode.patch
>
>
> in our Test,It is observed that Bulk load performance is degraded in HBase 2 .
>  Test Input: 
> 1: Table with 500 region(300 column family)
> 2:  data =2 TB
> Data Sample
> 18600000001201502051000000068110,18600000001,20150205,5,404,735412,2938,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111111111111111111111111111111111111111111111111111111111111111111111111111111111
> 3: Cluster: 7 node(2 master+5 Region Server)
>  4: No of Container Launched are same in both case
> HBase 2 took 10% more time then HBase 1.3 where test input is same for both cluster
>  
> |Feature|HBase 2.2.3
>  Time(Sec)|HBase 1.3.1
>  Time(Sec)|Diff%|Snappy lib:
>   |
> |BulkLoad|21837|19686.16|-10.93|Snappy lib:
>  HBase 2.2.3: 1.4
>  HBase 1.3.1: 1.4|



--
This message was sent by Atlassian Jira
(v8.3.4#803005)