You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Yechao Chen (Jira)" <ji...@apache.org> on 2020/08/07 07:07:00 UTC
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using
reflection in SnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172940#comment-17172940 ]
Yechao Chen commented on HBASE-24831:
-------------------------------------
with the pr we get about 7.6%/9.1% performance improve
in my test case with ScanPerformanceEvaluation tools
like that
{{hbase org.apache.hadoop.hbase.ScanPerformanceEvaluation -t snapshotscanmapreduce -rs /tmp/hbase/rs1 -sn sp_test}}
*with pr*
1#
total time to scan: 143943 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 109.1 MB/s
total rows : 10485760
throughput : 71.6 K rows/s
total cells : 104857600
throughput : 716.1 K cells/s
2#
total time to scan: 145948 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 107.6 MB/s
total rows : 10485760
throughput : 70.6 K rows/s
total cells : 104857600
throughput : 706.2 K cells/s
total time to scan: 149962 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 104.7 MB/s
total rows : 10485760
throughput : 68.7 K rows/s
total cells : 104857600
throughput : 687.2 K cells/s
*without pr*
1th:
total time to scan: 156929 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 100 MB/s
total rows : 10485760
throughput : 65.6 K rows/s
total cells : 104857600
throughput : 656.4 K cells/s
2th
total time to scan: 156078 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 100 MB/s
total rows : 10485760
throughput : 65.6 K rows/s
total cells : 104857600
throughput : 656.4 K cells/s
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -------------------------------------------------------------
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
> Reporter: Yechao Chen
> Assignee: Yechao Chen
> Priority: Major
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)