You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2018/06/09 04:33:00 UTC
[jira] [Comment Edited] (HBASE-20483) [PERFORMANCE] Flushing is 2x
slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506812#comment-16506812 ]
stack edited comment on HBASE-20483 at 6/9/18 4:32 AM:
-------------------------------------------------------
bq. The other jira abt SegmentScanner fixed this also right?
[~anoop.hbase] Sorry. Took me a while to generate the new graphs.
Yes, HBASE-20628 SegmentScanner does over-comparing when one flushing plus work done on speeding up Scanning -- mainly HBASE-20620 Tighter ByteBufferKeyValue Cell Comparator; part 2 and part 1 -- have made it so flush in hbase2 is now as regular if not more so than hbase1. We flush at about the same rate and the same sizes (hbase2 is tighter to the limits in my tests).
I put new results on new sheet here https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1517830749. See diagrams for runs against 4 regions, 40, and 400 comparing hbase1 to hbase2. See how in the sheet June07, https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1517830749, where we diagram the flushes. For how it was before, see the previous sheet, 'Flush History', https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826.
Let me resolve.
Oh, one thing, we are still significantly slower writing. Working on it. This issue seems to remove flushing as bottleneck. Now we are held up by the WAL system (Default WAL is better when high contention but doesn't look good in straight perf compares....)
was (Author: stack):
bq. The other jira abt SegmentScanner fixed this also right?
[~anoop.hbase] Sorry. Took me a while to generate the new graphs.
Yes, HBASE-20628 SegmentScanner does over-comparing when one flushing plus work done on speeding up Scanning -- mainly HBASE-20620 Tighter ByteBufferKeyValue Cell Comparator; part 2 and part 1 -- have made it so flush in hbase2 is now as regular if not more so than hbase1. We flush at about the same rate and the same sizes (hbase2 is tighter to the limits in my tests).
I put new results on new sheet here https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1517830749. See diagrams for runs against 4 regions, 40, and 400 comparing hbase1 to hbase2. See how in the sheet June07, https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1517830749, where we diagram the flushes. For how it was before, see the previous sheet, 'Flush History', https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826.
Let me resolve.
> [PERFORMANCE] Flushing is 2x slower in hbase2.
> ----------------------------------------------
>
> Key: HBASE-20483
> URL: https://issues.apache.org/jira/browse/HBASE-20483
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
> Reporter: stack
> Assignee: stack
> Priority: Major
> Fix For: 2.0.1
>
> Attachments: 0001-HBASE-20483-perpertual-flush-and-memstore-fill-branch-2.0.patch, 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, 0001-HBASE-20483-perpetual.tests.patch, 0001-HBASE-20483-perpetual.tests.patch, 0001-Perpetual-flush-memstore-fill-etc.-tools.patch, PerpetualFlushingProgram.patch, PerpetualFlushingProgram1.2.patch, SnapshotSegmentScanner_2.0.0.patch
>
>
> When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 Flush History" https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 for compare. First noted by [~anoop.hbase]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)