You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "HBase QA (JIRA)" <ji...@apache.org> on 2019/05/07 19:12:00 UTC
[jira] [Commented] (HBASE-21418) Reduce a number of reseek
operations in MemstoreScanner when seek point is close to the current row.
[ https://issues.apache.org/jira/browse/HBASE-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835033#comment-16835033 ]
HBase QA commented on HBASE-21418:
----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 7s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 24s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_212 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_222 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 51s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m 40s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_212 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_222 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s{color} | {color:green} the patch passed with JDK v1.8.0_212 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s{color} | {color:green} the patch passed with JDK v1.7.0_222 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 16s{color} | {color:red} hbase-server: The patch generated 16 new + 377 unchanged - 4 fixed = 393 total (was 381) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m 23s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 8m 30s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1 2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s{color} | {color:green} the patch passed with JDK v1.8.0_212 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed with JDK v1.7.0_222 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 2s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m 8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 40s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m 32s{color} | {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/PreCommit-HBASE-Build/263/artifact/patchprocess/Dockerfile |
| JIRA Issue | HBASE-21418 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12947549/HBASE-21418.branch-1.2.002.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e3e4d35fc810 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-1.2 / af250d2 |
| maven | version: Apache Maven 3.0.5 |
| Default Java | 1.7.0_222 |
| Multi-JDK versions | /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_212 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_222 |
| checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/263/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/263/testReport/ |
| Max. process+thread count | 4372 (vs. ulimit of 10000) |
| modules | C: hbase-client hbase-server U: . |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/263/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> Reduce a number of reseek operations in MemstoreScanner when seek point is close to the current row.
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-21418
> URL: https://issues.apache.org/jira/browse/HBASE-21418
> Project: HBase
> Issue Type: Improvement
> Components: Scanners
> Affects Versions: 1.2.5
> Reporter: Jeongdae Kim
> Assignee: Jeongdae Kim
> Priority: Minor
> Labels: performance
> Attachments: HBASE-21418.branch-1.2.001.patch, HBASE-21418.branch-1.2.001.patch, HBASE-21418.branch-1.2.002.patch
>
>
> We observed “responseTooSlow” logs for Get requests in our production clusters. even some get requests were responded after 10 seconds.
> Affected get requests were done with the timerange, and target rows have many columns that have some versions.
> We reproduced this issue, and found this behavior happens only when scanning in the memstore. after flushing the HStore, this slow response issue for Get disappeared and all same get requests are responded very quickly.
>
> We investigated this case, and found this performance difference between memstore scanner and hfile scanner is caused by the number of reseek operations executed while scanning. When a store scanner needs to reseek the next column, Hfile scanner wisely decide whether it have to reseek or not by checking the seek point is in current block, whereas memstore scanner just do reseek without decision unlike Hfile scanner. In our case, almost all columns in the memstore have older timestamp than scan(get)’s timerange, and so many reseek operations occur as much as about the number of columns. This results in increasing the response time of Get requests sporadically.
>
> To improve the reseek operation of the memstore scanner, i think it’s better skipping than seeking when reseek requested, if seek point is quite close to current cell that the scanner is pointing now.(Actually, i changed MatchCode.SEEK_NEXT_COL to MatchCode.Skip in our case, and the response time of Get was 6x faster than before) But we can’t decide whether seek point is close to the current cell or not, because memstore scannner has no information such as next block index.
> Before HBASE-13109, Scan.HINT_LOOKAHEAD was introduced to handle like this case, and it may be deprecated someday. But, i think that hint is still be useful for the memstore scanner to try to skip first, before reseeking, and with this option we can make reseek operations of memstore scanner smarter.
>
> I tested this patch in our case, and got the same result as i changed matchcode (mentioned above).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)