You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2017/02/23 14:43:44 UTC
[jira] [Commented] (HBASE-17682) Region stuck in merging_new state
indefinitely
[ https://issues.apache.org/jira/browse/HBASE-17682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880565#comment-15880565 ]
Hadoop QA commented on HBASE-17682:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s {color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 46s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s {color} | {color:green} branch-1.3 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s {color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 2s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 21s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 20s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s {color} | {color:green} branch-1.3 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s {color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s {color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s {color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 17m 42s {color} | {color:green} The patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s {color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s {color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 32s {color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s {color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 114m 51s {color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat |
| | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:66fbe99 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12854204/HBASE-17682.branch-1.3.001.patch |
| JIRA Issue | HBASE-17682 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 07b04de72ea6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1.3 / 8b0fabf |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-oracle:1.7.0_80 |
| findbugs | v3.0.0 |
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/5815/artifact/patchprocess/patch-unit-hbase-server.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/5815/artifact/patchprocess/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/5815/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/5815/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Region stuck in merging_new state indefinitely
> ----------------------------------------------
>
> Key: HBASE-17682
> URL: https://issues.apache.org/jira/browse/HBASE-17682
> Project: HBase
> Issue Type: Bug
> Affects Versions: 1.3.0
> Reporter: Abhishek Singh Chouhan
> Assignee: Abhishek Singh Chouhan
> Attachments: HBASE-17682.branch-1.3.001.patch, HBASE-17682.master.001.patch
>
>
> Ran into issue while tinkering around with a chaos monkey that did splits, merges and kills exclusively, which resulted in regions getting stuck in transition in merging new state indefinitely which i think happens when the rs is killed during the merge but before the ponr, in which case the new regions state in master is merging new. When the rs dies at this point the master executes RegionStates.serverOffline() for the rs which does
> for (RegionState state : regionsInTransition.values()) {
> HRegionInfo hri = state.getRegion();
> if (assignedRegions.contains(hri)) {
> // Region is open on this region server, but in transition.
> // This region must be moving away from this server, or splitting/merging.
> // SSH will handle it, either skip assigning, or re-assign.
> LOG.info("Transitioning " + state + " will be handled by ServerCrashProcedure for " + sn);
> } else if (sn.equals(state.getServerName())) {
> // Region is in transition on this region server, and this
> // region is not open on this server. So the region must be
> // moving to this server from another one (i.e. opening or
> // pending open on this server, was open on another one.
> // Offline state is also kind of pending open if the region is in
> // transition. The region could be in failed_close state too if we have
> // tried several times to open it while this region server is not reachable)
> if (state.isPendingOpenOrOpening() || state.isFailedClose() || state.isOffline()) {
> LOG.info("Found region in " + state +
> " to be reassigned by ServerCrashProcedure for " + sn);
> rits.add(hri);
> } else if(state.isSplittingNew()) {
> regionsToCleanIfNoMetaEntry.add(state.getRegion());
> } else {
> LOG.warn("THIS SHOULD NOT HAPPEN: unexpected " + state);
> }
> }
> }
> We donot handle merging new here and end up with "THIS SHOULD NOT HAPPEN: unexpected ...". Post this we have the new region which does not have any data stuck which leads to the balancer not running.
> I think we should handle mergingnew the same way as splittingnew.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)