You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2010/11/11 00:52:14 UTC
[jira] Created: (HBASE-3221) Race between splitting and disabling
Race between splitting and disabling
------------------------------------
Key: HBASE-3221
URL: https://issues.apache.org/jira/browse/HBASE-3221
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.90.0
There's a race when you disable a table, if one of the region servers started the split before the call and it reports after the master scanned the .META. regions, then you're not disabling the daughter regions. I see this in the master log:
{noformat}
2010-11-10 15:29:35,990 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attemping to disable table TestTable
2010-11-10 15:29:35,996 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 2 regions.
...
2010-11-10 15:29:39,014 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table is done=true
2010-11-10 15:29:39,105 INFO org.apache.hadoop.hbase.master.ServerManager: Received REGION_SPLIT: TestTable,,1289431761746.9de9305168be83098273e083f24ea5d8.:
Daughters; TestTable,,1289431775593.a08b127a61b89e268129aa022fd18ce1.,
TestTable,0001037720,1289431775593.4a5f831723ffbdb859d45510742d9926. from hbasedev,60020,1289431673756
...
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3221) Race between splitting and disabling
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-3221:
--------------------------------------
Attachment: HBASE-3221.patch
This patch survived an onslaught of disabling/enabling mixed with massive imports. It unassigns the daughter regions when the RS checks in with the split message if the table is disabled/disabling.
> Race between splitting and disabling
> ------------------------------------
>
> Key: HBASE-3221
> URL: https://issues.apache.org/jira/browse/HBASE-3221
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.90.0
>
> Attachments: HBASE-3221.patch
>
>
> There's a race when you disable a table, if one of the region servers started the split before the call and it reports after the master scanned the .META. regions, then you're not disabling the daughter regions. I see this in the master log:
> {noformat}
> 2010-11-10 15:29:35,990 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attemping to disable table TestTable
> 2010-11-10 15:29:35,996 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 2 regions.
> ...
> 2010-11-10 15:29:39,014 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table is done=true
> 2010-11-10 15:29:39,105 INFO org.apache.hadoop.hbase.master.ServerManager: Received REGION_SPLIT: TestTable,,1289431761746.9de9305168be83098273e083f24ea5d8.:
> Daughters; TestTable,,1289431775593.a08b127a61b89e268129aa022fd18ce1.,
> TestTable,0001037720,1289431775593.4a5f831723ffbdb859d45510742d9926. from hbasedev,60020,1289431673756
> ...
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3221) Race between splitting and disabling
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930873#action_12930873 ]
Jean-Daniel Cryans commented on HBASE-3221:
-------------------------------------------
For the record, this kind of stuff was impossible in the old master. Really exciting that it's finally fixed!
> Race between splitting and disabling
> ------------------------------------
>
> Key: HBASE-3221
> URL: https://issues.apache.org/jira/browse/HBASE-3221
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.90.0
>
> Attachments: HBASE-3221.patch
>
>
> There's a race when you disable a table, if one of the region servers started the split before the call and it reports after the master scanned the .META. regions, then you're not disabling the daughter regions. I see this in the master log:
> {noformat}
> 2010-11-10 15:29:35,990 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attemping to disable table TestTable
> 2010-11-10 15:29:35,996 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 2 regions.
> ...
> 2010-11-10 15:29:39,014 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table is done=true
> 2010-11-10 15:29:39,105 INFO org.apache.hadoop.hbase.master.ServerManager: Received REGION_SPLIT: TestTable,,1289431761746.9de9305168be83098273e083f24ea5d8.:
> Daughters; TestTable,,1289431775593.a08b127a61b89e268129aa022fd18ce1.,
> TestTable,0001037720,1289431775593.4a5f831723ffbdb859d45510742d9926. from hbasedev,60020,1289431673756
> ...
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-3221) Race between splitting and disabling
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-3221.
---------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Thanks Stack, I committed this to trunk.
> Race between splitting and disabling
> ------------------------------------
>
> Key: HBASE-3221
> URL: https://issues.apache.org/jira/browse/HBASE-3221
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.90.0
>
> Attachments: HBASE-3221.patch
>
>
> There's a race when you disable a table, if one of the region servers started the split before the call and it reports after the master scanned the .META. regions, then you're not disabling the daughter regions. I see this in the master log:
> {noformat}
> 2010-11-10 15:29:35,990 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attemping to disable table TestTable
> 2010-11-10 15:29:35,996 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 2 regions.
> ...
> 2010-11-10 15:29:39,014 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table is done=true
> 2010-11-10 15:29:39,105 INFO org.apache.hadoop.hbase.master.ServerManager: Received REGION_SPLIT: TestTable,,1289431761746.9de9305168be83098273e083f24ea5d8.:
> Daughters; TestTable,,1289431775593.a08b127a61b89e268129aa022fd18ce1.,
> TestTable,0001037720,1289431775593.4a5f831723ffbdb859d45510742d9926. from hbasedev,60020,1289431673756
> ...
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3221) Race between splitting and disabling
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930936#action_12930936 ]
stack commented on HBASE-3221:
------------------------------
+1
Small patch. Go commit.
> Race between splitting and disabling
> ------------------------------------
>
> Key: HBASE-3221
> URL: https://issues.apache.org/jira/browse/HBASE-3221
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.90.0
>
> Attachments: HBASE-3221.patch
>
>
> There's a race when you disable a table, if one of the region servers started the split before the call and it reports after the master scanned the .META. regions, then you're not disabling the daughter regions. I see this in the master log:
> {noformat}
> 2010-11-10 15:29:35,990 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attemping to disable table TestTable
> 2010-11-10 15:29:35,996 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 2 regions.
> ...
> 2010-11-10 15:29:39,014 INFO org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table is done=true
> 2010-11-10 15:29:39,105 INFO org.apache.hadoop.hbase.master.ServerManager: Received REGION_SPLIT: TestTable,,1289431761746.9de9305168be83098273e083f24ea5d8.:
> Daughters; TestTable,,1289431775593.a08b127a61b89e268129aa022fd18ce1.,
> TestTable,0001037720,1289431775593.4a5f831723ffbdb859d45510742d9926. from hbasedev,60020,1289431673756
> ...
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.