You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jonathan Gray (JIRA)" <ji...@apache.org> on 2009/10/15 20:24:31 UTC
[jira] Created: (HBASE-1908) ROOT not reassigned if only one
regionserver left
ROOT not reassigned if only one regionserver left
-------------------------------------------------
Key: HBASE-1908
URL: https://issues.apache.org/jira/browse/HBASE-1908
Project: Hadoop HBase
Issue Type: Bug
Affects Versions: 0.20.1
Reporter: Jonathan Gray
Priority: Blocker
Fix For: 0.20.2, 0.21.0
Yannis on the list uncovered an assignment bug:
{quote}
I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
{quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767610#action_12767610 ]
stack commented on HBASE-1908:
------------------------------
+1
What about issue where client would not go elsewhere after a connection refused exception? Is that a different issue?
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766168#action_12766168 ]
Jonathan Gray commented on HBASE-1908:
--------------------------------------
Binary logic ftw!
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1908:
---------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to branch and trunk, thanks for review stack.
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Dave Latham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766167#action_12766167 ]
Dave Latham commented on HBASE-1908:
------------------------------------
How about:
{{if (!isMetaServer || isSingleServer) {}}
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767629#action_12767629 ]
Jonathan Gray commented on HBASE-1908:
--------------------------------------
Opened HBASE-1920 for that issue, it is completely distinct I just uncovered it while testing for this.
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766160#action_12766160 ]
Jonathan Gray commented on HBASE-1908:
--------------------------------------
Yannis proposed fix:
{noformat}
diff RegionManager.java.FIXES RegionManager.java
414c414
< if ((!isMetaServer) || (isMetaServer && isSingleServer)) {
---
> if (!isMetaServer) {
{noformat}
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1908:
---------------------------------
Assignee: Jonathan Gray
Status: Patch Available (was: Open)
Will commit after someone reviews
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1908:
---------------------------------
Attachment: HBASE-1908-v1.patch
Trying to reproduce and then will test with this patch.
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766245#action_12766245 ]
Jonathan Gray commented on HBASE-1908:
--------------------------------------
Patch fixes the problem. It does re-assign META, but doesn't seem to matter (since it's being reassigned to regionserver already hosting it).
Also tested killing server hosting META instead of ROOT and all seems fine. Did uncover a client issue though.
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1908) ROOT not reassigned if only one
regionserver left
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766183#action_12766183 ]
Jonathan Gray commented on HBASE-1908:
--------------------------------------
Reproduced on clean branch as described. If you have two regionservers, one with META and one with ROOT, if you kill the regionserver hosting ROOT it doesn't ever get reassigned.
Odd behavior though. The META region gets reassigned to the regionserver tho it was never unassigned.
Master log: http://pastebin.com/mb4fed51
RegionServer log: http://pastebin.com/m3e079b36
> ROOT not reassigned if only one regionserver left
> -------------------------------------------------
>
> Key: HBASE-1908
> URL: https://issues.apache.org/jira/browse/HBASE-1908
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Jonathan Gray
> Priority: Blocker
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1908-v1.patch
>
>
> Yannis on the list uncovered an assignment bug:
> {quote}
> I performed additional testing with some alternate configurations and the problem arises (ONLY) when there is only one regionserver left which has the META table already assigned to it.
> In this case the ROOT table does not get assigned to the last regionserver (which holds the META table).
> Interestingly enough though when there is only one regionserver left that has the ROOT table already assign to it then it can also have the META table re-assigned to it (if again is the only server - i.e. in this scenario you can have one regionserver holding both the META and ROOT tables).
> Unless I am missing something I cannot find any reason why we cannot assign the ROOT table to the regionserver that manages the META table if it is the only one remaining (again it is an extreme case I agree that this can happen).
> I applied and tested a fix (at the hbase-0.20.0 codebase) in the RegionManager::regionsAwaitingAssignment where I add the root table in the regionstoAssign set if the it is the metaServer and also the only server.
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.