You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ramkrishna.s.vasudevan (Created) (JIRA)" <ji...@apache.org> on 2012/03/15 12:39:38 UTC
[jira] [Created] (HBASE-5583) Master restart on create table with
splitkeys does not recreate table with all the splitkey regions
Master restart on create table with splitkeys does not recreate table with all the splitkey regions
---------------------------------------------------------------------------------------------------
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0
-> Create table using splitkeys
-> MAster goes down before all regions are added to meta
-> On master restart the table is again enabled but with less number of regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with
splitkeys does not recreate table with all the splitkey regions
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245371#comment-13245371 ]
ramkrishna.s.vasudevan commented on HBASE-5583:
-----------------------------------------------
First step update the status in the below to CREATETABLE
/hbase/table/tableName
/hbase/table/tableName/currentstep
-> Create nodes with hregioninfo in the constructor of create table handler.
Create those many nodes as in the splitkeys.
-> /hbase/table/tableName/currentStep
CREATED_SPLIT_KEYS (after the creation of all splitkey nodes)
CREATING_TD
ADD_TO_META
ASSIGN_USER_REGIONS
if all are done 'ENABLED' will be the state of the table.
-> add regioninfo to META and on success delete the node created node one by one
-> IF master fail over happens
See in what step the node is
============================
If failed in ASSIGN_USER_REGIONS
Just reassign all the regions
If failed in ADD_TO_META
Check what regions are not yet added to meta and then do the assignment for all the regions
We can ensure that the nodes that are available in the zk are the ones for which the meta entry is not updated
If failed in CREATING_TD
If tabledescriptor not found create it again. if not go to the next step
If failed in CREATED_SPLIT_KEYS
Delete the node so that atleast the next time creation is successful. Just log it as we cannot throw error back to the client.
===================================================================================
We need to create one more api in client called 'isTableAvailable()' accepting splitKeys also. so that the user can use them to check if really
the table is created with all the splitkeys.
Currently the isTableAvailable() just checks if atleast one region is assigned to any RS.
The current changes does not guarentee that the user table will be created in a synchronous way to the client createTable api.
but will ensure that while creating any table if the master failover happens the table doesnot get created with less number of
regions but still the user thinks table creation is sucessful.
We now distinguish the states where a table was in create flow or in the enable flow.
Previously the ENABLING state was in master fail over scenario.
Pls provide your inputs.
> Master restart on create table with splitkeys does not recreate table with all the splitkey regions
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-5583
> URL: https://issues.apache.org/jira/browse/HBASE-5583
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> -> Create table using splitkeys
> -> MAster goes down before all regions are added to meta
> -> On master restart the table is again enabled but with less number of regions than specified in splitkeys
> Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists.
> Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with
splitkeys does not recreate table with all the splitkey regions
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233456#comment-13233456 ]
ramkrishna.s.vasudevan commented on HBASE-5583:
-----------------------------------------------
One more issue is there
If the master goes down just after writing the table descriptors and before adding anything to the META, on master restart we move the enabling state tables to enabled only if they have entry in META. So this is another area we need to address.
So ZK should ideally used in a better way to track the table creation steps.
> Master restart on create table with splitkeys does not recreate table with all the splitkey regions
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-5583
> URL: https://issues.apache.org/jira/browse/HBASE-5583
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> -> Create table using splitkeys
> -> MAster goes down before all regions are added to meta
> -> On master restart the table is again enabled but with less number of regions than specified in splitkeys
> Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists.
> Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with
splitkeys does not recreate table with all the splitkey regions
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233252#comment-13233252 ]
ramkrishna.s.vasudevan commented on HBASE-5583:
-----------------------------------------------
@Jon
Yes we should go with some form of persisting the splitkeys. ZK should be the best place for that.
> Master restart on create table with splitkeys does not recreate table with all the splitkey regions
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-5583
> URL: https://issues.apache.org/jira/browse/HBASE-5583
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> -> Create table using splitkeys
> -> MAster goes down before all regions are added to meta
> -> On master restart the table is again enabled but with less number of regions than specified in splitkeys
> Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists.
> Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with
splitkeys does not recreate table with all the splitkey regions
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246311#comment-13246311 ]
Zhihong Yu commented on HBASE-5583:
-----------------------------------
Expanding table creation state coverage makes sense.
> Master restart on create table with splitkeys does not recreate table with all the splitkey regions
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-5583
> URL: https://issues.apache.org/jira/browse/HBASE-5583
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> -> Create table using splitkeys
> -> MAster goes down before all regions are added to meta
> -> On master restart the table is again enabled but with less number of regions than specified in splitkeys
> Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists.
> Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with
splitkeys does not recreate table with all the splitkey regions
Posted by "Jonathan Hsieh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233150#comment-13233150 ]
Jonathan Hsieh commented on HBASE-5583:
---------------------------------------
Maybe we should put plans for master actions into ZK, and then remove each step as they are completed. This would allow a backup master or the restarted master to pick up and continue where the previous master had died.
So this we may have a zk structure act like a queue with region infos and then let the masters drain from there to create .META. rows.
I was thinking about adding a flag to info:regioninfo or extra column ("info:unfinshedPresplitCreate") but this would seem to suffer from a similar problem if the master failed as these were being removed after all regions created.
> Master restart on create table with splitkeys does not recreate table with all the splitkey regions
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-5583
> URL: https://issues.apache.org/jira/browse/HBASE-5583
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> -> Create table using splitkeys
> -> MAster goes down before all regions are added to meta
> -> On master restart the table is again enabled but with less number of regions than specified in splitkeys
> Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists.
> Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira