You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Subbu M Iyer (JIRA)" <ji...@apache.org> on 2011/08/17 16:07:27 UTC
[jira] [Created] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
--------------------------------------------------------------------------------------------------------------------
Key: HBASE-4213
URL: https://issues.apache.org/jira/browse/HBASE-4213
Project: HBase
Issue Type: Improvement
Reporter: Subbu M Iyer
This Jira is a slight variation in approach to what is being done as part of
https://issues.apache.org/jira/browse/HBASE-1730
Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
1. With out enable/disabling the table.
2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154333#comment-13154333 ]
Ted Yu commented on HBASE-4213:
-------------------------------
Compared to the implementation from HBASE-1730, Subbu's code would take less amount of effort to stabilize.
Thanks for the support, Lars.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: schema-update.png
Task status showing the completion of instant schema change
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126326#comment-13126326 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Latest patch supports:
1. LB aware schema change process.
2. Monitoring support to show the alter status/error etc.
3. Progress reporting.
4. Added extensive test scenarios to test various failover/failure scenarios. (17 test cases so far)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-trunk-v7.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151042#comment-13151042 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-11-16 06:05:02.324776)
Review request for hbase, Todd Lipcon, Andrew Purtell, and Subbu Iyer.
Changes
-------
New patch uses master.synchronousBalanceSwitch() to ensure that balancer is switched off during instant schema change
TestInstantSchemaChange#testLoadBalancerBlocksDuringSchemaChangeRequests passed.
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
Diffs (updated)
-----
/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202523
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
/src/main/resources/hbase-default.xml 1202381
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213.addendum
Addendum to temporarily disable testInstantSchemaOperationsInZKForMasterFailover which relies on predetermined sleep interval for schema janitor to clean up schema change request
Subbu will come up with better test.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086326#comment-13086326 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Attached patch is based on the following pattern.
On Startup:
1. On startup, create a new ZK node called "schema" if it doesn't exisit already. (Master and RS)
2. Master starts a MasterSchemaChangeTracker and sets a watch for childerns of /hbase/schema.
3. Each RS will start SchemaChangeTracker and set a watch childrens of /hbase/schema.
On Alter DDL: (modify table, add family, modify family)
1. Master will bypass the table disabled check for alter DDL requests.
2. On successful HTD change, master will create a new ZK node /hbase/schema/<table name>.
3. Each RS will get notified of the schema change through the new ZK node /hbase/schema/<table name>.
a. If the RS currently has online regions for the table, refresh the regions as follows:
a.1: Close the region.
a.2: Remove from online regions.
a.3: Get the latest HTD descriptor.
a.4: Reopen the region with new HTD.
a.5: Add the region to list of online regions.
b.RS will signal that it has successfully processed the schema change by
creating a new ZK node /hbase/schema/<table name>/<RS server name>
This will happen either if currently the RS does not have any online regions for the table,
or if has successfully completed the region refresh with new schema changes.
4. Master will get notified of the list of region servers who have successfully carried out the
schema change operation based on childrens of zknode /hbase/schema/<table name>.
5. Master will recursively delete the node /hbase/schema/<table name>, if the number of childrens
of /hbase/schema/<table name> is greater than or equal to current number of active region servers.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087503#comment-13087503 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Attached version 1 with most of call outs addressed.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-v9.txt
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-Instant_Schema_change_through_ZK.patch
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151433#comment-13151433 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-11-16 19:12:03.308667)
Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
Changes
-------
Patch addresses Lars' comments
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
Diffs (updated)
-----
/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202523
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
/src/main/resources/hbase-default.xml 1202381
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086751#comment-13086751 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Regarding:
public boolean refreshSchema(byte[] tableName) throws IOException {
I agree that it could be a void instead of a boolean. Will change that as well.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102281#comment-13102281 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I got the following when applying patch v4:
{code}
patching file src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java
Hunk #1 FAILED at 135.
1 out of 1 hunk FAILED -- saving rejects to file src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java.rej
{code}
When I tried to compile the code:
{code}
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:[1179,2] method does not override or implement a method from a supertype
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java:[127,13] modifyTable(byte[],org.apache.hadoop.hbase.HTableDescriptor,boolean) in org.apache.hadoop.hbase.client.HBaseAdmin cannot be applied to (byte[],org.apache.hadoop.hbase.HTableDescriptor)
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java:[157,17] cannot find symbol
symbol : method modifyColumn(byte[],org.apache.hadoop.hbase.HColumnDescriptor)
location: class org.apache.hadoop.hbase.client.HBaseAdmin
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java:[159,17] cannot find symbol
symbol : method addColumn(byte[],org.apache.hadoop.hbase.HColumnDescriptor)
location: class org.apache.hadoop.hbase.client.HBaseAdmin
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java:[285,6] modifyTable(byte[],org.apache.hadoop.hbase.HTableDescriptor,boolean) in org.apache.hadoop.hbase.client.HBaseAdmin cannot be applied to (byte[],org.apache.hadoop.hbase.HTableDescriptor)
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java:[355,6] cannot find symbol
symbol : method addColumn(byte[],org.apache.hadoop.hbase.HColumnDescriptor)
location: class org.apache.hadoop.hbase.client.HBaseAdmin
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java:[368,6] cannot find symbol
symbol : method deleteColumn(byte[],byte[])
location: class org.apache.hadoop.hbase.client.HBaseAdmin
/Users/tyu/top-hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java:[380,6] cannot find symbol
symbol : method modifyColumn(byte[],org.apache.hadoop.hbase.HColumnDescriptor)
location: class org.apache.hadoop.hbase.client.HBaseAdmin
{code}
Also, was progress reporting implemented in v4 ?
It can be done in the separate JIRA.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087112#comment-13087112 ]
Ted Yu commented on HBASE-4213:
-------------------------------
Thanks for the explanation.
Tracing where path variable
{code}
+ int rsCount = ZKUtil.getNumberOfChildren(this.watcher, path);
{code}
comes from:
{code}
+ if (path.startsWith(watcher.schemaZNode)
{code}
It doesn't align with your description.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150886#comment-13150886 ]
Ted Yu commented on HBASE-4213:
-------------------------------
https://reviews.apache.org/r/1786/ has been updated with 4213-0.92.v2
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-V9-Support_instant_schema_changes_through_ZK.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Open (was: Patch Available)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153235#comment-13153235 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-11-18 23:39:40.651341)
Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
Changes
-------
Latest patch with updated tests.
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
Diffs (updated)
-----
/pom.xml 1203789
/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202941
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202523
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1203441
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202941
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
/src/main/resources/hbase-default.xml 1202381
/src/test/java/org/apache/hadoop/hbase/client/InstantSchemaChangeTestBase.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeSplit.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: HBASE-4213_Instant_schema_change_-Version_2_.patch
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102335#comment-13102335 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Ted,
I addressed all the issues + a merge conflict and also ran all the tests on my local. All tests pass except the following which has nothing to do with this effort.
Results :
Failed tests: testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting): expected:<1> but was:<0>
testMultipleResubmits(org.apache.hadoop.hbase.master.TestSplitLogManager): expected:<2> but was:<3>
testMergeTool(org.apache.hadoop.hbase.util.TestMergeTool): 'merging regions 0 and 1' failed with errCode -1
Tests in error:
testScanMultipleVersions(org.apache.hadoop.hbase.TestScanMultipleVersions): Cannot lock storage /work/HBase0909/hbase/build/hbase/test/dfs/name1. The directory is already locked.
testAll(org.apache.hadoop.hbase.thrift.TestThriftServer): Cannot lock storage /work/HBase0909/hbase/build/hbase/test/dfs/name2. The directory is already locked.
Tests run: 970, Failures: 3, Errors: 2, Skipped: 9
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151681#comment-13151681 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-11-17 00:28:04.519410)
Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
Changes
-------
Patch addresses Lars' latest comments (except for the last comment about String where table changes in each iteration)
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
Diffs (updated)
-----
/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202941
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202523
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202941
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
/src/main/resources/hbase-default.xml 1202381
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-Fixed_NPE_in_RS_during_alter_.patch
This patch goes on top of previous commit.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151727#comment-13151727 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I saw some 'Too many open files' error:
https://builds.apache.org/job/PreCommit-HBASE-Build/271//testReport/org.apache.hadoop.hbase.client/TestInstantSchemaChange/testInstantSchemaChangeExclusions/
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104288#comment-13104288 ]
Andrew Purtell commented on HBASE-4213:
---------------------------------------
Just want to clarify that my +1 is conditional on having the config option that selects between the schema update options, with the old one the default; and an expectation that HBASE-4370 will be developed and go in soon after, along with many tests for failures at different points in the update processing.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104010#comment-13104010 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Attached V7 with all the review comments + above mentioned additions.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086651#comment-13086651 ]
Ted Yu commented on HBASE-4213:
-------------------------------
In EventHandler.java, should isSchemaChangeEvent() include C_M_DELETE_FAMILY as well ?
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087406#comment-13087406 ]
Ted Yu commented on HBASE-4213:
-------------------------------
One important decision we need to make about this feature is whether we support simultaneous schema changes for more than one table.
If it is supported, master needs to maintain a set of region servers per table at the time schema change for that table starts. This way, master knows when the change can be deemed completed.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144773#comment-13144773 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
JD,
Fixed the NPE that is causing the RS to die. All tests pass and don't see this exception.
Would be great if you can give it one more try with your concurrent alter + split scenario. I am also adding a test case for the concurrent split + alter scenario and I am validating the same.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143401#comment-13143401 ]
Jean-Daniel Cryans commented on HBASE-4213:
-------------------------------------------
The latest patch doesn't apply to 0.92, would it be possible to have another version for it? Thanks.
Also looking over the patch names, it would sure be nice if they followed some sort of convention.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104255#comment-13104255 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Andrew, thanks for your comments.
In V8.
1. Axed all boolean based public API's from HBaseAdmin and HMAster.
2. Schema changes will now drive off the config param.
3. alter_instant removed.
4. We have new JIRA 4370 where we plan to address the Master/RS failover scenarios in depth. So, we will probably address your new test case scenario there. The current JIRA is to put a basic framework to support schema changes through ZK in place.
Currently, when a RS dies halfway through the change the ZK node might stay on for ever, although the schema changes would have been completed when the RS comes back alive. This will be addressed intelligently in 4370 by stamping the RS hostname in ZK and so on. (more details in 4370)
Hope this helps and let me know for any thing else.
thanks
Subbu
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103104#comment-13103104 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review1862
-----------------------------------------------------------
/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/1786/#comment4254>
Please add javadoc for instantAdd.
/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/1786/#comment4255>
Add javadoc for instantAdd.
/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/1786/#comment4256>
Javadoc.
/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/1786/#comment4257>
Please wrap long lines.
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
<https://reviews.apache.org/r/1786/#comment4258>
Javadoc for new param.
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<https://reviews.apache.org/r/1786/#comment4260>
@Override should be kept.
- Ted
On 2011-09-12 18:36:02, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-12 18:36:02)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. This addresses bug HBASE-4213.
bq. https://issues.apache.org/jira/browse/HBASE-4213
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1169522
bq. /src/main/ruby/hbase/admin.rb 1169522
bq. /src/main/ruby/shell.rb 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1169522
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-V5-Support_instant_schema_changes_through_ZK.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151707#comment-13151707 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12503966/4213-trunk.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 58 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/271//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/271//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/271//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104266#comment-13104266 ]
Andrew Purtell commented on HBASE-4213:
---------------------------------------
+1 Thanks Subbu
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Open (was: Patch Available)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151583#comment-13151583 ]
Ted Yu commented on HBASE-4213:
-------------------------------
HBASE-4370 has mostly been covered by latest patch.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v2.txt
Patch after addressing Lars' comments
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087514#comment-13087514 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
I did test delete family from shell and it worked with out any additional support. I just found that TableDeleteFamilyHandler is using event type "EventType.C_M_ADD_FAMILY" instead of EventType.C_M_DELETE_FAMILY.
So, delete family is already taken care but i will fix the DeleteHandler to use the correct event type as well as well as incorporate the required changes to reflect the same during instant schema change.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088733#comment-13088733 ]
Ted Yu commented on HBASE-4213:
-------------------------------
bq. 4. Master stores the number of online RS while creating the schema change node.
Suppose RS1 was included in this count. Later it crashed and was brought up (either manually or by automated script). We should be able to tell that RS1 has completed schema change (by loading updated HTD).
My suggestion is to record the names of region servers in the above step.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Fix Version/s: (was: 0.92.0)
0.94.0
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154297#comment-13154297 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504508/4213-trunk-v7.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/318//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/318//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/318//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154396#comment-13154396 ]
Hudson commented on HBASE-4213:
-------------------------------
Integrated in HBase-TRUNK #2468 (See [https://builds.apache.org/job/HBase-TRUNK/2468/])
HBASE-4213 Support for fault tolerant, instant schema updates with out master's intervention through ZK
tedyu :
Files :
* /hbase/trunk/pom.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/trunk/src/main/resources/hbase-default.xml
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/InstantSchemaChangeTestBase.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeSplit.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-trunk-v7.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213.v6
Patch version 6 is same as version 5.
It is accepted by review board.
I think the original author of the patch should be the person sending out review request.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151864#comment-13151864 ]
Ted Yu commented on HBASE-4213:
-------------------------------
>From https://builds.apache.org/job/PreCommit-HBASE-Build/275//testReport/org.apache.hadoop.hbase.client/TestInstantSchemaChange/testInstantSchemaJanitor/:
{code}
java.net.SocketException: Too many open files
at sun.nio.ch.Net.socket0(Native Method)
at sun.nio.ch.Net.socket(Net.java:97)
at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:84)
{code}
The resource usage for tests in TestInstantSchemaChange is high.
I was able to run TestInstantSchemaChange successfully for several times on MacBook.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-4213:
-----------------------------
Assignee: Subbu M Iyer
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-trunk-v3.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104142#comment-13104142 ]
Ted Yu commented on HBASE-4213:
-------------------------------
@Andy:
4213-V7-Support_instant_schema_changes_through_ZK.patch is the version 7 patch.
The new methods in HBaseAdmin allow overriding 'hbase.instant.schema.change.enabled' config parameter. Is this Okay ?
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153233#comment-13153233 ]
Ted Yu commented on HBASE-4213:
-------------------------------
See schema-update.png for screen snapshot.
Note, since the schema change completed so fast, I wasn't able to try to trigger region splitting.
Also, in CompactSplitThread.requestSplit(final HRegion r, byte[] midKey), call to waitForInflightSchemaChange() has been added.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Open (was: Patch Available)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088462#comment-13088462 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Latest patch includes the following changes:
1. Load balancer will be turned off (if already enabled) before an instant schema change operation.
2. Load balancer will be turned on (if was on before the instant schema change).
3. Added a ZK based check instead of sleep timer for couple of tests.
4. Master stores the number of online RS while creating the schema change node. On completion the master compares against this count (rather than current number of active RS which might have changed during the schema change operation) with number of active RS to determine if all the RS have completed the schema change operation.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150887#comment-13150887 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-11-15 23:28:55.604278)
Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
Changes
-------
Updated with 4213-0.92.v2
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
This addresses bug HBASE-4213.
https://issues.apache.org/jira/browse/HBASE-4213
Diffs (updated)
-----
/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
/src/main/resources/hbase-default.xml 1202381
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk.txt
Latest patch adds test category.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151020#comment-13151020 ]
Lars Hofhansl commented on HBASE-4213:
--------------------------------------
@Ted: Hmm... But we haven't started the schema change at that point, right? So the master would not know about the intention.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v7.txt
Rebased after HBASE-4815 went in.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-Nov-2-2011_patch_.patch
Patch from Subbu for TRUNK.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-v9.txt
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104152#comment-13104152 ]
Andrew Purtell commented on HBASE-4213:
---------------------------------------
bq. The new methods in HBaseAdmin allow overriding 'hbase.instant.schema.change.enabled' config parameter. Is this Okay ?
Isn't the configuration variable there to support a transition, if people are not comfortable using ZK for schema updates in the beginning? Surely we will not have two ways of updating schema coexisting for a long time. There would not be a need to toggle between them. Adding new APIs that we'd have to quickly deprecate does not make sense to me.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-v9.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-0.92.txt
Patch 4213-0.92.txt is rebased for 0.92 branch.
HMaster.waitForCurrentSchemaChange() has been removed.
TestInstantSchemaChangeFailover and TestInstantSchemaChange passed.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145961#comment-13145961 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
- New patch to address concurrent schema alter and region splits.
- Added two new tests to test the concurrent split + alter scenarios.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v4.txt
Updated usage of MonitoredTask.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-v9.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-4213) Support instant schema
updates with out master's intervention (i.e with out enable/disable and
bulk assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088733#comment-13088733 ]
Ted Yu edited comment on HBASE-4213 at 8/22/11 2:33 PM:
--------------------------------------------------------
bq. 4. Master stores the number of online RS while creating the schema change node.
Suppose RS1 was included in this count. Later it crashes and is brought up (either manually or by automated script). We should be able to tell that RS1 has completed schema change (by loading updated HTD).
My suggestion is to record the names of region servers in the above step.
was (Author: yuzhihong@gmail.com):
bq. 4. Master stores the number of online RS while creating the schema change node.
Suppose RS1 was included in this count. Later it crashed and was brought up (either manually or by automated script). We should be able to tell that RS1 has completed schema change (by loading updated HTD).
My suggestion is to record the names of region servers in the above step.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154300#comment-13154300 ]
Ted Yu commented on HBASE-4213:
-------------------------------
3 out of the 4 test failures were due to 'Too many open files'.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153361#comment-13153361 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504313/4213-trunk-v7.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/302//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/302//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/302//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102336#comment-13102336 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Yes, I agree with you that we shall address 1) progress reporting and 2) secondary master take over through separate JIRA's.
Let me know if you have any comments. The changes are same except new additions to address the 1730 merge.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153451#comment-13153451 ]
Ted Yu commented on HBASE-4213:
-------------------------------
TestInstantSchemaChange failures were due to 'Too many open files'.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089934#comment-13089934 ]
Ted Yu commented on HBASE-4213:
-------------------------------
Implementing progress report is not difficult.
ZK node /hbase/schema/<table name>/<RS server name> would be created when region server receives schema change notification and carry value in the form of <completed regions>/<total regions>.
MasterSchemaChangeTracker would be able to compute overall progress.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153140#comment-13153140 ]
Ted Yu commented on HBASE-4213:
-------------------------------
The test failures were caused by 'Too many open files'.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151774#comment-13151774 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504009/4213-trunk-v2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 59 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/275//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/275//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/275//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154328#comment-13154328 ]
Lars Hofhansl commented on HBASE-4213:
--------------------------------------
Awesome. Thanks for all the work on this Subbu and Ted. This will be incredibly useful! (Once online schema changes stabilize in general)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126348#comment-13126348 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Thanks for your support on this Ted.
Also, on my local (with only one RS), for a 70 region test table I see a more than 100+% improvement in alter performance.
For a table in a bigger cluster with large number of regions, we should see drastic difference in performance.
thanks
Subbu
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135707#comment-13135707 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Attached latest patch with trunk merge.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Open (was: Patch Available)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102353#comment-13102353 ]
Ted Yu commented on HBASE-4213:
-------------------------------
In HBaseAdmin.java, javadoc should be added for instantAdd parameter:
{code}
* @param column column descriptor of column to be added
* @throws IOException if a remote or network exception occurs
*/
- public void addColumn(final String tableName, HColumnDescriptor column)
+ public void addColumn(final String tableName, HColumnDescriptor column, boolean instantAdd)
throws IOException {
{code}
Personally I prefer creating an enum, possibly called SCHEMA_OP_TYPE, which should contain NORMAL and INSTANT. There are two benefits:
1. the enum is more meaningful for readers
2. SCHEMA_OP_TYPE.INSTANT covers add/delete. This may provide benefit for related code.
This is minor, though.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504313/4213-trunk-v7.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/302//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/302//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/302//console
This message is automatically generated.)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143546#comment-13143546 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502207/4213-0.92.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -161 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 52 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.util.TestFSUtils
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/162//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/162//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/162//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v3.txt
Move waitForInflightSchemaChange() to the right place.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
Patch testing v3.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-trunk-v4.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153132#comment-13153132 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504247/4213-trunk-v5.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 68 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/296//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/296//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/296//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Summary: Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK. (was: Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign))
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152606#comment-13152606 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504154/4213-trunk-v3.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/287//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/287//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/287//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154503#comment-13154503 ]
stack commented on HBASE-4213:
------------------------------
Want to do the test fix in another issue Ted and Subbu?
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087383#comment-13087383 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Ted,
Yes, you are absolutely right and very good catch.
The second check should be like:
int rsCount = ZKUtil.getNumberOfChildren(this.watcher, watcher.rsZNode);
instead of
int rsCount = ZKUtil.getNumberOfChildren(this.watcher, path);
I have taken care of this.
thanks
Subbu
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-trunk-v5.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104372#comment-13104372 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review1886
-----------------------------------------------------------
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment4372>
Master checks balanceSwitch at the beginning of balance(). Meaning, the toggle of balanceSwitch would take effect in the next round of balancing.
We have two options in order to make balanceSwitch() call synchronous with balance():
1. Change balanceSwitch() to be synchronous directly. This breaks the API compatibility
2. Add new method, e.g. blockingBalanceSwitch(), to HMasterInterface()
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4373>
From listChildrenAndWatchThem():
if (children == null) {
return null;
}
So I guess the answer is yes.
- Ted
On 2011-09-12 18:36:02, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-12 18:36:02)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. This addresses bug HBASE-4213.
bq. https://issues.apache.org/jira/browse/HBASE-4213
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1169522
bq. /src/main/ruby/hbase/admin.rb 1169522
bq. /src/main/ruby/shell.rb 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1169522
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151654#comment-13151654 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review3307
-----------------------------------------------------------
Looks good to me now (I admit, I only browsed over the new tracker code).
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
<https://reviews.apache.org/r/1786/#comment7392>
This is no longer the case.
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment7394>
It also might not return the current list of all online regions, as onlineRegions is changed by other callers (like the one above) without a lock. Is that a problem here?
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment7395>
Super minor nit: Make this string just once?
- Lars
On 2011-11-16 19:12:03, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-16 19:12:03)
bq.
bq.
bq. Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202523
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
bq. /src/main/resources/hbase-default.xml 1202381
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
bq. /src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153413#comment-13153413 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504332/4213-trunk-v7.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.TestDrainingServer
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/306//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/306//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/306//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v7.txt
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-0.92.v2
Version 2 of patch for 0.92 branch.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087110#comment-13087110 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
They both are checking different counts. One counts the number of
active RS at this moment (based on number of childrens of /hbase/rs)
and other counts the number of RS successfully processed the schema
change (based on number of childrens of /hbase/schema/<table name> ).
If number of RS who have processed the schema change is >= number of
active RS at this moment then master presumes that all RS have
acknowledged the schema change and hence goes ahead with delete
operation.
On Thu, Aug 18, 2011 at 6:45 AM, Ted Yu (JIRA) <ji...@apache.org> wrote:
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-102511.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103925#comment-13103925 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Based on further discussions, here are some call outs:
1. Provide separate public API's for alter instant operations so as to not break existing public API's with out deprecation or prior notice.
2. Provide a config level setting to enable instant schema update feature. (defaults to false). This will also enable us to release this feature in a more controlled and transparent manner.
3. We don't want to intimidate developers with scary boolean flags
that does things in a such a way that they may not completely
understand or care about.
4.
Providing a developer level API to use instant-alter is good in the
sense they can fully capitalize a scalable/fault tolerant variants. At
the same time it might be confusing to some in the sense that why we
are even providing a "not scalable/fault tolerant" variant in the
first place.
5. We don't want to expose implementation details such as "this flag
uses ZK to track schema changes and so on"
So, long story short:
In addition to review comments, V7 will include the following:
1. Add new config parameter "hbase.instant.schema.change.enabled"
(exact name of flag is open) and default to "false" so that all
existing API's will go over the current path untouched.
2. Separate public API's for all alter operations which supports the
new pattern in addition to existing public API's. New API's will take
in a boolean parameter that overrides the config setting on a per
request basis.
3. Internally both the API's will go through the same pipeline to
promote reuse as well as maintainability.
Please let me know your thoughts/comments.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102344#comment-13102344 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Ted,
Can you help me post it to RB? I have problems posting the same.
thanks
Subbu
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-101211-Support_instant_schema_changes_through_ZK.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102897#comment-13102897 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
This addresses bug HBASE-4213.
https://issues.apache.org/jira/browse/HBASE-4213
Diffs
-----
/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java 1169522
/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1169522
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1169522
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1169522
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1169522
/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java 1169522
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1169522
/src/main/ruby/hbase/admin.rb 1169522
/src/main/ruby/shell.rb 1169522
/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 1169522
/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1169522
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 1169522
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1169522
/src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1169522
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087521#comment-13087521 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I think load balancer should be temporarily disabled at the beginning of instant schema change.
At the end, balancer should be enabled.
We should perform some testing on a table which has over 400 regions to see how long it takes for a cluster under normal load.
If it is not too long, disabling balancer should be fine.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087393#comment-13087393 ]
Ted Yu commented on HBASE-4213:
-------------------------------
@Subbu:
Glad I was able to help.
I think nothing is absolute. I believe in relativity :-)
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088461#comment-13088461 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Latest version attached.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107005#comment-13107005 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-09-17 03:16:42.534475)
Review request for hbase.
Changes
-------
bq. From Subbu:
In addition to most of the review comments, V9 adds:
1. Handle cases when the table is not in enabled state.
2. Ensures that concurrent alter requests are not missed.
3. Validate that alter can survive cases when one of the RS goes down during alter, or multiple RS going down during alter or adding new RS servers to the cloud during an alter.
4. Handle cases when primary master is brought down during alter and secondary takes over.
5. Added new test cases for all of the above mentioned cases.
To-Do:
1. Revisit the load balancer toggling. It doesn't seem right.
2. Add monitoring support for alter process.
3. Gracefully handle cases when ZooKeeper throws exceptions during an alter.
4. Add fine grained tests to validate master failover cases.
5. Address any other comments/suggestions.
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
This addresses bug HBASE-4213.
https://issues.apache.org/jira/browse/HBASE-4213
Diffs (updated)
-----
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1171786
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1171840
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1171786
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1171786
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1171786
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1171786
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1171786
/src/main/resources/hbase-default.xml 1171786
/src/main/ruby/hbase/admin.rb 1171786
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1171786
/src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1171786
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-v9.txt
Subbu's patch v9
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126331#comment-13126331 ]
Ted Yu commented on HBASE-4213:
-------------------------------
Nice work, Subbu.
Please adjust your patch for TRUNK:
{code}
patching file src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java
Hunk #1 FAILED at 18.
Hunk #2 succeeded at 52 (offset 1 line).
1 out of 2 hunks FAILED -- saving rejects to file src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java.rej
...
Hunk #4 succeeded at 241 (offset 14 lines).
1 out of 4 hunks FAILED -- saving rejects to file src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java.rej
...
2 out of 20 hunks FAILED -- saving rejects to file src/main/java/org/apache/hadoop/hbase/master/HMaster.java.rej
...
1 out of 2 hunks FAILED -- saving rejects to file src/main/java/org/apache/hadoop/hbase/master/MasterServices.java.rej
{code}
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
TRUNK build passed.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126333#comment-13126333 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Ted,
Yes, I will. My trunk is from 09/12. I wanted to get some initial feedback.
This time I could successfully request for a RB request. :-):-) Let me know how we handle this LB integration makes sense to you.
thanks
Subbu
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126344#comment-13126344 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I only looked at part of the patch.
For getAndStartSchemaJanitorChore(), javadoc should be corrected:
{code}
+ // Start up the load balancer chore
+ Chore chore = new Chore(name, schemaJanitorPeriod, master) {
{code}
For getAndStartBalancerChore():
{code}
+ if (doesLoadBalancerNodeExists()) {
{code}
The trailing s in the above name is unnecessary.
For doesLoadBalancerNodeExists():
{code}
+ int r = ZKUtil.checkExists(this.zooKeeper, zooKeeper.loadBalancerZNode);
+ return ZKUtil.checkExists(this.zooKeeper, zooKeeper.loadBalancerZNode) != -1;
{code}
The assignment above is for debugging, I guess. Please remove it.
I think a better name for waitForCurrentSchemaChange() would be waitForCurrentSchemaChangeToComplete().
But the following debug should be removed:
{code}
+ LOG.debug("Schema change operation is in progress. Waiting for " +
+ "it to complete before running the load balancer.");
+ Thread.sleep(1000);
{code}
In balance():
{code}
+ // I don't think we need the following check because we already have
+ // acquired the lock on balancer ??
{code}
So the following isLoadBalancerRunning() call can be removed :-)
For TableEventHandler.waitForLoadBalancerToComplete(), since master.isLoadBalancerRunning() checks zk node, should isLoadBalancerRunning() be moved to ZKUtil ? This would save RPC call.
MasterSchemaChangeTracker.handleFailedExpiredSchemaChanges() should be called handleFailedOrExpiredSchemaChanges()
So latest patch doesn't use the new synchronous switchBalancer(). Interesting.
There're pending questions in the patch.
{code}
+ // Should we block here until we successfully create a LB node in ZK?
+ if (createLoadBalancerNode()) {
{code}
createLoadBalancerNode() should distinguish KeeperException's that may arise and inform Master so that Master can make better decision.
For getAlterStatusFromSchemaChangeTracker():
{code}
+ LOG.debug("MasterAlterStatus is NULL for table = "
+ + Bytes.toString(tableName));
+ // should we throw IOException here as it makes more sense?
+ return new Pair<Integer, Integer>(0,0);
{code}
I think IOE makes more sense in the above case.
After the modified patch is posted to review board, comments would have better context.
Thanks for your perseverance, Subbu.
This feature has been requested by many customers.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246782#comment-13246782 ]
stack commented on HBASE-4213:
------------------------------
Reverted over in HBASE-5715
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151858#comment-13151858 ]
Lars Hofhansl commented on HBASE-4213:
--------------------------------------
Failure in TestInstantSchemaChange is suspicious. :)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151554#comment-13151554 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12503931/4213-0.92.v4
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 58 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/267//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/267//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/267//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: (was: 4213-trunk-v6.txt)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: HBASE_Instant_schema_change-version_3_.patch
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089188#comment-13089188 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Sure. We have two approaches.
1730 + master failover
Or
4213 + throttling + progress status.
Let me know which one we want and I can address it accordingly.
One question I have is:
Do we really need to go through bulk unassign/assign(through ZK) for schema changes ?
What is the real drawback of simply signaling the RS cloud to refresh their regions on schema changes?
Sent from my iPhone
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249195#comment-13249195 ]
Hudson commented on HBASE-4213:
-------------------------------
Integrated in HBase-0.94-security #8 (See [https://builds.apache.org/job/HBase-0.94-security/8/])
HBASE-5715 Revert 'Instant schema alter' for now, HBASE-4213 (Revision 1310017)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/branches/0.94/src/main/resources/hbase-default.xml
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/InstantSchemaChangeTestBase.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeSplit.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151012#comment-13151012 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review3293
-----------------------------------------------------------
Late to this review...
Awesome feature. Thanks for doing this Subbu!
Looks good to me in general.
Minor comments inline.
As a general comment, I think it would be better if we could drive from the ZK notifications, rather than doing all this sleeping in a loop waiting for some other action to complete by rechecking the ZK node; would probably require a lot of refactoring to make the code event based.
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<https://reviews.apache.org/r/1786/#comment7373>
whitespace
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment7372>
whitespace... here and below
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment7366>
How do you know for sure that while we waited for inflightSplits no new round of load balancing has started?
Or maybe that is not a problem?
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment7369>
Maybe put this method on schema tracker?
Similar code is found CompactSplitThread.java.
Also, can we use the ZK notifications here instead of waiting in a loop?
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment7368>
We're not added the "year" line anymore.
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment7371>
year line...
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
<https://reviews.apache.org/r/1786/#comment7370>
Year line ...
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java
<https://reviews.apache.org/r/1786/#comment7367>
I think we're not putting this line anymore.
- Lars
On 2011-11-15 23:28:55, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-15 23:28:55)
bq.
bq.
bq. Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. This addresses bug HBASE-4213.
bq. https://issues.apache.org/jira/browse/HBASE-4213
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
bq. /src/main/resources/hbase-default.xml 1202381
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
bq. /src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152707#comment-13152707 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504174/4213-trunk-v4.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 64 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.util.TestFSUtils
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestShell
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/289//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/289//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/289//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104280#comment-13104280 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review1885
-----------------------------------------------------------
/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/1786/#comment4337>
coming late to the game on the review here...
but why do we need a separate "instantAdd" boolean. If this feature works, we should always use it. Otherwise we have to maintain multiple code paths, and the API is more complicated. Plus, this breaks a public API.
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<https://reviews.apache.org/r/1786/#comment4338>
the fact that these two constructors differ only by number of arguments is quite suspect. Same with the two functions below.
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment4339>
we need to somehow expose this to the user in a way aside from just the logs. For example, we could make a MonitoredTask for the table operation, and mark it as failed here.
Ideally it would actually be a response to the original RPC that the user called in HTableAdmin.
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment4340>
we need some kind of try/finally here. Plus, this can easily race against other things enabling/disabling the balancer. We need a kind of "balancer disable stack" so that if multiple operations need to disable the balancer for a period, they can all work together. Plus, if the balancer is mid-balance, does balanceSwitch(false) wait for the balance to finish?
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment4341>
definitely need a sleep or yield. Seems like a code smell as well
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment4342>
why not just make this the while condition?
Since ZK is notification-based, can't we wait on this occuring rather than polling in a loop?
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4343>
!isEmpty() instead of size() comparison
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4344>
what happens if on the reopen, an exception is thrown? does the region get orphaned? we need some kind of recovery path here.
Example case: what if you alter a table and change compression to a codec that this RS doesn't have?
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4345>
seems like DEBUG level.
This whole process should be TaskMonitored.
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4351>
this function never returns null. So at call sites, don't check for null - just adds cruft.
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4346>
thread safety properties here? I'm surprised we don't need any locking. For example, what happens to regions that are in the process of opening?
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4348>
!isEmpty()
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4349>
DEBUG level
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4350>
DEBUG
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/1786/#comment4347>
should be ERROR or at least WARN level.
Need to include the exception traceback.
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
<https://reviews.apache.org/r/1786/#comment4352>
empty javadoc, plus it doesn't @return anything, it's void
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4353>
this should be registered _before_ the above line, right?
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4354>
if this fails, shouldn't we abort the server or something? We don't want a half-functional RS running that doesn't respond to schema changes.
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4355>
empty javadoc
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4356>
debug
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4358>
debug
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4359>
why is this the right behavior? could use a comment here, because it seems wrong.
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4360>
why create and then set data, instead of just creating with the correct data?
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4357>
rename to "schemaChangeNodeExists" or "doesSchemaChangeNodeExist"
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4361>
dead code
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4362>
would servers ever be null?
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4363>
what would cause this and what are the ramifications?
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4364>
this would be on *any* ZK change anywhere, right? Maybe better to invert this whole condition and just "return;" if it's for a path you're not interested in, reducing the nesting level of the function
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4365>
this order seems wrong, again
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4366>
debug
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4367>
again error paths seem suspect
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4368>
again: when would these error paths get hit, and what state would we be left in? needs comments describing why it's safe to ignore the errors
/src/main/ruby/hbase/admin.rb
<https://reviews.apache.org/r/1786/#comment4369>
similar to above, I see no reason to duplicate APIs here... having multiple alter commands is just going to confuse users and mean we have more code paths to test
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
<https://reviews.apache.org/r/1786/#comment4371>
need test cases covering failure cases. EG modify a column to change the codec to a random string, or some other modification that will cause an error on open.
another test case: what if you use the "instant" API while the table is disabled?
what about while the table is in the process of opening?
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
<https://reviews.apache.org/r/1786/#comment4370>
these while conditions should at least yield if not sleep. Plus why not just put the condition in the while, rather than if/break?
- Todd
On 2011-09-12 18:36:02, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-12 18:36:02)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. This addresses bug HBASE-4213.
bq. https://issues.apache.org/jira/browse/HBASE-4213
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1169522
bq. /src/main/ruby/hbase/admin.rb 1169522
bq. /src/main/ruby/shell.rb 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1169522
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154306#comment-13154306 ]
Ted Yu commented on HBASE-4213:
-------------------------------
Integrated to TRUNK.
Thanks for the patch Subbu.
Thanks for the review, Lars, Todd and Andy.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102229#comment-13102229 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Ted/Stack,
As we discussed here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151051#comment-13151051 ]
Lars Hofhansl commented on HBASE-4213:
--------------------------------------
Thanks Ted.
{code}
protected void handleInstantSchemaChanges(List<HRegionInfo> regions) {
...
try {
...
master.synchronousBalanceSwitch(false);
waitForInflightSplit(regions, status);
...
} finally {
master.synchronousBalanceSwitch(true);
}
{code}
Since balancing might be disabled for other reasons, should probably remember the old balancer state and restore that in the finally block.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504174/4213-trunk-v4.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 64 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.util.TestFSUtils
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestShell
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/289//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/289//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/289//console
This message is automatically generated.)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144778#comment-13144778 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502604/4213-Fixed_NPE_in_RS_during_alter_.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/187//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150921#comment-13150921 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I tried the following commands for table t1 which had one column f1 and they worked:
{code}
alter 't1', { NAME => 'f1', VERSIONS => 1 }
alter 't1', { NAME => 'f2', VERSIONS => 1 }
{code}
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103198#comment-13103198 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review1863
-----------------------------------------------------------
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4261>
Javadoc should explain the role of this class.
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
<https://reviews.apache.org/r/1786/#comment4262>
I think this should be: ... SchemaChangeTracker
- Ted
On 2011-09-12 18:36:02, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-12 18:36:02)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. This addresses bug HBASE-4213.
bq. https://issues.apache.org/jira/browse/HBASE-4213
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1169522
bq. /src/main/ruby/hbase/admin.rb 1169522
bq. /src/main/ruby/shell.rb 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1169522
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087507#comment-13087507 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
>> If a new region for this table comes in meantime, it'll be ok? It'll find new schema?
Yes. New regions are perfectly fine as they will see new HTD.
>> What happens if master decides to balance a region at this time? Or disable it? While reopenRegions is
>> running?
Hmm. This one is interesting. Let me take a closer look at how to address it. Any suggestions?
>> What Ted says on delete cf.
TBD. I am on it.
>>Can you look in zk or something rather than wait on a timer before moving on?
Yup. Will do.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104053#comment-13104053 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/#review1883
-----------------------------------------------------------
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
<https://reviews.apache.org/r/1786/#comment4335>
How many times is this expected to spin? Should there be a sleep here?
- Andrew
On 2011-09-12 18:36:02, Ted Yu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1786/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-12 18:36:02)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. From Subbu:
bq. here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
bq.
bq. 1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
bq.
bq. 2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
bq.
bq. 3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
bq.
bq.
bq. This addresses bug HBASE-4213.
bq. https://issues.apache.org/jira/browse/HBASE-4213
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java 1169522
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
bq. /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1169522
bq. /src/main/ruby/hbase/admin.rb 1169522
bq. /src/main/ruby/shell.rb 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
bq. /src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1169522
bq. /src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java 1169522
bq.
bq. Diff: https://reviews.apache.org/r/1786/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Ted
bq.
bq.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-0.92.v4
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247572#comment-13247572 ]
Hudson commented on HBASE-4213:
-------------------------------
Integrated in HBase-0.94 #89 (See [https://builds.apache.org/job/HBase-0.94/89/])
HBASE-5715 Revert 'Instant schema alter' for now, HBASE-4213 (Revision 1310017)
Result = FAILURE
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/branches/0.94/src/main/resources/hbase-default.xml
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/InstantSchemaChangeTestBase.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeSplit.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: HBASE-4213-Instant_schema_change.patch
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Open (was: Patch Available)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153203#comment-13153203 ]
Lars Hofhansl commented on HBASE-4213:
--------------------------------------
Seems like this is getting ready for check-in. The most important part to have confidence in is whether this change will impact stability even when the feature is switched off.
What do you think Ted?
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153315#comment-13153315 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504296/4213-trunk-v6.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 66 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/297//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/297//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/297//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-4213) Support instant schema
updates with out master's intervention (i.e with out enable/disable and
bulk assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086356#comment-13086356 ]
Ted Yu edited comment on HBASE-4213 at 8/17/11 2:47 PM:
--------------------------------------------------------
I haven't gone through the patch yet.
bq. 5. Master will recursively delete the node /hbase/schema/<table name>, if the number of childrens of /hbase/schema/<table name> is greater than or equal to current number of active region servers.
You meant recursively deleting the children of node /hbase/schema/<table name>, right ?
For region server which joins the cluster after the creation of /hbase/schema/<table name>, it should be able to find out that it already reads the most recent HTD. Does it create a child node under /hbase/schema/<table name> ?
was (Author: yuzhihong@gmail.com):
I haven't gone through the patch yet.
bq. 5. Master will recursively delete the node /hbase/schema/<table name>, if the number of childrens
of /hbase/schema/<table name> is greater than or equal to current number of active region servers.
You meant recursively deleting the children of node /hbase/schema/<table name>, right ?
For region server which joins the cluster after the creation of /hbase/schema/<table name>, it should be able to find out that it already reads the most recent HTD. Does it create a child node under /hbase/schema/<table name> ?
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086645#comment-13086645 ]
Ted Yu commented on HBASE-4213:
-------------------------------
In HRegionServer.java, the following method would always return true if there is no intervening IOException:
{code}
+ public boolean refreshSchema(byte[] tableName) throws IOException {
{code}
I wonder if the return type should be void.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104295#comment-13104295 ]
Andrew Purtell commented on HBASE-4213:
---------------------------------------
In fact I'm wondering now why HBASE-4370 is separate.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151581#comment-13151581 ]
Ted Yu commented on HBASE-4213:
-------------------------------
TestDistributedLogSplitting#testWorkerAbort passed on my MacBook.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086356#comment-13086356 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I haven't gone through the patch yet.
bq. 5. Master will recursively delete the node /hbase/schema/<table name>, if the number of childrens
of /hbase/schema/<table name> is greater than or equal to current number of active region servers.
You meant recursively deleting the children of node /hbase/schema/<table name>, right ?
For region server which joins the cluster after the creation of /hbase/schema/<table name>, it should be able to find out that it already reads the most recent HTD. Does it create a child node under /hbase/schema/<table name> ?
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v6.txt
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151049#comment-13151049 ]
jiraposter@reviews.apache.org commented on HBASE-4213:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1786/
-----------------------------------------------------------
(Updated 2011-11-16 06:12:33.670038)
Review request for Todd Lipcon, Andrew Purtell and Subbu Iyer.
Changes
-------
New patch remembers prevBalanceSwitch and uses it to restore load balancer switch.
Summary
-------
bq. From Subbu:
here is the latest patch that support alter_instant, an instant schema change command that supports (Add, Modify, Delete column and Modify table) actions through ZK.
1. This pattern capitalizes on the fact that HRI's are now in HDFS and need not be sent over again from Master to RS cloud on every schema change event.
2. Offers real time instant schema change as we bypass the explicit bulk reassign (unassign + assign) of regions from master to RS.
3. Offers fault tolerant schema change support as schema changes now go through ZK. Secondary master taking over a failed schema change will be addressed through a separate JIRA.
Diffs (updated)
-----
/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 1202381
/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java 1202523
/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1202381
/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java 1202381
/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java PRE-CREATION
/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1202381
/src/main/resources/hbase-default.xml 1202381
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java PRE-CREATION
/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1202381
/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 1202381
Diff: https://reviews.apache.org/r/1786/diff
Testing
-------
Unit tests pass.
Thanks,
Ted
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154417#comment-13154417 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504535/4213.addendum
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/320//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247671#comment-13247671 ]
Hudson commented on HBASE-4213:
-------------------------------
Integrated in HBase-TRUNK-security #158 (See [https://builds.apache.org/job/HBase-TRUNK-security/158/])
HBASE-5715 Revert 'Instant schema alter' for now, HBASE-4213 (Revision 1310012)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/trunk/src/main/resources/hbase-default.xml
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/InstantSchemaChangeTestBase.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeSplit.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086626#comment-13086626 ]
Ted Yu commented on HBASE-4213:
-------------------------------
In TableEventHandler.java, Exception handling should be enhanced:
{code}
+ } catch (KeeperException e) {
+ LOG.warn("Instant schema change failed for table " + tableName );
+ }
{code}
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102342#comment-13102342 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I created HBASE-4370 for the follow-on work.
I think publishing patch version 5 on review board should make it easier for other developers to comment.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502207/4213-0.92.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -161 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 52 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.util.TestFSUtils
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/162//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/162//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/162//console
This message is automatically generated.)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-4213:
----------------------------------
Fix Version/s: 0.92.0
Whatever the resolution this should have the same fix version as HBASE-1730
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142666#comment-13142666 ]
Ted Yu commented on HBASE-4213:
-------------------------------
TestInstantSchemaChange failures were due to 'Too many open files'
TestThriftHBaseServiceHandler failures were due to another hanging minicluster.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088126#comment-13088126 ]
Ted Yu commented on HBASE-4213:
-------------------------------
@Subbu:
You are going to upload another patch with load balancer related code, right ?
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Open (was: Patch Available)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086328#comment-13086328 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
As earlier, for some reasons I am not able to attach my patch to the review board.
Ted/Stack: Can one of you help me with this?
thanks
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089001#comment-13089001 ]
Ted Yu commented on HBASE-4213:
-------------------------------
Discussed this feature with Nileema, Stack, Karthick.
Desired characteristics include:
1. Accommodating master failover.
2. Ability for displaying progress information. This would be useful for Ops.
3. Throttling. HBASE-1730 provides the ability to ensure that only a configurable number of regions are closed per region server at a time.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104054#comment-13104054 ]
Andrew Purtell commented on HBASE-4213:
---------------------------------------
I don't see the V7 patch attached?
bq. We don't want to intimidate developers with scary boolean flags that does things in a such a way that they may not completely understand or care about.
Yes, this jumped out almost immediately. Should not have the boolean flags on the admin methods. If 'hbase.instant.schema.change.enabled' is true, then just do it? Is another set of admin APIs necessary?
Shell shouldn't need a new command, just do it "instant" if config var is enabled?
The unit test looks ok but it might be useful to include a case where a RS is killed while in the middle of a schema change. Confirm that the updates are all applied successfully when recovery is complete.
Other comment on the V6 patch up on RB.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502604/4213-Fixed_NPE_in_RS_during_alter_.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/187//console
This message is automatically generated.)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088224#comment-13088224 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Yes. I will.. I am on it.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087523#comment-13087523 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Yup. sure that makes perfect sense.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150820#comment-13150820 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12503785/4213-0.92.v2
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 58 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestHFileBlock
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/258//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/258//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/258//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151018#comment-13151018 ]
Ted Yu commented on HBASE-4213:
-------------------------------
@Lars:
Master would check whether schema change is in progress before running balancer:
{code}
+ if (schemaChangeTracker.isSchemaChangeInProgress()) {
+ LOG.debug("Schema change operation is in progress. Waiting for " +
+ "it to complete before running the load balancer.");
+ return false;
+ }
+ loadBalancerRunning = true;
{code}
In handleInstantSchemaChanges() where you asked the question:
{code}
waitForLoadBalancerToComplete(status);
waitForInflightSplit(regions, status);
{code}
See also testLoadBalancerBlocksDuringSchemaChangeRequests()
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088691#comment-13088691 ]
Ted Yu commented on HBASE-4213:
-------------------------------
{code}
+ boolean currentBalanceSwitch = master.isBalancerEnabaled();
+ if (this.master.isBalancerEnabaled()) {
+ LOG.info("Load Balancer enabled. Disabling load balancer for " +
+ "instant schema change");
+ this.master.balanceSwitch(false);
+ }
{code}
I think there is no need to introduce a new method. master.balanceSwitch() returns the old value which can be stored in currentBalanceSwitch.
{code}
+ while(true) {
+ if (!masterSchemaChangeTracker.isSchemaChangeNodeExists(
+ Bytes.toString(tableName))) {
+ break;
+ }
+ }
{code}
Should we introduce fault tolerance logic in the above loop ? E.g. if a region server goes down and thus would not acknowledge completion of the schema change.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145973#comment-13145973 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502847/4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/215//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087012#comment-13087012 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I wonder if the following check is accurate because servers and rsCount are both returned from ZKUtil methods:
{code}
+ int rsCount = ZKUtil.getNumberOfChildren(this.watcher, path);
+ if (servers != null && servers.size() >= rsCount) {
{code}
In TestInstantSchemaChange:
{code}
miniHBaseCluster = TEST_UTIL.startMiniCluster(3);
{code}
I added rsCount in your log statement.
Then I saw the following in org.apache.hadoop.hbase.client.TestInstantSchemaChange-output.txt:
{code}
tyumac:trunk tyu$ grep ' region servers have successfully ' target/surefire-reports/org.apache.hadoop.hbase.client.TestInstantSchemaChange-output.txt
2011-08-17 21:54:54,995 DEBUG [main-EventThread] zookeeper.MasterSchemaChangeTracker(69): All 2 region servers have successfully processed the schema changes for table = testSchemachange . Deleting the schema change node = /hbase/schema/testSchemachange
2011-08-17 21:54:57,459 DEBUG [main-EventThread] zookeeper.MasterSchemaChangeTracker(69): All 2 region servers have successfully processed the schema changes for table = testSchemachangeForAddColumn . Deleting the schema change node = /hbase/schema/testSchemachangeForAddColumn
2011-08-17 21:54:58,120 DEBUG [main-EventThread] zookeeper.MasterSchemaChangeTracker(69): All 2 region servers have successfully processed the schema changes for table = testSchemachangeForModifyColumn . Deleting the schema change node = /hbase/schema/testSchemachangeForModifyColumn
2011-08-17 21:54:58,330 DEBUG [main-EventThread] zookeeper.MasterSchemaChangeTracker(69): All 3 region servers have successfully processed the schema changes for table = testSchemachangeNode . Deleting the schema change node = /hbase/schema/testSchemachangeNode
{code}
Looks like master might delete schema change node prematurely.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-V7-Support_instant_schema_changes_through_ZK.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106776#comment-13106776 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
In addition to most of the review comments, V9 adds:
1. Handle cases when the table is not in enabled state.
2. Ensures that concurrent alter requests are not missed.
3. Validate that alter can survive cases when one of the RS goes down during alter, or multiple RS going down during alter or adding new RS servers to the cloud during an alter.
4. Handle cases when primary master is brought down during alter and secondary takes over.
5. Added new test cases for all of the above mentioned cases.
To-Do:
1. Revisit the load balancer toggling. It doesn't seem right.
2. Add monitoring support for alter process.
3. Gracefully handle cases when ZooKeeper throws exceptions during an alter.
4. Add fine grained tests to validate master failover cases.
5. Address any other comments/suggestions.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153220#comment-13153220 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I have tested this feature on dev cluster and observed satisfactory behavior.
For a 75 region table with 280K rows, instant schema update took 2.2 seconds.
HBASE-4815 is introducing a config parameter which would govern online (current) and instant schema changes.
If schema change is totally disabled, impact on stability is minimal.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150854#comment-13150854 ]
Ted Yu commented on HBASE-4213:
-------------------------------
It turns out that 4213-0.92.v2 can be cleanly applied to TRUNK as well.
{code}
635 mt -Dtest=TestInstantSchemaChange#testConcurrentInstantSchemaChangeAndSplit
636 mt -Dtest=TestInstantSchemaChange#testInstantSchemaJanitor
637 mt -Dtest=TestInstantSchemaChange#testInstantSchemaChangeExclusions
638 mt -Dtest=TestInstantSchemaChange#testInstantSchemaChangeWhileRSOpenRegionFailure
{code}
The above tests passed individually on MacBook.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v7.txt
Forgot to set hbase.online.schema.update.enable to true for the unit tests.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk-v6.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151026#comment-13151026 ]
Ted Yu commented on HBASE-4213:
-------------------------------
I wasn't able to find switchBalancer() or synchronousBalanceSwitch(created for instant schema change) in the patch.
I think synchronousBalanceSwitch(false) should be called at line 258.
synchronousBalanceSwitch(true) should be called in a finally block at line 285.
Thanks for the finding, Lars.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143605#comment-13143605 ]
Jean-Daniel Cryans commented on HBASE-4213:
-------------------------------------------
First time I tried it on my cluster, all 15 region servers died with this just after calling the alter (which returned successfully):
{quote}
2011-11-03 23:17:49,996 ERROR org.apache.hadoop.hbase.master.HMaster: Region server sv4r27s44,62023,1320362227032 reported a fatal error:
ABORTING region server sv4r27s44,62023,1320362227032: Unhandled exception: null
Cause:
java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:1001)
at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800)
at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678)
at java.lang.Thread.run(Thread.java:662)
{quote}
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-V10-Support_instant_schema_changes_through_ZK.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087369#comment-13087369 ]
stack commented on HBASE-4213:
------------------------------
bq. If number of RS who have processed the schema change is >= number of active RS at this moment then master presumes that all RS have acknowledged the schema change and hence goes ahead with delete operation.
If I add ten regionservers to a cluster in between start of a schema change and before it completes, could 'if (servers != null && servers.size() >= rsCount)' never trigger?
Just remove this:
{code}
# Table should be disabled
- raise(ArgumentError, "Table #{table_name} is enabled. Disable it first before altering.") if enabled?(table_name)
+ # raise(ArgumentError, "Table #{table_name} is enabled. Disable it first before altering.") if enabled?(table_name)
{code}
... including the comment.
You should log the exception... add it as arg to below:
{code}
+ } catch (KeeperException e) {
+ LOG.warn("Instant schema change failed for table " + tableName );
+ }
{code}
You do this:
{code}
+ this.schemaChangeTracker = new MasterSchemaChangeTracker(getZooKeeper(),
+ this.zooKeeper.schemaZNode, this);
{code}
Do you have to pass schema znode? Can you not ask the object returned by getZooKeeper for schemaZNode so only need to have two args in this method?
Whats this?
{code}
+ public void checkTableModifiable(final byte [] tableName,
+ EventHandler.EventType eventType)
throws IOException {
+ preCheckTableModifiable(tableName);
+ if (!eventType.isSchemaChangeEvent()) {
+ if (!getAssignmentManager().getZKTable().
+ isDisabledTable(Bytes.toString(tableName))) {
+ throw new TableNotDisabledException(tableName);
+ }
+ }
+ }
{code}
Whats the is disabled check for? We need this any more? Oh, I see... its needed if its NOT a schema change event.
Javadoc says '+ * @return true if region is opened successfully with new schema changes.' but method returns void.
Does this need to be synchronized (you syncrhonize on it later in a different method):
+ for (HRegion hRegion : onlineRegionsOfTable) {
If a new region for this table comes in meantime, it'll be ok? It'll find new schema?
What happens if master decides to balance a region at this time? Or disable it? While reopenRegions is running?
What Ted says on delete cf.
Can you look in zk or something rather than wait on a timer before moving on?
{code}
+ // Take a mini nap for changes to take effect.
+ try {
+ Thread.sleep(200);
+ } catch (InterruptedException e) {
+ }
{code}
Remove the '*' from log messages. You'll start an arms race where every log message will try and add a new type of flashing glyph so it can stand out from the crowd.
This patch looks great.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142639#comment-13142639 ]
Hadoop QA commented on HBASE-4213:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502012/4213-Nov-2-2011_patch_.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -161 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 52 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/144//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/144//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/144//console
This message is automatically generated.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154882#comment-13154882 ]
ramkrishna.s.vasudevan commented on HBASE-4213:
-----------------------------------------------
@Ted
One very small comment. Javadoc for MasterSchemaChangeTracker is not found. Can be added as part of some related fix.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Attachment: 4213-trunk-v5.txt
Created InstantSchemaChangeTestBase and
separated some tests into TestInstantSchemaChangeSplit
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247582#comment-13247582 ]
Hudson commented on HBASE-4213:
-------------------------------
Integrated in HBase-TRUNK #2714 (See [https://builds.apache.org/job/HBase-TRUNK/2714/])
HBASE-5715 Revert 'Instant schema alter' for now, HBASE-4213 (Revision 1310012)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFamilyHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterSchemaChangeTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/SchemaChangeTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
* /hbase/trunk/src/main/resources/hbase-default.xml
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/InstantSchemaChangeTestBase.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChange.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeFailover.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestInstantSchemaChangeSplit.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v7.txt, 4213-trunk.txt, 4213-v9.txt, 4213.addendum, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch, schema-update.png
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4213:
--------------------------
Status: Patch Available (was: Open)
Patch testing v5.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.94.0
>
> Attachments: 4213-0.92.txt, 4213-0.92.v2, 4213-0.92.v4, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-trunk-v2.txt, 4213-trunk-v3.txt, 4213-trunk-v4.txt, 4213-trunk-v5.txt, 4213-trunk.txt, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch
This is a delta patch and goes on top of the latest 0.90.2 patch that is already uploaded.
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4213) Support for fault tolerant, instant
schema updates with out master's intervention (i.e with out enable/disable
and bulk assign/unassign) through ZK.
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Subbu M Iyer updated HBASE-4213:
--------------------------------
Attachment: 4213-V8-Support_instant_schema_changes_through_ZK.patch
> Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: 4213-Instant_Schema_change_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support instant schema updates with
out master's intervention (i.e with out enable/disable and bulk
assign/unassign)
Posted by "Subbu M Iyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086745#comment-13086745 ]
Subbu M Iyer commented on HBASE-4213:
-------------------------------------
Ted,
Yes. DELETE_FAMILY also needs to be added. Will take care of that.
>> You meant recursively deleting the children of node /hbase/schema/<table name>, right ?
Yes.
>> For region server which joins the cluster after the creation of /hbase/schema/<table name>, it should be able to find out that it already reads the most recent HTD. Does it create a child node under /hbase/schema/<table name> ?
The newly joined RS will not create a child under /hbase/schema/<table name> as it will not be processing the schema change event.
> Support instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4213
> URL: https://issues.apache.org/jira/browse/HBASE-4213
> Project: HBase
> Issue Type: Improvement
> Reporter: Subbu M Iyer
> Assignee: Subbu M Iyer
> Fix For: 0.92.0
>
> Attachments: HBASE-4213-Instant_schema_change.patch
>
>
> This Jira is a slight variation in approach to what is being done as part of
> https://issues.apache.org/jira/browse/HBASE-1730
> Support instant schema updates such as Modify Table, Add Column, Modify Column operations:
> 1. With out enable/disabling the table.
> 2. With out bulk unassign/assign of regions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira