You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/09/12 07:20:57 UTC
[jira] Created: (HBASE-1832) Faster enable/disable/delete
Faster enable/disable/delete
----------------------------
Key: HBASE-1832
URL: https://issues.apache.org/jira/browse/HBASE-1832
Project: Hadoop HBase
Issue Type: Improvement
Reporter: stack
Fix For: 0.20.1
The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762364#action_12762364 ]
Jonathan Gray commented on HBASE-1832:
--------------------------------------
Have not tested, but reviewed patch closely and it looks good. Are there any good enable/disable/delete tests out there?
+1 if stack is confident it works
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1, 0.21.0
>
> Attachments: fasterenabledisable-v3.patch
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1832:
-------------------------
Fix Version/s: 0.20.1
Bringing back into 0.20.1 for another look... An organization embedding hbase has hundreds of tests to run. If this process ran smoother, it would save them a bunch of headache.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1832:
-------------------------
Attachment: fasterenabledisable-v3.patch
This is a bit of miscellany. Adds batching of meta scans so should be faster figuring if table enabled//disabled, etc. We were checking at least one time too many enable/disable state. Fixed other minor items like the rough count at head of master ui so now its more likely close to actual region count. This is probably good enough for 0.20.1 methinks.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
> Attachments: fasterenabledisable-v3.patch
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754487#action_12754487 ]
stack commented on HBASE-1832:
------------------------------
Could try HTable.getHRegionInfos(). This returns HRIs for a table. The implementation of getHRIs doesn't do bulk scanning. Fix. Then, bulk change the HRIs and then bulk write them. Pity there is no bulk delete....
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-1832:
----------------------------
Assignee: stack
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Fix For: 0.20.1, 0.21.0
>
> Attachments: fasterenabledisable-v3.patch
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1832:
-------------------------
Fix Version/s: 0.21.0
Status: Patch Available (was: Open)
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1, 0.21.0
>
> Attachments: fasterenabledisable-v3.patch
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759921#action_12759921 ]
stack commented on HBASE-1832:
------------------------------
Can I do anything about HBASE-1636 too?
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762361#action_12762361 ]
Jean-Daniel Cryans commented on HBASE-1832:
-------------------------------------------
Code looks good. Tried the patch, works well but I guess this won't make a difference for < 50 regions table. +1 for commit
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1, 0.21.0
>
> Attachments: fasterenabledisable-v3.patch
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1832:
-------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Resolving for now. This will all change in 0.21 (instantaneous?).
Thanks for the reviews lads.
Applied branch and trunk.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1, 0.21.0
>
> Attachments: fasterenabledisable-v3.patch
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754603#action_12754603 ]
stack commented on HBASE-1832:
------------------------------
A 1600 region table the first time through times out when I try to disable it but then subsequent enable/deletes take < 18 seconds and even < 10 if I do it enough. The delete runs just as fast. Moving out of 0.20.1 till learn more.
These table transitions are done internally by the master. Should just be a client-side action. There ain't nothing special that run from the master adds.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "Andrei Dragomir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761512#action_12761512 ]
Andrei Dragomir commented on HBASE-1832:
----------------------------------------
As I see in the HBaseAdmin, all the table modification operations go to the master, and everything there is synchronized on the RegionManager on the HMaster. Can we change these semantics ? What happens if client code disables the table while another client enables it ?
The second observation is that, there are about three or four different patterns in the code to scan the meta / get a table's regions (MetaScanner, MetaUtils, direct HTable(meta), etc). I think it would be better to have only one pattern.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761524#action_12761524 ]
stack commented on HBASE-1832:
------------------------------
@Andrei Thanks for taking a look. Yes, the way things currently work is way broke. Your second observation above particularly bugs me. Master is being rewritten for 0.21 to move state transitions to go via zk (and to clean up the hairball master has become). This issue is just about trying to figure if I can make enable/disable run faster in 0.20.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1832) Faster enable/disable/delete
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1832:
-------------------------
Fix Version/s: (was: 0.20.1)
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1832) Faster enable/disable/delete
Posted by "Andrei Dragomir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761552#action_12761552 ]
Andrei Dragomir commented on HBASE-1832:
----------------------------------------
I see. I think that the simplest solution then is to have something like bulk scanner in TableOperations, instead of single row server.next call that we use right now. For large tables with lots of regions, I think that will improve the performance a bit.
> Faster enable/disable/delete
> ----------------------------
>
> Key: HBASE-1832
> URL: https://issues.apache.org/jira/browse/HBASE-1832
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.20.1
>
>
> The enable/disable/delete is slow. Looking at code, its heavyweight. It doesn't do bulk scanning nor bulk writing. Try doing some code client side that does bulk scan and bulk puts. It might run faster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.