You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2011/04/15 04:42:05 UTC
[jira] [Created] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Enhance MasterCoprocessorHost to include notification of balancing of each region
---------------------------------------------------------------------------------
Key: HBASE-3786
URL: https://issues.apache.org/jira/browse/HBASE-3786
Project: HBase
Issue Type: Improvement
Components: coprocessors
Affects Versions: 0.90.2
Reporter: Ted Yu
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020299#comment-13020299 ]
Jason Rutherglen commented on HBASE-3786:
-----------------------------------------
I'm not sure the location of the previous region's necessary?
Ideally we'd simply notify the NameNode to replicate the existing index files to the new RegionServer/DataNode, however that doesn't seem possible today?
Stack and I discussed performing a partial/full Lucene index optimize while 'copying' the existing index files to the local DataNode (because the merge'll be initiated and executed on the new RS). Does that sound tenable? There'll be a cost to the CPU due to the index merging however it'll be single threaded.
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020243#comment-13020243 ]
stack commented on HBASE-3786:
------------------------------
I think that would work for the fellow who wants it. Let me ask him Gary.
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020356#comment-13020356 ]
Andrew Purtell commented on HBASE-3786:
---------------------------------------
This issue may not have the appropriate scope for it, but I do think we should provide hooks into the balancer function.
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455389#comment-13455389 ]
Lars Hofhansl commented on HBASE-3786:
--------------------------------------
I'm a bit fuzzy as to what the current state is. In MasterObserver I find {pre|post}Balance, but those are not per region, so I guess they do not provide what is being asked here...?
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020415#comment-13020415 ]
Jason Rutherglen commented on HBASE-3786:
-----------------------------------------
> This issue may not have the appropriate scope for it, but I do think we should provide hooks into the balancer function
I thought that existed today? Yes it'd be great to log the query times for a given region and balance based on system wide search system statistics. I think it'll be useful (and simple) to build a query logging system (eg, HBase table) that automatically logs the search queries into HBase for later analysis, generating the statistics useful for balancing.
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020277#comment-13020277 ]
Ted Yu commented on HBASE-3786:
-------------------------------
I looked at preOpen():
{code}
public void preOpen(final ObserverContext<RegionCoprocessorEnvironment> c);
{code}
It is not clear to me how we can obtain the original region server where the region came from.
I think semantically, that information shouldn't be provided here.
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3786) Enhance MasterCoprocessorHost to
include notification of balancing of each region
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020223#comment-13020223 ]
Gary Helmling commented on HBASE-3786:
--------------------------------------
Does this need to be handled on the region balancing end? Could it be performed in the RegionObserver.preOpen() hook instead?
> Enhance MasterCoprocessorHost to include notification of balancing of each region
> ---------------------------------------------------------------------------------
>
> Key: HBASE-3786
> URL: https://issues.apache.org/jira/browse/HBASE-3786
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Affects Versions: 0.90.2
> Reporter: Ted Yu
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira