You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Zhihong Yu (Created) (JIRA)" <ji...@apache.org> on 2012/01/23 18:20:40 UTC
[jira] [Created] (HBASE-5258) Move coprocessors set out of
RegionLoad
Move coprocessors set out of RegionLoad
---------------------------------------
Key: HBASE-5258
URL: https://issues.apache.org/jira/browse/HBASE-5258
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492669#comment-13492669 ]
stack commented on HBASE-5258:
------------------------------
[~sershe] Currently the RegionLoad in ServerLoad is badly overloaded carrying both load and attributes. Historically it was just easiest adding attributes to the message the RS was passing the Master on a heartbeat. Now we are pb'ing everywhere, we can entertain breaking load and attributes apart. Or, as below outlines, we'd not put load nor attributes on the heartbeat at all.
Load is currently sent by the regionservers to the master on the old heartbeat channel that has been around since the early days. The only reason the master gets the load is so its UI can give a bit of a clue as to what is going on clusterwise. The load the regionservers put on the heartbeat is mostly a duplicate of what the regionserver emits over its metrics channel. It would be good if we did not duplicate this load composition. It would be good if instead the master listened to the regionserver metrics emissions rather than get load off the heartbeat (or, RS wrote their load/metrics to a table and master read that whenever it wanted to show a page on cluster state). That would make it so the heartbeat now only carries attributes. But maybe it should not carry this even. Most of the time the attributes do not change so it is almost as though they should be written once and then the master would consult them when it needs them. The master could, say, write them to zk into the body of the RS ephemeral znode. Master can read them when it joins the cluster or when they are updated by the RS.
So, then the heartbeat would seem to have no purpose given zk client in the RS is already heartbeating so we could drop it.
But maybe we shouldn't and rather, we should rethread it. FB are starting to go down the path where they are starting to rely on this RS heartbeat rather than the ephemeral node in zk and letting master make the call on whether a RS should be let go or not. They are tending this way because the zk ephemeral node is dumb -- no logic other than timeout -- and because if master is allowed say what is dead and what is alive, it has the opportunity of applying rules; e.g. if all regionservers in the cluster are unresponsive all at the same time, its probably a network issue so hold tight till the network comes back (as opposed to zk which would just expire all sessions and all of the cluster would go down).
Hope this helps.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191589#comment-13191589 ]
Andrew Purtell commented on HBASE-5258:
---------------------------------------
bq. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
No doubt it is redundant to have each region report a coprocessor given how the framework currently works: All regions for a table will have an identical set of coprocessors loaded, or there is something bad happening.
bq. Would HServerLoad be a better place for this set ?
I have no major objection.
However, maybe we want a way to know if something bad happened on a region and a coprocessor on it went away? One could comb logs but that is hardly a convenient way to get online state.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5258:
------------------------------
Description:
When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
Would HServerLoad be a better place for this set ?
If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
was:
When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
Would HServerLoad be a better place for this set ?
Summary: Move coprocessors set out of RegionLoad (was: Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad)
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Eugene Koontz (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191443#comment-13191443 ]
Eugene Koontz commented on HBASE-5258:
--------------------------------------
Hi Ted,
Do you have an estimate for how much network traffic or heap footprint that this would save?
Just curious, not an objection.
-Eugene
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192338#comment-13192338 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
I was looking for HBASE-4660 yesterday, thanks for finding it.
Please work on this JIRA before tackling webuiport.
> Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191639#comment-13191639 ]
Andrew Purtell commented on HBASE-5258:
---------------------------------------
bq. for a mis-behaving coprocessor we either remove the buggy coprocessor
... from the coprocessor host for the given region (in the case of RegionCoprocessorHost) only...
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511077#comment-13511077 ]
Andrew Purtell commented on HBASE-5258:
---------------------------------------
bq. I don't see getting coprocessors from RegionLoad actually being used anywhere.
+1
I searched for users of RegionLoad#getCoprocessors and didn't find any either.
Originally we put this in so the loaded coprocessors on a region would show up next to other region information in the master UI. Unfortunately the result was not that useful, just clutter.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191594#comment-13191594 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
To my knowledge, for a mis-behaving coprocessor we either remove the buggy coprocessor or abort.
I wonder what scenario would lead to imbalanced coprocessors on a region.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198298#comment-13198298 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
During meeting with Stack, Liyin and Karthick today, Stack mentioned that embedding coprocessor information in RegionLoad is not the right thing to do.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5258) Move coprocessors set out of
RegionLoad, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5258:
------------------------------
Priority: Critical (was: Major)
Raising priority as this task is about making the correct design choices.
> Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Sergey Shelukhin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493441#comment-13493441 ]
Sergey Shelukhin commented on HBASE-5258:
-----------------------------------------
There may be potential for using info about region server load that is more realtime/critical than metrics channel, in master, for balancing/etc. Probably not important short term.
W.r.t. zk - why cannot master read zk nodes and apply rules too? ZK nodes may need to be made smarter for this purpose.
Assuming RSs will have to talk to ZK anyway, for large clusters it will avoid extra all-to-one communication. Although master reading ZK to make decisions may introduce additional delay, especially if master just watches the nodes and builds up internal state, instead of querying entire state periodically.
Regardless, does having different coprocessors on the regions of the same table make sense? If user has logic relying on coprocessors for some data querying or correctness it seems dangerous.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5258) Move coprocessors set out of
RegionLoad, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5258:
------------------------------
Summary: Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad (was: Move coprocessors set out of RegionLoad)
> Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Sergey Shelukhin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492066#comment-13492066 ]
Sergey Shelukhin commented on HBASE-5258:
-----------------------------------------
hi. is there any progress to be made on this?
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
Posted by "Eugene Koontz (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192336#comment-13192336 ]
Eugene Koontz commented on HBASE-5258:
--------------------------------------
@Zhihong, I think it should be ok to raise coprocessor reporting out of HServerLoad.HRegionLoad to just HServerLoad.
How do you see this as it relates to HBASE-4660 ? Does this fix make it (HBASE-4660) easier?
> Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191646#comment-13191646 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
I agree with the last comment @ 23/Jan/12 22:58
My understanding of the feature is that user should validate coprocessor by choosing the Abort policy for buggy coprocessor in pre-deployment stage. In production, the chance of buggy coprocessor dropping from individual region(s) should be low.
The ability of querying imbalanced coprocessors on a region server should be on-demand feature.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191862#comment-13191862 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
@Andrew, @Eugene:
Do you agree with the proposal as outlined in the title of this JIRA ?
> Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191461#comment-13191461 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
Since each coprocessor is represented by a string, the potential savings can be considerable, especially if many regions are hosted on each region server.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Sergey Shelukhin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510958#comment-13510958 ]
Sergey Shelukhin commented on HBASE-5258:
-----------------------------------------
Hi. Is there any objections to removing them for now? I don't see getting coprocessors from RegionLoad actually being used anywhere.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of
RegionLoad
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191651#comment-13191651 ]
Zhihong Yu commented on HBASE-5258:
-----------------------------------
Since the combination of coprocessors on a region server is limited, I was suggesting that the report of uneven coprocessor presence be embedded in HServerLoad.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Zhihong Yu
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira