[jira] [Comment Edited] (HBASE-6721) RegionServer Group based Assignment

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876096#comment-14876096 ] 

Andrew Purtell edited comment on HBASE-6721 at 9/18/15 6:31 PM:
----------------------------------------------------------------

bq. Andrew Purtell The new patch is based off of current master. Should we just replace the current branch with a new branch to go with the new patch?

I would say so if you want it. 

bq. Also was wondering since this is now cp-based. Do we still need a branch?

Up to you. We can rebase or delete, either way let me know.  

Some issues with the current security integration. Coprocessors can't call into the internals of other coprocessors. I understand why this was done, but we can't have it. Coprocessors calling into the internals of other coprocessors, this is a non-negotiable point for the sake of sanity in maintenance of separate optional extensions. It's a catch-22 imposed on this change by the requirement it be a coprocessor only implementation.

What I would suggest is introduce into the MasterObserver API hooks for the group admin APIs. Let the implementation of the group admin APIs and the authoritative security decisions both be separate mix-ins provided by different coprocessors. There needs to be common plumbing for the two. That belongs in MasterObserver. The plumbing could look like:
- MasterObserver support for pre/post group admin API action hooks
- In GroupAdminEndpoint, get the coprocessor host with getMasterCoprocessorHost()
- Invoke the public (technically, LimitedPrivate(COPROC)) APIs for pre/post group admin API actions.
- AccessController implements the new MasterObserver APIs to provide security for the group admin APIs.

This is much more in spirit with current interfaces and audience scoping. It decouples GroupAdminEndpoint from AccessController. (If the AC is not installed, no harm, no NPEs, no security checking (by intention), it's all good.) It also addresses concerns about zero impact in the default case. Those upcalls will never be made unless the GroupAdminEndpoint is installed.


was (Author: apurtell):
bq. Andrew Purtell The new patch is based off of current master. Should we just replace the current branch with a new branch to go with the new patch?

I would say so if you want it. 

bq. Also was wondering since this is now cp-based. Do we still need a branch?

Up to you. We can rebase or delete, either way let me know.  

Some issues with the current security integration. Coprocessors can't call into the internals of other coprocessors. I understand why this was done, but we can't have it. Coprocessors calling into the internals of other coprocessors, this is a non-negotiable point for the sake of sanity in maintenance of separate optional extensions. It's a catch-22 imposed on this change by the requirement it be a coprocessor only implementation.

What I would suggest is introduce into the MasterObserver API hooks for the group admin APIs. Let the implementation of the group admin APIs and the authoritative security decisions both be separate mix-ins provided by different coprocessors. There needs to be common plumbing for the two. That belongs in MasterObserver. The plumbing could look like:
- MasterObserver support for pre/post group admin API action hooks
- In GroupAdminEndpoint, get the coprocessor host with getMasterCoprocessorHost()
- Invoke the public (technically, LimitedPrivate(COPROC)) APIs for pre/post group admin API actions.

This is much more in spirit with current interfaces and audience scoping. It also addresses concerns about zero impact in the default case. Those upcalls will never be made unless the GroupAdminEndpoint is installed.

> RegionServer Group based Assignment
> -----------------------------------
>
>                 Key: HBASE-6721
>                 URL: https://issues.apache.org/jira/browse/HBASE-6721
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Francis Liu
>            Assignee: Francis Liu
>              Labels: hbase-6721
>         Attachments: 6721-master-webUI.patch, HBASE-6721 GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, immediateAssignments Sequence Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving out regions from a number of different tables owned by various client applications. Being able to group a subset of running RegionServers and assign specific tables to it, provides a client application a level of isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of RegionServer groups and assigns tables to region servers based on groupings. Load balancing will occur on a per group basis as well. 
> This is essentially a simplification of the approach taken in HBASE-4120. See attached document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)