You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by xiaolong ran <ra...@gmail.com> on 2019/10/30 11:49:37 UTC
PIP 49: Permission levels and inheritance
Hello all:
When using pulsar-admin, I found that the current permission verification mechanism has some problems.
We are starting a proposal about permission levels and inheritance in Apache Pulsar.
The proposal is in:
https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance <https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance>
To ensure compatibility, these changes will be made in v4's admin api. About the v4 admin API, see:
https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
Looking forward to any feedback.
Re: PIP 49: Permission levels and inheritance
Posted by xiaolong ran <ra...@gmail.com>.
Hello all committers:
I reorganize the relevant content of this PIP. PTAL again.
https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance <https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance>
Thanks Matteo and Joe feedback.
Best Regards,
Xiaolong Ran
> 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
>
> Hello all:
>
> When using pulsar-admin, I found that the current permission verification mechanism has some problems.
>
> We are starting a proposal about permission levels and inheritance in Apache Pulsar.
>
> The proposal is in:
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance <https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance>
>
> To ensure compatibility, these changes will be made in v4's admin api. About the v4 admin API, see:
>
> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>
> Looking forward to any feedback.
Re: PIP 49: Permission levels and inheritance
Posted by Addison Higham <ad...@gmail.com>.
I can't speak to if these are all the correct level of access (thought it
looks sane to me) but I am very excited about more granular permissions.
I would also say that I I think even an initial simpler implementation of
this with just the "namespace admin" would go a long way to improving it if
for some reason there is more complexity of realizing the whole plan of
this PIP.
This is something that we would likely have been tackling in the coming
months, so happy to help wherever possible assuming that this PIP gets
approved.
On Wed, Nov 6, 2019 at 3:14 AM xiaolong ran <ra...@gmail.com>
wrote:
> Hello all committers:
>
> About this PIP, do you have any other good ideas or propose.
>
>
> --
>
> Thanks
> Xioalong Ran
>
>
> > 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
> >
> > Hello all:
> >
> > When using pulsar-admin, I found that the current permission
> verification mechanism has some problems.
> >
> > We are starting a proposal about permission levels and inheritance in
> Apache Pulsar.
> >
> > The proposal is in:
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> <
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> >
> >
> > To ensure compatibility, these changes will be made in v4's admin api.
> About the v4 admin API, see:
> >
> > https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <
> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> >
> > Looking forward to any feedback.
>
>
Re: PIP 49: Permission levels and inheritance
Posted by Addison Higham <ad...@gmail.com>.
Thanks for the response Joe.
I think I understand where you are coming, but as I look at the current
capabilities for putting limits on a namespace (backlog quotas, publish and
subscribe rate limits, ttl, etc) there does appear to be a fair number of
controls in place today that do allow for *some* control over what can be
done within a given namespace such that I can reasonably delegate *some*
responsibility. Notice that in the permissions I listed, I gave very little
control to "namespace admins" for any namespace polices, even with just
topic creation for namespace admins, I think that would be very beneficial
to have that capability. I understand that usage of these resources aren't
hierarchical (i.e. inherited from a tenant) and therefore I am still
placing a fair amount of trust on tenant admins to set those values
reasonably (that can have global impact) and to then delegate that
permission to namespace admins. However, I would put forth that in many
situations, it may be much less effort to monitor and alert on these sort
of situations of using too many resources than it is to build external
automation to "safely" create and manage resources or use a tenant strategy
that doesn't map well to my business (where the visibility of multiple
components that make up a product is very natural mapping to a namespace
and a tenant)
In other words, I fully understand it isn't necessarily ideal, but I think
it makes sense for me to be able to make those trade-offs as I implement
Pulsar. I am positive this will improve over time using some of the longer
term ideas discussed in this thread, but I would also advocate that
iteration and a bit more flexibility today would have some real utility
(with some trade-offs that need to be understood by operators of Pulsar).
Pivoting a bit, I am curious to hear any opinions regarding a more "full
featured" approach to defining policies, such as a policy language to allow
arbitrary sets of permissions. I think that obviously gives the most
flexibility, but does shift a large amount of complexity onto the user to
figure out what policies are safe.
Once again, thanks for the discussion and happy to add anything else if it
is useful in helping figure out the right answer for Pulsar as well as
seeing how we can help out in making it happen :)
Addison
On Tue, Nov 12, 2019 at 12:02 AM Joe F <jo...@apache.org> wrote:
> >Is there any appetite for considering a "namespace admin"?
>
> I think this is where there is an issue. As Sijie aptly pointed out, this
> namespace "admin", as a concept, does not exist in Pulsar for resource
> allocation. It will upend the whole design of how resources are managed
> in Pulsar. While we started this PIP as a discussion on permissions, it is
> not possible do many of these asks as simply permissions. Implicitly, it's
> reworking the whole resource management model in Pulsar. And I think that
> if we are attempting to re-work resource management, it should be
> discussed as such, and not occur as an unnoticed side effect of attempting
> to manage permissions .
>
> >I think that set of permissions (with some deletions or adjustments) could
> go a long way to making the current permissions model work a fair bit
> better for orgs trying to deploy pulsar.
>
> Ultimately, when you give someone access to a resource, they are going to
> use it. And that use has a cost. Who pays the cost, and how is it managed
> and accounted? Power to manage access is power to spend. To what level do
> we do resource controls?
>
> I am a little lost on the idea of sub-tenants (namespace admins). This is
> the point where I would think that the system model would be better off
> having those namespace as first-class tenants, and not as namespaces.
> Pulsar is, not as it is, designed to have sub-tenants.
>
> Joe
>
>
>
>
> On Mon, Nov 11, 2019 at 9:35 AM Addison Higham <ad...@gmail.com> wrote:
>
> > Is there any appetite for considering a "namespace admin"? We could
> really
> > have use of this as currently we have to give more people than we would
> > want tenant admin to ease administration, when many users basically just
> > need permissions to create topics and grant permissions to those topics.
> >
> > From my reading of this chain, it is obvious that it probably won't be
> > given a ton of permissions and it may be complex to add, but it seems
> like
> > starting with a conservative approach of permissions for a namespace
> admin
> > initially could be done fairly quickly and safely.
> >
> > I would say the following could make a a lot of sense to be in a
> namespace
> > admin permissions:
> >
> > Topics
> > - compaction/compaction status
> > - offload/offload-status
> > - create/delete non partitioned topics
> > - create/delete partitioned topics
> > - list/grant/revoke permissions
> > - create/delete/peek/seek subscriptions
> > - stats calls
> >
> > Namespaces
> > - list/grant/revoke permissions
> > - set/get clusters (maybe not?)
> > - get/set ttl
> > - get/set persistence (maybe not?)
> >
> > I think that set of permissions (with some deletions or adjustments)
> could
> > go a long way to making the current permissions model work a fair bit
> > better for orgs trying to deploy pulsar.
> >
> > Long term, it feels like the correct answer to this problem is probably
> to
> > introduce some sort of policy language with a role being tied to some
> > policy that can grant a wide range of individual permissions, with the
> > existing superuser/tenant-admin/etc being redefined in terms of some of
> > these policies (perhaps using something like https://casbin.org/en/). If
> > it
> > seems like the focus should be on the more general solution, then I am
> fine
> > leaving this for now, but this will be something we need to tackle in the
> > next 6 months or so for my org, so I would love to see any sort of
> > direction of where things should be heading so we can plan to that and
> > perhaps help where possible!
> >
> >
> >
> > On Mon, Nov 11, 2019 at 2:24 AM xiaolong ran <ra...@gmail.com>
> > wrote:
> >
> > > Hello everyone:
> > >
> > > I reorganize the relevant content of this PIP. PTAL again.
> > >
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > > <
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > > >
> > >
> > > In this PIP, I only fix the unreasonable permissions in the current
> > > command and shown in bold.
> > > About specific permission of command, if there are different opinions
> > > about this, pls let me know, thanks.
> > >
> > > --
> > >
> > > Thanks
> > > Xiaolong Ran
> > >
> > >
> > > > 在 2019年11月8日，下午4:41，xiaolong ran <ra...@gmail.com> 写道：
> > > >
> > > > Thanks Sijie, Joe F and Addison Highham feedback.
> > > >
> > > > As Joe said:
> > > >
> > > > > There are no simple answers here other than to understand the
> effect
> > > of the command.
> > > >
> > > > So in here, we can use sijie proposal, we only need to fix the
> > > unreasonable permissions in the current command.
> > > >
> > > > I will reorganize this PIP, what do you think about this?
> > > >
> > > > --
> > > >
> > > > Thanks
> > > > Xiaolong Ran
> > > >
> > > >
> > > >> 在 2019年11月8日，下午3:59，Sijie Guo <guosijie@gmail.com <mailto:
> > > guosijie@gmail.com>> 写道：
> > > >>
> > > >> If the problem is complex enough, can we first reduce the scope of
> > this
> > > PIP
> > > >> to focus on fixing the permissions of individual commands (e.g.
> > schemas
> > > >> don't require super-user permissions)?
> > > >> As you said, there is no simple answers. We can go through the
> > commands
> > > one
> > > >> by one and come up a table of commands and their permissions and
> > > document
> > > >> it.
> > > >> It can help users understand how the permissions work for each
> > command.
> > > >>
> > > >> We can revisit a larger scope till we add the capability of
> allocation
> > > >> resources at different levels.
> > > >>
> > > >> Does that make sense?
> > > >>
> > > >> - Sijie
> > > >>
> > > >>
> > > >> On Fri, Nov 8, 2019 at 3:22 PM Joe F <joef@apache.org <mailto:
> > > joef@apache.org>> wrote:
> > > >>
> > > >>> There are no simple answers here other than to understand the
> effect
> > > of the
> > > >>> command.
> > > >>>
> > > >>> The resource limit/control command always addresses an object. But
> > > giving
> > > >>> control of that command to the object owner, just because the
> command
> > > >>> addresses an object, will break all controls about resource
> > > utilization in
> > > >>> a shared system. A filesystem like model may look simple and
> elegant
> > > but
> > > >>> is fraught with all kind of unintended consequences if implemented
> in
> > > >>> Pulsar (Even in the filesystem rm, mv and chown, while all
> > addressing
> > > a
> > > >>> file, treats the file owner differently)
> > > >>>
> > > >>> This is a complex, and complicated effort, and my concern is that
> > > this PIP
> > > >>> does not even come close to the scope of the work that is
> required.
> > > Some
> > > >>> of them may be impossible to do (as allocating certain resources to
> > > >>> namespace as compared to tenant) till such capabilities are added
> to
> > > the
> > > >>> system. And some begs fundamental questions - when you start
> > managing
> > > a
> > > >>> namespace like a tenant, does it even make sense to have that
> > > namespace?
> > > >>> Should't that namespace be a tenant in the first place? Why
> have a
> > > >>> namespace admin at all?
> > > >>>
> > > >>> Joe
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wave4dave@comcast.net
> > > <ma...@comcast.net>> wrote:
> > > >>>
> > > >>>> I’m not diving in but thinking about the logical implication of
> this
> > > >>>> dichotomy. For any object’s attributes some ought to be controlled
> > by
> > > >>>> object level permissions and others by sysadmin permissions. How
> can
> > > >>>> developers tell?
> > > >>>>
> > > >>>> Best Regards,
> > > >>>> Dave
> > > >>>>
> > > >>>>
> > > >>>> Sent from my iPhone
> > > >>>>
> > > >>>>> On Nov 7, 2019, at 8:02 PM, Joe F <joef@apache.org <mailto:
> > > joef@apache.org>> wrote:
> > > >>>>>
> > > >>>>> This proposal has the same issues as the previous one . In
> general
> > > it
> > > >>> is
> > > >>>>> not correct to think of commands as being controlled by owner of
> > the
> > > >>>>> object on which the command operates. Eg: It will be an error to
> > > >>> assing
> > > >>>>> control of all namespace commands to the namespace owner
> > > >>>>>
> > > >>>>> For eg: set subscription-dispatch-rate operates on a
> subscription
> > > rate
> > > >>>> and
> > > >>>>> set-max-producers-per-topic These operate on namespaces. A naive
> > > >>> approach
> > > >>>>> is to think that this would be controlled by the namespace owner.
> > > But
> > > >>>>> essentially both these are resource management operations. That
> > > should
> > > >>>> be
> > > >>>>> controlled by the system admin, unless you want to deal with
> every
> > > >>>>> namespace owner having the power to destroy your SLAs fo the
> > cluster.
> > > >>>>>
> > > >>>>> I just picked 2 examples to indicate the drawback of approaching
> > > >>>>> permissions in the proposed manner.
> > > >>>>>
> > > >>>>> There are many such cases in the proposal, too many to list out
> > > here. I
> > > >>>>> suggest completely ditching this approach of equating objects
> with
> > > >>> admin
> > > >>>>> capability of object owner
> > > >>>>>
> > > >>>>> Joe
> > > >>>>>
> > > >>>>>
> > > >>>>>> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <
> > > ranxiaolong716@gmail.com <ma...@gmail.com>
> > > >>>>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>> Hello all committers:
> > > >>>>>>
> > > >>>>>> About this PIP, do you have any other good ideas or propose.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>>
> > > >>>>>> Thanks
> > > >>>>>> Xioalong Ran
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>>> 在 2019年10月30日，下午7:49，xiaolong ran <ranxiaolong716@gmail.com
> > > <ma...@gmail.com>> 写道：
> > > >>>>>>>
> > > >>>>>>> Hello all:
> > > >>>>>>>
> > > >>>>>>> When using pulsar-admin, I found that the current permission
> > > >>>>>> verification mechanism has some problems.
> > > >>>>>>>
> > > >>>>>>> We are starting a proposal about permission levels and
> > inheritance
> > > in
> > > >>>>>> Apache Pulsar.
> > > >>>>>>>
> > > >>>>>>> The proposal is in:
> > > >>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > > <
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > > >
> > > >>>>>> <
> > > >>>>>>
> > > >>>>
> > > >>>
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> To ensure compatibility, these changes will be made in v4's
> admin
> > > >>> api.
> > > >>>>>> About the v4 admin API, see:
> > > >>>>>>>
> > > >>>>>>>
> > > >>>
> > https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
> > > >>>> <
> > > >>>>>>
> > > https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> > > >>>>>>>
> > > >>>>>>> Looking forward to any feedback.
> > > >>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>>
> > > >>>
> > > >
> > >
> > >
> >
>
Re: PIP 49: Permission levels and inheritance
Posted by Joe F <jo...@apache.org>.
>Is there any appetite for considering a "namespace admin"?
I think this is where there is an issue. As Sijie aptly pointed out, this
namespace "admin", as a concept, does not exist in Pulsar for resource
allocation. It will upend the whole design of how resources are managed
in Pulsar. While we started this PIP as a discussion on permissions, it is
not possible do many of these asks as simply permissions. Implicitly, it's
reworking the whole resource management model in Pulsar. And I think that
if we are attempting to re-work resource management, it should be
discussed as such, and not occur as an unnoticed side effect of attempting
to manage permissions .
>I think that set of permissions (with some deletions or adjustments) could
go a long way to making the current permissions model work a fair bit
better for orgs trying to deploy pulsar.
Ultimately, when you give someone access to a resource, they are going to
use it. And that use has a cost. Who pays the cost, and how is it managed
and accounted? Power to manage access is power to spend. To what level do
we do resource controls?
I am a little lost on the idea of sub-tenants (namespace admins). This is
the point where I would think that the system model would be better off
having those namespace as first-class tenants, and not as namespaces.
Pulsar is, not as it is, designed to have sub-tenants.
Joe
On Mon, Nov 11, 2019 at 9:35 AM Addison Higham <ad...@gmail.com> wrote:
> Is there any appetite for considering a "namespace admin"? We could really
> have use of this as currently we have to give more people than we would
> want tenant admin to ease administration, when many users basically just
> need permissions to create topics and grant permissions to those topics.
>
> From my reading of this chain, it is obvious that it probably won't be
> given a ton of permissions and it may be complex to add, but it seems like
> starting with a conservative approach of permissions for a namespace admin
> initially could be done fairly quickly and safely.
>
> I would say the following could make a a lot of sense to be in a namespace
> admin permissions:
>
> Topics
> - compaction/compaction status
> - offload/offload-status
> - create/delete non partitioned topics
> - create/delete partitioned topics
> - list/grant/revoke permissions
> - create/delete/peek/seek subscriptions
> - stats calls
>
> Namespaces
> - list/grant/revoke permissions
> - set/get clusters (maybe not?)
> - get/set ttl
> - get/set persistence (maybe not?)
>
> I think that set of permissions (with some deletions or adjustments) could
> go a long way to making the current permissions model work a fair bit
> better for orgs trying to deploy pulsar.
>
> Long term, it feels like the correct answer to this problem is probably to
> introduce some sort of policy language with a role being tied to some
> policy that can grant a wide range of individual permissions, with the
> existing superuser/tenant-admin/etc being redefined in terms of some of
> these policies (perhaps using something like https://casbin.org/en/). If
> it
> seems like the focus should be on the more general solution, then I am fine
> leaving this for now, but this will be something we need to tackle in the
> next 6 months or so for my org, so I would love to see any sort of
> direction of where things should be heading so we can plan to that and
> perhaps help where possible!
>
>
>
> On Mon, Nov 11, 2019 at 2:24 AM xiaolong ran <ra...@gmail.com>
> wrote:
>
> > Hello everyone:
> >
> > I reorganize the relevant content of this PIP. PTAL again.
> >
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > <
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > >
> >
> > In this PIP, I only fix the unreasonable permissions in the current
> > command and shown in bold.
> > About specific permission of command, if there are different opinions
> > about this, pls let me know, thanks.
> >
> > --
> >
> > Thanks
> > Xiaolong Ran
> >
> >
> > > 在 2019年11月8日，下午4:41，xiaolong ran <ra...@gmail.com> 写道：
> > >
> > > Thanks Sijie, Joe F and Addison Highham feedback.
> > >
> > > As Joe said:
> > >
> > > > There are no simple answers here other than to understand the effect
> > of the command.
> > >
> > > So in here, we can use sijie proposal, we only need to fix the
> > unreasonable permissions in the current command.
> > >
> > > I will reorganize this PIP, what do you think about this?
> > >
> > > --
> > >
> > > Thanks
> > > Xiaolong Ran
> > >
> > >
> > >> 在 2019年11月8日，下午3:59，Sijie Guo <guosijie@gmail.com <mailto:
> > guosijie@gmail.com>> 写道：
> > >>
> > >> If the problem is complex enough, can we first reduce the scope of
> this
> > PIP
> > >> to focus on fixing the permissions of individual commands (e.g.
> schemas
> > >> don't require super-user permissions)?
> > >> As you said, there is no simple answers. We can go through the
> commands
> > one
> > >> by one and come up a table of commands and their permissions and
> > document
> > >> it.
> > >> It can help users understand how the permissions work for each
> command.
> > >>
> > >> We can revisit a larger scope till we add the capability of allocation
> > >> resources at different levels.
> > >>
> > >> Does that make sense?
> > >>
> > >> - Sijie
> > >>
> > >>
> > >> On Fri, Nov 8, 2019 at 3:22 PM Joe F <joef@apache.org <mailto:
> > joef@apache.org>> wrote:
> > >>
> > >>> There are no simple answers here other than to understand the effect
> > of the
> > >>> command.
> > >>>
> > >>> The resource limit/control command always addresses an object. But
> > giving
> > >>> control of that command to the object owner, just because the command
> > >>> addresses an object, will break all controls about resource
> > utilization in
> > >>> a shared system. A filesystem like model may look simple and elegant
> > but
> > >>> is fraught with all kind of unintended consequences if implemented in
> > >>> Pulsar (Even in the filesystem rm, mv and chown, while all
> addressing
> > a
> > >>> file, treats the file owner differently)
> > >>>
> > >>> This is a complex, and complicated effort, and my concern is that
> > this PIP
> > >>> does not even come close to the scope of the work that is required.
> > Some
> > >>> of them may be impossible to do (as allocating certain resources to
> > >>> namespace as compared to tenant) till such capabilities are added to
> > the
> > >>> system. And some begs fundamental questions - when you start
> managing
> > a
> > >>> namespace like a tenant, does it even make sense to have that
> > namespace?
> > >>> Should't that namespace be a tenant in the first place? Why have a
> > >>> namespace admin at all?
> > >>>
> > >>> Joe
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wave4dave@comcast.net
> > <ma...@comcast.net>> wrote:
> > >>>
> > >>>> I’m not diving in but thinking about the logical implication of this
> > >>>> dichotomy. For any object’s attributes some ought to be controlled
> by
> > >>>> object level permissions and others by sysadmin permissions. How can
> > >>>> developers tell?
> > >>>>
> > >>>> Best Regards,
> > >>>> Dave
> > >>>>
> > >>>>
> > >>>> Sent from my iPhone
> > >>>>
> > >>>>> On Nov 7, 2019, at 8:02 PM, Joe F <joef@apache.org <mailto:
> > joef@apache.org>> wrote:
> > >>>>>
> > >>>>> This proposal has the same issues as the previous one . In general
> > it
> > >>> is
> > >>>>> not correct to think of commands as being controlled by owner of
> the
> > >>>>> object on which the command operates. Eg: It will be an error to
> > >>> assing
> > >>>>> control of all namespace commands to the namespace owner
> > >>>>>
> > >>>>> For eg: set subscription-dispatch-rate operates on a subscription
> > rate
> > >>>> and
> > >>>>> set-max-producers-per-topic These operate on namespaces. A naive
> > >>> approach
> > >>>>> is to think that this would be controlled by the namespace owner.
> > But
> > >>>>> essentially both these are resource management operations. That
> > should
> > >>>> be
> > >>>>> controlled by the system admin, unless you want to deal with every
> > >>>>> namespace owner having the power to destroy your SLAs fo the
> cluster.
> > >>>>>
> > >>>>> I just picked 2 examples to indicate the drawback of approaching
> > >>>>> permissions in the proposed manner.
> > >>>>>
> > >>>>> There are many such cases in the proposal, too many to list out
> > here. I
> > >>>>> suggest completely ditching this approach of equating objects with
> > >>> admin
> > >>>>> capability of object owner
> > >>>>>
> > >>>>> Joe
> > >>>>>
> > >>>>>
> > >>>>>> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <
> > ranxiaolong716@gmail.com <ma...@gmail.com>
> > >>>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Hello all committers:
> > >>>>>>
> > >>>>>> About this PIP, do you have any other good ideas or propose.
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>>
> > >>>>>> Thanks
> > >>>>>> Xioalong Ran
> > >>>>>>
> > >>>>>>
> > >>>>>>>> 在 2019年10月30日，下午7:49，xiaolong ran <ranxiaolong716@gmail.com
> > <ma...@gmail.com>> 写道：
> > >>>>>>>
> > >>>>>>> Hello all:
> > >>>>>>>
> > >>>>>>> When using pulsar-admin, I found that the current permission
> > >>>>>> verification mechanism has some problems.
> > >>>>>>>
> > >>>>>>> We are starting a proposal about permission levels and
> inheritance
> > in
> > >>>>>> Apache Pulsar.
> > >>>>>>>
> > >>>>>>> The proposal is in:
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > <
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > >
> > >>>>>> <
> > >>>>>>
> > >>>>
> > >>>
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> To ensure compatibility, these changes will be made in v4's admin
> > >>> api.
> > >>>>>> About the v4 admin API, see:
> > >>>>>>>
> > >>>>>>>
> > >>>
> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
> > >>>> <
> > >>>>>>
> > https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> > >>>>>>>
> > >>>>>>> Looking forward to any feedback.
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>>
> > >>>
> > >
> >
> >
>
Re: PIP 49: Permission levels and inheritance
Posted by Addison Higham <ad...@gmail.com>.
Is there any appetite for considering a "namespace admin"? We could really
have use of this as currently we have to give more people than we would
want tenant admin to ease administration, when many users basically just
need permissions to create topics and grant permissions to those topics.
From my reading of this chain, it is obvious that it probably won't be
given a ton of permissions and it may be complex to add, but it seems like
starting with a conservative approach of permissions for a namespace admin
initially could be done fairly quickly and safely.
I would say the following could make a a lot of sense to be in a namespace
admin permissions:
Topics
- compaction/compaction status
- offload/offload-status
- create/delete non partitioned topics
- create/delete partitioned topics
- list/grant/revoke permissions
- create/delete/peek/seek subscriptions
- stats calls
Namespaces
- list/grant/revoke permissions
- set/get clusters (maybe not?)
- get/set ttl
- get/set persistence (maybe not?)
I think that set of permissions (with some deletions or adjustments) could
go a long way to making the current permissions model work a fair bit
better for orgs trying to deploy pulsar.
Long term, it feels like the correct answer to this problem is probably to
introduce some sort of policy language with a role being tied to some
policy that can grant a wide range of individual permissions, with the
existing superuser/tenant-admin/etc being redefined in terms of some of
these policies (perhaps using something like https://casbin.org/en/). If it
seems like the focus should be on the more general solution, then I am fine
leaving this for now, but this will be something we need to tackle in the
next 6 months or so for my org, so I would love to see any sort of
direction of where things should be heading so we can plan to that and
perhaps help where possible!
On Mon, Nov 11, 2019 at 2:24 AM xiaolong ran <ra...@gmail.com>
wrote:
> Hello everyone:
>
> I reorganize the relevant content of this PIP. PTAL again.
>
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> <
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> >
>
> In this PIP, I only fix the unreasonable permissions in the current
> command and shown in bold.
> About specific permission of command, if there are different opinions
> about this, pls let me know, thanks.
>
> --
>
> Thanks
> Xiaolong Ran
>
>
> > 在 2019年11月8日，下午4:41，xiaolong ran <ra...@gmail.com> 写道：
> >
> > Thanks Sijie, Joe F and Addison Highham feedback.
> >
> > As Joe said:
> >
> > > There are no simple answers here other than to understand the effect
> of the command.
> >
> > So in here, we can use sijie proposal, we only need to fix the
> unreasonable permissions in the current command.
> >
> > I will reorganize this PIP, what do you think about this?
> >
> > --
> >
> > Thanks
> > Xiaolong Ran
> >
> >
> >> 在 2019年11月8日，下午3:59，Sijie Guo <guosijie@gmail.com <mailto:
> guosijie@gmail.com>> 写道：
> >>
> >> If the problem is complex enough, can we first reduce the scope of this
> PIP
> >> to focus on fixing the permissions of individual commands (e.g. schemas
> >> don't require super-user permissions)?
> >> As you said, there is no simple answers. We can go through the commands
> one
> >> by one and come up a table of commands and their permissions and
> document
> >> it.
> >> It can help users understand how the permissions work for each command.
> >>
> >> We can revisit a larger scope till we add the capability of allocation
> >> resources at different levels.
> >>
> >> Does that make sense?
> >>
> >> - Sijie
> >>
> >>
> >> On Fri, Nov 8, 2019 at 3:22 PM Joe F <joef@apache.org <mailto:
> joef@apache.org>> wrote:
> >>
> >>> There are no simple answers here other than to understand the effect
> of the
> >>> command.
> >>>
> >>> The resource limit/control command always addresses an object. But
> giving
> >>> control of that command to the object owner, just because the command
> >>> addresses an object, will break all controls about resource
> utilization in
> >>> a shared system. A filesystem like model may look simple and elegant
> but
> >>> is fraught with all kind of unintended consequences if implemented in
> >>> Pulsar (Even in the filesystem rm, mv and chown, while all addressing
> a
> >>> file, treats the file owner differently)
> >>>
> >>> This is a complex, and complicated effort, and my concern is that
> this PIP
> >>> does not even come close to the scope of the work that is required.
> Some
> >>> of them may be impossible to do (as allocating certain resources to
> >>> namespace as compared to tenant) till such capabilities are added to
> the
> >>> system. And some begs fundamental questions - when you start managing
> a
> >>> namespace like a tenant, does it even make sense to have that
> namespace?
> >>> Should't that namespace be a tenant in the first place? Why have a
> >>> namespace admin at all?
> >>>
> >>> Joe
> >>>
> >>>
> >>>
> >>> On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wave4dave@comcast.net
> <ma...@comcast.net>> wrote:
> >>>
> >>>> I’m not diving in but thinking about the logical implication of this
> >>>> dichotomy. For any object’s attributes some ought to be controlled by
> >>>> object level permissions and others by sysadmin permissions. How can
> >>>> developers tell?
> >>>>
> >>>> Best Regards,
> >>>> Dave
> >>>>
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On Nov 7, 2019, at 8:02 PM, Joe F <joef@apache.org <mailto:
> joef@apache.org>> wrote:
> >>>>>
> >>>>> This proposal has the same issues as the previous one . In general
> it
> >>> is
> >>>>> not correct to think of commands as being controlled by owner of the
> >>>>> object on which the command operates. Eg: It will be an error to
> >>> assing
> >>>>> control of all namespace commands to the namespace owner
> >>>>>
> >>>>> For eg: set subscription-dispatch-rate operates on a subscription
> rate
> >>>> and
> >>>>> set-max-producers-per-topic These operate on namespaces. A naive
> >>> approach
> >>>>> is to think that this would be controlled by the namespace owner.
> But
> >>>>> essentially both these are resource management operations. That
> should
> >>>> be
> >>>>> controlled by the system admin, unless you want to deal with every
> >>>>> namespace owner having the power to destroy your SLAs fo the cluster.
> >>>>>
> >>>>> I just picked 2 examples to indicate the drawback of approaching
> >>>>> permissions in the proposed manner.
> >>>>>
> >>>>> There are many such cases in the proposal, too many to list out
> here. I
> >>>>> suggest completely ditching this approach of equating objects with
> >>> admin
> >>>>> capability of object owner
> >>>>>
> >>>>> Joe
> >>>>>
> >>>>>
> >>>>>> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <
> ranxiaolong716@gmail.com <ma...@gmail.com>
> >>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hello all committers:
> >>>>>>
> >>>>>> About this PIP, do you have any other good ideas or propose.
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>>
> >>>>>> Thanks
> >>>>>> Xioalong Ran
> >>>>>>
> >>>>>>
> >>>>>>>> 在 2019年10月30日，下午7:49，xiaolong ran <ranxiaolong716@gmail.com
> <ma...@gmail.com>> 写道：
> >>>>>>>
> >>>>>>> Hello all:
> >>>>>>>
> >>>>>>> When using pulsar-admin, I found that the current permission
> >>>>>> verification mechanism has some problems.
> >>>>>>>
> >>>>>>> We are starting a proposal about permission levels and inheritance
> in
> >>>>>> Apache Pulsar.
> >>>>>>>
> >>>>>>> The proposal is in:
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> <
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> >
> >>>>>> <
> >>>>>>
> >>>>
> >>>
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> >>>>>>>
> >>>>>>>
> >>>>>>> To ensure compatibility, these changes will be made in v4's admin
> >>> api.
> >>>>>> About the v4 admin API, see:
> >>>>>>>
> >>>>>>>
> >>> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
> >>>> <
> >>>>>>
> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> >>>>>>>
> >>>>>>> Looking forward to any feedback.
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >
>
>
Re: PIP 49: Permission levels and inheritance
Posted by xiaolong ran <ra...@gmail.com>.
Hello everyone:
I reorganize the relevant content of this PIP. PTAL again.
https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance <https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance>
In this PIP, I only fix the unreasonable permissions in the current command and shown in bold.
About specific permission of command, if there are different opinions about this, pls let me know, thanks.
--
Thanks
Xiaolong Ran
> 在 2019年11月8日，下午4:41，xiaolong ran <ra...@gmail.com> 写道：
>
> Thanks Sijie, Joe F and Addison Highham feedback.
>
> As Joe said:
>
> > There are no simple answers here other than to understand the effect of the command.
>
> So in here, we can use sijie proposal, we only need to fix the unreasonable permissions in the current command.
>
> I will reorganize this PIP, what do you think about this?
>
> --
>
> Thanks
> Xiaolong Ran
>
>
>> 在 2019年11月8日，下午3:59，Sijie Guo <guosijie@gmail.com <ma...@gmail.com>> 写道：
>>
>> If the problem is complex enough, can we first reduce the scope of this PIP
>> to focus on fixing the permissions of individual commands (e.g. schemas
>> don't require super-user permissions)?
>> As you said, there is no simple answers. We can go through the commands one
>> by one and come up a table of commands and their permissions and document
>> it.
>> It can help users understand how the permissions work for each command.
>>
>> We can revisit a larger scope till we add the capability of allocation
>> resources at different levels.
>>
>> Does that make sense?
>>
>> - Sijie
>>
>>
>> On Fri, Nov 8, 2019 at 3:22 PM Joe F <joef@apache.org <ma...@apache.org>> wrote:
>>
>>> There are no simple answers here other than to understand the effect of the
>>> command.
>>>
>>> The resource limit/control command always addresses an object. But giving
>>> control of that command to the object owner, just because the command
>>> addresses an object, will break all controls about resource utilization in
>>> a shared system. A filesystem like model may look simple and elegant but
>>> is fraught with all kind of unintended consequences if implemented in
>>> Pulsar (Even in the filesystem rm, mv and chown, while all addressing a
>>> file, treats the file owner differently)
>>>
>>> This is a complex, and complicated effort, and my concern is that this PIP
>>> does not even come close to the scope of the work that is required. Some
>>> of them may be impossible to do (as allocating certain resources to
>>> namespace as compared to tenant) till such capabilities are added to the
>>> system. And some begs fundamental questions - when you start managing a
>>> namespace like a tenant, does it even make sense to have that namespace?
>>> Should't that namespace be a tenant in the first place? Why have a
>>> namespace admin at all?
>>>
>>> Joe
>>>
>>>
>>>
>>> On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wave4dave@comcast.net <ma...@comcast.net>> wrote:
>>>
>>>> I’m not diving in but thinking about the logical implication of this
>>>> dichotomy. For any object’s attributes some ought to be controlled by
>>>> object level permissions and others by sysadmin permissions. How can
>>>> developers tell?
>>>>
>>>> Best Regards,
>>>> Dave
>>>>
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On Nov 7, 2019, at 8:02 PM, Joe F <joef@apache.org <ma...@apache.org>> wrote:
>>>>>
>>>>> This proposal has the same issues as the previous one . In general it
>>> is
>>>>> not correct to think of commands as being controlled by owner of the
>>>>> object on which the command operates. Eg: It will be an error to
>>> assing
>>>>> control of all namespace commands to the namespace owner
>>>>>
>>>>> For eg: set subscription-dispatch-rate operates on a subscription rate
>>>> and
>>>>> set-max-producers-per-topic These operate on namespaces. A naive
>>> approach
>>>>> is to think that this would be controlled by the namespace owner. But
>>>>> essentially both these are resource management operations. That should
>>>> be
>>>>> controlled by the system admin, unless you want to deal with every
>>>>> namespace owner having the power to destroy your SLAs fo the cluster.
>>>>>
>>>>> I just picked 2 examples to indicate the drawback of approaching
>>>>> permissions in the proposed manner.
>>>>>
>>>>> There are many such cases in the proposal, too many to list out here. I
>>>>> suggest completely ditching this approach of equating objects with
>>> admin
>>>>> capability of object owner
>>>>>
>>>>> Joe
>>>>>
>>>>>
>>>>>> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ranxiaolong716@gmail.com <ma...@gmail.com>
>>>>
>>>>>> wrote:
>>>>>>
>>>>>> Hello all committers:
>>>>>>
>>>>>> About this PIP, do you have any other good ideas or propose.
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Thanks
>>>>>> Xioalong Ran
>>>>>>
>>>>>>
>>>>>>>> 在 2019年10月30日，下午7:49，xiaolong ran <ranxiaolong716@gmail.com <ma...@gmail.com>> 写道：
>>>>>>>
>>>>>>> Hello all:
>>>>>>>
>>>>>>> When using pulsar-admin, I found that the current permission
>>>>>> verification mechanism has some problems.
>>>>>>>
>>>>>>> We are starting a proposal about permission levels and inheritance in
>>>>>> Apache Pulsar.
>>>>>>>
>>>>>>> The proposal is in:
>>>>>>>
>>>>>>
>>>>
>>> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance <https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance>
>>>>>> <
>>>>>>
>>>>
>>> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
>>>>>>>
>>>>>>>
>>>>>>> To ensure compatibility, these changes will be made in v4's admin
>>> api.
>>>>>> About the v4 admin API, see:
>>>>>>>
>>>>>>>
>>> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
>>>> <
>>>>>> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>>>>>>>
>>>>>>> Looking forward to any feedback.
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>
Re: PIP 49: Permission levels and inheritance
Posted by xiaolong ran <ra...@gmail.com>.
Thanks Sijie, Joe F and Addison Highham feedback.
As Joe said:
> There are no simple answers here other than to understand the effect of the command.
So in here, we can use sijie proposal, we only need to fix the unreasonable permissions in the current command.
I will reorganize this PIP, what do you think about this?
--
Thanks
Xiaolong Ran
> 在 2019年11月8日，下午3:59，Sijie Guo <gu...@gmail.com> 写道：
>
> If the problem is complex enough, can we first reduce the scope of this PIP
> to focus on fixing the permissions of individual commands (e.g. schemas
> don't require super-user permissions)?
> As you said, there is no simple answers. We can go through the commands one
> by one and come up a table of commands and their permissions and document
> it.
> It can help users understand how the permissions work for each command.
>
> We can revisit a larger scope till we add the capability of allocation
> resources at different levels.
>
> Does that make sense?
>
> - Sijie
>
>
> On Fri, Nov 8, 2019 at 3:22 PM Joe F <jo...@apache.org> wrote:
>
>> There are no simple answers here other than to understand the effect of the
>> command.
>>
>> The resource limit/control command always addresses an object. But giving
>> control of that command to the object owner, just because the command
>> addresses an object, will break all controls about resource utilization in
>> a shared system. A filesystem like model may look simple and elegant but
>> is fraught with all kind of unintended consequences if implemented in
>> Pulsar (Even in the filesystem rm, mv and chown, while all addressing a
>> file, treats the file owner differently)
>>
>> This is a complex, and complicated effort, and my concern is that this PIP
>> does not even come close to the scope of the work that is required. Some
>> of them may be impossible to do (as allocating certain resources to
>> namespace as compared to tenant) till such capabilities are added to the
>> system. And some begs fundamental questions - when you start managing a
>> namespace like a tenant, does it even make sense to have that namespace?
>> Should't that namespace be a tenant in the first place? Why have a
>> namespace admin at all?
>>
>> Joe
>>
>>
>>
>> On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wa...@comcast.net> wrote:
>>
>>> I’m not diving in but thinking about the logical implication of this
>>> dichotomy. For any object’s attributes some ought to be controlled by
>>> object level permissions and others by sysadmin permissions. How can
>>> developers tell?
>>>
>>> Best Regards,
>>> Dave
>>>
>>>
>>> Sent from my iPhone
>>>
>>>> On Nov 7, 2019, at 8:02 PM, Joe F <jo...@apache.org> wrote:
>>>>
>>>> This proposal has the same issues as the previous one . In general it
>> is
>>>> not correct to think of commands as being controlled by owner of the
>>>> object on which the command operates. Eg: It will be an error to
>> assing
>>>> control of all namespace commands to the namespace owner
>>>>
>>>> For eg: set subscription-dispatch-rate operates on a subscription rate
>>> and
>>>> set-max-producers-per-topic These operate on namespaces. A naive
>> approach
>>>> is to think that this would be controlled by the namespace owner. But
>>>> essentially both these are resource management operations. That should
>>> be
>>>> controlled by the system admin, unless you want to deal with every
>>>> namespace owner having the power to destroy your SLAs fo the cluster.
>>>>
>>>> I just picked 2 examples to indicate the drawback of approaching
>>>> permissions in the proposed manner.
>>>>
>>>> There are many such cases in the proposal, too many to list out here. I
>>>> suggest completely ditching this approach of equating objects with
>> admin
>>>> capability of object owner
>>>>
>>>> Joe
>>>>
>>>>
>>>>> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ranxiaolong716@gmail.com
>>>
>>>>> wrote:
>>>>>
>>>>> Hello all committers:
>>>>>
>>>>> About this PIP, do you have any other good ideas or propose.
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Thanks
>>>>> Xioalong Ran
>>>>>
>>>>>
>>>>>>> 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
>>>>>>
>>>>>> Hello all:
>>>>>>
>>>>>> When using pulsar-admin, I found that the current permission
>>>>> verification mechanism has some problems.
>>>>>>
>>>>>> We are starting a proposal about permission levels and inheritance in
>>>>> Apache Pulsar.
>>>>>>
>>>>>> The proposal is in:
>>>>>>
>>>>>
>>>
>> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
>>>>> <
>>>>>
>>>
>> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
>>>>>>
>>>>>>
>>>>>> To ensure compatibility, these changes will be made in v4's admin
>> api.
>>>>> About the v4 admin API, see:
>>>>>>
>>>>>>
>> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
>>> <
>>>>> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>>>>>>
>>>>>> Looking forward to any feedback.
>>>>>
>>>>>
>>>
>>>
>>
Re: PIP 49: Permission levels and inheritance
Posted by Sijie Guo <gu...@gmail.com>.
If the problem is complex enough, can we first reduce the scope of this PIP
to focus on fixing the permissions of individual commands (e.g. schemas
don't require super-user permissions)?
As you said, there is no simple answers. We can go through the commands one
by one and come up a table of commands and their permissions and document
it.
It can help users understand how the permissions work for each command.
We can revisit a larger scope till we add the capability of allocation
resources at different levels.
Does that make sense?
- Sijie
On Fri, Nov 8, 2019 at 3:22 PM Joe F <jo...@apache.org> wrote:
> There are no simple answers here other than to understand the effect of the
> command.
>
> The resource limit/control command always addresses an object. But giving
> control of that command to the object owner, just because the command
> addresses an object, will break all controls about resource utilization in
> a shared system. A filesystem like model may look simple and elegant but
> is fraught with all kind of unintended consequences if implemented in
> Pulsar (Even in the filesystem rm, mv and chown, while all addressing a
> file, treats the file owner differently)
>
> This is a complex, and complicated effort, and my concern is that this PIP
> does not even come close to the scope of the work that is required. Some
> of them may be impossible to do (as allocating certain resources to
> namespace as compared to tenant) till such capabilities are added to the
> system. And some begs fundamental questions - when you start managing a
> namespace like a tenant, does it even make sense to have that namespace?
> Should't that namespace be a tenant in the first place? Why have a
> namespace admin at all?
>
> Joe
>
>
>
> On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wa...@comcast.net> wrote:
>
> > I’m not diving in but thinking about the logical implication of this
> > dichotomy. For any object’s attributes some ought to be controlled by
> > object level permissions and others by sysadmin permissions. How can
> > developers tell?
> >
> > Best Regards,
> > Dave
> >
> >
> > Sent from my iPhone
> >
> > > On Nov 7, 2019, at 8:02 PM, Joe F <jo...@apache.org> wrote:
> > >
> > > This proposal has the same issues as the previous one . In general it
> is
> > > not correct to think of commands as being controlled by owner of the
> > > object on which the command operates. Eg: It will be an error to
> assing
> > > control of all namespace commands to the namespace owner
> > >
> > > For eg: set subscription-dispatch-rate operates on a subscription rate
> > and
> > > set-max-producers-per-topic These operate on namespaces. A naive
> approach
> > > is to think that this would be controlled by the namespace owner. But
> > > essentially both these are resource management operations. That should
> > be
> > > controlled by the system admin, unless you want to deal with every
> > > namespace owner having the power to destroy your SLAs fo the cluster.
> > >
> > > I just picked 2 examples to indicate the drawback of approaching
> > > permissions in the proposed manner.
> > >
> > > There are many such cases in the proposal, too many to list out here. I
> > > suggest completely ditching this approach of equating objects with
> admin
> > > capability of object owner
> > >
> > > Joe
> > >
> > >
> > >> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ranxiaolong716@gmail.com
> >
> > >> wrote:
> > >>
> > >> Hello all committers:
> > >>
> > >> About this PIP, do you have any other good ideas or propose.
> > >>
> > >>
> > >> --
> > >>
> > >> Thanks
> > >> Xioalong Ran
> > >>
> > >>
> > >>>> 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
> > >>>
> > >>> Hello all:
> > >>>
> > >>> When using pulsar-admin, I found that the current permission
> > >> verification mechanism has some problems.
> > >>>
> > >>> We are starting a proposal about permission levels and inheritance in
> > >> Apache Pulsar.
> > >>>
> > >>> The proposal is in:
> > >>>
> > >>
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > >> <
> > >>
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > >>>
> > >>>
> > >>> To ensure compatibility, these changes will be made in v4's admin
> api.
> > >> About the v4 admin API, see:
> > >>>
> > >>>
> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
> > <
> > >> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> > >>>
> > >>> Looking forward to any feedback.
> > >>
> > >>
> >
> >
>
Re: PIP 49: Permission levels and inheritance
Posted by Joe F <jo...@apache.org>.
There are no simple answers here other than to understand the effect of the
command.
The resource limit/control command always addresses an object. But giving
control of that command to the object owner, just because the command
addresses an object, will break all controls about resource utilization in
a shared system. A filesystem like model may look simple and elegant but
is fraught with all kind of unintended consequences if implemented in
Pulsar (Even in the filesystem rm, mv and chown, while all addressing a
file, treats the file owner differently)
This is a complex, and complicated effort, and my concern is that this PIP
does not even come close to the scope of the work that is required. Some
of them may be impossible to do (as allocating certain resources to
namespace as compared to tenant) till such capabilities are added to the
system. And some begs fundamental questions - when you start managing a
namespace like a tenant, does it even make sense to have that namespace?
Should't that namespace be a tenant in the first place? Why have a
namespace admin at all?
Joe
On Thu, Nov 7, 2019 at 8:16 PM Dave Fisher <wa...@comcast.net> wrote:
> I’m not diving in but thinking about the logical implication of this
> dichotomy. For any object’s attributes some ought to be controlled by
> object level permissions and others by sysadmin permissions. How can
> developers tell?
>
> Best Regards,
> Dave
>
>
> Sent from my iPhone
>
> > On Nov 7, 2019, at 8:02 PM, Joe F <jo...@apache.org> wrote:
> >
> > This proposal has the same issues as the previous one . In general it is
> > not correct to think of commands as being controlled by owner of the
> > object on which the command operates. Eg: It will be an error to assing
> > control of all namespace commands to the namespace owner
> >
> > For eg: set subscription-dispatch-rate operates on a subscription rate
> and
> > set-max-producers-per-topic These operate on namespaces. A naive approach
> > is to think that this would be controlled by the namespace owner. But
> > essentially both these are resource management operations. That should
> be
> > controlled by the system admin, unless you want to deal with every
> > namespace owner having the power to destroy your SLAs fo the cluster.
> >
> > I just picked 2 examples to indicate the drawback of approaching
> > permissions in the proposed manner.
> >
> > There are many such cases in the proposal, too many to list out here. I
> > suggest completely ditching this approach of equating objects with admin
> > capability of object owner
> >
> > Joe
> >
> >
> >> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ra...@gmail.com>
> >> wrote:
> >>
> >> Hello all committers:
> >>
> >> About this PIP, do you have any other good ideas or propose.
> >>
> >>
> >> --
> >>
> >> Thanks
> >> Xioalong Ran
> >>
> >>
> >>>> 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
> >>>
> >>> Hello all:
> >>>
> >>> When using pulsar-admin, I found that the current permission
> >> verification mechanism has some problems.
> >>>
> >>> We are starting a proposal about permission levels and inheritance in
> >> Apache Pulsar.
> >>>
> >>> The proposal is in:
> >>>
> >>
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> >> <
> >>
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> >>>
> >>>
> >>> To ensure compatibility, these changes will be made in v4's admin api.
> >> About the v4 admin API, see:
> >>>
> >>> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
> <
> >> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> >>>
> >>> Looking forward to any feedback.
> >>
> >>
>
>
Re: PIP 49: Permission levels and inheritance
Posted by Dave Fisher <wa...@comcast.net>.
I’m not diving in but thinking about the logical implication of this dichotomy. For any object’s attributes some ought to be controlled by object level permissions and others by sysadmin permissions. How can developers tell?
Best Regards,
Dave
Sent from my iPhone
> On Nov 7, 2019, at 8:02 PM, Joe F <jo...@apache.org> wrote:
>
> This proposal has the same issues as the previous one . In general it is
> not correct to think of commands as being controlled by owner of the
> object on which the command operates. Eg: It will be an error to assing
> control of all namespace commands to the namespace owner
>
> For eg: set subscription-dispatch-rate operates on a subscription rate and
> set-max-producers-per-topic These operate on namespaces. A naive approach
> is to think that this would be controlled by the namespace owner. But
> essentially both these are resource management operations. That should be
> controlled by the system admin, unless you want to deal with every
> namespace owner having the power to destroy your SLAs fo the cluster.
>
> I just picked 2 examples to indicate the drawback of approaching
> permissions in the proposed manner.
>
> There are many such cases in the proposal, too many to list out here. I
> suggest completely ditching this approach of equating objects with admin
> capability of object owner
>
> Joe
>
>
>> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ra...@gmail.com>
>> wrote:
>>
>> Hello all committers:
>>
>> About this PIP, do you have any other good ideas or propose.
>>
>>
>> --
>>
>> Thanks
>> Xioalong Ran
>>
>>
>>>> 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
>>>
>>> Hello all:
>>>
>>> When using pulsar-admin, I found that the current permission
>> verification mechanism has some problems.
>>>
>>> We are starting a proposal about permission levels and inheritance in
>> Apache Pulsar.
>>>
>>> The proposal is in:
>>>
>> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
>> <
>> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
>>>
>>>
>>> To ensure compatibility, these changes will be made in v4's admin api.
>> About the v4 admin API, see:
>>>
>>> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <
>> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>>>
>>> Looking forward to any feedback.
>>
>>
Re: PIP 49: Permission levels and inheritance
Posted by Sijie Guo <gu...@gmail.com>.
I haven't gone through the details of the new proposal yet.
But based on the comments in the email thread, it seems to me that one of
the major concerns is around the ownership and permissions of "resources".
The actual resources (cpu, memory, and storage) is shared between multiple
tenants within a cluster.
The resources are allocated to tenants in the form of namespace policy
(dispatch rate, backlog, storage quota and etc).
That means mutating a namespace policy is modifying the resource
allocation. Hence as a cluster owner, you don't want to allow non cluster
owners to
modify the resource allocations that can potentially impact other tenants
in the same system.
If my understanding is correct, then there are two options that we can
explore:
1) separate the configurations in namespace policy into 2 groups.
one is for resource allocations (configurations can change the resource
usage), and the other one for remaining other settings (e.g. encryption
required, ttl).
cluster-admin (aka super-user) can modify the first group of configuration;
while namespace-admin can modify the other group.
This can also set a guideline for people adding more configurations into
namespace policy in the future.
2) provide a hierarchical resource allocation.
The concern was raise mainly because we have only one level (via namespace
policy) to allocate resources.
Similar as what topic policy is doing, we add a tenant level policy for
resource management.
The tenant-level policy limits the total resources can be used by all the
namespace and topics.
Hence the resource allocation and ownership can be well defined, and it can
be aligned with permissions. So we can have a clear separation between
operations and users. Cluster-owners control the total resource allocated
to a tenant but it doesn't control the namespace level details. The
modifications within a tenant doesn't affect the total resource allocated
to a tenant.
To me, exploring and going down with second option is probably a right
approach to make Pulsar a better "multi-tenant" system.
However we can start with first option to set a clear guideline.
Just my two cents.
- Sijie
On Fri, Nov 8, 2019 at 12:02 PM Joe F <jo...@apache.org> wrote:
> This proposal has the same issues as the previous one . In general it is
> not correct to think of commands as being controlled by owner of the
> object on which the command operates. Eg: It will be an error to assing
> control of all namespace commands to the namespace owner
>
> For eg: set subscription-dispatch-rate operates on a subscription rate and
> set-max-producers-per-topic These operate on namespaces. A naive approach
> is to think that this would be controlled by the namespace owner. But
> essentially both these are resource management operations. That should be
> controlled by the system admin, unless you want to deal with every
> namespace owner having the power to destroy your SLAs fo the cluster.
>
> I just picked 2 examples to indicate the drawback of approaching
> permissions in the proposed manner.
>
> There are many such cases in the proposal, too many to list out here. I
> suggest completely ditching this approach of equating objects with admin
> capability of object owner
>
> Joe
>
>
> On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ra...@gmail.com>
> wrote:
>
> > Hello all committers:
> >
> > About this PIP, do you have any other good ideas or propose.
> >
> >
> > --
> >
> > Thanks
> > Xioalong Ran
> >
> >
> > > 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
> > >
> > > Hello all:
> > >
> > > When using pulsar-admin, I found that the current permission
> > verification mechanism has some problems.
> > >
> > > We are starting a proposal about permission levels and inheritance in
> > Apache Pulsar.
> > >
> > > The proposal is in:
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > <
> >
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > >
> > >
> > > To ensure compatibility, these changes will be made in v4's admin api.
> > About the v4 admin API, see:
> > >
> > > https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api
> <
> > https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> > >
> > > Looking forward to any feedback.
> >
> >
>
Re: PIP 49: Permission levels and inheritance
Posted by Joe F <jo...@apache.org>.
This proposal has the same issues as the previous one . In general it is
not correct to think of commands as being controlled by owner of the
object on which the command operates. Eg: It will be an error to assing
control of all namespace commands to the namespace owner
For eg: set subscription-dispatch-rate operates on a subscription rate and
set-max-producers-per-topic These operate on namespaces. A naive approach
is to think that this would be controlled by the namespace owner. But
essentially both these are resource management operations. That should be
controlled by the system admin, unless you want to deal with every
namespace owner having the power to destroy your SLAs fo the cluster.
I just picked 2 examples to indicate the drawback of approaching
permissions in the proposed manner.
There are many such cases in the proposal, too many to list out here. I
suggest completely ditching this approach of equating objects with admin
capability of object owner
Joe
On Wed, Nov 6, 2019 at 2:14 AM xiaolong ran <ra...@gmail.com>
wrote:
> Hello all committers:
>
> About this PIP, do you have any other good ideas or propose.
>
>
> --
>
> Thanks
> Xioalong Ran
>
>
> > 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
> >
> > Hello all:
> >
> > When using pulsar-admin, I found that the current permission
> verification mechanism has some problems.
> >
> > We are starting a proposal about permission levels and inheritance in
> Apache Pulsar.
> >
> > The proposal is in:
> >
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> <
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> >
> >
> > To ensure compatibility, these changes will be made in v4's admin api.
> About the v4 admin API, see:
> >
> > https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <
> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> >
> > Looking forward to any feedback.
>
>
Re: PIP 49: Permission levels and inheritance
Posted by xiaolong ran <ra...@gmail.com>.
Hello all committers:
About this PIP, do you have any other good ideas or propose.
--
Thanks
Xioalong Ran
> 在 2019年10月30日，下午7:49，xiaolong ran <ra...@gmail.com> 写道：
>
> Hello all:
>
> When using pulsar-admin, I found that the current permission verification mechanism has some problems.
>
> We are starting a proposal about permission levels and inheritance in Apache Pulsar.
>
> The proposal is in:
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance <https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance>
>
> To ensure compatibility, these changes will be made in v4's admin api. About the v4 admin API, see:
>
> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>
> Looking forward to any feedback.
Re: PIP 49: Permission levels and inheritance
Posted by xiaolong ran <ra...@gmail.com>.
Thanks Matteo and Joe feedback, I will reorganize the relevant content of this PIP.
> 在 2019年10月31日，上午5:29，Matteo Merli <mm...@apache.org> 写道：
>
> On Wed, Oct 30, 2019 at 10:11 AM xiaolong ran <ra...@gmail.com> wrote:
>>
>> Thanks Matteo and Joe.
>>
>> In here, I have two questions:
>>
>> 1. In `namespaces` and `topics`, most of the commands are `tenant admin`, are they reasonable?
>
> I think having the intermediate step of namespace admin is good.
>
> For topics, we need to revisit several of these. But the questions are
> anyway tricky:
> Should someone who has "consume" access to a topic also have access to
> this topic stats?
>
> On one hand it might make sense, though the topic stats also include
> info for other unrelated subscriptions,
> which one might not want to be visible.
>
>
>> 2. Whether permissions management should be inherited?
>>
>> For examples: in `bin/pulsar-admin topics create [topic name]`, should only the `tenant admin` have permission to operate,
>> even if the `super user ` should not have permission to create a topic, right?
>
> I think both views are valid there, though always augmenting is a bit
> easier to implement and reason about.
>
> Super-user itself was being treated a bit different in that it's also
> used for broker-to-broker communication. For geo-replication,
> a broker needs to be able to connect to any other broker and publish
> on any topic.
>
>> Yes, for some permissions of commands, we need to discuss further, this is also the purpose of this PIP. This PIP
>> proposes introducing permission levels and inheritance into Pulsar authorization system to make permission
>> check clearer across Pulsar codebase. We can discuss which of the permissions for the commands in this PIP
>> are reasonable, and which of the commands' permissions we should keep current permissions.
>
> I think this should start by categorizing the actions/operations
> instead of focusing on the individual handlers.
Re: PIP 49: Permission levels and inheritance
Posted by Matteo Merli <mm...@apache.org>.
On Wed, Oct 30, 2019 at 10:11 AM xiaolong ran <ra...@gmail.com> wrote:
>
> Thanks Matteo and Joe.
>
> In here, I have two questions:
>
> 1. In `namespaces` and `topics`, most of the commands are `tenant admin`, are they reasonable?
I think having the intermediate step of namespace admin is good.
For topics, we need to revisit several of these. But the questions are
anyway tricky:
Should someone who has "consume" access to a topic also have access to
this topic stats?
On one hand it might make sense, though the topic stats also include
info for other unrelated subscriptions,
which one might not want to be visible.
> 2. Whether permissions management should be inherited?
>
> For examples: in `bin/pulsar-admin topics create [topic name]`, should only the `tenant admin` have permission to operate,
> even if the `super user ` should not have permission to create a topic, right?
I think both views are valid there, though always augmenting is a bit
easier to implement and reason about.
Super-user itself was being treated a bit different in that it's also
used for broker-to-broker communication. For geo-replication,
a broker needs to be able to connect to any other broker and publish
on any topic.
> Yes, for some permissions of commands, we need to discuss further, this is also the purpose of this PIP. This PIP
> proposes introducing permission levels and inheritance into Pulsar authorization system to make permission
> check clearer across Pulsar codebase. We can discuss which of the permissions for the commands in this PIP
> are reasonable, and which of the commands' permissions we should keep current permissions.
I think this should start by categorizing the actions/operations
instead of focusing on the individual handlers.
Re: PIP 49: Permission levels and inheritance
Posted by xiaolong ran <ra...@gmail.com>.
Thanks Matteo and Joe.
In here, I have two questions:
1. In `namespaces` and `topics`, most of the commands are `tenant admin`, are they reasonable?
2. Whether permissions management should be inherited?
For examples: in `bin/pulsar-admin topics create [topic name]`, should only the `tenant admin` have permission to operate,
even if the `super user ` should not have permission to create a topic, right?
> From `set-subscribe-rate` to `set-max-producers-per-topic`, all these
methods are there for protecting the system and
give the operator the flexibility to box users into different shaped
boxes based on necessity.
Yes, for some permissions of commands, we need to discuss further, this is also the purpose of this PIP. This PIP
proposes introducing permission levels and inheritance into Pulsar authorization system to make permission
check clearer across Pulsar codebase. We can discuss which of the permissions for the commands in this PIP
are reasonable, and which of the commands' permissions we should keep current permissions.
Thanks again.
Best Regards,
Xiaolong Ran
> 在 2019年10月31日，上午12:33，Matteo Merli <mm...@apache.org> 写道：
>
> Agree with Joe,
>
> there are several methods in namespaces API that *really* meant for
> super-user only. It was not a mistake or oversight.
>
> From `set-subscribe-rate` to `set-max-producers-per-topic`, all these
> methods are there for protecting the system and
> give the operator the flexibility to box users into different shaped
> boxes based on necessity.
>
> Also, if we're talking about inheritance we should also include the
> topic level ACLs (which are already supported) and how
> the topic level ACLs interacts with the APIs.
>
> Eg: if I have topic level produce permission on a topic, I should be
> able to get/set the schema for that topic.
>
>
>
>
> --
> Matteo Merli
> <mm...@apache.org>
>
> On Wed, Oct 30, 2019 at 7:22 AM Joe F <jo...@apache.org> wrote:
>>
>> It is good that we are looking at revamping the system. But the proposal
>> as it is, is thin.
>>
>> First I would like this proposal to be split into two. One for inheritance
>> and another for changes from existing controls. They are completely
>> orthogonal and independent issues. Second, both of them (inheritance and
>> changes) need to have a clear rationales.
>>
>> There are 2 key underlying principles in Pulsar,. The cluster operators,
>> i.e. Pulsar admins (super-users) manage the system and system resources,
>> Their role is in operating the cluster and managing system resources (CPU,
>> storage, n/w bandwidth etc) and keeping the system healthy. Tenant admins
>> mange the tenant resources allocated to them) .Second, Pulsar has a model
>> of disintermediation. Owning, Producing and Consuming are separate concerns
>> and are abstracted from each other.
>>
>> Many changes in this proposal does not fit the model. So I would like to
>> see rationale for each permission change from existing permissions
>>
>> For eg: the proposed change for in namespace permissions for
>> set-clusters tenant-admin ==> super-user
>> breaks the resource model. A tenant is allocated resources in X,Y,Z
>> clusters. And it's up top the tenant to manage it - whether to use
>> resources in X, or in X& Y etc, and to which resources. There is no reason
>> for the super-user to get involved.
>>
>> Another example, unloading a namespace is a system operation that moves
>> load in the cluster around. Allowing namespace owners to interfere in
>> system operations is never a good idea.
>>
>> I see many such changes here which does not fit the resource model
>>
>> I also see that this breaks the existing model of topic level override of
>> permissions. Again, this does not seem to be well thought-out on that side
>> either.
>>
>> So I would like to see rationales that align with 1)resource management
>> principles 2) the separation of concerns between owner, producer and
>> consumer 2) zero trust (unless explicitly granted nothing is given)4) zero
>> discovery avenues - no information, however harmless it is, is revealed to
>> entities that don't need it. (eg: There is no reason for a namespace owner
>> to do get-dispatch-rate)
>>
>> I agree that we Pulsar can improve on what we have today, but this proposal
>> needs additional work and understanding of the complex underlying issue of
>> resource management related to granting control.
>>
>> Joe
>>
>> On Wed, Oct 30, 2019 at 4:49 AM xiaolong ran <ra...@gmail.com>
>> wrote:
>>
>>> Hello all:
>>>
>>> When using pulsar-admin, I found that the current permission verification
>>> mechanism has some problems.
>>>
>>> We are starting a proposal about permission levels and inheritance in
>>> Apache Pulsar.
>>>
>>> The proposal is in:
>>>
>>> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
>>> <
>>> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
>>>>
>>>
>>> To ensure compatibility, these changes will be made in v4's admin api.
>>> About the v4 admin API, see:
>>>
>>> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <
>>> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>>>
>>> Looking forward to any feedback.
>>>
Re: PIP 49: Permission levels and inheritance
Posted by Matteo Merli <mm...@apache.org>.
Agree with Joe,
there are several methods in namespaces API that *really* meant for
super-user only. It was not a mistake or oversight.
From `set-subscribe-rate` to `set-max-producers-per-topic`, all these
methods are there for protecting the system and
give the operator the flexibility to box users into different shaped
boxes based on necessity.
Also, if we're talking about inheritance we should also include the
topic level ACLs (which are already supported) and how
the topic level ACLs interacts with the APIs.
Eg: if I have topic level produce permission on a topic, I should be
able to get/set the schema for that topic.
--
Matteo Merli
<mm...@apache.org>
On Wed, Oct 30, 2019 at 7:22 AM Joe F <jo...@apache.org> wrote:
>
> It is good that we are looking at revamping the system. But the proposal
> as it is, is thin.
>
> First I would like this proposal to be split into two. One for inheritance
> and another for changes from existing controls. They are completely
> orthogonal and independent issues. Second, both of them (inheritance and
> changes) need to have a clear rationales.
>
> There are 2 key underlying principles in Pulsar,. The cluster operators,
> i.e. Pulsar admins (super-users) manage the system and system resources,
> Their role is in operating the cluster and managing system resources (CPU,
> storage, n/w bandwidth etc) and keeping the system healthy. Tenant admins
> mange the tenant resources allocated to them) .Second, Pulsar has a model
> of disintermediation. Owning, Producing and Consuming are separate concerns
> and are abstracted from each other.
>
> Many changes in this proposal does not fit the model. So I would like to
> see rationale for each permission change from existing permissions
>
> For eg: the proposed change for in namespace permissions for
> set-clusters tenant-admin ==> super-user
> breaks the resource model. A tenant is allocated resources in X,Y,Z
> clusters. And it's up top the tenant to manage it - whether to use
> resources in X, or in X& Y etc, and to which resources. There is no reason
> for the super-user to get involved.
>
> Another example, unloading a namespace is a system operation that moves
> load in the cluster around. Allowing namespace owners to interfere in
> system operations is never a good idea.
>
> I see many such changes here which does not fit the resource model
>
> I also see that this breaks the existing model of topic level override of
> permissions. Again, this does not seem to be well thought-out on that side
> either.
>
> So I would like to see rationales that align with 1)resource management
> principles 2) the separation of concerns between owner, producer and
> consumer 2) zero trust (unless explicitly granted nothing is given)4) zero
> discovery avenues - no information, however harmless it is, is revealed to
> entities that don't need it. (eg: There is no reason for a namespace owner
> to do get-dispatch-rate)
>
> I agree that we Pulsar can improve on what we have today, but this proposal
> needs additional work and understanding of the complex underlying issue of
> resource management related to granting control.
>
> Joe
>
> On Wed, Oct 30, 2019 at 4:49 AM xiaolong ran <ra...@gmail.com>
> wrote:
>
> > Hello all:
> >
> > When using pulsar-admin, I found that the current permission verification
> > mechanism has some problems.
> >
> > We are starting a proposal about permission levels and inheritance in
> > Apache Pulsar.
> >
> > The proposal is in:
> >
> > https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> > <
> > https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> > >
> >
> > To ensure compatibility, these changes will be made in v4's admin api.
> > About the v4 admin API, see:
> >
> > https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <
> > https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
> >
> > Looking forward to any feedback.
> >
Re: PIP 49: Permission levels and inheritance
Posted by Joe F <jo...@apache.org>.
It is good that we are looking at revamping the system. But the proposal
as it is, is thin.
First I would like this proposal to be split into two. One for inheritance
and another for changes from existing controls. They are completely
orthogonal and independent issues. Second, both of them (inheritance and
changes) need to have a clear rationales.
There are 2 key underlying principles in Pulsar,. The cluster operators,
i.e. Pulsar admins (super-users) manage the system and system resources,
Their role is in operating the cluster and managing system resources (CPU,
storage, n/w bandwidth etc) and keeping the system healthy. Tenant admins
mange the tenant resources allocated to them) .Second, Pulsar has a model
of disintermediation. Owning, Producing and Consuming are separate concerns
and are abstracted from each other.
Many changes in this proposal does not fit the model. So I would like to
see rationale for each permission change from existing permissions
For eg: the proposed change for in namespace permissions for
set-clusters tenant-admin ==> super-user
breaks the resource model. A tenant is allocated resources in X,Y,Z
clusters. And it's up top the tenant to manage it - whether to use
resources in X, or in X& Y etc, and to which resources. There is no reason
for the super-user to get involved.
Another example, unloading a namespace is a system operation that moves
load in the cluster around. Allowing namespace owners to interfere in
system operations is never a good idea.
I see many such changes here which does not fit the resource model
I also see that this breaks the existing model of topic level override of
permissions. Again, this does not seem to be well thought-out on that side
either.
So I would like to see rationales that align with 1)resource management
principles 2) the separation of concerns between owner, producer and
consumer 2) zero trust (unless explicitly granted nothing is given)4) zero
discovery avenues - no information, however harmless it is, is revealed to
entities that don't need it. (eg: There is no reason for a namespace owner
to do get-dispatch-rate)
I agree that we Pulsar can improve on what we have today, but this proposal
needs additional work and understanding of the complex underlying issue of
resource management related to granting control.
Joe
On Wed, Oct 30, 2019 at 4:49 AM xiaolong ran <ra...@gmail.com>
wrote:
> Hello all:
>
> When using pulsar-admin, I found that the current permission verification
> mechanism has some problems.
>
> We are starting a proposal about permission levels and inheritance in
> Apache Pulsar.
>
> The proposal is in:
>
> https://github.com/apache/pulsar/wiki/PIP-49%3A-Permission-levels-and-inheritance
> <
> https://github.com/apache/pulsar/wiki/PIP-49:-Permission-levels-and-inheritance
> >
>
> To ensure compatibility, these changes will be made in v4's admin api.
> About the v4 admin API, see:
>
> https://github.com/apache/pulsar/wiki/PIP-48%3A-hierarchical-admin-api <
> https://github.com/apache/pulsar/wiki/PIP-48:-hierarchical-admin-api>
>
> Looking forward to any feedback.
>