Question on ACL performance

[Jon Watte <jw...@gmail.com>]

I'm looking at the C++ broker/exchange.

When I have an ACL, the documentation seems to indicate that the ACL is
parsed for each message sent. Is this the case? If I have a very long ACL
(say a million entries), would this be a performance problem?

Also, when is the ACL file re-read? If I have a system that adds new users
all the time, and each user needs a separate ACL rule, how can that be
expressed?

Sincerely,

jw


--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.

Re: Question on ACL performance

[Rajith Attapattu <ra...@gmail.com>]

> When I have an ACL, the documentation seems to indicate that the ACL is
> parsed for each message sent. Is this the case? If I have a very long ACL
> (say a million entries), would this be a performance problem?

I would assume that you meant ACL lookup when you meant "parsed".
For completeness, let me describe how ACL works.

ACL has an abstraction which provides lookup methods to figure out if
a particular action requested by the user is authorized.
This layer also holds the ACL rules in memory which is used to
determine the above.

The important methods in this interface are
inline virtual bool doTransferAcl() {return transferAcl;};  -- which
we use as switch to avoid an ACL look up if no publish ACL is present.

The ACL lookup method used for publish ACL.
virtual bool authorise(const std::string& id, const Action& action,
const ObjectType& objType, const std::string& ExchangeName,const
std::string& RoutingKey);

The ACL lookup method used for all other ACL except publish ACL.
virtual bool authorise(const std::string& id, const Action& action,
const ObjectType& objType, const std::string& name, std::map<Property,
std::string>* params=0);


As mentioned, the above ACL abstraction has a data model which holds
the ACL rules.
These rules can be populated using any mechanism.

Currently the C++ broker ships with a file based ACL. Where the ACL
rules are described in a file.
This module parses the file and populates the data model described above.

There is also another effort to use SELinux based plugin to describe
the ACL, where the SELinux rules will be used to populate the the Qpid
ACL data model.
So in theory you could write any mechanism to populate the ACL data model.

Rajith

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: Question on ACL performance

[Jon Watte <jw...@gmail.com>]

Thanks for the reply. It sounds as if a custom module would be the only sane
way (as changes are quite incremental). In fact, in the intended usage,
there is another system that can do access control and gatekeeping entirely,
and just inform the system that it's to expect a certain user to connect
with a certain set of permissions, so a module that could handle that would
probably be necessary (similar to online management).

I need to grant access on a granular level, though. A given single user will
only be allowed to publish and subscribe to a small number of
exchanges/queues. (Generally, one output queue per user, and a large number
of exchanges, only a few of which get tied to the user's individual queue)

Sincerely,

jw


--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



On Mon, Jan 18, 2010 at 9:00 AM, Rajith Attapattu <ra...@gmail.com>wrote:

> On Sun, Jan 17, 2010 at 11:35 PM, Jon Watte <jw...@gmail.com> wrote:
> > I'm looking at the C++ broker/exchange.
> >
> > When I have an ACL, the documentation seems to indicate that the ACL is
> > parsed for each message sent. Is this the case? If I have a very long ACL
> > (say a million entries), would this be a performance problem?
>
> Even if ACL is enabled in the broker,  If you don't have ACL for
> publish (message transfer) then we don't check ACL at all.
> If you have ACL for publish then we do check ACL.
> Therefore If you don't have publish ACL then there want be any impact
> on performance during message transfers.
>
> I haven't tested ACL with a large amount of entries. Once the file is
> parsed we have to keep the data in memory.
> So it will likely take a bit of memory as well.
> Also such a large amount of entries are definitely going to increase
> broker startup (where we parse the file)
> Also if your ACL involves publish ACL, then the current mechanism will
> likely have a huge performance impact.
> (See some of the suggestion below to mitigate this).
>
> > Also, when is the ACL file re-read? If I have a system that adds new
> users
> > all the time, and each user needs a separate ACL rule, how can that be
> > expressed?
> You can use the management module to reload the ACL file.
> However this could affect the performance as the file is going to be huge.
> You will likely need to add,delete,modify user privileges, each time
> requiring a reload.
>
> So given your situation, I don't think using the file based approach
> is a good idea.
> You might have to write a custom module. Below are some options.
>
> 1.) A fragmented ACL file. So when you add a new user you only read
> the fragment that was modified, instead of reloading all million
> entries.
> 2.) You have a custom ACL model which gets updated via a 3rd party
> system using a simple RPC call. Ex addUser(...) which
> add/delete/modify a user in the memory structures contained within the
> ACL abstraction in the broker.
> 3.) Another option is to partition your users among your broker
> instances. That would likely require a load balancing/routing
> mechanism (for your connections) which has explicit knowledge about
> how you have partitioned your users.
>
> Your use case is very interesting. Please keep us posted with your
> progress.
>
> >
> > Sincerely,
> >
> > jw
> >
> >
> > --
> > Americans might object: there is no way we would sacrifice our living
> > standards for the benefit of people in the rest of the world.
> Nevertheless,
> > whether we get there willingly or not, we shall soon have lower
> consumption
> > rates, because our present rates are unsustainable.
> >
>
>
>
> --
> Regards,
>
> Rajith Attapattu
> Red Hat
> http://rajith.2rlabs.com/
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>

Re: Question on ACL performance

[Rajith Attapattu <ra...@gmail.com>]

On Sun, Jan 17, 2010 at 11:35 PM, Jon Watte <jw...@gmail.com> wrote:
> I'm looking at the C++ broker/exchange.
>
> When I have an ACL, the documentation seems to indicate that the ACL is
> parsed for each message sent. Is this the case? If I have a very long ACL
> (say a million entries), would this be a performance problem?

Even if ACL is enabled in the broker,  If you don't have ACL for
publish (message transfer) then we don't check ACL at all.
If you have ACL for publish then we do check ACL.
Therefore If you don't have publish ACL then there want be any impact
on performance during message transfers.

I haven't tested ACL with a large amount of entries. Once the file is
parsed we have to keep the data in memory.
So it will likely take a bit of memory as well.
Also such a large amount of entries are definitely going to increase
broker startup (where we parse the file)
Also if your ACL involves publish ACL, then the current mechanism will
likely have a huge performance impact.
(See some of the suggestion below to mitigate this).

> Also, when is the ACL file re-read? If I have a system that adds new users
> all the time, and each user needs a separate ACL rule, how can that be
> expressed?
You can use the management module to reload the ACL file.
However this could affect the performance as the file is going to be huge.
You will likely need to add,delete,modify user privileges, each time
requiring a reload.

So given your situation, I don't think using the file based approach
is a good idea.
You might have to write a custom module. Below are some options.

1.) A fragmented ACL file. So when you add a new user you only read
the fragment that was modified, instead of reloading all million
entries.
2.) You have a custom ACL model which gets updated via a 3rd party
system using a simple RPC call. Ex addUser(...) which
add/delete/modify a user in the memory structures contained within the
ACL abstraction in the broker.
3.) Another option is to partition your users among your broker
instances. That would likely require a load balancing/routing
mechanism (for your connections) which has explicit knowledge about
how you have partitioned your users.

Your use case is very interesting. Please keep us posted with your progress.

>
> Sincerely,
>
> jw
>
>
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world. Nevertheless,
> whether we get there willingly or not, we shall soon have lower consumption
> rates, because our present rates are unsustainable.
>



-- 
Regards,

Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org