Handling a large number of dynamic permissions

[jmcginn13 <jo...@railinc.com>]

Hi,

My scenario has a large number of rapidly changing permissions based on the
data in my domain model. We also have a large and growing set of
rules/policies for determining user permissions. The set of data objects and
number of permissions is in the order of tens of millions. In some cases we
need permissions down to the field level. The domain model changes at a high
enough velocity (100's of thousands of updates / day at a minimum) that
pre-calculating and storing the permissions doesn't seem very feasible. What
I want to do to solve this is the following:

Store in a in memory cache my identity database (the elements for my domain
objects needed for permission determination)
Encode my security policies/rules in business rules (i.e. Drools)
When a user requests access to an entity I want to:
* Load my object identity from my database
* Execute my security ruleset to generate the applicable permissions for
that user for the object being requested
* Allow the client system use the "isPermitted" methods to see if the user
has permission and react accordingly.

A concrete example:
The domain model is railcars. A limited example would be something like:
Car ID - UP-123456
Current Owner - GATX
Current Handling Carrier - BNSF
Current Repair Shop - NULL
Current Shipper - DOWX
Current Consignee - ECF
Damaged/Defect Flag - N
Component Recall Flag - N
WILD Alert Flag - Y
... and so on

The total data size is roughly 1.8 million railcars. There are other domain
objects I'll be adding to this as well that have even larger data sets.

Some example policies:
If user is owner can see everything for car
If user is handling carrier can see certain health records on the car
If car is damaged and user is handling carrier can see everything on car and
can update disposition on car
...

Now to my problem. The dynamic permission generation requires both the user
(PrincipalCollection) and the object being requested so that the
doGetAuthorizationInfo of my realm can populate the rule engine with the
objects it needs. My first thought is to add the object being requested to
the principal collection, but I'm not sure what would be the best part of
the API to extend to do this. Does anyone have experience with a similar
scenario they can share with me?

Thanks in advance

John



--
View this message in context: http://shiro-user.582556.n2.nabble.com/Handling-a-large-number-of-dynamic-permissions-tp7578918.html
Sent from the Shiro User mailing list archive at Nabble.com.

RE: Handling a large number of dynamic permissions

[jmcginn13 <jo...@railinc.com>]

Hi Ken, 

Since I created my own realm from scratch and implemented Realm and
Authorizer, but didn't extend AuthorizingRealm I don't have doGet***
methods. Also since I'm dealing with preauthenticated users via my own SSO
system (CA SiteMinder) I don't actually do authentication in my authorizing
realm. I've attached the custom realm below.

If you notice I implemented my own getAuthorizationInfo method in the realm,
this is where the permission loader is called. I had to do this because
AuthorizingRealm's method only accepted PrincipalCollection and I needed
both PrincipalCollection and the permission being requested. All of the
isPermitted and hasRole methods eventually invoke that method.

You can notice the authentication methods at the bottom are basically
disabled. I have a separate realm that "authenticates" my preauthenticated
user. Basically its main job is to convert my SSO user object into a Shiro
AuthenticationInfo. I've attached that realm further down.

Our SSO system has roles, and has been extended so it can have roles by
company as we have multi-tenant systems with users from many different
companies. If you notice in the example rule above the hasRole method is
checking to see if the user has a certain role for the company that matches
the company that owns the railcar. The SSO roles are converted into roles in
the AssignableAuthorizationInfo object when I initially build it. It would
be possible for the security rules to add additional roles to this (it has
an addRole method). Then a RolePermissionResolver could be used to convert
the role into the correct permissions. With our initial work though we have
just added permissions directly and not dealt with roles in this way.

The authorizing permission loading realm
DynamicPermissionAuthorizingRealm.java
<http://shiro-user.582556.n2.nabble.com/file/n7578987/DynamicPermissionAuthorizingRealm.java>  

The "authenticating" realm
SsoPreauthenticatedRealm.java
<http://shiro-user.582556.n2.nabble.com/file/n7578987/SsoPreauthenticatedRealm.java>  

What do you mean by a tech sheet? Do you have an example?



--
View this message in context: http://shiro-user.582556.n2.nabble.com/Handling-a-large-number-of-dynamic-permissions-tp7578918p7578987.html
Sent from the Shiro User mailing list archive at Nabble.com.

RE: Handling a large number of dynamic permissions

[Ken in Nashua <kc...@live.com>]

Cool...can you show us your final java module...
the one that calls 

doGetAuthentication
doGetAuthorization

That would be terrific.

And first I seen these KINDS of expressions expressed since I was wrestling with the same thing last February.

Actually a tech sheet would be terrific if you could post one on dynamic perms.

Roles too right?

 		 	   		  

Re: Handling a large number of dynamic permissions

[jmcginn13 <jo...@railinc.com>]

I wanted to give a follow up on how I solved this, and thanks Jared for the
input. I ended up going pretty much with your design.

I created my own realm for dynamically generating permissions at runtime as
they are requested. To do this I created some generic typed entity and
entity loader interfaces:
public interface PermissibleEntity  
and
public interface PermissibleEntityLoader <I extends Serializable, E extends
PermissibleEntity&lt;I>>

"Security providers" can code implementations of these for specific entities
that need permissions (e.g. RailcarEntity and RailcarEntityLoader). The
loader is meant to go to the source data and get any additional data needed
for permission determination (e.g. Railcar ownership information).

Central to this is my own custom Permission object that contains the entity
object and a separate Permission object to contain the rest of the
permission request. This is almost always WildcardPermission.

Then I abstracted out the actual permission generation with an interface:
public interface EntityPermissionLoader <I extends Serializable, E extends
PermissibleEntity&lt;I>>
  AssignableAuthorizationInfo loadEntityPermissions(PrincipalCollection
principals, E entity);

AssignableAuthorizationInfo is my own interface that extends
AuthorizationInfo to allow writing to the permission collections.

I provided a Drools based implementation of this interface in the framework.
It expects to be configured with a valid KnowledgeBase that has the security
rules written in them. It inserts the loaded entity that permissions are
being requested for, the principals, and the AssignableAuthorizationInfo and
fires the rules.

An example rule looks like:
rule "GiveConsistManagerUmlerOwnerViewAllPermission"
	when
		$eq : PermissibleEquipmentEntity($umow : umlerCarOwner)
		$user : AssignableAuthorizationInfo(hasRole("AHSICONMGR/" + $umow))
	then
		$user.addStringPermission("equipment:" + $eq.getEntityId() +
":health:*:view");
end		

This gives a user with the AHSICONMGR role for the company that is the
railcar owner (umow) permissions to see all health related data for that
railcar. 

My custom realm uses its configured EntityPermissionLoader for any of the
isPermitted type methods.

Caching is intended for the EntityLoaders via an abstract base class and for
the EntityPermissionLoader. I cached the generated
AssignableAuthorizationInfo along with the IDs of all entities that have
been already generated. Entities know when they are expired so they can be
reloaded and the permissions regenerated. I had to do a synchronized block
on the user to make it thread safe.

I have already implemented two domains with this model (railcars and train
events). I packaged them in a separate modular maven project from my core
api. Each entity type gets a module. These are pretty small with the entity
object itself, a entity loader if needed, and a permission resolver.

Overall the core api is pretty small with a few key interfaces, some base
abstract classes, the Drools implementation for permission "loading" and my
custom realm to use it all. There are also a few classes for converting our
preauthenticated SSO user object into a shiro subject and that's it. Then
each domain only has a handful of classes to implement the key interfaces.
Then there is a centralized Drools project to contain the business rules.

My clients just need to configure the domain and its required dependencies
(usually with spring config) then call something like:
SecurityUtils.getSubject().isPermitted("equipment:BNSF1:health:repairs:view")
...

If everything is configured correctly this will:
* Parse the string via the resolver into my EntityPermission object. The
equipment token identifies the entity type. The BNSF1 token is parsed into
the entity ID using our Railcar Equipment ID Parser. The rest is created as
a wildcard permission (health:repairs:view)
* Load the railcar data for ID BNSF1 from the database
* Execute the security rules (Drools) with the loaded railcar data and the
user's roles to populate all permissions for the user related to railcar
BNSF1
* Finally do the typical permission implies calls on the user's permissions
until true is found and return the result.

The key object model looks like:
<http://shiro-user.582556.n2.nabble.com/file/n7578985/shiro-security-authorization-class-model.jpg> 



--
View this message in context: http://shiro-user.582556.n2.nabble.com/Handling-a-large-number-of-dynamic-permissions-tp7578918p7578985.html
Sent from the Shiro User mailing list archive at Nabble.com.

Re: Handling a large number of dynamic permissions

[Jared Bunting <ja...@peachjean.com>]

I looked into doing this once when I was told that our domain was this
size.  (I was a new employee and later discovered that it wasn't this
large).

My first thought is that you should avoid thinking about
doGetAuthorizationInfo and implement isPermitted on the *Realm* yourself.
 This will avoid you having to recalculate *everything* all the time,
although you will have to do your own caching.

My second thought is that trying to add the requested object to the
PrincipalCollection is a bad idea.  This isn't what the PrincipalCollection
is meant for, and I foresee concurrency and data consistency issues.
    On addressing the issue of needing the domain object, my first approach
would be to encode an "id" in the permission string, and have my Realm do
the same lookup - if there's a cache for these lookups that's shared
between the Realm and your user code, then the extra work should be
trivial.  That would look something like this: "read:UP-123456:*",
"read:UP-123456:health-records", "update:UP-123456:disposition".  (note
that i tried to use your examples w/o knowing very much at all about your
domain)
    If this is not feasible, you could implement your own Permission class
- "RailcarPermission" for example.  Then you could do:
SecurityManager.getSubject().isPermitted(new RailcarPermission("view",
myRailcarObject, "health-records")).

I believe that encoding the ID in the permission string will give you more
flexibility in the future, but which direction to go is really up to you.

-Jared


On Wed, Jul 17, 2013 at 10:48 AM, jmcginn13 <jo...@railinc.com> wrote:

> Hi,
>
> My scenario has a large number of rapidly changing permissions based on the
> data in my domain model. We also have a large and growing set of
> rules/policies for determining user permissions. The set of data objects
> and
> number of permissions is in the order of tens of millions. In some cases we
> need permissions down to the field level. The domain model changes at a
> high
> enough velocity (100's of thousands of updates / day at a minimum) that
> pre-calculating and storing the permissions doesn't seem very feasible.
> What
> I want to do to solve this is the following:
>
> Store in a in memory cache my identity database (the elements for my domain
> objects needed for permission determination)
> Encode my security policies/rules in business rules (i.e. Drools)
> When a user requests access to an entity I want to:
> * Load my object identity from my database
> * Execute my security ruleset to generate the applicable permissions for
> that user for the object being requested
> * Allow the client system use the "isPermitted" methods to see if the user
> has permission and react accordingly.
>
> A concrete example:
> The domain model is railcars. A limited example would be something like:
> Car ID - UP-123456
> Current Owner - GATX
> Current Handling Carrier - BNSF
> Current Repair Shop - NULL
> Current Shipper - DOWX
> Current Consignee - ECF
> Damaged/Defect Flag - N
> Component Recall Flag - N
> WILD Alert Flag - Y
> ... and so on
>
> The total data size is roughly 1.8 million railcars. There are other domain
> objects I'll be adding to this as well that have even larger data sets.
>
> Some example policies:
> If user is owner can see everything for car
> If user is handling carrier can see certain health records on the car
> If car is damaged and user is handling carrier can see everything on car
> and
> can update disposition on car
> ...
>
> Now to my problem. The dynamic permission generation requires both the user
> (PrincipalCollection) and the object being requested so that the
> doGetAuthorizationInfo of my realm can populate the rule engine with the
> objects it needs. My first thought is to add the object being requested to
> the principal collection, but I'm not sure what would be the best part of
> the API to extend to do this. Does anyone have experience with a similar
> scenario they can share with me?
>
> Thanks in advance
>
> John
>
>
>
> --
> View this message in context:
> http://shiro-user.582556.n2.nabble.com/Handling-a-large-number-of-dynamic-permissions-tp7578918.html
> Sent from the Shiro User mailing list archive at Nabble.com.
>