Large number of groups impacts performance

["EXT-McTaggart, Peter" <pe...@boeing.com>]

Hi,

Env: TDK 2.3/Flux, MySQL 4/Oracle, JDK 1.4.1

In the system I am building I need to have data dependent security, ie
security access to
particular fields etc depends on the the application data in the fields.
I'm using Flux 
to do this.

To accomplish this I am creating a group per Item that the applicationd
deals with
and assigning users that require access to that item into the group.
While I
haven't measured yet I am worried about the performance of this
approach.

here are the numbers:

#roles: 7
#permissions: 50
#groups: 2500 growing by about 1000 per year
#users: 1000

Now in the debug log I see a large number of seemingly identifcal
repetitive SQL calls. And there are
now sizeable delays when logging in and generating screeens etc. Has
anyone implemented turbine with these
orer of numbers , if so, were there any tricks , tips, mods that you
used to improve performance?

Any thoughts or suggestions would be appreciated.

Thanks
Peter

Re: Large number of groups impacts performance

[Brian Lawler <br...@tribenetwork.com>]

We ran into that exact same problem and to fix it, I extended the 
SecurityService to override those bad behaviors.  The main culprit is 
getACL(), which goes through the nightmarish task of selecting all 
groups a few times which became generally displeasurable on our 
application.  I changed this to call getMyGroups(), which is a method I 
added to this service that gets your groups in a more efficient manner. 
  All you have to do then is go to TR.props and change your 
SecurityService to point to your new class and you should be good to 
go...

HTH.

-Brian

++++++

Here is the code that I used:

package com.tribe.security;

import java.util.Hashtable;
import java.util.Iterator;

import org.apache.torque.util.Criteria;

import org.apache.turbine.om.security.Group;
import org.apache.turbine.om.security.Permission;
import org.apache.turbine.om.security.Role;
import org.apache.turbine.om.security.User;
import org.apache.turbine.om.security.RedwoodUser;
import org.apache.turbine.om.security.peer.GroupPeer;
import org.apache.turbine.om.security.peer.PermissionPeer;
import org.apache.turbine.om.security.peer.RolePeer;
import org.apache.turbine.services.security.TurbineSecurity;
import org.apache.turbine.services.security.db.DBSecurityService;
import org.apache.turbine.util.Log;
import org.apache.turbine.util.security.AccessControlList;
import org.apache.turbine.util.security.DataBackendException;
import org.apache.turbine.util.security.GroupSet;
import org.apache.turbine.util.security.PermissionSet;
import org.apache.turbine.util.security.RoleSet;
import org.apache.turbine.util.security.UnknownEntityException;

public class SecurityService extends DBSecurityService {
     public AccessControlList getACL(User user)
             throws DataBackendException, UnknownEntityException
     {
         if (!TurbineSecurity.accountExists(user))
         {
             throw new UnknownEntityException("The account '"
                     + user.getName() + "' does not exist");
         }
         try
         {
             Hashtable roles = new Hashtable();
             Hashtable permissions = new Hashtable();
             // notify the state modifiers (writers) that we want to 
create
             // the snapshot.
             lockShared();

             // construct the snapshot:

             // foreach group in the system
             for (Iterator groupsIterator = getMyGroups(user).elements();
                  groupsIterator.hasNext();)
             {
                 Group group = (Group) groupsIterator.next();
                 // get roles of user in the group
                 RoleSet groupRoles = RolePeer.retrieveSet(user, group);
                 // put the Set into roles(group)
                 roles.put(group, groupRoles);
                 // collect all permissions in this group
                 PermissionSet groupPermissions = new PermissionSet();
                 // foreach role in Set
                 for (Iterator rolesIterator = groupRoles.elements();
                      rolesIterator.hasNext();)
                 {
                     Role role = (Role) rolesIterator.next();
                     // get permissions of the role
                     PermissionSet rolePermissions
                             = PermissionPeer.retrieveSet(role);
                     groupPermissions.add(rolePermissions);
                 }
                 // put the Set into permissions(group)
                 permissions.put(group, groupPermissions);
             }
             return new AccessControlList(roles, permissions);
         }
         catch (Exception e)
         {
             throw new DataBackendException("Failed to build ACL for 
user '"
                     + user.getName() + "'", e);
         }
         finally
         {
             // notify the state modifiers that we are done creating the 
snapshot
             unlockShared();
         }
     }

     private GroupSet getMyGroups(User u) throws DataBackendException,
      UnknownEntityException {
         return ((RedwoodUser)u).getGroups();
     }

     public Group getGroup(String name) throws DataBackendException,
      UnknownEntityException {
         Log.debug("Retrieving a group named: " + name);
         Group result = null;
         Criteria crit = new Criteria();
         crit.add(GroupPeer.NAME, name);
         GroupSet groups = getGroups(crit);
         if(groups.size() == 0) {
             throw new UnknownEntityException("Did not find a group 
named " +
              name);
         } else {
             result = (Group) groups.elements().next();
         }
         return result;
     }

}


On Thursday, November 20, 2003, at 02:04  PM, EXT-McTaggart, Peter 
wrote:

> Hi,
>
> Env: TDK 2.3/Flux, MySQL 4/Oracle, JDK 1.4.1
>
> In the system I am building I need to have data dependent security, ie
> security access to
> particular fields etc depends on the the application data in the 
> fields.
> I'm using Flux
> to do this.
>
> To accomplish this I am creating a group per Item that the applicationd
> deals with
> and assigning users that require access to that item into the group.
> While I
> haven't measured yet I am worried about the performance of this
> approach.
>
> here are the numbers:
>
> #roles: 7
> #permissions: 50
> #groups: 2500 growing by about 1000 per year
> #users: 1000
>
> Now in the debug log I see a large number of seemingly identifcal
> repetitive SQL calls. And there are
> now sizeable delays when logging in and generating screeens etc. Has
> anyone implemented turbine with these
> orer of numbers , if so, were there any tricks , tips, mods that you
> used to improve performance?
>
> Any thoughts or suggestions would be appreciated.
>
> Thanks
> Peter


---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-user-help@jakarta.apache.org