Re: Scalability - large numbers of users/groups in LDAP

[Don Bosco Durai <bo...@apache.org>]

Seems you are suggesting two scenarios.

1.    Ranger should have an option just to sync Group (without users). We should be already supporting it or there was an intention to support.  If we are not doing it for any reason, I am a strong +1 for doing it. 
2.    Direct LDAP would have been ideal, but we were worried about the load we might put on LDAP for real-time queries. Just FYI, Ranger uses LDAP/AD for authentication and easy selection of users/groups during policy create. For authentication, it is already real-time (even though I would have preferred to get the roles also in real-time). 

If you have a very high number of users/groups, then the short-term recommendation to is to apply LDAP filters and limit syncing users only to those using Hadoop.

Thanks

Bosco


On 2/10/17, 6:20 AM, "Nigel Jones" <jo...@uk.ibm.com> wrote:

    On 10/02/2017 09:58, Velmurugan Periasamy wrote:
     > Hi Nigel:
     >
     > Thanks for starting an interesting thread.
    
     > I believe this is already addressed by 
    https://issues.apache.org/jira/browse/RANGER-869. Please take a look.
    
    I took a look - indeed I had noticed this option to go via groups and 
    lookup "member" which does mitigate the issue somewhat, depending on the 
    number of groups
    
    In the environment I'm thinking of I can probably find an "interesting" 
    list of groups. So I could modify usersync to not just use the 
    group->member lookup, but also to ONLY do that for certain groups (I'll 
    probably need "groupsync" for that... !)
    
    Whether this work depends on how the ldap server is set up... I need to 
    take a look.. if so this is probably good enough for now.
    
    But I'm still wondering if we really need to sync users at all since at 
    some point any kind of connector/engine may well be doing an ldap lookup 
    anyway - certainly that's true in an engine -- Apache Derby based - that 
    I'm looking at (and developing a plugin for). This may become more 
    important for large numbers of groups and users especially if we 
    consider applying ranger plugins to technologies used by a broad set of 
    users.
    
    Out of interest I just noticed in the nifi mailing lists that there was 
    a recent thread on "LDAP Group Authorization". There is some discussion 
    of native nifi+ranger, but in either case the question about why not get 
    the info direct from ldap at connect time is being made. intriguing ...
    
    Thanks for the link ... mulling over some more :-)
    
    nigel.