Re: [rep:User] autosave when bulk creating users

[Angela Schreiber <an...@adobe.com>]

hi nicolas

On 31/07/15 10:48, "Nicolas Peltier" <np...@adobe.com> wrote:

>Hi,
>
>i¹ve seen that UserManager.autosave was an unsupported operation, and
>thus using LDAP sync with many users generates a save at each user/group
>creation which makes the sync very slow.

not sure i get that.... if autosave was enabled _every_ single
operation on the user management API would trigger a session.save.
isn't that just what you wish to avoid?

btw: autosave is not supported by default, but for backwards compatibility
it can be enabled... but it's really just meant for those cases where
this is inevitable.

>Installing a package of users is much quicker, so i¹m wondering if there
>is a way to create user with an API without saving at each user creation,
>group creation, membership change?

whenever you create or modify a user/group using regular user management
API the changes are not autosaved by default and it's up to the
caller to persist the changes by calling Session.save().

or do you specifically mean the creation of a user in the context of
sync from LDAP (or some other external identity provider)?

>Thanks for your answers,

i guess you need to clarify your question again...

kind regards
angela

>Nicolas
>
>

Re: [rep:User] autosave when bulk creating users

[Nicolas Peltier <np...@adobe.com>]

Hi Angela,

this mail just to say i was wrong at my first analysis to incriminate persistence, looks like i was using isMember before 
adding a user to a group which is #1 useless, and #2 more & more expensive as i was adding users to group.

Nicolas

> On 06 Aug 2015, at 10:56, Angela Schreiber <an...@adobe.com> wrote:
> 
> hi nicolas
> 
>>> i am not too familiar with the sync mechanism... but looking just
>>> quickly at the code it seems that it persists the sync of each
>>> user/group.
>>> that looks reasonable to me (even if it comes with some Root.commit()
>>> overhead) as it allows to specifically retry or revert the changes
>>> made during the sync of one single account. that's the defensive
>>> way of minimising the unexpected behaviour for the consumer in case
>>> of failure.
>>> 
>>> on the other hand you might argue that the default should be success
>>> and that in those cases less frequent calls to persist the changes
>>> might work as well and was probably faster.
>> mmm interesting, yeah i might argue that :-)
> 
> :-)
> 
>> and that we could probably in case of an error ³replay² user/group sync
>> persisting one by one,
>> from the last successful save. Will have a look.
> 
> i don't think this is feasible. you will have to revert everything
> again and start over again. in particular merge conflicts will be
> impossible to sort out once you get them with a lot of different
> changes made.
> 
> but anyway: there is always a balance to find and i am not familiar
> with those very design details in the sync handler... i am just reading the
> code and trying to explain. for sure tobi provide more insight here.
> 
> kind regards
> angela 
> 
>> 
>> Thanks a lot,
>> Nicolas
> 

Re: [rep:User] autosave when bulk creating users

[Angela Schreiber <an...@adobe.com>]

hi nicolas

>>i am not too familiar with the sync mechanism... but looking just
>> quickly at the code it seems that it persists the sync of each
>>user/group.
>> that looks reasonable to me (even if it comes with some Root.commit()
>> overhead) as it allows to specifically retry or revert the changes
>> made during the sync of one single account. that's the defensive
>> way of minimising the unexpected behaviour for the consumer in case
>> of failure.
>> 
>> on the other hand you might argue that the default should be success
>> and that in those cases less frequent calls to persist the changes
>> might work as well and was probably faster.
>mmm interesting, yeah i might argue that :-)

:-)

>and that we could probably in case of an error ³replay² user/group sync
>persisting one by one,
>from the last successful save. Will have a look.

i don't think this is feasible. you will have to revert everything
again and start over again. in particular merge conflicts will be
impossible to sort out once you get them with a lot of different
changes made.

but anyway: there is always a balance to find and i am not familiar
with those very design details in the sync handler... i am just reading the
code and trying to explain. for sure tobi provide more insight here.

kind regards
angela 

>
>Thanks a lot,
>Nicolas

Re: [rep:User] autosave when bulk creating users

[Nicolas Peltier <np...@adobe.com>]

> 
> i am not too familiar with the sync mechanism... but looking just
> quickly at the code it seems that it persists the sync of each user/group.
> that looks reasonable to me (even if it comes with some Root.commit()
> overhead) as it allows to specifically retry or revert the changes
> made during the sync of one single account. that's the defensive
> way of minimising the unexpected behaviour for the consumer in case
> of failure.
> 
> on the other hand you might argue that the default should be success
> and that in those cases less frequent calls to persist the changes
> might work as well and was probably faster.
mmm interesting, yeah i might argue that :-) and that we could probably in case of an error “replay” user/group sync persisting one by one, 
from the last successful save. Will have a look.

Thanks a lot,
Nicolas

Re: [rep:User] autosave when bulk creating users

[Angela Schreiber <an...@adobe.com>]

hi nicolas

>>whenever you create or modify a user/group using regular user management
>> API the changes are not autosaved by default and it's up to the
>> caller to persist the changes by calling Session.save().
>i didn¹t know that, great.

see also
http://jackrabbit.apache.org/oak/docs/security/user.html

>
>> or do you specifically mean the creation of a user in the context of
>> sync from LDAP (or some other external identity provider)?
>i¹m reusing DefaultSyncHandler¹s sync mechanism together with ldap
>identity provider, and was blaming something in there writing too
>frequently. 
>I guess then problem is between oak¹s code & the result i get = between
>my keyboard and my chair.

i am not too familiar with the sync mechanism... but looking just
quickly at the code it seems that it persists the sync of each user/group.
that looks reasonable to me (even if it comes with some Root.commit()
overhead) as it allows to specifically retry or revert the changes
made during the sync of one single account. that's the defensive
way of minimising the unexpected behaviour for the consumer in case
of failure.

on the other hand you might argue that the default should be success
and that in those cases less frequent calls to persist the changes
might work as well and was probably faster.

not sure what the right answer would be here... in any case: it's not
an autosave of the user management implementation but a conscious
decision of the sync-handler to frequently persist the changes :-)

hope that helps

>Thanks Angela for your answers!

you are welcome.
angela

>
>Nicolas
>

Re: [rep:User] autosave when bulk creating users

[Nicolas Peltier <np...@adobe.com>]

> whenever you create or modify a user/group using regular user management
> API the changes are not autosaved by default and it's up to the
> caller to persist the changes by calling Session.save().
i didn’t know that, great.

> or do you specifically mean the creation of a user in the context of
> sync from LDAP (or some other external identity provider)?
i’m reusing DefaultSyncHandler’s sync mechanism together with ldap identity provider, and was blaming something in there writing too frequently. 
I guess then problem is between oak’s code & the result i get = between my keyboard and my chair.

Thanks Angela for your answers!

Nicolas