You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Tobias Bocanegra <tr...@apache.org> on 2014/03/14 01:06:13 UTC
concurrent creation of users
Hi,
I tried to create users concurrently, and I get a constraint violation:
"Cannot create user/group: Intermediate folders must be of type
rep:AuthorizableFolder."
on the "addExistingNode" parent, because the User validator traverses
up the following tree:
"/rep:security/rep:authorizables/rep:users/:conflict/addExistingNode/u/us/user-5"
How should this work? should the user validator stop on the
":conflict" node? btw: there is never a real conflict here. all nodes
are the same.
regards, toby
Re: concurrent creation of users
Posted by Michael Dürig <md...@apache.org>.
On 17.3.14 10:46 , Michael Dürig wrote:
>
> There indeed seem to be differences in the way the such conflicts are
> handled by the different implementations. I'll follow up with respective
> issues.
See https://issues.apache.org/jira/browse/OAK-1549 and
https://issues.apache.org/jira/browse/OAK-1550.
Michael
Re: concurrent creation of users
Posted by Michael Dürig <md...@apache.org>.
On 16.3.14 6:34 , Tobias Bocanegra wrote:
> Hi,
>
>
> On Sat, Mar 15, 2014 at 8:02 AM, Michael Dürig <md...@apache.org> wrote:
>> Ah I see. The respective validators are missing from the test setup. But
>> maybe this was done intentionally? I can't really tell. However If I add the
>> and run the test I get stack traces that resemble more what I would expect:
>
> yes. but the change is trivial. there are 2 commits adding the same
> childnode to a parent. the childnode is identical in name, properties
> and type, and is not referenceable.
There indeed seem to be differences in the way the such conflicts are
handled by the different implementations. I'll follow up with respective
issues.
Michael
>
> this is a typical use-case in highly concurrent systems, so I think
> that we should find a solution that can merge trivial changes.
> regards, toby
>
Re: concurrent creation of users
Posted by Michael Dürig <md...@apache.org>.
On 17.3.14 7:22 , Tobias Bocanegra wrote:
> well, then the merge resolver is not clever enough. the node 'f' is
> identical, it got just added different child nodes.
>
> IMO:
>
> A) f.add("fo").add(u1); commit();
> B) f.add("fo").add(u2); commit();
>
> must work for any concurrent execution order.
> regards, toby
I created https://issues.apache.org/jira/browse/OAK-1553 to track this.
Michael
Re: concurrent creation of users
Posted by Tobias Bocanegra <tr...@apache.org>.
On Mon, Mar 17, 2014 at 3:28 AM, Michael Dürig <md...@apache.org> wrote:
> I'm however also seeing real conflicts in that test case like:
>
> +"/rep:security/rep:authorizables/rep:users/f":{
> "jcr:primaryType":"nam:rep:AuthorizableFolder",
> "fo":{
> "jcr:primaryType":"nam:rep:AuthorizableFolder",
> "foo-user-2":{
>
> "rep:password":"{SHA-256}f65757d780eb4850-1000-f3c2a571c41ff56f879ccc94ae151012d9b9c2bf970a0407f982ad06df1a043e",
> "rep:principalName":"foo-user-2",
> "rep:authorizableId":"foo-user-2",
> "jcr:uuid":"651a0628-4ed9-3db8-b98e-ca6dcd92f615",
> "jcr:primaryType":"nam:rep:User"}}}
>
>
> +"/rep:security/rep:authorizables/rep:users/f":{
> "jcr:primaryType":"nam:rep:AuthorizableFolder",
> "fo":{
> "jcr:primaryType":"nam:rep:AuthorizableFolder",
> "foo-user-1":{
>
> "rep:password":"{SHA-256}5080678fba260f84-1000-a76311735279849147b390c96f6f8a0f92736ca25991d88349fb26b1c17578d1",
> "rep:principalName":"foo-user-1",
> "rep:authorizableId":"foo-user-1",
> "jcr:primaryType":"nam:rep:User",
> "jcr:uuid":"95427c0c-ac0b-3f4c-a6fe-a25d8781d263"}}}
>
> This causes a conflict since the new node f is not the same in both cases.
> See the Javadoc of org.apache.jackrabbit.mk.api.MicroKernel#rebase for what
> constitutes a conflict.
well, then the merge resolver is not clever enough. the node 'f' is
identical, it got just added different child nodes.
IMO:
A) f.add("fo").add(u1); commit();
B) f.add("fo").add(u2); commit();
must work for any concurrent execution order.
regards, toby
>
> Michael
>
>
>>
>> this is a typical use-case in highly concurrent systems, so I think
>> that we should find a solution that can merge trivial changes.
>> regards, toby
>>
>
Re: concurrent creation of users
Posted by Michael Dürig <md...@apache.org>.
On 16.3.14 6:34 , Tobias Bocanegra wrote:
> Hi,
>
>
> On Sat, Mar 15, 2014 at 8:02 AM, Michael Dürig <md...@apache.org> wrote:
>> Ah I see. The respective validators are missing from the test setup. But
>> maybe this was done intentionally? I can't really tell. However If I add the
>> and run the test I get stack traces that resemble more what I would expect:
>
> yes. but the change is trivial. there are 2 commits adding the same
> childnode to a parent. the childnode is identical in name, properties
> and type, and is not referenceable.
I'm however also seeing real conflicts in that test case like:
+"/rep:security/rep:authorizables/rep:users/f":{
"jcr:primaryType":"nam:rep:AuthorizableFolder",
"fo":{
"jcr:primaryType":"nam:rep:AuthorizableFolder",
"foo-user-2":{
"rep:password":"{SHA-256}f65757d780eb4850-1000-f3c2a571c41ff56f879ccc94ae151012d9b9c2bf970a0407f982ad06df1a043e",
"rep:principalName":"foo-user-2",
"rep:authorizableId":"foo-user-2",
"jcr:uuid":"651a0628-4ed9-3db8-b98e-ca6dcd92f615",
"jcr:primaryType":"nam:rep:User"}}}
+"/rep:security/rep:authorizables/rep:users/f":{
"jcr:primaryType":"nam:rep:AuthorizableFolder",
"fo":{
"jcr:primaryType":"nam:rep:AuthorizableFolder",
"foo-user-1":{
"rep:password":"{SHA-256}5080678fba260f84-1000-a76311735279849147b390c96f6f8a0f92736ca25991d88349fb26b1c17578d1",
"rep:principalName":"foo-user-1",
"rep:authorizableId":"foo-user-1",
"jcr:primaryType":"nam:rep:User",
"jcr:uuid":"95427c0c-ac0b-3f4c-a6fe-a25d8781d263"}}}
This causes a conflict since the new node f is not the same in both
cases. See the Javadoc of
org.apache.jackrabbit.mk.api.MicroKernel#rebase for what constitutes a
conflict.
Michael
>
> this is a typical use-case in highly concurrent systems, so I think
> that we should find a solution that can merge trivial changes.
> regards, toby
>
Re: concurrent creation of users
Posted by Tobias Bocanegra <tr...@apache.org>.
Hi,
On Sat, Mar 15, 2014 at 8:02 AM, Michael Dürig <md...@apache.org> wrote:
> Ah I see. The respective validators are missing from the test setup. But
> maybe this was done intentionally? I can't really tell. However If I add the
> and run the test I get stack traces that resemble more what I would expect:
yes. but the change is trivial. there are 2 commits adding the same
childnode to a parent. the childnode is identical in name, properties
and type, and is not referenceable.
this is a typical use-case in highly concurrent systems, so I think
that we should find a solution that can merge trivial changes.
regards, toby
Re: concurrent creation of users
Posted by Michael Dürig <md...@apache.org>.
On 14.3.14 11:22 , Tobias Bocanegra wrote:
> Hi,
>
> On Fri, Mar 14, 2014 at 2:43 PM, Michael Dürig <md...@apache.org> wrote:
>>
>>
>> On 14.3.14 1:06 , Tobias Bocanegra wrote:
>>>
>>> Hi,
>>>
>>> I tried to create users concurrently, and I get a constraint violation:
>>>
>>> "Cannot create user/group: Intermediate folders must be of type
>>> rep:AuthorizableFolder."
>>>
>>> on the "addExistingNode" parent, because the User validator traverses
>>> up the following tree:
>>>
>>>
>>> "/rep:security/rep:authorizables/rep:users/:conflict/addExistingNode/u/us/user-5"
>>
>>
>> This is already strange.
>> o.a.j.o.plugins.commit.JcrConflictHandler#JCR_CONFLICT_HANDLER should run
>> before any other hook (see o.a.j.o..jcr.Jcr#Jcr(Oak)). This conflict handler
>> replaces the :conflict markers with rep:MergeConflict markers. Later in the
>> chain of hooks o.a.j.o..plugins.commit.ConflictValidatorProvider fails the
>> commit when there are unresolved conflicts. I.e. when there are
>> rep:MergeConflict markers left.
> well, this is an oak-core test where there is no JCR. I tried adding
> the JcrConflictHandler but then I get NPEs.
Ah I see. The respective validators are missing from the test setup. But
maybe this was done intentionally? I can't really tell. However If I add
the and run the test I get stack traces that resemble more what I would
expect:
org.apache.jackrabbit.oak.api.CommitFailedException: OakState0001:
Unresolved conflicts in /rep:security/rep:authorizables/rep:users
at
org.apache.jackrabbit.oak.plugins.commit.ConflictValidator.failOnMergeConflict(ConflictValidator.java:84)
at
org.apache.jackrabbit.oak.plugins.commit.ConflictValidator.propertyAdded(ConflictValidator.java:54)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.propertyAdded(CompositeEditor.java:83)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.propertyAdded(EditorDiff.java:82)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:375)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
at
org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
at
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeStore$MemoryNodeStoreBranch.merge(MemoryNodeStore.java:217)
at
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeStore.merge(MemoryNodeStore.java:96)
at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:242)
at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:253)
at
org.apache.jackrabbit.oak.security.user.UserManagerImplTest$1.run(UserManagerImplTest.java:267)
Here is what I did:
@@ -81,12 +83,14 @@
public void before() throws Exception {
Oak oak = new Oak()
.with(new InitialContent())
+ .with(JcrConflictHandler.JCR_CONFLICT_HANDLER)
.with(new NamespaceEditorProvider())
.with(new ReferenceEditorProvider())
.with(new ReferenceIndexProvider())
.with(new PropertyIndexEditorProvider())
.with(new PropertyIndexProvider())
.with(new TypeEditorProvider())
+ .with(new ConflictValidatorProvider())
.with(getSecurityProvider());
withEditors(oak);
contentRepository = oak.createContentRepository();
Michael
>
>
>>> How should this work? should the user validator stop on the
>>> ":conflict" node? btw: there is never a real conflict here. all nodes
>>> are the same.
>>
>>
>> I vaguely remember that we might have though this to be too expensive, but I
>> might be wrong.
>> The AbstractRebaseDiff (which is the in memory version for conflict
>> handling) certainly behaves as Toby describes. Jukka, Marcel what about the
>> respective node store implementations? We might even have different
>> behaviours here depending on who does the diffing.
>>
>
> regards, toby
>
Re: concurrent creation of users
Posted by Tobias Bocanegra <tr...@apache.org>.
Hi,
On Fri, Mar 14, 2014 at 2:43 PM, Michael Dürig <md...@apache.org> wrote:
>
>
> On 14.3.14 1:06 , Tobias Bocanegra wrote:
>>
>> Hi,
>>
>> I tried to create users concurrently, and I get a constraint violation:
>>
>> "Cannot create user/group: Intermediate folders must be of type
>> rep:AuthorizableFolder."
>>
>> on the "addExistingNode" parent, because the User validator traverses
>> up the following tree:
>>
>>
>> "/rep:security/rep:authorizables/rep:users/:conflict/addExistingNode/u/us/user-5"
>
>
> This is already strange.
> o.a.j.o.plugins.commit.JcrConflictHandler#JCR_CONFLICT_HANDLER should run
> before any other hook (see o.a.j.o..jcr.Jcr#Jcr(Oak)). This conflict handler
> replaces the :conflict markers with rep:MergeConflict markers. Later in the
> chain of hooks o.a.j.o..plugins.commit.ConflictValidatorProvider fails the
> commit when there are unresolved conflicts. I.e. when there are
> rep:MergeConflict markers left.
well, this is an oak-core test where there is no JCR. I tried adding
the JcrConflictHandler but then I get NPEs.
>> How should this work? should the user validator stop on the
>> ":conflict" node? btw: there is never a real conflict here. all nodes
>> are the same.
>
>
> I vaguely remember that we might have though this to be too expensive, but I
> might be wrong.
> The AbstractRebaseDiff (which is the in memory version for conflict
> handling) certainly behaves as Toby describes. Jukka, Marcel what about the
> respective node store implementations? We might even have different
> behaviours here depending on who does the diffing.
>
regards, toby
Re: concurrent creation of users
Posted by Michael Dürig <md...@apache.org>.
On 14.3.14 1:06 , Tobias Bocanegra wrote:
> Hi,
>
> I tried to create users concurrently, and I get a constraint violation:
>
> "Cannot create user/group: Intermediate folders must be of type
> rep:AuthorizableFolder."
>
> on the "addExistingNode" parent, because the User validator traverses
> up the following tree:
>
> "/rep:security/rep:authorizables/rep:users/:conflict/addExistingNode/u/us/user-5"
This is already strange.
o.a.j.o.plugins.commit.JcrConflictHandler#JCR_CONFLICT_HANDLER should
run before any other hook (see o.a.j.o..jcr.Jcr#Jcr(Oak)). This conflict
handler replaces the :conflict markers with rep:MergeConflict markers.
Later in the chain of hooks
o.a.j.o..plugins.commit.ConflictValidatorProvider fails the commit when
there are unresolved conflicts. I.e. when there are rep:MergeConflict
markers left.
>
> How should this work? should the user validator stop on the
> ":conflict" node? btw: there is never a real conflict here. all nodes
> are the same.
I vaguely remember that we might have though this to be too expensive,
but I might be wrong.
The AbstractRebaseDiff (which is the in memory version for conflict
handling) certainly behaves as Toby describes. Jukka, Marcel what about
the respective node store implementations? We might even have different
behaviours here depending on who does the diffing.
Michael
>
> regards, toby
>
Re: concurrent creation of users
Posted by Tobias Bocanegra <tr...@apache.org>.
Hi,
I created https://issues.apache.org/jira/browse/OAK-1541 and added an
additional test that shows the same problem.
it looks like the 'UserValidator' is executed before the merge
conflict is resolved. As I don't fully understand how it should work,
maybe someone that implemented the merging could take a look at this.
thanks.
regards, toby
On Thu, Mar 13, 2014 at 5:06 PM, Tobias Bocanegra <tr...@apache.org> wrote:
> Hi,
>
> I tried to create users concurrently, and I get a constraint violation:
>
> "Cannot create user/group: Intermediate folders must be of type
> rep:AuthorizableFolder."
>
> on the "addExistingNode" parent, because the User validator traverses
> up the following tree:
>
> "/rep:security/rep:authorizables/rep:users/:conflict/addExistingNode/u/us/user-5"
>
> How should this work? should the user validator stop on the
> ":conflict" node? btw: there is never a real conflict here. all nodes
> are the same.
>
> regards, toby