You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Tudor Rogoz <ro...@adobe.com> on 2013/03/26 13:13:01 UTC
ConstraintViolationException on heavy load
Hi,
I get this kind of exception on the majority of the oak nodes when I'm trying to concurrently write into a mongo cluster[0].This seems to happen only with some nodes structures and is not reproducing all the time.
I created a small test that can reproduce the issue, you can find it the attachment.Just make sure that you have a local mongo db running on the 27017 port.
Is this a known issue? Or i'm doing something wrong?
Thanks,
Tudor
[0]
javax.jcr.nodetype.ConstraintViolationException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.jackrabbit.oak.api.CommitFailedException.throwRepositoryException(CommitFailedException.java:57)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:234)
at org.apache.jackrabbit.oak.jcr.SessionImpl.save(SessionImpl.java:320)
at org.apache.jackrabbit.oakmongomk.RepoWrite.run(ConstraintViolationExceptionTest.java:83)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: javax.jcr.nodetype.ConstraintViolationException: Incorrect node type of child node nodeZ442730
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor.constraintViolation(TypeEditor.java:158)
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor.getDefinition(TypeEditor.java:381)
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor.enter(TypeEditor.java:101)
at org.apache.jackrabbit.oak.spi.commit.VisibleEditor.enter(VisibleEditor.java:58)
at org.apache.jackrabbit.oak.spi.commit.CompositeEditor.enter(CompositeEditor.java:66)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.process(EditorHook.java:104)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.process(EditorHook.java:80)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.access$0(EditorHook.java:73)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.childNodeAdded(EditorHook.java:163)
at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:335)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.process(EditorHook.java:109)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.process(EditorHook.java:80)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.access$0(EditorHook.java:73)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.childNodeAdded(EditorHook.java:163)
at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:335)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.process(EditorHook.java:109)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.process(EditorHook.java:80)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.access$0(EditorHook.java:73)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.childNodeAdded(EditorHook.java:163)
at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:335)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.process(EditorHook.java:109)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.process(EditorHook.java:80)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.access$0(EditorHook.java:73)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.childNodeChanged(EditorHook.java:176)
at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:337)
at org.apache.jackrabbit.oak.spi.commit.EditorHook$EditorDiff.process(EditorHook.java:109)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.process(EditorHook.java:80)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:59)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:59)
at org.apache.jackrabbit.oak.kernel.KernelNodeStoreBranch.merge(KernelNodeStoreBranch.java:144)
at org.apache.jackrabbit.oak.core.RootImpl$2.run(RootImpl.java:278)
at org.apache.jackrabbit.oak.core.RootImpl$2.run(RootImpl.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:337)
at org.apache.jackrabbit.oak.core.RootImpl.commit(RootImpl.java:273)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:231)
... 5 more
Caused by: javax.jcr.nodetype.ConstraintViolationException: Incorrect node type of child node nodeZ442730
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor.constraintViolation(TypeEditor.java:159)
... 41 more
Re: ConstraintViolationException on heavy load
Posted by Tudor Rogoz <ro...@adobe.com>.
Thanks Michael for looking on this, I just raised OAK-730
<https://issues.apache.org/jira/browse/OAK-730>
Tudor
On 03/29/2013 04:38 PM, Michael Dürig wrote:
> Hi,
>
> Just to confirm, this reproduces immediately on OSX. My checkout is at
> revision 1462232.
>
> Michael
>
> On 29.3.13 13:50, Tudor Rogoz wrote:
>> Hi,
>>
>> Seems that the issue still reproduces.
>> The microkernels may fail on a local non-clustered mongod instance, too
>> (not only in cluster).
>>
>> So what I did, I hope you'll get the exception on your machine too:
>>
>> - I used the test attached to this email and trigger it 3 times from my
>> IDE.
>> - the test uses a "database barrier" (
>> dbWriter.syncMongos(writersNumber, "syncOAK") ) and waits for all the
>> java processes before starting to write
>> - i put a breakpoint on "e.printStackTrace();" just to catch the exception
>>
>> I was able to reproduce it multiple times with an oak build from Friday
>> (29 march) on both my Ubuntu station, and on a Windows station (on this
>> platform i had to wait longer).
>> Can you please try again with the attached test, and see if reproduces
>> on your station?
>>
>> Tudor
>>
>> On 03/27/2013 04:42 PM, Jukka Zitting wrote:
>>> Hi,
>>>
>>> On Wed, Mar 27, 2013 at 4:28 PM, Tudor Rogoz <ro...@adobe.com> wrote:
>>>> I just made a pull, and is not reproducing on my side, too.It was
>>>> probably
>>>> fixed somehow in the latest commits.
>>> OK, good to know!
>>>
>>> BR,
>>>
>>> Jukka Zitting
Re: ConstraintViolationException on heavy load
Posted by Michael Dürig <md...@apache.org>.
Hi,
Just to confirm, this reproduces immediately on OSX. My checkout is at
revision 1462232.
Michael
On 29.3.13 13:50, Tudor Rogoz wrote:
> Hi,
>
> Seems that the issue still reproduces.
> The microkernels may fail on a local non-clustered mongod instance, too
> (not only in cluster).
>
> So what I did, I hope you'll get the exception on your machine too:
>
> - I used the test attached to this email and trigger it 3 times from my
> IDE.
> - the test uses a "database barrier" (
> dbWriter.syncMongos(writersNumber, "syncOAK") ) and waits for all the
> java processes before starting to write
> - i put a breakpoint on "e.printStackTrace();" just to catch the exception
>
> I was able to reproduce it multiple times with an oak build from Friday
> (29 march) on both my Ubuntu station, and on a Windows station (on this
> platform i had to wait longer).
> Can you please try again with the attached test, and see if reproduces
> on your station?
>
> Tudor
>
> On 03/27/2013 04:42 PM, Jukka Zitting wrote:
>> Hi,
>>
>> On Wed, Mar 27, 2013 at 4:28 PM, Tudor Rogoz <ro...@adobe.com> wrote:
>>> I just made a pull, and is not reproducing on my side, too.It was
>>> probably
>>> fixed somehow in the latest commits.
>> OK, good to know!
>>
>> BR,
>>
>> Jukka Zitting
>
Re: ConstraintViolationException on heavy load
Posted by Tudor Rogoz <ro...@adobe.com>.
Hi,
Seems that the issue still reproduces.
The microkernels may fail on a local non-clustered mongod instance, too (not only in cluster).
So what I did, I hope you'll get the exception on your machine too:
- I used the test attached to this email and trigger it 3 times from my IDE.
- the test uses a "database barrier" ( dbWriter.syncMongos(writersNumber, "syncOAK") ) and waits for all the java processes before starting to write
- i put a breakpoint on "e.printStackTrace();" just to catch the exception
I was able to reproduce it multiple times with an oak build from Friday (29 march) on both my Ubuntu station, and on a Windows station (on this platform i had to wait longer).
Can you please try again with the attached test, and see if reproduces on your station?
Tudor
On 03/27/2013 04:42 PM, Jukka Zitting wrote:
> Hi,
>
> On Wed, Mar 27, 2013 at 4:28 PM, Tudor Rogoz <ro...@adobe.com> wrote:
>> I just made a pull, and is not reproducing on my side, too.It was probably
>> fixed somehow in the latest commits.
> OK, good to know!
>
> BR,
>
> Jukka Zitting
Re: ConstraintViolationException on heavy load
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Wed, Mar 27, 2013 at 4:28 PM, Tudor Rogoz <ro...@adobe.com> wrote:
> I just made a pull, and is not reproducing on my side, too.It was probably
> fixed somehow in the latest commits.
OK, good to know!
BR,
Jukka Zitting
Re: ConstraintViolationException on heavy load
Posted by Tudor Rogoz <ro...@adobe.com>.
On 03/27/2013 01:33 PM, Jukka Zitting wrote:
> Hi,
>
> On Tue, Mar 26, 2013 at 4:22 PM, Tudor Rogoz <ro...@adobe.com> wrote:
>> The attached tests (from the initial mail) will reproduce the issue on a
>> single mongo instance.
> I tried running the test (without modifications, accessing a local
> non-clustered mongod instance) a few times (killing it after ten or so
> minutes) but couldn't yet reproduce the problem. Do you still see it
> with the latest trunk?
I just made a pull, and is not reproducing on my side, too.It was probably fixed somehow in the latest commits.
Thanks,
Tudor
> BR,
>
> Jukka Zitting
Re: ConstraintViolationException on heavy load
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Tue, Mar 26, 2013 at 4:22 PM, Tudor Rogoz <ro...@adobe.com> wrote:
> The attached tests (from the initial mail) will reproduce the issue on a
> single mongo instance.
I tried running the test (without modifications, accessing a local
non-clustered mongod instance) a few times (killing it after ten or so
minutes) but couldn't yet reproduce the problem. Do you still see it
with the latest trunk?
BR,
Jukka Zitting
Re: ConstraintViolationException on heavy load
Posted by Tudor Rogoz <ro...@adobe.com>.
On 03/26/2013 02:56 PM, Jukka Zitting wrote:
> Hi,
>
> On Tue, Mar 26, 2013 at 2:13 PM, Tudor Rogoz <ro...@adobe.com> wrote:
>> I get this kind of exception on the majority of the oak nodes when I'm
>> trying to concurrently write into a mongo cluster[0].This seems to happen
>> only with some nodes structures and is not reproducing all the time.
> Does this only happen on a MongoDB cluster, or also with a single Mongo node?
The attached tests (from the initial mail) will reproduce the issue on a
single mongo instance.There, I have 2 oaks that are concurrently writing
into the same mongo database.
In the cluster, the problem reproduces too, only that the oak instances
are running on different mongo instances.So, the problem is not cluster
related.
More than that, the oak instances are not failing directly, all of them
are performing several saves into the database before crashing.
Tudor
> The fact that the problem is non-deterministic makes it sound like a
> MongoMK issue (perhaps a non-uniform view of the node type registry?)
> rather than a problem with the type validator (whose recent changes
> might well have triggered this issue).
>
> BR,
>
> Jukka Zitting
Re: ConstraintViolationException on heavy load
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Tue, Mar 26, 2013 at 2:13 PM, Tudor Rogoz <ro...@adobe.com> wrote:
> I get this kind of exception on the majority of the oak nodes when I'm
> trying to concurrently write into a mongo cluster[0].This seems to happen
> only with some nodes structures and is not reproducing all the time.
Does this only happen on a MongoDB cluster, or also with a single Mongo node?
The fact that the problem is non-deterministic makes it sound like a
MongoMK issue (perhaps a non-uniform view of the node type registry?)
rather than a problem with the type validator (whose recent changes
might well have triggered this issue).
BR,
Jukka Zitting