Invalid item state while adding nodes on different sessions on multi thread environment

[Luiz Fernando Teston <fe...@caravelatech.com>]

Hi,

I just found an strange behavior while using multiple sessions on multi
thread environment. I'd like to know if I did something wrong.
My application code do this:
1 - before start to work, it creates a root node inside a new session. It
saves and close this session.
2 - create a bunch of workers. Each worker has its own session.
3 - each worker adds nodes inside the root node retrieved by its session. It
saves and do a logout properly.
4 - fill this workers on a list of Callables to be passed to an java 5
ThreadPoolExecutor.
5 - it executes all workers on the ThreadPool.
6 - it do some asserts to verify if it gots some errors or not.

During phase 5 it gots an exception telling this:

javax.jcr.InvalidItemStateException: Item cannot be saved because it has
been modified externally: node /
at org.apache.jackrabbit.core.ItemImpl.getTransientStates(ItemImpl.java:245)
at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:939)
at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:915)

I just wrote a test to simulate this behavior. Sometimes it got this error
sometimes not. Also to be easy to simulate this error, just change the
THREAD_SIZE and NODE_SIZE variables.

I'm very thankful for your help!


Fernando Teston

---- test code -----

package org.openspotlight.jcr.provider.test;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import javax.jcr.Node;
import javax.jcr.Session;

import org.junit.Assert;
import org.junit.Test;

public class MultithreadedSessionTest {

enum Status {
OK, ERROR
}

class Worker implements Callable<Status> {

private final Session session;
private final int i;

public Worker(final Session session, final int i) {
this.session = session;
this.i = i;
}

public Status call() throws Exception {
try {

Node parent1 = session.getRootNode().getNode("root");
for (int j = 0; j < NODES_SIZE; j++) {
parent1 = parent1.addNode("node_" + i + "_" + j);
}
session.save();
session.logout();
return Status.OK;
} catch (final Exception e) {
e.printStackTrace();
return Status.ERROR;
}
}

}

private final int THREAD_SIZE = 100;

private final int NODES_SIZE = 10;

private Session openSession() {
                // TODO - here opens a new session
}
@Test
public void shouldInsertNodesInParallel() throws Exception {
final Session s = openSession();
s.getRootNode().addNode("root");
s.save();
s.logout();
final List<Callable<Status>> workers = new ArrayList<Callable<Status>>(
THREAD_SIZE);

for (int i = 0; i < THREAD_SIZE; i++) {
final Session session = openSession();
workers.add(new Worker(session, i));
}

final ExecutorService threadPool = Executors.newFixedThreadPool(4);
final List<Future<Status>> resultList = threadPool.invokeAll(workers);
for (final Future<Status> result : resultList) {
Assert.assertTrue(result.get().equals(Status.OK));
}

}

}

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[Patricio Echagüe <pa...@gmail.com>]

Hi guys, I'm running into the same problem. Has there been any update on
this?

I looked at Jira and there is no comments on this "bug" yet.

Regards,
Patricio

On Wed, Dec 9, 2009 at 1:30 AM, Luiz Fernando Teston <
feu.teston@caravelatech.com> wrote:

> Thomas,
>
> I created the JIRA issue JCR-2428 </jira/browse/JCR-2428> . I'm really
> thankful for your help! Since my application uses multi threaded
> environment, this fix would help us a lot!
>
> Best regards,
>
>
>
> Fernando Teston
>
> On Wed, Dec 9, 2009 at 7:13 AM, Thomas Müller <thomas.mueller@day.com
> >wrote:
>
> > Hi,
> >
> > Actually, it turns out, concurrently adding nodes to the same parent
> > node should work, when using different node names (so only same name
> > siblings are problematic). Your test case should work as far as I see.
> > I will try to find out what the problem is... Could you create a new
> > Jira issue and attach your test case please?
> >
> > Regards,
> > Thomas
> >
>

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[Luiz Fernando Teston <fe...@caravelatech.com>]

Thomas,

I created the JIRA issue JCR-2428 </jira/browse/JCR-2428> . I'm really
thankful for your help! Since my application uses multi threaded
environment, this fix would help us a lot!

Best regards,



Fernando Teston

On Wed, Dec 9, 2009 at 7:13 AM, Thomas Müller <th...@day.com>wrote:

> Hi,
>
> Actually, it turns out, concurrently adding nodes to the same parent
> node should work, when using different node names (so only same name
> siblings are problematic). Your test case should work as far as I see.
> I will try to find out what the problem is... Could you create a new
> Jira issue and attach your test case please?
>
> Regards,
> Thomas
>

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[Thomas Müller <th...@day.com>]

Hi,

Actually, it turns out, concurrently adding nodes to the same parent
node should work, when using different node names (so only same name
siblings are problematic). Your test case should work as far as I see.
I will try to find out what the problem is... Could you create a new
Jira issue and attach your test case please?

Regards,
Thomas

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[Luiz Fernando Teston <fe...@caravelatech.com>]

Hi,

The e-mails and also the wiki are clear. I also did a test writing on
different threads and different nodes and it works. If you guys are planning
to change this behavior please let me know, maybe sending me the jira issue
(so I could follow it).

Regards,


Fernando Teston


On Wed, Dec 9, 2009 at 6:51 AM, Thomas Müller <th...@day.com>wrote:

> Hi,
>
> I updated the wiki:
> http://wiki.apache.org/jackrabbit/QuestionsAndAnswers#Concurrency
>
> Please tell me if you have more question or if it's unclear.
>
> Regards,
> Thomas
>

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[Thomas Müller <th...@day.com>]

Hi,

I updated the wiki:
http://wiki.apache.org/jackrabbit/QuestionsAndAnswers#Concurrency

Please tell me if you have more question or if it's unclear.

Regards,
Thomas

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[Thomas Müller <th...@day.com>]

Hi,

> This problem will come in concurrent request to repository to either add
> content or read content.

That's not fully correct. Multiple session can read from the
repository concurrently. Also, multiple session can write to the
repository concurrently, given that they write in different nodes.

The test case tries to concurrently modify the node '/root'. I know it
doesn't look like it would. But internally, Jackrabbit stores the list
of child nodes at the node itself, that means if you add a node to
'/root' then you implicitly update the node '/root'. And that's where
the conflict occurs.

The exception message is quite strange, it sounds like the problem
occurs on the node '/' (on the root node). I'm not sure why it doesn't
say '/root'.

Regards,
Thomas

Re: Invalid item state while adding nodes on different sessions on multi thread environment

[sunild <su...@coreobjects.com>]

Luiz,

Even I have faced the same problem.
This problem will come in concurrent request to repository to either add 
content or read content.
This can be resolved using locks.
There are two cases
1. You can use java 5 concurrent.ReentrantReadWritelock, to lock the 
repository and then unlock after session.save();
2. Make your nodes mix:lockable and then invoke node.lock() before you 
do addNode and then invoke node.unlock() after session.save();

Regards,
Sunil Dhage

Luiz Fernando Teston wrote:
> Hi,
>
> I just found an strange behavior while using multiple sessions on multi
> thread environment. I'd like to know if I did something wrong.
> My application code do this:
> 1 - before start to work, it creates a root node inside a new session. It
> saves and close this session.
> 2 - create a bunch of workers. Each worker has its own session.
> 3 - each worker adds nodes inside the root node retrieved by its session. It
> saves and do a logout properly.
> 4 - fill this workers on a list of Callables to be passed to an java 5
> ThreadPoolExecutor.
> 5 - it executes all workers on the ThreadPool.
> 6 - it do some asserts to verify if it gots some errors or not.
>
> During phase 5 it gots an exception telling this:
>
> javax.jcr.InvalidItemStateException: Item cannot be saved because it has
> been modified externally: node /
> at org.apache.jackrabbit.core.ItemImpl.getTransientStates(ItemImpl.java:245)
> at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:939)
> at org.apache.jackrabbit.core.SessionImpl.save(SessionImpl.java:915)
>
> I just wrote a test to simulate this behavior. Sometimes it got this error
> sometimes not. Also to be easy to simulate this error, just change the
> THREAD_SIZE and NODE_SIZE variables.
>
> I'm very thankful for your help!
>
>
> Fernando Teston
>
> ---- test code -----
>
> package org.openspotlight.jcr.provider.test;
>
> import java.util.ArrayList;
> import java.util.List;
> import java.util.concurrent.Callable;
> import java.util.concurrent.ExecutorService;
> import java.util.concurrent.Executors;
> import java.util.concurrent.Future;
>
> import javax.jcr.Node;
> import javax.jcr.Session;
>
> import org.junit.Assert;
> import org.junit.Test;
>
> public class MultithreadedSessionTest {
>
> enum Status {
> OK, ERROR
> }
>
> class Worker implements Callable<Status> {
>
> private final Session session;
> private final int i;
>
> public Worker(final Session session, final int i) {
> this.session = session;
> this.i = i;
> }
>
> public Status call() throws Exception {
> try {
>
> Node parent1 = session.getRootNode().getNode("root");
> for (int j = 0; j < NODES_SIZE; j++) {
> parent1 = parent1.addNode("node_" + i + "_" + j);
> }
> session.save();
> session.logout();
> return Status.OK;
> } catch (final Exception e) {
> e.printStackTrace();
> return Status.ERROR;
> }
> }
>
> }
>
> private final int THREAD_SIZE = 100;
>
> private final int NODES_SIZE = 10;
>
> private Session openSession() {
>                 // TODO - here opens a new session
> }
> @Test
> public void shouldInsertNodesInParallel() throws Exception {
> final Session s = openSession();
> s.getRootNode().addNode("root");
> s.save();
> s.logout();
> final List<Callable<Status>> workers = new ArrayList<Callable<Status>>(
> THREAD_SIZE);
>
> for (int i = 0; i < THREAD_SIZE; i++) {
> final Session session = openSession();
> workers.add(new Worker(session, i));
> }
>
> final ExecutorService threadPool = Executors.newFixedThreadPool(4);
> final List<Future<Status>> resultList = threadPool.invokeAll(workers);
> for (final Future<Status> result : resultList) {
> Assert.assertTrue(result.get().equals(Status.OK));
> }
>
> }
>
> }
>