Creating a split Document in a unit test

[Robert Munteanu <ro...@apache.org>]

Hi,

I'm trying to create a split document in a unit test to reproduce I'm
failure I'm seeing sporadically with my multiplexing document store
POC.

What I tried was

            Session session = getAdminSession();
            try {

                Node node = session.getRootNode().addNode("bla");
                session.save();

                for ( int i = 0 ; i < 1000 ; i++ ) {
                    node = session.getRootNode().getNode("bla");
                    node.setProperty("prop-" + i, largeString());
                    session.save();
                }

            } finally {
                session.logout();
            }

While this code creates a split candidate in the DocumentNodeStore,
NodeDocument.split() creates no update ops ; I traced down the logic
to SplitOperations.create() but it got a bit too complicated for me to
follow.

Does anyone have a suggestion on how to create a split document in a unit test?

Thanks,

Robert

Re: Creating a split Document in a unit test

[Robert Munteanu <ro...@apache.org>]

On Thu, 2015-08-06 at 20:03 +0530, Vikas Saurabh wrote:
> > - the @Before instantiates a DocumentNodeStore backed by my
> > implementation of the MultiplexingDocumentStore
> And it's working with say MemoryDocumentStore? IIRC, the decision to
> split is done on the basis of NodeDocument.getMemory() (and I think
> certain other parameters too). BTW, I might be wrong here.
> 
> So, this not working with your MultiplexingDS could either be due 
> some
> issue in test case <-> DocumentNodeStore interaction OR that
> DocumentNodeStore is deciding not to split based on the info it gets
> from MultiplexingDS.
> 
> > - the validation is simply running the background operations
> Well, the documents should get split after this under the premise 
> that
> DocumentNodeStore decides to do so.

In the end, the simplest way to achieve this was to delegate to the
DocumentSplitTest

    @Test
    public void splitRevisions() throws Exception {

        MultiplexingBasedDocumentSplitTest delegate = new
MultiplexingBasedDocumentSplitTest();
        delegate.initDocumentMK(); // not managed by JUnit so call the
@Before manually
        delegate.splitRevisions();
    }

where MultiplexingBasedDocumentSplitTest extends the DocumentSplitTest
and creates the DocumentMK based on my DocumentStore implementation.

Thanks,

Robert

Re: Creating a split Document in a unit test

[Vikas Saurabh <vi...@gmail.com>]

> - the @Before instantiates a DocumentNodeStore backed by my
> implementation of the MultiplexingDocumentStore
And it's working with say MemoryDocumentStore? IIRC, the decision to
split is done on the basis of NodeDocument.getMemory() (and I think
certain other parameters too). BTW, I might be wrong here.

So, this not working with your MultiplexingDS could either be due some
issue in test case <-> DocumentNodeStore interaction OR that
DocumentNodeStore is deciding not to split based on the info it gets
from MultiplexingDS.

> - the validation is simply running the background operations
Well, the documents should get split after this under the premise that
DocumentNodeStore decides to do so.

Thanks,
Vikas

Re: Creating a split Document in a unit test

[Robert Munteanu <ro...@apache.org>]

On Thu, 2015-08-06 at 19:36 +0530, Vikas Saurabh wrote:
> Hi Robert,
> 
> > I am running that operation manually, so it should trigger ...
> Umm, I'm sure what you meant by manual? Do you mean you the updates
> once and then check whether actual split happened in a different
> execution which is launched manually?

I end the test with a 

  nodeStore.runBackgroundOperations();



> Can you describe your current test setup a bit more?

Sure

- the @Before instantiates a DocumentNodeStore backed by my
implementation of the MultiplexingDocumentStore
- the test is as I pasted before , an attempt to generate enough data
in order to create a split document
- the validation is simply running the background operations

Thanks,

Robert

Re: Creating a split Document in a unit test

[Vikas Saurabh <vi...@gmail.com>]

Hi Robert,

> I am running that operation manually, so it should trigger ...
Umm, I'm sure what you meant by manual? Do you mean you the updates
once and then check whether actual split happened in a different
execution which is launched manually?

Can you describe your current test setup a bit more?

Thanks,
Vikas

Re: Creating a split Document in a unit test

[Robert Munteanu <ro...@apache.org>]

Hi Vikas,

On Thu, 2015-08-06 at 19:11 +0530, Vikas Saurabh wrote:
> >                 for ( int i = 0 ; i < 1000 ; i++ ) {
> BTW, just doing it a 100 times should do it
> (ref:NodeDocument.NUM_REVS_THRESHOLD)
> 
> 
> > While this code creates a split candidate in the DocumentNodeStore,
> > NodeDocument.split() creates no update ops
> Actual split docs are created in background update thread -- which,
> would hit almost every second, so you can sleep a while and it should
> do it.

I am running that operation manually, so it should trigger ...

> 
> 
> > Does anyone have a suggestion on how to create a split document in 
> > a unit test?
> To me, although, depending on split docs from Session level seems a
> little too dependend on implementation. I think it'd be easier (more
> controllable) to have the test build up at a more basic level -- 
> check
> out DocumentSplitTest. Basic idea is that do a lot of changes and 
> then
> forcefully do a NodeStore.runBackgroundOperations()

I'll try to do whatever DocumentSplitTest does ; having all my
regressions in one test is nice, but not a must-have.

Thanks,

Robert

Re: Creating a split Document in a unit test

[Vikas Saurabh <vi...@gmail.com>]

>                 for ( int i = 0 ; i < 1000 ; i++ ) {
BTW, just doing it a 100 times should do it
(ref:NodeDocument.NUM_REVS_THRESHOLD)


> While this code creates a split candidate in the DocumentNodeStore,
> NodeDocument.split() creates no update ops
Actual split docs are created in background update thread -- which,
would hit almost every second, so you can sleep a while and it should
do it.


> Does anyone have a suggestion on how to create a split document in a unit test?
To me, although, depending on split docs from Session level seems a
little too dependend on implementation. I think it'd be easier (more
controllable) to have the test build up at a more basic level -- check
out DocumentSplitTest. Basic idea is that do a lot of changes and then
forcefully do a NodeStore.runBackgroundOperations()


Thanks,
Vikas