Multiplexing Document Store

[Ian Boston <ia...@gmail.com>]

Hi,
I am trying to enhance a multiplexing document store written initially by
Robert Munteanu to support multiplexing of content under /jcr:system/** in
particular the version store and the permissions store. I have a scheme
that should theoretically work, encoding the target store in the entry name
of the map key.

However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
objects created by a DocumentNodeStore get a reference to that
DocumentNodeStore, bypassing any multiplexing document node store. This is
all ok if all the calls relate to content in the same DocumentNodeStore,
but as soon as anything performs a call into DocumentNodeStore that relates
to a path not within that DocumentNodeStore the multiplexing breaks down.
Code that reads and writes to /jcr:system/** does this.

I have tried hacking the code to ensure that the reference to
DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore, however
when I do that, MultiplexingDocumentNodeStore gets calls that have
insufficient context to route to the correct DocumentNodeStore. I could
hack some more, but if I do, anything that works is unlikely to be
acceptable to Oak as a patch.

Any suggestions ?

I haven't even started to look at /oak:index and the code that writes/reads
from there.

Best Regards
Ian

Re: Multiplexing Document Store

[Ian Boston <ie...@tfd.co.uk>]

On 3 December 2015 at 04:15, Chetan Mehrotra <ch...@gmail.com>
wrote:

> Hi Ian,
>
> On Wed, Dec 2, 2015 at 9:24 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> > Hence all MutableTrees get their
> > NodeBuilder from the root DocumentNodeState that is the DocumentNodeStore
> > owning the root node and not the MultiplexingDocumentNodeStore.
>
> Some confusion here. What we have is a MultiplexDocumentStore and
> there is no MultiplexingDocumentNodeStore. All the above objects
> DocumentNodeState, DocumentNodeBuilder refer to DocumentNodeStore (not
> DocumentStore).
>

I meant MultiplexDocumentStore. So many stores and so many layers make it
hard for someone who doesn't work in Oak every day to find there way around.


>
> Use of MultiplexDocumentStore is an implementation detail of
> DocumentNodeStore hence these objects need not be affected and aware
> of multiplexing logic. May be I am missing something here?
>

Looking at the code again, I don't think you are missing anything.
DocumentNodeStore gets its store from the DocumentMK.builder which is a
singleton.

I need to re-run my tests of the 30 October to find out why I was seeing
the behaviour I was seeing, however IIRC you have recently been looking at
/jcr:system etc and the branch I was working may now be obsolete ? If it
is, please let me know as there is no point in duplication or wasted effort.

Best Regards
Ian



>
> Chetan Mehrotra
>

Re: Multiplexing Document Store

[Chetan Mehrotra <ch...@gmail.com>]

Hi Ian,

On Wed, Dec 2, 2015 at 9:24 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> Hence all MutableTrees get their
> NodeBuilder from the root DocumentNodeState that is the DocumentNodeStore
> owning the root node and not the MultiplexingDocumentNodeStore.

Some confusion here. What we have is a MultiplexDocumentStore and
there is no MultiplexingDocumentNodeStore. All the above objects
DocumentNodeState, DocumentNodeBuilder refer to DocumentNodeStore (not
DocumentStore).

Use of MultiplexDocumentStore is an implementation detail of
DocumentNodeStore hence these objects need not be affected and aware
of multiplexing logic. May be I am missing something here?

Chetan Mehrotra

Re: Multiplexing Document Store

[Ian Boston <ie...@tfd.co.uk>]

Hi,

On 1 December 2015 at 13:10, Chetan Mehrotra <ch...@gmail.com>
wrote:

> Reviving this old thread as I just started looking into this
>
> > However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
> > objects created by a DocumentNodeStore get a reference to that
> > DocumentNodeStore, bypassing any multiplexing document node store.
>
> Multiplexing is being implemented at DocumentStore level and
> DocumentNodeStore gets a reference to that. And that internally has a
> reference to MultiPlexingDocumentStore so logically looks ok
>
> > but as soon as anything performs a call into DocumentNodeStore that
> relates
> > to a path not within that DocumentNodeStore the multiplexing breaks down.
> > Code that reads and writes to /jcr:system/** does this.
>
> Can you provide a testcase on Robert's branch illustrating the
> problem? That would help to understand the problem you are facing
> better
>

It is quite possible that the behaviour I was seeing was due to some other
problem I introduced in attempting to make multiplexed content in
/jcr:system/** work, but I will try and explain with pointers to code in
the 1.3.11 release, which, in this area is the same as the branch I was
working on a month ago. I should go and check Roberts branch to see if this
has changed since I pulled from it.

MutableTree objects (other than the root object are created) at [1]. The
second last parameter is the node builder, which as the tree is built
references the NodeBuilder stored in the root object. Hence all all
MutableTree objects have the same nodeBuilder, which comes from the store
at the MutableRoot [2]. The root DocumentNodeState is probably in the
cache, but when it was created it was created by the DocumentNodeStore that
owned it eg [3]. Although the root NodeState may have been looked up by the
MultiplexingDocumentNodeStore, that will delegate to the real
DocumentNodeStore owning the root node. Hence all MutableTrees get their
NodeBuilder from the root DocumentNodeState that is the DocumentNodeStore
owning the root node and not the MultiplexingDocumentNodeStore. Since that
NodeBuilder gets used everywhere, all sorts of operations will be being
performed against the wrong DocumentNodeStore. It's the same structure that
blocked a number of PoC's to allow multiple DocumentStores to be isolated
from one another inside Oak.

Obviously, its software so anything is possible and patchable. The patch
that I ended up with to wrap the this param in [3] so that the correct
NodeBuilder was used, resulted in a patch that looked too large to be
acceptable as a contribution (touches almost every class in oak-core) Hence
the question to list. Perhaps there is a different way of achieving the
isolation between DocumentNodeStores and preventing leakage ?


Best Regards
Ian



1
https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.3.11/oak-core/src/main/java/org/apache/jackrabbit/oak/core/MutableTree.java#L290
https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.3.11/oak-core/src/main/java/org/apache/jackrabbit/oak/core/MutableTree.java#L92
2
https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.3.11/oak-core/src/main/java/org/apache/jackrabbit/oak/core/MutableRoot.java#L160
3
https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.3.11/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java#L2411



> Chetan Mehrotra
>
>
> On Fri, Oct 30, 2015 at 4:32 PM, Ian Boston <ia...@gmail.com> wrote:
> > Hi,
> > I am trying to enhance a multiplexing document store written initially by
> > Robert Munteanu to support multiplexing of content under /jcr:system/**
> in
> > particular the version store and the permissions store. I have a scheme
> > that should theoretically work, encoding the target store in the entry
> name
> > of the map key.
> >
> > However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
> > objects created by a DocumentNodeStore get a reference to that
> > DocumentNodeStore, bypassing any multiplexing document node store. This
> is
> > all ok if all the calls relate to content in the same DocumentNodeStore,
> > but as soon as anything performs a call into DocumentNodeStore that
> relates
> > to a path not within that DocumentNodeStore the multiplexing breaks down.
> > Code that reads and writes to /jcr:system/** does this.
> >
> > I have tried hacking the code to ensure that the reference to
> > DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore,
> however
> > when I do that, MultiplexingDocumentNodeStore gets calls that have
> > insufficient context to route to the correct DocumentNodeStore. I could
> > hack some more, but if I do, anything that works is unlikely to be
> > acceptable to Oak as a patch.
> >
> > Any suggestions ?
> >
> > I haven't even started to look at /oak:index and the code that
> writes/reads
> > from there.
> >
> > Best Regards
> > Ian
>

Re: Multiplexing Document Store

[Chetan Mehrotra <ch...@gmail.com>]

Reviving this old thread as I just started looking into this

> However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
> objects created by a DocumentNodeStore get a reference to that
> DocumentNodeStore, bypassing any multiplexing document node store.

Multiplexing is being implemented at DocumentStore level and
DocumentNodeStore gets a reference to that. And that internally has a
reference to MultiPlexingDocumentStore so logically looks ok

> but as soon as anything performs a call into DocumentNodeStore that relates
> to a path not within that DocumentNodeStore the multiplexing breaks down.
> Code that reads and writes to /jcr:system/** does this.

Can you provide a testcase on Robert's branch illustrating the
problem? That would help to understand the problem you are facing
better
Chetan Mehrotra


On Fri, Oct 30, 2015 at 4:32 PM, Ian Boston <ia...@gmail.com> wrote:
> Hi,
> I am trying to enhance a multiplexing document store written initially by
> Robert Munteanu to support multiplexing of content under /jcr:system/** in
> particular the version store and the permissions store. I have a scheme
> that should theoretically work, encoding the target store in the entry name
> of the map key.
>
> However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
> objects created by a DocumentNodeStore get a reference to that
> DocumentNodeStore, bypassing any multiplexing document node store. This is
> all ok if all the calls relate to content in the same DocumentNodeStore,
> but as soon as anything performs a call into DocumentNodeStore that relates
> to a path not within that DocumentNodeStore the multiplexing breaks down.
> Code that reads and writes to /jcr:system/** does this.
>
> I have tried hacking the code to ensure that the reference to
> DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore, however
> when I do that, MultiplexingDocumentNodeStore gets calls that have
> insufficient context to route to the correct DocumentNodeStore. I could
> hack some more, but if I do, anything that works is unlikely to be
> acceptable to Oak as a patch.
>
> Any suggestions ?
>
> I haven't even started to look at /oak:index and the code that writes/reads
> from there.
>
> Best Regards
> Ian