You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Carsten Ziegeler <cz...@apache.org> on 2013/02/27 16:09:53 UTC
Behaviour of Move Operations
Hi,
I'm wondering how move operations in a cluster will work in Oak - my
scenario is that more than one cluster node tries concurrently to move
a node to a different location, but each of the instances tries to
move the same source to a different destination. So e.g. we have a
node at /a/node and cluster node 1 tries to move this to /1/node while
cluster node 2 tries to move this to /2/node.
Afaik, with current Jackrabbit 2, the "first" instance succeeds while
the "second" would get an exception.
How will this be handled with Oak? Could it happen that due to this
happening concurrently that the node ends up twice in the repository
(at /1/node and /2/node in my example)?
Regards
Carsten
--
Carsten Ziegeler
cziegeler@apache.org
Re: Behaviour of Move Operations
Posted by Carsten Ziegeler <cz...@apache.org>.
Thanks Jukka,
is the configuration of the journal a repository wide configuration or
is it configurable for sub trees?
Regards
Carsten
2013/2/27 Jukka Zitting <ju...@gmail.com>:
> Hi,
>
> On Wed, Feb 27, 2013 at 5:09 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>> How will this be handled with Oak? Could it happen that due to this
>> happening concurrently that the node ends up twice in the repository
>> (at /1/node and /2/node in my example)?
>
> The behavior depends on the underlying MicroKernel implementation.
>
> With the new SegmentMK I've been working on, you can control the behavior:
>
> * If both cluster nodes use the same (root) journal, then only one of
> them succeeds and the other one will fail with an exception. The
> behavior is more or less the same as with current Jackrabbit.
>
> * If the cluster nodes use different journals (with background
> merging), then one of the moves will succeed and depending on timing
> the other one either fails or ends up producing a duplicate copy of
> the tree.
>
> The latter option is designed to boost write concurrency in scenarios
> where it's OK for some operations to get lost or produce somewhat
> inconsistent results (high-volume commenting or logging systems,
> etc.). Operations for which such behavior is not desirable should use
> the first option.
>
> BR,
>
> Jukka Zitting
--
Carsten Ziegeler
cziegeler@apache.org
Re: Behaviour of Move Operations
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>>(/libs is always consistent,
>> /comments is eventually consistent or so).
>
>I think (but dont know for certain) that that would require the paths
>being store in different column families.
Yes, that's true.
> Is it possible to
>differentiate between subtrees at the DocumentStore interface level or
>are the objects too abstract by then ?
As for the MongoMK, the DocumentStore implementation would need to do
that. There, the primary key for a document is the path. I think it would
be quite easy to support.
Regards,
Thomas
Re: Behaviour of Move Operations
Posted by Ian Boston <ie...@tfd.co.uk>.
On 4 March 2013 21:00, Thomas Mueller <mu...@adobe.com> wrote:
> Hi,
>
>>Are there any plans (or has someone already done) a CassandraMK
>
> I'm not aware of plans to write a CassandraMK. Currently we concentrate on
> MongoDB. Possibly Cassandra could be supported quite easily by
> implementing the org.apache.jackrabbit.mongomk.prototype.DocumentStore
> interface, but I didn't look into that.
thanks for the pointer.
>
>>is
>>there something in the Oak design that would prevent that level of
>>eventual consistency ? (I guess that might trading consistency for
>>scalability)
>
> ... or consistency for performance.
true.
>
> I didn't test it, but I guess eventual consistency would work within Oak.
> I would actually be more worried about how *users* (and developers) can
> deal with inconsistencies. It might be confusing for a user if he can
> successfully save changes, but then his changes are lost or garbled
> because somebody else changed the same item at the same time. If we do
> allow inconsistencies within the MicroKernel, I guess it would help if
> this could be configured on a path level (/libs is always consistent,
> /comments is eventually consistent or so).
I think (but dont know for certain) that that would require the paths
being store in different column families. Is it possible to
differentiate between subtrees at the DocumentStore interface level or
are the objects too abstract by then ?
I can bet the order in which values become eventually consistent is
non-deterministic.
Ian
>
> Regards,
> Thomas
>
Re: Behaviour of Move Operations
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>Are there any plans (or has someone already done) a CassandraMK
I'm not aware of plans to write a CassandraMK. Currently we concentrate on
MongoDB. Possibly Cassandra could be supported quite easily by
implementing the org.apache.jackrabbit.mongomk.prototype.DocumentStore
interface, but I didn't look into that.
>is
>there something in the Oak design that would prevent that level of
>eventual consistency ? (I guess that might trading consistency for
>scalability)
... or consistency for performance.
I didn't test it, but I guess eventual consistency would work within Oak.
I would actually be more worried about how *users* (and developers) can
deal with inconsistencies. It might be confusing for a user if he can
successfully save changes, but then his changes are lost or garbled
because somebody else changed the same item at the same time. If we do
allow inconsistencies within the MicroKernel, I guess it would help if
this could be configured on a path level (/libs is always consistent,
/comments is eventually consistent or so).
Regards,
Thomas
Re: Behaviour of Move Operations
Posted by Ian Boston <ie...@tfd.co.uk>.
On 2 March 2013 01:38, Thomas Mueller <mu...@adobe.com> wrote:
> Hi,
>
>>One point of Oak is exactly that: to provide a sensible way for trading
>>off consistency for availability.
>
> As far as I understood our goals, we want to have a scalable solution.
>
>> While I agree that consistency is
>>important it should however not have an impact on scenarios where it is
>>irrelevant. Other NoSQL systems have similar means for such a trade offs.
>
> In the case of MongoDB, writes always happen on the primary node first.
> That way you get consistency:
> http://docs.mongodb.org/manual/faq/replica-sets/#what-do-the-terms-primary-
> and-master-mean - MongoDB does provide scalable writes by sharding the
> data. Improved read scalability is achieved by using replication. I think
> we should try to do something similar on Oak.
>
> Trading consistency for scalability is not necessary.
Apache Cassandra does this using Quorum at a column family level [1]
and allows writes to any node (ie no master or primary node).
Are there any plans (or has someone already done) a CassandraMK, or is
there something in the Oak design that would prevent that level of
eventual consistency ? (I guess that might trading consistency for
scalability)
1 http://wiki.apache.org/cassandra/API
>
> Regards,
> Thomas
>
Re: Behaviour of Move Operations
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>One point of Oak is exactly that: to provide a sensible way for trading
>off consistency for availability.
As far as I understood our goals, we want to have a scalable solution.
> While I agree that consistency is
>important it should however not have an impact on scenarios where it is
>irrelevant. Other NoSQL systems have similar means for such a trade offs.
In the case of MongoDB, writes always happen on the primary node first.
That way you get consistency:
http://docs.mongodb.org/manual/faq/replica-sets/#what-do-the-terms-primary-
and-master-mean - MongoDB does provide scalable writes by sharding the
data. Improved read scalability is achieved by using replication. I think
we should try to do something similar on Oak.
Trading consistency for scalability is not necessary.
Regards,
Thomas
Re: Behaviour of Move Operations
Posted by Michael Dürig <md...@apache.org>.
On 1.3.13 12:52, Felix Meschberger wrote:
> Hi,
>
> Am 01.03.2013 um 13:47 schrieb Michael Dürig:
>
>>
>>
>> What Jukka is saying is that the repository gives you a choice between
>> consistency and availability. Since both you cannot have.
>
> I think you don't want to given the user that choice ...
>
> I'd opt for best possible availability (or probably you mean performance?) with "guaranteed" consistency.
One point of Oak is exactly that: to provide a sensible way for trading
off consistency for availability. While I agree that consistency is
important it should however not have an impact on scenarios where it is
irrelevant. Other NoSQL systems have similar means for such a trade offs.
See
http://blog.mongodb.org/post/523516007/on-distributed-consistency-part-6-consistency-chart
and http://en.wikipedia.org/wiki/CAP_theorem for some background.
Michael
>
> Regards
> Felix
>
>>
>> Michael
>>
>> On 1.3.13 12:40, Felix Meschberger wrote:
>>> So you essentially say: Behaviour of the repository is "best effort" and we -- at the end of the day -- cannot trust the repository ?
>>>
>>> Sounds frightening.
>>>
>>> IMHO the repository should be failsafe and thus eventually solve the issue making sure we don't end up with two copies of the same node (actually both copies, if referenceable, will even have the same node ID....
>>>
>>> Regards
>>> Felix
>>>
>>> Am 27.02.2013 um 16:35 schrieb Jukka Zitting:
>>>
>>>> Hi,
>>>>
>>>> On Wed, Feb 27, 2013 at 5:09 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>>>>> How will this be handled with Oak? Could it happen that due to this
>>>>> happening concurrently that the node ends up twice in the repository
>>>>> (at /1/node and /2/node in my example)?
>>>>
>>>> The behavior depends on the underlying MicroKernel implementation.
>>>>
>>>> With the new SegmentMK I've been working on, you can control the behavior:
>>>>
>>>> * If both cluster nodes use the same (root) journal, then only one of
>>>> them succeeds and the other one will fail with an exception. The
>>>> behavior is more or less the same as with current Jackrabbit.
>>>>
>>>> * If the cluster nodes use different journals (with background
>>>> merging), then one of the moves will succeed and depending on timing
>>>> the other one either fails or ends up producing a duplicate copy of
>>>> the tree.
>>>>
>>>> The latter option is designed to boost write concurrency in scenarios
>>>> where it's OK for some operations to get lost or produce somewhat
>>>> inconsistent results (high-volume commenting or logging systems,
>>>> etc.). Operations for which such behavior is not desirable should use
>>>> the first option.
>>>>
>>>> BR,
>>>>
>>>> Jukka Zitting
>>>
>>>
>>> --
>>> Felix Meschberger | Principal Scientist | Adobe
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
> --
> Felix Meschberger | Principal Scientist | Adobe
>
>
>
>
>
>
>
Re: Behaviour of Move Operations
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 01.03.2013 um 13:47 schrieb Michael Dürig:
>
>
> What Jukka is saying is that the repository gives you a choice between
> consistency and availability. Since both you cannot have.
I think you don't want to given the user that choice ...
I'd opt for best possible availability (or probably you mean performance?) with "guaranteed" consistency.
Regards
Felix
>
> Michael
>
> On 1.3.13 12:40, Felix Meschberger wrote:
>> So you essentially say: Behaviour of the repository is "best effort" and we -- at the end of the day -- cannot trust the repository ?
>>
>> Sounds frightening.
>>
>> IMHO the repository should be failsafe and thus eventually solve the issue making sure we don't end up with two copies of the same node (actually both copies, if referenceable, will even have the same node ID....
>>
>> Regards
>> Felix
>>
>> Am 27.02.2013 um 16:35 schrieb Jukka Zitting:
>>
>>> Hi,
>>>
>>> On Wed, Feb 27, 2013 at 5:09 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>>>> How will this be handled with Oak? Could it happen that due to this
>>>> happening concurrently that the node ends up twice in the repository
>>>> (at /1/node and /2/node in my example)?
>>>
>>> The behavior depends on the underlying MicroKernel implementation.
>>>
>>> With the new SegmentMK I've been working on, you can control the behavior:
>>>
>>> * If both cluster nodes use the same (root) journal, then only one of
>>> them succeeds and the other one will fail with an exception. The
>>> behavior is more or less the same as with current Jackrabbit.
>>>
>>> * If the cluster nodes use different journals (with background
>>> merging), then one of the moves will succeed and depending on timing
>>> the other one either fails or ends up producing a duplicate copy of
>>> the tree.
>>>
>>> The latter option is designed to boost write concurrency in scenarios
>>> where it's OK for some operations to get lost or produce somewhat
>>> inconsistent results (high-volume commenting or logging systems,
>>> etc.). Operations for which such behavior is not desirable should use
>>> the first option.
>>>
>>> BR,
>>>
>>> Jukka Zitting
>>
>>
>> --
>> Felix Meschberger | Principal Scientist | Adobe
>>
>>
>>
>>
>>
>>
>>
--
Felix Meschberger | Principal Scientist | Adobe
Re: Behaviour of Move Operations
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Mar 1, 2013 at 4:47 AM, Michael Dürig <md...@apache.org> wrote:
> ...What Jukka is saying is that the repository gives you a choice between
> consistency and availability. Since both you cannot have....
+1 to having that, as long as those choices are made clear to users.
Log messages that explain what's happening (*) would be good to have,
in all cases where settings affect CAP behavior.
-Bertrand
(*) for example "CAP:the FOO option is currently set for availability.
at the expense of consistency"
Re: Behaviour of Move Operations
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>What Jukka is saying is that the repository gives you a choice between
>consistency and availability. Since both you cannot have.
Actually, I challenge this. You can have both consistency and availability
at the same time. I guess you refer to the CAP theorem, which says you
can't have all of the 3 following at the same time: consistency,
availability, and partition tolerance. But you can have 2 of them at the
same time (for example consistency and availability).
The way I see it, it's just a limitation of Jukka SegementNodeStore model.
Regards,
Thomas
Re: Behaviour of Move Operations
Posted by Michael Dürig <md...@apache.org>.
What Jukka is saying is that the repository gives you a choice between
consistency and availability. Since both you cannot have.
Michael
On 1.3.13 12:40, Felix Meschberger wrote:
> So you essentially say: Behaviour of the repository is "best effort" and we -- at the end of the day -- cannot trust the repository ?
>
> Sounds frightening.
>
> IMHO the repository should be failsafe and thus eventually solve the issue making sure we don't end up with two copies of the same node (actually both copies, if referenceable, will even have the same node ID....
>
> Regards
> Felix
>
> Am 27.02.2013 um 16:35 schrieb Jukka Zitting:
>
>> Hi,
>>
>> On Wed, Feb 27, 2013 at 5:09 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>>> How will this be handled with Oak? Could it happen that due to this
>>> happening concurrently that the node ends up twice in the repository
>>> (at /1/node and /2/node in my example)?
>>
>> The behavior depends on the underlying MicroKernel implementation.
>>
>> With the new SegmentMK I've been working on, you can control the behavior:
>>
>> * If both cluster nodes use the same (root) journal, then only one of
>> them succeeds and the other one will fail with an exception. The
>> behavior is more or less the same as with current Jackrabbit.
>>
>> * If the cluster nodes use different journals (with background
>> merging), then one of the moves will succeed and depending on timing
>> the other one either fails or ends up producing a duplicate copy of
>> the tree.
>>
>> The latter option is designed to boost write concurrency in scenarios
>> where it's OK for some operations to get lost or produce somewhat
>> inconsistent results (high-volume commenting or logging systems,
>> etc.). Operations for which such behavior is not desirable should use
>> the first option.
>>
>> BR,
>>
>> Jukka Zitting
>
>
> --
> Felix Meschberger | Principal Scientist | Adobe
>
>
>
>
>
>
>
Re: Behaviour of Move Operations
Posted by Felix Meschberger <fm...@adobe.com>.
So you essentially say: Behaviour of the repository is "best effort" and we -- at the end of the day -- cannot trust the repository ?
Sounds frightening.
IMHO the repository should be failsafe and thus eventually solve the issue making sure we don't end up with two copies of the same node (actually both copies, if referenceable, will even have the same node ID....
Regards
Felix
Am 27.02.2013 um 16:35 schrieb Jukka Zitting:
> Hi,
>
> On Wed, Feb 27, 2013 at 5:09 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>> How will this be handled with Oak? Could it happen that due to this
>> happening concurrently that the node ends up twice in the repository
>> (at /1/node and /2/node in my example)?
>
> The behavior depends on the underlying MicroKernel implementation.
>
> With the new SegmentMK I've been working on, you can control the behavior:
>
> * If both cluster nodes use the same (root) journal, then only one of
> them succeeds and the other one will fail with an exception. The
> behavior is more or less the same as with current Jackrabbit.
>
> * If the cluster nodes use different journals (with background
> merging), then one of the moves will succeed and depending on timing
> the other one either fails or ends up producing a duplicate copy of
> the tree.
>
> The latter option is designed to boost write concurrency in scenarios
> where it's OK for some operations to get lost or produce somewhat
> inconsistent results (high-volume commenting or logging systems,
> etc.). Operations for which such behavior is not desirable should use
> the first option.
>
> BR,
>
> Jukka Zitting
--
Felix Meschberger | Principal Scientist | Adobe
Re: Behaviour of Move Operations
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
With the MongoMK I'm working on, the first move would work, and the second
move would fail.
Regards,
Thomas
Re: Behaviour of Move Operations
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Wed, Feb 27, 2013 at 5:09 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> How will this be handled with Oak? Could it happen that due to this
> happening concurrently that the node ends up twice in the repository
> (at /1/node and /2/node in my example)?
The behavior depends on the underlying MicroKernel implementation.
With the new SegmentMK I've been working on, you can control the behavior:
* If both cluster nodes use the same (root) journal, then only one of
them succeeds and the other one will fail with an exception. The
behavior is more or less the same as with current Jackrabbit.
* If the cluster nodes use different journals (with background
merging), then one of the moves will succeed and depending on timing
the other one either fails or ends up producing a duplicate copy of
the tree.
The latter option is designed to boost write concurrency in scenarios
where it's OK for some operations to get lost or produce somewhat
inconsistent results (high-volume commenting or logging systems,
etc.). Operations for which such behavior is not desirable should use
the first option.
BR,
Jukka Zitting