oak's consistency tuning options

[Tyson Norris <tn...@adobe.com>]

Hi  -
I was wondering if there are plans to surface consistency options in oak such that certain save() operations can be guaranteed to be replicated?

For example, in mongo getLastError() [0] can be executed with parameters to indicate that the call will block until a certain level of consistency is achieved (write has reached the server, write has been replicated to n nodes, write has been written to journal, etc)

Are there any examples of how oak might provide this type of function, so that applications could implement features where certain write operations block until consistency is achieved, and other write operations never block?

Thanks
Tyson


[0]http://www.mongodb.org/display/DOCS/getLastError+Command

Re: oak's consistency tuning options

[Tyson Norris <tn...@adobe.com>]

Thanks Michael

On 9/4/12 4:02 AM, "Michael Dürig" <md...@apache.org> wrote:

>
>Hi Tyson,
>
>The current direction we are going is that clients may not make too much
>assumptions on consistency. That is, we effectively trade consistency
>for availability [1, 2].

For clustered systems, this leaves the requirement to use sticky sessions
in place for any user-modified data (including auth tokens stored in jcr);
I am mainly asking to see if there is a way to avoid this at some point.

It would be nice to specify per-operation what the consistency
requirements are, at least in a clustered scenario.

>
>In recent discussions about locking [3] and referential integrity [4]
>this issue surfaced again. There I suggested to provide lock support for
>non clustered setups and provide a hook where lock support could be
>plugged in for other cases. This way custom lock implementation -
>possibly through external coordinators like Apache Zookeeper - are still
>possible [5].

For non-clustered setups, I guess the alternative would be to explicitly
replicate the content immediately+synchronously as part of the application
logic (e.g. A pluggable lock implementation), correct?
I think in some cases it may be good to consider these separately -
locking and replicating - e.g. When the content is not required to be
locked, but simply copied into place on each repo instance.


>
>I think this approach could be further generalised such that consistency
>guarantees in general would be enforced through an pluggable/external
>coordinator.

Yes that would be great - there are several use cases where as soon as
users are allowed to modify content the system configuration becomes so
much more delicate (I.e. User must be bound to that instance). The ability
to enforce consistency rules per operation would alleviate this to some
extent. 

Thanks
Tyson

>
>Michael
>
>
>[1] 
>http://wiki.apache.org/jackrabbit/Goals%20and%20non%20goals%20for%20Jackra
>bbit%203
>[2] http://markmail.org/thread/kypn7c4bi5gjnspu
>[3] https://issues.apache.org/jira/browse/OAK-150
>[4] https://issues.apache.org/jira/browse/OAK-270
>[5] 
>https://issues.apache.org/jira/browse/OAK-150?focusedCommentId=13401328&pa
>ge=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#commen
>t-13401328
>
>On 1.9.12 0:08, Tyson Norris wrote:
>> Hi  -
>> I was wondering if there are plans to surface consistency options in
>>oak such that certain save() operations can be guaranteed to be
>>replicated?
>>
>> For example, in mongo getLastError() [0] can be executed with
>>parameters to indicate that the call will block until a certain level of
>>consistency is achieved (write has reached the server, write has been
>>replicated to n nodes, write has been written to journal, etc)
>>
>> Are there any examples of how oak might provide this type of function,
>>so that applications could implement features where certain write
>>operations block until consistency is achieved, and other write
>>operations never block?
>>
>> Thanks
>> Tyson
>>
>>
>> [0]http://www.mongodb.org/display/DOCS/getLastError+Command
>>

Re: oak's consistency tuning options

[Michael Dürig <md...@apache.org>]

Hi Tyson,

The current direction we are going is that clients may not make too much 
assumptions on consistency. That is, we effectively trade consistency 
for availability [1, 2].

In recent discussions about locking [3] and referential integrity [4] 
this issue surfaced again. There I suggested to provide lock support for 
non clustered setups and provide a hook where lock support could be 
plugged in for other cases. This way custom lock implementation - 
possibly through external coordinators like Apache Zookeeper - are still 
possible [5].

I think this approach could be further generalised such that consistency 
guarantees in general would be enforced through an pluggable/external 
coordinator.

Michael


[1] 
http://wiki.apache.org/jackrabbit/Goals%20and%20non%20goals%20for%20Jackrabbit%203
[2] http://markmail.org/thread/kypn7c4bi5gjnspu
[3] https://issues.apache.org/jira/browse/OAK-150
[4] https://issues.apache.org/jira/browse/OAK-270
[5] 
https://issues.apache.org/jira/browse/OAK-150?focusedCommentId=13401328&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13401328

On 1.9.12 0:08, Tyson Norris wrote:
> Hi  -
> I was wondering if there are plans to surface consistency options in oak such that certain save() operations can be guaranteed to be replicated?
>
> For example, in mongo getLastError() [0] can be executed with parameters to indicate that the call will block until a certain level of consistency is achieved (write has reached the server, write has been replicated to n nodes, write has been written to journal, etc)
>
> Are there any examples of how oak might provide this type of function, so that applications could implement features where certain write operations block until consistency is achieved, and other write operations never block?
>
> Thanks
> Tyson
>
>
> [0]http://www.mongodb.org/display/DOCS/getLastError+Command
>