You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by Michael Dürig <md...@apache.org> on 2011/11/30 13:38:42 UTC
[jr3 microkernel] Write skew
Hi,
As documented earlier [1] new Mircokernel based JCR implementations will
most likely introduce snapshot isolation for sessions. While I think
this is a good thing in general, it also introduces potentially
troublesome write skew: application level constraints might hold locally
(i.e. per session) but might fail globally. That is, in order to enforce
some constraints, applications might need to implement explicit cross
session synchronization mechanisms instead of being able to rely on
session isolation. The following test case demonstrates the issue.
/**
* Trans-session isolation differs from Jackrabbit 2.
* Snapshot isolation can result in write skew as this
* test demonstrates: the check method enforces an
* application logic constraint which says that the sum
* of the properties p1 and p2 must not be negative. While
* session1 and session2 each enforce this constraint before
* saving, the constraint might not hold globally as can be
* seen in session3.
*/
@Test
public void testSessionIsolation() throws RepositoryException {
Repository repository = getRepository();
Session session0 = repository.login();
Node testNode = session0.getNode("/").addNode("testNode");
testNode.setProperty("p1", 1);
testNode.setProperty("p2", 1);
session0.save();
check(getSession());
Session session1 = repository.login();
Session session2 = repository.login();
session1.getNode("/testNode").setProperty("p1", -1);
check(session1);
session1.save();
session2.getNode("/testNode").setProperty("p2", -1);
check(session2); // Throws on JR2 but not on JR3
session2.save();
Session session3 = repository.login();
check(session3); // Throws on JR3
}
private static void check(Session session) throws RepositoryException {
if (session.getNode("/testNode").getProperty("p1").getLong() +
session.getNode("/testNode").getProperty("p2").getLong() < 0) {
fail("p1 + p2 < 0");
}
}
Michael
[1]
http://wiki.apache.org/jackrabbit/Transactional%20model%20of%20the%20Microkernel%20based%20Jackrabbit%20prototype
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 30.11.11 13:27, Felix Meschberger wrote:
>>
>> That's a different story for a different thread ;-) In a nutshell: under
>> snapshot isolation sessions will see ADD events for items which do not
>> exist for them. OTHO when they get an DELETE event the item will still
>> exist for them. Note how this is reverse from the situation in JR2.
>
> Hmm, this will probably effectively break many if not most current ObservationListener implementations. Good to know.
>
> Still what could listeners possibly do here ? Sesssion.refresh ? Or create a short-lived session to handle the event ?
Both approaches should work. If the behaviour is acceptable is another
story though...
Michael
Re: [jr3 microkernel] Write skew
Posted by Alexander Klimetschek <ak...@adobe.com>.
On 30.11.11 18:47, "Michael Dürig" <md...@apache.org> wrote:
>We should clarify that quickly then. AFAIU it form that list of
>goals/non-goals which is floating around on of the goals is "mvcc, only
>update sessions on refresh()" which is in direct violation of the spec.
If you can make the snapshot isolation feature configurable, that would be
cool. At least for the JCR 1.0 spec the repository implementation could
make the choice, hence this might still be considered in line with the 2.0
spec as well (but just my guess). I expect the spec authors must have
discussed this topic, so maybe they can give a clear answer.
Cheers,
Alex
--
Alexander Klimetschek
Developer // Adobe (Day) // Berlin - Basel
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 30.11.11 17:13, Stefan Guggisberg wrote:
>> Good catch! Two points:
>>
>> 1) Does visible mean immediately visible on next access or visible after
>> refresh? The second case would work with snapshot isolation.
>>
>> 2) Event though I'd like it to be different, spec. compliance hasn't been a
>> top priority on this project so far.
>
> i guess what you mean by 'spec compliance' is 'support of all and every
> optional features in the JCR spec'. and i agree with that.
>
> jr3 should IMO be spec compliant in the sense that it passes the TCK.
> which optional features should be supported is TBD.
We should clarify that quickly then. AFAIU it form that list of
goals/non-goals which is floating around on of the goals is "mvcc, only
update sessions on refresh()" which is in direct violation of the spec.
Also a non-goal on that list is nodetype consistency which AFAIK would
also violate the spec. As my example on the Wiki [1] shows, with
snapshot isolation as it is currently implemented we will not be able to
enforce node type consistency.
Michael
[1]
http://wiki.apache.org/jackrabbit/Transactional%20model%20of%20the%20Microkernel%20based%20Jackrabbit%20prototype
>
> we don't need another JCR 2.0/2.1 reference implementation.
> we already have one with jackrabbit 2.x
>
> cheers
> stefan
>
>>
>> Michael
>>
>>
>>>
>>> [0]
>>>
>>> http://www.day.com/specs/jcr/1.0/7.1.3.4_Seeing_Changes_Made_by_Other_Sessions.html
>>> [1]
>>>
>>> http://www.day.com/specs/jcr/2.0/10_Writing.html#10.1.4%20Visibility%20of%20Changes
>>>
>>> Cheers,
>>> Alex
>>>
>>> --
>>> Alexander Klimetschek
>>> Developer // Adobe (Day) // Berlin - Basel
>>>
>>
Re: [jr3 microkernel] Write skew
Posted by Stefan Guggisberg <st...@gmail.com>.
On Wed, Nov 30, 2011 at 3:21 PM, Michael Dürig <md...@apache.org> wrote:
>
>
> On 30.11.11 13:57, Alexander Klimetschek wrote:
>>
>> I expect there is a lot of code outside that relies on the copy-on-write
>> nature of JR 2 - i.e. that anything the session did not touch yet is
>> always "live". Introducing snapshot isolation (which would be
>> copy-on-read IIUC) would break those cases (but usually these errors
>> will depend on concurrency, thus hard to spot).
>>
>> Now looking at the specs, I am confused: in JCR 1.0 it was up to the
>> implementation to be using copy-on-read or copy-on-write [0]. In JCR 2.0
>> that text was replaced by [1] (IIUC, didn't find anything else) which
>> seems to be defining copy-on-write as the standard behavior now:
>>
>> "A change that is persisted is visible to all other sessions bound to
>> the same persistent workspace that have sufficient read permission."
>>
>> That would mean that JR 3 cannot do snapshot isolation! But I might be
>> missing something...
>
>
> Good catch! Two points:
>
> 1) Does visible mean immediately visible on next access or visible after
> refresh? The second case would work with snapshot isolation.
>
> 2) Event though I'd like it to be different, spec. compliance hasn't been a
> top priority on this project so far.
i guess what you mean by 'spec compliance' is 'support of all and every
optional features in the JCR spec'. and i agree with that.
jr3 should IMO be spec compliant in the sense that it passes the TCK.
which optional features should be supported is TBD.
we don't need another JCR 2.0/2.1 reference implementation.
we already have one with jackrabbit 2.x
cheers
stefan
>
> Michael
>
>
>>
>> [0]
>>
>> http://www.day.com/specs/jcr/1.0/7.1.3.4_Seeing_Changes_Made_by_Other_Sessions.html
>> [1]
>>
>> http://www.day.com/specs/jcr/2.0/10_Writing.html#10.1.4%20Visibility%20of%20Changes
>>
>> Cheers,
>> Alex
>>
>> --
>> Alexander Klimetschek
>> Developer // Adobe (Day) // Berlin - Basel
>>
>
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 12:26, Felix Meschberger wrote:
>>> So we (thinking of Sling amongst other things) might have to adapt
>>> our event listeners to do a Session.refresh at the beginning of the
>>> onEvent method (in case long lived sessions are used).
>>
>> For obvious reasons a snapshot-based implementation would need to
>> implicitly call refresh(true) on a session before delivering
>> observation events.
>
> I don't think so, I would even deem it dangerous. Rather I would accept the fact of the session not being accurate (since this is ok per the spec) and have the listeners refresh themselves as needed.
>
> Maybe there are listeners, that do not even use the session, with which they have been registered in the first place. Then such a refresh by the repository would just burn cycles to no avail.
Another approach we discussed some time ago is to have read only
sessions which are always on the newest revision (i.e. see all saved
changes). Observation listeners on such sessions would then behave as
they did with JR2. Since the underlying Microkernel implements an MVCC
approach such read only sessions would be very easy and cheap to implement.
Michael
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 01.12.2011 um 13:11 schrieb Jukka Zitting:
> On Thu, Dec 1, 2011 at 11:59 AM, Felix Meschberger <fm...@adobe.com> wrote:
>
>> So we (thinking of Sling amongst other things) might have to adapt
>> our event listeners to do a Session.refresh at the beginning of the
>> onEvent method (in case long lived sessions are used).
>
> For obvious reasons a snapshot-based implementation would need to
> implicitly call refresh(true) on a session before delivering
> observation events.
I don't think so, I would even deem it dangerous. Rather I would accept the fact of the session not being accurate (since this is ok per the spec) and have the listeners refresh themselves as needed.
Maybe there are listeners, that do not even use the session, with which they have been registered in the first place. Then such a refresh by the repository would just burn cycles to no avail.
Regards
Felix
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 13:21, Jukka Zitting wrote:
>> IIUC you want to "auto-refresh" all sessions each time observation events
>> are being delivered?
>
> Only those sessions to which events are being delivered, i.e. ones
> that have explicitly registered an observation listener.
Hmm I remain skeptical ;-)
>> Why do we 'obviously' need this? I understand, there will be a change from
>> the current behaviour: while today sessions can't see items anymore which
>> they get DELETE events for, in the new implementation sessions can't see
>> items yet which they get ADD events for. From that perspective the new
>> behaviour seems more flexible and complete: sessions can see deleted
>> items until they do a refresh. Afterwards they can see the added items.
>
> One of the few hard guarantees that we make to an observation listener
> is that when an event is received, that change has already happened.
> If the session state isn't refreshed when an even is delivered, then
> the session would essentially be seeing into the future. I guess that
> might be useful for some things like you mention, but to me that
> sounds counterintuitive and as mentioned by Felix will likely trip off
> many existing clients.
Yes I share this concerns. As I mentioned in earlier reply another
approach we discussed some time ago is to have read only sessions which
are always on the newest revision (i.e. see all saved changes).
Observation listeners on such sessions would then behave as they did
with JR2. Since the underlying Microkernel implements an MVCC approach
such read only sessions would be very easy and cheap to implement.
Michael
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 1:21 PM, Michael Dürig <md...@apache.org> wrote:
> IIUC you want to "auto-refresh" all sessions each time observation events
> are being delivered?
Only those sessions to which events are being delivered, i.e. ones
that have explicitly registered an observation listener.
> Why do we 'obviously' need this? I understand, there will be a change from
> the current behaviour: while today sessions can't see items anymore which
> they get DELETE events for, in the new implementation sessions can't see
> items yet which they get ADD events for. From that perspective the new
> behaviour seems more flexible and complete: sessions can see deleted
> items until they do a refresh. Afterwards they can see the added items.
One of the few hard guarantees that we make to an observation listener
is that when an event is received, that change has already happened.
If the session state isn't refreshed when an even is delivered, then
the session would essentially be seeing into the future. I guess that
might be useful for some things like you mention, but to me that
sounds counterintuitive and as mentioned by Felix will likely trip off
many existing clients.
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
>> So we (thinking of Sling amongst other things) might have to adapt
>> our event listeners to do a Session.refresh at the beginning of the
>> onEvent method (in case long lived sessions are used).
>
> For obvious reasons a snapshot-based implementation would need to
> implicitly call refresh(true) on a session before delivering
> observation events.
IIUC you want to "auto-refresh" all sessions each time observation
events are being delivered?
Why do we 'obviously' need this? I understand, there will be a change
from the current behaviour: while today sessions can't see items anymore
which they get DELETE events for, in the new implementation sessions
can't see items yet which they get ADD events for. From that perspective
the new behaviour seems more flexible and complete: sessions can see
deleted items until they do a refresh. Afterwards they can see the added
items.
Michael
>
>> Sure. I think the main problem with long lived sessions I know of
>> (in Sling based applications) is with Observation Listeners which
>> are quite isolated and can be "fixed" easily -- there just has to
>> be awareness.
>
> The above approach should limit the impact on observation listeners.
>
> Also, to help prevent issues with other kinds of long-lived sessions,
> it might be a good idea to have a configurable auto-refresh policy.
>
> BR,
>
> Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 11:59 AM, Felix Meschberger <fm...@adobe.com> wrote:
> I always thought, what Jackrabbit currently does is required ...
The spec is pretty flexible. The current Jackrabbit implementation is
just one interpretation of it.
> So we (thinking of Sling amongst other things) might have to adapt
> our event listeners to do a Session.refresh at the beginning of the
> onEvent method (in case long lived sessions are used).
For obvious reasons a snapshot-based implementation would need to
implicitly call refresh(true) on a session before delivering
observation events.
> Sure. I think the main problem with long lived sessions I know of
> (in Sling based applications) is with Observation Listeners which
> are quite isolated and can be "fixed" easily -- there just has to
> be awareness.
The above approach should limit the impact on observation listeners.
Also, to help prevent issues with other kinds of long-lived sessions,
it might be a good idea to have a configurable auto-refresh policy.
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 30.11.2011 um 21:53 schrieb Jukka Zitting:
> Hi,
>
> On Wed, Nov 30, 2011 at 3:21 PM, Michael Dürig <md...@apache.org> wrote:
>> 1) Does visible mean immediately visible on next access or visible after
>> refresh? The second case would work with snapshot isolation.
>
> That's up the implementation.
>
> Section 10.11.1 of JSR 283 [1] explicitly allows changes to become
> visible only after a refresh: "[...] items that do not have changes
> pending have their state refreshed to reflect the current persisted
> state, thus revealing changes made by other sessions."
Ok. Thanks for the clarification.
I always thought, what Jackrabbit currently does is required ... So we (thinking of Sling amongst other things) might have to adapt our event listeners to do a Session.refresh at the beginning of the onEvent method (in case long lived sessions are used).
>
> More generally about the write skew -issue; I don't think that's a
> common problem in practice. There aren't too many clients that
> explicitly interleave operations of multiple sessions within a single
> thread. And in a multithreaded setup a client would even with
> Jackrabbit 2.x need to use explicit synchronization to enforce more
> complex content constraints.
Sure. I think the main problem with long lived sessions I know of (in Sling based applications) is with Observation Listeners which are quite isolated and can be "fixed" easily -- there just has to be awareness.
Regards
Felix
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 2:01 PM, Thomas Mueller <mu...@adobe.com> wrote:
>>The save() implementation will need to do an internal refresh(true),
>>check any type and other constraints, and only persist the changes if
>>everything is OK.
>
> What if the node type changes after the refresh, but before the save?
See the second paragraph of that message:
"To prevent this from being a too big blocking operation, we can use
optimistic locking by making the last step of the process conditional
on the repository still being at the state it was when the
refresh(true) call was made. Otherwise the refresh-check-persist cycle
should be restarted."
Or, as mentioned in my reply to Michael, an eventual consistency model
could be used to avoid all blocking.
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 01.12.2011 um 14:47 schrieb Michael Dürig:
>
>>> Currently the microkernel does nothing about this. That's what this whole
>>> discussion is about after all ;-)
>>
>> and please bear in mind that the microkernel project in the jackrabbit sandbox
>> is a prototype and represents WIP... ;)
>
> I do. That's exactly why we should challenge it: to discover possible
> paint points and to discuss how to handle them.
Ok, I am out-a-here -- I cannot paint ;-)
Regards
Felix
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
>> Currently the microkernel does nothing about this. That's what this whole
>> discussion is about after all ;-)
>
> and please bear in mind that the microkernel project in the jackrabbit sandbox
> is a prototype and represents WIP... ;)
I do. That's exactly why we should challenge it: to discover possible
paint points and to discuss how to handle them.
Michael
>
> cheers
> stefan
>
>
>>
>> Eventually consistent approaches are IMO very hard to get right with JCR
>> since most operation are assumed to be persisted after dispatch. One
>> solution I see is to have changes caused by conflict resolution (i.e. during
>> convergence) to appear as normal changes as if they where done by other
>> sessions (see [1]). This would however require changing the revision model
>> of the microkernel from linear shaped to tree shaped.
>>
>> For the current problem I'd rather have the microkernel to expose some
>> test-and-set semantics as Tom describes in [2]. That way a client of the
>> microkernel could apply changes atomically.
>>
>> A third and most general solution which puts away with write skew for good
>> is described in [3]. This requires examining the commit log for certain
>> read-write conflicts on save.
>>
>> Michael
>>
>> [1]
>> http://wiki.apache.org/jackrabbit/Clustering%20the%20Microkernel#Replication
>>
>> [2] http://markmail.org/message/c32jpeoxklgcrybr
>>
>> [3] http://dl.acm.org/citation.cfm?id=1376690
Re: [jr3 microkernel] Write skew
Posted by Stefan Guggisberg <st...@gmail.com>.
On Thu, Dec 1, 2011 at 2:26 PM, Michael Dürig <md...@apache.org> wrote:
>
> On 1.12.11 13:01, Jukka Zitting wrote:
>>
>> Hi,
>>
>> On Thu, Dec 1, 2011 at 1:13 PM, Michael Dürig<md...@apache.org> wrote:
>>>
>>> But this will introduce a race condition. With other words: this only
>>> make
>>> the time window during which such a inconsistency could slip in smaller.
>>> I
>>> think what we'd really need is an atomic test and set operation.
>>
>>
>> Right, either the refresh-check-persist model for save() needs to be
>> atomic (possibly with optimistic locking as described) or we need an
>> eventual consistency model that can automatically resolve such
>> conflicts.
>>
>> Has this already been thought about? How does the current microkernel
>> implementation handle such cases?
>
>
> Currently the microkernel does nothing about this. That's what this whole
> discussion is about after all ;-)
and please bear in mind that the microkernel project in the jackrabbit sandbox
is a prototype and represents WIP... ;)
cheers
stefan
>
> Eventually consistent approaches are IMO very hard to get right with JCR
> since most operation are assumed to be persisted after dispatch. One
> solution I see is to have changes caused by conflict resolution (i.e. during
> convergence) to appear as normal changes as if they where done by other
> sessions (see [1]). This would however require changing the revision model
> of the microkernel from linear shaped to tree shaped.
>
> For the current problem I'd rather have the microkernel to expose some
> test-and-set semantics as Tom describes in [2]. That way a client of the
> microkernel could apply changes atomically.
>
> A third and most general solution which puts away with write skew for good
> is described in [3]. This requires examining the commit log for certain
> read-write conflicts on save.
>
> Michael
>
> [1]
> http://wiki.apache.org/jackrabbit/Clustering%20the%20Microkernel#Replication
>
> [2] http://markmail.org/message/c32jpeoxklgcrybr
>
> [3] http://dl.acm.org/citation.cfm?id=1376690
Re: [jr3 microkernel] Write skew
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>One of the main goals of the Microkernel architecture was to get true
>write scalability.
If you mean concurrent writes within a cluster, then I agree. This is a
problem in Jackrabbit 2, and we need to solve it.
If you mean concurrent writes within the same repository is a goal, then
I'm not aware that we defined it as a goal. If you think it is a goal,
please explain and provide details where it is a problem. Please note
"throughput" problems are unrelated to "concurrency" problems. Saying
"Jackrabbit 2 is to slow therefore it needs to be more concurrent" is the
wrong conclusion. I do agree Jackrabbit 2 is slow. But if Jackrabbit 2
(unclustered) is too slow then we need to improve (and test for)
throughput. Not concurrency.
>If we go down that route, we have to accept that we
>are trading this for consistency.
If we switch to snapshot isolation, then this will affect how collisions
are handled. The behavior will change, and this might be unexpected.
However, there is not just one solution even if using snapshot isolation.
For example the commit could detect write collisions to the same node and
rollback, or merge. It's up to us what behavior we implement. Currently
all MicroKernel implementations merge changes, but this could be changed
if it's a problem.
Regards,
Thomas
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 15:34, Jukka Zitting wrote:
>> But I don't think we should try to increase concurrency of
>> write operations within the *same* repository [...]
>
> That's a valid solution, but it means that we explicitly drop the idea
> of being able to scale concurrent writes linearly by adding new
> cluster nodes. See also http://markmail.org/message/nuxclhqsbihanbjg
> for the original discussion about this.
>
>> - in a cluster, writes do not scale as writing is basically synchronized
>
> This I think is the key issue here. Do we want to break the
> synchronization requirement and thus eliminate this bottleneck
> entirely, or are we happy with more localized performance improvements
> for the concurrent write case?
>
> Ultimately the question boils down to whether we want to support
> massively parallel systems where the number of concurrent writers
> (measured as writes happening within the communication latency across
> the entire cluster) starts at thousands and can ultimately scale go up
> millions. Such systems are practically impossible to implement with
> any non-zero level of global synchronization, which is why I think
> this is such a crucial design decision to be made.
>
> I'm fine if we explicitly decide not to target such use cases. Then we
> can avoid having to worry about the complexities of an eventual
> consistency model. With good optimization it's still possible to
> achieve pretty impressive concurrent performance even with a
> synchronized cluster.
One of the main goals of the Microkernel architecture was to get true
write scalability. If we go down that route, we have to accept that we
are trading this for consistency.
Michael
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 3:49 PM, Thomas Mueller <mu...@adobe.com> wrote:
> But I don't think we should try to increase concurrency of
> write operations within the *same* repository [...]
That's a valid solution, but it means that we explicitly drop the idea
of being able to scale concurrent writes linearly by adding new
cluster nodes. See also http://markmail.org/message/nuxclhqsbihanbjg
for the original discussion about this.
> - in a cluster, writes do not scale as writing is basically synchronized
This I think is the key issue here. Do we want to break the
synchronization requirement and thus eliminate this bottleneck
entirely, or are we happy with more localized performance improvements
for the concurrent write case?
Ultimately the question boils down to whether we want to support
massively parallel systems where the number of concurrent writers
(measured as writes happening within the communication latency across
the entire cluster) starts at thousands and can ultimately scale go up
millions. Such systems are practically impossible to implement with
any non-zero level of global synchronization, which is why I think
this is such a crucial design decision to be made.
I'm fine if we explicitly decide not to target such use cases. Then we
can avoid having to worry about the complexities of an eventual
consistency model. With good optimization it's still possible to
achieve pretty impressive concurrent performance even with a
synchronized cluster.
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>>>But I don't think we should try to increase concurrency of write
>>> operations within the *same* repository because that's not a problem at
>>> all.
>>
>> i beg to differ ;)
>>
>> in jr2 saves are serialized. IMO that's a *real* problem, especially
>>when
>> saving large change sets. this problem can be addressed e.g. with an
>> MVCC based model.
The problem with Jackrabbit isn't that concurrency for write operations is
bad: throughput is bad. This is the main problem. Increasing concurrency
in the save operation will not affect throughput in a meaningful way
(well, most likely it will decrease throughput).
I'm not aware that there is a big problem with large change sets. Anyway
large change sets should be split up into smaller set.
For me, increasing throughput is a lot more important than increasing
concurrency.
>Yes, I agree. It's something I've seen many times in the field
>(consider saving a large pdf in a cms).
Large PDFs are stored in the data store. Large binaries are stored there
well before the save operation, so this is not part of the save operation
at all. Increasing concurrency in the save operation doesn't affect that
in any way.
>you can't scale out the writes in a
>cluster since all writes are serialized for the whole cluster.
Yes, this is a big problem. We need to solve it. One idea is to
synchronize cluster nodes asynchronously, and better support splitting
data into multiple repositories (sharding), for example using virtual
repositories that can be linked together.
Regards,
Thomas
Re: [jr3 microkernel] Write skew
Posted by Bart van der Schans <b....@onehippo.com>.
On Thu, Dec 1, 2011 at 4:26 PM, Stefan Guggisberg
<st...@gmail.com> wrote:
> On Thu, Dec 1, 2011 at 3:49 PM, Thomas Mueller <mu...@adobe.com> wrote:
>> Hi,
>>
>>> A test-and-set operation necessarily requires at least some level of
>>> atomicity which can quickly become a bottleneck for a clustered setup.
>>
>> Test-and-set is a problem in a clustered 'eventually consistent' model,
>> that's true. I don't know how test-and-set could be used in that way.
>>
>> Possibly the easiest solution is that each node modification sets the node
>> type again (to the expected value) even if the user didn't change it.
>>
>>
>> But I don't think we should try to increase concurrency of write
>> operations within the *same* repository because that's not a problem at
>> all.
>
> i beg to differ ;)
>
> in jr2 saves are serialized. IMO that's a *real* problem, especially when
> saving large change sets. this problem can be addressed e.g. with an
> MVCC based model.
Yes, I agree. It's something I've seen many times in the field
(consider saving a large pdf in a cms). What makes the current
situation even worse is that you can't scale out the writes in a
cluster since all writes are serialized for the whole cluster. If you
could scale out and have concurrent writes on different cluster nodes
the problem would be much less worse.
Bart
Re: [jr3 microkernel] Write skew
Posted by Stefan Guggisberg <st...@gmail.com>.
On Thu, Dec 1, 2011 at 3:49 PM, Thomas Mueller <mu...@adobe.com> wrote:
> Hi,
>
>> A test-and-set operation necessarily requires at least some level of
>> atomicity which can quickly become a bottleneck for a clustered setup.
>
> Test-and-set is a problem in a clustered 'eventually consistent' model,
> that's true. I don't know how test-and-set could be used in that way.
>
> Possibly the easiest solution is that each node modification sets the node
> type again (to the expected value) even if the user didn't change it.
>
>
> But I don't think we should try to increase concurrency of write
> operations within the *same* repository because that's not a problem at
> all.
i beg to differ ;)
in jr2 saves are serialized. IMO that's a *real* problem, especially when
saving large change sets. this problem can be addressed e.g. with an
MVCC based model.
cheers
stefan
> Jackrabbit 2 is slow for other reasons. In my view, the main problems
> for Jackrabbit 2 are:
>
> - the more nodes the slower because of the randomly distributed node ids
> - the more open sessions the slower because of internal event processing
> - the more child nodes the slower because the list is stored as one unit
> - jackrabbit core is slow internally for other reasons we didn't analyze
> - indexing everything with Lucene is a performance problem at some point
> - in a cluster, writes do not scale as writing is basically synchronized
>
> None of that problems is related to transaction isolation. I'm not saying
> we shouldn't try to use MVCC. But it never was a problem that Jackrabbit 2
> doesn't use MVCC.
>
> Regards,
>
> Thomas
>
Re: [jr3 microkernel] Write skew
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
> A test-and-set operation necessarily requires at least some level of
> atomicity which can quickly become a bottleneck for a clustered setup.
Test-and-set is a problem in a clustered 'eventually consistent' model,
that's true. I don't know how test-and-set could be used in that way.
Possibly the easiest solution is that each node modification sets the node
type again (to the expected value) even if the user didn't change it.
But I don't think we should try to increase concurrency of write
operations within the *same* repository because that's not a problem at
all. Jackrabbit 2 is slow for other reasons. In my view, the main problems
for Jackrabbit 2 are:
- the more nodes the slower because of the randomly distributed node ids
- the more open sessions the slower because of internal event processing
- the more child nodes the slower because the list is stored as one unit
- jackrabbit core is slow internally for other reasons we didn't analyze
- indexing everything with Lucene is a performance problem at some point
- in a cluster, writes do not scale as writing is basically synchronized
None of that problems is related to transaction isolation. I'm not saying
we shouldn't try to use MVCC. But it never was a problem that Jackrabbit 2
doesn't use MVCC.
Regards,
Thomas
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 15:10, Jukka Zitting wrote:
>>> If I understand the paper correctly, also this approach requires
>>> synchronization to establish the serialization order of transactions.
>>
>> No I don't think so. What it does is, it cancels problematic transactions.
>> Detecting these requires finding certain read-write conflicts between
>> transactions.
>
> But if there are two concurrent transactions, the commit of the other
> one needs to wait until the first one has been processed before it can
> complete the check against conflicts. Otherwise how can a conflicting
> commit be made to fail?
>
I see, that's right. Unless the other transaction has access to the
operations of the first transaction. In that case - and if it decides to
abort due to a conflict and later the first also decides to abort - the
other transaction would have aborted unnecessarily. That shouldn't hurt
too much though.
Michael
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 4:02 PM, Michael Dürig <md...@apache.org> wrote:
> On 1.12.11 14:25, Jukka Zitting wrote:
>> If I understand the paper correctly, also this approach requires
>> synchronization to establish the serialization order of transactions.
>
> No I don't think so. What it does is, it cancels problematic transactions.
> Detecting these requires finding certain read-write conflicts between
> transactions.
But if there are two concurrent transactions, the commit of the other
one needs to wait until the first one has been processed before it can
complete the check against conflicts. Otherwise how can a conflicting
commit be made to fail?
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 14:25, Jukka Zitting wrote:
>> A third and most general solution which puts away with write skew for good
>> is described in [3]. This requires examining the commit log for certain
>> read-write conflicts on save.
>
> If I understand the paper correctly, also this approach requires
> synchronization to establish the serialization order of transactions.
No I don't think so. What it does is, it cancels problematic
transactions. Detecting these requires finding certain read-write
conflicts between transactions. In a way this resembles serialization
graph testing. Under snapshot isolation this seems to become easier than
in the general case due to some 'nice' properties of the serialization
graphs involved here. See the remarks about 'vulnerable edges' and
'dangerous structures' in section 2.2 of the paper.
Michael
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 15:24, Bart van der Schans wrote:
>> I think the toughest question here is how to merge conflicting commits.
>> There are some quite pathological cases [1]: consider the case where a node
>> x is moved such that it becomes a child of node y in one session and at them
>> same time node y is moved such that it becomes a child of node x in another
>> session. The situation is inherently symmetric and neither move can follow
>> the other. On merge there seems to be no better way than to undo one (or
>> both?) of the changes of the sessions.
>
> Isn't the simplest conflicting commit case something like two sessions
> setting the same property on the same node to a different value? You
> can't merge that and you basically have the same options: one session
> (eventually) "wins" (the first or the second write? what is first?),
> or both (eventually) loose or you end up with two different versions
> of the property/node co-existing, both being valid.
Yes that's probably the simplest conflict case. But there is a subtle
difference: in this case one operation can follow the other. So some
arbitration rule (vector clocks for example) could be used for making
them serializable. In the case of the conflicting move operations such
an arbitration is not possible.
> I would expect these kind of scenarios have been worked out in other
> "eventual consistent" nosql stores. Could we learn something from them
> about conflict resolution? Or can we even use one of those as
> persistence store and let the store itself work out the conflict
> resolution?
One big difference here is the hierarchical nature of JCR. Other nosql
data bases might not have to cope with hierarchical move conflicts like
the above.
Michael
>
> Bart
>
>
>>
>> Michael
>>
>> [1]
>> http://wiki.apache.org/jackrabbit/Clustering%20the%20Microkernel#Replication
>>
>
>
>
Re: [jr3 microkernel] Write skew
Posted by Bart van der Schans <b....@onehippo.com>.
On Thu, Dec 1, 2011 at 3:50 PM, Michael Dürig <md...@apache.org> wrote:
>
>
> On 1.12.11 14:25, Jukka Zitting wrote:
>>>
>>> Eventually consistent approaches are IMO very hard to get right with JCR
>>> since most operation are assumed to be persisted after dispatch. One
>>> solution I see is to have changes caused by conflict resolution (i.e.
>>> during
>>> convergence) to appear as normal changes as if they where done by other
>>> sessions (see [1]). This would however require changing the revision
>>> model
>>> of the microkernel from linear shaped to tree shaped.
>>
>>
>> This to me sounds like the best approach to take if only there's a way
>> to solve the problems you mention. The main benefit of an eventually
>> consistent model is that a save() call can safely return without the
>> repository having to synchronizes with other cluster nodes or
>> ultimately even other threads within a single node.
>
>
> I think the toughest question here is how to merge conflicting commits.
> There are some quite pathological cases [1]: consider the case where a node
> x is moved such that it becomes a child of node y in one session and at them
> same time node y is moved such that it becomes a child of node x in another
> session. The situation is inherently symmetric and neither move can follow
> the other. On merge there seems to be no better way than to undo one (or
> both?) of the changes of the sessions.
Isn't the simplest conflicting commit case something like two sessions
setting the same property on the same node to a different value? You
can't merge that and you basically have the same options: one session
(eventually) "wins" (the first or the second write? what is first?),
or both (eventually) loose or you end up with two different versions
of the property/node co-existing, both being valid.
I would expect these kind of scenarios have been worked out in other
"eventual consistent" nosql stores. Could we learn something from them
about conflict resolution? Or can we even use one of those as
persistence store and let the store itself work out the conflict
resolution?
Bart
>
> Michael
>
> [1]
> http://wiki.apache.org/jackrabbit/Clustering%20the%20Microkernel#Replication
>
--
Amsterdam - Oosteinde 11, 1017 WT Amsterdam
Boston - 1 Broadway, Cambridge, MA 02142
US +1 877 414 4776 (toll free)
Europe +31(0)20 522 4466
www.onehippo.com
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 14:25, Jukka Zitting wrote:
>> Eventually consistent approaches are IMO very hard to get right with JCR
>> since most operation are assumed to be persisted after dispatch. One
>> solution I see is to have changes caused by conflict resolution (i.e. during
>> convergence) to appear as normal changes as if they where done by other
>> sessions (see [1]). This would however require changing the revision model
>> of the microkernel from linear shaped to tree shaped.
>
> This to me sounds like the best approach to take if only there's a way
> to solve the problems you mention. The main benefit of an eventually
> consistent model is that a save() call can safely return without the
> repository having to synchronizes with other cluster nodes or
> ultimately even other threads within a single node.
I think the toughest question here is how to merge conflicting commits.
There are some quite pathological cases [1]: consider the case where a
node x is moved such that it becomes a child of node y in one session
and at them same time node y is moved such that it becomes a child of
node x in another session. The situation is inherently symmetric and
neither move can follow the other. On merge there seems to be no better
way than to undo one (or both?) of the changes of the sessions.
Michael
[1]
http://wiki.apache.org/jackrabbit/Clustering%20the%20Microkernel#Replication
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 14:25, Jukka Zitting wrote:
>> For the current problem I'd rather have the microkernel to expose some
>> test-and-set semantics as Tom describes in [2].
>
> A test-and-set operation necessarily requires at least some level of
> atomicity which can quickly become a bottleneck for a clustered setup.
Yes that's right. I don't think there is a big problem however in this
case. i.e. when setting node types.
Michael
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 14:25, Jukka Zitting wrote:
>> For the current problem I'd rather have the microkernel to expose some
>> test-and-set semantics as Tom describes in [2].
>
> A test-and-set operation necessarily requires at least some level of
> atomicity which can quickly become a bottleneck for a clustered setup.
Yes but that critical section can be very small: since the changes are
written to disk ahead, locking is only required when writing the new
revision id to the journal. With the test and set approach, the only
addition is to do the test inside the lock. With (correct) double
checked locking this can even be further optimized.
Michael
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 2:26 PM, Michael Dürig <md...@apache.org> wrote:
> Currently the microkernel does nothing about this. That's what this whole
> discussion is about after all ;-)
I see this as probably the most significant design decision to be
taken for Jackrabbit 3 since the approach taken will have a major
impact on repository scalability and performance in terms of
concurrent writes. Good that this discussion is happening on the list!
> Eventually consistent approaches are IMO very hard to get right with JCR
> since most operation are assumed to be persisted after dispatch. One
> solution I see is to have changes caused by conflict resolution (i.e. during
> convergence) to appear as normal changes as if they where done by other
> sessions (see [1]). This would however require changing the revision model
> of the microkernel from linear shaped to tree shaped.
This to me sounds like the best approach to take if only there's a way
to solve the problems you mention. The main benefit of an eventually
consistent model is that a save() call can safely return without the
repository having to synchronizes with other cluster nodes or
ultimately even other threads within a single node.
> For the current problem I'd rather have the microkernel to expose some
> test-and-set semantics as Tom describes in [2].
A test-and-set operation necessarily requires at least some level of
atomicity which can quickly become a bottleneck for a clustered setup.
> A third and most general solution which puts away with write skew for good
> is described in [3]. This requires examining the commit log for certain
> read-write conflicts on save.
If I understand the paper correctly, also this approach requires
synchronization to establish the serialization order of transactions.
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 13:01, Jukka Zitting wrote:
> Hi,
>
> On Thu, Dec 1, 2011 at 1:13 PM, Michael Dürig<md...@apache.org> wrote:
>> But this will introduce a race condition. With other words: this only make
>> the time window during which such a inconsistency could slip in smaller. I
>> think what we'd really need is an atomic test and set operation.
>
> Right, either the refresh-check-persist model for save() needs to be
> atomic (possibly with optimistic locking as described) or we need an
> eventual consistency model that can automatically resolve such
> conflicts.
>
> Has this already been thought about? How does the current microkernel
> implementation handle such cases?
Currently the microkernel does nothing about this. That's what this
whole discussion is about after all ;-)
Eventually consistent approaches are IMO very hard to get right with JCR
since most operation are assumed to be persisted after dispatch. One
solution I see is to have changes caused by conflict resolution (i.e.
during convergence) to appear as normal changes as if they where done by
other sessions (see [1]). This would however require changing the
revision model of the microkernel from linear shaped to tree shaped.
For the current problem I'd rather have the microkernel to expose some
test-and-set semantics as Tom describes in [2]. That way a client of the
microkernel could apply changes atomically.
A third and most general solution which puts away with write skew for
good is described in [3]. This requires examining the commit log for
certain read-write conflicts on save.
Michael
[1]
http://wiki.apache.org/jackrabbit/Clustering%20the%20Microkernel#Replication
[2] http://markmail.org/message/c32jpeoxklgcrybr
[3] http://dl.acm.org/citation.cfm?id=1376690
Re: [jr3 microkernel] Write skew
Posted by Stefan Guggisberg <st...@gmail.com>.
On Thu, Dec 1, 2011 at 2:01 PM, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> On Thu, Dec 1, 2011 at 1:13 PM, Michael Dürig <md...@apache.org> wrote:
>> But this will introduce a race condition. With other words: this only make
>> the time window during which such a inconsistency could slip in smaller. I
>> think what we'd really need is an atomic test and set operation.
>
> Right, either the refresh-check-persist model for save() needs to be
> atomic (possibly with optimistic locking as described) or we need an
> eventual consistency model that can automatically resolve such
> conflicts.
>
> Has this already been thought about? How does the current microkernel
> implementation handle such cases?
the current microkernel prototype commit roughly works as follows:
1. all changes are persisted
2. within a synchronized block,
2a. - if the head revision has changed since we
started persisting our changes,
a 3-way merge is done starting from the
root node (ideally only the root node
needs to be merged)
2b. - the new head revision is persisted
if an unmergeable conflict is detected in step 2a,
an exception is thrown. that currently only happens
on concurrent property modifications.
details can be found here (doCommit & mergeNode methods) : [0]
cheers
stefan
[0] http://svn.apache.org/viewvc/jackrabbit/sandbox/microkernel/src/main/java/org/apache/jackrabbit/mk/model/CommitBuilder.java?view=markup
>
> BR,
>
> Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 1:13 PM, Michael Dürig <md...@apache.org> wrote:
> But this will introduce a race condition. With other words: this only make
> the time window during which such a inconsistency could slip in smaller. I
> think what we'd really need is an atomic test and set operation.
Right, either the refresh-check-persist model for save() needs to be
atomic (possibly with optimistic locking as described) or we need an
eventual consistency model that can automatically resolve such
conflicts.
Has this already been thought about? How does the current microkernel
implementation handle such cases?
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 1.12.11 12:05, Jukka Zitting wrote:
> Hi,
>
> On Thu, Dec 1, 2011 at 11:07 AM, Michael Dürig<md...@apache.org> wrote:
>> Note however, that for the same reason we are currently not able to
>> guarantee node type consistency:
>> [...]
>> Both sessions will successfully complete since for each of them the
>> consistency property (nt:file cannot have a child named "foo"). holds.
>
> The save() implementation will need to do an internal refresh(true),
> check any type and other constraints, and only persist the changes if
> everything is OK.
>
> To prevent this from being a too big blocking operation, we can use
> optimistic locking by making the last step of the process conditional
> on the repository still being at the state it was when the
> refresh(true) call was made. Otherwise the refresh-check-persist cycle
> should be restarted.
But this will introduce a race condition. With other words: this only
make the time window during which such a inconsistency could slip in
smaller. I think what we'd really need is an atomic test and set operation.
Michael
>
> BR,
>
> Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>The save() implementation will need to do an internal refresh(true),
>check any type and other constraints, and only persist the changes if
>everything is OK.
What if the node type changes after the refresh, but before the save?
Regards,
Thomas
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Dec 1, 2011 at 11:07 AM, Michael Dürig <md...@apache.org> wrote:
> Note however, that for the same reason we are currently not able to
> guarantee node type consistency:
> [...]
> Both sessions will successfully complete since for each of them the
> consistency property (nt:file cannot have a child named "foo"). holds.
The save() implementation will need to do an internal refresh(true),
check any type and other constraints, and only persist the changes if
everything is OK.
To prevent this from being a too big blocking operation, we can use
optimistic locking by making the last step of the process conditional
on the repository still being at the state it was when the
refresh(true) call was made. Otherwise the refresh-check-persist cycle
should be restarted.
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 30.11.11 20:53, Jukka Zitting wrote:
>> 1) Does visible mean immediately visible on next access or visible after
>> refresh? The second case would work with snapshot isolation.
>
> That's up the implementation.
>
> Section 10.11.1 of JSR 283 [1] explicitly allows changes to become
> visible only after a refresh: "[...] items that do not have changes
> pending have their state refreshed to reflect the current persisted
> state, thus revealing changes made by other sessions."
>
> More generally about the write skew -issue; I don't think that's a
> common problem in practice. There aren't too many clients that
> explicitly interleave operations of multiple sessions within a single
> thread. And in a multithreaded setup a client would even with
> Jackrabbit 2.x need to use explicit synchronization to enforce more
> complex content constraints.
Ok, sounds reasonable to me as far as application constraints are
involved. Note however, that for the same reason we are currently not
able to guarantee node type consistency:
session1 = login()
session2 = login()
n1 = session1.getNode("n")
n1.setPrimaryType(nt:file)
n1.save()
n2 = session2.getNode("n")
n2.addNode("foo")
n2.save()
Both sessions will successfully complete since for each of them the
consistency property (nt:file cannot have a child named "foo"). holds.
The combined effect however will result in node n being of type nt:file
and having a child node foo.
We could fix this by implementing Tom's test-and-set operator [1] in the
microkernel.
Michael
[1] http://markmail.org/message/c32jpeoxklgcrybr
>
> [1] http://www.day.com/specs/jcr/2.0/10_Writing.html#10.11.1%20Refresh
>
> BR,
>
> Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Wed, Nov 30, 2011 at 3:21 PM, Michael Dürig <md...@apache.org> wrote:
> 1) Does visible mean immediately visible on next access or visible after
> refresh? The second case would work with snapshot isolation.
That's up the implementation.
Section 10.11.1 of JSR 283 [1] explicitly allows changes to become
visible only after a refresh: "[...] items that do not have changes
pending have their state refreshed to reflect the current persisted
state, thus revealing changes made by other sessions."
More generally about the write skew -issue; I don't think that's a
common problem in practice. There aren't too many clients that
explicitly interleave operations of multiple sessions within a single
thread. And in a multithreaded setup a client would even with
Jackrabbit 2.x need to use explicit synchronization to enforce more
complex content constraints.
[1] http://www.day.com/specs/jcr/2.0/10_Writing.html#10.11.1%20Refresh
BR,
Jukka Zitting
Re: [jr3 microkernel] Write skew
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
>>2) Event though I'd like it to be different, spec. compliance hasn't
>> been a top priority on this project so far.
>
>Autsch ! You are going to implement a spec, so compliance is IMHO one of
>the goals and as such is one of the top priorities IMHO .. Even more so
>than performance !
It depends on the spec. Sometimes it's better to be not compliant or to
only support a subset.
In many cases performance is more important than the number of features.
Regards,
Thomas
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 30.11.2011 um 15:21 schrieb Michael Dürig:
> 2) Event though I'd like it to be different, spec. compliance hasn't
> been a top priority on this project so far.
Autsch ! You are going to implement a spec, so compliance is IMHO one of the goals and as such is one of the top priorities IMHO .. Even more so than performance !
Any implementation which is not compliant may be as compliant as it may be is not worth it because the user never knows what to expect.
Regards
Felix
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 30.11.2011 um 15:21 schrieb Michael Dürig:
>
>
> On 30.11.11 13:57, Alexander Klimetschek wrote:
>> I expect there is a lot of code outside that relies on the copy-on-write
>> nature of JR 2 - i.e. that anything the session did not touch yet is
>> always "live". Introducing snapshot isolation (which would be
>> copy-on-read IIUC) would break those cases (but usually these errors
>> will depend on concurrency, thus hard to spot).
>>
>> Now looking at the specs, I am confused: in JCR 1.0 it was up to the
>> implementation to be using copy-on-read or copy-on-write [0]. In JCR 2.0
>> that text was replaced by [1] (IIUC, didn't find anything else) which
>> seems to be defining copy-on-write as the standard behavior now:
>>
>> "A change that is persisted is visible to all other sessions bound to
>> the same persistent workspace that have sufficient read permission."
>>
>> That would mean that JR 3 cannot do snapshot isolation! But I might be
>> missing something...
>
> Good catch! Two points:
>
> 1) Does visible mean immediately visible on next access or visible after
> refresh? The second case would work with snapshot isolation.
Well "visible after refresh" is a conditional visibility IMHO, while "visible" as in the spec is more like unconditional (always under the assumption of having read access, of course).
Regards
Felix
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 30.11.11 13:57, Alexander Klimetschek wrote:
> I expect there is a lot of code outside that relies on the copy-on-write
> nature of JR 2 - i.e. that anything the session did not touch yet is
> always "live". Introducing snapshot isolation (which would be
> copy-on-read IIUC) would break those cases (but usually these errors
> will depend on concurrency, thus hard to spot).
>
> Now looking at the specs, I am confused: in JCR 1.0 it was up to the
> implementation to be using copy-on-read or copy-on-write [0]. In JCR 2.0
> that text was replaced by [1] (IIUC, didn't find anything else) which
> seems to be defining copy-on-write as the standard behavior now:
>
> "A change that is persisted is visible to all other sessions bound to
> the same persistent workspace that have sufficient read permission."
>
> That would mean that JR 3 cannot do snapshot isolation! But I might be
> missing something...
Good catch! Two points:
1) Does visible mean immediately visible on next access or visible after
refresh? The second case would work with snapshot isolation.
2) Event though I'd like it to be different, spec. compliance hasn't
been a top priority on this project so far.
Michael
>
> [0]
> http://www.day.com/specs/jcr/1.0/7.1.3.4_Seeing_Changes_Made_by_Other_Sessions.html
> [1]
> http://www.day.com/specs/jcr/2.0/10_Writing.html#10.1.4%20Visibility%20of%20Changes
>
> Cheers,
> Alex
>
> --
> Alexander Klimetschek
> Developer // Adobe (Day) // Berlin - Basel
>
Re: [jr3 microkernel] Write skew
Posted by Alexander Klimetschek <ak...@adobe.com>.
On 30.11.11 14:27, "Felix Meschberger" <fm...@adobe.com>> wrote:
Am 30.11.2011 um 14:20 schrieb Michael Dürig:
That's a different story for a different thread ;-) In a nutshell: under
snapshot isolation sessions will see ADD events for items which do not
exist for them. OTHO when they get an DELETE event the item will still
exist for them. Note how this is reverse from the situation in JR2.
Hmm, this will probably effectively break many if not most current ObservationListener implementations. Good to know.
I expect there is a lot of code outside that relies on the copy-on-write nature of JR 2 - i.e. that anything the session did not touch yet is always "live". Introducing snapshot isolation (which would be copy-on-read IIUC) would break those cases (but usually these errors will depend on concurrency, thus hard to spot).
Now looking at the specs, I am confused: in JCR 1.0 it was up to the implementation to be using copy-on-read or copy-on-write [0]. In JCR 2.0 that text was replaced by [1] (IIUC, didn't find anything else) which seems to be defining copy-on-write as the standard behavior now:
"A change that is persisted is visible to all other sessions bound to the same persistent workspace that have sufficient read permission."
That would mean that JR 3 cannot do snapshot isolation! But I might be missing something...
[0] http://www.day.com/specs/jcr/1.0/7.1.3.4_Seeing_Changes_Made_by_Other_Sessions.html
[1] http://www.day.com/specs/jcr/2.0/10_Writing.html#10.1.4%20Visibility%20of%20Changes
Cheers,
Alex
--
Alexander Klimetschek
Developer // Adobe (Day) // Berlin - Basel
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 30.11.2011 um 14:20 schrieb Michael Dürig:
>
> On 30.11.11 13:07, Felix Meschberger wrote:
>> Say, s1 listens on changes to node x, s2 updates node x and on receiving the event s1 would access the new property. What happens ?
>
> That's a different story for a different thread ;-) In a nutshell: under
> snapshot isolation sessions will see ADD events for items which do not
> exist for them. OTHO when they get an DELETE event the item will still
> exist for them. Note how this is reverse from the situation in JR2.
Hmm, this will probably effectively break many if not most current ObservationListener implementations. Good to know.
Still what could listeners possibly do here ? Sesssion.refresh ? Or create a short-lived session to handle the event ?
Regards
Felix
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
On 30.11.11 13:07, Felix Meschberger wrote:
> Hi,
>
> Oops, sorry, Tom just hinted at my misreading of the calculation. Of course session3 is expected to have -1 (the op is + not -).
>
> The problem here is for session2 really. What could session2 possibly do ?
The repository implementation could check whether there are certain
read-write conflicts between session2 and session1 and throw on save if
so. Details are here: http://dl.acm.org/citation.cfm?id=1376690
> to a refresh before calculating ?
That would only result in a race condition since session1 might well
save after session2 refreshed.
> Are we postulating short-lived sessions ? What about sessions backing ObservationListeners (which are inherently long-lived) ? Do they get accurate data back ?
>
> Say, s1 listens on changes to node x, s2 updates node x and on receiving the event s1 would access the new property. What happens ?
That's a different story for a different thread ;-) In a nutshell: under
snapshot isolation sessions will see ADD events for items which do not
exist for them. OTHO when they get an DELETE event the item will still
exist for them. Note how this is reverse from the situation in JR2.
Michael
>
>
> Regards
> Felix
>
> Am 30.11.2011 um 14:02 schrieb Michael Dürig:
>
>>
>>
>>> This kind of scares me a bit: What could session3 possibly do about this here ?
>>>
>>> It looks like session3 is created after everything is set and done, thus it is expected that p1==p2==-1 and thus p1-p2==0.
>>
>> session3 can't do anything about it, session 2 could. See [1]. This is a
>> consequence inherent to snapshot isolation.
>>
>> [1] http://dl.acm.org/citation.cfm?id=1376690
>>
>>
>>>
>>> Regards
>>> Felix
>>>
>>> Am 30.11.2011 um 13:38 schrieb Michael Dürig:
>>>
>>>>
>>>> Hi,
>>>>
>>>> As documented earlier [1] new Mircokernel based JCR implementations will
>>>> most likely introduce snapshot isolation for sessions. While I think
>>>> this is a good thing in general, it also introduces potentially
>>>> troublesome write skew: application level constraints might hold locally
>>>> (i.e. per session) but might fail globally. That is, in order to enforce
>>>> some constraints, applications might need to implement explicit cross
>>>> session synchronization mechanisms instead of being able to rely on
>>>> session isolation. The following test case demonstrates the issue.
>>>>
>>>> /**
>>>> * Trans-session isolation differs from Jackrabbit 2.
>>>> * Snapshot isolation can result in write skew as this
>>>> * test demonstrates: the check method enforces an
>>>> * application logic constraint which says that the sum
>>>> * of the properties p1 and p2 must not be negative. While
>>>> * session1 and session2 each enforce this constraint before
>>>> * saving, the constraint might not hold globally as can be
>>>> * seen in session3.
>>>> */
>>>> @Test
>>>> public void testSessionIsolation() throws RepositoryException {
>>>> Repository repository = getRepository();
>>>>
>>>> Session session0 = repository.login();
>>>> Node testNode = session0.getNode("/").addNode("testNode");
>>>> testNode.setProperty("p1", 1);
>>>> testNode.setProperty("p2", 1);
>>>> session0.save();
>>>> check(getSession());
>>>>
>>>> Session session1 = repository.login();
>>>> Session session2 = repository.login();
>>>>
>>>> session1.getNode("/testNode").setProperty("p1", -1);
>>>> check(session1);
>>>> session1.save();
>>>>
>>>> session2.getNode("/testNode").setProperty("p2", -1);
>>>> check(session2); // Throws on JR2 but not on JR3
>>>> session2.save();
>>>>
>>>> Session session3 = repository.login();
>>>> check(session3); // Throws on JR3
>>>> }
>>>>
>>>> private static void check(Session session) throws RepositoryException {
>>>> if (session.getNode("/testNode").getProperty("p1").getLong() +
>>>> session.getNode("/testNode").getProperty("p2").getLong()< 0) {
>>>> fail("p1 + p2< 0");
>>>> }
>>>> }
>>>>
>>>> Michael
>>>>
>>>> [1]
>>>> http://wiki.apache.org/jackrabbit/Transactional%20model%20of%20the%20Microkernel%20based%20Jackrabbit%20prototype
>>>
>
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Oops, sorry, Tom just hinted at my misreading of the calculation. Of course session3 is expected to have -1 (the op is + not -).
The problem here is for session2 really. What could session2 possibly do ? to a refresh before calculating ? Are we postulating short-lived sessions ? What about sessions backing ObservationListeners (which are inherently long-lived) ? Do they get accurate data back ?
Say, s1 listens on changes to node x, s2 updates node x and on receiving the event s1 would access the new property. What happens ?
Regards
Felix
Am 30.11.2011 um 14:02 schrieb Michael Dürig:
>
>
>> This kind of scares me a bit: What could session3 possibly do about this here ?
>>
>> It looks like session3 is created after everything is set and done, thus it is expected that p1==p2==-1 and thus p1-p2==0.
>
> session3 can't do anything about it, session 2 could. See [1]. This is a
> consequence inherent to snapshot isolation.
>
> [1] http://dl.acm.org/citation.cfm?id=1376690
>
>
>>
>> Regards
>> Felix
>>
>> Am 30.11.2011 um 13:38 schrieb Michael Dürig:
>>
>>>
>>> Hi,
>>>
>>> As documented earlier [1] new Mircokernel based JCR implementations will
>>> most likely introduce snapshot isolation for sessions. While I think
>>> this is a good thing in general, it also introduces potentially
>>> troublesome write skew: application level constraints might hold locally
>>> (i.e. per session) but might fail globally. That is, in order to enforce
>>> some constraints, applications might need to implement explicit cross
>>> session synchronization mechanisms instead of being able to rely on
>>> session isolation. The following test case demonstrates the issue.
>>>
>>> /**
>>> * Trans-session isolation differs from Jackrabbit 2.
>>> * Snapshot isolation can result in write skew as this
>>> * test demonstrates: the check method enforces an
>>> * application logic constraint which says that the sum
>>> * of the properties p1 and p2 must not be negative. While
>>> * session1 and session2 each enforce this constraint before
>>> * saving, the constraint might not hold globally as can be
>>> * seen in session3.
>>> */
>>> @Test
>>> public void testSessionIsolation() throws RepositoryException {
>>> Repository repository = getRepository();
>>>
>>> Session session0 = repository.login();
>>> Node testNode = session0.getNode("/").addNode("testNode");
>>> testNode.setProperty("p1", 1);
>>> testNode.setProperty("p2", 1);
>>> session0.save();
>>> check(getSession());
>>>
>>> Session session1 = repository.login();
>>> Session session2 = repository.login();
>>>
>>> session1.getNode("/testNode").setProperty("p1", -1);
>>> check(session1);
>>> session1.save();
>>>
>>> session2.getNode("/testNode").setProperty("p2", -1);
>>> check(session2); // Throws on JR2 but not on JR3
>>> session2.save();
>>>
>>> Session session3 = repository.login();
>>> check(session3); // Throws on JR3
>>> }
>>>
>>> private static void check(Session session) throws RepositoryException {
>>> if (session.getNode("/testNode").getProperty("p1").getLong() +
>>> session.getNode("/testNode").getProperty("p2").getLong()< 0) {
>>> fail("p1 + p2< 0");
>>> }
>>> }
>>>
>>> Michael
>>>
>>> [1]
>>> http://wiki.apache.org/jackrabbit/Transactional%20model%20of%20the%20Microkernel%20based%20Jackrabbit%20prototype
>>
Re: [jr3 microkernel] Write skew
Posted by Michael Dürig <md...@apache.org>.
> This kind of scares me a bit: What could session3 possibly do about this here ?
>
> It looks like session3 is created after everything is set and done, thus it is expected that p1==p2==-1 and thus p1-p2==0.
session3 can't do anything about it, session 2 could. See [1]. This is a
consequence inherent to snapshot isolation.
[1] http://dl.acm.org/citation.cfm?id=1376690
>
> Regards
> Felix
>
> Am 30.11.2011 um 13:38 schrieb Michael Dürig:
>
>>
>> Hi,
>>
>> As documented earlier [1] new Mircokernel based JCR implementations will
>> most likely introduce snapshot isolation for sessions. While I think
>> this is a good thing in general, it also introduces potentially
>> troublesome write skew: application level constraints might hold locally
>> (i.e. per session) but might fail globally. That is, in order to enforce
>> some constraints, applications might need to implement explicit cross
>> session synchronization mechanisms instead of being able to rely on
>> session isolation. The following test case demonstrates the issue.
>>
>> /**
>> * Trans-session isolation differs from Jackrabbit 2.
>> * Snapshot isolation can result in write skew as this
>> * test demonstrates: the check method enforces an
>> * application logic constraint which says that the sum
>> * of the properties p1 and p2 must not be negative. While
>> * session1 and session2 each enforce this constraint before
>> * saving, the constraint might not hold globally as can be
>> * seen in session3.
>> */
>> @Test
>> public void testSessionIsolation() throws RepositoryException {
>> Repository repository = getRepository();
>>
>> Session session0 = repository.login();
>> Node testNode = session0.getNode("/").addNode("testNode");
>> testNode.setProperty("p1", 1);
>> testNode.setProperty("p2", 1);
>> session0.save();
>> check(getSession());
>>
>> Session session1 = repository.login();
>> Session session2 = repository.login();
>>
>> session1.getNode("/testNode").setProperty("p1", -1);
>> check(session1);
>> session1.save();
>>
>> session2.getNode("/testNode").setProperty("p2", -1);
>> check(session2); // Throws on JR2 but not on JR3
>> session2.save();
>>
>> Session session3 = repository.login();
>> check(session3); // Throws on JR3
>> }
>>
>> private static void check(Session session) throws RepositoryException {
>> if (session.getNode("/testNode").getProperty("p1").getLong() +
>> session.getNode("/testNode").getProperty("p2").getLong()< 0) {
>> fail("p1 + p2< 0");
>> }
>> }
>>
>> Michael
>>
>> [1]
>> http://wiki.apache.org/jackrabbit/Transactional%20model%20of%20the%20Microkernel%20based%20Jackrabbit%20prototype
>
Re: [jr3 microkernel] Write skew
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
This kind of scares me a bit: What could session3 possibly do about this here ?
It looks like session3 is created after everything is set and done, thus it is expected that p1==p2==-1 and thus p1-p2==0.
Regards
Felix
Am 30.11.2011 um 13:38 schrieb Michael Dürig:
>
> Hi,
>
> As documented earlier [1] new Mircokernel based JCR implementations will
> most likely introduce snapshot isolation for sessions. While I think
> this is a good thing in general, it also introduces potentially
> troublesome write skew: application level constraints might hold locally
> (i.e. per session) but might fail globally. That is, in order to enforce
> some constraints, applications might need to implement explicit cross
> session synchronization mechanisms instead of being able to rely on
> session isolation. The following test case demonstrates the issue.
>
> /**
> * Trans-session isolation differs from Jackrabbit 2.
> * Snapshot isolation can result in write skew as this
> * test demonstrates: the check method enforces an
> * application logic constraint which says that the sum
> * of the properties p1 and p2 must not be negative. While
> * session1 and session2 each enforce this constraint before
> * saving, the constraint might not hold globally as can be
> * seen in session3.
> */
> @Test
> public void testSessionIsolation() throws RepositoryException {
> Repository repository = getRepository();
>
> Session session0 = repository.login();
> Node testNode = session0.getNode("/").addNode("testNode");
> testNode.setProperty("p1", 1);
> testNode.setProperty("p2", 1);
> session0.save();
> check(getSession());
>
> Session session1 = repository.login();
> Session session2 = repository.login();
>
> session1.getNode("/testNode").setProperty("p1", -1);
> check(session1);
> session1.save();
>
> session2.getNode("/testNode").setProperty("p2", -1);
> check(session2); // Throws on JR2 but not on JR3
> session2.save();
>
> Session session3 = repository.login();
> check(session3); // Throws on JR3
> }
>
> private static void check(Session session) throws RepositoryException {
> if (session.getNode("/testNode").getProperty("p1").getLong() +
> session.getNode("/testNode").getProperty("p2").getLong() < 0) {
> fail("p1 + p2 < 0");
> }
> }
>
> Michael
>
> [1]
> http://wiki.apache.org/jackrabbit/Transactional%20model%20of%20the%20Microkernel%20based%20Jackrabbit%20prototype