You are viewing a plain text version of this content. The canonical link for it is here.
Posted to slide-dev@jakarta.apache.org by "Nevermann, Dr., Peter" <Pe...@softwareag.com> on 2003/12/27 15:58:16 UTC
T-Locking and Resolving
Hi,
the week before Xmas, my colleague Michael Hartmeier and me were dealing
with making two techniques which we used in the Tamino store generally
available in Slide: t-locking and resolving.
What it is.
T-locking stands for "transient locking" and makes store implementations
more robust against the parallel access ... even if the underlying datastore
(e.g. filesystem) doesn't support the required level of isolation. By means
of t-locking, during a Slide transaction, resources can be protected by READ
or WRITE t-locks.
Resolving is required to make a store implementation *binding* enabled.
Binding refers to the ability of assigning multiple URIs to the same
resource (see http://webdav.org/bind). As in a binding enabled store
implementation the URI does not anymore uniquely identify a resource, an
alternate unique resource identifier (resource-ID) is required. Resolving
refers to the process of finding the associated resource-ID for a given URI.
What we did.
So, we provided a new parent store implementation "ParentStore" (still
looking for a more appealing name :)) which we made extend "ExtendedStore"
to take advantage of Oliver's caching facilities. ParentStore adds t-locking
and resolving, of course. As ExtendedStore is still the default parent store
implementation, ParentStore has to be enabled explicitly in the store
definition:
<store name="tx" classname="org.apache.slide.store.ParentStore">
<parameter name="useBinding">true</parameter>
<nodestore
classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
<parameter name="rootpath">tx/store/metadata</parameter>
<parameter name="workpath">tx/work/metadata</parameter>
</nodestore>
...
<contentstore
classname="org.apache.slide.store.txfile.TxFileContentStore">
<parameter name="rootpath">tx/store/content</parameter>
<parameter name="workpath">tx/work/content</parameter>
</contentstore>
</store>
Binding can be en-/disabled by means of the per-store parameter "useBinding"
(see above). Be sure that the global switch for binding in slide.properties
is ON:
org.apache.slide.binding=true
We tested with Oliver's TX store. In order to binding-enable it, the only
thing we needed to do was to make sure that the node store saves the
*binding* vectors instead of the *children* vector of ObjectNode.
We added a class "ResourceId" extending org.apache.slide.common.Uri. It
holds a "unique URI" in its state which we internally called UURI and which
uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
converts between URI and UURI. If binding is disabled, URI=UURI holds and no
conversion takes place.
Roughly speaking, for a method of the store interface (e.g.
revokePermissions), the implementation in ParentStore looks like:
public void revokePermissions(Uri uri) throws ServiceAccessException {
try {
super.revokePermissions(obtainResourceId(uri));
}
catch (ObjectNotFoundException e) {
throw new ServiceAccessException(this, e);
}
}
So, if binding is enabled, caching in the ExtendedStore takes place on the
base of UURIs.
Our test results.
We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
testsuite with/without binding, and by including/excluding the "multi-user"
testcases inside "functional" (E=ExtendedStore, P=ParentStore):
-----------------------------------------
Store E E P P P P
Binding Y Y
Multi-user Y Y Y
-----------------------------------------
Total 210 212 210 210 212 212
Failed 0 40 0 0 3 0
Elapsed 200 482 193 189 337 397
-----------------------------------------
We started the testsuite as follows:
TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
Conclusions.
We observed that the ParentStore is as stable as the ExtendedStore, but adds
stability in the case of multi-user tests. Also, we could not observe any
substantial loss of performance by using the ParentStore.
Open issues.
1) All child store implementations (MySQL, JDBC, etc.) should save the
*binding* vectors instead of the children vector of ObjectNode.
2) ParentStore could become the default parent store implementation as soon
as it proves to be stable.
3) As resolving appears to be expensive, it seems senseful to cache the
mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
caching is not yet fully enabled (see doResolve method). I am still fighting
with (a) performance drawbacks or (b) data inconsistencies when I fully
enable resolve caching.
4) The UURI is currently generated in the class ResourceId. We thought that
maybe the child store implementation should get an opportunity to determine
the UURI policy?
5) Currently, with binding enabled, a filesystem-based content store maps to
files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
Here, an enhancement would be to map to the URI and take advantage of
linking capabilities of the underlying filesystem.
I wish you all a Happy New Year!
Regards,
Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: TxFileStore binding file format
Posted by Oliver Zeigermann <oz...@c1-fse.de>.
Martin Holz wrote:
> Hello Oliver,
>
> Martin Holz <ho...@fiz-chemie.de> writes:
>
>
>>On Wednesday 07 January 2004 11:10, Oliver Zeigermann wrote:
>>
>>>Martin Holz wrote:
>>>
>>>>Oliver Zeigermann <oz...@apache.org> writes:
>>>>
>>>>>There is one other thing that just reached my pea-sized brain: If
>>>>>we change the XML representation of the file store - which is
>>>>>necessary to have binding - it will no longer be compatible to
>>>>>the old non-tx file store. What to do about it? Just say, well
>>>>>that's the way it is, or should we provide something like a
>>>>>compatibility mode?
>>>>
>>>>No need for a compatibility mode, since nobody would
>>>>switch back to the old file store, but there might be
>>>>the need for a converter. Probably this can be done
>>>>with a simple XSL stylesheet.
>>>
>>>Sounds reasonable - the XSLT would look really simple I guess.
>>>Martin, would you volunteer do this? Or shall I?
>>
>>I will do it, if you can wait until Monday.
>
>
> Did you already checkin the changes for binding or know
> what the new file format will be?
Peter checked them in. And I guess the changes are pretty obvious. The
section in the XML descriptors for children once called "childs" having
child sub-elements now look like (for action folder):
<children>
<child name="read" uuri="/actions/read" />
<child name="read-acl" uuri="/actions/read-acl" />
<child name="read-current-user-privilege-set"
uuri="/actions/read-current-user-privilege-set" />
<child name="write" uuri="/actions/write" />
<child name="write-acl" uuri="/actions/write-acl" />
<child name="write-properties" uuri="/actions/write-properties" />
<child name="write-content" uuri="/actions/write-content" />
<child name="bind" uuri="/actions/bind" />
<child name="unbind" uuri="/actions/unbind" />
<child name="unlock" uuri="/actions/unlock" />
</children>
<parents>
<parent name="actions" uuri="/" />
</parents>
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
TxFileStore binding file format
Posted by Martin Holz <ho...@fiz-chemie.de>.
Hello Oliver,
Martin Holz <ho...@fiz-chemie.de> writes:
> On Wednesday 07 January 2004 11:10, Oliver Zeigermann wrote:
> > Martin Holz wrote:
> > > Oliver Zeigermann <oz...@apache.org> writes:
> > >>There is one other thing that just reached my pea-sized brain: If
> > >> we change the XML representation of the file store - which is
> > >> necessary to have binding - it will no longer be compatible to
> > >> the old non-tx file store. What to do about it? Just say, well
> > >> that's the way it is, or should we provide something like a
> > >> compatibility mode?
> > >
> > > No need for a compatibility mode, since nobody would
> > > switch back to the old file store, but there might be
> > > the need for a converter. Probably this can be done
> > > with a simple XSL stylesheet.
> >
> > Sounds reasonable - the XSLT would look really simple I guess.
> > Martin, would you volunteer do this? Or shall I?
>
> I will do it, if you can wait until Monday.
Did you already checkin the changes for binding or know
what the new file format will be?
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Martin Holz <ho...@fiz-chemie.de>.
On Wednesday 07 January 2004 11:10, Oliver Zeigermann wrote:
> Martin Holz wrote:
> > Oliver Zeigermann <oz...@apache.org> writes:
> >>There is one other thing that just reached my pea-sized brain: If
> >> we change the XML representation of the file store - which is
> >> necessary to have binding - it will no longer be compatible to
> >> the old non-tx file store. What to do about it? Just say, well
> >> that's the way it is, or should we provide something like a
> >> compatibility mode?
> >
> > No need for a compatibility mode, since nobody would
> > switch back to the old file store, but there might be
> > the need for a converter. Probably this can be done
> > with a simple XSL stylesheet.
>
> Sounds reasonable - the XSLT would look really simple I guess.
> Martin, would you volunteer do this? Or shall I?
I will do it, if you can wait until Monday.
Martin
--
Martin Holz <ho...@fiz-chemie.de>
Softwareentwicklung / Vernetztes Studium - Chemie
FIZ CHEMIE Berlin
Franklin Str. 11
D-10587 Berlin
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Martin Holz wrote:
> Oliver Zeigermann <oz...@apache.org> writes:
>
>
>>There is one other thing that just reached my pea-sized brain: If we
>>change the XML representation of the file store - which is necessary
>>to have binding - it will no longer be compatible to the old non-tx
>>file store. What to do about it? Just say, well that's the way it is,
>>or should we provide something like a compatibility mode?
>
>
> No need for a compatibility mode, since nobody would
> switch back to the old file store, but there might be
> the need for a converter. Probably this can be done
> with a simple XSL stylesheet.
Sounds reasonable - the XSLT would look really simple I guess. Martin,
would you volunteer do this? Or shall I?
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Martin Holz <ho...@fiz-chemie.de>.
Oliver Zeigermann <oz...@apache.org> writes:
> There is one other thing that just reached my pea-sized brain: If we
> change the XML representation of the file store - which is necessary
> to have binding - it will no longer be compatible to the old non-tx
> file store. What to do about it? Just say, well that's the way it is,
> or should we provide something like a compatibility mode?
No need for a compatibility mode, since nobody would
switch back to the old file store, but there might be
the need for a converter. Probably this can be done
with a simple XSL stylesheet.
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
There is one other thing that just reached my pea-sized brain: If we
change the XML representation of the file store - which is necessary to
have binding - it will no longer be compatible to the old non-tx file
store. What to do about it? Just say, well that's the way it is, or
should we provide something like a compatibility mode?
Oliver
Nevermann, Dr., Peter wrote:
> Hi,
>
> the week before Xmas, my colleague Michael Hartmeier and me were dealing
> with making two techniques which we used in the Tamino store generally
> available in Slide: t-locking and resolving.
>
> What it is.
>
> T-locking stands for "transient locking" and makes store implementations
> more robust against the parallel access ... even if the underlying datastore
> (e.g. filesystem) doesn't support the required level of isolation. By means
> of t-locking, during a Slide transaction, resources can be protected by READ
> or WRITE t-locks.
>
> Resolving is required to make a store implementation *binding* enabled.
> Binding refers to the ability of assigning multiple URIs to the same
> resource (see http://webdav.org/bind). As in a binding enabled store
> implementation the URI does not anymore uniquely identify a resource, an
> alternate unique resource identifier (resource-ID) is required. Resolving
> refers to the process of finding the associated resource-ID for a given URI.
>
> What we did.
>
> So, we provided a new parent store implementation "ParentStore" (still
> looking for a more appealing name :)) which we made extend "ExtendedStore"
> to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> and resolving, of course. As ExtendedStore is still the default parent store
> implementation, ParentStore has to be enabled explicitly in the store
> definition:
>
> <store name="tx" classname="org.apache.slide.store.ParentStore">
> <parameter name="useBinding">true</parameter>
> <nodestore
> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> <parameter name="rootpath">tx/store/metadata</parameter>
> <parameter name="workpath">tx/work/metadata</parameter>
> </nodestore>
> ...
> <contentstore
> classname="org.apache.slide.store.txfile.TxFileContentStore">
> <parameter name="rootpath">tx/store/content</parameter>
> <parameter name="workpath">tx/work/content</parameter>
> </contentstore>
> </store>
>
> Binding can be en-/disabled by means of the per-store parameter "useBinding"
> (see above). Be sure that the global switch for binding in slide.properties
> is ON:
>
> org.apache.slide.binding=true
>
> We tested with Oliver's TX store. In order to binding-enable it, the only
> thing we needed to do was to make sure that the node store saves the
> *binding* vectors instead of the *children* vector of ObjectNode.
>
> We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> holds a "unique URI" in its state which we internally called UURI and which
> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> conversion takes place.
>
> Roughly speaking, for a method of the store interface (e.g.
> revokePermissions), the implementation in ParentStore looks like:
>
> public void revokePermissions(Uri uri) throws ServiceAccessException {
> try {
> super.revokePermissions(obtainResourceId(uri));
> }
> catch (ObjectNotFoundException e) {
> throw new ServiceAccessException(this, e);
> }
> }
>
> So, if binding is enabled, caching in the ExtendedStore takes place on the
> base of UURIs.
>
> Our test results.
>
> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> testsuite with/without binding, and by including/excluding the "multi-user"
> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>
> -----------------------------------------
> Store E E P P P P
> Binding Y Y
> Multi-user Y Y Y
> -----------------------------------------
> Total 210 212 210 210 212 212
> Failed 0 40 0 0 3 0
> Elapsed 200 482 193 189 337 397
> -----------------------------------------
>
> We started the testsuite as follows:
> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>
> Conclusions.
>
> We observed that the ParentStore is as stable as the ExtendedStore, but adds
> stability in the case of multi-user tests. Also, we could not observe any
> substantial loss of performance by using the ParentStore.
>
> Open issues.
>
> 1) All child store implementations (MySQL, JDBC, etc.) should save the
> *binding* vectors instead of the children vector of ObjectNode.
>
> 2) ParentStore could become the default parent store implementation as soon
> as it proves to be stable.
>
> 3) As resolving appears to be expensive, it seems senseful to cache the
> mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> caching is not yet fully enabled (see doResolve method). I am still fighting
> with (a) performance drawbacks or (b) data inconsistencies when I fully
> enable resolve caching.
>
> 4) The UURI is currently generated in the class ResourceId. We thought that
> maybe the child store implementation should get an opportunity to determine
> the UURI policy?
>
> 5) Currently, with binding enabled, a filesystem-based content store maps to
> files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> Here, an enhancement would be to map to the URI and take advantage of
> linking capabilities of the underlying filesystem.
>
> I wish you all a Happy New Year!
>
> Regards,
> Peter
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Oliver Zeigermann wrote:
> I am also almost done with the RDBMS port to binding and also applied
> the same trick to it, so deadlocks are handled more gracefully there as
> well. I also added some framework (only reasonably implemented for
> Sybase and MSSQLServer, yet) to handle contraint violations that might
> occur with low isolation levels like READ COMMITTED.
Done, just checked it in. Tests were running successfully.
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Inspired by the way deadlocks are handled in ParentStore, file store now
throws a nested ConflictException upon deadlock. This way deadlocks can
be resolved with response code "Conflict" passing multi user tests
copyDisjoint and copySame. It seems with this change t-locking is no
longer needed for the file store.
It is worth noting there is no functional change in the file store, the
only thing changed is the way this conflict is reported. After all file
store really provides *SERIALIZABLE* as it's isolation level. Thus there
never were any correctness issues with it.
I am also almost done with the RDBMS port to binding and also applied
the same trick to it, so deadlocks are handled more gracefully there as
well. I also added some framework (only reasonably implemented for
Sybase and MSSQLServer, yet) to handle contraint violations that might
occur with low isolation levels like READ COMMITTED.
All this raises the question if t-locks are still needed... Is it
*really* necessary for Tamino?
Oliver
Nevermann, Dr., Peter wrote:
> Hi,
>
> the week before Xmas, my colleague Michael Hartmeier and me were dealing
> with making two techniques which we used in the Tamino store generally
> available in Slide: t-locking and resolving.
>
> What it is.
>
> T-locking stands for "transient locking" and makes store implementations
> more robust against the parallel access ... even if the underlying datastore
> (e.g. filesystem) doesn't support the required level of isolation. By means
> of t-locking, during a Slide transaction, resources can be protected by READ
> or WRITE t-locks.
>
> Resolving is required to make a store implementation *binding* enabled.
> Binding refers to the ability of assigning multiple URIs to the same
> resource (see http://webdav.org/bind). As in a binding enabled store
> implementation the URI does not anymore uniquely identify a resource, an
> alternate unique resource identifier (resource-ID) is required. Resolving
> refers to the process of finding the associated resource-ID for a given URI.
>
> What we did.
>
> So, we provided a new parent store implementation "ParentStore" (still
> looking for a more appealing name :)) which we made extend "ExtendedStore"
> to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> and resolving, of course. As ExtendedStore is still the default parent store
> implementation, ParentStore has to be enabled explicitly in the store
> definition:
>
> <store name="tx" classname="org.apache.slide.store.ParentStore">
> <parameter name="useBinding">true</parameter>
> <nodestore
> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> <parameter name="rootpath">tx/store/metadata</parameter>
> <parameter name="workpath">tx/work/metadata</parameter>
> </nodestore>
> ...
> <contentstore
> classname="org.apache.slide.store.txfile.TxFileContentStore">
> <parameter name="rootpath">tx/store/content</parameter>
> <parameter name="workpath">tx/work/content</parameter>
> </contentstore>
> </store>
>
> Binding can be en-/disabled by means of the per-store parameter "useBinding"
> (see above). Be sure that the global switch for binding in slide.properties
> is ON:
>
> org.apache.slide.binding=true
>
> We tested with Oliver's TX store. In order to binding-enable it, the only
> thing we needed to do was to make sure that the node store saves the
> *binding* vectors instead of the *children* vector of ObjectNode.
>
> We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> holds a "unique URI" in its state which we internally called UURI and which
> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> conversion takes place.
>
> Roughly speaking, for a method of the store interface (e.g.
> revokePermissions), the implementation in ParentStore looks like:
>
> public void revokePermissions(Uri uri) throws ServiceAccessException {
> try {
> super.revokePermissions(obtainResourceId(uri));
> }
> catch (ObjectNotFoundException e) {
> throw new ServiceAccessException(this, e);
> }
> }
>
> So, if binding is enabled, caching in the ExtendedStore takes place on the
> base of UURIs.
>
> Our test results.
>
> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> testsuite with/without binding, and by including/excluding the "multi-user"
> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>
> -----------------------------------------
> Store E E P P P P
> Binding Y Y
> Multi-user Y Y Y
> -----------------------------------------
> Total 210 212 210 210 212 212
> Failed 0 40 0 0 3 0
> Elapsed 200 482 193 189 337 397
> -----------------------------------------
>
> We started the testsuite as follows:
> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>
> Conclusions.
>
> We observed that the ParentStore is as stable as the ExtendedStore, but adds
> stability in the case of multi-user tests. Also, we could not observe any
> substantial loss of performance by using the ParentStore.
>
> Open issues.
>
> 1) All child store implementations (MySQL, JDBC, etc.) should save the
> *binding* vectors instead of the children vector of ObjectNode.
>
> 2) ParentStore could become the default parent store implementation as soon
> as it proves to be stable.
>
> 3) As resolving appears to be expensive, it seems senseful to cache the
> mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> caching is not yet fully enabled (see doResolve method). I am still fighting
> with (a) performance drawbacks or (b) data inconsistencies when I fully
> enable resolve caching.
>
> 4) The UURI is currently generated in the class ResourceId. We thought that
> maybe the child store implementation should get an opportunity to determine
> the UURI policy?
>
> 5) Currently, with binding enabled, a filesystem-based content store maps to
> files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> Here, an enhancement would be to map to the URI and take advantage of
> linking capabilities of the underlying filesystem.
>
> I wish you all a Happy New Year!
>
> Regards,
> Peter
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Oliver Zeigermann wrote:
> Michael Hartmeier wrote:
>> What's your favorite book about all this? I need some background about
>> isolation levels ... (I picked ReadWrite locks from Concurrent
>> Programming in Java, Design principles and patterns, I like this book)
Ah, yes, isolation levels, I am no expert there either, but I guess you
can turn to just any database text book.
In short:
There are four common isolation levels which form a hierarchy:
1. READ UNCOMMITTED
2. READ COMMITTED
3. REPEATABLE READ
4. SERIALIZABLE
SERIALIZABLE is the highest level possible. If this is the isolation
level some *sequential* ordering of concurrent transaction can be found
that will leave the database in exactly the same state as the
concurrently executed transactions. Now when each transaction leaves the
database in a consistent state - which is your assumption - it will
still be consistent after those transactions have been executed.
Depending on the strategy to achive this isolation level seen from the
code executed inside a transaction it should behave just the same as if
there were no other transactions running in parallel and no other
transaction should not be influenced by this transaction (except for
some waiting/deadlocks when using blocking locks or commits failing
rarely which optimisitc strategies).
READ UNCOMMITTED is a lousy isolation. Data written inside of one
transaction can be seen inside others running in parallel even before
this first transaction commits those changes. This can lead to quite a
number of problems.
READ COMMITTED is used often, but still is rather weak. The difference
to READ UNCOMMITTED is data will be visible to other transactions
running in parallel after commit only.
REPEATABLE READ guarantees that data once read will be the same when
read for a second time - even if it has been physically changed in the
meantime by some transaction.
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Michael Hartmeier wrote:
> On Wed, 2004-01-07 at 10:08, Oliver Zeigermann wrote:
>
> I had a lock a your GenericLocks, and I'd like to see TLocks replaced by
> your implementation. As far as I understand, GenericLocks with
> maxLevel=2 can emulate TLocks, so why not dump the special-case
> implementation?
Yes, maxLevel=2 could represent NO_LOCK, READ_LOCK and WRITE_LOCK.
> You mentioned priorities ... does that mean, e.g. that threads waiting
> for read-locks should wake up before threads waiting before write-locks?
> That's not implemented in TLocks either, so that's not a reason to keep
> TLocks either.
Not quite sure... What I have seen, writers are preferred over readers
when more than one thread is waiting to acquire a lock, correct?
GenericLock does not support preferences like this.
> What's your favorite book about all this? I need some background about
> isolation levels ... (I picked ReadWrite locks from Concurrent
> Programming in Java, Design principles and patterns, I like this book)
Apart from the web I have read the Doug Lea stuff only. I do not
remember the title, but I think it is just the one you named...
Oliver
> Michael
>
>
>>Michael Hartmeier wrote:
>>
>>
>>>Aaaarghhh! Your code looks definitely better than mine ...
>>
>>As I said my code in GenericLock was wrong as well! It does not seem to
>>be that easy... I will take over the corrected for loop stated below...
>>
>>
>>>Where did you checkin your GenericLock? We should change ParentStore to
>>>use your lock (or even better: use a common lock manager).
>>
>>Well, as it seems we do not need this locking in the ParentStore any
>>more. Still it might be a good idea to use the read/write lock idea from
>>TLock. GenericLock is needed for the file store and does well without
>>priorities. But - as it seems - caching will need non-blocking locks to
>>support isolation levels higher than READ COMMITTED. read/write locks
>>would be nice for this...
>>
>>Locks are in org.apache.slide.util.locking{.impl}... When you have a
>>look at it, have in mind GenericLock is a multi level lock, i.e. there
>>is at most *one* multi level lock per resource, but the lock itself can
>>have more than one lock level and more than one owner. This is a bit
>>excentric, but I saw it fit just right...
>>
>>Oliver
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
On Wed, 2004-01-07 at 10:08, Oliver Zeigermann wrote:
I had a lock a your GenericLocks, and I'd like to see TLocks replaced by
your implementation. As far as I understand, GenericLocks with
maxLevel=2 can emulate TLocks, so why not dump the special-case
implementation?
You mentioned priorities ... does that mean, e.g. that threads waiting
for read-locks should wake up before threads waiting before write-locks?
That's not implemented in TLocks either, so that's not a reason to keep
TLocks either.
What's your favorite book about all this? I need some background about
isolation levels ... (I picked ReadWrite locks from Concurrent
Programming in Java, Design principles and patterns, I like this book)
Michael
> Michael Hartmeier wrote:
>
> > Aaaarghhh! Your code looks definitely better than mine ...
>
> As I said my code in GenericLock was wrong as well! It does not seem to
> be that easy... I will take over the corrected for loop stated below...
>
> > Where did you checkin your GenericLock? We should change ParentStore to
> > use your lock (or even better: use a common lock manager).
>
> Well, as it seems we do not need this locking in the ParentStore any
> more. Still it might be a good idea to use the read/write lock idea from
> TLock. GenericLock is needed for the file store and does well without
> priorities. But - as it seems - caching will need non-blocking locks to
> support isolation levels higher than READ COMMITTED. read/write locks
> would be nice for this...
>
> Locks are in org.apache.slide.util.locking{.impl}... When you have a
> look at it, have in mind GenericLock is a multi level lock, i.e. there
> is at most *one* multi level lock per resource, but the lock itself can
> have more than one lock level and more than one owner. This is a bit
> excentric, but I saw it fit just right...
>
> Oliver
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Michael Hartmeier wrote:
> Aaaarghhh! Your code looks definitely better than mine ...
As I said my code in GenericLock was wrong as well! It does not seem to
be that easy... I will take over the corrected for loop stated below...
> Where did you checkin your GenericLock? We should change ParentStore to
> use your lock (or even better: use a common lock manager).
Well, as it seems we do not need this locking in the ParentStore any
more. Still it might be a good idea to use the read/write lock idea from
TLock. GenericLock is needed for the file store and does well without
priorities. But - as it seems - caching will need non-blocking locks to
support isolation levels higher than READ COMMITTED. read/write locks
would be nice for this...
Locks are in org.apache.slide.util.locking{.impl}... When you have a
look at it, have in mind GenericLock is a multi level lock, i.e. there
is at most *one* multi level lock per resource, but the lock itself can
have more than one lock level and more than one owner. This is a bit
excentric, but I saw it fit just right...
Oliver
> Michael
>
> On Tue, 2004-01-06 at 16:31, Oliver Zeigermann wrote:
>
>>Arghhh! After investigating both TLock and GenericLock for comparison it
>> occurs to me *both* "acquire" methods are wrong as they use the wrong
>>timeout in wait.
>>
>>In TLock the timeout may be too *short* when the loop cicles more than
>>once while in GenericLock the timeout may be too *long* as wait always
>>waits the maximum timeout.
>>
>>To solve this the for loop in awaitReader and awaitWriter in TLock might
>>look like this:
>>
>>
>>> for (remaining = timeout; remaining > 0; remaining = timeout - (System.currentTimeMillis() - started)) {
>>
>>instead of the current loop
>>
>>
>>> for (remaining = timeout; remaining > 0; remaining -= System.currentTimeMillis() - started) {
>>
>>
>>As I obviously failed once in this task that seemed so simple to me I
>>may be wrong here again. Could someone smarter than me check this, please :)
>>
>>Oliver
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
Aaaarghhh! Your code looks definitely better than mine ...
Where did you checkin your GenericLock? We should change ParentStore to
use your lock (or even better: use a common lock manager).
Michael
On Tue, 2004-01-06 at 16:31, Oliver Zeigermann wrote:
> Arghhh! After investigating both TLock and GenericLock for comparison it
> occurs to me *both* "acquire" methods are wrong as they use the wrong
> timeout in wait.
>
> In TLock the timeout may be too *short* when the loop cicles more than
> once while in GenericLock the timeout may be too *long* as wait always
> waits the maximum timeout.
>
> To solve this the for loop in awaitReader and awaitWriter in TLock might
> look like this:
>
> > for (remaining = timeout; remaining > 0; remaining = timeout - (System.currentTimeMillis() - started)) {
>
> instead of the current loop
>
> > for (remaining = timeout; remaining > 0; remaining -= System.currentTimeMillis() - started) {
>
>
> As I obviously failed once in this task that seemed so simple to me I
> may be wrong here again. Could someone smarter than me check this, please :)
>
> Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Arghhh! After investigating both TLock and GenericLock for comparison it
occurs to me *both* "acquire" methods are wrong as they use the wrong
timeout in wait.
In TLock the timeout may be too *short* when the loop cicles more than
once while in GenericLock the timeout may be too *long* as wait always
waits the maximum timeout.
To solve this the for loop in awaitReader and awaitWriter in TLock might
look like this:
> for (remaining = timeout; remaining > 0; remaining = timeout - (System.currentTimeMillis() - started)) {
instead of the current loop
> for (remaining = timeout; remaining > 0; remaining -= System.currentTimeMillis() - started) {
As I obviously failed once in this task that seemed so simple to me I
may be wrong here again. Could someone smarter than me check this, please :)
Oliver
Nevermann, Dr., Peter wrote:
> Hi,
>
> the week before Xmas, my colleague Michael Hartmeier and me were dealing
> with making two techniques which we used in the Tamino store generally
> available in Slide: t-locking and resolving.
>
> What it is.
>
> T-locking stands for "transient locking" and makes store implementations
> more robust against the parallel access ... even if the underlying datastore
> (e.g. filesystem) doesn't support the required level of isolation. By means
> of t-locking, during a Slide transaction, resources can be protected by READ
> or WRITE t-locks.
>
> Resolving is required to make a store implementation *binding* enabled.
> Binding refers to the ability of assigning multiple URIs to the same
> resource (see http://webdav.org/bind). As in a binding enabled store
> implementation the URI does not anymore uniquely identify a resource, an
> alternate unique resource identifier (resource-ID) is required. Resolving
> refers to the process of finding the associated resource-ID for a given URI.
>
> What we did.
>
> So, we provided a new parent store implementation "ParentStore" (still
> looking for a more appealing name :)) which we made extend "ExtendedStore"
> to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> and resolving, of course. As ExtendedStore is still the default parent store
> implementation, ParentStore has to be enabled explicitly in the store
> definition:
>
> <store name="tx" classname="org.apache.slide.store.ParentStore">
> <parameter name="useBinding">true</parameter>
> <nodestore
> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> <parameter name="rootpath">tx/store/metadata</parameter>
> <parameter name="workpath">tx/work/metadata</parameter>
> </nodestore>
> ...
> <contentstore
> classname="org.apache.slide.store.txfile.TxFileContentStore">
> <parameter name="rootpath">tx/store/content</parameter>
> <parameter name="workpath">tx/work/content</parameter>
> </contentstore>
> </store>
>
> Binding can be en-/disabled by means of the per-store parameter "useBinding"
> (see above). Be sure that the global switch for binding in slide.properties
> is ON:
>
> org.apache.slide.binding=true
>
> We tested with Oliver's TX store. In order to binding-enable it, the only
> thing we needed to do was to make sure that the node store saves the
> *binding* vectors instead of the *children* vector of ObjectNode.
>
> We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> holds a "unique URI" in its state which we internally called UURI and which
> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> conversion takes place.
>
> Roughly speaking, for a method of the store interface (e.g.
> revokePermissions), the implementation in ParentStore looks like:
>
> public void revokePermissions(Uri uri) throws ServiceAccessException {
> try {
> super.revokePermissions(obtainResourceId(uri));
> }
> catch (ObjectNotFoundException e) {
> throw new ServiceAccessException(this, e);
> }
> }
>
> So, if binding is enabled, caching in the ExtendedStore takes place on the
> base of UURIs.
>
> Our test results.
>
> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> testsuite with/without binding, and by including/excluding the "multi-user"
> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>
> -----------------------------------------
> Store E E P P P P
> Binding Y Y
> Multi-user Y Y Y
> -----------------------------------------
> Total 210 212 210 210 212 212
> Failed 0 40 0 0 3 0
> Elapsed 200 482 193 189 337 397
> -----------------------------------------
>
> We started the testsuite as follows:
> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>
> Conclusions.
>
> We observed that the ParentStore is as stable as the ExtendedStore, but adds
> stability in the case of multi-user tests. Also, we could not observe any
> substantial loss of performance by using the ParentStore.
>
> Open issues.
>
> 1) All child store implementations (MySQL, JDBC, etc.) should save the
> *binding* vectors instead of the children vector of ObjectNode.
>
> 2) ParentStore could become the default parent store implementation as soon
> as it proves to be stable.
>
> 3) As resolving appears to be expensive, it seems senseful to cache the
> mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> caching is not yet fully enabled (see doResolve method). I am still fighting
> with (a) performance drawbacks or (b) data inconsistencies when I fully
> enable resolve caching.
>
> 4) The UURI is currently generated in the class ResourceId. We thought that
> maybe the child store implementation should get an opportunity to determine
> the UURI policy?
>
> 5) Currently, with binding enabled, a filesystem-based content store maps to
> files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> Here, an enhancement would be to map to the URI and take advantage of
> linking capabilities of the underlying filesystem.
>
> I wish you all a Happy New Year!
>
> Regards,
> Peter
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
The relation between ParentStore / ExtendedStore and all the others is
really confusing - I think you are right there. I guess if ParentStore
will finally take the place of ExtendedStore it should be nice to keep
the name ParentStore as then there is no need to distiguish it from
ExtendedStore any more.
Guten Rutsch :)
Oliver
Michael Hartmeier wrote:
> On Sat, 2003-12-27 at 16:41, Oliver Zeigermann wrote:
>
>
>>...
>>
>>Well, it does transient locking, why not call it TLockingStore or so?
>>ExtendedStore is such an ugly name, let's have a nice one for
>>ParentStore ;)
>>
>
>
> I kind of like the name because it helps me distinguish between
> "aggregating" stores (ParentStore, ExtendedStore) and "aspect" stores
> (NodeStore, PermissionStore, ...) that serve individual aspects only.
> I'd like to see something like a ChildStore in the class hierachie ...
>
> TLockingStore doesn't express the resolving functionality.
>
> Guten Rutsch :)
>
> Michael
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
On Sat, 2003-12-27 at 16:41, Oliver Zeigermann wrote:
> ...
>
> Well, it does transient locking, why not call it TLockingStore or so?
> ExtendedStore is such an ugly name, let's have a nice one for
> ParentStore ;)
>
I kind of like the name because it helps me distinguish between
"aggregating" stores (ParentStore, ExtendedStore) and "aspect" stores
(NodeStore, PermissionStore, ...) that serve individual aspects only.
I'd like to see something like a ChildStore in the class hierachie ...
TLockingStore doesn't express the resolving functionality.
Guten Rutsch :)
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Mike Oliver <ol...@matrix-media.com>.
Amen!
Stefano Mazzocchi wrote:
>
> On 27 Dec 2003, at 07:41, Oliver Zeigermann wrote:
>
>> Nevermann, Dr., Peter wrote:
>>
>>> Hi,
>>> the week before Xmas, my colleague Michael Hartmeier and me were
>>> dealing
>>> with making two techniques which we used in the Tamino store generally
>>> available in Slide: t-locking and resolving.
>>> What it is.
>>> T-locking stands for "transient locking" and makes store
>>> implementations
>>> more robust against the parallel access ... even if the underlying
>>> datastore
>>> (e.g. filesystem) doesn't support the required level of isolation.
>>> By means
>>> of t-locking, during a Slide transaction, resources can be protected
>>> by READ
>>> or WRITE t-locks.
>>
>>
>> *Just great* :) :) :) :) That was what I meant when I talked about
>> *global locks* all the time! Here they are :) :) :) :)
>>
>>> Resolving is required to make a store implementation *binding* enabled.
>>> Binding refers to the ability of assigning multiple URIs to the same
>>> resource (see http://webdav.org/bind). As in a binding enabled store
>>> implementation the URI does not anymore uniquely identify a
>>> resource, an
>>> alternate unique resource identifier (resource-ID) is required.
>>> Resolving
>>> refers to the process of finding the associated resource-ID for a
>>> given URI.
>>> What we did.
>>> So, we provided a new parent store implementation "ParentStore" (still
>>> looking for a more appealing name :))
>>
>>
>> Well, it does transient locking, why not call it TLockingStore or so?
>> ExtendedStore is such an ugly name, let's have a nice one for
>> ParentStore ;)
>>
>> which we made extend "ExtendedStore"
>>
>>> to take advantage of Oliver's caching facilities. ParentStore adds
>>> t-locking
>>> and resolving, of course. As ExtendedStore is still the default
>>> parent store
>>> implementation, ParentStore has to be enabled explicitly in the store
>>> definition:
>>> <store name="tx" classname="org.apache.slide.store.ParentStore">
>>> <parameter name="useBinding">true</parameter>
>>> <nodestore
>>> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
>>> <parameter name="rootpath">tx/store/metadata</parameter>
>>> <parameter name="workpath">tx/work/metadata</parameter>
>>> </nodestore>
>>> ...
>>> <contentstore
>>> classname="org.apache.slide.store.txfile.TxFileContentStore">
>>> <parameter name="rootpath">tx/store/content</parameter>
>>> <parameter name="workpath">tx/work/content</parameter>
>>> </contentstore>
>>> </store>
>>> Binding can be en-/disabled by means of the per-store parameter
>>> "useBinding"
>>> (see above). Be sure that the global switch for binding in
>>> slide.properties
>>> is ON:
>>> org.apache.slide.binding=true
>>> We tested with Oliver's TX store. In order to binding-enable it, the
>>> only
>>> thing we needed to do was to make sure that the node store saves the
>>> *binding* vectors instead of the *children* vector of ObjectNode. We
>>> added a class "ResourceId" extending org.apache.slide.common.Uri. It
>>> holds a "unique URI" in its state which we internally called UURI
>>> and which
>>> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
>>> converts between URI and UURI. If binding is disabled, URI=UURI
>>> holds and no
>>> conversion takes place.
>>> Roughly speaking, for a method of the store interface (e.g.
>>> revokePermissions), the implementation in ParentStore looks like:
>>> public void revokePermissions(Uri uri) throws
>>> ServiceAccessException {
>>> try {
>>> super.revokePermissions(obtainResourceId(uri));
>>> }
>>> catch (ObjectNotFoundException e) {
>>> throw new ServiceAccessException(this, e);
>>> }
>>> }
>>> So, if binding is enabled, caching in the ExtendedStore takes place
>>> on the
>>> base of UURIs.
>>> Our test results.
>>> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
>>> testsuite with/without binding, and by including/excluding the
>>> "multi-user"
>>> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>>> -----------------------------------------
>>> Store E E P P P P
>>> Binding Y Y Multi-user
>>> Y Y Y -----------------------------------------
>>> Total 210 212 210 210 212 212 Failed 0
>>> 40 0 0 3 0 Elapsed 200 482 193 189 337
>>> 397 -----------------------------------------
>>> We started the testsuite as follows:
>>> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
>>> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>>> Conclusions.
>>> We observed that the ParentStore is as stable as the ExtendedStore,
>>> but adds
>>> stability in the case of multi-user tests. Also, we could not
>>> observe any
>>> substantial loss of performance by using the ParentStore.
>>> Open issues.
>>> 1) All child store implementations (MySQL, JDBC, etc.) should save the
>>> *binding* vectors instead of the children vector of ObjectNode.
>>
>>
>> This should mainly be the J2EE / JDBC store. This will be on my list!
>>
>>> 2) ParentStore could become the default parent store implementation
>>> as soon
>>> as it proves to be stable.
>>
>>
>> As soon as possible. I agree!
>>
>>> 3) As resolving appears to be expensive, it seems senseful to cache the
>>> mapping URI -> UURI inside a transaction/WebDAV-request. Currently,
>>> resolve
>>> caching is not yet fully enabled (see doResolve method). I am still
>>> fighting
>>> with (a) performance drawbacks or (b) data inconsistencies when I fully
>>> enable resolve caching.
>>> 4) The UURI is currently generated in the class ResourceId. We
>>> thought that
>>> maybe the child store implementation should get an opportunity to
>>> determine
>>> the UURI policy?
>>> 5) Currently, with binding enabled, a filesystem-based content store
>>> maps to
>>> files where the filename is the resource-ID, e.g.
>>> 1072522837857.198_1.0.
>>> Here, an enhancement would be to map to the URI and take advantage of
>>> linking capabilities of the underlying filesystem.
>>> I wish you all a Happy New Year!
>>> Regards,
>>> Peter
>>
>>
>> Happy new year from me too! Let's continue this prolific work on Slide!
>
>
> You guys rock!! :-)
>
> --
> Stefano.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Stefano Mazzocchi <st...@apache.org>.
On 27 Dec 2003, at 07:41, Oliver Zeigermann wrote:
> Nevermann, Dr., Peter wrote:
>> Hi,
>> the week before Xmas, my colleague Michael Hartmeier and me were
>> dealing
>> with making two techniques which we used in the Tamino store generally
>> available in Slide: t-locking and resolving.
>> What it is.
>> T-locking stands for "transient locking" and makes store
>> implementations
>> more robust against the parallel access ... even if the underlying
>> datastore
>> (e.g. filesystem) doesn't support the required level of isolation. By
>> means
>> of t-locking, during a Slide transaction, resources can be protected
>> by READ
>> or WRITE t-locks.
>
> *Just great* :) :) :) :) That was what I meant when I talked about
> *global locks* all the time! Here they are :) :) :) :)
>
>> Resolving is required to make a store implementation *binding*
>> enabled.
>> Binding refers to the ability of assigning multiple URIs to the same
>> resource (see http://webdav.org/bind). As in a binding enabled store
>> implementation the URI does not anymore uniquely identify a resource,
>> an
>> alternate unique resource identifier (resource-ID) is required.
>> Resolving
>> refers to the process of finding the associated resource-ID for a
>> given URI.
>> What we did.
>> So, we provided a new parent store implementation "ParentStore" (still
>> looking for a more appealing name :))
>
> Well, it does transient locking, why not call it TLockingStore or so?
> ExtendedStore is such an ugly name, let's have a nice one for
> ParentStore ;)
>
> which we made extend "ExtendedStore"
>> to take advantage of Oliver's caching facilities. ParentStore adds
>> t-locking
>> and resolving, of course. As ExtendedStore is still the default
>> parent store
>> implementation, ParentStore has to be enabled explicitly in the store
>> definition:
>> <store name="tx" classname="org.apache.slide.store.ParentStore">
>> <parameter name="useBinding">true</parameter>
>> <nodestore
>> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
>> <parameter name="rootpath">tx/store/metadata</parameter>
>> <parameter name="workpath">tx/work/metadata</parameter>
>> </nodestore>
>> ...
>> <contentstore
>> classname="org.apache.slide.store.txfile.TxFileContentStore">
>> <parameter name="rootpath">tx/store/content</parameter>
>> <parameter name="workpath">tx/work/content</parameter>
>> </contentstore>
>> </store>
>> Binding can be en-/disabled by means of the per-store parameter
>> "useBinding"
>> (see above). Be sure that the global switch for binding in
>> slide.properties
>> is ON:
>> org.apache.slide.binding=true
>> We tested with Oliver's TX store. In order to binding-enable it, the
>> only
>> thing we needed to do was to make sure that the node store saves the
>> *binding* vectors instead of the *children* vector of ObjectNode. We
>> added a class "ResourceId" extending org.apache.slide.common.Uri. It
>> holds a "unique URI" in its state which we internally called UURI and
>> which
>> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
>> converts between URI and UURI. If binding is disabled, URI=UURI holds
>> and no
>> conversion takes place.
>> Roughly speaking, for a method of the store interface (e.g.
>> revokePermissions), the implementation in ParentStore looks like:
>> public void revokePermissions(Uri uri) throws
>> ServiceAccessException {
>> try {
>> super.revokePermissions(obtainResourceId(uri));
>> }
>> catch (ObjectNotFoundException e) {
>> throw new ServiceAccessException(this, e);
>> }
>> }
>> So, if binding is enabled, caching in the ExtendedStore takes place
>> on the
>> base of UURIs.
>> Our test results.
>> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
>> testsuite with/without binding, and by including/excluding the
>> "multi-user"
>> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>> -----------------------------------------
>> Store E E P P P P Binding
>> Y Y Multi-user Y Y Y
>> -----------------------------------------
>> Total 210 212 210 210 212 212 Failed 0 40
>> 0 0 3 0 Elapsed 200 482 193 189 337 397
>> -----------------------------------------
>> We started the testsuite as follows:
>> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
>> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>> Conclusions.
>> We observed that the ParentStore is as stable as the ExtendedStore,
>> but adds
>> stability in the case of multi-user tests. Also, we could not observe
>> any
>> substantial loss of performance by using the ParentStore.
>> Open issues.
>> 1) All child store implementations (MySQL, JDBC, etc.) should save the
>> *binding* vectors instead of the children vector of ObjectNode.
>
> This should mainly be the J2EE / JDBC store. This will be on my list!
>
>> 2) ParentStore could become the default parent store implementation
>> as soon
>> as it proves to be stable.
>
> As soon as possible. I agree!
>
>> 3) As resolving appears to be expensive, it seems senseful to cache
>> the
>> mapping URI -> UURI inside a transaction/WebDAV-request. Currently,
>> resolve
>> caching is not yet fully enabled (see doResolve method). I am still
>> fighting
>> with (a) performance drawbacks or (b) data inconsistencies when I
>> fully
>> enable resolve caching.
>> 4) The UURI is currently generated in the class ResourceId. We
>> thought that
>> maybe the child store implementation should get an opportunity to
>> determine
>> the UURI policy?
>> 5) Currently, with binding enabled, a filesystem-based content store
>> maps to
>> files where the filename is the resource-ID, e.g.
>> 1072522837857.198_1.0.
>> Here, an enhancement would be to map to the URI and take advantage of
>> linking capabilities of the underlying filesystem.
>> I wish you all a Happy New Year!
>> Regards,
>> Peter
>
> Happy new year from me too! Let's continue this prolific work on Slide!
You guys rock!! :-)
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
I' not familiar with sql stuff, but there a two things that might be
useful:
o the URI_ID field should store the binding name only, not the full
uri. Otherwise, you'll have trouble to update your descriptors
if a binding above is changed.
o the same applied to parent bindings. In addition, the parent binding
name is not unique, the same resource may be bound into different
folders with the same name. This might cause trouble in a relational
database ...
Michael
On Mon, 2004-01-05 at 09:50, Oliver Zeigermann wrote:
> Oliver Zeigermann wrote:
> >> Open issues.
> >>
> >> 1) All child store implementations (MySQL, JDBC, etc.) should save
> the
> >> *binding* vectors instead of the children vector of ObjectNode.
> >
> >
> > This should mainly be the J2EE / JDBC store. This will be on my
> list!
>
> I checked the implementation and it really seems easy to make this
> adaption.
>
> I propose:
>
> 1.) A general change in DB schema: replace children table with a
> binding
> and a parentBinding table like this:
>
> CREATE TABLE BINDING (
> URI_ID id_type NOT NULL
> REFERENCES URI (URI_ID),
> CHILD_UURI_ID id_type NOT NULL
> REFERENCES URI (URI_ID)
> )
>
> CREATE TABLE PARENT_BINDING (
> URI_ID id_type NOT NULL
> REFERENCES URI (URI_ID),
> PARENT_UURI_ID id_type NOT NULL
> REFERENCES URI (URI_ID)
> )
>
>
> 2.) An update of StandardRDBMSAdapter to reflect the change in schema
> and usage of ctor ObjectNode(String uuri, Vector bindings, Vector
> parentBindings, Vector links).
>
> I will do so if there are no objections,
>
> Oliver
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Michael Hartmeier wrote:
> I don't know what a ctor is, but I think you're right: We didn't change
> links when we updated the file store to support bindings.
Ooops, ctor is my abbreviation of constructor, I'm a lazy guy...
> I don't understand LinkNodes, but if its similar to links in unix file
> systems it's obsolete.
I guess it is...
Oliver
> Michael
>
>
> On Mon, 2004-01-05 at 11:54, Oliver Zeigermann wrote:
>
>>While doing this: LinkNode has no ctor for bindings. This may make
>>sense
>>as it seems obsolete now. Can anyone confirm this?
>>
>>Anyway, we should keep it for backward compatibility, right?
>>
>>Oliver
>>
>>Oliver Zeigermann wrote:
>>
>>
>>>Oliver Zeigermann wrote:
>>>
>>>
>>>>>Open issues.
>>>>>
>>>>>1) All child store implementations (MySQL, JDBC, etc.) should save
>>
>>the
>>
>>>>>*binding* vectors instead of the children vector of ObjectNode.
>>>>
>>>>
>>>>
>>>>This should mainly be the J2EE / JDBC store. This will be on my
>>
>>list!
>>
>>>
>>>I checked the implementation and it really seems easy to make this
>>>adaption.
>>>
>>>I propose:
>>>
>>>1.) A general change in DB schema: replace children table with a
>>
>>binding
>>
>>>and a parentBinding table like this:
>>>
>>>CREATE TABLE BINDING (
>>> URI_ID id_type NOT NULL
>>> REFERENCES URI (URI_ID),
>>> CHILD_UURI_ID id_type NOT NULL
>>> REFERENCES URI (URI_ID)
>>>)
>>>
>>>CREATE TABLE PARENT_BINDING (
>>> URI_ID id_type NOT NULL
>>> REFERENCES URI (URI_ID),
>>> PARENT_UURI_ID id_type NOT NULL
>>> REFERENCES URI (URI_ID)
>>>)
>>>
>>>
>>>2.) An update of StandardRDBMSAdapter to reflect the change in
>>
>>schema
>>
>>>and usage of ctor ObjectNode(String uuri, Vector bindings, Vector
>>>parentBindings, Vector links).
>>>
>>>I will do so if there are no objections,
>>>
>>>Oliver
>>>
>>>
>>>
>>>
>>
>>---------------------------------------------------------------------
>>
>>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>>
>>>.
>>>
>>
>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
I don't know what a ctor is, but I think you're right: We didn't change
links when we updated the file store to support bindings.
I don't understand LinkNodes, but if its similar to links in unix file
systems it's obsolete.
Michael
On Mon, 2004-01-05 at 11:54, Oliver Zeigermann wrote:
> While doing this: LinkNode has no ctor for bindings. This may make
> sense
> as it seems obsolete now. Can anyone confirm this?
>
> Anyway, we should keep it for backward compatibility, right?
>
> Oliver
>
> Oliver Zeigermann wrote:
>
> > Oliver Zeigermann wrote:
> >
> >>> Open issues.
> >>>
> >>> 1) All child store implementations (MySQL, JDBC, etc.) should save
> the
> >>> *binding* vectors instead of the children vector of ObjectNode.
> >>
> >>
> >>
> >> This should mainly be the J2EE / JDBC store. This will be on my
> list!
> >
> >
> > I checked the implementation and it really seems easy to make this
> > adaption.
> >
> > I propose:
> >
> > 1.) A general change in DB schema: replace children table with a
> binding
> > and a parentBinding table like this:
> >
> > CREATE TABLE BINDING (
> > URI_ID id_type NOT NULL
> > REFERENCES URI (URI_ID),
> > CHILD_UURI_ID id_type NOT NULL
> > REFERENCES URI (URI_ID)
> > )
> >
> > CREATE TABLE PARENT_BINDING (
> > URI_ID id_type NOT NULL
> > REFERENCES URI (URI_ID),
> > PARENT_UURI_ID id_type NOT NULL
> > REFERENCES URI (URI_ID)
> > )
> >
> >
> > 2.) An update of StandardRDBMSAdapter to reflect the change in
> schema
> > and usage of ctor ObjectNode(String uuri, Vector bindings, Vector
> > parentBindings, Vector links).
> >
> > I will do so if there are no objections,
> >
> > Oliver
> >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >
> >
> > .
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
While doing this: LinkNode has no ctor for bindings. This may make sense
as it seems obsolete now. Can anyone confirm this?
Anyway, we should keep it for backward compatibility, right?
Oliver
Oliver Zeigermann wrote:
> Oliver Zeigermann wrote:
>
>>> Open issues.
>>>
>>> 1) All child store implementations (MySQL, JDBC, etc.) should save the
>>> *binding* vectors instead of the children vector of ObjectNode.
>>
>>
>>
>> This should mainly be the J2EE / JDBC store. This will be on my list!
>
>
> I checked the implementation and it really seems easy to make this
> adaption.
>
> I propose:
>
> 1.) A general change in DB schema: replace children table with a binding
> and a parentBinding table like this:
>
> CREATE TABLE BINDING (
> URI_ID id_type NOT NULL
> REFERENCES URI (URI_ID),
> CHILD_UURI_ID id_type NOT NULL
> REFERENCES URI (URI_ID)
> )
>
> CREATE TABLE PARENT_BINDING (
> URI_ID id_type NOT NULL
> REFERENCES URI (URI_ID),
> PARENT_UURI_ID id_type NOT NULL
> REFERENCES URI (URI_ID)
> )
>
>
> 2.) An update of StandardRDBMSAdapter to reflect the change in schema
> and usage of ctor ObjectNode(String uuri, Vector bindings, Vector
> parentBindings, Vector links).
>
> I will do so if there are no objections,
>
> Oliver
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Oliver Zeigermann wrote:
> Oliver Zeigermann wrote:
>
>>> Open issues.
>>>
>>> 1) All child store implementations (MySQL, JDBC, etc.) should save the
>>> *binding* vectors instead of the children vector of ObjectNode.
>>
>>
>>
>> This should mainly be the J2EE / JDBC store. This will be on my list!
>
>
> I checked the implementation and it really seems easy to make this
> adaption.
>
> I propose:
>
> 1.) A general change in DB schema: replace children table with a binding
> and a parentBinding table like this:
>
> CREATE TABLE BINDING (
> URI_ID id_type NOT NULL
> REFERENCES URI (URI_ID),
> CHILD_UURI_ID id_type NOT NULL
> REFERENCES URI (URI_ID)
> )
>
> CREATE TABLE PARENT_BINDING (
> URI_ID id_type NOT NULL
> REFERENCES URI (URI_ID),
> PARENT_UURI_ID id_type NOT NULL
> REFERENCES URI (URI_ID)
> )
...which lacks the name, so what about this:
CREATE TABLE BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
NAME uri_str_type NOT NULL,
CHILD_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID)
)
CREATE TABLE PARENT_BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
NAME uri_str_type NOT NULL,
PARENT_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID)
)
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Oliver Zeigermann wrote:
>> Open issues.
>>
>> 1) All child store implementations (MySQL, JDBC, etc.) should save the
>> *binding* vectors instead of the children vector of ObjectNode.
>
>
> This should mainly be the J2EE / JDBC store. This will be on my list!
I checked the implementation and it really seems easy to make this
adaption.
I propose:
1.) A general change in DB schema: replace children table with a binding
and a parentBinding table like this:
CREATE TABLE BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
CHILD_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID)
)
CREATE TABLE PARENT_BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
PARENT_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID)
)
2.) An update of StandardRDBMSAdapter to reflect the change in schema
and usage of ctor ObjectNode(String uuri, Vector bindings, Vector
parentBindings, Vector links).
I will do so if there are no objections,
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Nevermann, Dr., Peter wrote:
> Hi,
>
> the week before Xmas, my colleague Michael Hartmeier and me were dealing
> with making two techniques which we used in the Tamino store generally
> available in Slide: t-locking and resolving.
>
> What it is.
>
> T-locking stands for "transient locking" and makes store implementations
> more robust against the parallel access ... even if the underlying datastore
> (e.g. filesystem) doesn't support the required level of isolation. By means
> of t-locking, during a Slide transaction, resources can be protected by READ
> or WRITE t-locks.
*Just great* :) :) :) :) That was what I meant when I talked about
*global locks* all the time! Here they are :) :) :) :)
> Resolving is required to make a store implementation *binding* enabled.
> Binding refers to the ability of assigning multiple URIs to the same
> resource (see http://webdav.org/bind). As in a binding enabled store
> implementation the URI does not anymore uniquely identify a resource, an
> alternate unique resource identifier (resource-ID) is required. Resolving
> refers to the process of finding the associated resource-ID for a given URI.
>
> What we did.
>
> So, we provided a new parent store implementation "ParentStore" (still
> looking for a more appealing name :))
Well, it does transient locking, why not call it TLockingStore or so?
ExtendedStore is such an ugly name, let's have a nice one for ParentStore ;)
which we made extend "ExtendedStore"
> to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> and resolving, of course. As ExtendedStore is still the default parent store
> implementation, ParentStore has to be enabled explicitly in the store
> definition:
>
> <store name="tx" classname="org.apache.slide.store.ParentStore">
> <parameter name="useBinding">true</parameter>
> <nodestore
> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> <parameter name="rootpath">tx/store/metadata</parameter>
> <parameter name="workpath">tx/work/metadata</parameter>
> </nodestore>
> ...
> <contentstore
> classname="org.apache.slide.store.txfile.TxFileContentStore">
> <parameter name="rootpath">tx/store/content</parameter>
> <parameter name="workpath">tx/work/content</parameter>
> </contentstore>
> </store>
>
> Binding can be en-/disabled by means of the per-store parameter "useBinding"
> (see above). Be sure that the global switch for binding in slide.properties
> is ON:
>
> org.apache.slide.binding=true
>
> We tested with Oliver's TX store. In order to binding-enable it, the only
> thing we needed to do was to make sure that the node store saves the
> *binding* vectors instead of the *children* vector of ObjectNode.
>
> We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> holds a "unique URI" in its state which we internally called UURI and which
> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> conversion takes place.
>
> Roughly speaking, for a method of the store interface (e.g.
> revokePermissions), the implementation in ParentStore looks like:
>
> public void revokePermissions(Uri uri) throws ServiceAccessException {
> try {
> super.revokePermissions(obtainResourceId(uri));
> }
> catch (ObjectNotFoundException e) {
> throw new ServiceAccessException(this, e);
> }
> }
>
> So, if binding is enabled, caching in the ExtendedStore takes place on the
> base of UURIs.
>
> Our test results.
>
> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> testsuite with/without binding, and by including/excluding the "multi-user"
> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>
> -----------------------------------------
> Store E E P P P P
> Binding Y Y
> Multi-user Y Y Y
> -----------------------------------------
> Total 210 212 210 210 212 212
> Failed 0 40 0 0 3 0
> Elapsed 200 482 193 189 337 397
> -----------------------------------------
>
> We started the testsuite as follows:
> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>
> Conclusions.
>
> We observed that the ParentStore is as stable as the ExtendedStore, but adds
> stability in the case of multi-user tests. Also, we could not observe any
> substantial loss of performance by using the ParentStore.
>
> Open issues.
>
> 1) All child store implementations (MySQL, JDBC, etc.) should save the
> *binding* vectors instead of the children vector of ObjectNode.
This should mainly be the J2EE / JDBC store. This will be on my list!
> 2) ParentStore could become the default parent store implementation as soon
> as it proves to be stable.
As soon as possible. I agree!
> 3) As resolving appears to be expensive, it seems senseful to cache the
> mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> caching is not yet fully enabled (see doResolve method). I am still fighting
> with (a) performance drawbacks or (b) data inconsistencies when I fully
> enable resolve caching.
>
> 4) The UURI is currently generated in the class ResourceId. We thought that
> maybe the child store implementation should get an opportunity to determine
> the UURI policy?
>
> 5) Currently, with binding enabled, a filesystem-based content store maps to
> files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> Here, an enhancement would be to map to the URI and take advantage of
> linking capabilities of the underlying filesystem.
>
> I wish you all a Happy New Year!
>
> Regards,
> Peter
Happy new year from me too! Let's continue this prolific work on Slide!
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
I don't understand why doResolve skips locking if the resourceId is
found in the cache!
In my optionen, we have to lock in either case because we cannot be sure
that to resolve call that filled the cache was performed with the same
lock type we got in the current doResolve.
For example: the first doResolve might request lock type "READ_LOCK",
but the second call might request lock type WRITE_LOCK. In this case,
the first call fills the cache and obtains read locks, but the second
call does *not* obtain write locks!?
I'm not really happy with all this caching ... I'd love to see
resourceIds used in the slide layer - we could beform a single resolve
there and always call stores with resourceIds later. This would make the
cache obsolete. But that's a major change ...
Michael
On Tue, 2004-01-06 at 13:29, Oliver Zeigermann wrote:
> I may have missed something, but in ParentStore I found no evidence
> anything is locked when objects are stored or removed. Maybe doTLock in
> was intended for this, but is never used...
>
> Could someone explain thit to me, please?
>
> Thanks,
>
> Oliver
>
>
> Nevermann, Dr., Peter wrote:
>
> > Hi,
> >
> > the week before Xmas, my colleague Michael Hartmeier and me were dealing
> > with making two techniques which we used in the Tamino store generally
> > available in Slide: t-locking and resolving.
> >
> > What it is.
> >
> > T-locking stands for "transient locking" and makes store implementations
> > more robust against the parallel access ... even if the underlying datastore
> > (e.g. filesystem) doesn't support the required level of isolation. By means
> > of t-locking, during a Slide transaction, resources can be protected by READ
> > or WRITE t-locks.
> >
> > Resolving is required to make a store implementation *binding* enabled.
> > Binding refers to the ability of assigning multiple URIs to the same
> > resource (see http://webdav.org/bind). As in a binding enabled store
> > implementation the URI does not anymore uniquely identify a resource, an
> > alternate unique resource identifier (resource-ID) is required. Resolving
> > refers to the process of finding the associated resource-ID for a given URI.
> >
> > What we did.
> >
> > So, we provided a new parent store implementation "ParentStore" (still
> > looking for a more appealing name :)) which we made extend "ExtendedStore"
> > to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> > and resolving, of course. As ExtendedStore is still the default parent store
> > implementation, ParentStore has to be enabled explicitly in the store
> > definition:
> >
> > <store name="tx" classname="org.apache.slide.store.ParentStore">
> > <parameter name="useBinding">true</parameter>
> > <nodestore
> > classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> > <parameter name="rootpath">tx/store/metadata</parameter>
> > <parameter name="workpath">tx/work/metadata</parameter>
> > </nodestore>
> > ...
> > <contentstore
> > classname="org.apache.slide.store.txfile.TxFileContentStore">
> > <parameter name="rootpath">tx/store/content</parameter>
> > <parameter name="workpath">tx/work/content</parameter>
> > </contentstore>
> > </store>
> >
> > Binding can be en-/disabled by means of the per-store parameter "useBinding"
> > (see above). Be sure that the global switch for binding in slide.properties
> > is ON:
> >
> > org.apache.slide.binding=true
> >
> > We tested with Oliver's TX store. In order to binding-enable it, the only
> > thing we needed to do was to make sure that the node store saves the
> > *binding* vectors instead of the *children* vector of ObjectNode.
> >
> > We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> > holds a "unique URI" in its state which we internally called UURI and which
> > uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> > converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> > conversion takes place.
> >
> > Roughly speaking, for a method of the store interface (e.g.
> > revokePermissions), the implementation in ParentStore looks like:
> >
> > public void revokePermissions(Uri uri) throws ServiceAccessException {
> > try {
> > super.revokePermissions(obtainResourceId(uri));
> > }
> > catch (ObjectNotFoundException e) {
> > throw new ServiceAccessException(this, e);
> > }
> > }
> >
> > So, if binding is enabled, caching in the ExtendedStore takes place on the
> > base of UURIs.
> >
> > Our test results.
> >
> > We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> > testsuite with/without binding, and by including/excluding the "multi-user"
> > testcases inside "functional" (E=ExtendedStore, P=ParentStore):
> >
> > -----------------------------------------
> > Store E E P P P P
> > Binding Y Y
> > Multi-user Y Y Y
> > -----------------------------------------
> > Total 210 212 210 210 212 212
> > Failed 0 40 0 0 3 0
> > Elapsed 200 482 193 189 337 397
> > -----------------------------------------
> >
> > We started the testsuite as follows:
> > TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> > TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
> >
> > Conclusions.
> >
> > We observed that the ParentStore is as stable as the ExtendedStore, but adds
> > stability in the case of multi-user tests. Also, we could not observe any
> > substantial loss of performance by using the ParentStore.
> >
> > Open issues.
> >
> > 1) All child store implementations (MySQL, JDBC, etc.) should save the
> > *binding* vectors instead of the children vector of ObjectNode.
> >
> > 2) ParentStore could become the default parent store implementation as soon
> > as it proves to be stable.
> >
> > 3) As resolving appears to be expensive, it seems senseful to cache the
> > mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> > caching is not yet fully enabled (see doResolve method). I am still fighting
> > with (a) performance drawbacks or (b) data inconsistencies when I fully
> > enable resolve caching.
> >
> > 4) The UURI is currently generated in the class ResourceId. We thought that
> > maybe the child store implementation should get an opportunity to determine
> > the UURI policy?
> >
> > 5) Currently, with binding enabled, a filesystem-based content store maps to
> > files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> > Here, an enhancement would be to map to the URI and take advantage of
> > linking capabilities of the underlying filesystem.
> >
> > I wish you all a Happy New Year!
> >
> > Regards,
> > Peter
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >
> >
> > .
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
Very impressive -- you're right!
We just compared ParentStore with our proprietary implementation:
ParentStore has to lock upon store and remove! Somehow, the appropriate
call got lost.
It probably doesn't cause problems in the testsuite because 1) there are
retrieveObject calls with proper locking before storeObject calls, and
2) the parent folder will always be locked before removeObject is
called.
Peter just tries a patch on his machine, it shouldn't be too difficult.
We're still discussing a possible problem with the resolve cache, but
that's independent from the above bug.
Michael
On Tue, 2004-01-06 at 13:44, Oliver Zeigermann wrote:
> Michael Hartmeier wrote:
>
> > It's in doResolve. This method locks all parent folders and the resource
> > itself. The actual locking is done by the
> > tlockManager.lock(currentResourceId, currentUriStr, lockType);
> > calls.
>
> Well, yes, but I was missing this in storeObject and removeObject, only
> found locking in createObject and retrieveObject. Has it been it
> forgotten? Or is it I simply do not understand (most likely...)? I
> thought storeObject could look like this
>
> public void storeObject(Uri uri, ObjectNode object)
> throws ServiceAccessException, ObjectNotFoundException {
>
> ResourceId resourceId = obtainResourceId(uri, object);
> ObjectNode objectClone = object.cloneObject();
> objectClone.setUri(resourceId.getUuri()); // switch to uuri
> try {
> tlockManager.lock(resourceId, uri.toString(),
> TLock.WRITE_LOCK);
> } catch (TLockedException e) {
> throw new ServiceAccessException(this, new
> ConflictException(uri.toString()));
> }
> super.storeObject(resourceId, objectClone);
> }
>
>
> and removeObject like that
>
> public void removeObject(Uri uri, ObjectNode object)
> throws ServiceAccessException, ObjectNotFoundException {
>
> ResourceId resourceId = obtainResourceId(uri, object);
> ObjectNode objectClone = object.cloneObject();
> objectClone.setUri(resourceId.getUuri()); // switch to uuri
> try {
> tlockManager.lock(resourceId, uri.toString(),
> TLock.WRITE_LOCK);
> } catch (TLockedException e) {
> throw new ServiceAccessException(this, new
> ConflictException(uri.toString()));
> }
> super.removeObject(resourceId, objectClone);
> }
>
>
> Could you explain?
>
> Thanks :)
>
> Oliver
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
Michael Hartmeier wrote:
> It's in doResolve. This method locks all parent folders and the resource
> itself. The actual locking is done by the
> tlockManager.lock(currentResourceId, currentUriStr, lockType);
> calls.
Well, yes, but I was missing this in storeObject and removeObject, only
found locking in createObject and retrieveObject. Has it been it
forgotten? Or is it I simply do not understand (most likely...)? I
thought storeObject could look like this
public void storeObject(Uri uri, ObjectNode object)
throws ServiceAccessException, ObjectNotFoundException {
ResourceId resourceId = obtainResourceId(uri, object);
ObjectNode objectClone = object.cloneObject();
objectClone.setUri(resourceId.getUuri()); // switch to uuri
try {
tlockManager.lock(resourceId, uri.toString(),
TLock.WRITE_LOCK);
} catch (TLockedException e) {
throw new ServiceAccessException(this, new
ConflictException(uri.toString()));
}
super.storeObject(resourceId, objectClone);
}
and removeObject like that
public void removeObject(Uri uri, ObjectNode object)
throws ServiceAccessException, ObjectNotFoundException {
ResourceId resourceId = obtainResourceId(uri, object);
ObjectNode objectClone = object.cloneObject();
objectClone.setUri(resourceId.getUuri()); // switch to uuri
try {
tlockManager.lock(resourceId, uri.toString(),
TLock.WRITE_LOCK);
} catch (TLockedException e) {
throw new ServiceAccessException(this, new
ConflictException(uri.toString()));
}
super.removeObject(resourceId, objectClone);
}
Could you explain?
Thanks :)
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Michael Hartmeier <Mi...@softwareag.com>.
It's in doResolve. This method locks all parent folders and the resource
itself. The actual locking is done by the
tlockManager.lock(currentResourceId, currentUriStr, lockType);
calls.
You're right - doTLock is never called. I suspect Peter has moved the
code into doResolve and forgot to delete the old method ...
Michael
On Tue, 2004-01-06 at 13:29, Oliver Zeigermann wrote:
> I may have missed something, but in ParentStore I found no evidence
> anything is locked when objects are stored or removed. Maybe doTLock in
> was intended for this, but is never used...
>
> Could someone explain thit to me, please?
>
> Thanks,
>
> Oliver
>
>
> Nevermann, Dr., Peter wrote:
>
> > Hi,
> >
> > the week before Xmas, my colleague Michael Hartmeier and me were dealing
> > with making two techniques which we used in the Tamino store generally
> > available in Slide: t-locking and resolving.
> >
> > What it is.
> >
> > T-locking stands for "transient locking" and makes store implementations
> > more robust against the parallel access ... even if the underlying datastore
> > (e.g. filesystem) doesn't support the required level of isolation. By means
> > of t-locking, during a Slide transaction, resources can be protected by READ
> > or WRITE t-locks.
> >
> > Resolving is required to make a store implementation *binding* enabled.
> > Binding refers to the ability of assigning multiple URIs to the same
> > resource (see http://webdav.org/bind). As in a binding enabled store
> > implementation the URI does not anymore uniquely identify a resource, an
> > alternate unique resource identifier (resource-ID) is required. Resolving
> > refers to the process of finding the associated resource-ID for a given URI.
> >
> > What we did.
> >
> > So, we provided a new parent store implementation "ParentStore" (still
> > looking for a more appealing name :)) which we made extend "ExtendedStore"
> > to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> > and resolving, of course. As ExtendedStore is still the default parent store
> > implementation, ParentStore has to be enabled explicitly in the store
> > definition:
> >
> > <store name="tx" classname="org.apache.slide.store.ParentStore">
> > <parameter name="useBinding">true</parameter>
> > <nodestore
> > classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> > <parameter name="rootpath">tx/store/metadata</parameter>
> > <parameter name="workpath">tx/work/metadata</parameter>
> > </nodestore>
> > ...
> > <contentstore
> > classname="org.apache.slide.store.txfile.TxFileContentStore">
> > <parameter name="rootpath">tx/store/content</parameter>
> > <parameter name="workpath">tx/work/content</parameter>
> > </contentstore>
> > </store>
> >
> > Binding can be en-/disabled by means of the per-store parameter "useBinding"
> > (see above). Be sure that the global switch for binding in slide.properties
> > is ON:
> >
> > org.apache.slide.binding=true
> >
> > We tested with Oliver's TX store. In order to binding-enable it, the only
> > thing we needed to do was to make sure that the node store saves the
> > *binding* vectors instead of the *children* vector of ObjectNode.
> >
> > We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> > holds a "unique URI" in its state which we internally called UURI and which
> > uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> > converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> > conversion takes place.
> >
> > Roughly speaking, for a method of the store interface (e.g.
> > revokePermissions), the implementation in ParentStore looks like:
> >
> > public void revokePermissions(Uri uri) throws ServiceAccessException {
> > try {
> > super.revokePermissions(obtainResourceId(uri));
> > }
> > catch (ObjectNotFoundException e) {
> > throw new ServiceAccessException(this, e);
> > }
> > }
> >
> > So, if binding is enabled, caching in the ExtendedStore takes place on the
> > base of UURIs.
> >
> > Our test results.
> >
> > We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> > testsuite with/without binding, and by including/excluding the "multi-user"
> > testcases inside "functional" (E=ExtendedStore, P=ParentStore):
> >
> > -----------------------------------------
> > Store E E P P P P
> > Binding Y Y
> > Multi-user Y Y Y
> > -----------------------------------------
> > Total 210 212 210 210 212 212
> > Failed 0 40 0 0 3 0
> > Elapsed 200 482 193 189 337 397
> > -----------------------------------------
> >
> > We started the testsuite as follows:
> > TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> > TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
> >
> > Conclusions.
> >
> > We observed that the ParentStore is as stable as the ExtendedStore, but adds
> > stability in the case of multi-user tests. Also, we could not observe any
> > substantial loss of performance by using the ParentStore.
> >
> > Open issues.
> >
> > 1) All child store implementations (MySQL, JDBC, etc.) should save the
> > *binding* vectors instead of the children vector of ObjectNode.
> >
> > 2) ParentStore could become the default parent store implementation as soon
> > as it proves to be stable.
> >
> > 3) As resolving appears to be expensive, it seems senseful to cache the
> > mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> > caching is not yet fully enabled (see doResolve method). I am still fighting
> > with (a) performance drawbacks or (b) data inconsistencies when I fully
> > enable resolve caching.
> >
> > 4) The UURI is currently generated in the class ResourceId. We thought that
> > maybe the child store implementation should get an opportunity to determine
> > the UURI policy?
> >
> > 5) Currently, with binding enabled, a filesystem-based content store maps to
> > files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> > Here, an enhancement would be to map to the URI and take advantage of
> > linking capabilities of the underlying filesystem.
> >
> > I wish you all a Happy New Year!
> >
> > Regards,
> > Peter
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >
> >
> > .
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: T-Locking and Resolving
Posted by Oliver Zeigermann <oz...@apache.org>.
I may have missed something, but in ParentStore I found no evidence
anything is locked when objects are stored or removed. Maybe doTLock in
was intended for this, but is never used...
Could someone explain thit to me, please?
Thanks,
Oliver
Nevermann, Dr., Peter wrote:
> Hi,
>
> the week before Xmas, my colleague Michael Hartmeier and me were dealing
> with making two techniques which we used in the Tamino store generally
> available in Slide: t-locking and resolving.
>
> What it is.
>
> T-locking stands for "transient locking" and makes store implementations
> more robust against the parallel access ... even if the underlying datastore
> (e.g. filesystem) doesn't support the required level of isolation. By means
> of t-locking, during a Slide transaction, resources can be protected by READ
> or WRITE t-locks.
>
> Resolving is required to make a store implementation *binding* enabled.
> Binding refers to the ability of assigning multiple URIs to the same
> resource (see http://webdav.org/bind). As in a binding enabled store
> implementation the URI does not anymore uniquely identify a resource, an
> alternate unique resource identifier (resource-ID) is required. Resolving
> refers to the process of finding the associated resource-ID for a given URI.
>
> What we did.
>
> So, we provided a new parent store implementation "ParentStore" (still
> looking for a more appealing name :)) which we made extend "ExtendedStore"
> to take advantage of Oliver's caching facilities. ParentStore adds t-locking
> and resolving, of course. As ExtendedStore is still the default parent store
> implementation, ParentStore has to be enabled explicitly in the store
> definition:
>
> <store name="tx" classname="org.apache.slide.store.ParentStore">
> <parameter name="useBinding">true</parameter>
> <nodestore
> classname="org.apache.slide.store.txfile.TxXMLFileDescriptorsStore">
> <parameter name="rootpath">tx/store/metadata</parameter>
> <parameter name="workpath">tx/work/metadata</parameter>
> </nodestore>
> ...
> <contentstore
> classname="org.apache.slide.store.txfile.TxFileContentStore">
> <parameter name="rootpath">tx/store/content</parameter>
> <parameter name="workpath">tx/work/content</parameter>
> </contentstore>
> </store>
>
> Binding can be en-/disabled by means of the per-store parameter "useBinding"
> (see above). Be sure that the global switch for binding in slide.properties
> is ON:
>
> org.apache.slide.binding=true
>
> We tested with Oliver's TX store. In order to binding-enable it, the only
> thing we needed to do was to make sure that the node store saves the
> *binding* vectors instead of the *children* vector of ObjectNode.
>
> We added a class "ResourceId" extending org.apache.slide.common.Uri. It
> holds a "unique URI" in its state which we internally called UURI and which
> uses the registered URI-scheme urn:uuid:. So, the parentStore mainly
> converts between URI and UURI. If binding is disabled, URI=UURI holds and no
> conversion takes place.
>
> Roughly speaking, for a method of the store interface (e.g.
> revokePermissions), the implementation in ParentStore looks like:
>
> public void revokePermissions(Uri uri) throws ServiceAccessException {
> try {
> super.revokePermissions(obtainResourceId(uri));
> }
> catch (ObjectNotFoundException e) {
> throw new ServiceAccessException(this, e);
> }
> }
>
> So, if binding is enabled, caching in the ExtendedStore takes place on the
> base of UURIs.
>
> Our test results.
>
> We used Tomcat 4.1.27 and Oliver's TX store. We run the "functional"
> testsuite with/without binding, and by including/excluding the "multi-user"
> testcases inside "functional" (E=ExtendedStore, P=ParentStore):
>
> -----------------------------------------
> Store E E P P P P
> Binding Y Y
> Multi-user Y Y Y
> -----------------------------------------
> Total 210 212 210 210 212 212
> Failed 0 40 0 0 3 0
> Elapsed 200 482 193 189 337 397
> -----------------------------------------
>
> We started the testsuite as follows:
> TProcessor.CMD -pattern *cases\\functional* -exclude *multi-user*
> TProcessor.CMD -pattern *cases\\functional* -users 4 -iterations 4
>
> Conclusions.
>
> We observed that the ParentStore is as stable as the ExtendedStore, but adds
> stability in the case of multi-user tests. Also, we could not observe any
> substantial loss of performance by using the ParentStore.
>
> Open issues.
>
> 1) All child store implementations (MySQL, JDBC, etc.) should save the
> *binding* vectors instead of the children vector of ObjectNode.
>
> 2) ParentStore could become the default parent store implementation as soon
> as it proves to be stable.
>
> 3) As resolving appears to be expensive, it seems senseful to cache the
> mapping URI -> UURI inside a transaction/WebDAV-request. Currently, resolve
> caching is not yet fully enabled (see doResolve method). I am still fighting
> with (a) performance drawbacks or (b) data inconsistencies when I fully
> enable resolve caching.
>
> 4) The UURI is currently generated in the class ResourceId. We thought that
> maybe the child store implementation should get an opportunity to determine
> the UURI policy?
>
> 5) Currently, with binding enabled, a filesystem-based content store maps to
> files where the filename is the resource-ID, e.g. 1072522837857.198_1.0.
> Here, an enhancement would be to map to the URI and take advantage of
> linking capabilities of the underlying filesystem.
>
> I wish you all a Happy New Year!
>
> Regards,
> Peter
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org