You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Marcel Reutegger <ma...@gmx.net> on 2007/06/02 11:25:30 UTC
Re: concurrent writes (JCR-314)
Rafał Kwiecień wrote:
> The patch ( FineGrainedISMLocking.patch) allows to read (from cache) during
> writing but concurrent writes are still not possible. I need to know when
> concurrent writes in the jackrabbit will be possible. In a month, year ?
I'd say most probably not within a month.
What persistence manager do you use? with the patch you mentioned it should be
possible to read not just from the cache but also from the persistence manager
during a non-conflicting write.
regards
marcel
Re: concurrent writes (JCR-314)
Posted by Miguel Ángel Jiménez <mi...@gmail.com>.
Well, perhaps you can apply the same pattern with cache. If cache integrity
is needed and hence locking, you could delegate the actual lock management
on the cache itself. The point is to convert all the objects that require
exclusive access into thread-safe objects (reviewing its public methods and
protecting only those needed) instead of manage the lock outside them. This
strategy minimizes the time a lock is held so potentially you can achieve
more performance.
I've not reviewed the code deeply, so perhaps I'm missing something. Let me
know anyway, I'm very interested on improving the jackrabbit performance in
multi-threaded environments, so I'm completely at your disposal.
Best regards,
On 06/06/07, Marcel Reutegger <ma...@gmx.net> wrote:
>
> Hi Miguel,
>
> Miguel Ángel Jiménez wrote:
> > I would also like to see jackrabbit doing concurrent writes at its core,
> > delegating the lock management to the PersistenceManager implementation.
>
> hmm, that's an interesting idea, but I'm not sure if it will work. ISM
> locking
> in jackrabbit has specific requirements because it also uses locking to
> ensure
> cache integrity. the requirements are documented in the interface
> description of
> ISMLocking.
>
> regards
> marcel
>
--
Miguel.
Re: concurrent writes (JCR-314)
Posted by Marcel Reutegger <ma...@gmx.net>.
Hi Miguel,
Miguel Ángel Jiménez wrote:
> I would also like to see jackrabbit doing concurrent writes at its core,
> delegating the lock management to the PersistenceManager implementation.
hmm, that's an interesting idea, but I'm not sure if it will work. ISM locking
in jackrabbit has specific requirements because it also uses locking to ensure
cache integrity. the requirements are documented in the interface description of
ISMLocking.
regards
marcel
Re: concurrent writes (JCR-314)
Posted by Alexandru Popescu ☀ <th...@gmail.com>.
On 6/5/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> 2007/6/5, Alexandru Popescu ☀ <th...@gmail.com>:
> > On 6/5/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> > > 2007/6/4, Alexandru Popescu ☀ <th...@gmail.com>:
> > > > On 6/4/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> > > > > I am not deeply into the problem, but why is there locking on the Java
> > > > > side anyway? It really should be possible to get along with DB locks
> > > > > only.
> > > > >
> > > > > Calling the DB from synchronized blocks always bears the danger of deadlocks.
> > > > >
> > > >
> > > > Basically because the implementation of JCR is not required to work on
> > > > top of a RDBMS. Moreover, the persistence managers have been usually
> > > > created to be used over different persistence solutions (and over
> > > > different RDBMS, which have different locking support/mechanisms).
> > >
> > > If that is so, concurrency code really should be inside the individual
> > > persistence solutions and not inside the Jackrabbit core. If it is,
> > > there always is a deadlock hazard.
> > >
> >
> > It is. Maybe before going further you should check the code. If you
> > have some suggestions I guess everybody would be happy to hear about
> > different approaches.
>
> Oh. Sorry for being ignorant :(
>
No worries. But I am not a Jackrabbit developer, so sometimes I may be
mistaking.
> It would be great if you could help me getting smarter. Two questions
>
> (1) Why is there Java side locking in the BundleDbPersistenceManager anyway?
I don't know the details of BundleDbPersistenceManager. However, this
question (or the more generic one about PMs) has been discussed
extensively in the past on the ml, so a search will reveal lots of
answers (I think Steffan is the guy)
> (2) If all locking is inside the individual persistence managers, what
> is the purpose of SharedItemStateManager locking?
>
Think about SharedItemStateManger as a 2nd level cache. A cache
without correct synchronization would not be a real cache. Basically,
it is meant to fulfill the spec requirements about item visibility
between different sessions.
bests,
./alex
--
.w( the_mindstorm )p.
> Thanks in advance
>
> Oliver
>
Re: concurrent writes (JCR-314)
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
2007/6/5, Alexandru Popescu ☀ <th...@gmail.com>:
> On 6/5/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> > 2007/6/4, Alexandru Popescu ☀ <th...@gmail.com>:
> > > On 6/4/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> > > > I am not deeply into the problem, but why is there locking on the Java
> > > > side anyway? It really should be possible to get along with DB locks
> > > > only.
> > > >
> > > > Calling the DB from synchronized blocks always bears the danger of deadlocks.
> > > >
> > >
> > > Basically because the implementation of JCR is not required to work on
> > > top of a RDBMS. Moreover, the persistence managers have been usually
> > > created to be used over different persistence solutions (and over
> > > different RDBMS, which have different locking support/mechanisms).
> >
> > If that is so, concurrency code really should be inside the individual
> > persistence solutions and not inside the Jackrabbit core. If it is,
> > there always is a deadlock hazard.
> >
>
> It is. Maybe before going further you should check the code. If you
> have some suggestions I guess everybody would be happy to hear about
> different approaches.
Oh. Sorry for being ignorant :(
It would be great if you could help me getting smarter. Two questions
(1) Why is there Java side locking in the BundleDbPersistenceManager anyway?
(2) If all locking is inside the individual persistence managers, what
is the purpose of SharedItemStateManager locking?
Thanks in advance
Oliver
Re: concurrent writes (JCR-314)
Posted by Alexandru Popescu ☀ <th...@gmail.com>.
On 6/5/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> 2007/6/4, Alexandru Popescu ☀ <th...@gmail.com>:
> > On 6/4/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> > > I am not deeply into the problem, but why is there locking on the Java
> > > side anyway? It really should be possible to get along with DB locks
> > > only.
> > >
> > > Calling the DB from synchronized blocks always bears the danger of deadlocks.
> > >
> >
> > Basically because the implementation of JCR is not required to work on
> > top of a RDBMS. Moreover, the persistence managers have been usually
> > created to be used over different persistence solutions (and over
> > different RDBMS, which have different locking support/mechanisms).
>
> If that is so, concurrency code really should be inside the individual
> persistence solutions and not inside the Jackrabbit core. If it is,
> there always is a deadlock hazard.
>
It is. Maybe before going further you should check the code. If you
have some suggestions I guess everybody would be happy to hear about
different approaches.
./alex
--
.w( the_mindstorm )p.
> The way to go would be to remove synchronization from Jackrabbits core.
>
> What do you say?
>
> Oliver
>
Re: concurrent writes (JCR-314)
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
2007/6/4, Alexandru Popescu ☀ <th...@gmail.com>:
> On 6/4/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> > I am not deeply into the problem, but why is there locking on the Java
> > side anyway? It really should be possible to get along with DB locks
> > only.
> >
> > Calling the DB from synchronized blocks always bears the danger of deadlocks.
> >
>
> Basically because the implementation of JCR is not required to work on
> top of a RDBMS. Moreover, the persistence managers have been usually
> created to be used over different persistence solutions (and over
> different RDBMS, which have different locking support/mechanisms).
If that is so, concurrency code really should be inside the individual
persistence solutions and not inside the Jackrabbit core. If it is,
there always is a deadlock hazard.
The way to go would be to remove synchronization from Jackrabbits core.
What do you say?
Oliver
Re: concurrent writes (JCR-314)
Posted by Alexandru Popescu ☀ <th...@gmail.com>.
On 6/4/07, Oliver Zeigermann <ol...@zeigermann.de> wrote:
> I am not deeply into the problem, but why is there locking on the Java
> side anyway? It really should be possible to get along with DB locks
> only.
>
> Calling the DB from synchronized blocks always bears the danger of deadlocks.
>
Basically because the implementation of JCR is not required to work on
top of a RDBMS. Moreover, the persistence managers have been usually
created to be used over different persistence solutions (and over
different RDBMS, which have different locking support/mechanisms).
./alex
--
.w( the_mindstorm )p.
> Oliver
>
> 2007/6/4, Miguel Ángel Jiménez <mi...@gmail.com>:
> > I did some tests removing synchronized blocks from
> > BundleDbPersistenceManager and AbstractBundlePersistenceManager and all went
> > fine, but I think thorough testing is needed in this field.
> >
> > I would also like to see jackrabbit doing concurrent writes at its core,
> > delegating the lock management to the PersistenceManager implementation. A
> > BundleDbPersistenceManager could possibly offer concurrent writes whereas a
> > file based persistence manager might do one write at a time.
> >
> > Best regards,
> >
> > On 04/06/07, Marcel Reutegger <ma...@gmx.net> wrote:
> > >
> > > Hi Rafał,
> > >
> > > Rafał Kwiecień wrote:
> > > > Ok.So, how long it can take? 2 or 3 months? half year ?
> > > > It is important to me to know when this feature will be available.
> > >
> > > jackrabbit is an open source project and does not have a fixed road map
> > > nor
> > > detailed release plan. if such a feature is important to you, you are very
> > > welcome to participate.
> > >
> > > afaics the following steps are required to be able to support concurrent
> > > writes:
> > >
> > > - implement a ISMLocking that supports multiple write locks at a time
> > > - extend an existing or create a new persistence manager to support
> > > concurrent
> > > writes
> > > - make sure the jackrabbit core is able to handle concurrent writes. e.g.
> > > check
> > > if caches are synchronized properly.
> > > - optionally: enhance the search handler implementation to support
> > > concurrent
> > > writes. (this is not a hard requirement because when jackrabbit indexes
> > > content
> > > the write lock had been downgraded to a read lock)
> > >
> > > > I use BundlePersistenceManager. Methods in that persistence manager are
> > > > synchronized. So, there is not possible to read anything during write.
> > >
> > > can you please file a jira issue about this and if possible attach a
> > > patch? thanks.
> > >
> > > > BTW. If I use FineGrainedISMLocking, sometimes I see a warning in logs:
> > > > WARN [.core.query.lucene.SearchIndex] Exception while creating document
> > > for
> > > > node: aad7aa6a-5baf-4a33-b88d-f39f713aad1a:
> > > javax.jcr.RepositoryException:
> > > > Missing child node entry for node with id:
> > > > aad7aa6a-5baf-4a33-b88d-f39f713aad1a
> > > > Does it mean that some node has not been indexed ?
> > > > When I use DefaultISMLocking, I don't get warnings.
> > >
> > > please note that FineGrainedISMLocking is work in progress. there are some
> > > implications when using this class that need to be resolved first before
> > > it can
> > > be used. e.g. access to caches are not properly synchronized when using
> > > FineGrainedISMLocking.
> > >
> > > regards
> > > marcel
> > >
> >
> >
> >
> > --
> > Miguel.
> >
>
Re: concurrent writes (JCR-314)
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
I am not deeply into the problem, but why is there locking on the Java
side anyway? It really should be possible to get along with DB locks
only.
Calling the DB from synchronized blocks always bears the danger of deadlocks.
Oliver
2007/6/4, Miguel Ángel Jiménez <mi...@gmail.com>:
> I did some tests removing synchronized blocks from
> BundleDbPersistenceManager and AbstractBundlePersistenceManager and all went
> fine, but I think thorough testing is needed in this field.
>
> I would also like to see jackrabbit doing concurrent writes at its core,
> delegating the lock management to the PersistenceManager implementation. A
> BundleDbPersistenceManager could possibly offer concurrent writes whereas a
> file based persistence manager might do one write at a time.
>
> Best regards,
>
> On 04/06/07, Marcel Reutegger <ma...@gmx.net> wrote:
> >
> > Hi Rafał,
> >
> > Rafał Kwiecień wrote:
> > > Ok.So, how long it can take? 2 or 3 months? half year ?
> > > It is important to me to know when this feature will be available.
> >
> > jackrabbit is an open source project and does not have a fixed road map
> > nor
> > detailed release plan. if such a feature is important to you, you are very
> > welcome to participate.
> >
> > afaics the following steps are required to be able to support concurrent
> > writes:
> >
> > - implement a ISMLocking that supports multiple write locks at a time
> > - extend an existing or create a new persistence manager to support
> > concurrent
> > writes
> > - make sure the jackrabbit core is able to handle concurrent writes. e.g.
> > check
> > if caches are synchronized properly.
> > - optionally: enhance the search handler implementation to support
> > concurrent
> > writes. (this is not a hard requirement because when jackrabbit indexes
> > content
> > the write lock had been downgraded to a read lock)
> >
> > > I use BundlePersistenceManager. Methods in that persistence manager are
> > > synchronized. So, there is not possible to read anything during write.
> >
> > can you please file a jira issue about this and if possible attach a
> > patch? thanks.
> >
> > > BTW. If I use FineGrainedISMLocking, sometimes I see a warning in logs:
> > > WARN [.core.query.lucene.SearchIndex] Exception while creating document
> > for
> > > node: aad7aa6a-5baf-4a33-b88d-f39f713aad1a:
> > javax.jcr.RepositoryException:
> > > Missing child node entry for node with id:
> > > aad7aa6a-5baf-4a33-b88d-f39f713aad1a
> > > Does it mean that some node has not been indexed ?
> > > When I use DefaultISMLocking, I don't get warnings.
> >
> > please note that FineGrainedISMLocking is work in progress. there are some
> > implications when using this class that need to be resolved first before
> > it can
> > be used. e.g. access to caches are not properly synchronized when using
> > FineGrainedISMLocking.
> >
> > regards
> > marcel
> >
>
>
>
> --
> Miguel.
>
Re: concurrent writes (JCR-314)
Posted by Miguel Ángel Jiménez <mi...@gmail.com>.
I did some tests removing synchronized blocks from
BundleDbPersistenceManager and AbstractBundlePersistenceManager and all went
fine, but I think thorough testing is needed in this field.
I would also like to see jackrabbit doing concurrent writes at its core,
delegating the lock management to the PersistenceManager implementation. A
BundleDbPersistenceManager could possibly offer concurrent writes whereas a
file based persistence manager might do one write at a time.
Best regards,
On 04/06/07, Marcel Reutegger <ma...@gmx.net> wrote:
>
> Hi Rafał,
>
> Rafał Kwiecień wrote:
> > Ok.So, how long it can take? 2 or 3 months? half year ?
> > It is important to me to know when this feature will be available.
>
> jackrabbit is an open source project and does not have a fixed road map
> nor
> detailed release plan. if such a feature is important to you, you are very
> welcome to participate.
>
> afaics the following steps are required to be able to support concurrent
> writes:
>
> - implement a ISMLocking that supports multiple write locks at a time
> - extend an existing or create a new persistence manager to support
> concurrent
> writes
> - make sure the jackrabbit core is able to handle concurrent writes. e.g.
> check
> if caches are synchronized properly.
> - optionally: enhance the search handler implementation to support
> concurrent
> writes. (this is not a hard requirement because when jackrabbit indexes
> content
> the write lock had been downgraded to a read lock)
>
> > I use BundlePersistenceManager. Methods in that persistence manager are
> > synchronized. So, there is not possible to read anything during write.
>
> can you please file a jira issue about this and if possible attach a
> patch? thanks.
>
> > BTW. If I use FineGrainedISMLocking, sometimes I see a warning in logs:
> > WARN [.core.query.lucene.SearchIndex] Exception while creating document
> for
> > node: aad7aa6a-5baf-4a33-b88d-f39f713aad1a:
> javax.jcr.RepositoryException:
> > Missing child node entry for node with id:
> > aad7aa6a-5baf-4a33-b88d-f39f713aad1a
> > Does it mean that some node has not been indexed ?
> > When I use DefaultISMLocking, I don't get warnings.
>
> please note that FineGrainedISMLocking is work in progress. there are some
> implications when using this class that need to be resolved first before
> it can
> be used. e.g. access to caches are not properly synchronized when using
> FineGrainedISMLocking.
>
> regards
> marcel
>
--
Miguel.
Re: concurrent writes (JCR-314)
Posted by Marcel Reutegger <ma...@gmx.net>.
Pablo Rios wrote:
> Regarding the steps below required to support concurrent writes, what
> considerations should be taken into account in connection with JCR-18 issue ?
IMO JCR-18 should be closed as fixed and new issues should be created as new
deadlocks are observed. e.g. JCR-962.
as jackrabbit evolved, new synchronization and locking was added to the core. I
think their usage should be reviewed and changed accordingly. most important,
the sequence when locks are acquired.
> How does the Shared ISM instance for the global VersionManager works ?
it works just like a SharedISM of a regular workspace, except that it doesn't
have virtual item state providers attached.
regards
marcel
RE: concurrent writes (JCR-314)
Posted by Pablo Rios <pr...@bea.com>.
Regarding the steps below required to support concurrent writes, what considerations should be taken into account in connection with JCR-18 issue ? How does the Shared ISM instance for the global VersionManager works ?
Thanks,
Pablo
________________________________
From: Marcel Reutegger [mailto:marcel.reutegger@gmx.net]
Sent: Mon 6/4/2007 1:56 AM
To: users@jackrabbit.apache.org
Subject: Re: concurrent writes (JCR-314)
Hi Rafal,
Rafal Kwiecien wrote:
> Ok.So, how long it can take? 2 or 3 months? half year ?
> It is important to me to know when this feature will be available.
jackrabbit is an open source project and does not have a fixed road map nor
detailed release plan. if such a feature is important to you, you are very
welcome to participate.
afaics the following steps are required to be able to support concurrent writes:
- implement a ISMLocking that supports multiple write locks at a time
- extend an existing or create a new persistence manager to support concurrent
writes
- make sure the jackrabbit core is able to handle concurrent writes. e.g. check
if caches are synchronized properly.
- optionally: enhance the search handler implementation to support concurrent
writes. (this is not a hard requirement because when jackrabbit indexes content
the write lock had been downgraded to a read lock)
> I use BundlePersistenceManager. Methods in that persistence manager are
> synchronized. So, there is not possible to read anything during write.
can you please file a jira issue about this and if possible attach a patch? thanks.
> BTW. If I use FineGrainedISMLocking, sometimes I see a warning in logs:
> WARN [.core.query.lucene.SearchIndex] Exception while creating document for
> node: aad7aa6a-5baf-4a33-b88d-f39f713aad1a: javax.jcr.RepositoryException:
> Missing child node entry for node with id:
> aad7aa6a-5baf-4a33-b88d-f39f713aad1a
> Does it mean that some node has not been indexed ?
> When I use DefaultISMLocking, I don't get warnings.
please note that FineGrainedISMLocking is work in progress. there are some
implications when using this class that need to be resolved first before it can
be used. e.g. access to caches are not properly synchronized when using
FineGrainedISMLocking.
regards
marcel
Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
Re: concurrent writes (JCR-314)
Posted by Marcel Reutegger <ma...@gmx.net>.
Hi Rafał,
Rafał Kwiecień wrote:
> Ok.So, how long it can take? 2 or 3 months? half year ?
> It is important to me to know when this feature will be available.
jackrabbit is an open source project and does not have a fixed road map nor
detailed release plan. if such a feature is important to you, you are very
welcome to participate.
afaics the following steps are required to be able to support concurrent writes:
- implement a ISMLocking that supports multiple write locks at a time
- extend an existing or create a new persistence manager to support concurrent
writes
- make sure the jackrabbit core is able to handle concurrent writes. e.g. check
if caches are synchronized properly.
- optionally: enhance the search handler implementation to support concurrent
writes. (this is not a hard requirement because when jackrabbit indexes content
the write lock had been downgraded to a read lock)
> I use BundlePersistenceManager. Methods in that persistence manager are
> synchronized. So, there is not possible to read anything during write.
can you please file a jira issue about this and if possible attach a patch? thanks.
> BTW. If I use FineGrainedISMLocking, sometimes I see a warning in logs:
> WARN [.core.query.lucene.SearchIndex] Exception while creating document for
> node: aad7aa6a-5baf-4a33-b88d-f39f713aad1a: javax.jcr.RepositoryException:
> Missing child node entry for node with id:
> aad7aa6a-5baf-4a33-b88d-f39f713aad1a
> Does it mean that some node has not been indexed ?
> When I use DefaultISMLocking, I don't get warnings.
please note that FineGrainedISMLocking is work in progress. there are some
implications when using this class that need to be resolved first before it can
be used. e.g. access to caches are not properly synchronized when using
FineGrainedISMLocking.
regards
marcel
Re: concurrent writes (JCR-314)
Posted by Rafał Kwiecień <ra...@consol.pl>.
Dnia sobota, 2 czerwca 2007 11:25, Marcel Reutegger napisał:
> Rafał Kwiecień wrote:
> > The patch ( FineGrainedISMLocking.patch) allows to read (from cache)
> > during writing but concurrent writes are still not possible. I need to
> > know when concurrent writes in the jackrabbit will be possible. In a
> > month, year ?
>
> I'd say most probably not within a month.
Ok.So, how long it can take? 2 or 3 months? half year ?
It is important to me to know when this feature will be available.
>
> What persistence manager do you use? with the patch you mentioned it should
> be possible to read not just from the cache but also from the persistence
> manager during a non-conflicting write.
I use BundlePersistenceManager. Methods in that persistence manager are
synchronized. So, there is not possible to read anything during write.
BTW. If I use FineGrainedISMLocking, sometimes I see a warning in logs:
WARN [.core.query.lucene.SearchIndex] Exception while creating document for
node: aad7aa6a-5baf-4a33-b88d-f39f713aad1a: javax.jcr.RepositoryException:
Missing child node entry for node with id:
aad7aa6a-5baf-4a33-b88d-f39f713aad1a
Does it mean that some node has not been indexed ?
When I use DefaultISMLocking, I don't get warnings.
Thanks,
Rafał
--
Rafał Kwiecień
ConSol* Consulting & Solutions Software Poland Sp. z o.o.
ul. Piastowska 44C, 30-070 Kraków
http://www.consol.pl/