You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by jason rutherglen <ja...@yahoo.com> on 2006/06/08 22:33:16 UTC
ReentrantReadWriteLock
Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?
Re: ReentrantLock
Posted by Yonik Seeley <ys...@gmail.com>.
Of course it might take fewer CPUs to show a difference:
http://www-128.ibm.com/developerworks/java/library/j-jtp10264/
But that's in the inner loop.... and I can't think of anything in Solr
that synchronizes at that low of a level. Where this could really pay
off is Lucene's IndexReader.isDeleted() call that Scorers make... but
Lucene is Java1.4 compatible for now.
-Yonik
On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> What about ReentrantLock? I like it because it cleans up sync blocks. Adds a few nice methods. Makes threading more OO.
>
> ----- Original Message ----
> From: Yonik Seeley <ys...@gmail.com>
> To: solr-dev@lucene.apache.org; jason rutherglen <ja...@yahoo.com>
> Sent: Thursday, June 8, 2006 1:50:45 PM
> Subject: Re: ReentrantReadWriteLock
>
> On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> > Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?
>
> Not really... ReentrantReadWriteLocks beat normal synchronization when
> there are many readers, fewer writers, and heavy contention for the
> locks.
>
> In the case of our LRUCache, there aren't any pure readers... It's
> implemented with LinkedHashMap, and every lookup changes the ordering
> of the entries.
>
> ConcurrentHashMap is a cool class, but there is no
> LinkedConcurrentHashMap. If you want to develop one feel free :-) In
> profiling though, cache lookups aren't that significant. Are you
> seeing something that indicates otherwise?
>
> -Yonik
> http://incubator.apache.org/solr Solr, the open-source Lucene search server
>
>
>
>
>
>
--
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server
Re: ReentrantLock
Posted by Yonik Seeley <ys...@gmail.com>.
On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> What about ReentrantLock? I like it because it cleans up sync blocks. Adds a few nice methods. Makes threading more OO.
I don't know about cleaning up.... the syntax is slightly messier
because you always need a finally block.
It could have it's place if we needed lock polling or timed lock
waits. It's really only faster under heavy contention AFAIK. So if
you are running Solr on one of those monster 128 or 256 CPU machines,
then it might matter more :-) Of course, at that point, Lucene might
still be the weakest link in the concurrency chain too. Only way to
tell is to have someone with a high CPU count do some
testing/profiling.
-Yonik
Re: ReentrantReadWriteLock
Posted by Yonik Seeley <ys...@gmail.com>.
On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> Yeah
> LinkedConcurrentHashMap would be nice, that could be done. Is it needed for
> anything right now?
I don't think so... if it existed, I would have tried it out against
the standard synchronized LinkedHashMap. For most uses I doubt it
would be an improvement though... at least for our hardware with 2-4
CPUs per box.
-Yonik
Re: ReentrantReadWriteLock
Posted by jason rutherglen <ja...@yahoo.com>.
I don't see any place in particular, though since there is a lot of locking in Solr, and a lot I don't fully understand, thought I'd ask. I don't see it used much in projects, but it is a nice piece of work. Yeah LinkedConcurrentHashMap would be nice, that could be done. Is it needed for anything right now?
----- Original Message ----
From: Yonik Seeley <ys...@gmail.com>
To: solr-dev@lucene.apache.org; jason rutherglen <ja...@yahoo.com>
Sent: Thursday, June 8, 2006 1:50:45 PM
Subject: Re: ReentrantReadWriteLock
On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?
Not really... ReentrantReadWriteLocks beat normal synchronization when
there are many readers, fewer writers, and heavy contention for the
locks.
In the case of our LRUCache, there aren't any pure readers... It's
implemented with LinkedHashMap, and every lookup changes the ordering
of the entries.
ConcurrentHashMap is a cool class, but there is no
LinkedConcurrentHashMap. If you want to develop one feel free :-) In
profiling though, cache lookups aren't that significant. Are you
seeing something that indicates otherwise?
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server
ReentrantLock
Posted by jason rutherglen <ja...@yahoo.com>.
What about ReentrantLock? I like it because it cleans up sync blocks. Adds a few nice methods. Makes threading more OO.
----- Original Message ----
From: Yonik Seeley <ys...@gmail.com>
To: solr-dev@lucene.apache.org; jason rutherglen <ja...@yahoo.com>
Sent: Thursday, June 8, 2006 1:50:45 PM
Subject: Re: ReentrantReadWriteLock
On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?
Not really... ReentrantReadWriteLocks beat normal synchronization when
there are many readers, fewer writers, and heavy contention for the
locks.
In the case of our LRUCache, there aren't any pure readers... It's
implemented with LinkedHashMap, and every lookup changes the ordering
of the entries.
ConcurrentHashMap is a cool class, but there is no
LinkedConcurrentHashMap. If you want to develop one feel free :-) In
profiling though, cache lookups aren't that significant. Are you
seeing something that indicates otherwise?
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server
Re: ReentrantReadWriteLock
Posted by Yonik Seeley <ys...@gmail.com>.
On 6/8/06, jason rutherglen <ja...@yahoo.com> wrote:
> Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?
Not really... ReentrantReadWriteLocks beat normal synchronization when
there are many readers, fewer writers, and heavy contention for the
locks.
In the case of our LRUCache, there aren't any pure readers... It's
implemented with LinkedHashMap, and every lookup changes the ordering
of the entries.
ConcurrentHashMap is a cool class, but there is no
LinkedConcurrentHashMap. If you want to develop one feel free :-) In
profiling though, cache lookups aren't that significant. Are you
seeing something that indicates otherwise?
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server