Threads locking around getFSDirectory

[Archie Cowan <ar...@gmail.com>]

Lucene users,

We run an EJB search application that has been experiencing some locking
issues. By taking thread dumps when the jvm seems to be frozen, many threads
in the application are locked around calls to getFSDirectory.

"Thread-84" daemon prio=1 tid=0x053117b0 nid=0x2274 waiting for monitor
entry [4666000..466686c]
at org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:109)
- waiting to lock <0x4d3a86a0> (a org.apache.lucene.store.FSDirectory)
at org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:83)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:95)
at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:38)

Details about our app:
We run inside JBoss 4.0.0
Sun JDK 1.4.2
Lucene 1.4.3
Compound Index Format
Red Hat Linux 3

Our initial research into the problem only brought up comments around
performance improvements made to lucene last year that are present in
version 1.4.3. We're wondering if the lucene user community can give any
advice for pursuing this further. Some questions we have are:

How many IndexSearchers do people usually use in their applications?
- In previous lucene versions, we found we had better concurrency using
multiple IndexSearchers
- Our current app uses an separate IndexSearcher for each thread (as needed)
from a pool

We've seen reports that the multi file indexes offer a faster search. Are
there any performance numbers around on concurrency with the multifile index
format? It seems like this could either improve or further increase
contention with locking depending on what our problem is.

Some of our searching around this problem revealed some work on something
using the java nio package.... NioFSDirectory that offered better
concurrency (
http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200505.mbox/%3C4295F65B.8030100@apache.org%3E).
We haven't found this in the lucene repository... where or is this available
for use?

Thanks in advance.

Regards,

Archie

Re: Threads locking around getFSDirectory

[Chris Hostetter <ho...@fucit.org>]

: How many IndexSearchers do people usually use in their applications?
: - In previous lucene versions, we found we had better concurrency using
: multiple IndexSearchers
: - Our current app uses an separate IndexSearcher for each thread (as needed)
: from a pool

In my experience, one "live" searcher is allways plenty, with an "on-deck"
searcher that is opened and used to warm up FieldCaches and filehandles
before being swaped over to replce the "live" searcher when updates have
been made.


: using the java nio package.... NioFSDirectory that offered better
: concurrency (
: http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200505.mbox/%3C4295F65B.8030100@apache.org%3E).
: We haven't found this in the lucene repository... where or is this available
: for use?

I think this is what you are looking for, but i'm not familiar with it at
all...

	http://issues.apache.org/jira/browse/LUCENE-414


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org