You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Shailendra Mudgal <mu...@gmail.com> on 2010/10/07 07:12:48 UTC
Multiple searcher blocked threads
Hi Everyone,
Recently we have migrated from lucene 2.2 to lucene 2.9.3.
We are having some issues in search. During the load, searchers are getting
hung up. When we took a process stack, we sound that there are around 450
threads are in the blocked state.
Thread t@885: (state = BLOCKED)
-
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(byte[],
int, int) @bci=7, line=115 (Compiled frame)
- org.apache.lucene.store.BufferedIndexInput.readBytes(byte[], int, int,
boolean) @bci=203, line=136 (Compiled frame)
-
org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(byte[],
int, int) @bci=51, line=247 (Compiled frame)
- org.apache.lucene.store.BufferedIndexInput.refill() @bci=87, line=157
(Compiled frame)
- org.apache.lucene.store.BufferedIndexInput.readByte() @bci=12, line=38
(Compiled frame)
- org.apache.lucene.store.IndexInput.readVInt() @bci=1, line=78 (Compiled
frame)
-
org.apache.lucene.index.TermBuffer.read(org.apache.lucene.store.IndexInput,
org.apache.lucene.index.FieldInfos) @bci=6, line=64 (Compiled frame)
- org.apache.lucene.index.SegmentTermEnum.next() @bci=64, line=129
(Compiled frame)
-
org.apache.lucene.index.SegmentTermEnum.scanTo(org.apache.lucene.index.Term)
@bci=25, line=160 (Compiled frame)
- org.apache.lucene.index.TermInfosReader.get(org.apache.lucene.index.Term,
boolean) @bci=211, line=232 (Compiled frame)
- org.apache.lucene.index.TermInfosReader.get(org.apache.lucene.index.Term)
@bci=3, line=179 (Compiled frame)
-
org.apache.lucene.index.SegmentReader.docFreq(org.apache.lucene.index.Term)
@bci=12, line=997 (Compiled frame)
-
org.apache.lucene.index.DirectoryReader.docFreq(org.apache.lucene.index.Term)
@bci=25, line=633 (Compiled frame)
-
org.apache.lucene.index.FilterIndexReader.docFreq(org.apache.lucene.index.Term)
@bci=9, line=195 (Compiled frame)
-
org.apache.lucene.search.IndexSearcher.docFreq(org.apache.lucene.index.Term)
@bci=5, line=147 (Compiled frame)
-
org.apache.lucene.search.Similarity.idfExplain(org.apache.lucene.index.Term,
org.apache.lucene.search.Searcher) @bci=29, line=765 (Compiled frame)
-
org.apache.lucene.search.TermQuery$TermWeight.<init>(org.apache.lucene.search.TermQuery,
org.apache.lucene.search.Searcher) @bci=28, line=46 (Compiled frame)
-
org.apache.lucene.search.TermQuery.createWeight(org.apache.lucene.search.Searcher)
@bci=6, line=146 (Compiled frame)
-
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(org.apache.lucene.search.BooleanQuery,
org.apache.lucene.search.Searcher) @bci=72, line=184 (Compiled f
rame)
-
org.apache.lucene.search.BooleanQuery.createWeight(org.apache.lucene.search.Searcher)
@bci=6, line=415 (Interpreted frame)
-
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(org.apache.lucene.search.BooleanQuery,
org.apache.lucene.search.Searcher) @bci=72, line=184 (Compiled f
rame)
-
org.apache.lucene.search.BooleanQuery.createWeight(org.apache.lucene.search.Searcher)
@bci=6, line=415 (Interpreted frame)
- org.apache.lucene.search.Query.weight(org.apache.lucene.search.Searcher)
@bci=8, line=99 (Interpreted frame)
-
org.apache.lucene.search.Searcher.createWeight(org.apache.lucene.search.Query)
@bci=2, line=230 (Interpreted frame)
- org.apache.lucene.search.Searcher.search(org.apache.lucene.search.Query,
org.apache.lucene.search.HitCollector) @bci=3, line=110 (Interpreted frame)
- lucene.searcher.MySearcherThread.run() @bci=99, line=52 (Compiled frame)
I have a feeling that these blocked threads are the causing searcher not to
respond.
Our application is running on SunOS and we are using jdk1.6_04
Thanks in advance..
Vipin
Re: Multiple searcher blocked threads
Posted by Shailendra Mudgal <mu...@gmail.com>.
Hi Yannis,
Thanks for your reply.
It fixed the problem.
Thanks again.
On Thu, Oct 7, 2010 at 10:52 AM, Yannis Pavlidis <yp...@oneriot.com>wrote:
> I would recommend you use NIOFSDirectory. We had similar issues and after
> we switched to NIOFSDirectory these issues disappeared (dramatically
> reduced).
>
> Also if you are using filters what has really helped us is to use cached
> filters (CachingWrapperFilters).
>
> Hope that helps.
>
> Yannis.
>
> On Oct 6, 2010, at 11:12 PM, Shailendra Mudgal wrote:
>
> > Hi Everyone,
> >
> > Recently we have migrated from lucene 2.2 to lucene 2.9.3.
> >
> > We are having some issues in search. During the load, searchers are
> getting
> > hung up. When we took a process stack, we sound that there are around 450
> > threads are in the blocked state.
> >
> >
> > Thread t@885: (state = BLOCKED)
> > -
> >
> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(byte[],
> > int, int) @bci=7, line=115 (Compiled frame)
> > - org.apache.lucene.store.BufferedIndexInput.readBytes(byte[], int, int,
> > boolean) @bci=203, line=136 (Compiled frame)
> > -
> >
> org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(byte[],
> > int, int) @bci=51, line=247 (Compiled frame)
> > - org.apache.lucene.store.BufferedIndexInput.refill() @bci=87, line=157
> > (Compiled frame)
> > - org.apache.lucene.store.BufferedIndexInput.readByte() @bci=12, line=38
> > (Compiled frame)
> > - org.apache.lucene.store.IndexInput.readVInt() @bci=1, line=78 (Compiled
> > frame)
> > -
> >
> org.apache.lucene.index.TermBuffer.read(org.apache.lucene.store.IndexInput,
> > org.apache.lucene.index.FieldInfos) @bci=6, line=64 (Compiled frame)
> > - org.apache.lucene.index.SegmentTermEnum.next() @bci=64, line=129
> > (Compiled frame)
> > -
> >
> org.apache.lucene.index.SegmentTermEnum.scanTo(org.apache.lucene.index.Term)
> > @bci=25, line=160 (Compiled frame)
> > -
> org.apache.lucene.index.TermInfosReader.get(org.apache.lucene.index.Term,
> > boolean) @bci=211, line=232 (Compiled frame)
> > -
> org.apache.lucene.index.TermInfosReader.get(org.apache.lucene.index.Term)
> > @bci=3, line=179 (Compiled frame)
> > -
> >
> org.apache.lucene.index.SegmentReader.docFreq(org.apache.lucene.index.Term)
> > @bci=12, line=997 (Compiled frame)
> > -
> >
> org.apache.lucene.index.DirectoryReader.docFreq(org.apache.lucene.index.Term)
> > @bci=25, line=633 (Compiled frame)
> > -
> >
> org.apache.lucene.index.FilterIndexReader.docFreq(org.apache.lucene.index.Term)
> > @bci=9, line=195 (Compiled frame)
> > -
> >
> org.apache.lucene.search.IndexSearcher.docFreq(org.apache.lucene.index.Term)
> > @bci=5, line=147 (Compiled frame)
> > -
> >
> org.apache.lucene.search.Similarity.idfExplain(org.apache.lucene.index.Term,
> > org.apache.lucene.search.Searcher) @bci=29, line=765 (Compiled frame)
> > -
> >
> org.apache.lucene.search.TermQuery$TermWeight.<init>(org.apache.lucene.search.TermQuery,
> > org.apache.lucene.search.Searcher) @bci=28, line=46 (Compiled frame)
> > -
> >
> org.apache.lucene.search.TermQuery.createWeight(org.apache.lucene.search.Searcher)
> > @bci=6, line=146 (Compiled frame)
> > -
> >
> org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(org.apache.lucene.search.BooleanQuery,
> > org.apache.lucene.search.Searcher) @bci=72, line=184 (Compiled f
> > rame)
> > -
> >
> org.apache.lucene.search.BooleanQuery.createWeight(org.apache.lucene.search.Searcher)
> > @bci=6, line=415 (Interpreted frame)
> > -
> >
> org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(org.apache.lucene.search.BooleanQuery,
> > org.apache.lucene.search.Searcher) @bci=72, line=184 (Compiled f
> > rame)
> > -
> >
> org.apache.lucene.search.BooleanQuery.createWeight(org.apache.lucene.search.Searcher)
> > @bci=6, line=415 (Interpreted frame)
> > -
> org.apache.lucene.search.Query.weight(org.apache.lucene.search.Searcher)
> > @bci=8, line=99 (Interpreted frame)
> > -
> >
> org.apache.lucene.search.Searcher.createWeight(org.apache.lucene.search.Query)
> > @bci=2, line=230 (Interpreted frame)
> > -
> org.apache.lucene.search.Searcher.search(org.apache.lucene.search.Query,
> > org.apache.lucene.search.HitCollector) @bci=3, line=110 (Interpreted
> frame)
> > - lucene.searcher.MySearcherThread.run() @bci=99, line=52 (Compiled
> frame)
> >
> >
> >
> > I have a feeling that these blocked threads are the causing searcher not
> to
> > respond.
> >
> > Our application is running on SunOS and we are using jdk1.6_04
> >
> > Thanks in advance..
> > Vipin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Multiple searcher blocked threads
Posted by Yannis Pavlidis <yp...@oneriot.com>.
I would recommend you use NIOFSDirectory. We had similar issues and after we switched to NIOFSDirectory these issues disappeared (dramatically reduced).
Also if you are using filters what has really helped us is to use cached filters (CachingWrapperFilters).
Hope that helps.
Yannis.
On Oct 6, 2010, at 11:12 PM, Shailendra Mudgal wrote:
> Hi Everyone,
>
> Recently we have migrated from lucene 2.2 to lucene 2.9.3.
>
> We are having some issues in search. During the load, searchers are getting
> hung up. When we took a process stack, we sound that there are around 450
> threads are in the blocked state.
>
>
> Thread t@885: (state = BLOCKED)
> -
> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(byte[],
> int, int) @bci=7, line=115 (Compiled frame)
> - org.apache.lucene.store.BufferedIndexInput.readBytes(byte[], int, int,
> boolean) @bci=203, line=136 (Compiled frame)
> -
> org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(byte[],
> int, int) @bci=51, line=247 (Compiled frame)
> - org.apache.lucene.store.BufferedIndexInput.refill() @bci=87, line=157
> (Compiled frame)
> - org.apache.lucene.store.BufferedIndexInput.readByte() @bci=12, line=38
> (Compiled frame)
> - org.apache.lucene.store.IndexInput.readVInt() @bci=1, line=78 (Compiled
> frame)
> -
> org.apache.lucene.index.TermBuffer.read(org.apache.lucene.store.IndexInput,
> org.apache.lucene.index.FieldInfos) @bci=6, line=64 (Compiled frame)
> - org.apache.lucene.index.SegmentTermEnum.next() @bci=64, line=129
> (Compiled frame)
> -
> org.apache.lucene.index.SegmentTermEnum.scanTo(org.apache.lucene.index.Term)
> @bci=25, line=160 (Compiled frame)
> - org.apache.lucene.index.TermInfosReader.get(org.apache.lucene.index.Term,
> boolean) @bci=211, line=232 (Compiled frame)
> - org.apache.lucene.index.TermInfosReader.get(org.apache.lucene.index.Term)
> @bci=3, line=179 (Compiled frame)
> -
> org.apache.lucene.index.SegmentReader.docFreq(org.apache.lucene.index.Term)
> @bci=12, line=997 (Compiled frame)
> -
> org.apache.lucene.index.DirectoryReader.docFreq(org.apache.lucene.index.Term)
> @bci=25, line=633 (Compiled frame)
> -
> org.apache.lucene.index.FilterIndexReader.docFreq(org.apache.lucene.index.Term)
> @bci=9, line=195 (Compiled frame)
> -
> org.apache.lucene.search.IndexSearcher.docFreq(org.apache.lucene.index.Term)
> @bci=5, line=147 (Compiled frame)
> -
> org.apache.lucene.search.Similarity.idfExplain(org.apache.lucene.index.Term,
> org.apache.lucene.search.Searcher) @bci=29, line=765 (Compiled frame)
> -
> org.apache.lucene.search.TermQuery$TermWeight.<init>(org.apache.lucene.search.TermQuery,
> org.apache.lucene.search.Searcher) @bci=28, line=46 (Compiled frame)
> -
> org.apache.lucene.search.TermQuery.createWeight(org.apache.lucene.search.Searcher)
> @bci=6, line=146 (Compiled frame)
> -
> org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(org.apache.lucene.search.BooleanQuery,
> org.apache.lucene.search.Searcher) @bci=72, line=184 (Compiled f
> rame)
> -
> org.apache.lucene.search.BooleanQuery.createWeight(org.apache.lucene.search.Searcher)
> @bci=6, line=415 (Interpreted frame)
> -
> org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(org.apache.lucene.search.BooleanQuery,
> org.apache.lucene.search.Searcher) @bci=72, line=184 (Compiled f
> rame)
> -
> org.apache.lucene.search.BooleanQuery.createWeight(org.apache.lucene.search.Searcher)
> @bci=6, line=415 (Interpreted frame)
> - org.apache.lucene.search.Query.weight(org.apache.lucene.search.Searcher)
> @bci=8, line=99 (Interpreted frame)
> -
> org.apache.lucene.search.Searcher.createWeight(org.apache.lucene.search.Query)
> @bci=2, line=230 (Interpreted frame)
> - org.apache.lucene.search.Searcher.search(org.apache.lucene.search.Query,
> org.apache.lucene.search.HitCollector) @bci=3, line=110 (Interpreted frame)
> - lucene.searcher.MySearcherThread.run() @bci=99, line=52 (Compiled frame)
>
>
>
> I have a feeling that these blocked threads are the causing searcher not to
> respond.
>
> Our application is running on SunOS and we are using jdk1.6_04
>
> Thanks in advance..
> Vipin
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org