You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Daniel Pfeifer <Da...@tradedoubler.com> on 2006/01/24 14:52:19 UTC
Two strange things in Lucene
Today I've been alerted by one of my collegues that our Lucene-based
indexing solution no longer refreshes the searchers and thus we never
get any new indexed documents.
Since I didn't find anything in the log from log4j I did a "kill -3" on
the process and found two very interesting things:
Almost all multisearcher threads were in this state:
"MultiSearcher thread #1" daemon prio=10 tid=0x0000000001900960
nid=0x81442c waiting for monitor entry
[0xfffffd7d269ff000..0xfffffd7d269ffb50]
at java.util.Vector.size(Vector.java:270)
- waiting to lock <0xfffffd7f0114ea28> (a java.util.Vector)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.
java:95)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:220
)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.
java:97)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:220
)
at org.apache.lucene.search.Query.weight(Query.java:85)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:85)
at
org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.j
ava:251)
And, additionally I found another stacktrace in the stdout-log which I
find interesting:
Exception in thread "MultiSearcher thread #1"
org.apache.lucene.search.BooleanQuery$TooManyClauses
at
org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:79)
at
org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:71)
at
org.apache.lucene.search.PrefixQuery.rewrite(PrefixQuery.java:50)
at
org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:243)
at
org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:243)
at
org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:166)
at org.apache.lucene.search.Query.weight(Query.java:84)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:85)
at
org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.j
ava:251)
Has anyone ever seen these errors and knows why they might occour?
Thanks in advance,
Daniel Pfeifer
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Two strange things in Lucene
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Jan 24, 2006, at 8:52 AM, Daniel Pfeifer wrote:
> Today I've been alerted by one of my collegues that our Lucene-based
> indexing solution no longer refreshes the searchers and thus we never
> get any new indexed documents.
>
> Since I didn't find anything in the log from log4j I did a "kill
> -3" on
> the process and found two very interesting things:
>
> Almost all multisearcher threads were in this state:
>
> "MultiSearcher thread #1" daemon prio=10 tid=0x0000000001900960
> nid=0x81442c waiting for monitor entry
> [0xfffffd7d269ff000..0xfffffd7d269ffb50]
> at java.util.Vector.size(Vector.java:270)
> - waiting to lock <0xfffffd7f0114ea28> (a java.util.Vector)
> at
> org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>
> (BooleanQuery.
> java:95)
I don't know about this one, but guessing that it just happens to be
a normal state of the system when you killed the process. *shrugs*
>
> And, additionally I found another stacktrace in the stdout-log which I
> find interesting:
>
> Exception in thread "MultiSearcher thread #1"
> org.apache.lucene.search.BooleanQuery$TooManyClauses
This is a typical occurrence when using Query's that expand such as
WildcardQuery, RangeQuery, FuzzyQuery, etc. If users are doing
queries like a* and there are over 1024 terms that start with "a"
then you will, by default, blow up WildcardQuery's expansion into a
BooleanQuery. You can up that limit on BooleanQuery, or disallow
those types of queries perhaps.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Two strange things in Lucene
Posted by Pa...@saaconsultants.com.
The TooManyClauses exception is due to the prefix query being rewritten to
a boolean query that exceeds the boolean queries maximum number of clauses.
Its an unchecked exception from the search method that you should probably
explicitly catch and then return a helpful message to the user maybe
suggesting they refine their search?
try
{
Hits h = searcher.search(query, filter, sort);
}
catch (TooManyClauses e)
{
// return a msg to user that the query was too general
}
Regards
Paul I.
"Daniel Pfeifer"
<Daniel.Pfeifer@t
radedoubler.com> To
<ja...@lucene.apache.org>
24/01/2006 13:52 cc
Subject
Please respond to Two strange things in Lucene
java-user@lucene.
apache.org
Today I've been alerted by one of my collegues that our Lucene-based
indexing solution no longer refreshes the searchers and thus we never
get any new indexed documents.
Since I didn't find anything in the log from log4j I did a "kill -3" on
the process and found two very interesting things:
Almost all multisearcher threads were in this state:
"MultiSearcher thread #1" daemon prio=10 tid=0x0000000001900960
nid=0x81442c waiting for monitor entry
[0xfffffd7d269ff000..0xfffffd7d269ffb50]
at java.util.Vector.size(Vector.java:270)
- waiting to lock <0xfffffd7f0114ea28> (a java.util.Vector)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.
java:95)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:220
)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.
java:97)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:220
)
at org.apache.lucene.search.Query.weight(Query.java:85)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:85)
at
org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.j
ava:251)
And, additionally I found another stacktrace in the stdout-log which I
find interesting:
Exception in thread "MultiSearcher thread #1"
org.apache.lucene.search.BooleanQuery$TooManyClauses
at
org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:79)
at
org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:71)
at
org.apache.lucene.search.PrefixQuery.rewrite(PrefixQuery.java:50)
at
org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:243)
at
org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:243)
at
org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:166)
at org.apache.lucene.search.Query.weight(Query.java:84)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:85)
at
org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.j
ava:251)
Has anyone ever seen these errors and knows why they might occour?
Thanks in advance,
Daniel Pfeifer
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org