You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by miki sun <su...@hotmail.com> on 2006/04/04 13:29:14 UTC
Re: QueryParser error + solution
Thanks Erik and Michael!
I copied some code from demo.SearchFiles.java, I do not have a more clearer
tracing message. Now it works.
But do you have a better way than this:
//escaping special chars for query parser
for (int i = 0; i < qr.length(); i ++)
{
if( (qr.charAt(i) == '+') ||(qr.charAt(i) == '-')
||(qr.charAt(i) == '&')||(qr.charAt(i) == '~')
||(qr.charAt(i) == '|')|| (qr.charAt(i) == '!')
||(qr.charAt(i) == '[')||(qr.charAt(i) == ']')
||(qr.charAt(i) == '(')||(qr.charAt(i) == ')')
||(qr.charAt(i) == '{')||(qr.charAt(i) == '}')
||(qr.charAt(i) == '^')||(qr.charAt(i) == '"')
||(qr.charAt(i) == '*')||(qr.charAt(i) == '?')
||(qr.charAt(i) == ':')||(qr.charAt(i) == '\\') )
theQueries += "\\" + qr.charAt(i);
else
theQueries += qr.charAt(i) ;
}
----Original Message Follows----
From: Erik Hatcher <er...@ehatchersolutions.com>
Reply-To: java-user@lucene.apache.org
To: java-user@lucene.apache.org
Subject: Re: QueryParser error
Date: Tue, 4 Apr 2006 07:01:41 -0400
Miki hasn't posted a complete stack trace, and if the NPE is coming from
QueryParser it would likely give stack trace information pinpointing
precisely where the issue is. My hunch is some other code is obfuscating
the real issue. A complete stack trace would be much more helpful.
Erik
On Apr 4, 2006, at 6:55 AM, Michael Barry wrote:
>Exclamation point too ... check out the "Escaping Special Characters"
>section in the QueryParser syntax guide (http://lucene.apache.org/
>java/docs/queryparsersyntax.html)
>
>miki sun wrote:
>>Still got error even without colon:
>>
>>Error in parse query :The light of the body is the eye if therefore thine
>>eye be single, thy whole body shall be full of light. But if thine eye be
>>evil, thy whole body shall be full of darkness. If therefore the light
>>that is in thee be darkness, how great is that darkness!
>>
>>java.lang.NullPointerException
>>
>>
>>
>>----Original Message Follows----
>>From: Michael Barry <mb...@cos.com>
>>Reply-To: java-user@lucene.apache.org
>>To: java-user@lucene.apache.org
>>Subject: Re: QueryParser error
>>Date: Tue, 04 Apr 2006 06:44:02 -0400
>>
>>You need to escape the colons.
>>
>>miki sun wrote:
>>>Hi there
>>>
>>>Who can tell me why I got the the queryParser error for the following
>>>query:
>>>
>>>Error in parse query :The light of the body is the eye: if therefore
>>>thine eye be single, thy whole body shall be full of light. But if thine
>>>eye be evil, thy whole body shall be full of darkness. If therefore the
>>>light that is in thee be darkness, how great is that darkness!
>>>java.lang.NullPointerException
>>>
>>>Appart from [, ], (, ), whatelse should be exculded in the query?
>>>
>>>Thanks
>>>
>>>Miki
>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: QueryParser error + solution
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
If you're going to escape all the special characters used as
operators in the query expression, then what is the point of using
QueryParser? Seems like you could just tokenize the text yourself
and build a Query instance via API to support your needs. Do you
need to leverage any special operators of QueryParser? Or are you
just using it to take a string of words and turn it into a BooleanQuery?
Erik
On Apr 4, 2006, at 7:29 AM, miki sun wrote:
> Thanks Erik and Michael!
>
> I copied some code from demo.SearchFiles.java, I do not have a more
> clearer tracing message. Now it works.
>
> But do you have a better way than this:
>
> //escaping special chars for query parser
> for (int i = 0; i < qr.length(); i ++)
> {
> if( (qr.charAt(i) == '+') ||(qr.charAt(i) == '-')
> ||(qr.charAt(i) == '&')||(qr.charAt(i) == '~')
> ||(qr.charAt(i) == '|')|| (qr.charAt(i) == '!')
> ||(qr.charAt(i) == '[')||(qr.charAt(i) == ']')
> ||(qr.charAt(i) == '(')||(qr.charAt(i) == ')')
> ||(qr.charAt(i) == '{')||(qr.charAt(i) == '}')
> ||(qr.charAt(i) == '^')||(qr.charAt(i) == '"')
> ||(qr.charAt(i) == '*')||(qr.charAt(i) == '?')
> ||(qr.charAt(i) == ':')||(qr.charAt(i) == '\\') )
> theQueries += "\\" + qr.charAt(i);
> else
> theQueries += qr.charAt(i) ;
> }
>
>
>
> ----Original Message Follows----
> From: Erik Hatcher <er...@ehatchersolutions.com>
> Reply-To: java-user@lucene.apache.org
> To: java-user@lucene.apache.org
> Subject: Re: QueryParser error
> Date: Tue, 4 Apr 2006 07:01:41 -0400
>
> Miki hasn't posted a complete stack trace, and if the NPE is
> coming from QueryParser it would likely give stack trace
> information pinpointing precisely where the issue is. My hunch is
> some other code is obfuscating the real issue. A complete stack
> trace would be much more helpful.
>
> Erik
>
>
> On Apr 4, 2006, at 6:55 AM, Michael Barry wrote:
>
>> Exclamation point too ... check out the "Escaping Special Characters"
>> section in the QueryParser syntax guide (http://lucene.apache.org/
>> java/docs/queryparsersyntax.html)
>>
>> miki sun wrote:
>>> Still got error even without colon:
>>>
>>> Error in parse query :The light of the body is the eye if
>>> therefore thine eye be single, thy whole body shall be full of
>>> light. But if thine eye be evil, thy whole body shall be full of
>>> darkness. If therefore the light that is in thee be darkness,
>>> how great is that darkness!
>>>
>>> java.lang.NullPointerException
>>>
>>>
>>>
>>> ----Original Message Follows----
>>> From: Michael Barry <mb...@cos.com>
>>> Reply-To: java-user@lucene.apache.org
>>> To: java-user@lucene.apache.org
>>> Subject: Re: QueryParser error
>>> Date: Tue, 04 Apr 2006 06:44:02 -0400
>>>
>>> You need to escape the colons.
>>>
>>> miki sun wrote:
>>>> Hi there
>>>>
>>>> Who can tell me why I got the the queryParser error for the
>>>> following query:
>>>>
>>>> Error in parse query :The light of the body is the eye: if
>>>> therefore thine eye be single, thy whole body shall be full of
>>>> light. But if thine eye be evil, thy whole body shall be full
>>>> of darkness. If therefore the light that is in thee be
>>>> darkness, how great is that darkness!
>>>> java.lang.NullPointerException
>>>>
>>>> Appart from [, ], (, ), whatelse should be exculded in the query?
>>>>
>>>> Thanks
>>>>
>>>> Miki
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: search.Similarity
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
In case anyone misses the smiley, I'm just teasing. Yes, there are
years of research and heavy duty experience that are behind Lucene.
There are quite a number of research documents and books that
describe information retrieval theory and practice, several of them
linked here:
<http://wiki.apache.org/jakarta-lucene/InformationRetrieval>
On Apr 11, 2006, at 2:06 PM, Erik Hatcher wrote:
> On Apr 11, 2006, at 1:46 PM, miki sun wrote:
>> Is there any theory behind the similarity measure of Lucene?
>>
>> http://lucene.apache.org/java/docs/api/org/apache/lucene/search/
>> Similarity.html
>
> No, Doug just made it up with some random mathematical formulas,
> just for fun :)
>
> Erik
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: search.Similarity
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Apr 11, 2006, at 1:46 PM, miki sun wrote:
> Is there any theory behind the similarity measure of Lucene?
>
> http://lucene.apache.org/java/docs/api/org/apache/lucene/search/
> Similarity.html
No, Doug just made it up with some random mathematical formulas, just
for fun :)
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
search.Similarity
Posted by miki sun <su...@hotmail.com>.
Hi there
Is there any theory behind the similarity measure of Lucene?
http://lucene.apache.org/java/docs/api/org/apache/lucene/search/Similarity.html
Thanks
Miki
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: doc.get("contents")
Posted by miki sun <su...@hotmail.com>.
Thanks Chris
I just realize the "contents" in the index is not the "contents" in the
original document.
Miki
----Original Message Follows----
From: Chris Hostetter <ho...@fucit.org>
Reply-To: java-user@lucene.apache.org
To: java-user@lucene.apache.org
Subject: Re: doc.get("contents")
Date: Thu, 6 Apr 2006 14:58:25 -0700 (PDT)
Fields, by default, are not stored. if you look at the FileDocument.java
file in the demo, you should see that the contents field is created this
way...
// Add the contents of the file to a field named "contents". Specify a
Reader,
// so that the text of the file is tokenized and indexed, but not
stored.
// Note that FileReader expects the file to be in the system's default
encoding.
// If that's not the case searching for special characters will fail.
doc.add(new Field("contents", new FileReader(f)));
..since it is not stored, the call to doc.get("contents") returns null ...
you're not checking for it to be null, before passing it to StringReader,
so you get the null pointer exception then.
You could change the code to made contents a stored field, but that would
make you index much bigger. you could also open the orriginal file on
disk, and pass it to the analyzer for highlighting instead.
: Date: Thu, 06 Apr 2006 20:19:24 +0000
: From: miki sun <su...@hotmail.com>
: Reply-To: java-user@lucene.apache.org
: To: java-user@lucene.apache.org
: Subject: doc.get("contents")
:
: Dear all
:
: I got a java.lang.NullPointerException at
: java.io.StringReader.<init>(StringReader.java:33) error when processing
the
: following code:
:
: for (int i = 0; i < theHits.length(); i++)
: {
: Document doc = theHits.doc(i);
: String contents = doc.get("contents") ;
: TokenStream tokenStream = analyzer.tokenStream( "contents", new
: StringReader(contents)) ;
: System.out.println(theHighlighter.getBestFragment(tokenStream, contents));
: }
:
: I used demo.IndexFiles to index my directory. Who can tell me what's
wrong?
: Thanks a lot!
:
: Miki
:
:
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-user-help@lucene.apache.org
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: doc.get("contents")
Posted by Chris Hostetter <ho...@fucit.org>.
Fields, by default, are not stored. if you look at the FileDocument.java
file in the demo, you should see that the contents field is created this
way...
// Add the contents of the file to a field named "contents". Specify a Reader,
// so that the text of the file is tokenized and indexed, but not stored.
// Note that FileReader expects the file to be in the system's default encoding.
// If that's not the case searching for special characters will fail.
doc.add(new Field("contents", new FileReader(f)));
..since it is not stored, the call to doc.get("contents") returns null ...
you're not checking for it to be null, before passing it to StringReader,
so you get the null pointer exception then.
You could change the code to made contents a stored field, but that would
make you index much bigger. you could also open the orriginal file on
disk, and pass it to the analyzer for highlighting instead.
: Date: Thu, 06 Apr 2006 20:19:24 +0000
: From: miki sun <su...@hotmail.com>
: Reply-To: java-user@lucene.apache.org
: To: java-user@lucene.apache.org
: Subject: doc.get("contents")
:
: Dear all
:
: I got a java.lang.NullPointerException at
: java.io.StringReader.<init>(StringReader.java:33) error when processing the
: following code:
:
: for (int i = 0; i < theHits.length(); i++)
: {
: Document doc = theHits.doc(i);
: String contents = doc.get("contents") ;
: TokenStream tokenStream = analyzer.tokenStream( "contents", new
: StringReader(contents)) ;
: System.out.println(theHighlighter.getBestFragment(tokenStream, contents));
: }
:
: I used demo.IndexFiles to index my directory. Who can tell me what's wrong?
: Thanks a lot!
:
: Miki
:
:
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-user-help@lucene.apache.org
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
doc.get("contents")
Posted by miki sun <su...@hotmail.com>.
Dear all
I got a java.lang.NullPointerException at
java.io.StringReader.<init>(StringReader.java:33) error when processing the
following code:
for (int i = 0; i < theHits.length(); i++)
{
Document doc = theHits.doc(i);
String contents = doc.get("contents") ;
TokenStream tokenStream = analyzer.tokenStream( "contents", new
StringReader(contents)) ;
System.out.println(theHighlighter.getBestFragment(tokenStream, contents));
}
I used demo.IndexFiles to index my directory. Who can tell me what's wrong?
Thanks a lot!
Miki
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: QueryParser error + solution
Posted by miki sun <su...@hotmail.com>.
Daniel you are very clever! Your solution remind me this:
No temptation has overtaken you but such as is common to man; and God is
faithful, who will not allow you to be tempted beyond what you are able, but
with the temptation will provide the way of escape also, so that you will be
able to endure it.
1 Corinthians 10:13 (New American Standard Version)
Well done Erik!
----Original Message Follows----
From: Daniel Noll <da...@nuix.com.au>
Reply-To: java-user@lucene.apache.org
To: java-user@lucene.apache.org
Subject: Re: QueryParser error + solution
Date: Wed, 05 Apr 2006 14:26:20 +1000
miki sun wrote:
>Thanks Erik and Michael!
>
>I copied some code from demo.SearchFiles.java, I do not have a more clearer
>tracing message. Now it works.
>
>But do you have a better way than this:
>
[snip]
Something like this?
String str = "Really bad query string: lots of evil stuff!";
str = QueryParser.escape(str);
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://www.nuix.com.au/ Fax: +61 2 9212 6902
This message is intended only for the named recipient. If you are not
the intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
message or attachment is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: QueryParser error + solution
Posted by Daniel Noll <da...@nuix.com.au>.
miki sun wrote:
> Thanks Erik and Michael!
>
> I copied some code from demo.SearchFiles.java, I do not have a more
> clearer tracing message. Now it works.
>
> But do you have a better way than this:
>
[snip]
Something like this?
String str = "Really bad query string: lots of evil stuff!";
str = QueryParser.escape(str);
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://www.nuix.com.au/ Fax: +61 2 9212 6902
This message is intended only for the named recipient. If you are not
the intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
message or attachment is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org