You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Omar Didi <od...@Cyveillance.com> on 2005/04/01 00:00:46 UTC
RE: error when query contains numbers
I have used luke to look inside the index and i haven't found any numbers except for the fields that were indexed as keywords. i have used the AnalyzerDemo class in LIA, I got the following output:
Analyzing "welcome to the year 2005 yahoo123"
WhitespaceAnalyzer:
[welcome] [to] [the] [year] [2005] [yahoo123]
SimpleAnalyzer:
[welcome] [to] [the] [year] [yahoo]
StopAnalyzer:
[welcome] [year] [yahoo]
StandardAnalyzer:
[welcome] [year] [2005] [yahoo123]
CyAnalyzer:
[year] [2005] [yahoo123]
the analyzer I am using doesn t discard numbers but I can t find them. any ideas??
-----Original Message-----
From: Erik Hatcher [mailto:erik@ehatchersolutions.com]
Sent: Wednesday, March 30, 2005 8:58 PM
To: java-user@lucene.apache.org
Subject: Re: error when query contains numbers
On Mar 30, 2005, at 8:05 PM, Omar Didi wrote:
> the .toString() looks excactly like the query I enter: if I search for
> "yahoo AND 200" it returns 0 hits. I am sure there are documents that
> have 200 hundreds in them. The analyzer I am using is a custom
> analyzer that has a list of stop words. I don t know much about the
> way data was indexed, I am just developing an aplication to search
> using the analyzer that was used while indexing.
Try the tips here:
http://wiki.apache.org/jakarta-lucene/AnalysisParalysis - you need to
analyze your analyzer and ensure what you think was indexed actually
was. Also, look into using Luke - http://www.getopt.org/luke/ - to see
what makes your index tick.
> my concern now is if there is an error with the way the indexing was
> do I have to reindex the documents?
Yes. That's just the nature of how it works. Getting the analysis
right is important stuff, and if you didn't index it, you can't search
for it!
Feel free to share more details of your analyzer, and we'd be happy to
"analyze" it.
Erik
> thanks
>
> On Mar 30, 2005, at 4:41 PM, Omar Didi wrote:
>> I am using a QueryParser to search the index. when the query has
>> numbers, i don t get any results??
>> any suggestions??
>
> What is the .toString of the Query object instance returned from
> QueryParser? What Analyzer are you using? How did you index the
> field(s) being queried?
>
> Erik
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: error when query contains numbers
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Mar 31, 2005, at 5:00 PM, Omar Didi wrote:
> CyAnalyzer:
> [year] [2005] [yahoo123]
> the analyzer I am using doesn t discard numbers but I can t find them.
> any ideas??
If that is the analyzer you are using for indexing and you are using a
tokenized field type then those tokens will be indexed as you see
there.
My only theory now is that you've got something misconfigured. Back up
and index a very simple single document index to the filesystem and
check it out with Luke. Send us that indexing code if you do not see
numbers there.
Erik
>
>
> -----Original Message-----
> From: Erik Hatcher [mailto:erik@ehatchersolutions.com]
> Sent: Wednesday, March 30, 2005 8:58 PM
> To: java-user@lucene.apache.org
> Subject: Re: error when query contains numbers
>
>
>
> On Mar 30, 2005, at 8:05 PM, Omar Didi wrote:
>> the .toString() looks excactly like the query I enter: if I search for
>> "yahoo AND 200" it returns 0 hits. I am sure there are documents that
>> have 200 hundreds in them. The analyzer I am using is a custom
>> analyzer that has a list of stop words. I don t know much about the
>> way data was indexed, I am just developing an aplication to search
>> using the analyzer that was used while indexing.
>
> Try the tips here:
> http://wiki.apache.org/jakarta-lucene/AnalysisParalysis - you need to
> analyze your analyzer and ensure what you think was indexed actually
> was. Also, look into using Luke - http://www.getopt.org/luke/ - to see
> what makes your index tick.
>
>> my concern now is if there is an error with the way the indexing was
>> do I have to reindex the documents?
>
> Yes. That's just the nature of how it works. Getting the analysis
> right is important stuff, and if you didn't index it, you can't search
> for it!
>
> Feel free to share more details of your analyzer, and we'd be happy to
> "analyze" it.
>
> Erik
>
>
>> thanks
>>
>> On Mar 30, 2005, at 4:41 PM, Omar Didi wrote:
>>> I am using a QueryParser to search the index. when the query has
>>> numbers, i don t get any results??
>>> any suggestions??
>>
>> What is the .toString of the Query object instance returned from
>> QueryParser? What Analyzer are you using? How did you index the
>> field(s) being queried?
>>
>> Erik
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org