Is comma a special character in Lucene query parsing.

[Umer Khalid Qureshi <om...@gmail.com>]

Hi,

I have indexed text using standard analyzer. Say, there are two words, *
10000* and *10,000*.
When i am searching, i am parsing the text as following.

QueryParser queryParser = new
QueryParser(Lucene.Net.Util.Version.LUCENE_29,"Contents", new
Lucene.Net.Analysis.Standard.StandardAnalyzer(new string[] { "" }));
queryParser.SetMultiTermRewriteMethod(MultiTermQuery.SCORING_BOOLEAN_QUERY_REWRITE);
Query questionQuery = queryParser.Parse("10000");
         //AND
questionQuery = queryParser.Parse("10,000");

Now, what i want is that while searching it should be ignoring '*,*' in the
search term. Apparently, my code doesn't support that. Can any one tell me
what i don't know or missing???
one more thing, It doesn't matter whether i search *Nov 23, 2010*  or *Nov
23 2010*. The results are same but not in case when '*,*' appeared  between
digits.


Regards,
Umer.

Re: Is comma a special character in Lucene query parsing.

[digy digy <di...@gmail.com>]

Again same answer.

http://mail-archives.apache.org/mod_mbox/lucene-lucene-net-user/201011.mbox/%3CAANLkTim6kyuZHwb8P7g=Hvqx6DY1FkARChro0hyw+uFt@mail.gmail.com%3E

Use the code and see what StandardAnalyzer does with your input string.

DIGY

On Tue, Nov 23, 2010 at 11:34 AM, Umer Khalid Qureshi <omer.khalid@gmail.com
> wrote:

> Hi,
>
> I have indexed text using standard analyzer. Say, there are two words, *
> 10000* and *10,000*.
> When i am searching, i am parsing the text as following.
>
> QueryParser queryParser = new
> QueryParser(Lucene.Net.Util.Version.LUCENE_29,"Contents", new
> Lucene.Net.Analysis.Standard.StandardAnalyzer(new string[] { "" }));
>
> queryParser.SetMultiTermRewriteMethod(MultiTermQuery.SCORING_BOOLEAN_QUERY_REWRITE);
> Query questionQuery = queryParser.Parse("10000");
>         //AND
> questionQuery = queryParser.Parse("10,000");
>
> Now, what i want is that while searching it should be ignoring '*,*' in the
> search term. Apparently, my code doesn't support that. Can any one tell me
> what i don't know or missing???
> one more thing, It doesn't matter whether i search *Nov 23, 2010*  or *Nov
> 23 2010*. The results are same but not in case when '*,*' appeared  between
> digits.
>
>
> Regards,
> Umer.
>

RE: Is comma a special character in Lucene query parsing.

[Andrew Busby <an...@aimstrategic.com>]

Hi Umer,

I would think this is more to do with white space than commas.

Looking at the data example you provided the StandardAnaylzer will break
this into tokens on the white space, so the tokenized strings are 
Nov | 23 |,| 2010
Nov | 23, | 2010

(I actually think the lone comma is removed).  So it would be the Nov and
the 2010 that are being matched on the 23, or 23

As your 10000 and 10,000 are string values with no white space they are
treated as different search term.

This is a guess but it might help fire some thoughts in your mind.

Andrew

-----Original Message-----
From: Umer Khalid Qureshi [mailto:omer.khalid@gmail.com] 
Sent: 23 November 2010 09:35
To: lucene-net-user@lucene.apache.org
Subject: Is comma a special character in Lucene query parsing.

Hi,

I have indexed text using standard analyzer. Say, there are two words, *
10000* and *10,000*.
When i am searching, i am parsing the text as following.

QueryParser queryParser = new
QueryParser(Lucene.Net.Util.Version.LUCENE_29,"Contents", new
Lucene.Net.Analysis.Standard.StandardAnalyzer(new string[] { "" }));
queryParser.SetMultiTermRewriteMethod(MultiTermQuery.SCORING_BOOLEAN_QUERY_R
EWRITE);
Query questionQuery = queryParser.Parse("10000");
         //AND
questionQuery = queryParser.Parse("10,000");

Now, what i want is that while searching it should be ignoring '*,*' in the
search term. Apparently, my code doesn't support that. Can any one tell me
what i don't know or missing???
one more thing, It doesn't matter whether i search *Nov 23, 2010*  or *Nov
23 2010*. The results are same but not in case when '*,*' appeared  between
digits.


Regards,
Umer.