You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by dabboo <ag...@sapient.com> on 2009/04/03 09:41:09 UTC
Phrase Query Issue
Hi,
I am implementing phrase search query using my application. I am trying to
search "Glorious Revolution", I am expecting all the records which contains
this as whole string in any of its field should display.
Below is my query:
?q=fieldname:"Glorious Revolution"&qt=dismaxrequest
I am passing this to my solr search engine using httpclient. But it is
thrwoing me Invaild Query exception. If I append double quotes with
backslash, it passes the query but it doesnt shows the right values.
Instead it creates the query like:
isbn13_product_s:\"Glorious Revolution\"
Please suggest how I should tackle this. If I give the phrase search in my
solr console, it returns me the correct results.
Thanks,
Amit Garg
--
View this message in context: http://www.nabble.com/Phrase-Query-Issue-tp22863529p22863529.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase Query Issue
Posted by Erick Erickson <er...@gmail.com>.
Well, I don't think this is done normally. I'm assuming here that
you've simplified the example and your *real* requirements aren't
quite what you're showing, because what you've described seems
really confusing to a user.
That said, I don't think SOLR has a way to handle this out-of-the-box.
What you're asking for is for the query to search different fields based
on the *results* of the parsing. You have to deal with someone searching
for "tests" OR "run", which presumably should return anything
with "test", "tests", "testing", "run". But should NOT return a doc
with only "running" in the title. But to the parser, this starts out as
title:(tests run)....
So, you really need to see the results of feeding any term through
the stemmer. If it comes back the same as what you fed in, go
against one field. If it comes back stemmed, go against a
different field.
You'll probably have to create a custom parser that pre-processes
each term and determines which field should be searched, search
the SOLR list as creating custom parsers has been discussed
before by people more knowledgeable than me...
If my assumption that there are parts of this problem that you've
simplified is NOT correct, perhaps you could explain the use-case
that leads to this requirement....
But if anyone has a better idea, now's the time to speak up <G>.
Best
Erick
On Tue, Dec 1, 2009 at 12:13 AM, ravicv <ra...@gmail.com> wrote:
>
> Thanks Erick But their any way to get phrase search and normal search
> together.
>
> for Example if I search for "test" it should only give test result but not
> tests..
> And if we search for tests it should give all results matching with
> tests(this should include stemmer results also)
>
> Please clarify me how normally this scenario is implemented?
>
> I am using field title with text type which contains
> SnowballPorterFilterFactory filter.
>
> And another field title_exact with out SnowballPorterFilterFactory filter.
>
> Both are copied using copy field.
>
> <copyField source="title" dest="text"/>
> <copyField source="title_exact" dest="text_exact"/>
> But as text type is having a SnowballPorterFilterFactory filter it stores
> tests as test.So always if I search for "test" tests also will come in my
> search result.
> Please tell me a way to avoid it.
>
> Thanks
> Ravichandra
> --
> View this message in context:
> http://old.nabble.com/Phrase-Query-Issue-tp22863529p26586788.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: Phrase Query Issue
Posted by ravicv <ra...@gmail.com>.
Thanks Erick But their any way to get phrase search and normal search
together.
for Example if I search for "test" it should only give test result but not
tests..
And if we search for tests it should give all results matching with
tests(this should include stemmer results also)
Please clarify me how normally this scenario is implemented?
I am using field title with text type which contains
SnowballPorterFilterFactory filter.
And another field title_exact with out SnowballPorterFilterFactory filter.
Both are copied using copy field.
<copyField source="title" dest="text"/>
<copyField source="title_exact" dest="text_exact"/>
But as text type is having a SnowballPorterFilterFactory filter it stores
tests as test.So always if I search for "test" tests also will come in my
search result.
Please tell me a way to avoid it.
Thanks
Ravichandra
--
View this message in context: http://old.nabble.com/Phrase-Query-Issue-tp22863529p26586788.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase Query Issue
Posted by Erick Erickson <er...@gmail.com>.
You didn't say what field you're searching on, but if it's any
field with one of the stemmers involved, then this is expected
behavior. That is, "test" and "tests" are both stemmed to
"test". I do see SnowballPorterFilterFactory in your schema,
so this is what I'd check first....
Think carefully about removing this though. Depending on the
intent of the field users may well *want* this behavior.
Best
Erick
On Mon, Nov 30, 2009 at 6:02 AM, ravicv <ra...@gmail.com> wrote:
>
> Hi I have a problem with phrase search.
>
> If I search test then tests is also coming in search result.
>
> Please help me ...
>
> My schema continues...
> "
> <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
> < type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <!-- in this example, we will only use synonyms at query time
> <filter class="solr.SynonymFilterFactory"
> synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
> -->
>
> <filter class="solr.StopFilterFactory"
> ignoreCase="true"
> words="stopwords.txt"
> enablePositionIncrements="true"
> />
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="0" generateNumberParts="0" catenateWords="1"
> catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
> preserveOriginal="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.SnowballPorterFilterFactory" language="German"
> protected="protwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
>
> < a nalyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="false"/>
> <filter class="solr.StopFilterFactory"
> ignoreCase="true"
> words="stopwords.txt"
> enablePositionIncrements="true"
> />
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="0" generateNumberParts="0" catenateWords="1"
> catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
> preserveOriginal="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.SnowballPorterFilterFactory" language="German"
> protected="protwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
>
>
> <fieldType name="text_exact" class="solr.TextField"
> positionIncrementGap="100">
> <ana lyzer type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <!-- in this example, we will only use synonyms at query time
> <filter class="solr.SynonymFilterFactory"
> synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
> -->
> <!-- Case insensitive stop word removal.
> add enablePositionIncrements=true in both the index and query
> analyzers to leave a 'gap' for more accurate phrase queries.
> -->
> <filter class="solr.StopFilterFactory"
> ignoreCase="true"
> words="stopwords.txt"
> enablePositionIncrements="true"
> />
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="0" generateNumberParts="0" catenateWords="1"
> catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
> preserveOriginal="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyz er>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="false"/>
> <filter class="solr.StopFilterFactory"
> ignoreCase="true"
> words="stopwords.txt"
> enablePositionIncrements="true"
> />
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="0" generateNumberParts="0" catenateWords="1"
> catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
> preserveOriginal="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analy zer>
> </fieldType>"
>
> and copy field
>
> <copyField source="title_de" dest="text"/>
> <copyField source="title_de" dest="text_exact"/>
>
> Please help me ...
>
>
>
>
> Otis Gospodnetic wrote:
> >
> >
> > Let me second this. People ask for this pretty often.
> >
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> >
> >
> > ----- Original Message ----
> >> From: Erik Hatcher <er...@ehatchersolutions.com>
> >> To: solr-user@lucene.apache.org
> >> Sent: Saturday, April 4, 2009 8:33:46 PM
> >> Subject: Re: Phrase Query Issue
> >>
> >>
> >> On Apr 4, 2009, at 1:25 AM, dabboo wrote:
> >>
> >> >
> >> > Erik,
> >> >
> >> > Thanks a lot for your reply. I have made some changes in the solr code
> >> and
> >> > now field clauses are working fine with dismax request. Not only this,
> >> > wildcard characters are also working with dismax and q query
> parameter.
> >> >
> >> > If you want I can share modified code with you.
> >>
> >> That'd be good to share. Simply open a Solr JIRA issue with this
> >> enhancement
> >> request and post your code there. Test cases and documentation always
> >> appreciated too, but working code to start with is fine.
> >>
> >> Erik
> >
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Phrase-Query-Issue-tp22863529p26572797.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: Phrase Query Issue
Posted by ravicv <ra...@gmail.com>.
Hi I have a problem with phrase search.
If I search test then tests is also coming in search result.
Please help me ...
My schema continues...
"
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
< type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<!-- in this example, we will only use synonyms at query time
<filter class="solr.SynonymFilterFactory"
synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
-->
<filter class="solr.StopFilterFactory"
ignoreCase="true"
words="stopwords.txt"
enablePositionIncrements="true"
/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="0" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SnowballPorterFilterFactory" language="German"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
< a nalyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="false"/>
<filter class="solr.StopFilterFactory"
ignoreCase="true"
words="stopwords.txt"
enablePositionIncrements="true"
/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="0" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SnowballPorterFilterFactory" language="German"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
<fieldType name="text_exact" class="solr.TextField"
positionIncrementGap="100">
<ana lyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<!-- in this example, we will only use synonyms at query time
<filter class="solr.SynonymFilterFactory"
synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
-->
<!-- Case insensitive stop word removal.
add enablePositionIncrements=true in both the index and query
analyzers to leave a 'gap' for more accurate phrase queries.
-->
<filter class="solr.StopFilterFactory"
ignoreCase="true"
words="stopwords.txt"
enablePositionIncrements="true"
/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="0" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyz er>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="false"/>
<filter class="solr.StopFilterFactory"
ignoreCase="true"
words="stopwords.txt"
enablePositionIncrements="true"
/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="0" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"
preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analy zer>
</fieldType>"
and copy field
<copyField source="title_de" dest="text"/>
<copyField source="title_de" dest="text_exact"/>
Please help me ...
Otis Gospodnetic wrote:
>
>
> Let me second this. People ask for this pretty often.
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> ----- Original Message ----
>> From: Erik Hatcher <er...@ehatchersolutions.com>
>> To: solr-user@lucene.apache.org
>> Sent: Saturday, April 4, 2009 8:33:46 PM
>> Subject: Re: Phrase Query Issue
>>
>>
>> On Apr 4, 2009, at 1:25 AM, dabboo wrote:
>>
>> >
>> > Erik,
>> >
>> > Thanks a lot for your reply. I have made some changes in the solr code
>> and
>> > now field clauses are working fine with dismax request. Not only this,
>> > wildcard characters are also working with dismax and q query parameter.
>> >
>> > If you want I can share modified code with you.
>>
>> That'd be good to share. Simply open a Solr JIRA issue with this
>> enhancement
>> request and post your code there. Test cases and documentation always
>> appreciated too, but working code to start with is fine.
>>
>> Erik
>
>
>
--
View this message in context: http://old.nabble.com/Phrase-Query-Issue-tp22863529p26572797.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase Query Issue
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Let me second this. People ask for this pretty often.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
> From: Erik Hatcher <er...@ehatchersolutions.com>
> To: solr-user@lucene.apache.org
> Sent: Saturday, April 4, 2009 8:33:46 PM
> Subject: Re: Phrase Query Issue
>
>
> On Apr 4, 2009, at 1:25 AM, dabboo wrote:
>
> >
> > Erik,
> >
> > Thanks a lot for your reply. I have made some changes in the solr code and
> > now field clauses are working fine with dismax request. Not only this,
> > wildcard characters are also working with dismax and q query parameter.
> >
> > If you want I can share modified code with you.
>
> That'd be good to share. Simply open a Solr JIRA issue with this enhancement
> request and post your code there. Test cases and documentation always
> appreciated too, but working code to start with is fine.
>
> Erik
Re: Phrase Query Issue
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Apr 4, 2009, at 1:25 AM, dabboo wrote:
>
> Erik,
>
> Thanks a lot for your reply. I have made some changes in the solr
> code and
> now field clauses are working fine with dismax request. Not only this,
> wildcard characters are also working with dismax and q query
> parameter.
>
> If you want I can share modified code with you.
That'd be good to share. Simply open a Solr JIRA issue with this
enhancement request and post your code there. Test cases and
documentation always appreciated too, but working code to start with
is fine.
Erik
Re: Phrase Query Issue
Posted by dabboo <ag...@sapient.com>.
Erik,
Thanks a lot for your reply. I have made some changes in the solr code and
now field clauses are working fine with dismax request. Not only this,
wildcard characters are also working with dismax and q query parameter.
If you want I can share modified code with you.
Thanks,
Amit Garg
Erik Hatcher wrote:
>
> Also note that the dismax parser does NOT support fielded clauses, so
> fieldname:"some phrase" is not parsed as you might expect with dismax.
>
> Erik
>
> On Apr 3, 2009, at 3:41 AM, dabboo wrote:
>
>>
>> Hi,
>>
>> I am implementing phrase search query using my application. I am
>> trying to
>> search "Glorious Revolution", I am expecting all the records which
>> contains
>> this as whole string in any of its field should display.
>>
>> Below is my query:
>>
>> ?q=fieldname:"Glorious Revolution"&qt=dismaxrequest
>>
>> I am passing this to my solr search engine using httpclient. But it is
>> thrwoing me Invaild Query exception. If I append double quotes with
>> backslash, it passes the query but it doesnt shows the right values.
>>
>> Instead it creates the query like:
>>
>> isbn13_product_s:\"Glorious Revolution\"
>>
>> Please suggest how I should tackle this. If I give the phrase search
>> in my
>> solr console, it returns me the correct results.
>>
>> Thanks,
>> Amit Garg
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Phrase-Query-Issue-tp22863529p22863529.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
--
View this message in context: http://www.nabble.com/Phrase-Query-Issue-tp22863529p22880202.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase Query Issue
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Also note that the dismax parser does NOT support fielded clauses, so
fieldname:"some phrase" is not parsed as you might expect with dismax.
Erik
On Apr 3, 2009, at 3:41 AM, dabboo wrote:
>
> Hi,
>
> I am implementing phrase search query using my application. I am
> trying to
> search "Glorious Revolution", I am expecting all the records which
> contains
> this as whole string in any of its field should display.
>
> Below is my query:
>
> ?q=fieldname:"Glorious Revolution"&qt=dismaxrequest
>
> I am passing this to my solr search engine using httpclient. But it is
> thrwoing me Invaild Query exception. If I append double quotes with
> backslash, it passes the query but it doesnt shows the right values.
>
> Instead it creates the query like:
>
> isbn13_product_s:\"Glorious Revolution\"
>
> Please suggest how I should tackle this. If I give the phrase search
> in my
> solr console, it returns me the correct results.
>
> Thanks,
> Amit Garg
>
>
> --
> View this message in context: http://www.nabble.com/Phrase-Query-Issue-tp22863529p22863529.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Phrase Query Issue
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Fri, Apr 3, 2009 at 1:11 PM, dabboo <ag...@sapient.com> wrote:
>
> I am passing this to my solr search engine using httpclient. But it is
> thrwoing me Invaild Query exception.
>
I suggest that you use the Solrj client. It will save a you a lot of effort.
http://wiki.apache.org/solr/Solrj
--
Regards,
Shalin Shekhar Mangar.
Re: Phrase Query Issue
Posted by Nicolas <ni...@aidel.com>.
Maybe you can have to escape quotes with the % folowed by the exa code
of the quote like is done by the php urlencode fonction.
"Returns a string in which all non-alphanumeric characters except /-_./
have been replaced with a percent (/%/) sign followed by two hex digits
and spaces encoded as plus (/+/) signs."
Nicolas.
dabboo a écrit :
> Hi,
>
> I am implementing phrase search query using my application. I am trying to
> search "Glorious Revolution", I am expecting all the records which contains
> this as whole string in any of its field should display.
>
> Below is my query:
>
> ?q=fieldname:"Glorious Revolution"&qt=dismaxrequest
>
> I am passing this to my solr search engine using httpclient. But it is
> thrwoing me Invaild Query exception. If I append double quotes with
> backslash, it passes the query but it doesnt shows the right values.
>
> Instead it creates the query like:
>
> isbn13_product_s:\"Glorious Revolution\"
>
> Please suggest how I should tackle this. If I give the phrase search in my
> solr console, it returns me the correct results.
>
> Thanks,
> Amit Garg
>
>
>