You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Carlos Alexandro Becker <ca...@gmail.com> on 2013/03/04 20:16:02 UTC
requestHandler ignoring synonyms
I have a request handler like this:
<requestHandler name="/defaultSearch" class="solr.SearchHandler">
<!-- Adds filters to all queries-->
<lst name="appends">
<str name="fq">-DELETED:true</str>
</lst>
<lst name="defaults">
<!-- See http://wiki.apache.org/solr/DisMaxQParserPlugin -->
<str name="defType">edismax</str>
<str name="echoParams">explicit</str>
<str name="df">text</str>
<!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
<float name="tie">1.00</float>
<!-- Query Fields - Search fileds with its height -->
<str name="qf">
DocumentDisplayName^10 DocumentName^9
DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
</str>
<!-- Phrase Field - frases fields used on quer as phrase >> "" -->
<str name="pf">
DocumentDisplayName^10 DocumentName^9
DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
</str>
<!-- See http://wiki.apache.org/solr/DisMaxQParserPlugin -->
<str name="mm">
2<-1 5<-2 6<90%
</str>
<!-- Phrase whitespace ignore -->
<int name="ps">10</int>
</lst>
</requestHandler>
but the synonyms and stopwords are ignored. If I use the default /select
query it works just as expected.
What can be wrong?
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Carlos Alexandro Becker <ca...@gmail.com>.
OK, thanks. I will do this and try to make this thing work.
Thank you very much for your help.
On Tue, Mar 5, 2013 at 11:34 AM, Jack Krupansky <ja...@basetechnology.com>wrote:
> Follow the advice you've already been given: 1) switch from the keyword
> tokenizer to the standard tokenizer, 2) get rid of regex replace (for now),
> and otherwise simplify your analyzers as much as possible. Then run a test
> with a simple, consistent example, and review the debugQuery and parsed
> query output to see what actually happens. And focus on using the Solr
> Admin UI Analysis page to see how your analyzers are really behaving.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Carlos Alexandro Becker
> Sent: Tuesday, March 05, 2013 8:54 AM
>
> To: solr-user
> Subject: Re: requestHandler ignoring synonyms
>
> Hi Jack,
>
> Thanks for your answer, and yes, I'm pretty confused.
> The thing is: This problem is going on in one of my job applications, and I
> must fix it.
>
> Can you give me some tips or links that I should read to clear my mind and
> understand it?
>
> Thanks in advance.
>
>
> On Tue, Mar 5, 2013 at 10:48 AM, Jack Krupansky <ja...@basetechnology.com>*
> *wrote:
>
> You're getting the 400 error because you are using the keyword tokenizer
>> which means that there will be lots of "terms" (really just raw strings"
>> that begin with "java". That simply isn't going to work. Stick with the
>> standard tokenizer.
>>
>> You have way too much going on here that is clearly way beyond your
>> current skill level. You need to back way off, way, way off, and focus on
>> doing simple things, getting simple things to work and then build
>> incrementally.
>>
>> Get rid of the reg ex pattern replace filter as well. Maybe you will
>> ultimately need it, but at this stage of your confusion it only adds to
>> your confusion.
>>
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Carlos Alexandro Becker
>> Sent: Tuesday, March 05, 2013 8:33 AM
>>
>> To: solr-user
>> Subject: Re: requestHandler ignoring synonyms
>>
>> Thanks for your answer Erik!
>>
>> I changed the FieldType to:
>> https://gist.github.com/****caarlos0/89b7c0484b154550bc63<https://gist.github.com/**caarlos0/89b7c0484b154550bc63>
>> <**https://gist.github.com/**caarlos0/89b7c0484b154550bc63<https://gist.github.com/caarlos0/89b7c0484b154550bc63>
>> >
>>
>>
>> And got a 400 error with message "analyzer returned too many terms for
>> multiTerm term: java".
>>
>> I also tried to change the query to do not use wildcard, but it still
>> ignoring the synonyms..
>>
>> Any idea?
>>
>> Thanks
>>
>>
>> On Tue, Mar 5, 2013 at 10:18 AM, Erik Hatcher <erik.hatcher@gmail.com
>> >wrote:
>>
>> In that last example you're doing a wildcard query (java*), and by
>> default
>>
>>> that does not run (all of) the analysis chain you have defined.
>>>
>>> If you need to expand synonyms for wildcarded terms like this, you'll
>>> need
>>> to define a "multiterm" analysis chain. See here for more details: <
>>> http://wiki.apache.org/solr/****MultitermQueryAnalysis<http://wiki.apache.org/solr/**MultitermQueryAnalysis>
>>> <http://**wiki.apache.org/solr/**MultitermQueryAnalysis<http://wiki.apache.org/solr/MultitermQueryAnalysis>
>>> >
>>>
>>> >
>>>
>>> Erik
>>>
>>>
>>> On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:
>>>
>>> > Hi Jack, I've updated the gist:
>>> > https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec>
>>> <**https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>>> >
>>>
>>> >
>>> > I'm give you the wrong browser tab result yesterday, sorry.
>>> >
>>> > The schema seems right to me. I have a field name BoosterField, with >
>>> the
>>> > synonyms and etc enabled in its fieldtype...
>>> >
>>> > can't figure out what's wrong. :S
>>> >
>>> >
>>> > Jack, thanks for your help.
>>> >
>>> >
>>> > On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <
>>> jack@basetechnology.com
>>> >wrote:
>>> >
>>> >> Where it says:
>>> >>
>>> >> "querystring":"******DocumentContent:java",
>>> >> "parsedquery":"(+((******DocumentContent:java DocumentContent:notare
>>> >> DocumentContent:jre)~2/no_******coord) () () () () ())/no_coord",
>>>
>>> >>
>>> >> That indicates that "java" was expanded to be equivalent to "java",
>>> >> "notare", or "jre".
>>> >>
>>> >> Are you sure you have documents that have "notare" or "jre" in the
>>> >> DocumentContent field?
>>> >>
>>> >> Suggestion: leave off the "DocumentContent:" field from the query >>
>>> since
>>> it
>>> >> is included in the qf parameter.
>>> >>
>>> >>
>>> >> -- Jack Krupansky
>>> >>
>>> >> -----Original Message----- From: Carlos Alexandro Becker
>>> >> Sent: Monday, March 04, 2013 4:09 PM
>>> >> To: solr-user
>>> >> Subject: Re: requestHandler ignoring synonyms
>>> >>
>>> >>
>>> >> I've done this, but I'm not a specialist, so, I see nothing >>
>>> interesting.
>>> >>
>>> >> The log is: >> https://gist.github.com/******
>>> caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec>
>>> <**https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec>
>>> >
>>>
>>> <
>>> https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec>
>>> <**https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>>> >
>>> >
>>> >>
>>> >> Thanks.
>>> >>
>>> >>
>>> >> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <
>>> jack@basetechnology.com
>>> >**
>>> >> wrote:
>>> >>
>>> >> You can simply test whether synonyms are being ignored or how they are
>>> >>> being processed by using the Solr Admin UI Analysis page. Select the
>>> field
>>> >>> and enter the text to test. It will show you exactly what the synonym
>>> >>> filter does.
>>> >>>
>>> >>> Make sure the synonym file is exactly as specified in the token >>>
>>> filter.
>>> >>>
>>> >>> If there are still problems, show us the field name, field type with
>>> >>> analyzer, and the query response with &debugQuery=true. The parsed
>>> query
>>> >>> string should have query-time synonyms expanded.
>>> >>>
>>> >>> -- Jack Krupansky
>>> >>>
>>> >>> -----Original Message----- From: Carlos Alexandro Becker
>>> >>> Sent: Monday, March 04, 2013 2:16 PM
>>> >>> To: solr-user
>>> >>> Subject: requestHandler ignoring synonyms
>>> >>>
>>> >>>
>>> >>> I have a request handler like this:
>>> >>>
>>> >>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
>>> >>> <!-- Adds filters to all queries-->
>>> >>> <lst name="appends">
>>> >>> <str name="fq">-DELETED:true</str>
>>> >>> </lst>
>>> >>>
>>> >>> <lst name="defaults">
>>> >>> <!-- See >>> http://wiki.apache.org/solr/****
>>> ****DisMaxQParserPlugin<http://wiki.apache.org/solr/******DisMaxQParserPlugin>
>>> <http:/**/wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>>> >
>>> <
>>> http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>>> <http://**wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> >
>>> >
>>> >>> <http://**wiki.apache.org/****solr/**DisMaxQParserPlugin<http://wiki.apache.org/**solr/**DisMaxQParserPlugin>
>>> <htt**p://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> >
>>> <
>>> http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>>> >
>>>
>>> >
>>> >>>> -->
>>> >>>
>>> >>> <str name="defType">edismax</str>
>>> >>> <str name="echoParams">explicit</********str>
>>>
>>> >>>
>>> >>> <str name="df">text</str>
>>> >>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
>>> >>> <float name="tie">1.00</float>
>>> >>> <!-- Query Fields - Search fileds with its height -->
>>> >>> <str name="qf">
>>> >>> DocumentDisplayName^10 DocumentName^9
>>> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>>> >>> </str>
>>> >>> <!-- Phrase Field - frases fields used on quer as phrase >> >>>
>>> "" -->
>>> >>> <str name="pf">
>>> >>> DocumentDisplayName^10 DocumentName^9
>>> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>>> >>> </str>
>>> >>> <!-- See >>> http://wiki.apache.org/solr/****
>>> ****DisMaxQParserPlugin<http://wiki.apache.org/solr/******DisMaxQParserPlugin>
>>> <http:/**/wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>>> >
>>> <
>>> http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>>> <http://**wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> >
>>> >
>>> >>> <http://**wiki.apache.org/****solr/**DisMaxQParserPlugin<http://wiki.apache.org/**solr/**DisMaxQParserPlugin>
>>> <htt**p://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> >
>>> <
>>> http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>>> >
>>>
>>> >
>>> >>>> -->
>>> >>>
>>> >>> <str name="mm">
>>> >>> 2<-1 5<-2 6<90%
>>> >>> </str>
>>> >>> <!-- Phrase whitespace ignore -->
>>> >>> <int name="ps">10</int>
>>> >>> </lst>
>>> >>> </requestHandler>
>>> >>>
>>> >>>
>>> >>> but the synonyms and stopwords are ignored. If I use the default
>>> /select
>>> >>> query it works just as expected.
>>> >>>
>>> >>> What can be wrong?
>>> >>>
>>> >>> --
>>> >>> Atenciosamente,
>>> >>> *Carlos Alexandro Becker*
>>> >>> http://caarlos0.github.com/********about<http://caarlos0.github.com/******about>
>>> <http://caarlos0.**github.com/****about<http://caarlos0.github.com/****about>
>>> ><
>>>
>>> http://caarlos0.github.com/******about<http://caarlos0.github.com/****about><
>>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about>
>>> >
>>> ><
>>> >>> http://caarlos0.github.com/******about<http://caarlos0.github.com/****about>
>>> <http://caarlos0.github.**com/**about<http://caarlos0.github.com/**about>
>>> ><
>>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>>>
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Atenciosamente,
>>> >> *Carlos Alexandro Becker*
>>> >> http://caarlos0.github.com/******about<http://caarlos0.github.com/****about>
>>> <http://caarlos0.github.**com/**about<http://caarlos0.github.com/**about>
>>> ><
>>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Atenciosamente,
>>> > *Carlos Alexandro Becker*
>>> > http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>>>
>>>
>>>
>>>
>> --
>> Atenciosamente,
>> *Carlos Alexandro Becker*
>> http://caarlos0.github.com/****about <http://caarlos0.github.com/**about><
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>>
>>
>
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Jack Krupansky <ja...@basetechnology.com>.
Follow the advice you've already been given: 1) switch from the keyword
tokenizer to the standard tokenizer, 2) get rid of regex replace (for now),
and otherwise simplify your analyzers as much as possible. Then run a test
with a simple, consistent example, and review the debugQuery and parsed
query output to see what actually happens. And focus on using the Solr Admin
UI Analysis page to see how your analyzers are really behaving.
-- Jack Krupansky
-----Original Message-----
From: Carlos Alexandro Becker
Sent: Tuesday, March 05, 2013 8:54 AM
To: solr-user
Subject: Re: requestHandler ignoring synonyms
Hi Jack,
Thanks for your answer, and yes, I'm pretty confused.
The thing is: This problem is going on in one of my job applications, and I
must fix it.
Can you give me some tips or links that I should read to clear my mind and
understand it?
Thanks in advance.
On Tue, Mar 5, 2013 at 10:48 AM, Jack Krupansky
<ja...@basetechnology.com>wrote:
> You're getting the 400 error because you are using the keyword tokenizer
> which means that there will be lots of "terms" (really just raw strings"
> that begin with "java". That simply isn't going to work. Stick with the
> standard tokenizer.
>
> You have way too much going on here that is clearly way beyond your
> current skill level. You need to back way off, way, way off, and focus on
> doing simple things, getting simple things to work and then build
> incrementally.
>
> Get rid of the reg ex pattern replace filter as well. Maybe you will
> ultimately need it, but at this stage of your confusion it only adds to
> your confusion.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Carlos Alexandro Becker
> Sent: Tuesday, March 05, 2013 8:33 AM
>
> To: solr-user
> Subject: Re: requestHandler ignoring synonyms
>
> Thanks for your answer Erik!
>
> I changed the FieldType to:
> https://gist.github.com/**caarlos0/89b7c0484b154550bc63<https://gist.github.com/caarlos0/89b7c0484b154550bc63>
>
> And got a 400 error with message "analyzer returned too many terms for
> multiTerm term: java".
>
> I also tried to change the query to do not use wildcard, but it still
> ignoring the synonyms..
>
> Any idea?
>
> Thanks
>
>
> On Tue, Mar 5, 2013 at 10:18 AM, Erik Hatcher <erik.hatcher@gmail.com
> >wrote:
>
> In that last example you're doing a wildcard query (java*), and by
> default
>> that does not run (all of) the analysis chain you have defined.
>>
>> If you need to expand synonyms for wildcarded terms like this, you'll
>> need
>> to define a "multiterm" analysis chain. See here for more details: <
>> http://wiki.apache.org/solr/**MultitermQueryAnalysis<http://wiki.apache.org/solr/MultitermQueryAnalysis>
>> >
>>
>> Erik
>>
>>
>> On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:
>>
>> > Hi Jack, I've updated the gist:
>> > https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>> >
>> > I'm give you the wrong browser tab result yesterday, sorry.
>> >
>> > The schema seems right to me. I have a field name BoosterField, with
>> > the
>> > synonyms and etc enabled in its fieldtype...
>> >
>> > can't figure out what's wrong. :S
>> >
>> >
>> > Jack, thanks for your help.
>> >
>> >
>> > On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <jack@basetechnology.com
>> >wrote:
>> >
>> >> Where it says:
>> >>
>> >> "querystring":"****DocumentContent:java",
>> >> "parsedquery":"(+((****DocumentContent:java DocumentContent:notare
>> >> DocumentContent:jre)~2/no_****coord) () () () () ())/no_coord",
>> >>
>> >> That indicates that "java" was expanded to be equivalent to "java",
>> >> "notare", or "jre".
>> >>
>> >> Are you sure you have documents that have "notare" or "jre" in the
>> >> DocumentContent field?
>> >>
>> >> Suggestion: leave off the "DocumentContent:" field from the query
>> >> since
>> it
>> >> is included in the qf parameter.
>> >>
>> >>
>> >> -- Jack Krupansky
>> >>
>> >> -----Original Message----- From: Carlos Alexandro Becker
>> >> Sent: Monday, March 04, 2013 4:09 PM
>> >> To: solr-user
>> >> Subject: Re: requestHandler ignoring synonyms
>> >>
>> >>
>> >> I've done this, but I'm not a specialist, so, I see nothing >>
>> interesting.
>> >>
>> >> The log is:
>> >> https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec>
>> <
>> https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>> >
>> >>
>> >> Thanks.
>> >>
>> >>
>> >> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <
>> jack@basetechnology.com
>> >**
>> >> wrote:
>> >>
>> >> You can simply test whether synonyms are being ignored or how they are
>> >>> being processed by using the Solr Admin UI Analysis page. Select the
>> field
>> >>> and enter the text to test. It will show you exactly what the synonym
>> >>> filter does.
>> >>>
>> >>> Make sure the synonym file is exactly as specified in the token >>>
>> filter.
>> >>>
>> >>> If there are still problems, show us the field name, field type with
>> >>> analyzer, and the query response with &debugQuery=true. The parsed
>> query
>> >>> string should have query-time synonyms expanded.
>> >>>
>> >>> -- Jack Krupansky
>> >>>
>> >>> -----Original Message----- From: Carlos Alexandro Becker
>> >>> Sent: Monday, March 04, 2013 2:16 PM
>> >>> To: solr-user
>> >>> Subject: requestHandler ignoring synonyms
>> >>>
>> >>>
>> >>> I have a request handler like this:
>> >>>
>> >>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
>> >>> <!-- Adds filters to all queries-->
>> >>> <lst name="appends">
>> >>> <str name="fq">-DELETED:true</str>
>> >>> </lst>
>> >>>
>> >>> <lst name="defaults">
>> >>> <!-- See
>> >>> http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> >
>> >>> <http://**wiki.apache.org/**solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>> >
>> >>>> -->
>> >>>
>> >>> <str name="defType">edismax</str>
>> >>> <str name="echoParams">explicit</******str>
>> >>>
>> >>> <str name="df">text</str>
>> >>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
>> >>> <float name="tie">1.00</float>
>> >>> <!-- Query Fields - Search fileds with its height -->
>> >>> <str name="qf">
>> >>> DocumentDisplayName^10 DocumentName^9
>> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>> >>> </str>
>> >>> <!-- Phrase Field - frases fields used on quer as phrase >>
>> >>> "" -->
>> >>> <str name="pf">
>> >>> DocumentDisplayName^10 DocumentName^9
>> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>> >>> </str>
>> >>> <!-- See
>> >>> http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> >
>> >>> <http://**wiki.apache.org/**solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>> >
>> >>>> -->
>> >>>
>> >>> <str name="mm">
>> >>> 2<-1 5<-2 6<90%
>> >>> </str>
>> >>> <!-- Phrase whitespace ignore -->
>> >>> <int name="ps">10</int>
>> >>> </lst>
>> >>> </requestHandler>
>> >>>
>> >>>
>> >>> but the synonyms and stopwords are ignored. If I use the default
>> /select
>> >>> query it works just as expected.
>> >>>
>> >>> What can be wrong?
>> >>>
>> >>> --
>> >>> Atenciosamente,
>> >>> *Carlos Alexandro Becker*
>> >>> http://caarlos0.github.com/******about<http://caarlos0.github.com/****about><
>> http://caarlos0.github.com/****about <http://caarlos0.github.com/**about>
>> ><
>> >>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Atenciosamente,
>> >> *Carlos Alexandro Becker*
>> >> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>> >>
>> >
>> >
>> >
>> > --
>> > Atenciosamente,
>> > *Carlos Alexandro Becker*
>> > http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>>
>>
>>
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Carlos Alexandro Becker <ca...@gmail.com>.
Hi Jack,
Thanks for your answer, and yes, I'm pretty confused.
The thing is: This problem is going on in one of my job applications, and I
must fix it.
Can you give me some tips or links that I should read to clear my mind and
understand it?
Thanks in advance.
On Tue, Mar 5, 2013 at 10:48 AM, Jack Krupansky <ja...@basetechnology.com>wrote:
> You're getting the 400 error because you are using the keyword tokenizer
> which means that there will be lots of "terms" (really just raw strings"
> that begin with "java". That simply isn't going to work. Stick with the
> standard tokenizer.
>
> You have way too much going on here that is clearly way beyond your
> current skill level. You need to back way off, way, way off, and focus on
> doing simple things, getting simple things to work and then build
> incrementally.
>
> Get rid of the reg ex pattern replace filter as well. Maybe you will
> ultimately need it, but at this stage of your confusion it only adds to
> your confusion.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Carlos Alexandro Becker
> Sent: Tuesday, March 05, 2013 8:33 AM
>
> To: solr-user
> Subject: Re: requestHandler ignoring synonyms
>
> Thanks for your answer Erik!
>
> I changed the FieldType to:
> https://gist.github.com/**caarlos0/89b7c0484b154550bc63<https://gist.github.com/caarlos0/89b7c0484b154550bc63>
>
> And got a 400 error with message "analyzer returned too many terms for
> multiTerm term: java".
>
> I also tried to change the query to do not use wildcard, but it still
> ignoring the synonyms..
>
> Any idea?
>
> Thanks
>
>
> On Tue, Mar 5, 2013 at 10:18 AM, Erik Hatcher <erik.hatcher@gmail.com
> >wrote:
>
> In that last example you're doing a wildcard query (java*), and by default
>> that does not run (all of) the analysis chain you have defined.
>>
>> If you need to expand synonyms for wildcarded terms like this, you'll need
>> to define a "multiterm" analysis chain. See here for more details: <
>> http://wiki.apache.org/solr/**MultitermQueryAnalysis<http://wiki.apache.org/solr/MultitermQueryAnalysis>
>> >
>>
>> Erik
>>
>>
>> On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:
>>
>> > Hi Jack, I've updated the gist:
>> > https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>> >
>> > I'm give you the wrong browser tab result yesterday, sorry.
>> >
>> > The schema seems right to me. I have a field name BoosterField, with the
>> > synonyms and etc enabled in its fieldtype...
>> >
>> > can't figure out what's wrong. :S
>> >
>> >
>> > Jack, thanks for your help.
>> >
>> >
>> > On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <jack@basetechnology.com
>> >wrote:
>> >
>> >> Where it says:
>> >>
>> >> "querystring":"****DocumentContent:java",
>> >> "parsedquery":"(+((****DocumentContent:java DocumentContent:notare
>> >> DocumentContent:jre)~2/no_****coord) () () () () ())/no_coord",
>> >>
>> >> That indicates that "java" was expanded to be equivalent to "java",
>> >> "notare", or "jre".
>> >>
>> >> Are you sure you have documents that have "notare" or "jre" in the
>> >> DocumentContent field?
>> >>
>> >> Suggestion: leave off the "DocumentContent:" field from the query since
>> it
>> >> is included in the qf parameter.
>> >>
>> >>
>> >> -- Jack Krupansky
>> >>
>> >> -----Original Message----- From: Carlos Alexandro Becker
>> >> Sent: Monday, March 04, 2013 4:09 PM
>> >> To: solr-user
>> >> Subject: Re: requestHandler ignoring synonyms
>> >>
>> >>
>> >> I've done this, but I'm not a specialist, so, I see nothing >>
>> interesting.
>> >>
>> >> The log is: https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec>
>> <
>> https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>> >
>> >>
>> >> Thanks.
>> >>
>> >>
>> >> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <
>> jack@basetechnology.com
>> >**
>> >> wrote:
>> >>
>> >> You can simply test whether synonyms are being ignored or how they are
>> >>> being processed by using the Solr Admin UI Analysis page. Select the
>> field
>> >>> and enter the text to test. It will show you exactly what the synonym
>> >>> filter does.
>> >>>
>> >>> Make sure the synonym file is exactly as specified in the token >>>
>> filter.
>> >>>
>> >>> If there are still problems, show us the field name, field type with
>> >>> analyzer, and the query response with &debugQuery=true. The parsed
>> query
>> >>> string should have query-time synonyms expanded.
>> >>>
>> >>> -- Jack Krupansky
>> >>>
>> >>> -----Original Message----- From: Carlos Alexandro Becker
>> >>> Sent: Monday, March 04, 2013 2:16 PM
>> >>> To: solr-user
>> >>> Subject: requestHandler ignoring synonyms
>> >>>
>> >>>
>> >>> I have a request handler like this:
>> >>>
>> >>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
>> >>> <!-- Adds filters to all queries-->
>> >>> <lst name="appends">
>> >>> <str name="fq">-DELETED:true</str>
>> >>> </lst>
>> >>>
>> >>> <lst name="defaults">
>> >>> <!-- See http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> >
>> >>> <http://**wiki.apache.org/**solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>> >
>> >>>> -->
>> >>>
>> >>> <str name="defType">edismax</str>
>> >>> <str name="echoParams">explicit</******str>
>> >>>
>> >>> <str name="df">text</str>
>> >>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
>> >>> <float name="tie">1.00</float>
>> >>> <!-- Query Fields - Search fileds with its height -->
>> >>> <str name="qf">
>> >>> DocumentDisplayName^10 DocumentName^9
>> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>> >>> </str>
>> >>> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
>> >>> <str name="pf">
>> >>> DocumentDisplayName^10 DocumentName^9
>> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>> >>> </str>
>> >>> <!-- See http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> >
>> >>> <http://**wiki.apache.org/**solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> <
>> http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>> >
>> >>>> -->
>> >>>
>> >>> <str name="mm">
>> >>> 2<-1 5<-2 6<90%
>> >>> </str>
>> >>> <!-- Phrase whitespace ignore -->
>> >>> <int name="ps">10</int>
>> >>> </lst>
>> >>> </requestHandler>
>> >>>
>> >>>
>> >>> but the synonyms and stopwords are ignored. If I use the default
>> /select
>> >>> query it works just as expected.
>> >>>
>> >>> What can be wrong?
>> >>>
>> >>> --
>> >>> Atenciosamente,
>> >>> *Carlos Alexandro Becker*
>> >>> http://caarlos0.github.com/******about<http://caarlos0.github.com/****about><
>> http://caarlos0.github.com/****about <http://caarlos0.github.com/**about>
>> ><
>> >>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Atenciosamente,
>> >> *Carlos Alexandro Becker*
>> >> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>> >>
>> >
>> >
>> >
>> > --
>> > Atenciosamente,
>> > *Carlos Alexandro Becker*
>> > http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>>
>>
>>
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Jack Krupansky <ja...@basetechnology.com>.
You're getting the 400 error because you are using the keyword tokenizer
which means that there will be lots of "terms" (really just raw strings"
that begin with "java". That simply isn't going to work. Stick with the
standard tokenizer.
You have way too much going on here that is clearly way beyond your current
skill level. You need to back way off, way, way off, and focus on doing
simple things, getting simple things to work and then build incrementally.
Get rid of the reg ex pattern replace filter as well. Maybe you will
ultimately need it, but at this stage of your confusion it only adds to your
confusion.
-- Jack Krupansky
-----Original Message-----
From: Carlos Alexandro Becker
Sent: Tuesday, March 05, 2013 8:33 AM
To: solr-user
Subject: Re: requestHandler ignoring synonyms
Thanks for your answer Erik!
I changed the FieldType to:
https://gist.github.com/caarlos0/89b7c0484b154550bc63
And got a 400 error with message "analyzer returned too many terms for
multiTerm term: java".
I also tried to change the query to do not use wildcard, but it still
ignoring the synonyms..
Any idea?
Thanks
On Tue, Mar 5, 2013 at 10:18 AM, Erik Hatcher <er...@gmail.com>wrote:
> In that last example you're doing a wildcard query (java*), and by default
> that does not run (all of) the analysis chain you have defined.
>
> If you need to expand synonyms for wildcarded terms like this, you'll need
> to define a "multiterm" analysis chain. See here for more details: <
> http://wiki.apache.org/solr/MultitermQueryAnalysis>
>
> Erik
>
>
> On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:
>
> > Hi Jack, I've updated the gist:
> > https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec
> >
> > I'm give you the wrong browser tab result yesterday, sorry.
> >
> > The schema seems right to me. I have a field name BoosterField, with the
> > synonyms and etc enabled in its fieldtype...
> >
> > can't figure out what's wrong. :S
> >
> >
> > Jack, thanks for your help.
> >
> >
> > On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <jack@basetechnology.com
> >wrote:
> >
> >> Where it says:
> >>
> >> "querystring":"**DocumentContent:java",
> >> "parsedquery":"(+((**DocumentContent:java DocumentContent:notare
> >> DocumentContent:jre)~2/no_**coord) () () () () ())/no_coord",
> >>
> >> That indicates that "java" was expanded to be equivalent to "java",
> >> "notare", or "jre".
> >>
> >> Are you sure you have documents that have "notare" or "jre" in the
> >> DocumentContent field?
> >>
> >> Suggestion: leave off the "DocumentContent:" field from the query since
> it
> >> is included in the qf parameter.
> >>
> >>
> >> -- Jack Krupansky
> >>
> >> -----Original Message----- From: Carlos Alexandro Becker
> >> Sent: Monday, March 04, 2013 4:09 PM
> >> To: solr-user
> >> Subject: Re: requestHandler ignoring synonyms
> >>
> >>
> >> I've done this, but I'm not a specialist, so, I see nothing
> >> interesting.
> >>
> >> The log is: https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<
> https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
> >>
> >> Thanks.
> >>
> >>
> >> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <jack@basetechnology.com
> >**
> >> wrote:
> >>
> >> You can simply test whether synonyms are being ignored or how they are
> >>> being processed by using the Solr Admin UI Analysis page. Select the
> field
> >>> and enter the text to test. It will show you exactly what the synonym
> >>> filter does.
> >>>
> >>> Make sure the synonym file is exactly as specified in the token
> >>> filter.
> >>>
> >>> If there are still problems, show us the field name, field type with
> >>> analyzer, and the query response with &debugQuery=true. The parsed
> query
> >>> string should have query-time synonyms expanded.
> >>>
> >>> -- Jack Krupansky
> >>>
> >>> -----Original Message----- From: Carlos Alexandro Becker
> >>> Sent: Monday, March 04, 2013 2:16 PM
> >>> To: solr-user
> >>> Subject: requestHandler ignoring synonyms
> >>>
> >>>
> >>> I have a request handler like this:
> >>>
> >>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
> >>> <!-- Adds filters to all queries-->
> >>> <lst name="appends">
> >>> <str name="fq">-DELETED:true</str>
> >>> </lst>
> >>>
> >>> <lst name="defaults">
> >>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<
> http://wiki.apache.org/solr/**DisMaxQParserPlugin>
> >>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<
> http://wiki.apache.org/solr/DisMaxQParserPlugin>
> >>>> -->
> >>>
> >>> <str name="defType">edismax</str>
> >>> <str name="echoParams">explicit</****str>
> >>>
> >>> <str name="df">text</str>
> >>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
> >>> <float name="tie">1.00</float>
> >>> <!-- Query Fields - Search fileds with its height -->
> >>> <str name="qf">
> >>> DocumentDisplayName^10 DocumentName^9
> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> >>> </str>
> >>> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
> >>> <str name="pf">
> >>> DocumentDisplayName^10 DocumentName^9
> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> >>> </str>
> >>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<
> http://wiki.apache.org/solr/**DisMaxQParserPlugin>
> >>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<
> http://wiki.apache.org/solr/DisMaxQParserPlugin>
> >>>> -->
> >>>
> >>> <str name="mm">
> >>> 2<-1 5<-2 6<90%
> >>> </str>
> >>> <!-- Phrase whitespace ignore -->
> >>> <int name="ps">10</int>
> >>> </lst>
> >>> </requestHandler>
> >>>
> >>>
> >>> but the synonyms and stopwords are ignored. If I use the default
> /select
> >>> query it works just as expected.
> >>>
> >>> What can be wrong?
> >>>
> >>> --
> >>> Atenciosamente,
> >>> *Carlos Alexandro Becker*
> >>> http://caarlos0.github.com/****about <
> http://caarlos0.github.com/**about><
> >>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Atenciosamente,
> >> *Carlos Alexandro Becker*
> >> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
> >>
> >
> >
> >
> > --
> > Atenciosamente,
> > *Carlos Alexandro Becker*
> > http://caarlos0.github.com/about
>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Carlos Alexandro Becker <ca...@gmail.com>.
Thanks for your answer Erik!
I changed the FieldType to:
https://gist.github.com/caarlos0/89b7c0484b154550bc63
And got a 400 error with message "analyzer returned too many terms for
multiTerm term: java".
I also tried to change the query to do not use wildcard, but it still
ignoring the synonyms..
Any idea?
Thanks
On Tue, Mar 5, 2013 at 10:18 AM, Erik Hatcher <er...@gmail.com>wrote:
> In that last example you're doing a wildcard query (java*), and by default
> that does not run (all of) the analysis chain you have defined.
>
> If you need to expand synonyms for wildcarded terms like this, you'll need
> to define a "multiterm" analysis chain. See here for more details: <
> http://wiki.apache.org/solr/MultitermQueryAnalysis>
>
> Erik
>
>
> On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:
>
> > Hi Jack, I've updated the gist:
> > https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec
> >
> > I'm give you the wrong browser tab result yesterday, sorry.
> >
> > The schema seems right to me. I have a field name BoosterField, with the
> > synonyms and etc enabled in its fieldtype...
> >
> > can't figure out what's wrong. :S
> >
> >
> > Jack, thanks for your help.
> >
> >
> > On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <jack@basetechnology.com
> >wrote:
> >
> >> Where it says:
> >>
> >> "querystring":"**DocumentContent:java",
> >> "parsedquery":"(+((**DocumentContent:java DocumentContent:notare
> >> DocumentContent:jre)~2/no_**coord) () () () () ())/no_coord",
> >>
> >> That indicates that "java" was expanded to be equivalent to "java",
> >> "notare", or "jre".
> >>
> >> Are you sure you have documents that have "notare" or "jre" in the
> >> DocumentContent field?
> >>
> >> Suggestion: leave off the "DocumentContent:" field from the query since
> it
> >> is included in the qf parameter.
> >>
> >>
> >> -- Jack Krupansky
> >>
> >> -----Original Message----- From: Carlos Alexandro Becker
> >> Sent: Monday, March 04, 2013 4:09 PM
> >> To: solr-user
> >> Subject: Re: requestHandler ignoring synonyms
> >>
> >>
> >> I've done this, but I'm not a specialist, so, I see nothing interesting.
> >>
> >> The log is: https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<
> https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
> >>
> >> Thanks.
> >>
> >>
> >> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <jack@basetechnology.com
> >**
> >> wrote:
> >>
> >> You can simply test whether synonyms are being ignored or how they are
> >>> being processed by using the Solr Admin UI Analysis page. Select the
> field
> >>> and enter the text to test. It will show you exactly what the synonym
> >>> filter does.
> >>>
> >>> Make sure the synonym file is exactly as specified in the token filter.
> >>>
> >>> If there are still problems, show us the field name, field type with
> >>> analyzer, and the query response with &debugQuery=true. The parsed
> query
> >>> string should have query-time synonyms expanded.
> >>>
> >>> -- Jack Krupansky
> >>>
> >>> -----Original Message----- From: Carlos Alexandro Becker
> >>> Sent: Monday, March 04, 2013 2:16 PM
> >>> To: solr-user
> >>> Subject: requestHandler ignoring synonyms
> >>>
> >>>
> >>> I have a request handler like this:
> >>>
> >>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
> >>> <!-- Adds filters to all queries-->
> >>> <lst name="appends">
> >>> <str name="fq">-DELETED:true</str>
> >>> </lst>
> >>>
> >>> <lst name="defaults">
> >>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<
> http://wiki.apache.org/solr/**DisMaxQParserPlugin>
> >>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<
> http://wiki.apache.org/solr/DisMaxQParserPlugin>
> >>>> -->
> >>>
> >>> <str name="defType">edismax</str>
> >>> <str name="echoParams">explicit</****str>
> >>>
> >>> <str name="df">text</str>
> >>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
> >>> <float name="tie">1.00</float>
> >>> <!-- Query Fields - Search fileds with its height -->
> >>> <str name="qf">
> >>> DocumentDisplayName^10 DocumentName^9
> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> >>> </str>
> >>> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
> >>> <str name="pf">
> >>> DocumentDisplayName^10 DocumentName^9
> >>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> >>> </str>
> >>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<
> http://wiki.apache.org/solr/**DisMaxQParserPlugin>
> >>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<
> http://wiki.apache.org/solr/DisMaxQParserPlugin>
> >>>> -->
> >>>
> >>> <str name="mm">
> >>> 2<-1 5<-2 6<90%
> >>> </str>
> >>> <!-- Phrase whitespace ignore -->
> >>> <int name="ps">10</int>
> >>> </lst>
> >>> </requestHandler>
> >>>
> >>>
> >>> but the synonyms and stopwords are ignored. If I use the default
> /select
> >>> query it works just as expected.
> >>>
> >>> What can be wrong?
> >>>
> >>> --
> >>> Atenciosamente,
> >>> *Carlos Alexandro Becker*
> >>> http://caarlos0.github.com/****about <
> http://caarlos0.github.com/**about><
> >>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Atenciosamente,
> >> *Carlos Alexandro Becker*
> >> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
> >>
> >
> >
> >
> > --
> > Atenciosamente,
> > *Carlos Alexandro Becker*
> > http://caarlos0.github.com/about
>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Erik Hatcher <er...@gmail.com>.
In that last example you're doing a wildcard query (java*), and by default that does not run (all of) the analysis chain you have defined.
If you need to expand synonyms for wildcarded terms like this, you'll need to define a "multiterm" analysis chain. See here for more details: <http://wiki.apache.org/solr/MultitermQueryAnalysis>
Erik
On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:
> Hi Jack, I've updated the gist:
> https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec
>
> I'm give you the wrong browser tab result yesterday, sorry.
>
> The schema seems right to me. I have a field name BoosterField, with the
> synonyms and etc enabled in its fieldtype...
>
> can't figure out what's wrong. :S
>
>
> Jack, thanks for your help.
>
>
> On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
>
>> Where it says:
>>
>> "querystring":"**DocumentContent:java",
>> "parsedquery":"(+((**DocumentContent:java DocumentContent:notare
>> DocumentContent:jre)~2/no_**coord) () () () () ())/no_coord",
>>
>> That indicates that "java" was expanded to be equivalent to "java",
>> "notare", or "jre".
>>
>> Are you sure you have documents that have "notare" or "jre" in the
>> DocumentContent field?
>>
>> Suggestion: leave off the "DocumentContent:" field from the query since it
>> is included in the qf parameter.
>>
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Carlos Alexandro Becker
>> Sent: Monday, March 04, 2013 4:09 PM
>> To: solr-user
>> Subject: Re: requestHandler ignoring synonyms
>>
>>
>> I've done this, but I'm not a specialist, so, I see nothing interesting.
>>
>> The log is: https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>>
>> Thanks.
>>
>>
>> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <ja...@basetechnology.com>**
>> wrote:
>>
>> You can simply test whether synonyms are being ignored or how they are
>>> being processed by using the Solr Admin UI Analysis page. Select the field
>>> and enter the text to test. It will show you exactly what the synonym
>>> filter does.
>>>
>>> Make sure the synonym file is exactly as specified in the token filter.
>>>
>>> If there are still problems, show us the field name, field type with
>>> analyzer, and the query response with &debugQuery=true. The parsed query
>>> string should have query-time synonyms expanded.
>>>
>>> -- Jack Krupansky
>>>
>>> -----Original Message----- From: Carlos Alexandro Becker
>>> Sent: Monday, March 04, 2013 2:16 PM
>>> To: solr-user
>>> Subject: requestHandler ignoring synonyms
>>>
>>>
>>> I have a request handler like this:
>>>
>>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
>>> <!-- Adds filters to all queries-->
>>> <lst name="appends">
>>> <str name="fq">-DELETED:true</str>
>>> </lst>
>>>
>>> <lst name="defaults">
>>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>>>> -->
>>>
>>> <str name="defType">edismax</str>
>>> <str name="echoParams">explicit</****str>
>>>
>>> <str name="df">text</str>
>>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
>>> <float name="tie">1.00</float>
>>> <!-- Query Fields - Search fileds with its height -->
>>> <str name="qf">
>>> DocumentDisplayName^10 DocumentName^9
>>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>>> </str>
>>> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
>>> <str name="pf">
>>> DocumentDisplayName^10 DocumentName^9
>>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>>> </str>
>>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>>>> -->
>>>
>>> <str name="mm">
>>> 2<-1 5<-2 6<90%
>>> </str>
>>> <!-- Phrase whitespace ignore -->
>>> <int name="ps">10</int>
>>> </lst>
>>> </requestHandler>
>>>
>>>
>>> but the synonyms and stopwords are ignored. If I use the default /select
>>> query it works just as expected.
>>>
>>> What can be wrong?
>>>
>>> --
>>> Atenciosamente,
>>> *Carlos Alexandro Becker*
>>> http://caarlos0.github.com/****about <http://caarlos0.github.com/**about><
>>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>>>
>>>
>>
>>
>> --
>> Atenciosamente,
>> *Carlos Alexandro Becker*
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>>
>
>
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Carlos Alexandro Becker <ca...@gmail.com>.
Hi Jack, I've updated the gist:
https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec
I'm give you the wrong browser tab result yesterday, sorry.
The schema seems right to me. I have a field name BoosterField, with the
synonyms and etc enabled in its fieldtype...
can't figure out what's wrong. :S
Jack, thanks for your help.
On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
> Where it says:
>
> "querystring":"**DocumentContent:java",
> "parsedquery":"(+((**DocumentContent:java DocumentContent:notare
> DocumentContent:jre)~2/no_**coord) () () () () ())/no_coord",
>
> That indicates that "java" was expanded to be equivalent to "java",
> "notare", or "jre".
>
> Are you sure you have documents that have "notare" or "jre" in the
> DocumentContent field?
>
> Suggestion: leave off the "DocumentContent:" field from the query since it
> is included in the qf parameter.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Carlos Alexandro Becker
> Sent: Monday, March 04, 2013 4:09 PM
> To: solr-user
> Subject: Re: requestHandler ignoring synonyms
>
>
> I've done this, but I'm not a specialist, so, I see nothing interesting.
>
> The log is: https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>
> Thanks.
>
>
> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <ja...@basetechnology.com>**
> wrote:
>
> You can simply test whether synonyms are being ignored or how they are
>> being processed by using the Solr Admin UI Analysis page. Select the field
>> and enter the text to test. It will show you exactly what the synonym
>> filter does.
>>
>> Make sure the synonym file is exactly as specified in the token filter.
>>
>> If there are still problems, show us the field name, field type with
>> analyzer, and the query response with &debugQuery=true. The parsed query
>> string should have query-time synonyms expanded.
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Carlos Alexandro Becker
>> Sent: Monday, March 04, 2013 2:16 PM
>> To: solr-user
>> Subject: requestHandler ignoring synonyms
>>
>>
>> I have a request handler like this:
>>
>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
>> <!-- Adds filters to all queries-->
>> <lst name="appends">
>> <str name="fq">-DELETED:true</str>
>> </lst>
>>
>> <lst name="defaults">
>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>> >-->
>>
>> <str name="defType">edismax</str>
>> <str name="echoParams">explicit</****str>
>>
>> <str name="df">text</str>
>> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
>> <float name="tie">1.00</float>
>> <!-- Query Fields - Search fileds with its height -->
>> <str name="qf">
>> DocumentDisplayName^10 DocumentName^9
>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>> </str>
>> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
>> <str name="pf">
>> DocumentDisplayName^10 DocumentName^9
>> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>> </str>
>> <!-- See http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>> <http://**wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>> >-->
>>
>> <str name="mm">
>> 2<-1 5<-2 6<90%
>> </str>
>> <!-- Phrase whitespace ignore -->
>> <int name="ps">10</int>
>> </lst>
>> </requestHandler>
>>
>>
>> but the synonyms and stopwords are ignored. If I use the default /select
>> query it works just as expected.
>>
>> What can be wrong?
>>
>> --
>> Atenciosamente,
>> *Carlos Alexandro Becker*
>> http://caarlos0.github.com/****about <http://caarlos0.github.com/**about><
>> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>>
>>
>
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Jack Krupansky <ja...@basetechnology.com>.
Where it says:
"querystring":"DocumentContent:java",
"parsedquery":"(+((DocumentContent:java DocumentContent:notare
DocumentContent:jre)~2/no_coord) () () () () ())/no_coord",
That indicates that "java" was expanded to be equivalent to "java",
"notare", or "jre".
Are you sure you have documents that have "notare" or "jre" in the
DocumentContent field?
Suggestion: leave off the "DocumentContent:" field from the query since it
is included in the qf parameter.
-- Jack Krupansky
-----Original Message-----
From: Carlos Alexandro Becker
Sent: Monday, March 04, 2013 4:09 PM
To: solr-user
Subject: Re: requestHandler ignoring synonyms
I've done this, but I'm not a specialist, so, I see nothing interesting.
The log is: https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec
Thanks.
On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky
<ja...@basetechnology.com>wrote:
> You can simply test whether synonyms are being ignored or how they are
> being processed by using the Solr Admin UI Analysis page. Select the field
> and enter the text to test. It will show you exactly what the synonym
> filter does.
>
> Make sure the synonym file is exactly as specified in the token filter.
>
> If there are still problems, show us the field name, field type with
> analyzer, and the query response with &debugQuery=true. The parsed query
> string should have query-time synonyms expanded.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Carlos Alexandro Becker
> Sent: Monday, March 04, 2013 2:16 PM
> To: solr-user
> Subject: requestHandler ignoring synonyms
>
>
> I have a request handler like this:
>
> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
> <!-- Adds filters to all queries-->
> <lst name="appends">
> <str name="fq">-DELETED:true</str>
> </lst>
>
> <lst name="defaults">
> <!-- See
> http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>-->
> <str name="defType">edismax</str>
> <str name="echoParams">explicit</**str>
> <str name="df">text</str>
> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
> <float name="tie">1.00</float>
> <!-- Query Fields - Search fileds with its height -->
> <str name="qf">
> DocumentDisplayName^10 DocumentName^9
> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> </str>
> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
> <str name="pf">
> DocumentDisplayName^10 DocumentName^9
> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> </str>
> <!-- See
> http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>-->
> <str name="mm">
> 2<-1 5<-2 6<90%
> </str>
> <!-- Phrase whitespace ignore -->
> <int name="ps">10</int>
> </lst>
> </requestHandler>
>
>
> but the synonyms and stopwords are ignored. If I use the default /select
> query it works just as expected.
>
> What can be wrong?
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Carlos Alexandro Becker <ca...@gmail.com>.
I've done this, but I'm not a specialist, so, I see nothing interesting.
The log is: https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec
Thanks.
On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
> You can simply test whether synonyms are being ignored or how they are
> being processed by using the Solr Admin UI Analysis page. Select the field
> and enter the text to test. It will show you exactly what the synonym
> filter does.
>
> Make sure the synonym file is exactly as specified in the token filter.
>
> If there are still problems, show us the field name, field type with
> analyzer, and the query response with &debugQuery=true. The parsed query
> string should have query-time synonyms expanded.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Carlos Alexandro Becker
> Sent: Monday, March 04, 2013 2:16 PM
> To: solr-user
> Subject: requestHandler ignoring synonyms
>
>
> I have a request handler like this:
>
> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
> <!-- Adds filters to all queries-->
> <lst name="appends">
> <str name="fq">-DELETED:true</str>
> </lst>
>
> <lst name="defaults">
> <!-- See http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>-->
> <str name="defType">edismax</str>
> <str name="echoParams">explicit</**str>
> <str name="df">text</str>
> <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
> <float name="tie">1.00</float>
> <!-- Query Fields - Search fileds with its height -->
> <str name="qf">
> DocumentDisplayName^10 DocumentName^9
> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> </str>
> <!-- Phrase Field - frases fields used on quer as phrase >> "" -->
> <str name="pf">
> DocumentDisplayName^10 DocumentName^9
> DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
> </str>
> <!-- See http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>-->
> <str name="mm">
> 2<-1 5<-2 6<90%
> </str>
> <!-- Phrase whitespace ignore -->
> <int name="ps">10</int>
> </lst>
> </requestHandler>
>
>
> but the synonyms and stopwords are ignored. If I use the default /select
> query it works just as expected.
>
> What can be wrong?
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>
>
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about
Re: requestHandler ignoring synonyms
Posted by Jack Krupansky <ja...@basetechnology.com>.
You can simply test whether synonyms are being ignored or how they are being
processed by using the Solr Admin UI Analysis page. Select the field and
enter the text to test. It will show you exactly what the synonym filter
does.
Make sure the synonym file is exactly as specified in the token filter.
If there are still problems, show us the field name, field type with
analyzer, and the query response with &debugQuery=true. The parsed query
string should have query-time synonyms expanded.
-- Jack Krupansky
-----Original Message-----
From: Carlos Alexandro Becker
Sent: Monday, March 04, 2013 2:16 PM
To: solr-user
Subject: requestHandler ignoring synonyms
I have a request handler like this:
<requestHandler name="/defaultSearch" class="solr.SearchHandler">
<!-- Adds filters to all queries-->
<lst name="appends">
<str name="fq">-DELETED:true</str>
</lst>
<lst name="defaults">
<!-- See http://wiki.apache.org/solr/DisMaxQParserPlugin -->
<str name="defType">edismax</str>
<str name="echoParams">explicit</str>
<str name="df">text</str>
<!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
<float name="tie">1.00</float>
<!-- Query Fields - Search fileds with its height -->
<str name="qf">
DocumentDisplayName^10 DocumentName^9
DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
</str>
<!-- Phrase Field - frases fields used on quer as phrase >> "" -->
<str name="pf">
DocumentDisplayName^10 DocumentName^9
DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
</str>
<!-- See http://wiki.apache.org/solr/DisMaxQParserPlugin -->
<str name="mm">
2<-1 5<-2 6<90%
</str>
<!-- Phrase whitespace ignore -->
<int name="ps">10</int>
</lst>
</requestHandler>
but the synonyms and stopwords are ignored. If I use the default /select
query it works just as expected.
What can be wrong?
--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about