You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Akihiro Ito <ak...@enigmo.co.jp> on 2019/07/25 06:24:49 UTC
autoGeneratePhraseQueries does not work?
HI,
I am using Solr 7.7.1 in SolrCloud mode.
I’m getting a document I shouldn’t when searching with a TextField.
It looks like autoGeneratePhaseQuery is not working as it should,
but I have no idea what is causing it.
The schema definition I use is as follows.
<fieldType name=“trigram_type” class=“solr.TextField”
positionIncrementGap=“100” autoGeneratePhraseQueries=“true”>
<analyzer type=“index”>
<charFilter class=“solr.ICUNormalizer2CharFilterFactory” name=“nfkc”/>
<charFilter class=“solr.PatternReplaceCharFilterFactory” pattern=“$”
replacement=“**“/>
<tokenizer class=“solr.NGramTokenizerFactory” maxGramSize=“3"
minGramSize=“3” />
<filter class=“solr.PatternReplaceFilterFactory”
pattern=“([^\s])\s[^\s]” replacement=“$1 ” replace=“all”/>
<filter class=“solr.PatternReplaceFilterFactory” pattern=“^\s.*$”
replacement=“” replace=“all”/>
<filter class=“solr.LengthFilterFactory” min=“3” max=“3"/>
<filter class=“solr.LowerCaseFilterFactory”/>
<filter class=“solr.ICUTransformFilterFactory” id=“Hiragana-Katakana”/>
</analyzer>
<analyzer type=“query”>
<charFilter class=“solr.ICUNormalizer2CharFilterFactory” name=“nfkc”/>
<tokenizer class=“solr.NGramTokenizerFactory” maxGramSize=“3"
minGramSize=“3” />
<filter class=“solr.LowerCaseFilterFactory”/>
<filter class=“solr.ICUTransformFilterFactory” id=“Hiragana-Katakana”/>
</analyzer>
</fieldType>
Following sample document is in Solr.
docs: [
{
syo_id: “1237”,
trigram: “aaabbbccc”,
＿version＿: 1639992506850476000,
timestamp: “2019-07-25T01:38:52.894Z”
}
]
If I execute the following query,it will hit the above document
q=trigram:aaaa&fq=syo_id:1237&debugQuery=on
Thanks,
Akihiro.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: autoGeneratePhraseQueries does not work?
Posted by Akihiro Ito <ak...@enigmo.co.jp>.
Thank you for your reply.
I did not know about the sow parameter.
By specifying "sow = true" explicitly,
it has been confirmed that the search results are as expected.
akihiro:~$curl
"http://localhost:28983/solr/bmitems_A/select?q=trigram:aaaa&fq=syo_id:1237&sow=true&echoParams=none"
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":5},
"response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]
}}
akihiro:~$curl
"http://localhost:28983/solr/bmitems_A/select?q=trigram:aaaa&fq=syo_id:1237&echoParams=none"
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":7},
"response":{"numFound":1,"start":0,"maxScore":26.75299,"docs":[
{
"syo_id":"1237",
"trigram":"aaabbbccc",
"_version_":1640006469631868928,
"timestamp":"2019-07-25T05:20:48.834Z",
"s_syo_name":"aaabbbccc"}]
}}
Thanks,
Akihiro.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: autoGeneratePhraseQueries does not work?
Posted by Yasufumi Mizoguchi <ya...@gmail.com>.
Hi,
Since Solr 7.0, sow(Split-on-Whitespace) parameter has set false as
implicit default.
Because autoGeneratePhraseQueries option's behavior depends on that
parameter,
you should add sow=true parameter to your query.
e.g)
q=trigram:aaaa&fq=syo_id:1237&debugQuery=on&sow=true
Thanks,
Yasufumi.
2019年7月25日(木) 15:25 Akihiro Ito <ak...@enigmo.co.jp>:
> HI,
>
> I am using Solr 7.7.1 in SolrCloud mode.
>
> I’m getting a document I shouldn’t when searching with a TextField.
> It looks like autoGeneratePhaseQuery is not working as it should,
> but I have no idea what is causing it.
>
> The schema definition I use is as follows.
>
> <fieldType name=“trigram_type” class=“solr.TextField”
> positionIncrementGap=“100” autoGeneratePhraseQueries=“true”>
> <analyzer type=“index”>
> <charFilter class=“solr.ICUNormalizer2CharFilterFactory” name=“nfkc”/>
> <charFilter class=“solr.PatternReplaceCharFilterFactory” pattern=“$”
> replacement=“**“/>
> <tokenizer class=“solr.NGramTokenizerFactory” maxGramSize=“3"
> minGramSize=“3” />
> <filter class=“solr.PatternReplaceFilterFactory”
> pattern=“([^\s])\s[^\s]” replacement=“$1 ” replace=“all”/>
> <filter class=“solr.PatternReplaceFilterFactory” pattern=“^\s.*$”
> replacement=“” replace=“all”/>
> <filter class=“solr.LengthFilterFactory” min=“3” max=“3"/>
> <filter class=“solr.LowerCaseFilterFactory”/>
> <filter class=“solr.ICUTransformFilterFactory”
> id=“Hiragana-Katakana”/>
> </analyzer>
> <analyzer type=“query”>
> <charFilter class=“solr.ICUNormalizer2CharFilterFactory” name=“nfkc”/>
> <tokenizer class=“solr.NGramTokenizerFactory” maxGramSize=“3"
> minGramSize=“3” />
> <filter class=“solr.LowerCaseFilterFactory”/>
>
> <filter class=“solr.ICUTransformFilterFactory”
> id=“Hiragana-Katakana”/>
> </analyzer>
> </fieldType>
>
>
> Following sample document is in Solr.
>
> docs: [
> {
> syo_id: “1237”,
> trigram: “aaabbbccc”,
> ＿version＿: 1639992506850476000,
> timestamp: “2019-07-25T01:38:52.894Z”
> }
> ]
>
> If I execute the following query,it will hit the above document
>
> q=trigram:aaaa&fq=syo_id:1237&debugQuery=on
>
>
> Thanks,
> Akihiro.
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
Re: autoGeneratePhraseQueries does not work?
Posted by Alexandros Paramythis <al...@contexity.ch>.
Hi Akihiro,
This behavior is caused by the NGramTokenizerFactory, which is used both
during analysis and querying in the configuration.
In short:
* During indexing, this filter causes the token 'aaa' to be generated
from the string 'aaabbbccc' (because minGramSize="3") and indexed
* At query time, this filter causes the token 'aaa' to be generated from
the string "aaaa" (again, because minGramSize="3")
Therefore, the matching should indeed occur exactly as you are seeing it
happen.
You may find the second example in the N-Gram Tokenizer section of the
reference guide useful:
https://lucene.apache.org/solr/guide/7_7/tokenizers.html#n-gram-tokenizer
Hope this helps,
Alex
On 25/07/2019 08:24, Akihiro Ito wrote:
> HI,
>
> I am using Solr 7.7.1 in SolrCloud mode.
>
> I’m getting a document I shouldn’t when searching with a TextField.
> It looks like autoGeneratePhaseQuery is not working as it should,
> but I have no idea what is causing it.
>
> The schema definition I use is as follows.
>
> <fieldType name=“trigram_type” class=“solr.TextField”
> positionIncrementGap=“100” autoGeneratePhraseQueries=“true”>
> <analyzer type=“index”>
> <charFilter class=“solr.ICUNormalizer2CharFilterFactory” name=“nfkc”/>
> <charFilter class=“solr.PatternReplaceCharFilterFactory” pattern=“$”
> replacement=“**“/>
> <tokenizer class=“solr.NGramTokenizerFactory” maxGramSize=“3"
> minGramSize=“3” />
> <filter class=“solr.PatternReplaceFilterFactory”
> pattern=“([^\s])\s[^\s]” replacement=“$1 ” replace=“all”/>
> <filter class=“solr.PatternReplaceFilterFactory” pattern=“^\s.*$”
> replacement=“” replace=“all”/>
> <filter class=“solr.LengthFilterFactory” min=“3” max=“3"/>
> <filter class=“solr.LowerCaseFilterFactory”/>
> <filter class=“solr.ICUTransformFilterFactory” id=“Hiragana-Katakana”/>
> </analyzer>
> <analyzer type=“query”>
> <charFilter class=“solr.ICUNormalizer2CharFilterFactory” name=“nfkc”/>
> <tokenizer class=“solr.NGramTokenizerFactory” maxGramSize=“3"
> minGramSize=“3” />
> <filter class=“solr.LowerCaseFilterFactory”/>
>
> <filter class=“solr.ICUTransformFilterFactory” id=“Hiragana-Katakana”/>
> </analyzer>
> </fieldType>
>
>
> Following sample document is in Solr.
>
> docs: [
> {
> syo_id: “1237”,
> trigram: “aaabbbccc”,
> ＿version＿: 1639992506850476000,
> timestamp: “2019-07-25T01:38:52.894Z”
> }
> ]
>
> If I execute the following query,it will hit the above document
>
> q=trigram:aaaa&fq=syo_id:1237&debugQuery=on
>
>
> Thanks,
> Akihiro.
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html