You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Zhu JiaJun (JIRA)" <ji...@apache.org> on 2019/04/03 08:22:05 UTC
[jira] [Commented] (LUCENE-7698) CommonGramsQueryFilter in the
query analyzer chain breaks phrase queries
[ https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808465#comment-16808465 ]
Zhu JiaJun commented on LUCENE-7698:
------------------------------------
Hi [~emaijala],
I followed the steps and found a query on "hello with an accent" get empty result which should get match on the field "features":"Good unicode support: héllo (hello with an accent over the e)" of document (id: SOLR1000). I'm trying to apply "CommonGramsFilterFactory + StopFilterFactory" in the query analyzer for our solr environment, while this issue cause some query get empty result.
The issue seems to be happens when 2 stopwords are siblings in the a phrase search. for example: "with an", also I didn't add "an" in the the stopwords.txt file, form the Analysis Tool, I found the CommonGramsFilterFactory still regard it as a stopword.
JiaJun
> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> ------------------------------------------------------------------------
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/queryparser
> Affects Versions: 6.4, 6.4.1
> Reporter: Ere Maijala
> Priority: Blocker
> Labels: regression
> Fix For: 6.4.2, 7.0
>
> Attachments: LUCENE-7698.patch
>
>
> (Please pardon me if the project or component are wrong!)
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify server/solr/configsets/sample_techproducts_configs/conf/managed-schema and modify text_general fieldType by adding CommonGrams(Query)Filter before stopWordFilter:
> <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.CommonGramsFilterFactory" ignoreCase="true" words="stopwords.txt" />
> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
> <!-- in this example, we will only use synonyms at query time
> <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
> -->
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.CommonGramsQueryFilterFactory" ignoreCase="true" words="stopwords.txt"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> </fieldType>
> 3.) Add "with" to server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and make sure the file has correct line endings (extracted from Solr zip it seems to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22&debugQuery=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22&debugQuery=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not with_video.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org