Analysis tool vs search query

[nonrenewable <no...@gmail.com>]

Hi,

I've run into this issue that I have no way of resolving, since the analysis
tool doesn't show me there is an error. I copy the exact field value into
the analysis tool and i type in the exact query request i'm issuing and the
tool finds it a match. However running the query with that exact same
request doesn't return the item. 

I know the item is there, since I can find it based on another field. It
appears that the problem occurs when i add a second word in my query. So I
also tried replacing all whitespaces with _, just to make sure that there's
a mismatch there but there isn't. Here is my field type definition in case
i'm missing something
Thanks,
Tony

    <fieldType name="prefix_search" class="solr.TextField"
positionIncrementGap="1">
      <analyzer type="index">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.ISOLatin1AccentFilterFactory" />
        <filter class="solr.PatternReplaceFilterFactory"
                pattern="[\-.,()]" replacement="" replace="all"
        />
        <filter class="solr.PatternReplaceFilterFactory"
                pattern="\s+" replacement="_" replace="all"
        />
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="40"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.ISOLatin1AccentFilterFactory" />
        <filter class="solr.StopFilterFactory"
        ignoreCase="true"
        words="stopwords.txt"
        enablePositionIncrements="true"
        />
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true" />
        <filter class="solr.PatternReplaceFilterFactory"
                pattern="[\-.,()]" replacement="" replace="all"
        />
        <filter class="solr.PatternReplaceFilterFactory"
                pattern="\s+" replacement="_" replace="all"
        />
      </analyzer>
    </fieldType>

Example inputs for analysis:
Index value: Banana, Veggie
Query value: banana veggie

-- 
View this message in context: http://old.nabble.com/Analysis-tool-vs-search-query-tp27316047p27316047.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Analysis tool vs search query

[Chris Hostetter <ho...@fucit.org>]

: I've run into this issue that I have no way of resolving, since the analysis
: tool doesn't show me there is an error. I copy the exact field value into
: the analysis tool and i type in the exact query request i'm issuing and the
: tool finds it a match. However running the query with that exact same

the analysis tool doesn'ty do query parsing .. so pasing a *query* 
string into the analysis tool isn't going to give you any meaningful information.

what the "query" section of the analysis tool lets you do is see what the 
"query time analyzer" (that is used by most query parsers at query time) 
will do with your input ... but the QueryParser is still in control, and 
it decides which input to pass to your analyser -- special characters 
(like whitespace) have meaning to most query parsers, before they ever 
have a chance of getting passed to the analyzer.

:         <tokenizer class="solr.KeywordTokenizerFactory"/>

A keyword tokenizer results in a single token for each input string, but 
the (default) query parser is going to chunk the input up on whitespace 
before the analyzer is ever invoked, unless you put it in a quoted string.


-Hoss