facet query when using word 'AND'

[Noriyuki TAKEI <nt...@sios.com>]

Hi,all

I use facet query,but I found it dose not work when using 'AND'.

I woud like to use facet query using 'AND' as not Operator but simple word.

At first,Solr Config is as below.

<searchComponent class="solr.SpellCheckComponent" name="suggest_ja">
    <lst name="spellchecker">
      <str name="name">suggest_dict</str>
      <str name="classname">solr.Suggester</str>
      <str name="lookupImpl">AnalyzingLookupFactory</str>
      <str name="field">suggest</str>
      <str name="storeDir">suggest_ja</str>
      <str name="buildOnStartup">true</str>
      <str name="buildOnCommit">true</str>
      <str name="suggestAnalyzerFieldType">text_ja_romaji</str>
    </lst>
  </searchComponent>

  <requestHandler name="/suggest_ja" class="solr.SearchHandler"
startup="lazy">
    <lst name="defaults">
      <str name="df">suggest</str>
      <str name="q.op">AND</str>
      <str name="rows">0</str>
      <str name="omitHeader">true</str>

      <str name="facet">true</str>
      <str name="facet.field">suggest</str>
      <str name="facet.limit">1000</str>
      <str name="facet.mincount">1</str>

      <str name="spellcheck">true</str>
      <str name="spellcheck.dictionary">suggest_dict</str>
      <str name="spellcheck.count">10</str>
      <str name="spellcheck.collate">true</str>
      <str name="spellcheck.maxCollations">30</str>
      <str name="spellcheck.maxCollationTries">10</str>
      <str name="spellcheck.collateExtendedResults">true</str>
    </lst>
    <arr name="last-components">
      <str>suggest_ja</str>
    </arr>
  </requestHandler>

I executed the query below,but Solr gave unexpected result.

$ curl
"http://localhost:8983/solr/kms/suggest_ja?wt=json&indent=true&q=\"AND\"&facet=true"
{
  "response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]
  },
  "facet_counts":{
    "facet_queries":{},
    "facet_fields":{
      "suggest":[]},
    "facet_ranges":{},
    "facet_intervals":{},
    "facet_heatmaps":{}},
  "spellcheck":{
    "suggestions":[],
    "collations":[]}}

I'd like to use facet search including the word "AND",so I surrounded "AND"
by
double quotes and then  appended the escappe parameter befoe
dobule quote as below.

\"AND\"

The Document included word "AND"(I have a pen AND an apple) is already
indexed.The evidence is as below.

$ curl "http://localhost:8983/solr/kms/select?wt=json&indent=true&q=\"AND\""

{

  "responseHeader":{
    "zkConnected":true,
    "status":0,
    "QTime":9,
    "params":{
      "q":"\"AND\"",
      "indent":"true",
      "wt":"json"}},
  "response":{"numFound":1,"start":0,"maxScore":0.2770272,"docs":[
      {
        "pub_date":"2017-03-09T12:34:56.789Z",
        "body":"I have a pen AND an apple",
        "title":"test",
        "url":"http://10.16.44.180:8080/#/management/two/",
        "system_cd":"hoge",
        "document_id":"001",
        "id":"hoge001",
        "content_type":"doc",
        "_version_":1577777862221496320}]
  }}


Therefore I expected the result as below.

{
  "response":{"numFound":1,"start":0,"maxScore":0.66747504,"docs":[]
  },
  "facet_counts":{
    "facet_queries":{},
    "facet_fields":{
      "suggest":[
        "AND",1,
        "apple",1,
        "pen",1,
]},
    "facet_ranges":{},
    "facet_intervals":{},
    "facet_heatmaps":{}},
  "spellcheck":{
    "suggestions":[],
    "collations":[]}}


Actually facet fields includes nothing.

How do I solve this?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: facet query when using word 'AND'

[Erick Erickson <er...@gmail.com>]

I typed the below then noticed that the field that has "I have a pen
AND an apple" is called "body" and you're faceting and searching on a
field called "suggest". The below is still relevant if there is still
a problem though ;)

Your problem isn't faceting, right? It's that you aren't matching any
docs at all when you expect to. Once we solve that then we can move on
to faceting ;)

First, I'd add &debug=query and see what the parsed query looks like.
Does it fit your expectations? See "parsedQuery_toString" in the debug
output.

Next, use the admin/analysis page to see how the input is analyzed at
index and query time. The first thing to verify is that the "and" is
not being removed since it's often considered a stopword. Do you get
facets when searching for "apple" (no quotes)? If you do is "and"
included as a facet? If not then you need to modify the stopwords
associated with the field in your schema.

You can use the admin/<select core>/schema browser to see the actual
terms in the index that'll help you verify that the _tokens_ in the
index include what you expect. Faceting works on indexed tokens BTW.

This is different than returning the field, since when you specify
fl=suggest you're getting back the _stored_ values, not the searchable
indexed tokens. This fools everybody at first. Stored values are a
verbatim copy of the input, no analysis at all. However, what you can
search/facet/etc. on is the tokenized version, which are _not_
returned via the "fl" field.

Best,
Erick

P.S. My compliments for including enough data to offer a diagnosis the
first time. It's refreshing not to have to go through 3 or 4 exchanges
before having enough information to say anything even potentially
useful.



On Wed, Sep 6, 2017 at 6:13 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 9/6/2017 3:48 AM, Noriyuki TAKEI wrote:
>> I use facet query,but I found it dose not work when using 'AND'.
>>
>> I woud like to use facet query using 'AND' as not Operator but simple word.
>
> With the standard or edismax parser, AND in all uppercase is interpreted
> as an operator.  There are two ways to deal with this.  One is to change
> the word to lowercase, which might not do what you want depending on
> your text analysis, the other is to escape part of it -- use A\ND
> instead of AND.  I have verified that escaping one of the letters in the
> word *does* work.
>
> If you're using the edismax query parser and the lowercase option, be
> sure that the lowercaseOperators parameter is set to false.  The default
> setting depends on luceneMatchVersion.  It's false when that's 7.0.0 or
> later, true if it's lower.  Which means that until version 7.0 is
> released, it will default to true.
>
> https://issues.apache.org/jira/browse/SOLR-4646
>
> Thanks,
> Shawn
>

Re: facet query when using word 'AND'

[Shawn Heisey <ap...@elyograg.org>]

On 9/6/2017 3:48 AM, Noriyuki TAKEI wrote:
> I use facet query,but I found it dose not work when using 'AND'.
>
> I woud like to use facet query using 'AND' as not Operator but simple word.

With the standard or edismax parser, AND in all uppercase is interpreted
as an operator.  There are two ways to deal with this.  One is to change
the word to lowercase, which might not do what you want depending on
your text analysis, the other is to escape part of it -- use A\ND
instead of AND.  I have verified that escaping one of the letters in the
word *does* work.

If you're using the edismax query parser and the lowercase option, be
sure that the lowercaseOperators parameter is set to false.  The default
setting depends on luceneMatchVersion.  It's false when that's 7.0.0 or
later, true if it's lower.  Which means that until version 7.0 is
released, it will default to true.

https://issues.apache.org/jira/browse/SOLR-4646

Thanks,
Shawn