You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by jefferyyuan <yu...@gmail.com> on 2013/07/20 00:01:00 UTC

The way edismax parses colon seems weird

In our application, user may search error code like 12:34.

We define default search field, like: <str name="qf">title^10 body_stored^8
content^5</str>
So when user search: 12:34, we want to search the error code in the
specified fields.

In the code, if we search q=12:34 directly, this can't find anything. It's
expected as it'ss to search 34 on 12 field.

Then we try to escape the colon, search: 12\:34, the parsedquery would be
+12\:34, still can't find the expected page.
<str name="parsedquery">(+12\:34)/no_coord</str>
<str name="parsedquery_toString">+12\:34</str>
<str name="QParser">ExtendedDismaxQParser</str>

If I type 2 \\, seems it can find the error page:
q=12\\:34
<str name="parsedquery">
(+DisjunctionMaxQuery((content:"12 34"^0.5 | body_stored:"(12\:34 12)
34"^0.8 | title:"12 34"^1.1)))/no_coord
</str>
<str name="parsedquery_toString">
+(content:"12 34"^0.5 | body_stored:"(12\:34 12) 34"^0.8 | title:"12
34"^1.1)
</str>
<str name="QParser">ExtendedDismaxQParser</str>

Is this a bug in Solr edismax or not?



--
View this message in context: http://lucene.472066.n3.nabble.com/The-way-edismax-parses-colon-seems-weird-tp4079226.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: The way edismax parses colon seems weird

Posted by Jack Krupansky <ja...@basetechnology.com>.
Very good chance that is it.

-- Jack Krupansky

-----Original Message----- 
From: Alexandre Rafalovitch
Sent: Friday, July 19, 2013 7:16 PM
To: solr-user@lucene.apache.org
Subject: Re: The way edismax parses colon seems weird

Could this be related:
https://issues.apache.org/jira/browse/SOLR-4333(Fixed in 4.4, so you
could even run your test against RC1)

Regards,
   Alex.

Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)


On Fri, Jul 19, 2013 at 6:01 PM, jefferyyuan <yu...@gmail.com> wrote:

> In our application, user may search error code like 12:34.
>
> We define default search field, like: <str name="qf">title^10 
> body_stored^8
> content^5</str>
> So when user search: 12:34, we want to search the error code in the
> specified fields.
>
> In the code, if we search q=12:34 directly, this can't find anything. It's
> expected as it'ss to search 34 on 12 field.
>
> Then we try to escape the colon, search: 12\:34, the parsedquery would be
> +12\:34, still can't find the expected page.
> <str name="parsedquery">(+12\:34)/no_coord</str>
> <str name="parsedquery_toString">+12\:34</str>
> <str name="QParser">ExtendedDismaxQParser</str>
>
> If I type 2 \\, seems it can find the error page:
> q=12\\:34
> <str name="parsedquery">
> (+DisjunctionMaxQuery((content:"12 34"^0.5 | body_stored:"(12\:34 12)
> 34"^0.8 | title:"12 34"^1.1)))/no_coord
> </str>
> <str name="parsedquery_toString">
> +(content:"12 34"^0.5 | body_stored:"(12\:34 12) 34"^0.8 | title:"12
> 34"^1.1)
> </str>
> <str name="QParser">ExtendedDismaxQParser</str>
>
> Is this a bug in Solr edismax or not?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/The-way-edismax-parses-colon-seems-weird-tp4079226.html
> Sent from the Solr - User mailing list archive at Nabble.com.
> 


Re: The way edismax parses colon seems weird

Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Could this be related:
https://issues.apache.org/jira/browse/SOLR-4333(Fixed in 4.4, so you
could even run your test against RC1)

Regards,
   Alex.

Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)


On Fri, Jul 19, 2013 at 6:01 PM, jefferyyuan <yu...@gmail.com> wrote:

> In our application, user may search error code like 12:34.
>
> We define default search field, like: <str name="qf">title^10 body_stored^8
> content^5</str>
> So when user search: 12:34, we want to search the error code in the
> specified fields.
>
> In the code, if we search q=12:34 directly, this can't find anything. It's
> expected as it'ss to search 34 on 12 field.
>
> Then we try to escape the colon, search: 12\:34, the parsedquery would be
> +12\:34, still can't find the expected page.
> <str name="parsedquery">(+12\:34)/no_coord</str>
> <str name="parsedquery_toString">+12\:34</str>
> <str name="QParser">ExtendedDismaxQParser</str>
>
> If I type 2 \\, seems it can find the error page:
> q=12\\:34
> <str name="parsedquery">
> (+DisjunctionMaxQuery((content:"12 34"^0.5 | body_stored:"(12\:34 12)
> 34"^0.8 | title:"12 34"^1.1)))/no_coord
> </str>
> <str name="parsedquery_toString">
> +(content:"12 34"^0.5 | body_stored:"(12\:34 12) 34"^0.8 | title:"12
> 34"^1.1)
> </str>
> <str name="QParser">ExtendedDismaxQParser</str>
>
> Is this a bug in Solr edismax or not?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/The-way-edismax-parses-colon-seems-weird-tp4079226.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: The way edismax parses colon seems weird

Posted by Chris Hostetter <ho...@fucit.org>.
You havne't told us anything about how you have the anslysis configured 
for the fileds you are using -- and those details probably contain the 
specifics of your problem.

once you've escaped hte colon so that eismax no longer recognizes it as 
"search this specific user field" syntax, any other questions about what 
the final query for that clauses windsup being, and what it does or 
doesn't match are entirely depending on your analyis.

When I use the Solr 4.3.1 example configs, and index a document like 
this...

$ java -Ddata=args -jar post.jar '<add><doc><field name="id">HOSS</field><field name="title">12:34</field><field name="cat">12:34</field></doc></add>'

Then the following query will find it... 
http://localhost:8983/solr/select?debugQuery=true&defType=edismax&qf=title&q=12\:34 
...and the parsed query, because of the fieldType & analyzier for the 
title field, looks like...

	(+DisjunctionMaxQuery(((title:12 title:34))))/no_coord

This query will also find it...
http://localhost:8983/solr/select?debugQuery=true&defType=edismax&qf=cat&q=56\:78
	(+DisjunctionMaxQuery((cat:56:78)))/no_coord

As will this one...
http://localhost:8983/solr/select?debugQuery=true&defType=edismax&qf=sku&q=99\:00
	(+DisjunctionMaxQuery((sku:9900)))/no_coord


As does combining them all together...
http://localhost:8983/solr/select?debugQuery=true&q.op=AND&defType=edismax&qf=sku+title+cat&q=12\:34+56\:78+99\:00

Note that if you use to "uf" option to tighten down which field names 
edismax allows with the ":" syntax, you don't even have to escape it...

http://localhost:8983/solr/select?debugQuery=true&q.op=AND&defType=edismax&uf=-*&qf=sku+title+cat&q=12:34+56:78+99:00



-Hoss

Re: The way edismax parses colon seems weird

Posted by Jack Krupansky <ja...@basetechnology.com>.
I noticed that a single backslash in the URL query got turned into a 
backslash in the parsed query, which implies that the backslash was escaped 
(improperly) by Solr:

http://localhost:8080/solr/select?q=12\:34&defType=edismax&debug=query&qf=content

<str name="parsedquery_toString">+12\:34</str>

As a workaround, enclose the term in quotes, without the escaping:

http://localhost:8080/solr/select?q="12:34"&defType=edismax&debug=query&qf=content

-- Jack Krupansky

-----Original Message----- 
From: jefferyyuan
Sent: Friday, July 19, 2013 7:09 PM
To: solr-user@lucene.apache.org
Subject: Re: The way edismax parses colon seems weird

Thanks very much for the reply.
We are querying solr directly from browser:
http://localhost:8080/solr/select?q=12\:34&defType=edismax&debug=query&qf=content

<str name="rawquerystring">12\:34</str>
<str name="querystring">12\:34</str>
<str name="parsedquery">(+12\:34)/no_coord</str>
<str name="parsedquery_toString">+12\:34</str>
<str name="QParser">ExtendedDismaxQParser</str>

And seems this is not related with which (default) field I use to query.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/The-way-edismax-parses-colon-seems-weird-tp4079226p4079234.html
Sent from the Solr - User mailing list archive at Nabble.com. 


Re: The way edismax parses colon seems weird

Posted by jefferyyuan <yu...@gmail.com>.
Thanks very much for the reply. 
We are querying solr directly from browser:
http://localhost:8080/solr/select?q=12\:34&defType=edismax&debug=query&qf=content

<str name="rawquerystring">12\:34</str>
<str name="querystring">12\:34</str>
<str name="parsedquery">(+12\:34)/no_coord</str>
<str name="parsedquery_toString">+12\:34</str>
<str name="QParser">ExtendedDismaxQParser</str>

And seems this is not related with which (default) field I use to query.



--
View this message in context: http://lucene.472066.n3.nabble.com/The-way-edismax-parses-colon-seems-weird-tp4079226p4079234.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: The way edismax parses colon seems weird

Posted by Shawn Heisey <so...@elyograg.org>.
On 7/19/2013 4:01 PM, jefferyyuan wrote:
> If I type 2 \\, seems it can find the error page:
> q=12\\:34
> <str name="parsedquery">
> (+DisjunctionMaxQuery((content:"12 34"^0.5 | body_stored:"(12\:34 12)
> 34"^0.8 | title:"12 34"^1.1)))/no_coord
> </str>
> <str name="parsedquery_toString">
> +(content:"12 34"^0.5 | body_stored:"(12\:34 12) 34"^0.8 | title:"12
> 34"^1.1)
> </str>
> <str name="QParser">ExtendedDismaxQParser</str>
>
> Is this a bug in Solr edismax or not?

It sounds like it's a requirement for whatever you are using to 
construct your queries.  When building Strings in Java, for instance, a 
double backslash is required for a literal backslash, because a single 
backslash is used for special characters, like \n for newline.  It's 
similar for Perl, and probably PHP as well as other programming languages.

Thanks,
Shawn


Re: The way edismax parses colon seems weird

Posted by Jack Krupansky <ja...@basetechnology.com>.
What field type analyzer and tokenizer are you using, and what does a sample 
of the input data look like?

Generally, a single backslash I all that is needed for escaping.

And, escaping is not needed within a quoted phrase, except for quotes and 
literal backslashes.

-- Jack Krupansky

-----Original Message----- 
From: jefferyyuan
Sent: Friday, July 19, 2013 6:01 PM
To: solr-user@lucene.apache.org
Subject: The way edismax parses colon seems weird

In our application, user may search error code like 12:34.

We define default search field, like: <str name="qf">title^10 body_stored^8
content^5</str>
So when user search: 12:34, we want to search the error code in the
specified fields.

In the code, if we search q=12:34 directly, this can't find anything. It's
expected as it'ss to search 34 on 12 field.

Then we try to escape the colon, search: 12\:34, the parsedquery would be
+12\:34, still can't find the expected page.
<str name="parsedquery">(+12\:34)/no_coord</str>
<str name="parsedquery_toString">+12\:34</str>
<str name="QParser">ExtendedDismaxQParser</str>

If I type 2 \\, seems it can find the error page:
q=12\\:34
<str name="parsedquery">
(+DisjunctionMaxQuery((content:"12 34"^0.5 | body_stored:"(12\:34 12)
34"^0.8 | title:"12 34"^1.1)))/no_coord
</str>
<str name="parsedquery_toString">
+(content:"12 34"^0.5 | body_stored:"(12\:34 12) 34"^0.8 | title:"12
34"^1.1)
</str>
<str name="QParser">ExtendedDismaxQParser</str>

Is this a bug in Solr edismax or not?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/The-way-edismax-parses-colon-seems-weird-tp4079226.html
Sent from the Solr - User mailing list archive at Nabble.com.