You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "David Smiley (JIRA)" <ji...@apache.org> on 2008/09/05 21:22:44 UTC
[jira] Created: (SOLR-756) Make DisjunctionMaxQueryParser generally
useful by supporting all query types.
Make DisjunctionMaxQueryParser generally useful by supporting all query types.
------------------------------------------------------------------------------
Key: SOLR-756
URL: https://issues.apache.org/jira/browse/SOLR-756
Project: Solr
Issue Type: Improvement
Affects Versions: 1.3
Reporter: David Smiley
This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
(patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "Peter Wolanin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751038#action_12751038 ]
Peter Wolanin commented on SOLR-756:
------------------------------------
We are regularly hitting this wall and users are very frustrated by not being able to use wildcards becuase we wanted the other advantages of the dismax parser.
Any chance to get some of these changes in 1.4?
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Fix For: 1.5
>
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-756) Make DisjunctionMaxQueryParser generally
useful by supporting all query types.
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-756:
---------------------------------------
Fix Version/s: 1.5
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Fix For: 1.5
>
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "David Smiley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630217#action_12630217 ]
David Smiley commented on SOLR-756:
-----------------------------------
The DisjunctionMaxQueryParser in Solr does more than limit the user's syntax. It offers the DisjunctionMax capability which is unique to this query parser, it has a boostingQuery, and a boostingFunctions capability too. Furthermore, the DisjunctionMaxQueryParser prescribes a particular way of limiting the user's syntax that I think applications should be able to specify if they so choose.
Sure, I could "write my own query parser" but that's a pain... why should everyone who wants the DisMax capability have to write their own just because we don't want it to make assumptions about a limited syntax?
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630231#action_12630231 ]
Mark Miller commented on SOLR-756:
----------------------------------
Ah, your right, I've gotten a bit lost floating between these issues and the mailing list email that led me here.
I am not opposed to anything you have done. Just enriching the issue with some discussion <g> My point is only what I have said, and as you say, while mildly connected to this issue, is more leveled at 758.
I don't really have anything to say against this patch in particular, and I apologize for the confusion. Looking at the code, seems reasonable to me.
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754337#action_12754337 ]
Hoss Man commented on SOLR-756:
-------------------------------
at this stage in the process for 1.4, i think most committers are focusing on bug fixes and avoiding adding new features ... especially new features that have been added to existing functionality, so they could trigger new bugs in *existing* use cases.
FWIW:
* the patch doesn't seem to have any test cases demonstrating the new functionality, which makes me leary of committing
* i'm not certain just from skimming the patch, but it seems like this should break behavior for any existing users of DisjunctionMaxQueryParser who are expecting the field aliases to only affect getFieldQuery ... i could be wrong about that though.
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Fix For: 1.5
>
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630208#action_12630208 ]
Mark Miller commented on SOLR-756:
----------------------------------
I think the whole point of dismax is to support a very limited subset of the query syntax right? I think you want to add a new queryparser not modify dismax...
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630220#action_12630220 ]
Mark Miller commented on SOLR-756:
----------------------------------
I am not suggesting that you maintain your own, but perhaps that a new queryparser would be a better option. I am not positive in my answer, and am only thinking out loud. I got the impression that dismax goes out of its way to escape as much as it can to avoid parse errors and allow copy paste of text type of uses. The functionality you mention appears all to be from components below the dismax parser. In my mind, I like the idea of keeping dismax lean and mean and starting something else to become a kitchen sink - Lucene's queryparser became a bit of a kitchen sink and I think dismax looks to be coded so as to avoid some of the problems that can emerge from that...
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645566#action_12645566 ]
Otis Gospodnetic commented on SOLR-756:
---------------------------------------
My colleagues and I have recently hit a similar wall -- DisMax not allowing wildcards.
I think Hoss will be the loudest (not in a negative sense) here, so here are some specific questions:
# are we OK with adding wildcard (and other) support to "dismax"?
# if yes, which part of "dismax" should that be added to? (David described the 2 meanings of "dismax")
I've hit this wall a couple of times that I think it's worth thinking about how to get support for non-plain-keywords-only queries somethere in the DisMax area. Like David, I too want to make use of the other functionality that already exists there, such as bf, ps, etc.
Thanks.
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-756) Make DisjunctionMaxQueryParser
generally useful by supporting all query types.
Posted by "David Smiley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630223#action_12630223 ]
David Smiley commented on SOLR-756:
-----------------------------------
When anyone says "dismax", there is ambiguity because it might mean one of two things:
* SolrPluginUtils.DisjunctionMaxQueryParser which extends Solr's QueryParser which extends Lucene's QueryParser. In this JIRA issue I've offered a patch to this file. This code has to deal with the foundational capability of tapping into Lucene's DisjunctionMaxQuery so that multiple fields can be a default instead of Lucene's just one. I believe this change isn't controversial compared to others, IMO.
* The DisMaxQParserPlugin triggered with "qt=dismax" which uses the QueryParser mentioned above and does things like escape user syntax, ,boost various things, min-should-match, etc. Issue SOLR-758 includes my patch to that. I think this plugin is named poorly because dismax is just one thing it does.
I suspect your comments might be more applicable to SOLR-758 than they are here. Have you actually seen the source I attached? Are you apposed to the changes I make in this issue?
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-756) Make DisjunctionMaxQueryParser generally
useful by supporting all query types.
Posted by "David Smiley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Smiley updated SOLR-756:
------------------------------
Attachment: SolrPluginUtilsDisMax.patch
The patch file is only to be applied on SolrPluginUtils.java. The patch file might need to be modified to remove the header that indicates the path to this file which is unique to my development system.
Testing this would be a little tricky because this parser is only used by one QParser and it prohibits doing the type of queries that would exercise these changes. I have tests on my system but they involve other things that I have not contributed (yet, any way).
> Make DisjunctionMaxQueryParser generally useful by supporting all query types.
> ------------------------------------------------------------------------------
>
> Key: SOLR-756
> URL: https://issues.apache.org/jira/browse/SOLR-756
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: David Smiley
> Attachments: SolrPluginUtilsDisMax.patch
>
>
> This is an enhancement to the DisjunctionMaxQueryParser to work on all the query variants such as wildcard, prefix, and fuzzy queries, and to support working in "AND" scenarios that are not processed by the min-should-match DisMax QParser. This was not in Solr already because DisMax was only used for a very limited syntax that didn't use those features. In my opinion, this makes a more suitable base parser for general use because unlike the Lucene/Solr parser, this one supports multiple default fields whereas other ones (say Yonik's {!prefix} one for example, can't do dismax). The notion of a single default field is antiquated and a technical under-the-hood detail of Lucene that I think Solr should shield the user from by on-the-fly using a DisMax when multiple fields are used.
> (patch to be attached soon)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.