You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2009/07/30 10:05:15 UTC
[jira] Created: (LUCENE-1768) NumericRange support for new query
parser
NumericRange support for new query parser
-----------------------------------------
Key: LUCENE-1768
URL: https://issues.apache.org/jira/browse/LUCENE-1768
Project: Lucene - Java
Issue Type: New Feature
Components: QueryParser
Affects Versions: 2.9
Reporter: Uwe Schindler
It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743334#action_12743334 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
<quote>
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
<quote>
+1
<quote>
I am not sure how to do this with the new parser. I think of the same like the MTQRewriteMethod (final static singletons in MTQ that do the rewrite and can be passed as parameter).
<quote>
I think we probably should have TermRangeQueryNode a NumericQueryNode and 2 builders classes that match that, change ParametricRangeQueryNodeProcessor to do the dirty work and create the new TermRangeQueryNode and NumericQueryNode in the correct places, based on the a map with [field name,RangeTools.TYPE] or something similar.
The builders should simple and just convert each type to the correct Lucene Object.
- we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
- create the new NumericQueryNode that extends from TermRangeQueryNode
- change the ParametricRangeQueryNodeProcessor to read the configuration passed by the user and create the correct QueryNode objects.
- create a new NumericQueryNodeBuilder add it to the StandardQueryTreeBuilder mapping.
I hope this helps
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738052#action_12738052 ]
Uwe Schindler commented on LUCENE-1768:
---------------------------------------
Luis,
I will post an example of queries and the constructed Query objects when I am back from holidays (Thursday+). In principle the syntax would be the same like for normal range queries, only that the min/max arguments may be double, float, int, long or dates. You would create instances of NumericRangeQuery from it using one of the static factories for each data type (for dates a conversion to long using Date.getTime() would be done). The datatype must be somehow predefined for the field names using some type of schema (per field). Open ends use "*" and the [], (), {} would define if incl. NumericRangeQuery is a subclass of MultiTermQuery so the rewrite method also applies to this query.
Example code for creating the NRQ are in the JavaDocs and there are 2 JUnits in trunk (TestNumericRangeQuery*) showing how it is used. Also the new LIA2 contains a chapter about it.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Adriano Crestani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742223#action_12742223 ]
Adriano Crestani commented on LUCENE-1768:
------------------------------------------
{quote}
The proposed RangeTools seems like a good approach, and I like how it
cleanly absorbs the Date precisions that the old queryParser also
supports.
{quote}
You meant DateTools, right?! I don't see so much difference to use this same approach over the "option1". You have a map based from field name to the DateTools.Resolution used for that field. Which is the same feature we want to implement on this JIRA, something you could configure how you are going to resolve the value defined on a range query based on the field name. The only difference is that we are expanding the options the user will have to resolve the values: RangeUtils.NUMERIC, RangeUtils.DATE, RangeUtils.FLOAT, etc...let me know if I missed or missunderstood something on this part.
{quote}
Here's one side-question, about back compat promises for the new
QueryParser: we are suggesting the users can start from all the
building blocks in StandardQueryParser, and override the processors,
create new nodes, builders, etc. with their own. But this is
potentially dangerous, in that the next version of Lucene might change
things up such that your custom code doesn't work anymore? It's alot
like a core class being subclassed externally, and then change to the
core class break those external subclasses.
EG say we had not handled numerics for 2.9, and users go and do
"option 2" (the quick & dirty, but simplest, way to get
NumericRangeQueries out). Then, say in 3.1 we implement the proposed
fix here ("option 1"). Suddenly, we've altered what nodes come out of
the processor pipeline, because we've created a new NumericRangeQuery
node, and so the builders that users had added, for the RangeQuery
node, will no loner be invoked. How are we going to handle
back-compat here?
{quote}
I think it's already happening with the "old" QP. It used to output RangeQuery objects and now it outputs TermRangeQuery objects. How is it going to be handled buy users expecting RangeQuery objects?
The "new" QP builder, delegates a query node based on its class to a builder, if there is no builder that knows how to build an object from that class it keeps looking up in the class hierarchy until it finds a builder that knows how to. Query nodes are supposed to be conceptual objects, they just represent some concept X, and ideally anything that fits in this concept should inherit from it, this way the user can create their own specific query nodes with no need to change how they are built (if there is no need for that). What I'm trying to say here is that if I create a node Y which extends X, I don't need to specify a new YBuilder for it, the XBuilder will be used. So, ideally, NumericRangeQueryNode should extends RangeQueryNode, the problem here is that we also need to specify a builder for the NumericRangeNode, and if the user sets a builder for RangeNode it will never be invoked for NumericRangeNode objects. Maybe it shouldn't at all, because if a new builder was specified for NumericRangeNode, it means a new kind of object should be built from it, something the user probably don't know yet, since it's a new kind of node, and his custom code needs to be updated anyway to support it.
Howerver, there is a solution for this kind of back-compat problem (which I don't think it is). In a future release, if a new XRangeQueryNode is created, instead of set
{code}
luceneBuilderMap.setBuilder(RangeQueryNode.class, new RangeQueryNodeBuilder());
luceneBuilderMap.setBuilder(XRangeQueryNode.class, new XRangeQueryNodeBuilder());
{code}
We could do:
{code}
rangeBuilderMap.setBuilder(RangeQueryNode.class, new RangeQueryNodeBuilder());
rangeBuilderMap.setBuilder(XRangeQueryNode.class, new XRangeQueryNodeBuilder());
// then
luceneBuilderMap.setBuilder(RangeQueryNode.class, rangeBuilderMap);
{code}
This way, if the user reset the RangeQueryNode builder to its own builder, it will still be called for XRangeQueryNode and RangeQueryNode objects.
Let me know if there is any question about what I just described.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744383#action_12744383 ]
Uwe Schindler commented on LUCENE-1768:
---------------------------------------
bq. we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
I would not do this. RangeQueryNode is in the syntax tree and the syntax of numeric and term ranges is equal, so the query parser cannot know what type of query it is. When this issue is fixed 3.1, this node will use the configuration of data types for field names (date, numeric, term) to create the correct range query.
bq. +1 on pushing this. getRangeQuery() will still be first class.
As noted in my comment on java-dev: We should add a comment in Javadocs, that the old (and also new) query parser do not work automatically with NumericRangeQuery, and that you should override getRangeQuery() and do a case-switch on the field name. I will do this later this day.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1768) NumericRange support for
new query parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740718#action_12740718 ]
Luis Alves edited comment on LUCENE-1768 at 8/7/09 1:43 PM:
------------------------------------------------------------
{quote}
Neither is your version with rangeTypes.put("money", RangeUtils.getType(RangeUtils.NUMERIC...
That's the application specific configuration code and doesn't need (or want) to be committed.
{quote}
You are correct, I was describing the use case from the user perspective.
That code was a example how to use the API's if we implement them in the future, those API's are not currently available.
{quote}
Directly instantiating the query you want is simple, ultimately configurable, and avoids adding a ton of unnecessary classes or methods that need to be kept in sync with everything that a user may want to do.
{quote}
I'm not sure what to say here. So I'll point to the documentation that we currently have:
You can read https://issues.apache.org/jira/secure/attachment/12410046/QueryParser_restructure_meetup_june2009_v2.pdf
and the java docs for
package org.apache.lucene.queryParser.core
class org.apache.lucene.queryParser.standard.StandardQueryParser
You can also look at TestSpanQueryParserSimpleSample junit for another example how the API's can be used,
in a completely different way.
The new QueryParser was designed to be extensible,
allow the implementation of languages extensions or different languages,
and have reusable components like the processors and builders
We use SyntaxParsers, Processors and Builders, all are replaceable components at runtime.
Any user can build it's own pipeline and create new processors, builders, querynodes and integrate them
with the existing ones to create the features they require.
Some of the features are:
- Syntax Tree optimization
- Syntax Tree expansion
- Syntax Tree validation and error reporting
- Tokenization and normalization of the query
- Makes it easy to create extensions
- Support for translation of error messages
- Allows users to plug and play processors and builders, without having to modify lucene code.
- Allow lucene users to implement features much faster
- Allow users to change default behavior in a easy way without having to modify lucene code.
{quote}
Is there a simple way to provide a custom QueryBuilder for range queries (or any other query type?) I'm sure there must be, but there are so many classes in the new QP, I'm having a little difficulty finding my way around.
{quote}
Below is the java code for option 2. It's not the recomend way to use the new queryparser,
but is the shortest way to do what you want.
{code}
class NumericQueryNodeBuilder extends RangeQueryNodeBuilder {
public TermRangeQuery build(QueryNode queryNode) throws QueryNodeException {
RangeQueryNode rangeNode = (RangeQueryNode) queryNode;
if (rangeNode.getField().toString().equals("money")) {
// do whatever you need here with queryNode.
return new NumericRangeQuery(field,...)
}
else {
return super.build(queryNode);
}
}
}
public void testNewRangeQueryBuilder() throws Exception {
StandardQueryParser qp = new StandardQueryParser();
QueryTreeBuilder builder = (QueryTreeBuilder)qp.getQueryBuilder();
builder.setBuilder(RangeQueryNode.class, new NumericQueryNodeBuilder());
String startDate = getLocalizedDate(2002, 1, 1, false);
String endDate = getLocalizedDate(2002, 1, 4, false);
StandardAnalyzer oneStopAnalyzer = new StandardAnalyzer();
qp.setAnalyzer(oneStopAnalyzer);
Query a = qp.parse("date:[" + startDate + " TO " + endDate + "]", null);
System.out.print(a);
}
{code}
was (Author: lafa):
{quote}
Neither is your version with rangeTypes.put("money", RangeUtils.getType(RangeUtils.NUMERIC...
That's the application specific configuration code and doesn't need (or want) to be committed.
{quote}
You are correct, I was describing the use case from the user perspective.
That code was a example how to use the API's if we implement them in the future, those API's are not currently available.
{quote}
Directly instantiating the query you want is simple, ultimately configurable, and avoids adding a ton of unnecessary classes or methods that need to be kept in sync with everything that a user may want to do.
{quote}
I'm not sure what to say here. So I'll point to the documentation that we currently have:
You can read https://issues.apache.org/jira/secure/attachment/12410046/QueryParser_restructure_meetup_june2009_v2.pdf
and the java docs for
package org.apache.lucene.queryParser.core
class org.apache.lucene.queryParser.standard.StandardQueryParser
You can also look at TestSpanQueryParserSimpleSample junit for another example how the API's can be used,
in a completely different way.
The new QueryParser was designed to be extensible,
allow the implementation of languages extensions or different languages,
and have reusable components like the processors and builders
We use SyntaxParsers, Processors and Builders, all are replaceable components at runtime.
Any user can build it's own pipeline and create new processors, builders, querynodes and integrate them
with the existing ones to create the features they require.
Some of the features are:
- Syntax Tree optimization
- Syntax Tree expansion
- Syntax Tree validation and error reporting
- Tokenization and normalization of the query
- Makes it easy to create extensions
- Support for translation of error messages
- Allows users to plug and play processors and builders, without having to modify lucene code.
- Allow lucene users to implement features much faster
- Allow users to change default behavior in a easy way without having to modify lucene code.
{quote}
Is there a simple way to provide a custom QueryBuilder for range queries (or any other query type?) I'm sure there must be, but there are so many classes in the new QP, I'm having a little difficulty finding my way around.
{quote}
{code}
class NumericQueryNodeBuilder extends RangeQueryNodeBuilder {
public TermRangeQuery build(QueryNode queryNode) throws QueryNodeException {
RangeQueryNode rangeNode = (RangeQueryNode) queryNode;
if (rangeNode.getField().toString().equals("money")) {
// do whatever you need here with queryNode.
return new NumericRangeQuery(field,...)
}
else {
return super.build(queryNode);
}
}
}
public void testNewRangeQueryBuilder() throws Exception {
StandardQueryParser qp = new StandardQueryParser();
QueryTreeBuilder builder = (QueryTreeBuilder)qp.getQueryBuilder();
builder.setBuilder(RangeQueryNode.class, new NumericQueryNodeBuilder());
String startDate = getLocalizedDate(2002, 1, 1, false);
String endDate = getLocalizedDate(2002, 1, 4, false);
StandardAnalyzer oneStopAnalyzer = new StandardAnalyzer();
qp.setAnalyzer(oneStopAnalyzer);
Query a = qp.parse("date:[" + startDate + " TO " + endDate + "]", null);
System.out.print(a);
}
{code}
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740659#action_12740659 ]
Yonik Seeley commented on LUCENE-1768:
--------------------------------------
bq. It's not reusable by other users (we can't commit your code to lucene)
Neither is your version with rangeTypes.put("money", RangeUtils.getType(RangeUtils.NUMERIC...
That's the application specific configuration code and doesn't need (or want) to be committed.
Directly instantiating the query you want is simple, ultimately configurable, and avoids adding a ton of unnecessary classes or methods that need to be kept in sync with everything that a user *may* want to do.
Is there a simple way to provide a custom QueryBuilder for range queries (or any other query type?) I'm sure there must be, but there are so many classes in the new QP, I'm having a little difficulty finding my way around.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1768:
------------------------------------------
Assignee: Uwe Schindler
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740608#action_12740608 ]
Michael McCandless commented on LUCENE-1768:
--------------------------------------------
bq. You could still do something similar by simply override RangeQueryNodeBuilder.build(QueryNode queryNode), but this is not clean (it is kind of a hack).
What's the cleaner way to do this? EG could I make my own ParametricRangeQueryNodeProcessor, subclassing the current one in the "standard.processors" package, that overrides postProcessNode to do its own conversion?
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740575#action_12740575 ]
Yonik Seeley commented on LUCENE-1768:
--------------------------------------
It feels like going that route would add much code and complexity.
If the user already knows how to create a range query in code, it's much more straightforward to just do
{code}
if ("money".equals(field)) return new NumericRangeQuery(field,...)
else return super.getRangeQuery(field,...)
{code}
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740222#action_12740222 ]
Yonik Seeley commented on LUCENE-1768:
--------------------------------------
bq. I think, this should be in 2.9.
The standard way in the past was for the app to simply override getRangeQuery() to handle different fields differently.
This still seems the most flexible.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740340#action_12740340 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
You could still do something similar by simply override RangeQueryNodeBuilder.build(QueryNode queryNode), but this is not clean (it is kind of a hack).
A clean implementation would allow the user to configure the field types (which the "new flexible queryparser" does).
I'm new to NumericRange Queries and Rangequeries in general, but here is what I think it should look like.
Here is a seudo java example:
{code}
final String defaultField = "default";
final String monthField = "month";
final String hourField = "hour";
final String distanceField = "distance";
final String moneyField = "money";
Map<CharSequence, RangeTools.Type> rangeTypes = new HashMap<CharSequence, RangeTools.Type>();
// set a field specific range type per field
rangeTypes.put(monthField, new RangeTools.Type(RangeUtils.DATE, DateTools.Resolution.MONTH) );
rangeTypes.put(hourField, new RangeUtils.Type(RangeUtils.DATE, DateTools.Resolution.HOUR) );
rangeTypes.put(distanceField, RangeUtils.getType(RangeUtils.NUMERIC, RangeUtils.NumericType.LONG, NumericUtils.PRECISION_STEP_DEFAULT) );
rangeTypes.put(moneyField, RangeUtils.getType(RangeUtils.NUMERIC, RangeUtils.NumericType.Type.FLOAT, NumericUtils.PRECISION_STEP_DEFAULT) );
StandardQueryParser qp = new StandardQueryParser();
// set default range type to Int default precision
qp.setDefaultRangeType(RangeUtils.getType(RangeUtils.NUMERIC, RangeUtils.NumericType.INT, NumericUtils.PRECISION_STEP_DEFAULT));
// set field range types
qp.setRangeTypes(rangeTypes);
Query q = qp.parser(" month:[01/01/2004 TO 01/01/2005] distance:[1000 to 2000] money: [23.50 to 50.99]");
{code}
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1768) NumericRange support for
new query parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738052#action_12738052 ]
Uwe Schindler edited comment on LUCENE-1768 at 8/2/09 5:10 AM:
---------------------------------------------------------------
Luis,
I will post an example of queries and the constructed Query objects when I am back from holidays (Thursday+). In principle the syntax would be the same like for normal range queries, only that the min/max arguments may be double, float, int, long or dates. You would create instances of NumericRangeQuery from it using one of the static factories for each data type (for dates a conversion to long using Date.getTime() would be done). The datatype must be somehow predefined for the field names using some type of schema (per field).. Open ends use "*" and the [], (), {} would define if incl. NumericRangeQuery is a subclass of MultiTermQuery so the rewrite method also applies to this query. For NRQ there is also a config parameter precisionStep which default value is 4, but should be also configureable per-field together with the data type.
Example code for creating the NRQ are in the JavaDocs and there are 2 JUnits in trunk (TestNumericRangeQuery*) showing how it is used. Also the new LIA2 contains a chapter about it.
was (Author: thetaphi):
Luis,
I will post an example of queries and the constructed Query objects when I am back from holidays (Thursday+). In principle the syntax would be the same like for normal range queries, only that the min/max arguments may be double, float, int, long or dates. You would create instances of NumericRangeQuery from it using one of the static factories for each data type (for dates a conversion to long using Date.getTime() would be done). The datatype must be somehow predefined for the field names using some type of schema (per field). Open ends use "*" and the [], (), {} would define if incl. NumericRangeQuery is a subclass of MultiTermQuery so the rewrite method also applies to this query.
Example code for creating the NRQ are in the JavaDocs and there are 2 JUnits in trunk (TestNumericRangeQuery*) showing how it is used. Also the new LIA2 contains a chapter about it.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Adriano Crestani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742458#action_12742458 ]
Adriano Crestani commented on LUCENE-1768:
------------------------------------------
{quote}
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
{quote}
+1 this way is easier for the user to config
{quote}
I was thinking about that, too. But here the API clearly defines, that getRangeQuery() returns a Query object without further specification. So the change was correct from the API/BW side. The change that another object is returned is documented in CHANGES.txt (as far as I know). We have here the same problem: You change the inner class implementations, but the abstract QueryParser's API is stable. The general contract when doing such things is, that you use instanceof checks before you try to cast some abstract return type to something specific, not documented.
{quote}
Agreed, I also think it's fine as long as it's documented
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740728#action_12740728 ]
Uwe Schindler commented on LUCENE-1768:
---------------------------------------
To go back to the idea why I opened the issue (and I think, this is also Mike's intention):
>From what you see on java-user, where users asking questions about how to use Lucene:
Most users are not aware of the fact, that they can create Query classes themselves. Most examplecode on the list is just: "I have such query string and I pass it to lucene and it does not work as exspected." It is hard to explain them, that they should simply not use a query parser for their queries and just instantiate the query classes directly. For such users it is even harder to customize this query parser.
My intention behind is: Make the RangeQueryNodeBuilder somehow configureable like Luis proposed, that you can set the type of a field (what we do not have in Lucene currently). If the type is undefined or explicite set to "string/term", create a TermRangeQuery. If it is set to any numeric type, create a NumericRangeQuery.newXxxRange(field,....).
The same can currently be done by the original Lucene query parser, but only for dates (and it is really a hack using this DateField class). I simply want to extend it that you can say: "this field is of type 'int' and create automatically the correct range query for it." Because the old query parser is now "deprecated", I want to do it for the new one. This would also be an intention for new users to throw away the old parser and use the new one, because it can be configured easily to create numeric ranges in addition to term ranges.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743349#action_12743349 ]
Yonik Seeley commented on LUCENE-1768:
--------------------------------------
If the existing query parser is not being deprecated, should this issue be pushed out to 3.0 or 3.1 to give it more time? In the meantime, people can use the existing override getRangeQuery() method. 2.9 is looking really close.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1768) NumericRange support for
new query parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743334#action_12743334 ]
Luis Alves edited comment on LUCENE-1768 at 8/14/09 11:32 AM:
--------------------------------------------------------------
{quote}
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
{quote}
+1
{quote}
I am not sure how to do this with the new parser. I think of the same like the MTQRewriteMethod (final static singletons in MTQ that do the rewrite and can be passed as parameter).
{quote}
I think we probably should have TermRangeQueryNode a NumericRangeQueryNode and 2 builders classes that match that, change ParametricRangeQueryNodeProcessor to do the dirty work and create the new TermRangeQueryNode and NumericRangeQueryNode in the correct places, based on the a map with [field name,RangeTools.TYPE] or something similar.
The builders should simple and just convert each type to the correct Lucene Object.
- we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
- create the new NumericRangeQueryNode that extends from TermRangeQueryNode
- change the ParametricRangeQueryNodeProcessor to read the configuration passed by the user and create the correct QueryNode objects.
- create a new NumericRangeQueryNodeBuilder add it to the StandardQueryTreeBuilder mapping.
I hope this helps
was (Author: lafa):
{quote}
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
{quote}
+1
{quote}
I am not sure how to do this with the new parser. I think of the same like the MTQRewriteMethod (final static singletons in MTQ that do the rewrite and can be passed as parameter).
{quote}
I think we probably should have TermRangeQueryNode a NumericQueryNode and 2 builders classes that match that, change ParametricRangeQueryNodeProcessor to do the dirty work and create the new TermRangeQueryNode and NumericQueryNode in the correct places, based on the a map with [field name,RangeTools.TYPE] or something similar.
The builders should simple and just convert each type to the correct Lucene Object.
- we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
- create the new NumericQueryNode that extends from TermRangeQueryNode
- change the ParametricRangeQueryNodeProcessor to read the configuration passed by the user and create the correct QueryNode objects.
- create a new NumericQueryNodeBuilder add it to the StandardQueryTreeBuilder mapping.
I hope this helps
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740643#action_12740643 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
Hi Yonik,
As I said before you can do that in the RangeQueryNodeBuilder.build(QueryNode queryNode),
but it's ugly and this is not what we intended when using the "new flexible query parser".
The "new flexible query parser" does not follow the concept of method overwriting has the old one.
So solutions that worked in the old queryparser, like overwriting a method, have to be implemented
using a programmatic way.
Your approach requires creating a new class, overwrite a method.
you still need to create a instance of your QueryParser and is not reusable.
Here is a sample of what your approach is:
{code}
Class YonikQueryParser extends QueryParser{
Query getRangeQuery(field,...) {
if ("money".equals(field)) return new NumericRangeQuery(field,...)
else return super.getRangeQuery(field,...)
}
}
...
QueryParser yqp = new YonikQueryParser(...);
yqp.parser(query);
{code}
vs
What I am proposing:
{code}
Map<CharSequence, RangeTools.Type> rangeTypes = new HashMap<CharSequence, RangeTools.Type>();
rangeTypes.put("money", RangeUtils.getType(RangeUtils.NUMERIC, RangeUtils.NumericType.Type.FLOAT, NumericUtils.PRECISION_STEP_DEFAULT) );
StandardQueryParser qp = new StandardQueryParser();
qp.setRangeTypes(rangeTypes);
qp.parser(query);
{code}
The second approach is programmatic does not require a new class,
or the overwrite of a method and is reusable by other users, and it's
backward compatible, meaning we can integrate this on the current
"Flexible query parser" and deliver this feature on 2.9 without affecting
any current usecase.
Your approach is not compatible, it does require new class, and is not programmatic,
It's not reusable by other users (we can't commit your code to lucene),
since fields are hard-coded.
Also the approach I proposing is very similar to setFieldsBoost setDateResolution,
already available on the old QP and the new flexible query parser.
I also want to say, that extending the old QP vs extending the "New flexible Query Parser" approaches
are never going to be similar, they completely different implementations.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740662#action_12740662 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
{quote}
What's the cleaner way to do this? EG could I make my own ParametricRangeQueryNodeProcessor, subclassing the current one in the "standard.processors" package, that overrides postProcessNode to do its own conversion?
{quote}
For Yonik simple requirement, you could
Option 1 (more flexible):
- make your own ParametricRangeQueryNodeProcessor, subclassing the current, returning NumericQueryNodes where needed
- create a NumericQueryNode that extends RangeQueryNode (node extra code needed)
- create a NumericQueryNodeBuilder that handles NumericQueryNodes, and set the map in StandardQueryTreeBuilder, ex: setBuilder(NumericQueryNode.class, new NumericQueryNodeBuilder()),. RangeQueryNodes will still be normally handled by the RangeQueryNodeBuilder.
Option 2, (less flexible):
- make your own RangeQueryNodeBuilder subclassing the current(ex: NumericQueryNodeBuilder) , set the map in StandardQueryTreeBuilder, ex: setBuilder(RangeQueryNode.class, new NumericQueryNodeBuilder())
Option 1, implements the correct usage of the APIs. It's more flexible and "dirty work" is done in the processors pipeline.
Option 2, is not the correct use case for the APIs, requires less code and it will work, but the builder will be performing the tasks the Processor should be doing.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740718#action_12740718 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
{quote}
Neither is your version with rangeTypes.put("money", RangeUtils.getType(RangeUtils.NUMERIC...
That's the application specific configuration code and doesn't need (or want) to be committed.
{quote}
You are correct, I was describing the use case from the user perspective.
That code was a example how to use the API's if we implement them in the future, those API's are not currently available.
{quote}
Directly instantiating the query you want is simple, ultimately configurable, and avoids adding a ton of unnecessary classes or methods that need to be kept in sync with everything that a user may want to do.
{quote}
I'm not sure what to say here. So I'll point to the documentation that we currently have:
You can read https://issues.apache.org/jira/secure/attachment/12410046/QueryParser_restructure_meetup_june2009_v2.pdf
and the java docs for
package org.apache.lucene.queryParser.core
class org.apache.lucene.queryParser.standard.StandardQueryParser
You can also look at TestSpanQueryParserSimpleSample junit for another example how the API's can be used,
in a completely different way.
The new QueryParser was designed to be extensible,
allow the implementation of languages extensions or different languages,
and have reusable components like the processors and builders
We use SyntaxParsers, Processors and Builders, all are replaceable components at runtime.
Any user can build it's own pipeline and create new processors, builders, querynodes and integrate them
with the existing ones to create the features they require.
Some of the features are:
- Syntax Tree optimization
- Syntax Tree expansion
- Syntax Tree validation and error reporting
- Tokenization and normalization of the query
- Makes it easy to create extensions
- Support for translation of error messages
- Allows users to plug and play processors and builders, without having to modify lucene code.
- Allow lucene users to implement features much faster
- Allow users to change default behavior in a easy way without having to modify lucene code.
{quote}
Is there a simple way to provide a custom QueryBuilder for range queries (or any other query type?) I'm sure there must be, but there are so many classes in the new QP, I'm having a little difficulty finding my way around.
{quote}
{code}
class NumericQueryNodeBuilder extends RangeQueryNodeBuilder {
public TermRangeQuery build(QueryNode queryNode) throws QueryNodeException {
RangeQueryNode rangeNode = (RangeQueryNode) queryNode;
if (rangeNode.getField().toString().equals("money")) {
// do whatever you need here with queryNode.
return new NumericRangeQuery(field,...)
}
else {
return super.build(queryNode);
}
}
}
public void testNewRangeQueryBuilder() throws Exception {
StandardQueryParser qp = new StandardQueryParser();
QueryTreeBuilder builder = (QueryTreeBuilder)qp.getQueryBuilder();
builder.setBuilder(RangeQueryNode.class, new NumericQueryNodeBuilder());
String startDate = getLocalizedDate(2002, 1, 1, false);
String endDate = getLocalizedDate(2002, 1, 4, false);
StandardAnalyzer oneStopAnalyzer = new StandardAnalyzer();
qp.setAnalyzer(oneStopAnalyzer);
Query a = qp.parse("date:[" + startDate + " TO " + endDate + "]", null);
System.out.print(a);
}
{code}
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1768:
----------------------------------
Fix Version/s: (was: 2.9)
3.1
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 3.1
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737669#action_12737669 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
Uwe,
Thanks for creating the jira issue.
Can you add some simple query examples.
What would be the lucene Query objects for those queries, if it was produce by a QP that supported that feature.
Also elaborate what is the current expect behavior for those queries.
If you can write a junit with one or 2 indexed docs,
and a lucene Query that retrives just one of those docs and not the other
without using the queryparser, that would be helpful.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744347#action_12744347 ]
Mark Miller commented on LUCENE-1768:
-------------------------------------
Finally read through this whole issue.
bq. If the existing query parser is not being deprecated, should this issue be pushed out to 3.0 or 3.1 to give it more time? In the meantime, people can use the existing override getRangeQuery() method. 2.9 is looking really close.
+1 on pushing this. getRangeQuery() will still be first class.
It does seem like we should at least do this though:
we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1768) NumericRange support for
new query parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743334#action_12743334 ]
Luis Alves edited comment on LUCENE-1768 at 8/14/09 11:16 AM:
--------------------------------------------------------------
{quote}
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
{quote}
+1
{quote}
I am not sure how to do this with the new parser. I think of the same like the MTQRewriteMethod (final static singletons in MTQ that do the rewrite and can be passed as parameter).
{quote}
I think we probably should have TermRangeQueryNode a NumericQueryNode and 2 builders classes that match that, change ParametricRangeQueryNodeProcessor to do the dirty work and create the new TermRangeQueryNode and NumericQueryNode in the correct places, based on the a map with [field name,RangeTools.TYPE] or something similar.
The builders should simple and just convert each type to the correct Lucene Object.
- we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
- create the new NumericQueryNode that extends from TermRangeQueryNode
- change the ParametricRangeQueryNodeProcessor to read the configuration passed by the user and create the correct QueryNode objects.
- create a new NumericQueryNodeBuilder add it to the StandardQueryTreeBuilder mapping.
I hope this helps
was (Author: lafa):
<quote>
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
<quote>
+1
<quote>
I am not sure how to do this with the new parser. I think of the same like the MTQRewriteMethod (final static singletons in MTQ that do the rewrite and can be passed as parameter).
<quote>
I think we probably should have TermRangeQueryNode a NumericQueryNode and 2 builders classes that match that, change ParametricRangeQueryNodeProcessor to do the dirty work and create the new TermRangeQueryNode and NumericQueryNode in the correct places, based on the a map with [field name,RangeTools.TYPE] or something similar.
The builders should simple and just convert each type to the correct Lucene Object.
- we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
- create the new NumericQueryNode that extends from TermRangeQueryNode
- change the ParametricRangeQueryNodeProcessor to read the configuration passed by the user and create the correct QueryNode objects.
- create a new NumericQueryNodeBuilder add it to the StandardQueryTreeBuilder mapping.
I hope this helps
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Luis Alves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744374#action_12744374 ]
Luis Alves commented on LUCENE-1768:
------------------------------------
{quote}
If the existing query parser is not being deprecated, should this issue be pushed out to 3.0 or 3.1 to give it more time? In the meantime, people can use the existing override getRangeQuery() method. 2.9 is looking really close.
{quote}
+1
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740861#action_12740861 ]
Michael McCandless commented on LUCENE-1768:
--------------------------------------------
Given the complexity of customizing the new QueryParser, and given
that numeric fields will likely be commonly used, I think this is an
important issue. I think we should try to have the new QueryParser
cleanly produce NumericRangeQuery, in 2.9.
EG expecting a user to do "option 1" (the "clean", more flexible
option) is a tall order. Simple things should be simple...
The proposed RangeTools seems like a good approach, and I like how it
cleanly absorbs the Date precisions that the old queryParser also
supports.
But we better get cracking here since 2.9 is real close....!
Here's one side-question, about back compat promises for the new
QueryParser: we are suggesting the users can start from all the
building blocks in StandardQueryParser, and override the processors,
create new nodes, builders, etc. with their own. But this is
potentially dangerous, in that the next version of Lucene might change
things up such that your custom code doesn't work anymore? It's alot
like a core class being subclassed externally, and then change to the
core class break those external subclasses.
EG say we had not handled numerics for 2.9, and users go and do
"option 2" (the quick & dirty, but simplest, way to get
NumericRangeQueries out). Then, say in 3.1 we implement the proposed
fix here ("option 1"). Suddenly, we've altered what nodes come out of
the processor pipeline, because we've created a new NumericRangeQuery
node, and so the builders that users had added, for the RangeQuery
node, will no loner be invoked. How are we going to handle
back-compat here?
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741432#action_12741432 ]
Mark Miller commented on LUCENE-1768:
-------------------------------------
Personally, I don't think we should deprecate the standard QueryParser yet - and the new one should carry no back compat policy. It needs to be flushed out in a release before we tell users to move to it IMO. Not enough Committers have enough experience with it to promise back compat at this point I think.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742308#action_12742308 ]
Michael McCandless commented on LUCENE-1768:
--------------------------------------------
bq. I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class
+1
bq. Howerver, there is a solution for this kind of back-compat problem (which I don't think it is).
Actually, on reading your explanation I agree it's not really a back compat break, since the user's custom builder for RangeQueryNode would still be invoked, and the core's builder for NumericRangeQuery would handle the newly added numeric range support. I think this is reasonable.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Adriano Crestani (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745236#action_12745236 ]
Adriano Crestani commented on LUCENE-1768:
------------------------------------------
{quote}
we should rename RangeQueryNode to TermRangeQueryNode (to match lucene name)
I would not do this. RangeQueryNode is in the syntax tree and the syntax of numeric and term ranges is equal, so the query parser cannot know what type of query it is. When this issue is fixed 3.1, this node will use the configuration of data types for field names (date, numeric, term) to create the correct range query.
{quote}
I think it's ok to rename, as far as I know, the standard.parser.SyntaxParser generates ParametricRangeQueryNode from a range query, which has 2 ParametricQueryNode as child. So, the range processor, will need to convert the 2 ParametricQueryNode to the respective type, based on the user config: TermRangeQueryNode (renamed from RangeQueryNode) or NumericRangeQueryNode.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 3.1
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1768:
----------------------------------
Fix Version/s: 2.9
I think, this should be in 2.9. Any Chance to do this. In my Opinion, it should be not so hard. I will prepare something tomorrow.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1768) NumericRange support for new query
parser
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742232#action_12742232 ]
Uwe Schindler commented on LUCENE-1768:
---------------------------------------
I would propose to absorb the RangeTools/Utils and DateTools/Utils (ehat is the correct name???) in one configuration class (just a bigger enumeration with a good name, not *Utils/*.Tools. e.g. RangeQueryDataType). By that you can define simply the type of a range query: term, numeric-int, numeric-float, numeric-double, date-precision-xxx,... Based on this enumeration, the upper/lower terms are parsed differently and different query objects are created. We just need to list all possible combinations of data types, the user could create: We could make this class extensible, if it is a Lucene Parameter class also supporting the parsing and building: One could simply create a new constant for his specific range type and supply methods to parse and build the query in the constant's implementation (so each constant contains also code to parse/build). I am not sure how to do this with the new parser. I think of the same like the MTQRewriteMethod (final static singletons in MTQ that do the rewrite and can be passed as parameter).
Maybe we can use this also to upgrade the old query parser if it gets not deprecated.
bq. I think it's already happening with the "old" QP. It used to output RangeQuery objects and now it outputs TermRangeQuery objects. How is it going to be handled buy users expecting RangeQuery objects?
I was thinking about that, too. But here the API clearly defines, that getRangeQuery() returns a Query object without further specification. So the change was correct from the API/BW side. The change that another object is returned is documented in CHANGES.txt (as far as I know). We have here the same problem: You change the inner class implementations, but the abstract QueryParser's API is stable. The general contract when doing such things is, that you use instanceof checks before you try to cast some abstract return type to something specific, not documented.
You have the same in various factories also in the very bw-oriented JDK: XML factories create things like SAXParser and so on. If you cast the returned objects to some special implementation class, its your problem, because you remove the abstraction and work with implementations. This happened e.g. from the change between Java 1.4 to 1.5, when the internal SAX parsers were exchanged and their class names changed. A lot of programs broke by that, because the developers casted the objects returned from factories without instanceof checks.
> NumericRange support for new query parser
> -----------------------------------------
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
> Issue Type: New Feature
> Components: QueryParser
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
>
> It would be good to specify some type of "schema" for the query parser in future, to automatically create NumericRangeQuery for different numeric types? It would then be possible to index a numeric value (double,float,long,int) using NumericField and then the query parser knows, which type of field this is and so it correctly creates a NumericRangeQuery for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so the user will have to configure the FieldConfig objects in the ConfigHandler. But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the instantiation of the correct Query type and conversion of the entered numeric values (simple Number.valueOf(...) cast of the user entered numbers). Evenerything else is identical, NumericRangeQuery also supports the MTQ rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in the current parser that tells it how to handle dates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org