You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Yonik Seeley (JIRA)" <ji...@apache.org> on 2007/09/15 00:13:32 UTC
[jira] Created: (SOLR-356) pluggable functions (value sources)
pluggable functions (value sources)
-----------------------------------
Key: SOLR-356
URL: https://issues.apache.org/jira/browse/SOLR-356
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-356) pluggable functions (value sources)
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley resolved SOLR-356.
-------------------------------
Resolution: Fixed
I just committed this. Thanks Doug!
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559206#action_12559206 ]
Hoss Man commented on SOLR-356:
-------------------------------
bq. Maybe I'm missing something from the recommendation though. Were you recommending using XML to express the functions themselves, or something else about the xml-query-parser?
I was not suggesting an XML syntax ... just that the approach the xml-query-parser takes to deal with recursively parsing/eval-ing the XML structure using "user" configured implementations for each type of XML node seems to map closely to the idea of recusively parsing/eval-ing parenthetical function syntax using "user" configured implantations for each function name.
In both cases the XML/paren parsing is trivial, it's deciding how to let the "user" tell you what Java objects to build based on each node/function name that gets interesting.
(like i said, i haven't looked at the xml-query-parser code since it was orriginally being written .. i may be over romanticizing the idea behind it's design and how applicable it could be in this case)
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-356) pluggable functions (value sources)
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated SOLR-356:
------------------------------
Attachment: pluggableFunctions.patch
Here's a patch for the second option, hiding the low-level details from ValueSourceParser implementations.
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-356) pluggable functions (value sources)
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated SOLR-356:
------------------------------
Attachment: pluggableFunctions.patch
This is a quick attempt at pluggable functions using the same style as Yonik's QParser plugins. it takes the path suggested in http://www.nabble.com/forum/ViewPost.jtp?post=12770704&framed=y by letting people register new functions as ValueSourceParser plugins in solrconfig.
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-356) pluggable functions (value sources)
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated SOLR-356:
------------------------------
Attachment: pluggableFunctions.patch
Added a sample ValueSourceParser plugin and some tests for it in TestFunctionQuery. The sample plugin is for an "nvl" function that replaces a null value in a doc with a parameter float value. It works much like the oracle SQL function of the same name. It also takes in an initialization parameter to test that functionality out.
I also made TestFunctionQuery use a new copy of solrconfig.xml (in solrconfig-functionquery.xml) to avoid polluting the standard one with plugins.
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560207#action_12560207 ]
Yonik Seeley commented on SOLR-356:
-----------------------------------
Looking good! I think the current API is straight forward and relatively easy to support, even if we changed underlying parsing technologies.
I think all we need now is a test that exercises plugging in a new function from solrconfig.xml...
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558434#action_12558434 ]
Doug Daniels commented on SOLR-356:
-----------------------------------
I agree that the ValueSourceParser should be isolated from the low-level details like grabbing commas between args.
I thought about the first option you suggested, and it seems difficult with recursive functions. When the ValueSourceParser.createValueSource method expects another ValueSource as an arg, it would need to invoke whatever code created it (presumably from the FunctionQParser). Alternately, the FunctionQParser could ensure that the innermost functions are run first, passing their completed values out to enclosing functions as params.
It seems simpler to me to go with the second option though -- locking down the API on FunctionQParser. The ValueSourceParser would already have access to FunctionQParser, which it could call when it needs to parse a ValueSource argument.
What do you think?
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559075#action_12559075 ]
Doug Daniels commented on SOLR-356:
-----------------------------------
I found that thread at [http://marc.info/?l=lucene-dev&m=113355526731460&w=2]. It's quite a lengthy thread, but from what I read I agree that it's trying to solve a similar problem (plus a few additional problems that solr has since solved nicely).
Perhaps it's just personal preference, but I find XML description of functions, though powerful and expressive, quite a bit clunky. I far prefer the functional style for this sort of task.
Maybe I'm missing something from the recommendation though. Were you recommending using XML to express the functions themselves, or something else about the xml-query-parser?
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558423#action_12558423 ]
Yonik Seeley commented on SOLR-356:
-----------------------------------
Thanks for taking a crack at this Doug!
My initial thought was perhaps to simply have a Map from function name to ValueSource class, and the ValueSource could either
1) take a List<Object> (or NamedList if we want to start supporting named params like python, etc)
2) specify the argment list so that the function parser could validate the parameters (but on second thought, I think this could get too complex)
But your use of ValueSourceParser looks to have advantages, as it's essentially a factory and can act as a virtual constructor, and can be initialized with different static params from config.
One question would be if we really want to expose StrParser to the ValueSourceParser.
StrParser is a really quick hack I threw together (that's grown) and I could see it changing in the future (esp if we eventually implement an infix parser). Two ways of isolating the ValueSourceParser from the low level details of parsing I see are:
1) have a ValueSourceParser.createValueSource(List params), and the function parser would create the list
and pass it to the parser
2) keep the current style, and lock down the public APIs on FunctionQParser. Remove some of the details of parsing (like reading separators). So the following code from your patch
{code}
standardValueSourceParsers.put("max", new ValueSourceParser() {
public ValueSource parse(FunctionQParser fp) throws ParseException {
ValueSource source = fp.parseValSource();
fp.getStrParser().expect(",");
float val = fp.getStrParser().getFloat();
return new MaxFloatFunction(source,val);
}
});
{code}
Would look something more like
{code}
standardValueSourceParsers.put("max", new ValueSourceParser() {
public ValueSource parse(FunctionQParser fp) throws ParseException {
ValueSource source = fp.getValSource();
float val = fp.getFloat();
return new MaxFloatFunction(source,val);
}
});
{code}
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558525#action_12558525 ]
Hoss Man commented on SOLR-356:
-------------------------------
I'm not really following this issue, but in skimming the comments i just wanted to toss out the idea that the approach taken by the Lucene-java xml-query-parser contrib might make sense here.
it's got the same basic problem: support parsing and building of nested (query/function) structures where user configuration tells you which (query/function) name maps to which implementation.
the code may not be reusable in this case, but the pattern may be (not that i remember much about what the pattern was, just that it made a lot of sense when it was being fleshed out)
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: pluggableFunctions.patch, pluggableFunctions.patch
>
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-356) pluggable functions (value sources)
Posted by "Jon Pierce (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528896 ]
Jon Pierce commented on SOLR-356:
---------------------------------
A related thread on the list:
http://www.mail-archive.com/solr-user@lucene.apache.org/msg06073.html
http://www.nabble.com/pluggable-functions-tf4476995.html
> pluggable functions (value sources)
> -----------------------------------
>
> Key: SOLR-356
> URL: https://issues.apache.org/jira/browse/SOLR-356
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
>
> allow configuration of new value sources ot be created by the function query parser.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.