You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Yonik Seeley (JIRA)" <ji...@apache.org> on 2007/09/08 18:34:29 UTC
[jira] Created: (SOLR-349) boost query by a function of other
fields
boost query by a function of other fields
-----------------------------------------
Key: SOLR-349
URL: https://issues.apache.org/jira/browse/SOLR-349
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
User should be able to boost a query by a function of other fields
Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) new functions for FunctionQuery
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527649 ]
Yonik Seeley commented on SOLR-349:
-----------------------------------
Being able to add new functions to the parser is probably useful to people... I opened SOLR-356 for this.
> new functions for FunctionQuery
> -------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) new functions for FunctionQuery
Posted by "Tom Hill (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527205 ]
Tom Hill commented on SOLR-349:
-------------------------------
It looks like you are removing the hard coded parsing of the functions in QueryParsing.java.
How hard would it be to allow user configured functions? We use several of our own functions, and currently have to hack up QueryParsing.java.
I'd much prefer to be able to add the classname to the config file, and have it just be picked up and added to vsParsers.
> new functions for FunctionQuery
> -------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525935 ]
Yonik Seeley commented on SOLR-349:
-----------------------------------
This issue is more for a top-down approach (less concerned with underlying implementation, more with just making it possible via http get).
- We need ways to add multiple fields together, multiply fields together, etc.
- We need a scaling function... for example, the range of all the components may not be known, but one may want the final value to be between 1 and 2. For something like this to be done efficiently, it seems like a min and max would be needed on ValueSource? Barring that, this value source would need to iterate over *all* documents and scale appropriately. Perhaps do the latter for the first implementation and we could always optimize later?
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-349) new functions for FunctionQuery
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley resolved SOLR-349.
-------------------------------
Resolution: Fixed
Added pow(a,b) and div(a,b) and committed.
Still open for discussion+changes of course.
> new functions for FunctionQuery
> -------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-349:
------------------------------
Attachment: FunctionQuery.patch
Attatching a new version of the patch with tests.
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526340 ]
Yonik Seeley commented on SOLR-349:
-----------------------------------
OK, my next step is to do tests for all these new functions provided no one sees an issue with the general approach.
I chose to stick with VaueSource as the basis for new functions rather than CustomScoreQuery because of greater complexities with weights, scorers, explanations, etc, and performance issues wrt scorers (skipTo, next needing to know deleted docs).
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525945 ]
Yonik Seeley commented on SOLR-349:
-----------------------------------
Yes, I suppose that sum() makes more sense... and ValueSources are so light weight it shouldn't really cause a performance difference. Although I think perhaps a constant should be allowed in sum(), otherwise people will be trying the very reasonable sum(a,1) and it won't work. Of course we could make a ConstantValueSource, so constants and value-sources would be interchangeable...
For "mult", I was thinking "product".
Also perhaps a map() to pseudo-handle default values.
product(x, map(y,0,1)) // map all zeros to 1 before multiplying
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) boost query by a function of other
fields
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525940 ]
Hoss Man commented on SOLR-349:
-------------------------------
the idea that funcname(a,b,c,d,e,f,g,....) maps to a*b + 1*c + d*e + f*g + ... seems a little weird, even weirder that funcname is "linear" which in theory only has 3 inputs.
wouldn't a new "sum(....)" function make more sense? ...
sum(linear(a,2,3), linear(b,4,0), linear(c,5,0))
(the key being that "sum" takes in a list of other ValueSources, while linear still takes only 1 and some constants)
we'd also need a "mult" function right?
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-349:
------------------------------
Attachment: linear_combination.patch
This patch extends the linear function to include other value sources.
so instead of
linear(a,2,3) // a*2+3
you can do
linear(a,2,3,b,4,c,5) // a*2+3+b*4+c*5
Everyone OK with extending LinearFloatFunction this way? It seemed like the simplest way to get this capability.
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-349) new functions for FunctionQuery
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-349:
------------------------------
Summary: new functions for FunctionQuery (was: boost query by a function of other fields)
> new functions for FunctionQuery
> -------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525938 ]
Yonik Seeley commented on SOLR-349:
-----------------------------------
And yes, the syntax is less than ideal.... it would be nice to extend the parser to accept infix, but that's a different issue for another day.
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-349) boost query by a function of other
fields
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-349:
------------------------------
Attachment: FunctionQuery.patch
Attaching functionquery.patch... adding many new functions:
log, sqrt, abs, sum, product, const
scale(source, min, max) // scale values from source to fall between min and max
map(source,min,max,target) // change any values that fall between min&max to target
query(lucene_query, defaultVal) // use relevancy score, w/ defaultVal when doc doesn't match
I also changed the function parser to use a hash lookup on function name since the list was getting long. The parser will now also accept a float constant as a value source, allowing things like add(a,2)
This is just a snapshot for feedback... no tests yet.
> boost query by a function of other fields
> -----------------------------------------
>
> Key: SOLR-349
> URL: https://issues.apache.org/jira/browse/SOLR-349
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: FunctionQuery.patch, linear_combination.patch
>
>
> User should be able to boost a query by a function of other fields
> Some background: http://www.nabble.com/boosting-a-query-by-a-function-of-other-fields-tf4387856.html#a12510092
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.