[GitHub] [lucene-solr] treygrainger opened a new pull request #941: SOLR-13836: Add 'streaming_expression' QParser

[GitBox <gi...@apache.org>]

treygrainger opened a new pull request #941: SOLR-13836: Add 'streaming_expression' QParser
URL: https://github.com/apache/lucene-solr/pull/941
 
 
   # Description
   
   It is currently possible to hit the search handler in a streaming expression ("search(...)"), but it is not currently possible to invoke a streaming expression from within a regular search within the search handler. In some cases, it would be useful to leverage the power of streaming expressions to generate a result set and then join that result set with a normal set of search results. This likely won't be particularly efficient for high cardinality streaming expression results, but it will be pretty powerful feature that could enable a bunch of use cases that aren't possible today within a normal search.
   
   See https://issues.apache.org/jira/browse/SOLR-13836 for usage information.
   
   # Solution
   
   The current solution adds a StreamingExpressionQParserPlugin which executes a streaming expression and joins the tuples returned on an id field with the main docset. The field name from the streaming expression tuples can be overridden ("f" param), as well as the method of joining ("method" parameter). 
   
   # Usage
   *Docs:*
   
   ```
   curl -X POST -H "Content-Type: application/json" http://localhost:8983/solr/food_collection/update?commit=true  --data-binary '
   [
   {"id": "1", "name_s":"donut","vector_fs":[5.0,0.0,1.0,5.0,0.0,4.0,5.0,1.0]},
   {"id": "2", "name_s":"apple juice","vector_fs":[1.0,5.0,0.0,0.0,0.0,4.0,4.0,3.0]},
   {"id": "3", "name_s":"cappuccino","vector_fs":[0.0,5.0,3.0,0.0,4.0,1.0,2.0,3.0]},
   {"id": "4", "name_s":"cheese pizza","vector_fs":[5.0,0.0,4.0,4.0,0.0,1.0,5.0,2.0]},
   {"id": "5", "name_s":"green tea","vector_fs":[0.0,5.0,0.0,0.0,2.0,1.0,1.0,5.0]},
   {"id": "6", "name_s":"latte","vector_fs":[0.0,5.0,4.0,0.0,4.0,1.0,3.0,3.0]},
   {"id": "7", "name_s":"soda","vector_fs":[0.0,5.0,0.0,0.0,3.0,5.0,5.0,0.0]},
   {"id": "8", "name_s":"cheese bread sticks","vector_fs":[5.0,0.0,4.0,5.0,0.0,1.0,4.0,2.0]},
   {"id": "9", "name_s":"water","vector_fs":[0.0,5.0,0.0,0.0,0.0,0.0,0.0,5.0]},
   {"id": "10", "name_s":"cinnamon bread sticks","vector_fs":[5.0,0.0,1.0,5.0,0.0,3.0,4.0,2.0]}
   ]
   ```
   
    
   
   *Query:*
   ```
   http://localhost:8983/solr/food/select?q=*:*&fq=\{!streaming_expression}top(select(search(food,%20q=%22*:*%22,%20fl=%22id,vector_fs%22,%20sort=%22id%20asc%22),%20cosineSimilarity(vector_fs,%20array(5.1,0.0,1.0,5.0,0.0,4.0,5.0,1.0))%20as%20cos,%20id),%20n=5,%20sort=%22cos%20desc%22)&fl=id,name_s
   ```
   
    
   
   *Response:*
   ```
   {
     "responseHeader":{
       "zkConnected":true,
       "status":0,
       "QTime":7,
       "params":{
         "q":"*:*",
         "fl":"id,name_s",
         "fq":"{!streaming_expression}top(select(search(food, q=\"*:*\", fl=\"id,vector_fs\", sort=\"id asc\"), cosineSimilarity(vector_fs, array(5.2,0.0,1.0,5.0,0.0,4.0,5.0,1.0)) as cos, id), n=5, sort=\"cos desc\")"}},
     "response":{"numFound":5,"start":0,"docs":[
         {
           "name_s":"donut",
           "id":"1"},
         {
           "name_s":"apple juice",
           "id":"2"},
         {
           "name_s":"cheese pizza",
           "id":"4"},
         {
           "name_s":"cheese bread sticks",
           "id":"8"},
         {
           "name_s":"cinnamon bread sticks",
           "id":"10"}]
     }}
   ```
   
   # Tests
   
   No tests written yet. First draft.
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability.
   - [x] I have created a Jira issue and added the issue ID to my pull request title.
   - [x] I am authorized to contribute this code to the ASF and have removed any code I do not have a license to distribute.
   - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
   - [x] I have developed this patch against the `master` branch.
   - [ ] I have run `ant precommit` and the appropriate test suite.
   - [ ] I have added tests for my changes.
   - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org