[jira] [Closed] (SOLR-6398) Add IterativeMergeStrategy to support running Parallel Iterative Algorithms inside of Solr

["Joel Bernstein (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joel Bernstein closed SOLR-6398.
--------------------------------
    Resolution: Fixed

> Add IterativeMergeStrategy to support running Parallel Iterative Algorithms inside of Solr
> ------------------------------------------------------------------------------------------
>
>                 Key: SOLR-6398
>                 URL: https://issues.apache.org/jira/browse/SOLR-6398
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Joel Bernstein
>            Priority: Minor
>             Fix For: Trunk
>
>         Attachments: SOLR-6398.patch, SOLR-6398.patch, SOLR-6398.patch, SOLR-6398.patch
>
>
> This ticket builds on the existing AnalyticsQuery / MergeStrategy framework by adding the abstract class IterativeMergeStrategy,  which has built-in support for call-backs to the shards. The IterativeMergeStrategy is designed to support the execution of Parallel iterative Algorithms, such as Gradient Descent, inside of Solr.
> To use the IterativeMergeStrategy you extend it and implement process(). This gives you access to the callback() method which makes it easy to make repeated calls to all the shards and run algorithms that require iteration.
> Below is an example of a class that extends IterativeMergeStrategy. In this example it collects the *count* from the shards and then calls back to shards executing the *!count* AnalyticsQuery and sending it merged counts from all the shards. 
> {code}
> class TestIterative extends IterativeMergeStrategy  {
>     public void process(ResponseBuilder rb, ShardRequest sreq) throws Exception {
>       int count = 0;
>       for(ShardResponse shardResponse : sreq.responses) {
>         NamedList response = shardResponse.getSolrResponse().getResponse();
>         NamedList analytics = (NamedList)response.get("analytics");
>         Integer c = (Integer)analytics.get("mycount");
>         count += c.intValue();
>       }
>       ModifiableSolrParams params = new ModifiableSolrParams();
>       params.add("distrib", "false");
>       params.add("fq","{!count base="+count+"}");
>       params.add("q","*:*");
>       /*
>       *  Call back to all the shards in the response and process the result.
>        */
>       QueryRequest request = new QueryRequest(params);
>       List<Future<CallBack>> futures = callBack(sreq.responses, request);
>       int nextCount = 0;
>       for(Future<CallBack> future : futures) {
>         QueryResponse response = future.get().getResponse();
>         NamedList analytics = (NamedList)response.getResponse().get("analytics");
>         Integer c = (Integer)analytics.get("mycount");
>         nextCount += c.intValue();
>       }
>       NamedList merged = new NamedList();
>       merged.add("mycount", nextCount);
>       rb.rsp.add("analytics", merged);
>     }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org