[jira] [Commented] (SOLR-9559) Add ExecutorStream to execute stored Streaming Expressions

["David Smiley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603936#comment-15603936 ] 

David Smiley commented on SOLR-9559:
------------------------------------

Hi Joel. What do you think about naming this {{eval}} instead?  That name seems more congruent with the name & purpose of the eval() method in various programming environments.  You are very close to the code so I can see how ExecutorStream came to your mind in light of it using an ExecutorService underneath.

I noticed that {{StreamTask}} loops over the tuples and does nothing with the result.  Why is that?  And might you use Java 7 try-with-resources over there?

I admit I'm a little confused as to the use-case -- why would someone embed a streaming expression be embedded in a tuple?  Perhaps some sort of persistent distributed work queue?  But then how are error conditions handled... do we concern ourselves with not running the same expression multiple times?

> Add ExecutorStream to execute stored Streaming Expressions
> ----------------------------------------------------------
>
>                 Key: SOLR-9559
>                 URL: https://issues.apache.org/jira/browse/SOLR-9559
>             Project: Solr
>          Issue Type: New Feature
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Joel Bernstein
>            Assignee: Joel Bernstein
>             Fix For: 6.3
>
>         Attachments: SOLR-9559.patch, SOLR-9559.patch, SOLR-9559.patch, SOLR-9559.patch
>
>
> The *ExecutorStream* will wrap a stream which contains Tuples with Streaming Expressions to execute. By default the ExecutorStream will look for the expression in the *expr_s* field in the Tuples.
> The ExecutorStream will have an internal thread pool so expressions can be executed in parallel on a single worker. The ExecutorStream can also be wrapped by the parallel function to partition the Streaming Expressions that need to be executed across a cluster of worker nodes.
> *Sample syntax*:
> {code}
> daemon(executor(threads=10, topic(storedExpressions, fl="expr_s", ...)))
> {code}
> In the example above a *daemon* wraps an *executor* which wraps a *topic* that is reading stored Streaming Expressions. The daemon will call the executor at intervals which will execute all the expressions retrieved by the topic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org