[jira] [Commented] (LIVY-421) support multi statements run at the same time

["Jeff Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LIVY-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265911#comment-16265911 ] 

Jeff Zhang commented on LIVY-421:
---------------------------------

It would be better to run multiple interpreter in one spark application.  Each interpreter runs in one thread, otherwise you can not control the ordering of statements if supporting statements for one interpreter. 

> support multi statements run at the same time
> ---------------------------------------------
>
>                 Key: LIVY-421
>                 URL: https://issues.apache.org/jira/browse/LIVY-421
>             Project: Livy
>          Issue Type: Improvement
>          Components: REPL
>    Affects Versions: 0.4
>            Reporter: zhuweibin
>            Priority: Critical
>              Labels: performance
>
> In the v0.4.0-incubating's’ implementation, the number of threads used to execute the statement code is hardcoded to one. When idle resources are sufficient to execute multiple statements, a lot of resources are wasted and blocking statements run. We are considering improvements in one of two ways:
> # 1. Add a configuration in livy.conf to globally control the number of threads in interpreterExecutor for all repl/sessions to execute statements
> # 2 in the session to create a rest api parameter to set the maximum number of statements can be run at the same time the number of statements
> hope to discuss this issue, thanks



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)