[jira] [Commented] (LIVY-395) Run statements in parallel in a session

["Hong Shen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LIVY-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210482#comment-16210482 ] 

Hong Shen commented on LIVY-395:
--------------------------------

We also need this feature in our cluster, think about that, if a app have to take a long time and large resource to init,  after init, we have a lot of statement to run, it sounds good if we can run statements in parallel.


> Run statements in parallel in a session
> ---------------------------------------
>
>                 Key: LIVY-395
>                 URL: https://issues.apache.org/jira/browse/LIVY-395
>             Project: Livy
>          Issue Type: Bug
>          Components: API
>    Affects Versions: 0.3
>            Reporter: kaushik ranjan
>
> Is there an option in Livy, similar to spark JobServer where we can run statements in an interactive session in parallel?
> Apache JobServer has a option to specify the number of requests which we want to run in parallel. Do we have something similar in Livy?
> Microsoft Azure HDI support team say - we can run as many statements in a session in parallel as are the number of cores in a headnode in the HDI cluster. But when we tested - it doesn't.
> Right now we see bad utilization of resources within a spark session (following interactive session model). Right now we have session per user model. I want to use the session for multiple users, but Livy's model of running only one statement at a time, is acting as a blocker for allowing multiple users to use the same session to run requests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)