[jira] [Updated] (LIVY-428) Does livy support concurrency for centain spark session?

["suheng.cloud (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LIVY-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

suheng.cloud updated LIVY-428:
------------------------------
    Issue Type: Bug  (was: Improvement)

> Does livy support concurrency for centain spark session?
> --------------------------------------------------------
>
>                 Key: LIVY-428
>                 URL: https://issues.apache.org/jira/browse/LIVY-428
>             Project: Livy
>          Issue Type: Bug
>          Components: API
>    Affects Versions: 0.4
>         Environment: livy 0.4
> spark2.2.0-cloudera1
>            Reporter: suheng.cloud
>         Attachments: spark-fair.png
>
>
> To use spark sql concurrency for internal fair schedule
> in [post] /sessions/{id}/statements use param "conf":
> {"spark.scheduler.mode":"FAIR","spark.scheduler.allocation.file":"/home/.../spark-schedule.xml"}
> spark-schedule.xml is as follows:
> <allocations>
>   <pool name="FAIR">
>     <schedulingMode>FAIR</schedulingMode>
>   </pool>
> </allocations>
> Log shows tow jobs surely be added into fair pool:
> INFO scheduler.FairSchedulableBuilder: Added task set TaskSet_10.0 tasks to pool FAIR
> ...
> INFO scheduler.FairSchedulableBuilder: Added task set TaskSet_11.0 tasks to pool FAIR
> But from spark web console the second job starts only when the first finished.
> I also tried with zeppelin, which concurrency works well with spark but not work in livy.
> Did I miss some configuration? Thanks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)