[jira] [Commented] (HIVE-7516) Add capacity control over queries running on Spark cluster [Spark Branch]

["Brock Noland (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101204#comment-14101204 ] 

Brock Noland commented on HIVE-7516:
------------------------------------

Tez controls capacity to it's "default" queues but I think generally I think the cluster manager (YARN, MesOS, Standalone) should be responsible for capacity control.

Thus I think the work for this JIRA is to essentially add an internal queue of "default" sessions so that users who are not running in a kerberos environment or are using Sentry or Hive's builtin SQL authorization mechanism can avoid JVM initialization time.

However, I don't think this can proceed until we better understand what our API will look like for interacting with multiple SparkContexts in a single JVM.

> Add capacity control over queries running on Spark cluster [Spark Branch]
> -------------------------------------------------------------------------
>
>                 Key: HIVE-7516
>                 URL: https://issues.apache.org/jira/browse/HIVE-7516
>             Project: Hive
>          Issue Type: Improvement
>          Components: Spark
>            Reporter: Xuefu Zhang
>            Assignee: Venki Korukanti
>
> Add a capacity control mechanism in Hive to limit the number queries running on Spark concurrently which might overwhelm the cluster. Idea can be borrowed from Tez.



--
This message was sent by Atlassian JIRA
(v6.2#6252)