[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

["BELUGA BEHR (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838224#comment-15838224 ] 

BELUGA BEHR commented on HIVE-14162:
------------------------------------

How about some sort of idle Spark timeout value that can be configured in the session?

If the Spark context is unused for X amount of time, then it is closed and the ApplicationMaster/Executor are stopped.  This will be good for performance of running several ad-hoc queries in short time frames (beeline/hue) but will return the cluster resources when the user goes out for lunch or goes home for the day.

> Allow disabling of long running job on Hive On Spark On YARN
> ------------------------------------------------------------
>
>                 Key: HIVE-14162
>                 URL: https://issues.apache.org/jira/browse/HIVE-14162
>             Project: Hive
>          Issue Type: New Feature
>          Components: Spark
>            Reporter: Thomas Scott
>            Assignee: Aihua Xu
>            Priority: Minor
>         Attachments: HIVE-14162.1.patch
>
>
> Hive On Spark launches a long running process on the first query to handle all queries for that user session. In some use cases this is not desired, for instance when using Hue with large intervals between query executions.
> Could we have a property that would cause long running spark jobs to be terminated after each query execution and started again for the next one?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)