You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Peter Wicks (JIRA)" <ji...@apache.org> on 2019/06/17 16:06:00 UTC
[jira] [Updated] (NIFI-6175) Spark Livy - Add Support for All
Missing Session Features
[ https://issues.apache.org/jira/browse/NIFI-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Wicks updated NIFI-6175:
------------------------------
Description:
The Livy Session Controller is missing many of the options available, and many of them I feel are critical for this service to be useful (queue? conf? num of executors?)
* Would like to see all available options there, with a blanket "conf" option for users to provide custom configuration.
* When the controller service shuts down, sessions are left running, with no option to shut them down. Add in functionality to shutdown open sessions.
* If the controller service finds no Idle Livy Sessions, it will create a new session... until the queue runs out of resources :). Need to have a Min/Max/Should be elastic or strict option
* When Livy starts up, it searches for existing sessions, but does not verify that those sessions belong to it.
** The Kerberos identity should be used to verify the identity on the session matches the identity on the controller service.
** Also, if a Proxy user has been specified, that should also be verified. If no proxy user was specified, then the Proxy user on the Livy session should match the Kerberos identity.
was:
The Livy Session Controller is missing many of the options available, and many of them I feel are critical for this service to be useful (queue? conf? num of executors?)
Would like to see all available options there, with a blanket "conf" option for users to provide custom configuration.
When the controller service shuts down, sessions are left running, with no option to shut them down. Add in functionality to shutdown open sessions.
When Livy starts up, it searches for existing sessions, but does not verify that those sessions belong to it. The Kerberos identity should be used to verify the identity on the session matches the identity on the controller service.
> Spark Livy - Add Support for All Missing Session Features
> ---------------------------------------------------------
>
> Key: NIFI-6175
> URL: https://issues.apache.org/jira/browse/NIFI-6175
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.9.1
> Reporter: Peter Wicks
> Assignee: Peter Wicks
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The Livy Session Controller is missing many of the options available, and many of them I feel are critical for this service to be useful (queue? conf? num of executors?)
> * Would like to see all available options there, with a blanket "conf" option for users to provide custom configuration.
> * When the controller service shuts down, sessions are left running, with no option to shut them down. Add in functionality to shutdown open sessions.
> * If the controller service finds no Idle Livy Sessions, it will create a new session... until the queue runs out of resources :). Need to have a Min/Max/Should be elastic or strict option
> * When Livy starts up, it searches for existing sessions, but does not verify that those sessions belong to it.
> ** The Kerberos identity should be used to verify the identity on the session matches the identity on the controller service.
> ** Also, if a Proxy user has been specified, that should also be verified. If no proxy user was specified, then the Proxy user on the Livy session should match the Kerberos identity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)