[jira] [Commented] (LIVY-1061) Add livy.server.app-name-prefix configuration for interactive session YARN application names

["Gyorgy Gal (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LIVY-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099766#comment-18099766 ] 

Gyorgy Gal commented on LIVY-1061:
----------------------------------

PR: https://github.com/apache/livy/pull/534

> Add livy.server.app-name-prefix configuration for interactive session YARN application names
> --------------------------------------------------------------------------------------------
>
>                 Key: LIVY-1061
>                 URL: https://issues.apache.org/jira/browse/LIVY-1061
>             Project: Livy
>          Issue Type: Improvement
>            Reporter: Nilesh Rathi
>            Assignee: Arun Kumar M
>            Priority: Major
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently, Apache Livy assigns a hardcoded YARN application name to interactive Spark sessions when no user-provided name is supplied in the session creation request. The default name follows the format {{livy-session-<id>}}, and there is no server-level configuration to override or customize this prefix.
> This creates a limitation for cluster administrators, especially in High Availability (HA) or multi-cluster environments, where Livy sessions launched across different Livy server instances or clusters all appear with identical naming patterns in the YARN Resource Manager UI. Operations and monitoring teams need a way to distinguish Livy-managed applications at the cluster or environment level without requiring every client to pass a custom name in each REST API request.
> h3. Current Behavior
> * *Interactive sessions:* When a client creates a session via {{POST /sessions}} without specifying a {{name}} field, Livy sets {{spark.app.name}} to {{livy-session-<id>}} (in {{InteractiveSession.scala}}).
> * *Batch sessions:* Users can already set the application name via the {{name}} field in {{POST /batches}} requests.
> h3. Expected Behavior
> Add a new Livy server configuration property, {{livy.server.app-name-prefix}}, that allows administrators to define a custom prefix for YARN application names of interactive sessions.
> * When unset (default): retain existing behavior — application names remain {{livy-session-<id>}}.
> * When set (e.g., {{livy.server.app-name-prefix=my-cluster}}): interactive session YARN application names use the configured prefix — {{my-cluster-<id>}}.
> h3. Scope
> * *Interactive sessions only.* Batch sessions already support per-request naming via the REST API {{name}} field; changing batch behavior could break existing automation or monitoring that relies on current naming conventions.
> h3. Use Cases
> * HA deployments with multiple Livy server instances where administrators need to identify which cluster or environment a session belongs to in the YARN RM UI.
> * Shared YARN clusters where ops teams want a consistent, cluster-specific prefix for all Livy interactive workloads.
> * Monitoring and alerting systems that filter or group YARN applications by a configurable prefix.
> h3. Proposed Implementation
> * Add {{livy.server.app-name-prefix}} to {{LivyConf}}.
> * In {{InteractiveSession.create()}}, use the configured prefix (if set) instead of the hardcoded {{livy-session}} when building the default {{spark.app.name}}.
> * Preserve backward compatibility: when the property is not configured, behavior is unchanged.
> h3. Example
> {code:bash}
> # livy.conf
> livy.server.app-name-prefix = my-cluster
> # Create an interactive session
> curl -X POST -H 'Content-Type: application/json' \
>   -d '{"kind":"spark"}' \
>   http://livy-host:8998/sessions
> # YARN RM shows application name: my-cluster-3
> yarn application -list | grep my-cluster
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)