[jira] [Created] (LIVY-978) Update the application tag generation logic in Livy

["Asif Khatri (Jira)" <ji...@apache.org>]

Asif Khatri created LIVY-978:
--------------------------------

             Summary: Update the application tag generation logic in Livy
                 Key: LIVY-978
                 URL: https://issues.apache.org/jira/browse/LIVY-978
             Project: Livy
          Issue Type: Bug
          Components: Server
    Affects Versions: 0.7.0
            Reporter: Asif Khatri
            Assignee: Asif Khatri


The following issue occurs:
{code:java}
ERROR org.apache.livy.utils.SparkYarnApp: Error whiling refreshing YARN state
java.lang.IllegalStateException: No YARN application is found with tag livy-batch-2-per2hlwa in 120 seconds. This may be because 1) spark-submit fail to submit application to YARN; or 2) YARN cluster doesn't have enough resources to start the application in time. Please check Livy log and YARN log to know the details.
{code}
if the below configuration parameter is set in YARN:
{code:java}
yarn.resourcemanager.application-tag-based-placement.force-lowercase=false
{code}
We can fix this in SparkYarnApp by updating app tag generation logic to generate appTag always in lowercase characters. SparkYarnApp code can be changed

from
{code:java}
val appTag = s"livy-session-$id-${Random.alphanumeric.take(8).mkString}"
{code}
to
{code:java}
val appTag = s"livy-session-$id-${Random.alphanumeric.take(8).mkString}".toLowerCase()
{code}
at [https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala#L92]




--
This message was sent by Atlassian Jira
(v8.20.10#820010)