You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Ron Gonzalez <zl...@yahoo.com.INVALID> on 2018/04/21 03:01:54 UTC

Get application id when using SparkSubmit.main from java

Hi,  I am trying to get the application id after I use SparkSubmit.main for a yarn submission.  I am able to make it asynchronous using spark.yarn.watForCompletion=false configuration option, but I can't seem to figure out how I can get the application id for this job. I read both SparkSubmit.scala and Client.scala.  Any thoughts on how I could do it? I'd prefer not to use Client.run directly that returns the application id since there's a lot of environment prep that SparkSubmit actually does.  Thanks in advance for any help...
Thanks,Ron