You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2025/10/30 17:48:00 UTC

[jira] [Commented] (SPARK-54067) Improve `SparkSubmit` to invoke `exitFn` with the root cause instead of `SparkUserAppException`

    [ https://issues.apache.org/jira/browse/SPARK-54067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18034226#comment-18034226 ] 

Dongjoon Hyun commented on SPARK-54067:
---------------------------------------

I collected this as a subtask of SPARK-51727 in order to improve the visibility.

> Improve `SparkSubmit` to invoke `exitFn` with the root cause instead of `SparkUserAppException`
> -----------------------------------------------------------------------------------------------
>
>                 Key: SPARK-54067
>                 URL: https://issues.apache.org/jira/browse/SPARK-54067
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Declarative Pipelines
>    Affects Versions: 4.1.0
>            Reporter: Sanford Ryza
>            Assignee: Sanford Ryza
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.1.0
>
>
> I hit this when I ran a pipeline that had no flows:
> ```
> org.apache.spark.SparkUserAppException: User application exited with 1
> 	at org.apache.spark.deploy.PythonRunner$.main(PythonRunner.scala:127)
> 	at org.apache.spark.deploy.PythonRunner.main(PythonRunner.scala)
> 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
> 	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> 	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1028)
> 	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:203)
> 	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:226)
> 	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:95)
> 	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1166)
> 	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1175)
> 	at org.apache.spark.deploy.SparkPipelines$.main(SparkPipelines.scala:42)
> 	at org.apache.spark.deploy.SparkPipelines.main(SparkPipelines.scala)
> ```
> This is not information that's relevant to the user.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org