[jira] [Resolved] (SPARK-44242) Spark job submission failed because Xmx string is available on one parameter provided into spark.driver.extraJavaOptions

["Mridul Muralidharan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-44242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mridul Muralidharan resolved SPARK-44242.
-----------------------------------------
    Fix Version/s: 4.0.0
       Resolution: Fixed

Issue resolved by pull request 41806
[https://github.com/apache/spark/pull/41806]

> Spark job submission failed because Xmx string is available on one parameter provided into spark.driver.extraJavaOptions
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-44242
>                 URL: https://issues.apache.org/jira/browse/SPARK-44242
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Submit
>    Affects Versions: 3.3.2, 3.4.1
>            Reporter: Nicolas Fraison
>            Assignee: Nicolas Fraison
>            Priority: Major
>             Fix For: 4.0.0
>
>
> The spark-submit command failed if Xmx string is found on any parameters provided to spark.driver.extraJavaOptions.
> For ex. running this spark-submit command line
> {code:java}
> ./bin/spark-submit --class org.apache.spark.examples.SparkPi --conf "spark.driver.extraJavaOptions=-Dtest=Xmx"  examples/jars/spark-examples_2.12-3.4.1.jar 100{code}
> failed due to
> {code:java}
> Error: Not allowed to specify max heap(Xmx) memory settings through java options (was -Dtest=Xmx). Use the corresponding --driver-memory or spark.driver.memory configuration instead.{code}
> The check performed in [https://github.com/apache/spark/blob/master/launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java#L314] seems to broad



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org