[jira] [Commented] (HIVE-9019) Avoid using SPARK_JAVA_OPTS [Spark Branch]

["Rui Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237833#comment-14237833 ] 

Rui Li commented on HIVE-9019:
------------------------------

Hi [~brocknoland], could you help to verify if this patch serves the purpose of HIVE-8998? Thanks!
I renamed the property because {{spark.executor.extraJavaOptions}} cannot take "spark" options:
{code}
// Validate spark.executor.extraJavaOptions
    settings.get(executorOptsKey).map { javaOpts =>
      if (javaOpts.contains("-Dspark")) {
        val msg = s"$executorOptsKey is not allowed to set Spark options (was '$javaOpts'). " +
          "Set them directly on a SparkConf or in a properties file when using ./bin/spark-submit."
        throw new Exception(msg)
      }
......
{code}

> Avoid using SPARK_JAVA_OPTS [Spark Branch]
> ------------------------------------------
>
>                 Key: HIVE-9019
>                 URL: https://issues.apache.org/jira/browse/HIVE-9019
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Rui Li
>            Assignee: Rui Li
>         Attachments: HIVE-9019.1-spark.patch
>
>
> SPARK_JAVA_OPTS has been deprecated, see {{SparkConf.validateSettings}}.
> Using it together with {{spark.driver.extraJavaOptions}} will cause SparkContext fail to start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)