You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "caijialiang (Jira)" <ji...@apache.org> on 2023/05/06 02:20:00 UTC
[jira] [Updated] (AMBARI-25932) wrong config file name in spark service advisor
[ https://issues.apache.org/jira/browse/AMBARI-25932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
caijialiang updated AMBARI-25932:
---------------------------------
Description:
When using Spark 3.x, the "spark-defaults.conf" file is required to configure the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In Ambari 2.8, all configurations for the Spark service are in the "spark-defaults.conf" file. However, the service advisor is still based on Spark 2, so the default Spark configuration file needs to be changed from "spark2-default" to "spark-default".
In addition, in Ambari 2.8, the Spark 3 Thrift Server also uses "spark-defaults.conf" at startup, so the "spark2-thrift-sparkconf" configuration in the advisor needs to be deprecated.
was:When using Spark 3.x, the "spark-defaults.conf" file is required to configure the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In Ambari 2.8, all configurations for the Spark service are in the "spark-defaults.conf" file. However, the service advisor is still based on Spark 2, so the default Spark configuration file needs to be changed from "spark2-default" to "spark-default".
> wrong config file name in spark service advisor
> ------------------------------------------------
>
> Key: AMBARI-25932
> URL: https://issues.apache.org/jira/browse/AMBARI-25932
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.8.0
> Reporter: caijialiang
> Priority: Major
>
> When using Spark 3.x, the "spark-defaults.conf" file is required to configure the Spark cluster, instead of "spark-default.xml" or "spark2-default.xml". In Ambari 2.8, all configurations for the Spark service are in the "spark-defaults.conf" file. However, the service advisor is still based on Spark 2, so the default Spark configuration file needs to be changed from "spark2-default" to "spark-default".
> In addition, in Ambari 2.8, the Spark 3 Thrift Server also uses "spark-defaults.conf" at startup, so the "spark2-thrift-sparkconf" configuration in the advisor needs to be deprecated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ambari.apache.org
For additional commands, e-mail: issues-help@ambari.apache.org