[jira] [Updated] (AMBARI-21572) Spark configuration mismatch post express upgrade

["Doroszlai, Attila (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-21572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doroszlai, Attila updated AMBARI-21572:
---------------------------------------
    Attachment: AMBARI-21572.patch

> Spark configuration mismatch post express upgrade
> -------------------------------------------------
>
>                 Key: AMBARI-21572
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21572
>             Project: Ambari
>          Issue Type: Bug
>          Components: service-advisor
>    Affects Versions: 2.5.2
>            Reporter: Eric Yang
>            Assignee: Doroszlai, Attila
>             Fix For: 2.5.2
>
>         Attachments: AMBARI-21572.patch
>
>
> During upgrade from IOP 4.2.x to HDP 2.6.2, there is a warning with the spark property spark.yarn.archive the value is:
> hdfs:///iop/apps/4.2.5.0-0000/spark2/spark2-iop-yarn-archive.tar.gz prior to the EU
> hdfs:///*hdp*/apps/4.2.5.0-0000/spark2/spark2-iop-yarn-archive.tar.gz post EU
> If this property isn't used by HDP it should be deleted during the stack upgrade. If it is used by HDP then the version number 4.2.5.0-0000 should be updated to a param.
> Spark2 alert had issues connecting to the thrift server post migration where it tried to ping port 4038 (defined by spark.thriftserver.ui.port, carried over from IOP before migration) while the process really was running on 4040. I got the alerts working again by updating *spark.thriftserver.ui.port* to 4040.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)