[jira] [Resolved] (AMBARI-25551) NodeManager doesn't work after adding a new host to a patch upgraded cluster

["Dmitry Lysnichenko (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-25551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitry Lysnichenko resolved AMBARI-25551.
-----------------------------------------
      Assignee: Dmitry Lysnichenko
    Resolution: Fixed

> NodeManager doesn't work after adding a new host to a patch upgraded cluster
> ----------------------------------------------------------------------------
>
>                 Key: AMBARI-25551
>                 URL: https://issues.apache.org/jira/browse/AMBARI-25551
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.7.5
>            Reporter: Dmytro Vitiuk
>            Assignee: Dmitry Lysnichenko
>            Priority: Major
>             Fix For: 2.7.6
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> By default "mapreduce.admin.user.env" property for mapreduce2 and "tez.am.launch.env"/"tez.task.launch.env" for tez contains a path with a specific hdp version, like the next one:
> {code}
> LD_LIBRARY_PATH=/usr/hdp/3.1.5.xx-x/hadoop/lib/native:/usr/hdp/3.1.5.xx-x/hadoop/lib/native/Linux-{{architecture}}-64
> {code}
> As result after patch upgrade (for Tez, MR2, YARN) tez/mapreduce jobs point to paths with old versions. And in the case we add new hosts with NodeManager only (without any client) we don't have these libraries by the old paths. So I am going to change the default value to the next one:
> {code}
> /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-{{architecture}}-64
> {code}
> This path is always exists (because NodeManager installs hadoop-client package as dependency).
> Also there is an issue with yarn-shuffle.jar path selection. By default we choose the spark client's version in the case a cluster has at least one spark client at any host:
> {code}
>  /usr/hdp/<sparkClientHDPVersion>/spark2/aux/spark-*-yarn-shuffle.jar
> {code}
> But this way stops working in the case we have hosts without spark client. I am going to use yarn's yarn-shuffle.jar path for the same hosts:
> {code}
> /usr/hdp/<yarnClientHDPVersion>/spark2/aux/spark-*-yarn-shuffle.jar
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)