You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2015/06/30 00:09:06 UTC
[jira] [Commented] (AMBARI-12202) RU: YARN daemons started by
Ambari have invalid classpath entries
[ https://issues.apache.org/jira/browse/AMBARI-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606518#comment-14606518 ]
Hudson commented on AMBARI-12202:
---------------------------------
SUCCESS: Integrated in Ambari-trunk-Commit #3033 (See [https://builds.apache.org/job/Ambari-trunk-Commit/3033/])
AMBARI-12202 - RU: YARN daemons started by Ambari have invalid classpath entries (jonathanhurley) (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=5afeaea3bcf567356fbfb28c88edea41f58f00fc)
* ambari-server/src/test/java/org/apache/ambari/server/checks/AbstractCheckDescriptorTest.java
* ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
* ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java
* ambari-server/src/main/java/org/apache/ambari/server/checks/UpgradeCheckGroup.java
* ambari-server/src/main/java/org/apache/ambari/server/checks/ServicesUpCheck.java
* ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
* ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
* ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml
* ambari-server/src/main/java/org/apache/ambari/server/state/stack/PrerequisiteCheck.java
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml
* ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
* ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
> RU: YARN daemons started by Ambari have invalid classpath entries
> -----------------------------------------------------------------
>
> Key: AMBARI-12202
> URL: https://issues.apache.org/jira/browse/AMBARI-12202
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.1.0
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Critical
> Fix For: 2.1.0
>
>
> After a rolling upgrade, there are invalid entries on the classpath for YARN:
> ResourceManager's command-line
> {code}
> yarn 9718 3.0 3.0 1876080 468084 ? Sl 19:56 2:07 /usr/jdk64/jdk1.7.0_67/bin/java -Dproc_resourcemanager -Xmx1024m -Dhdp.version=2.3.0.0-2410 -Dhadoop.log.dir=/grid/0/log/hadoop/yarn -Dyarn.log.dir=/grid/0/log/hadoop/yarn -Dhadoop.log.file=yarn-yarn-resourcemanager-ip-172-31-39-58.log -Dyarn.log.file=yarn-yarn-resourcemanager-ip-172-31-39-58.log -Dyarn.home.dir= -Dyarn.id.str=yarn -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.3.0.0-2410/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.0.0-2410/hadoop/lib/native:/grid/0/hdp/2.3.0.0-2410/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.0.0-2410/hadoop/lib/native -Dyarn.policy.file=hadoop-policy.xml -Dyarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY -Dhadoop.log.dir=/grid/0/log/hadoop/yarn -Dyarn.log.dir=/grid/0/log/hadoop/yarn -Dhadoop.log.file=yarn-yarn-resourcemanager-ip-172-31-39-58.log -Dyarn.log.file=yarn-yarn-resourcemanager-ip-172-31-39-58.log -Dyarn.home.dir=/usr/hdp/current/hadoop-yarn-resourcemanager -Dhadoop.home.dir=/grid/0/hdp/2.3.0.0-2410/hadoop -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.3.0.0-2410/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.0.0-2410/hadoop/lib/native:/grid/0/hdp/2.3.0.0-2410/hadoop/lib/native/Linux-amd64-64:/grid/0/hdp/2.3.0.0-2410/hadoop/lib/native -classpath /usr/hdp/2.3.0.0-2410/hadoop/conf:/usr/hdp/2.3.0.0-2410/hadoop/conf:/usr/hdp/2.3.0.0-2410/hadoop/conf:/usr/hdp/2.3.0.0-2410/hadoop/lib/*:/usr/hdp/2.3.0.0-2410/hadoop/.//*:/usr/hdp/2.3.0.0-2410/hadoop-hdfs/./:/usr/hdp/2.3.0.0-2410/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2410/hadoop-hdfs/.//*:/usr/hdp/2.3.0.0-2410/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2410/hadoop-yarn/.//*:/usr/hdp/2.3.0.0-2410/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2410/hadoop-mapreduce/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/tez-client/*:/usr/hdp/current/tez-client/lib/*:/etc/tez/conf/:/usr/hdp/2.3.0.0-2410/tez/*:/usr/hdp/2.3.0.0-2410/tez/lib/*:/etc/tez/conf:/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/tez-client/*:/usr/hdp/current/tez-client/lib/*:/etc/tez/conf/:/usr/hdp/2.3.0.0-2410/tez/*:/usr/hdp/2.3.0.0-2410/tez/lib/*:/etc/tez/conf:/usr/hdp/current/hadoop-yarn-resourcemanager/.//*:/usr/hdp/current/hadoop-yarn-resourcemanager/lib/*:/usr/hdp/2.3.0.0-2410/hadoop/conf/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
> {code}
> Invalid entries I see
> - /usr/hdp/2.3.0.0-2410/hadoop/conf,: Three (duplicate) entries for
> - /usr/hdp/current/hadoop-mapreduce-client/*
> - /usr/hdp/current/tez-client/*: 2 entries
> - /usr/hdp/current/tez-client/lib/*: 2 entries
> - /etc/tez/conf/: 4 entries
> - /usr/hdp/current/hadoop-mapreduce-client/*
> - /usr/hdp/current/hadoop-yarn-resourcemanager/.//*
> - /usr/hdp/current/hadoop-yarn-resourcemanager/lib/*
> Our {{hadoop-env.sh}} is sourcing the wrong mapr and tez:
> {code:title=hadoop-env.xml}
> #Add libraries required by nodemanager
> MAPREDUCE_LIBS={{mapreduce_libs_path}}
> export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}${JAVA_JDBC_LIBS}:${MAPREDUCE_LIBS}
> if [ -d "/usr/lib/tez" ]; then
> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/lib/tez/*:/usr/lib/tez/lib/*:/etc/tez/conf
> fi
> # Setting path to hdfs command line
> export HADOOP_LIBEXEC_DIR={{hadoop_libexec_dir}}
> {code}
> I see two issues here:
> - {{MAPREDUCE_LIBS}} is set to {{mapreduce_libs_path = "/usr/hdp/current/hadoop-mapreduce-client/*"}}
> - Tez is hard coded
> Both of these can be fixed; the question is do we need to fix them for Ambari 2.1?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)