You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Andrew Onischuk (JIRA)" <ji...@apache.org> on 2017/09/04 09:44:00 UTC
[jira] [Created] (AMBARI-21872) Deploys failing with MR service
check failure due to missing mapreduce.tar.gz
Andrew Onischuk created AMBARI-21872:
----------------------------------------
Summary: Deploys failing with MR service check failure due to missing mapreduce.tar.gz
Key: AMBARI-21872
URL: https://issues.apache.org/jira/browse/AMBARI-21872
Project: Ambari
Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
Fix For: 3.0.0
**Problem:**
2017-08-30 01:31:33,674 - ExecuteHadoop['jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-3.*.jar wordcount /user/ambari-qa/mapredsmokeinput /user/ambari-qa/mapredsmokeoutput'] {'bin_dir': '/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-client/bin', 'conf_dir': '/usr/hdp/current/hadoop-client/conf', 'logoutput': True, 'try_sleep': 5, 'tries': 1, 'user': 'ambari-qa'}
2017-08-30 01:31:33,675 - Execute['hadoop --config /usr/hdp/current/hadoop-client/conf jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-3.*.jar wordcount /user/ambari-qa/mapredsmokeinput /user/ambari-qa/mapredsmokeoutput'] {'logoutput': True, 'try_sleep': 5, 'environment': {}, 'tries': 1, 'user': 'ambari-qa', 'path': ['/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-client/bin']}
######## Hortonworks #############
This is MOTD message, added for testing in qe infra
17/08/30 01:31:35 INFO client.RMProxy: Connecting to ResourceManager at ctr-e134-1499953498516-113587-01-000003.hwx.site/172.27.31.9:8050
java.io.FileNotFoundException: File does not exist: hdfs://ctr-e134-1499953498516-113587-01-000003.hwx.site:8020/hdp/apps/3.0.0.0-229/mapreduce/mapreduce.tar.gz
at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:137)
at org.apache.hadoop.fs.AbstractFileSystem.resolvePath(AbstractFileSystem.java:480)
at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2191)
at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2187)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.resolve(FileContext.java:2193)
at org.apache.hadoop.fs.FileContext.resolvePath(FileContext.java:598)
at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:465)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:150)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1344)
**Investigation:**
After starting History Server, (which includes adding mapreduce.tar.gz to
HDFS), it complains about HDFS_CLIENT not present.
2017-09-01 21:16:04,194 - Called copy_to_hdfs tarball: mapreduce
2017-09-01 21:16:04,194 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.0.0-229 -> 3.0.0.0-229
2017-09-01 21:16:04,194 - Tarball version was calcuated as 3.0.0.0-229. Use Command Version: True
2017-09-01 21:16:04,194 - HDFS_CLIENT is not installed on the host. Skip copying /usr/hdp/3.0.0.0-229/hadoop/mapreduce.tar.gz
In Ambari-2.5.2 history server host does not have MR client/Hadoop client and
has MR service check passed.
**Solution:**
Seems like Ambari checks if a certain component is installed on the cluster
(like HDFS_CLIENT). However not only HDFS_CLIENT but namenode, datanode,
nfsgateway, journalnode etc.
would include tar.gz. We have to check instead if cluster has a service, but
not that host has a component.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)