You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Andrew Onischuk (JIRA)" <ji...@apache.org> on 2017/09/04 09:44:00 UTC
[jira] [Updated] (AMBARI-21872) Deploys failing with MR service
check failure due to missing mapreduce.tar.gz
[ https://issues.apache.org/jira/browse/AMBARI-21872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Onischuk updated AMBARI-21872:
-------------------------------------
Attachment: AMBARI-21872.patch
> Deploys failing with MR service check failure due to missing mapreduce.tar.gz
> -----------------------------------------------------------------------------
>
> Key: AMBARI-21872
> URL: https://issues.apache.org/jira/browse/AMBARI-21872
> Project: Ambari
> Issue Type: Bug
> Reporter: Andrew Onischuk
> Assignee: Andrew Onischuk
> Fix For: 3.0.0
>
> Attachments: AMBARI-21872.patch
>
>
> **Problem:**
>
>
>
> 2017-08-30 01:31:33,674 - ExecuteHadoop['jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-3.*.jar wordcount /user/ambari-qa/mapredsmokeinput /user/ambari-qa/mapredsmokeoutput'] {'bin_dir': '/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-client/bin', 'conf_dir': '/usr/hdp/current/hadoop-client/conf', 'logoutput': True, 'try_sleep': 5, 'tries': 1, 'user': 'ambari-qa'}
> 2017-08-30 01:31:33,675 - Execute['hadoop --config /usr/hdp/current/hadoop-client/conf jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-3.*.jar wordcount /user/ambari-qa/mapredsmokeinput /user/ambari-qa/mapredsmokeoutput'] {'logoutput': True, 'try_sleep': 5, 'environment': {}, 'tries': 1, 'user': 'ambari-qa', 'path': ['/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-client/bin']}
> ######## Hortonworks #############
> This is MOTD message, added for testing in qe infra
> 17/08/30 01:31:35 INFO client.RMProxy: Connecting to ResourceManager at ctr-e134-1499953498516-113587-01-000003.hwx.site/172.27.31.9:8050
> java.io.FileNotFoundException: File does not exist: hdfs://ctr-e134-1499953498516-113587-01-000003.hwx.site:8020/hdp/apps/3.0.0.0-229/mapreduce/mapreduce.tar.gz
> at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:137)
> at org.apache.hadoop.fs.AbstractFileSystem.resolvePath(AbstractFileSystem.java:480)
> at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2191)
> at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2187)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.resolve(FileContext.java:2193)
> at org.apache.hadoop.fs.FileContext.resolvePath(FileContext.java:598)
> at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:465)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:150)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1344)
>
> **Investigation:**
> After starting History Server, (which includes adding mapreduce.tar.gz to
> HDFS), it complains about HDFS_CLIENT not present.
>
>
> 2017-09-01 21:16:04,194 - Called copy_to_hdfs tarball: mapreduce
> 2017-09-01 21:16:04,194 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.0.0-229 -> 3.0.0.0-229
> 2017-09-01 21:16:04,194 - Tarball version was calcuated as 3.0.0.0-229. Use Command Version: True
> 2017-09-01 21:16:04,194 - HDFS_CLIENT is not installed on the host. Skip copying /usr/hdp/3.0.0.0-229/hadoop/mapreduce.tar.gz
>
> In Ambari-2.5.2 history server host does not have MR client/Hadoop client and
> has MR service check passed.
> **Solution:**
> Seems like Ambari checks if a certain component is installed on the cluster
> (like HDFS_CLIENT). However not only HDFS_CLIENT but namenode, datanode,
> nfsgateway, journalnode etc.
> would include tar.gz. We have to check instead if cluster has a service, but
> not that host has a component.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)