You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Krisztian Horvath <kr...@sequenceiq.com> on 2015/03/18 17:19:22 UTC
MR2 smoke test and all jobs fail
Hi,
Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails with the following error: java.io.FileNotFoundException: File does not exist: hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz . Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every job fails with the same error. Does any1 know about this issue?
Br,
Krisz
Re: MR2 smoke test and all jobs fail
Posted by Krisztian Horvath <kr...@sequenceiq.com>.
Adding the MR2 client does not solve it.
On March 18, 2015 at 6:20:52 PM, Krisztian Horvath (krisztian.horvath@sequenceiq.com) wrote:
As I see from the log during history server start: Could not verify HDP version by calling '/usr/bin/hdp-select versions > /tmp/tmpJsFu4o'. Return Code: 0, Output: 2.2.3.0-2611. There is no MR2 client in that hostgroup. Do you think I should add to that hostgroup as well?
On March 18, 2015 at 5:48:15 PM, John Speidel (jspeidel@hortonworks.com) wrote:
I think that this could occur if the MR2 client isn't installed on all hosts.
-John
________________________________________
From: Krisztian Horvath <kr...@sequenceiq.com>
Sent: Wednesday, March 18, 2015 12:19 PM
To: dev@ambari.apache.org
Subject: MR2 smoke test and all jobs fail
Hi,
Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails with the following error: java.io.FileNotFoundException: File does not exist: hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz . Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every job fails with the same error. Does any1 know about this issue?
Br,
Krisz
Re: MR2 smoke test and all jobs fail
Posted by Krisztian Horvath <kr...@sequenceiq.com>.
It works, someone said I should set the hdp.repo file to not use this version.
On March 18, 2015 at 7:02:12 PM, Krisztian Horvath (krisztian.horvath@sequenceiq.com) wrote:
Sure
On March 18, 2015 at 7:01:00 PM, Alejandro Fernandez (afernandez@hortonworks.com) wrote:
That definitely seems likely. Can you try that patch on top of dynamic_variable_interpretation.py?
Re: MR2 smoke test and all jobs fail
Posted by Krisztian Horvath <kr...@sequenceiq.com>.
Sure
On March 18, 2015 at 7:01:00 PM, Alejandro Fernandez (afernandez@hortonworks.com) wrote:
That definitely seems likely. Can you try that patch on top of dynamic_variable_interpretation.py?
Re: MR2 smoke test and all jobs fail
Posted by Alejandro Fernandez <af...@hortonworks.com>.
That definitely seems likely. Can you try that patch on top of dynamic_variable_interpretation.py?
Thanks,
Alejandro
From: Krisztian Horvath <kr...@sequenceiq.com>>
Date: Wednesday, March 18, 2015 at 10:49 AM
To: "dev@ambari.apache.org<ma...@ambari.apache.org>" <de...@ambari.apache.org>>, Alejandro Fernandez <af...@hortonworks.com>>, John Speidel <js...@hortonworks.com>>
Subject: Re: MR2 smoke test and all jobs fail
Hi Alejandro,
2.2.0.0 packages are downloaded, but not installed, here is the result anyway:
This might be relevant? https://github.com/apache/ambari/commit/a5c9225c2445884149c4c27aec041ebcc9fef4b4
accumulo-client - None
accumulo-gc - None
accumulo-master - None
accumulo-monitor - None
accumulo-tablet - None
accumulo-tracer - None
falcon-client - None
falcon-server - None
flume-server - None
hadoop-client - 2.2.3.0-2611
hadoop-hdfs-datanode - 2.2.3.0-2611
hadoop-hdfs-journalnode - 2.2.3.0-2611
hadoop-hdfs-namenode - 2.2.3.0-2611
hadoop-hdfs-nfs3 - 2.2.3.0-2611
hadoop-hdfs-portmap - 2.2.3.0-2611
hadoop-hdfs-secondarynamenode - 2.2.3.0-2611
hadoop-httpfs - None
hadoop-mapreduce-historyserver - 2.2.3.0-2611
hadoop-yarn-nodemanager - 2.2.3.0-2611
hadoop-yarn-resourcemanager - 2.2.3.0-2611
hadoop-yarn-timelineserver - 2.2.3.0-2611
hbase-client - None
hbase-master - None
hbase-regionserver - None
hive-metastore - None
hive-server2 - None
hive-webhcat - None
kafka-broker - None
knox-server - None
mahout-client - None
oozie-client - None
oozie-server - None
phoenix-client - None
ranger-admin - None
ranger-usersync - None
slider-client - None
spark-client - None
spark-historyserver - None
sqoop-client - None
sqoop-server - None
storm-client - None
storm-nimbus - None
storm-slider-client - None
storm-supervisor - None
zookeeper-client - 2.2.3.0-2611
zookeeper-server - None
On March 18, 2015 at 6:46:13 PM, Alejandro Fernandez (afernandez@hortonworks.com<ma...@hortonworks.com>) wrote:
When History Server starts, it will copy the mapreduce.tar.gz file to HDFS.
It determines where to copy the file to (e.g.,
hdfs:/prefix:8020/hdp/apps/<version>/mapreduce/mapreduce.tar.gz) by
finding the currently applied HDP version from the output of hdp-select.
What is the output of running?
hdp-select status
I suspect that two versions are installed, 2.2.0.0 GA bits and 2.2.3.0-2611
Thanks,
Alejandro
On 3/18/15, 10:20 AM, "Krisztian Horvath"
<kr...@sequenceiq.com>> wrote:
>As I see from the log during history server start: Could not verify HDP
>version by calling '/usr/bin/hdp-select versions > /tmp/tmpJsFu4o'.
>Return Code: 0, Output: 2.2.3.0-2611. There is no MR2 client in that
>hostgroup. Do you think I should add to that hostgroup as well?
>
>
>
>On March 18, 2015 at 5:48:15 PM, John Speidel (jspeidel@hortonworks.com<ma...@hortonworks.com>)
>wrote:
>
>I think that this could occur if the MR2 client isn't installed on all
>hosts.
>
>-John
>________________________________________
>From: Krisztian Horvath <kr...@sequenceiq.com>>
>Sent: Wednesday, March 18, 2015 12:19 PM
>To: dev@ambari.apache.org<ma...@ambari.apache.org>
>Subject: MR2 smoke test and all jobs fail
>
>Hi,
>
>Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails
>with the following error: java.io.FileNotFoundException: File does not
>exist:
>hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz
>. Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every
>job fails with the same error. Does any1 know about this issue?
>
>Br,
>Krisz
>
>
Re: MR2 smoke test and all jobs fail
Posted by Krisztian Horvath <kr...@sequenceiq.com>.
Hi Alejandro,
2.2.0.0 packages are downloaded, but not installed, here is the result anyway:
This might be relevant? https://github.com/apache/ambari/commit/a5c9225c2445884149c4c27aec041ebcc9fef4b4
accumulo-client - None
accumulo-gc - None
accumulo-master - None
accumulo-monitor - None
accumulo-tablet - None
accumulo-tracer - None
falcon-client - None
falcon-server - None
flume-server - None
hadoop-client - 2.2.3.0-2611
hadoop-hdfs-datanode - 2.2.3.0-2611
hadoop-hdfs-journalnode - 2.2.3.0-2611
hadoop-hdfs-namenode - 2.2.3.0-2611
hadoop-hdfs-nfs3 - 2.2.3.0-2611
hadoop-hdfs-portmap - 2.2.3.0-2611
hadoop-hdfs-secondarynamenode - 2.2.3.0-2611
hadoop-httpfs - None
hadoop-mapreduce-historyserver - 2.2.3.0-2611
hadoop-yarn-nodemanager - 2.2.3.0-2611
hadoop-yarn-resourcemanager - 2.2.3.0-2611
hadoop-yarn-timelineserver - 2.2.3.0-2611
hbase-client - None
hbase-master - None
hbase-regionserver - None
hive-metastore - None
hive-server2 - None
hive-webhcat - None
kafka-broker - None
knox-server - None
mahout-client - None
oozie-client - None
oozie-server - None
phoenix-client - None
ranger-admin - None
ranger-usersync - None
slider-client - None
spark-client - None
spark-historyserver - None
sqoop-client - None
sqoop-server - None
storm-client - None
storm-nimbus - None
storm-slider-client - None
storm-supervisor - None
zookeeper-client - 2.2.3.0-2611
zookeeper-server - None
On March 18, 2015 at 6:46:13 PM, Alejandro Fernandez (afernandez@hortonworks.com) wrote:
When History Server starts, it will copy the mapreduce.tar.gz file to HDFS.
It determines where to copy the file to (e.g.,
hdfs:/prefix:8020/hdp/apps/<version>/mapreduce/mapreduce.tar.gz) by
finding the currently applied HDP version from the output of hdp-select.
What is the output of running?
hdp-select status
I suspect that two versions are installed, 2.2.0.0 GA bits and 2.2.3.0-2611
Thanks,
Alejandro
On 3/18/15, 10:20 AM, "Krisztian Horvath"
<kr...@sequenceiq.com> wrote:
>As I see from the log during history server start: Could not verify HDP
>version by calling '/usr/bin/hdp-select versions > /tmp/tmpJsFu4o'.
>Return Code: 0, Output: 2.2.3.0-2611. There is no MR2 client in that
>hostgroup. Do you think I should add to that hostgroup as well?
>
>
>
>On March 18, 2015 at 5:48:15 PM, John Speidel (jspeidel@hortonworks.com)
>wrote:
>
>I think that this could occur if the MR2 client isn't installed on all
>hosts.
>
>-John
>________________________________________
>From: Krisztian Horvath <kr...@sequenceiq.com>
>Sent: Wednesday, March 18, 2015 12:19 PM
>To: dev@ambari.apache.org
>Subject: MR2 smoke test and all jobs fail
>
>Hi,
>
>Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails
>with the following error: java.io.FileNotFoundException: File does not
>exist:
>hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz
>. Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every
>job fails with the same error. Does any1 know about this issue?
>
>Br,
>Krisz
>
>
Re: MR2 smoke test and all jobs fail
Posted by Alejandro Fernandez <af...@hortonworks.com>.
When History Server starts, it will copy the mapreduce.tar.gz file to HDFS.
It determines where to copy the file to (e.g.,
hdfs:/prefix:8020/hdp/apps/<version>/mapreduce/mapreduce.tar.gz) by
finding the currently applied HDP version from the output of hdp-select.
What is the output of running?
hdp-select status
I suspect that two versions are installed, 2.2.0.0 GA bits and 2.2.3.0-2611
Thanks,
Alejandro
On 3/18/15, 10:20 AM, "Krisztian Horvath"
<kr...@sequenceiq.com> wrote:
>As I see from the log during history server start: Could not verify HDP
>version by calling '/usr/bin/hdp-select versions > /tmp/tmpJsFu4o'.
>Return Code: 0, Output: 2.2.3.0-2611. There is no MR2 client in that
>hostgroup. Do you think I should add to that hostgroup as well?
>
>
>
>On March 18, 2015 at 5:48:15 PM, John Speidel (jspeidel@hortonworks.com)
>wrote:
>
>I think that this could occur if the MR2 client isn't installed on all
>hosts.
>
>-John
>________________________________________
>From: Krisztian Horvath <kr...@sequenceiq.com>
>Sent: Wednesday, March 18, 2015 12:19 PM
>To: dev@ambari.apache.org
>Subject: MR2 smoke test and all jobs fail
>
>Hi,
>
>Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails
>with the following error: java.io.FileNotFoundException: File does not
>exist:
>hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz
>. Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every
>job fails with the same error. Does any1 know about this issue?
>
>Br,
>Krisz
>
>
Re: MR2 smoke test and all jobs fail
Posted by Krisztian Horvath <kr...@sequenceiq.com>.
As I see from the log during history server start: Could not verify HDP version by calling '/usr/bin/hdp-select versions > /tmp/tmpJsFu4o'. Return Code: 0, Output: 2.2.3.0-2611. There is no MR2 client in that hostgroup. Do you think I should add to that hostgroup as well?
On March 18, 2015 at 5:48:15 PM, John Speidel (jspeidel@hortonworks.com) wrote:
I think that this could occur if the MR2 client isn't installed on all hosts.
-John
________________________________________
From: Krisztian Horvath <kr...@sequenceiq.com>
Sent: Wednesday, March 18, 2015 12:19 PM
To: dev@ambari.apache.org
Subject: MR2 smoke test and all jobs fail
Hi,
Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails with the following error: java.io.FileNotFoundException: File does not exist: hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz . Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every job fails with the same error. Does any1 know about this issue?
Br,
Krisz
Re: MR2 smoke test and all jobs fail
Posted by John Speidel <js...@hortonworks.com>.
I think that this could occur if the MR2 client isn't installed on all hosts.
-John
________________________________________
From: Krisztian Horvath <kr...@sequenceiq.com>
Sent: Wednesday, March 18, 2015 12:19 PM
To: dev@ambari.apache.org
Subject: MR2 smoke test and all jobs fail
Hi,
Currently with Ambari 1.7 and HDP 2.2.3.0-2611 the MR2 smoke test fails with the following error: java.io.FileNotFoundException: File does not exist: hdfs:/<consulnode>:8020/hdp/apps/2.2.3.0-2611/mapreduce/mapreduce.tar.gz . Few days ago with version 2.2.0.0-2041 it worked fine. Btw now every job fails with the same error. Does any1 know about this issue?
Br,
Krisz