You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by xeonmailinglist-gmail <xe...@gmail.com> on 2015/05/28 15:14:51 UTC
Can't see logs of a finished job.
Hi,
I have configured the proxy server and the historyserver in Yarn
Mapreduce, and they are running properly [1]. But, I can’t see the logs
of an executed job [2]. I even have set history params in
|yarn-site.xml| [3]. Am I missing some configuration to see the logs of
a finished job?
[1] Processes running
|~/Programs/hadoop$ jps
15520 SecondaryNameNode
18800 WebAppProxyServer
17985 NodeManager
19154 Jps
15251 DataNode
15047 NameNode
16157 JobHistoryServer
17583 ResourceManager
|
[2] The error:
|
Failed redirect for container_1432817967879_0001_01_000001
Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
|
[3] Part of |yarn-site.xml| configuration
| <!-- job history -->
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
<property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
<property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
|
--
--
Re: Can't see logs of a finished job.
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey,
the job is finished. In order to see the logs for the finished job from webui, you need add one more configuration:
I see that you have already started jobhistoryserver (Assume the jhs address is localhost:19888).
Add this configuration to yarn-site.xml:
<property>
<name>yarn.log.server.url</name>
<value>http://localhost:19888/jobhistory/logs</value>
</property>
Thanks
Xuan Gong
From: xeonmailinglist-gmail <xe...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Thursday, May 28, 2015 at 6:14 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Can't see logs of a finished job.
Hi,
I have configured the proxy server and the historyserver in Yarn Mapreduce, and they are running properly [1]. But, I can’t see the logs of an executed job [2]. I even have set history params in yarn-site.xml [3]. Am I missing some configuration to see the logs of a finished job?
[1] Processes running
~/Programs/hadoop$ jps
15520 SecondaryNameNode
18800 WebAppProxyServer
17985 NodeManager
19154 Jps
15251 DataNode
15047 NameNode
16157 JobHistoryServer
17583 ResourceManager
[2] The error:
Failed redirect for container_1432817967879_0001_01_000001
Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
[3] Part of yarn-site.xml configuration
<!-- job history -->
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
<property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
<property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
--
--
Re: Can't see logs of a finished job.
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey,
the job is finished. In order to see the logs for the finished job from webui, you need add one more configuration:
I see that you have already started jobhistoryserver (Assume the jhs address is localhost:19888).
Add this configuration to yarn-site.xml:
<property>
<name>yarn.log.server.url</name>
<value>http://localhost:19888/jobhistory/logs</value>
</property>
Thanks
Xuan Gong
From: xeonmailinglist-gmail <xe...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Thursday, May 28, 2015 at 6:14 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Can't see logs of a finished job.
Hi,
I have configured the proxy server and the historyserver in Yarn Mapreduce, and they are running properly [1]. But, I can’t see the logs of an executed job [2]. I even have set history params in yarn-site.xml [3]. Am I missing some configuration to see the logs of a finished job?
[1] Processes running
~/Programs/hadoop$ jps
15520 SecondaryNameNode
18800 WebAppProxyServer
17985 NodeManager
19154 Jps
15251 DataNode
15047 NameNode
16157 JobHistoryServer
17583 ResourceManager
[2] The error:
Failed redirect for container_1432817967879_0001_01_000001
Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
[3] Part of yarn-site.xml configuration
<!-- job history -->
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
<property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
<property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
--
--
Re: Can't see logs of a finished job.
Posted by xeonmailinglist-gmail <xe...@gmail.com>.
In the |mapred-site.xml| I have also this configuration [4]:
[4]
|<property><name>mapreduce.jobhistory.address </name> <value>hadoop-coc-1:10020</value> </property>
<property><name>mapreduce.jobhistory.webapp.address</name> <value>hadoop-coc-1:19888</value> </property>
<property> <name>mapreduce.jobhistory.done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/done</value> </property>
<property> <name>mapreduce.jobhistory.intermediate-done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/intermediate-done-dir</value> </property>
|
On 05/28/2015 02:14 PM, xeonmailinglist-gmail wrote:
> Hi,
>
> I have configured the proxy server and the historyserver in Yarn
> Mapreduce, and they are running properly [1]. But, I can’t see the
> logs of an executed job [2]. I even have set history params in
> |yarn-site.xml| [3]. Am I missing some configuration to see the logs
> of a finished job?
>
> [1] Processes running
>
> |~/Programs/hadoop$ jps
> 15520 SecondaryNameNode
> 18800 WebAppProxyServer
> 17985 NodeManager
> 19154 Jps
> 15251 DataNode
> 15047 NameNode
> 16157 JobHistoryServer
> 17583 ResourceManager
> |
>
> [2] The error:
>
> |
> Failed redirect for container_1432817967879_0001_01_000001
>
> Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
>
> java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
> |
>
> [3] Part of |yarn-site.xml| configuration
>
> | <!-- job history -->
> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
> <property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
> |
>
> --
> --
--
--
Re: Can't see logs of a finished job.
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey,
the job is finished. In order to see the logs for the finished job from webui, you need add one more configuration:
I see that you have already started jobhistoryserver (Assume the jhs address is localhost:19888).
Add this configuration to yarn-site.xml:
<property>
<name>yarn.log.server.url</name>
<value>http://localhost:19888/jobhistory/logs</value>
</property>
Thanks
Xuan Gong
From: xeonmailinglist-gmail <xe...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Thursday, May 28, 2015 at 6:14 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Can't see logs of a finished job.
Hi,
I have configured the proxy server and the historyserver in Yarn Mapreduce, and they are running properly [1]. But, I can’t see the logs of an executed job [2]. I even have set history params in yarn-site.xml [3]. Am I missing some configuration to see the logs of a finished job?
[1] Processes running
~/Programs/hadoop$ jps
15520 SecondaryNameNode
18800 WebAppProxyServer
17985 NodeManager
19154 Jps
15251 DataNode
15047 NameNode
16157 JobHistoryServer
17583 ResourceManager
[2] The error:
Failed redirect for container_1432817967879_0001_01_000001
Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
[3] Part of yarn-site.xml configuration
<!-- job history -->
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
<property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
<property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
--
--
Re: Can't see logs of a finished job.
Posted by xeonmailinglist-gmail <xe...@gmail.com>.
In the |mapred-site.xml| I have also this configuration [4]:
[4]
|<property><name>mapreduce.jobhistory.address </name> <value>hadoop-coc-1:10020</value> </property>
<property><name>mapreduce.jobhistory.webapp.address</name> <value>hadoop-coc-1:19888</value> </property>
<property> <name>mapreduce.jobhistory.done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/done</value> </property>
<property> <name>mapreduce.jobhistory.intermediate-done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/intermediate-done-dir</value> </property>
|
On 05/28/2015 02:14 PM, xeonmailinglist-gmail wrote:
> Hi,
>
> I have configured the proxy server and the historyserver in Yarn
> Mapreduce, and they are running properly [1]. But, I can’t see the
> logs of an executed job [2]. I even have set history params in
> |yarn-site.xml| [3]. Am I missing some configuration to see the logs
> of a finished job?
>
> [1] Processes running
>
> |~/Programs/hadoop$ jps
> 15520 SecondaryNameNode
> 18800 WebAppProxyServer
> 17985 NodeManager
> 19154 Jps
> 15251 DataNode
> 15047 NameNode
> 16157 JobHistoryServer
> 17583 ResourceManager
> |
>
> [2] The error:
>
> |
> Failed redirect for container_1432817967879_0001_01_000001
>
> Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
>
> java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
> |
>
> [3] Part of |yarn-site.xml| configuration
>
> | <!-- job history -->
> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
> <property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
> |
>
> --
> --
--
--
Re: Can't see logs of a finished job.
Posted by xeonmailinglist-gmail <xe...@gmail.com>.
In the |mapred-site.xml| I have also this configuration [4]:
[4]
|<property><name>mapreduce.jobhistory.address </name> <value>hadoop-coc-1:10020</value> </property>
<property><name>mapreduce.jobhistory.webapp.address</name> <value>hadoop-coc-1:19888</value> </property>
<property> <name>mapreduce.jobhistory.done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/done</value> </property>
<property> <name>mapreduce.jobhistory.intermediate-done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/intermediate-done-dir</value> </property>
|
On 05/28/2015 02:14 PM, xeonmailinglist-gmail wrote:
> Hi,
>
> I have configured the proxy server and the historyserver in Yarn
> Mapreduce, and they are running properly [1]. But, I can’t see the
> logs of an executed job [2]. I even have set history params in
> |yarn-site.xml| [3]. Am I missing some configuration to see the logs
> of a finished job?
>
> [1] Processes running
>
> |~/Programs/hadoop$ jps
> 15520 SecondaryNameNode
> 18800 WebAppProxyServer
> 17985 NodeManager
> 19154 Jps
> 15251 DataNode
> 15047 NameNode
> 16157 JobHistoryServer
> 17583 ResourceManager
> |
>
> [2] The error:
>
> |
> Failed redirect for container_1432817967879_0001_01_000001
>
> Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
>
> java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
> |
>
> [3] Part of |yarn-site.xml| configuration
>
> | <!-- job history -->
> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
> <property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
> |
>
> --
> --
--
--
Re: Can't see logs of a finished job.
Posted by xeonmailinglist-gmail <xe...@gmail.com>.
In the |mapred-site.xml| I have also this configuration [4]:
[4]
|<property><name>mapreduce.jobhistory.address </name> <value>hadoop-coc-1:10020</value> </property>
<property><name>mapreduce.jobhistory.webapp.address</name> <value>hadoop-coc-1:19888</value> </property>
<property> <name>mapreduce.jobhistory.done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/done</value> </property>
<property> <name>mapreduce.jobhistory.intermediate-done-dir</name> <value>/home/xubuntu/Programs/hadoop/logs/history/intermediate-done-dir</value> </property>
|
On 05/28/2015 02:14 PM, xeonmailinglist-gmail wrote:
> Hi,
>
> I have configured the proxy server and the historyserver in Yarn
> Mapreduce, and they are running properly [1]. But, I can’t see the
> logs of an executed job [2]. I even have set history params in
> |yarn-site.xml| [3]. Am I missing some configuration to see the logs
> of a finished job?
>
> [1] Processes running
>
> |~/Programs/hadoop$ jps
> 15520 SecondaryNameNode
> 18800 WebAppProxyServer
> 17985 NodeManager
> 19154 Jps
> 15251 DataNode
> 15047 NameNode
> 16157 JobHistoryServer
> 17583 ResourceManager
> |
>
> [2] The error:
>
> |
> Failed redirect for container_1432817967879_0001_01_000001
>
> Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
>
> java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
> |
>
> [3] Part of |yarn-site.xml| configuration
>
> | <!-- job history -->
> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
> <property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
> <property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
> |
>
> --
> --
--
--
Re: Can't see logs of a finished job.
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey,
the job is finished. In order to see the logs for the finished job from webui, you need add one more configuration:
I see that you have already started jobhistoryserver (Assume the jhs address is localhost:19888).
Add this configuration to yarn-site.xml:
<property>
<name>yarn.log.server.url</name>
<value>http://localhost:19888/jobhistory/logs</value>
</property>
Thanks
Xuan Gong
From: xeonmailinglist-gmail <xe...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Thursday, May 28, 2015 at 6:14 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Can't see logs of a finished job.
Hi,
I have configured the proxy server and the historyserver in Yarn Mapreduce, and they are running properly [1]. But, I can’t see the logs of an executed job [2]. I even have set history params in yarn-site.xml [3]. Am I missing some configuration to see the logs of a finished job?
[1] Processes running
~/Programs/hadoop$ jps
15520 SecondaryNameNode
18800 WebAppProxyServer
17985 NodeManager
19154 Jps
15251 DataNode
15047 NameNode
16157 JobHistoryServer
17583 ResourceManager
[2] The error:
Failed redirect for container_1432817967879_0001_01_000001
Failed while trying to construct the redirect url to the log server. Log Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.
[3] Part of yarn-site.xml configuration
<!-- job history -->
<property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
<property> <name>yarn.nodemanager.log.retain-seconds</name> <value>900000</value> </property>
<property> <name>yarn.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property>
--
--