You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2014/03/18 08:38:55 UTC
issue of "Log aggregation has not completed or is not enabled."
hi,maillist:
i try look application log use the following process
# yarn application -list
Application-Id Application-Name
User Queue State
Final-State Tracking-URL
application_1395126130647_0014 select user_id as userid,
adverti...stattime(Stage-1) hive hive
FINISHED SUCCEEDED
ch18:19888/jobhistory/job/job_1395126130647_0014
# yarn logs -applicationId application_1395126130647_0014
Logs not available at
/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
Log aggregation has not completed or is not enabled.
but i do enable Log aggregation function ,here is my yarn-site.xml
configuration about log aggregation
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<description>Where to aggregate logs to.</description>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>
the application logs is not put on hdfs successfully,why?
# hadoop fs -ls
/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014':
No such file or directory
Re: issue of "Log aggregation has not completed or is not enabled."
Posted by Jian He <jh...@hortonworks.com>.
OTOH, if the application is still running, the logs are not yet uploaded,
you certainly can not see the logs.
Jian
On Tue, Mar 18, 2014 at 1:57 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Just for confirmation,
>
> 1. Does NodeManager is restarted after enabling LogAggregation? If
> Yes, check for NodeManager start up logs for Log Aggregation Service start
> is success.
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* ch huang [mailto:justlooks@gmail.com]
> *Sent:* 18 March 2014 13:09
> *To:* user@hadoop.apache.org
> *Subject:* issue of "Log aggregation has not completed or is not enabled."
>
>
>
> hi,maillist:
>
> i try look application log use the following process
>
>
>
> # yarn application -list
> Application-Id Application-Name
> User Queue State
> Final-State Tracking-URL
> application_1395126130647_0014 select user_id as userid,
> adverti...stattime(Stage-1) hive hive
> FINISHED SUCCEEDED
> ch18:19888/jobhistory/job/job_1395126130647_0014
>
> # yarn logs -applicationId application_1395126130647_0014
> Logs not available at
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> Log aggregation has not completed or is not enabled.
>
>
>
> but i do enable Log aggregation function ,here is my yarn-site.xml
> configuration about log aggregation
>
>
>
> <property>
> <name>yarn.log-aggregation-enable</name>
> <value>true</value>
> </property>
>
> <property>
> <description>Where to aggregate logs to.</description>
> <name>yarn.nodemanager.remote-app-log-dir</name>
> <value>/var/log/hadoop-yarn/apps</value>
> </property>
>
>
>
> the application logs is not put on hdfs successfully,why?
>
>
>
> # hadoop fs -ls
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014':
> No such file or directory
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: issue of "Log aggregation has not completed or is not enabled."
Posted by Jian He <jh...@hortonworks.com>.
OTOH, if the application is still running, the logs are not yet uploaded,
you certainly can not see the logs.
Jian
On Tue, Mar 18, 2014 at 1:57 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Just for confirmation,
>
> 1. Does NodeManager is restarted after enabling LogAggregation? If
> Yes, check for NodeManager start up logs for Log Aggregation Service start
> is success.
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* ch huang [mailto:justlooks@gmail.com]
> *Sent:* 18 March 2014 13:09
> *To:* user@hadoop.apache.org
> *Subject:* issue of "Log aggregation has not completed or is not enabled."
>
>
>
> hi,maillist:
>
> i try look application log use the following process
>
>
>
> # yarn application -list
> Application-Id Application-Name
> User Queue State
> Final-State Tracking-URL
> application_1395126130647_0014 select user_id as userid,
> adverti...stattime(Stage-1) hive hive
> FINISHED SUCCEEDED
> ch18:19888/jobhistory/job/job_1395126130647_0014
>
> # yarn logs -applicationId application_1395126130647_0014
> Logs not available at
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> Log aggregation has not completed or is not enabled.
>
>
>
> but i do enable Log aggregation function ,here is my yarn-site.xml
> configuration about log aggregation
>
>
>
> <property>
> <name>yarn.log-aggregation-enable</name>
> <value>true</value>
> </property>
>
> <property>
> <description>Where to aggregate logs to.</description>
> <name>yarn.nodemanager.remote-app-log-dir</name>
> <value>/var/log/hadoop-yarn/apps</value>
> </property>
>
>
>
> the application logs is not put on hdfs successfully,why?
>
>
>
> # hadoop fs -ls
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014':
> No such file or directory
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: issue of "Log aggregation has not completed or is not enabled."
Posted by Jian He <jh...@hortonworks.com>.
OTOH, if the application is still running, the logs are not yet uploaded,
you certainly can not see the logs.
Jian
On Tue, Mar 18, 2014 at 1:57 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Just for confirmation,
>
> 1. Does NodeManager is restarted after enabling LogAggregation? If
> Yes, check for NodeManager start up logs for Log Aggregation Service start
> is success.
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* ch huang [mailto:justlooks@gmail.com]
> *Sent:* 18 March 2014 13:09
> *To:* user@hadoop.apache.org
> *Subject:* issue of "Log aggregation has not completed or is not enabled."
>
>
>
> hi,maillist:
>
> i try look application log use the following process
>
>
>
> # yarn application -list
> Application-Id Application-Name
> User Queue State
> Final-State Tracking-URL
> application_1395126130647_0014 select user_id as userid,
> adverti...stattime(Stage-1) hive hive
> FINISHED SUCCEEDED
> ch18:19888/jobhistory/job/job_1395126130647_0014
>
> # yarn logs -applicationId application_1395126130647_0014
> Logs not available at
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> Log aggregation has not completed or is not enabled.
>
>
>
> but i do enable Log aggregation function ,here is my yarn-site.xml
> configuration about log aggregation
>
>
>
> <property>
> <name>yarn.log-aggregation-enable</name>
> <value>true</value>
> </property>
>
> <property>
> <description>Where to aggregate logs to.</description>
> <name>yarn.nodemanager.remote-app-log-dir</name>
> <value>/var/log/hadoop-yarn/apps</value>
> </property>
>
>
>
> the application logs is not put on hdfs successfully,why?
>
>
>
> # hadoop fs -ls
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014':
> No such file or directory
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: issue of "Log aggregation has not completed or is not enabled."
Posted by Jian He <jh...@hortonworks.com>.
OTOH, if the application is still running, the logs are not yet uploaded,
you certainly can not see the logs.
Jian
On Tue, Mar 18, 2014 at 1:57 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Just for confirmation,
>
> 1. Does NodeManager is restarted after enabling LogAggregation? If
> Yes, check for NodeManager start up logs for Log Aggregation Service start
> is success.
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* ch huang [mailto:justlooks@gmail.com]
> *Sent:* 18 March 2014 13:09
> *To:* user@hadoop.apache.org
> *Subject:* issue of "Log aggregation has not completed or is not enabled."
>
>
>
> hi,maillist:
>
> i try look application log use the following process
>
>
>
> # yarn application -list
> Application-Id Application-Name
> User Queue State
> Final-State Tracking-URL
> application_1395126130647_0014 select user_id as userid,
> adverti...stattime(Stage-1) hive hive
> FINISHED SUCCEEDED
> ch18:19888/jobhistory/job/job_1395126130647_0014
>
> # yarn logs -applicationId application_1395126130647_0014
> Logs not available at
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> Log aggregation has not completed or is not enabled.
>
>
>
> but i do enable Log aggregation function ,here is my yarn-site.xml
> configuration about log aggregation
>
>
>
> <property>
> <name>yarn.log-aggregation-enable</name>
> <value>true</value>
> </property>
>
> <property>
> <description>Where to aggregate logs to.</description>
> <name>yarn.nodemanager.remote-app-log-dir</name>
> <value>/var/log/hadoop-yarn/apps</value>
> </property>
>
>
>
> the application logs is not put on hdfs successfully,why?
>
>
>
> # hadoop fs -ls
> /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
> ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014':
> No such file or directory
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: issue of "Log aggregation has not completed or is not enabled."
Posted by Rohith Sharma K S <ro...@huawei.com>.
Just for confirmation,
1. Does NodeManager is restarted after enabling LogAggregation? If Yes, check for NodeManager start up logs for Log Aggregation Service start is success.
Thanks & Regards
Rohith Sharma K S
From: ch huang [mailto:justlooks@gmail.com]
Sent: 18 March 2014 13:09
To: user@hadoop.apache.org
Subject: issue of "Log aggregation has not completed or is not enabled."
hi,maillist:
i try look application log use the following process
# yarn application -list
Application-Id Application-Name User Queue State Final-State Tracking-URL
application_1395126130647_0014 select user_id as userid, adverti...stattime(Stage-1) hive hive FINISHED SUCCEEDED ch18:19888/jobhistory/job/job_1395126130647_0014
# yarn logs -applicationId application_1395126130647_0014
Logs not available at /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
Log aggregation has not completed or is not enabled.
but i do enable Log aggregation function ,here is my yarn-site.xml configuration about log aggregation
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<description>Where to aggregate logs to.</description>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>
the application logs is not put on hdfs successfully,why?
# hadoop fs -ls /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014': No such file or directory
Re: unsubscribe
Posted by Vimal Jain <vk...@gmail.com>.
Please send an email to user-unsubscribe@hadoop.apache.org.
On Tue, Mar 18, 2014 at 6:57 PM, Rananavare, Sunil <
sunil.rananavare@unify.com> wrote:
> Please remove me from the user distribution list.
>
> Thanks.
>
--
Thanks and Regards,
Vimal Jain
Re: unsubscribe
Posted by Vimal Jain <vk...@gmail.com>.
Please send an email to user-unsubscribe@hadoop.apache.org.
On Tue, Mar 18, 2014 at 6:57 PM, Rananavare, Sunil <
sunil.rananavare@unify.com> wrote:
> Please remove me from the user distribution list.
>
> Thanks.
>
--
Thanks and Regards,
Vimal Jain
Re: unsubscribe
Posted by Vimal Jain <vk...@gmail.com>.
Please send an email to user-unsubscribe@hadoop.apache.org.
On Tue, Mar 18, 2014 at 6:57 PM, Rananavare, Sunil <
sunil.rananavare@unify.com> wrote:
> Please remove me from the user distribution list.
>
> Thanks.
>
--
Thanks and Regards,
Vimal Jain
Re: unsubscribe
Posted by Vimal Jain <vk...@gmail.com>.
Please send an email to user-unsubscribe@hadoop.apache.org.
On Tue, Mar 18, 2014 at 6:57 PM, Rananavare, Sunil <
sunil.rananavare@unify.com> wrote:
> Please remove me from the user distribution list.
>
> Thanks.
>
--
Thanks and Regards,
Vimal Jain
unsubscribe
Posted by "Rananavare, Sunil" <su...@unify.com>.
Please remove me from the user distribution list.
Thanks.
unsubscribe
Posted by "Rananavare, Sunil" <su...@unify.com>.
Please remove me from the user distribution list.
Thanks.
unsubscribe
Posted by "Rananavare, Sunil" <su...@unify.com>.
Please remove me from the user distribution list.
Thanks.
RE: issue of "Log aggregation has not completed or is not enabled."
Posted by Rohith Sharma K S <ro...@huawei.com>.
Just for confirmation,
1. Does NodeManager is restarted after enabling LogAggregation? If Yes, check for NodeManager start up logs for Log Aggregation Service start is success.
Thanks & Regards
Rohith Sharma K S
From: ch huang [mailto:justlooks@gmail.com]
Sent: 18 March 2014 13:09
To: user@hadoop.apache.org
Subject: issue of "Log aggregation has not completed or is not enabled."
hi,maillist:
i try look application log use the following process
# yarn application -list
Application-Id Application-Name User Queue State Final-State Tracking-URL
application_1395126130647_0014 select user_id as userid, adverti...stattime(Stage-1) hive hive FINISHED SUCCEEDED ch18:19888/jobhistory/job/job_1395126130647_0014
# yarn logs -applicationId application_1395126130647_0014
Logs not available at /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
Log aggregation has not completed or is not enabled.
but i do enable Log aggregation function ,here is my yarn-site.xml configuration about log aggregation
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<description>Where to aggregate logs to.</description>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>
the application logs is not put on hdfs successfully,why?
# hadoop fs -ls /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014': No such file or directory
RE: issue of "Log aggregation has not completed or is not enabled."
Posted by Rohith Sharma K S <ro...@huawei.com>.
Just for confirmation,
1. Does NodeManager is restarted after enabling LogAggregation? If Yes, check for NodeManager start up logs for Log Aggregation Service start is success.
Thanks & Regards
Rohith Sharma K S
From: ch huang [mailto:justlooks@gmail.com]
Sent: 18 March 2014 13:09
To: user@hadoop.apache.org
Subject: issue of "Log aggregation has not completed or is not enabled."
hi,maillist:
i try look application log use the following process
# yarn application -list
Application-Id Application-Name User Queue State Final-State Tracking-URL
application_1395126130647_0014 select user_id as userid, adverti...stattime(Stage-1) hive hive FINISHED SUCCEEDED ch18:19888/jobhistory/job/job_1395126130647_0014
# yarn logs -applicationId application_1395126130647_0014
Logs not available at /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
Log aggregation has not completed or is not enabled.
but i do enable Log aggregation function ,here is my yarn-site.xml configuration about log aggregation
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<description>Where to aggregate logs to.</description>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>
the application logs is not put on hdfs successfully,why?
# hadoop fs -ls /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014': No such file or directory
RE: issue of "Log aggregation has not completed or is not enabled."
Posted by Rohith Sharma K S <ro...@huawei.com>.
Just for confirmation,
1. Does NodeManager is restarted after enabling LogAggregation? If Yes, check for NodeManager start up logs for Log Aggregation Service start is success.
Thanks & Regards
Rohith Sharma K S
From: ch huang [mailto:justlooks@gmail.com]
Sent: 18 March 2014 13:09
To: user@hadoop.apache.org
Subject: issue of "Log aggregation has not completed or is not enabled."
hi,maillist:
i try look application log use the following process
# yarn application -list
Application-Id Application-Name User Queue State Final-State Tracking-URL
application_1395126130647_0014 select user_id as userid, adverti...stattime(Stage-1) hive hive FINISHED SUCCEEDED ch18:19888/jobhistory/job/job_1395126130647_0014
# yarn logs -applicationId application_1395126130647_0014
Logs not available at /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
Log aggregation has not completed or is not enabled.
but i do enable Log aggregation function ,here is my yarn-site.xml configuration about log aggregation
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<description>Where to aggregate logs to.</description>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>
the application logs is not put on hdfs successfully,why?
# hadoop fs -ls /var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014
ls: `/var/log/hadoop-yarn/apps/root/logs/application_1395126130647_0014': No such file or directory
unsubscribe
Posted by "Rananavare, Sunil" <su...@unify.com>.
Please remove me from the user distribution list.
Thanks.