You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Silvina Caíno Lores <si...@gmail.com> on 2014/03/10 12:16:11 UTC
Hadoop 2.2.0 not showing progress
Hi all,
I've been noticing lately that sometimes my Hadoop jobs do not report
progress in the terminal. They seem like they are stuck at the "Running
job: job_ ...." message, however YarnChilds are running and properly
executing.
I know that my job didn't fail but it's very inconvenient not being able to
monitor how is it going, specially to estimate completion time. This
doesn't happen all the time, any clues on what is going on?
Thanks in advance,
Silvina
Re: Hadoop 2.2.0 not showing progress
Posted by Silvina Caíno Lores <si...@gmail.com>.
The job was actually running and it finished successfully. I was able to
monitor it by using wget and the resource manager web interface
(http://host:port/ws/v1/cluster/apps) and the yarn application command that
Xuan suggested.
Thank you all for your help.
On 11 March 2014 06:11, sudhakara st <su...@gmail.com> wrote:
> RM not able schdule your jobs it waiting indefinitely to schedule jobs,
> may due to not able communicate with NM's or not able create AM or
> sufiicient resoucrce are not avaialbel in containers etc. Check your
> configuration for Java heap, yarn.app.mapreduce.am.resource.mb, mapreduce.reduce.memory.mb,
> mapreduce.map.memory.mb, Check RM web URL for available resources and
> scheduler information and also check RM,NM logs
>
>
> On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
>
>> Hey, Silvina:
>>
>> You may find more information about this application from RM web UI or
>> Yarn Command Line (Type yarn application -help to find out commands).
>>
>> Thanks
>>
>> Xuan Gong
>>
>>
>> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
>> silvi.caino@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I've been noticing lately that sometimes my Hadoop jobs do not report
>>> progress in the terminal. They seem like they are stuck at the "Running
>>> job: job_ ...." message, however YarnChilds are running and properly
>>> executing.
>>>
>>> I know that my job didn't fail but it's very inconvenient not being able
>>> to monitor how is it going, specially to estimate completion time. This
>>> doesn't happen all the time, any clues on what is going on?
>>>
>>> Thanks in advance,
>>> Silvina
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
Re: Hadoop 2.2.0 not showing progress
Posted by Silvina Caíno Lores <si...@gmail.com>.
The job was actually running and it finished successfully. I was able to
monitor it by using wget and the resource manager web interface
(http://host:port/ws/v1/cluster/apps) and the yarn application command that
Xuan suggested.
Thank you all for your help.
On 11 March 2014 06:11, sudhakara st <su...@gmail.com> wrote:
> RM not able schdule your jobs it waiting indefinitely to schedule jobs,
> may due to not able communicate with NM's or not able create AM or
> sufiicient resoucrce are not avaialbel in containers etc. Check your
> configuration for Java heap, yarn.app.mapreduce.am.resource.mb, mapreduce.reduce.memory.mb,
> mapreduce.map.memory.mb, Check RM web URL for available resources and
> scheduler information and also check RM,NM logs
>
>
> On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
>
>> Hey, Silvina:
>>
>> You may find more information about this application from RM web UI or
>> Yarn Command Line (Type yarn application -help to find out commands).
>>
>> Thanks
>>
>> Xuan Gong
>>
>>
>> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
>> silvi.caino@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I've been noticing lately that sometimes my Hadoop jobs do not report
>>> progress in the terminal. They seem like they are stuck at the "Running
>>> job: job_ ...." message, however YarnChilds are running and properly
>>> executing.
>>>
>>> I know that my job didn't fail but it's very inconvenient not being able
>>> to monitor how is it going, specially to estimate completion time. This
>>> doesn't happen all the time, any clues on what is going on?
>>>
>>> Thanks in advance,
>>> Silvina
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
Re: Hadoop 2.2.0 not showing progress
Posted by Silvina Caíno Lores <si...@gmail.com>.
The job was actually running and it finished successfully. I was able to
monitor it by using wget and the resource manager web interface
(http://host:port/ws/v1/cluster/apps) and the yarn application command that
Xuan suggested.
Thank you all for your help.
On 11 March 2014 06:11, sudhakara st <su...@gmail.com> wrote:
> RM not able schdule your jobs it waiting indefinitely to schedule jobs,
> may due to not able communicate with NM's or not able create AM or
> sufiicient resoucrce are not avaialbel in containers etc. Check your
> configuration for Java heap, yarn.app.mapreduce.am.resource.mb, mapreduce.reduce.memory.mb,
> mapreduce.map.memory.mb, Check RM web URL for available resources and
> scheduler information and also check RM,NM logs
>
>
> On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
>
>> Hey, Silvina:
>>
>> You may find more information about this application from RM web UI or
>> Yarn Command Line (Type yarn application -help to find out commands).
>>
>> Thanks
>>
>> Xuan Gong
>>
>>
>> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
>> silvi.caino@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I've been noticing lately that sometimes my Hadoop jobs do not report
>>> progress in the terminal. They seem like they are stuck at the "Running
>>> job: job_ ...." message, however YarnChilds are running and properly
>>> executing.
>>>
>>> I know that my job didn't fail but it's very inconvenient not being able
>>> to monitor how is it going, specially to estimate completion time. This
>>> doesn't happen all the time, any clues on what is going on?
>>>
>>> Thanks in advance,
>>> Silvina
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
Re: Hadoop 2.2.0 not showing progress
Posted by Silvina Caíno Lores <si...@gmail.com>.
The job was actually running and it finished successfully. I was able to
monitor it by using wget and the resource manager web interface
(http://host:port/ws/v1/cluster/apps) and the yarn application command that
Xuan suggested.
Thank you all for your help.
On 11 March 2014 06:11, sudhakara st <su...@gmail.com> wrote:
> RM not able schdule your jobs it waiting indefinitely to schedule jobs,
> may due to not able communicate with NM's or not able create AM or
> sufiicient resoucrce are not avaialbel in containers etc. Check your
> configuration for Java heap, yarn.app.mapreduce.am.resource.mb, mapreduce.reduce.memory.mb,
> mapreduce.map.memory.mb, Check RM web URL for available resources and
> scheduler information and also check RM,NM logs
>
>
> On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
>
>> Hey, Silvina:
>>
>> You may find more information about this application from RM web UI or
>> Yarn Command Line (Type yarn application -help to find out commands).
>>
>> Thanks
>>
>> Xuan Gong
>>
>>
>> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
>> silvi.caino@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I've been noticing lately that sometimes my Hadoop jobs do not report
>>> progress in the terminal. They seem like they are stuck at the "Running
>>> job: job_ ...." message, however YarnChilds are running and properly
>>> executing.
>>>
>>> I know that my job didn't fail but it's very inconvenient not being able
>>> to monitor how is it going, specially to estimate completion time. This
>>> doesn't happen all the time, any clues on what is going on?
>>>
>>> Thanks in advance,
>>> Silvina
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
Re: Hadoop 2.2.0 not showing progress
Posted by sudhakara st <su...@gmail.com>.
RM not able schdule your jobs it waiting indefinitely to schedule jobs, may
due to not able communicate with NM's or not able create AM or
sufiicient resoucrce are not avaialbel in containers etc. Check your
configuration for Java heap, yarn.app.mapreduce.am.resource.mb,
mapreduce.reduce.memory.mb,
mapreduce.map.memory.mb, Check RM web URL for available resources and
scheduler information and also check RM,NM logs
On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> Hey, Silvina:
>
> You may find more information about this application from RM web UI or
> Yarn Command Line (Type yarn application -help to find out commands).
>
> Thanks
>
> Xuan Gong
>
>
> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
> silvi.caino@gmail.com> wrote:
>
>> Hi all,
>>
>> I've been noticing lately that sometimes my Hadoop jobs do not report
>> progress in the terminal. They seem like they are stuck at the "Running
>> job: job_ ...." message, however YarnChilds are running and properly
>> executing.
>>
>> I know that my job didn't fail but it's very inconvenient not being able
>> to monitor how is it going, specially to estimate completion time. This
>> doesn't happen all the time, any clues on what is going on?
>>
>> Thanks in advance,
>> Silvina
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Regards,
...sudhakara
Re: Hadoop 2.2.0 not showing progress
Posted by sudhakara st <su...@gmail.com>.
RM not able schdule your jobs it waiting indefinitely to schedule jobs, may
due to not able communicate with NM's or not able create AM or
sufiicient resoucrce are not avaialbel in containers etc. Check your
configuration for Java heap, yarn.app.mapreduce.am.resource.mb,
mapreduce.reduce.memory.mb,
mapreduce.map.memory.mb, Check RM web URL for available resources and
scheduler information and also check RM,NM logs
On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> Hey, Silvina:
>
> You may find more information about this application from RM web UI or
> Yarn Command Line (Type yarn application -help to find out commands).
>
> Thanks
>
> Xuan Gong
>
>
> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
> silvi.caino@gmail.com> wrote:
>
>> Hi all,
>>
>> I've been noticing lately that sometimes my Hadoop jobs do not report
>> progress in the terminal. They seem like they are stuck at the "Running
>> job: job_ ...." message, however YarnChilds are running and properly
>> executing.
>>
>> I know that my job didn't fail but it's very inconvenient not being able
>> to monitor how is it going, specially to estimate completion time. This
>> doesn't happen all the time, any clues on what is going on?
>>
>> Thanks in advance,
>> Silvina
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Regards,
...sudhakara
Re: Hadoop 2.2.0 not showing progress
Posted by sudhakara st <su...@gmail.com>.
RM not able schdule your jobs it waiting indefinitely to schedule jobs, may
due to not able communicate with NM's or not able create AM or
sufiicient resoucrce are not avaialbel in containers etc. Check your
configuration for Java heap, yarn.app.mapreduce.am.resource.mb,
mapreduce.reduce.memory.mb,
mapreduce.map.memory.mb, Check RM web URL for available resources and
scheduler information and also check RM,NM logs
On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> Hey, Silvina:
>
> You may find more information about this application from RM web UI or
> Yarn Command Line (Type yarn application -help to find out commands).
>
> Thanks
>
> Xuan Gong
>
>
> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
> silvi.caino@gmail.com> wrote:
>
>> Hi all,
>>
>> I've been noticing lately that sometimes my Hadoop jobs do not report
>> progress in the terminal. They seem like they are stuck at the "Running
>> job: job_ ...." message, however YarnChilds are running and properly
>> executing.
>>
>> I know that my job didn't fail but it's very inconvenient not being able
>> to monitor how is it going, specially to estimate completion time. This
>> doesn't happen all the time, any clues on what is going on?
>>
>> Thanks in advance,
>> Silvina
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Regards,
...sudhakara
Re: Hadoop 2.2.0 not showing progress
Posted by sudhakara st <su...@gmail.com>.
RM not able schdule your jobs it waiting indefinitely to schedule jobs, may
due to not able communicate with NM's or not able create AM or
sufiicient resoucrce are not avaialbel in containers etc. Check your
configuration for Java heap, yarn.app.mapreduce.am.resource.mb,
mapreduce.reduce.memory.mb,
mapreduce.map.memory.mb, Check RM web URL for available resources and
scheduler information and also check RM,NM logs
On Mon, Mar 10, 2014 at 9:54 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> Hey, Silvina:
>
> You may find more information about this application from RM web UI or
> Yarn Command Line (Type yarn application -help to find out commands).
>
> Thanks
>
> Xuan Gong
>
>
> On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores <
> silvi.caino@gmail.com> wrote:
>
>> Hi all,
>>
>> I've been noticing lately that sometimes my Hadoop jobs do not report
>> progress in the terminal. They seem like they are stuck at the "Running
>> job: job_ ...." message, however YarnChilds are running and properly
>> executing.
>>
>> I know that my job didn't fail but it's very inconvenient not being able
>> to monitor how is it going, specially to estimate completion time. This
>> doesn't happen all the time, any clues on what is going on?
>>
>> Thanks in advance,
>> Silvina
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Regards,
...sudhakara
Re: Hadoop 2.2.0 not showing progress
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Silvina:
You may find more information about this application from RM web UI or
Yarn Command Line (Type yarn application -help to find out commands).
Thanks
Xuan Gong
On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores
<si...@gmail.com>wrote:
> Hi all,
>
> I've been noticing lately that sometimes my Hadoop jobs do not report
> progress in the terminal. They seem like they are stuck at the "Running
> job: job_ ...." message, however YarnChilds are running and properly
> executing.
>
> I know that my job didn't fail but it's very inconvenient not being able
> to monitor how is it going, specially to estimate completion time. This
> doesn't happen all the time, any clues on what is going on?
>
> Thanks in advance,
> Silvina
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop 2.2.0 not showing progress
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Silvina:
You may find more information about this application from RM web UI or
Yarn Command Line (Type yarn application -help to find out commands).
Thanks
Xuan Gong
On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores
<si...@gmail.com>wrote:
> Hi all,
>
> I've been noticing lately that sometimes my Hadoop jobs do not report
> progress in the terminal. They seem like they are stuck at the "Running
> job: job_ ...." message, however YarnChilds are running and properly
> executing.
>
> I know that my job didn't fail but it's very inconvenient not being able
> to monitor how is it going, specially to estimate completion time. This
> doesn't happen all the time, any clues on what is going on?
>
> Thanks in advance,
> Silvina
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop 2.2.0 not showing progress
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Silvina:
You may find more information about this application from RM web UI or
Yarn Command Line (Type yarn application -help to find out commands).
Thanks
Xuan Gong
On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores
<si...@gmail.com>wrote:
> Hi all,
>
> I've been noticing lately that sometimes my Hadoop jobs do not report
> progress in the terminal. They seem like they are stuck at the "Running
> job: job_ ...." message, however YarnChilds are running and properly
> executing.
>
> I know that my job didn't fail but it's very inconvenient not being able
> to monitor how is it going, specially to estimate completion time. This
> doesn't happen all the time, any clues on what is going on?
>
> Thanks in advance,
> Silvina
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop 2.2.0 not showing progress
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Silvina:
You may find more information about this application from RM web UI or
Yarn Command Line (Type yarn application -help to find out commands).
Thanks
Xuan Gong
On Mon, Mar 10, 2014 at 4:16 AM, Silvina Caíno Lores
<si...@gmail.com>wrote:
> Hi all,
>
> I've been noticing lately that sometimes my Hadoop jobs do not report
> progress in the terminal. They seem like they are stuck at the "Running
> job: job_ ...." message, however YarnChilds are running and properly
> executing.
>
> I know that my job didn't fail but it's very inconvenient not being able
> to monitor how is it going, specially to estimate completion time. This
> doesn't happen all the time, any clues on what is going on?
>
> Thanks in advance,
> Silvina
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.