You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by "S.L" <si...@gmail.com> on 2014/09/17 18:29:08 UTC
Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Hi All,
I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is when
I submit this job YARN the application that is running in YARN is marked as
complete even as on console its reported as only 58% complete . I have
confirmed that its also not printing the log statements that its supposed
to print when the job is actually complete .
Please see the output from the job submission console below. It just stops
at 58% and job history server and YARN cluster UI reports that this job has
already succeeded. Can someone let me know why this is happening ?
4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
Thanks.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
Does anyone know what the issue could be , is there any property setting
that is causing this behavior ?
On Wed, Sep 17, 2014 at 3:11 PM, S.L <si...@gmail.com> wrote:
>
> I don't see any hs_err file in the current directory , so I don't think
> that is the case.
>
> On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
>>
>> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
>> this seems to happen in the 4th iteration consistently, how do I confirm
>> this possibility?
>>
>> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> Is it possible that the client JVM is somehow getting killed while the
>>> YARN application finishes as usual on the cluster in the background?
>>>
>>> +Vinod
>>>
>>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>>
>>>>
>>>> Hi All,
>>>>
>>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>>> when I submit this job YARN the application that is running in YARN is
>>>> marked as complete even as on console its reported as only 58% complete . I
>>>> have confirmed that its also not printing the log statements that its
>>>> supposed to print when the job is actually complete .
>>>>
>>>> Please see the output from the job submission console below. It just
>>>> stops at 58% and job history server and YARN cluster UI reports that this
>>>> job has already succeeded. Can someone let me know why this is happening ?
>>>>
>>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>>
>>>> Thanks.
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
Does anyone know what the issue could be , is there any property setting
that is causing this behavior ?
On Wed, Sep 17, 2014 at 3:11 PM, S.L <si...@gmail.com> wrote:
>
> I don't see any hs_err file in the current directory , so I don't think
> that is the case.
>
> On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
>>
>> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
>> this seems to happen in the 4th iteration consistently, how do I confirm
>> this possibility?
>>
>> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> Is it possible that the client JVM is somehow getting killed while the
>>> YARN application finishes as usual on the cluster in the background?
>>>
>>> +Vinod
>>>
>>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>>
>>>>
>>>> Hi All,
>>>>
>>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>>> when I submit this job YARN the application that is running in YARN is
>>>> marked as complete even as on console its reported as only 58% complete . I
>>>> have confirmed that its also not printing the log statements that its
>>>> supposed to print when the job is actually complete .
>>>>
>>>> Please see the output from the job submission console below. It just
>>>> stops at 58% and job history server and YARN cluster UI reports that this
>>>> job has already succeeded. Can someone let me know why this is happening ?
>>>>
>>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>>
>>>> Thanks.
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
Does anyone know what the issue could be , is there any property setting
that is causing this behavior ?
On Wed, Sep 17, 2014 at 3:11 PM, S.L <si...@gmail.com> wrote:
>
> I don't see any hs_err file in the current directory , so I don't think
> that is the case.
>
> On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
>>
>> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
>> this seems to happen in the 4th iteration consistently, how do I confirm
>> this possibility?
>>
>> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> Is it possible that the client JVM is somehow getting killed while the
>>> YARN application finishes as usual on the cluster in the background?
>>>
>>> +Vinod
>>>
>>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>>
>>>>
>>>> Hi All,
>>>>
>>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>>> when I submit this job YARN the application that is running in YARN is
>>>> marked as complete even as on console its reported as only 58% complete . I
>>>> have confirmed that its also not printing the log statements that its
>>>> supposed to print when the job is actually complete .
>>>>
>>>> Please see the output from the job submission console below. It just
>>>> stops at 58% and job history server and YARN cluster UI reports that this
>>>> job has already succeeded. Can someone let me know why this is happening ?
>>>>
>>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>>
>>>> Thanks.
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
Does anyone know what the issue could be , is there any property setting
that is causing this behavior ?
On Wed, Sep 17, 2014 at 3:11 PM, S.L <si...@gmail.com> wrote:
>
> I don't see any hs_err file in the current directory , so I don't think
> that is the case.
>
> On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
>>
>> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
>> this seems to happen in the 4th iteration consistently, how do I confirm
>> this possibility?
>>
>> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> Is it possible that the client JVM is somehow getting killed while the
>>> YARN application finishes as usual on the cluster in the background?
>>>
>>> +Vinod
>>>
>>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>>
>>>>
>>>> Hi All,
>>>>
>>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>>> when I submit this job YARN the application that is running in YARN is
>>>> marked as complete even as on console its reported as only 58% complete . I
>>>> have confirmed that its also not printing the log statements that its
>>>> supposed to print when the job is actually complete .
>>>>
>>>> Please see the output from the job submission console below. It just
>>>> stops at 58% and job history server and YARN cluster UI reports that this
>>>> job has already succeeded. Can someone let me know why this is happening ?
>>>>
>>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>>
>>>> Thanks.
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I don't see any hs_err file in the current directory , so I don't think
that is the case.
On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
> this seems to happen in the 4th iteration consistently, how do I confirm
> this possibility?
>
> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> Is it possible that the client JVM is somehow getting killed while the
>> YARN application finishes as usual on the cluster in the background?
>>
>> +Vinod
>>
>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>
>>>
>>> Hi All,
>>>
>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>> when I submit this job YARN the application that is running in YARN is
>>> marked as complete even as on console its reported as only 58% complete . I
>>> have confirmed that its also not printing the log statements that its
>>> supposed to print when the job is actually complete .
>>>
>>> Please see the output from the job submission console below. It just
>>> stops at 58% and job history server and YARN cluster UI reports that this
>>> job has already succeeded. Can someone let me know why this is happening ?
>>>
>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>
>>> Thanks.
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I don't see any hs_err file in the current directory , so I don't think
that is the case.
On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
> this seems to happen in the 4th iteration consistently, how do I confirm
> this possibility?
>
> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> Is it possible that the client JVM is somehow getting killed while the
>> YARN application finishes as usual on the cluster in the background?
>>
>> +Vinod
>>
>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>
>>>
>>> Hi All,
>>>
>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>> when I submit this job YARN the application that is running in YARN is
>>> marked as complete even as on console its reported as only 58% complete . I
>>> have confirmed that its also not printing the log statements that its
>>> supposed to print when the job is actually complete .
>>>
>>> Please see the output from the job submission console below. It just
>>> stops at 58% and job history server and YARN cluster UI reports that this
>>> job has already succeeded. Can someone let me know why this is happening ?
>>>
>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>
>>> Thanks.
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I don't see any hs_err file in the current directory , so I don't think
that is the case.
On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
> this seems to happen in the 4th iteration consistently, how do I confirm
> this possibility?
>
> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> Is it possible that the client JVM is somehow getting killed while the
>> YARN application finishes as usual on the cluster in the background?
>>
>> +Vinod
>>
>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>
>>>
>>> Hi All,
>>>
>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>> when I submit this job YARN the application that is running in YARN is
>>> marked as complete even as on console its reported as only 58% complete . I
>>> have confirmed that its also not printing the log statements that its
>>> supposed to print when the job is actually complete .
>>>
>>> Please see the output from the job submission console below. It just
>>> stops at 58% and job history server and YARN cluster UI reports that this
>>> job has already succeeded. Can someone let me know why this is happening ?
>>>
>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>
>>> Thanks.
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I don't see any hs_err file in the current directory , so I don't think
that is the case.
On Wed, Sep 17, 2014 at 2:21 PM, S.L <si...@gmail.com> wrote:
>
> I am not sure, I running a sequence of MRV1 jobs using a bash script ,
> this seems to happen in the 4th iteration consistently, how do I confirm
> this possibility?
>
> On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> Is it possible that the client JVM is somehow getting killed while the
>> YARN application finishes as usual on the cluster in the background?
>>
>> +Vinod
>>
>> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>>
>>>
>>> Hi All,
>>>
>>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>>> when I submit this job YARN the application that is running in YARN is
>>> marked as complete even as on console its reported as only 58% complete . I
>>> have confirmed that its also not printing the log statements that its
>>> supposed to print when the job is actually complete .
>>>
>>> Please see the output from the job submission console below. It just
>>> stops at 58% and job history server and YARN cluster UI reports that this
>>> job has already succeeded. Can someone let me know why this is happening ?
>>>
>>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>>
>>> Thanks.
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I am not sure, I running a sequence of MRV1 jobs using a bash script , this
seems to happen in the 4th iteration consistently, how do I confirm this
possibility?
On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Is it possible that the client JVM is somehow getting killed while the
> YARN application finishes as usual on the cluster in the background?
>
> +Vinod
>
> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
>>
>> Hi All,
>>
>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>> when I submit this job YARN the application that is running in YARN is
>> marked as complete even as on console its reported as only 58% complete . I
>> have confirmed that its also not printing the log statements that its
>> supposed to print when the job is actually complete .
>>
>> Please see the output from the job submission console below. It just
>> stops at 58% and job history server and YARN cluster UI reports that this
>> job has already succeeded. Can someone let me know why this is happening ?
>>
>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>
>> Thanks.
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I am not sure, I running a sequence of MRV1 jobs using a bash script , this
seems to happen in the 4th iteration consistently, how do I confirm this
possibility?
On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Is it possible that the client JVM is somehow getting killed while the
> YARN application finishes as usual on the cluster in the background?
>
> +Vinod
>
> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
>>
>> Hi All,
>>
>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>> when I submit this job YARN the application that is running in YARN is
>> marked as complete even as on console its reported as only 58% complete . I
>> have confirmed that its also not printing the log statements that its
>> supposed to print when the job is actually complete .
>>
>> Please see the output from the job submission console below. It just
>> stops at 58% and job history server and YARN cluster UI reports that this
>> job has already succeeded. Can someone let me know why this is happening ?
>>
>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>
>> Thanks.
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I am not sure, I running a sequence of MRV1 jobs using a bash script , this
seems to happen in the 4th iteration consistently, how do I confirm this
possibility?
On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Is it possible that the client JVM is somehow getting killed while the
> YARN application finishes as usual on the cluster in the background?
>
> +Vinod
>
> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
>>
>> Hi All,
>>
>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>> when I submit this job YARN the application that is running in YARN is
>> marked as complete even as on console its reported as only 58% complete . I
>> have confirmed that its also not printing the log statements that its
>> supposed to print when the job is actually complete .
>>
>> Please see the output from the job submission console below. It just
>> stops at 58% and job history server and YARN cluster UI reports that this
>> job has already succeeded. Can someone let me know why this is happening ?
>>
>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>
>> Thanks.
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by "S.L" <si...@gmail.com>.
I am not sure, I running a sequence of MRV1 jobs using a bash script , this
seems to happen in the 4th iteration consistently, how do I confirm this
possibility?
On Wed, Sep 17, 2014 at 1:34 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Is it possible that the client JVM is somehow getting killed while the
> YARN application finishes as usual on the cluster in the background?
>
> +Vinod
>
> On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
>>
>> Hi All,
>>
>> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is
>> when I submit this job YARN the application that is running in YARN is
>> marked as complete even as on console its reported as only 58% complete . I
>> have confirmed that its also not printing the log statements that its
>> supposed to print when the job is actually complete .
>>
>> Please see the output from the job submission console below. It just
>> stops at 58% and job history server and YARN cluster UI reports that this
>> job has already succeeded. Can someone let me know why this is happening ?
>>
>> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
>> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
>> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
>> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
>> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>>
>> Thanks.
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Is it possible that the client JVM is somehow getting killed while the YARN
application finishes as usual on the cluster in the background?
+Vinod
On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
> Hi All,
>
> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is when
> I submit this job YARN the application that is running in YARN is marked as
> complete even as on console its reported as only 58% complete . I have
> confirmed that its also not printing the log statements that its supposed
> to print when the job is actually complete .
>
> Please see the output from the job submission console below. It just stops
> at 58% and job history server and YARN cluster UI reports that this job has
> already succeeded. Can someone let me know why this is happening ?
>
> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>
> Thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Is it possible that the client JVM is somehow getting killed while the YARN
application finishes as usual on the cluster in the background?
+Vinod
On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
> Hi All,
>
> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is when
> I submit this job YARN the application that is running in YARN is marked as
> complete even as on console its reported as only 58% complete . I have
> confirmed that its also not printing the log statements that its supposed
> to print when the job is actually complete .
>
> Please see the output from the job submission console below. It just stops
> at 58% and job history server and YARN cluster UI reports that this job has
> already succeeded. Can someone let me know why this is happening ?
>
> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>
> Thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Is it possible that the client JVM is somehow getting killed while the YARN
application finishes as usual on the cluster in the background?
+Vinod
On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
> Hi All,
>
> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is when
> I submit this job YARN the application that is running in YARN is marked as
> complete even as on console its reported as only 58% complete . I have
> confirmed that its also not printing the log statements that its supposed
> to print when the job is actually complete .
>
> Please see the output from the job submission console below. It just stops
> at 58% and job history server and YARN cluster UI reports that this job has
> already succeeded. Can someone let me know why this is happening ?
>
> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>
> Thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Map Reduce Job is reported as complete on history server while on
console it shows as only half way thru
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Is it possible that the client JVM is somehow getting killed while the YARN
application finishes as usual on the cluster in the background?
+Vinod
On Wed, Sep 17, 2014 at 9:29 AM, S.L <si...@gmail.com> wrote:
>
> Hi All,
>
> I am running a MRV1 job on Hadoop YARN 2.3.0 cluster , the problem is when
> I submit this job YARN the application that is running in YARN is marked as
> complete even as on console its reported as only 58% complete . I have
> confirmed that its also not printing the log statements that its supposed
> to print when the job is actually complete .
>
> Please see the output from the job submission console below. It just stops
> at 58% and job history server and YARN cluster UI reports that this job has
> already succeeded. Can someone let me know why this is happening ?
>
> 4/08/28 08:36:19 INFO mapreduce.Job: map 54% reduce 0%
> 14/08/28 08:44:13 INFO mapreduce.Job: map 55% reduce 0%
> 14/08/28 08:52:16 INFO mapreduce.Job: map 56% reduce 0%
> 14/08/28 08:59:22 INFO mapreduce.Job: map 57% reduce 0%
> 14/08/28 09:07:33 INFO mapreduce.Job: map 58% reduce 0%
>
> Thanks.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.