You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by la...@gmail.com on 2009/12/27 11:20:08 UTC
回复: hadoop job progress going back
Task failed?
---- 原始信息 ----
发自: Doopah Shaf <do...@gmail.com>
发送时间:
收件人: common-user@hadoop.apache.org
主题: hadoop job progress going back
Running jobs sometimes displays progress as follows:
map 100% reduce 10%
and then
map 100% reduce 0%
What causes this? Is this speculative execution restarting tasks?
Cheers,
DS
Re: hadoop job progress going back
Posted by Jason Venner <ja...@gmail.com>.
There are some issues with the way counter values are collected and
summarized that cause this display behavior.
It partially has to do with speculative execution.
I believe it is fixed in the later versions of 0.19 and beyond.
I can't remember or find the jira associated with this.
On Sun, Dec 27, 2009 at 2:28 AM, Jeff Zhang <zj...@gmail.com> wrote:
> No, if you look at the click the job id and watch the details of this job,
> some task must failed.
>
>
> Jeff Zhang
>
> 2009/12/27 Doopah Shaf <do...@gmail.com>
>
> > The web UI doesn't show any failures.
> >
> > And it does show several reduce tasks with progress = 475.70%... any
> clues?
> >
> > On Sun, Dec 27, 2009 at 12:20 PM, <la...@gmail.com> wrote:
> >
> > > Task failed?
> > >
> > > ---- 原始信息 ----
> > > 发自: Doopah Shaf <do...@gmail.com>
> > > 发送时间:
> > > 收件人: common-user@hadoop.apache.org
> > > 主题: hadoop job progress going back
> > >
> > > Running jobs sometimes displays progress as follows:
> > >
> > > map 100% reduce 10%
> > >
> > > and then
> > >
> > > map 100% reduce 0%
> > >
> > > What causes this? Is this speculative execution restarting tasks?
> > >
> > > Cheers,
> > > DS
> > >
> >
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: hadoop job progress going back
Posted by Jeff Zhang <zj...@gmail.com>.
No, if you look at the click the job id and watch the details of this job,
some task must failed.
Jeff Zhang
2009/12/27 Doopah Shaf <do...@gmail.com>
> The web UI doesn't show any failures.
>
> And it does show several reduce tasks with progress = 475.70%... any clues?
>
> On Sun, Dec 27, 2009 at 12:20 PM, <la...@gmail.com> wrote:
>
> > Task failed?
> >
> > ---- 原始信息 ----
> > 发自: Doopah Shaf <do...@gmail.com>
> > 发送时间:
> > 收件人: common-user@hadoop.apache.org
> > 主题: hadoop job progress going back
> >
> > Running jobs sometimes displays progress as follows:
> >
> > map 100% reduce 10%
> >
> > and then
> >
> > map 100% reduce 0%
> >
> > What causes this? Is this speculative execution restarting tasks?
> >
> > Cheers,
> > DS
> >
>
Re: hadoop job progress going back
Posted by Farhan Husain <fa...@csebuet.org>.
What version are you using?
2009/12/27 Doopah Shaf <do...@gmail.com>
> The web UI doesn't show any failures.
>
> And it does show several reduce tasks with progress = 475.70%... any clues?
>
> On Sun, Dec 27, 2009 at 12:20 PM, <la...@gmail.com> wrote:
>
> > Task failed?
> >
> > ---- 原始信息 ----
> > 发自: Doopah Shaf <do...@gmail.com>
> > 发送时间:
> > 收件人: common-user@hadoop.apache.org
> > 主题: hadoop job progress going back
> >
> > Running jobs sometimes displays progress as follows:
> >
> > map 100% reduce 10%
> >
> > and then
> >
> > map 100% reduce 0%
> >
> > What causes this? Is this speculative execution restarting tasks?
> >
> > Cheers,
> > DS
> >
>
Re: hadoop job progress going back
Posted by Doopah Shaf <do...@gmail.com>.
The web UI doesn't show any failures.
And it does show several reduce tasks with progress = 475.70%... any clues?
On Sun, Dec 27, 2009 at 12:20 PM, <la...@gmail.com> wrote:
> Task failed?
>
> ---- 原始信息 ----
> 发自: Doopah Shaf <do...@gmail.com>
> 发送时间:
> 收件人: common-user@hadoop.apache.org
> 主题: hadoop job progress going back
>
> Running jobs sometimes displays progress as follows:
>
> map 100% reduce 10%
>
> and then
>
> map 100% reduce 0%
>
> What causes this? Is this speculative execution restarting tasks?
>
> Cheers,
> DS
>