You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by maisnam ns <ma...@gmail.com> on 2013/05/21 10:02:47 UTC
Clarifications on MAPREDUCE-5183
Hi,
I was looking into this issue but would be happy if someone could clarify
some of my doubts.
Is the issue related to the given below snapshot of log:
org.apache.hadoop.mapred.TaskTracker: Task
attempt_201305211246_0001_r_000001_0 is in commit-pending, task
state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
org.apache.hadoop.mapred.TaskTracker:
attempt_201305211246_0001_r_000001_0 0.0% reduce > copy > 2013-05-21
12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
Got heartbeatResponse from JobTracker with responseId: 27 and 2
actions 2013-05-21 12:48:51,769 INFO org.apache.hadoop.mapred.TaskTracker:
Received commit task action for
attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
INFO org.apache.hadoop.mapred.TaskTracker:
Received commit task action for
attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201305211246_0001_r_000001_0 1.0% reduce >
reduce 2013-05-21 12:48:53,777 INFO
The issue says 'TaskTracker#reportProgress logging of 0.0-1.0 progress
is followed by percent sign'
1.attempt_201305211246_0001_r_000001_0 0.0% reduce
2.attempt_201305211246_0001_r_000001_0 1.0% reduce
If my understanding is correct the percent sign at the end of 0.0 and 1.0
should be removed.
Regards
Niranjan Singh
Re: Clarifications on MAPREDUCE-5183
Posted by maisnam ns <ma...@gmail.com>.
@Chris Nauroth Thanks will look into that method.
On Wed, May 22, 2013 at 2:43 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
> BTW, there is a handy helper function in Hadoop Common that can help you
> with this: org.apache.hadoop.util.StringUtils.formatPercent.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Tue, May 21, 2013 at 11:06 AM, maisnam ns <ma...@gmail.com> wrote:
>
> > Hi Sandy,
> >
> > Sure, I would do exactly what you are suggesting.
> >
> > Regards
> > Niranjan Singh
> >
> >
> > On Tue, May 21, 2013 at 11:27 PM, Sandy Ryza <sandy.ryza@cloudera.com
> > >wrote:
> >
> > > If you're planning to fix it, it would probably look prettiest to keep
> > the
> > > percentage sign and have the numbers between 0 and 100.
> > >
> > > -Sandy
> > >
> > >
> > > On Tue, May 21, 2013 at 10:55 AM, maisnam ns <ma...@gmail.com>
> > wrote:
> > >
> > > > Thanks Sandy Ryza
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, May 21, 2013 at 11:20 PM, Sandy Ryza <
> sandy.ryza@cloudera.com
> > > > >wrote:
> > > >
> > > > > Hi Niranjan,
> > > > >
> > > > > Your understanding is correct.
> > > > >
> > > > > -Sandy
> > > > >
> > > > >
> > > > > On Tue, May 21, 2013 at 1:02 AM, maisnam ns <ma...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I was looking into this issue but would be happy if someone could
> > > > clarify
> > > > > > some of my doubts.
> > > > > >
> > > > > > Is the issue related to the given below snapshot of log:
> > > > > >
> > > > > > org.apache.hadoop.mapred.TaskTracker: Task
> > > > > > attempt_201305211246_0001_r_000001_0 is in commit-pending, task
> > > > > > state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
> > > > > > org.apache.hadoop.mapred.TaskTracker:
> > > > > > attempt_201305211246_0001_r_000001_0 0.0% reduce > copy >
> > 2013-05-21
> > > > > > 12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
> > > > > > Got heartbeatResponse from JobTracker with responseId: 27 and 2
> > > > > > actions 2013-05-21 12:48:51,769 INFO
> > > > > org.apache.hadoop.mapred.TaskTracker:
> > > > > > Received commit task action for
> > > > > > attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
> > > > > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > > > > Received commit task action for
> > > > > > attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
> > > > > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > > > > attempt_201305211246_0001_r_000001_0 1.0% reduce >
> > > > > > reduce 2013-05-21 12:48:53,777 INFO
> > > > > >
> > > > > > The issue says 'TaskTracker#reportProgress logging of 0.0-1.0
> > > progress
> > > > > > is followed by percent sign'
> > > > > >
> > > > > >
> > > > > > 1.attempt_201305211246_0001_r_000001_0 0.0% reduce
> > > > > > 2.attempt_201305211246_0001_r_000001_0 1.0% reduce
> > > > > >
> > > > > > If my understanding is correct the percent sign at the end of 0.0
> > and
> > > > 1.0
> > > > > >
> > > > > > should be removed.
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > Niranjan Singh
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Clarifications on MAPREDUCE-5183
Posted by Chris Nauroth <cn...@hortonworks.com>.
BTW, there is a handy helper function in Hadoop Common that can help you
with this: org.apache.hadoop.util.StringUtils.formatPercent.
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Tue, May 21, 2013 at 11:06 AM, maisnam ns <ma...@gmail.com> wrote:
> Hi Sandy,
>
> Sure, I would do exactly what you are suggesting.
>
> Regards
> Niranjan Singh
>
>
> On Tue, May 21, 2013 at 11:27 PM, Sandy Ryza <sandy.ryza@cloudera.com
> >wrote:
>
> > If you're planning to fix it, it would probably look prettiest to keep
> the
> > percentage sign and have the numbers between 0 and 100.
> >
> > -Sandy
> >
> >
> > On Tue, May 21, 2013 at 10:55 AM, maisnam ns <ma...@gmail.com>
> wrote:
> >
> > > Thanks Sandy Ryza
> > >
> > >
> > >
> > >
> > > On Tue, May 21, 2013 at 11:20 PM, Sandy Ryza <sandy.ryza@cloudera.com
> > > >wrote:
> > >
> > > > Hi Niranjan,
> > > >
> > > > Your understanding is correct.
> > > >
> > > > -Sandy
> > > >
> > > >
> > > > On Tue, May 21, 2013 at 1:02 AM, maisnam ns <ma...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I was looking into this issue but would be happy if someone could
> > > clarify
> > > > > some of my doubts.
> > > > >
> > > > > Is the issue related to the given below snapshot of log:
> > > > >
> > > > > org.apache.hadoop.mapred.TaskTracker: Task
> > > > > attempt_201305211246_0001_r_000001_0 is in commit-pending, task
> > > > > state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
> > > > > org.apache.hadoop.mapred.TaskTracker:
> > > > > attempt_201305211246_0001_r_000001_0 0.0% reduce > copy >
> 2013-05-21
> > > > > 12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
> > > > > Got heartbeatResponse from JobTracker with responseId: 27 and 2
> > > > > actions 2013-05-21 12:48:51,769 INFO
> > > > org.apache.hadoop.mapred.TaskTracker:
> > > > > Received commit task action for
> > > > > attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
> > > > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > > > Received commit task action for
> > > > > attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
> > > > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > > > attempt_201305211246_0001_r_000001_0 1.0% reduce >
> > > > > reduce 2013-05-21 12:48:53,777 INFO
> > > > >
> > > > > The issue says 'TaskTracker#reportProgress logging of 0.0-1.0
> > progress
> > > > > is followed by percent sign'
> > > > >
> > > > >
> > > > > 1.attempt_201305211246_0001_r_000001_0 0.0% reduce
> > > > > 2.attempt_201305211246_0001_r_000001_0 1.0% reduce
> > > > >
> > > > > If my understanding is correct the percent sign at the end of 0.0
> and
> > > 1.0
> > > > >
> > > > > should be removed.
> > > > >
> > > > > Regards
> > > > >
> > > > > Niranjan Singh
> > > > >
> > > >
> > >
> >
>
Re: Clarifications on MAPREDUCE-5183
Posted by maisnam ns <ma...@gmail.com>.
Hi Sandy,
Sure, I would do exactly what you are suggesting.
Regards
Niranjan Singh
On Tue, May 21, 2013 at 11:27 PM, Sandy Ryza <sa...@cloudera.com>wrote:
> If you're planning to fix it, it would probably look prettiest to keep the
> percentage sign and have the numbers between 0 and 100.
>
> -Sandy
>
>
> On Tue, May 21, 2013 at 10:55 AM, maisnam ns <ma...@gmail.com> wrote:
>
> > Thanks Sandy Ryza
> >
> >
> >
> >
> > On Tue, May 21, 2013 at 11:20 PM, Sandy Ryza <sandy.ryza@cloudera.com
> > >wrote:
> >
> > > Hi Niranjan,
> > >
> > > Your understanding is correct.
> > >
> > > -Sandy
> > >
> > >
> > > On Tue, May 21, 2013 at 1:02 AM, maisnam ns <ma...@gmail.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I was looking into this issue but would be happy if someone could
> > clarify
> > > > some of my doubts.
> > > >
> > > > Is the issue related to the given below snapshot of log:
> > > >
> > > > org.apache.hadoop.mapred.TaskTracker: Task
> > > > attempt_201305211246_0001_r_000001_0 is in commit-pending, task
> > > > state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
> > > > org.apache.hadoop.mapred.TaskTracker:
> > > > attempt_201305211246_0001_r_000001_0 0.0% reduce > copy > 2013-05-21
> > > > 12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
> > > > Got heartbeatResponse from JobTracker with responseId: 27 and 2
> > > > actions 2013-05-21 12:48:51,769 INFO
> > > org.apache.hadoop.mapred.TaskTracker:
> > > > Received commit task action for
> > > > attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
> > > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > > Received commit task action for
> > > > attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
> > > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > > attempt_201305211246_0001_r_000001_0 1.0% reduce >
> > > > reduce 2013-05-21 12:48:53,777 INFO
> > > >
> > > > The issue says 'TaskTracker#reportProgress logging of 0.0-1.0
> progress
> > > > is followed by percent sign'
> > > >
> > > >
> > > > 1.attempt_201305211246_0001_r_000001_0 0.0% reduce
> > > > 2.attempt_201305211246_0001_r_000001_0 1.0% reduce
> > > >
> > > > If my understanding is correct the percent sign at the end of 0.0 and
> > 1.0
> > > >
> > > > should be removed.
> > > >
> > > > Regards
> > > >
> > > > Niranjan Singh
> > > >
> > >
> >
>
Re: Clarifications on MAPREDUCE-5183
Posted by Sandy Ryza <sa...@cloudera.com>.
If you're planning to fix it, it would probably look prettiest to keep the
percentage sign and have the numbers between 0 and 100.
-Sandy
On Tue, May 21, 2013 at 10:55 AM, maisnam ns <ma...@gmail.com> wrote:
> Thanks Sandy Ryza
>
>
>
>
> On Tue, May 21, 2013 at 11:20 PM, Sandy Ryza <sandy.ryza@cloudera.com
> >wrote:
>
> > Hi Niranjan,
> >
> > Your understanding is correct.
> >
> > -Sandy
> >
> >
> > On Tue, May 21, 2013 at 1:02 AM, maisnam ns <ma...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I was looking into this issue but would be happy if someone could
> clarify
> > > some of my doubts.
> > >
> > > Is the issue related to the given below snapshot of log:
> > >
> > > org.apache.hadoop.mapred.TaskTracker: Task
> > > attempt_201305211246_0001_r_000001_0 is in commit-pending, task
> > > state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
> > > org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201305211246_0001_r_000001_0 0.0% reduce > copy > 2013-05-21
> > > 12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
> > > Got heartbeatResponse from JobTracker with responseId: 27 and 2
> > > actions 2013-05-21 12:48:51,769 INFO
> > org.apache.hadoop.mapred.TaskTracker:
> > > Received commit task action for
> > > attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
> > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > Received commit task action for
> > > attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
> > > INFO org.apache.hadoop.mapred.TaskTracker:
> > > attempt_201305211246_0001_r_000001_0 1.0% reduce >
> > > reduce 2013-05-21 12:48:53,777 INFO
> > >
> > > The issue says 'TaskTracker#reportProgress logging of 0.0-1.0 progress
> > > is followed by percent sign'
> > >
> > >
> > > 1.attempt_201305211246_0001_r_000001_0 0.0% reduce
> > > 2.attempt_201305211246_0001_r_000001_0 1.0% reduce
> > >
> > > If my understanding is correct the percent sign at the end of 0.0 and
> 1.0
> > >
> > > should be removed.
> > >
> > > Regards
> > >
> > > Niranjan Singh
> > >
> >
>
Re: Clarifications on MAPREDUCE-5183
Posted by maisnam ns <ma...@gmail.com>.
Thanks Sandy Ryza
On Tue, May 21, 2013 at 11:20 PM, Sandy Ryza <sa...@cloudera.com>wrote:
> Hi Niranjan,
>
> Your understanding is correct.
>
> -Sandy
>
>
> On Tue, May 21, 2013 at 1:02 AM, maisnam ns <ma...@gmail.com> wrote:
>
> > Hi,
> >
> > I was looking into this issue but would be happy if someone could clarify
> > some of my doubts.
> >
> > Is the issue related to the given below snapshot of log:
> >
> > org.apache.hadoop.mapred.TaskTracker: Task
> > attempt_201305211246_0001_r_000001_0 is in commit-pending, task
> > state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
> > org.apache.hadoop.mapred.TaskTracker:
> > attempt_201305211246_0001_r_000001_0 0.0% reduce > copy > 2013-05-21
> > 12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
> > Got heartbeatResponse from JobTracker with responseId: 27 and 2
> > actions 2013-05-21 12:48:51,769 INFO
> org.apache.hadoop.mapred.TaskTracker:
> > Received commit task action for
> > attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
> > INFO org.apache.hadoop.mapred.TaskTracker:
> > Received commit task action for
> > attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
> > INFO org.apache.hadoop.mapred.TaskTracker:
> > attempt_201305211246_0001_r_000001_0 1.0% reduce >
> > reduce 2013-05-21 12:48:53,777 INFO
> >
> > The issue says 'TaskTracker#reportProgress logging of 0.0-1.0 progress
> > is followed by percent sign'
> >
> >
> > 1.attempt_201305211246_0001_r_000001_0 0.0% reduce
> > 2.attempt_201305211246_0001_r_000001_0 1.0% reduce
> >
> > If my understanding is correct the percent sign at the end of 0.0 and 1.0
> >
> > should be removed.
> >
> > Regards
> >
> > Niranjan Singh
> >
>
Re: Clarifications on MAPREDUCE-5183
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Niranjan,
Your understanding is correct.
-Sandy
On Tue, May 21, 2013 at 1:02 AM, maisnam ns <ma...@gmail.com> wrote:
> Hi,
>
> I was looking into this issue but would be happy if someone could clarify
> some of my doubts.
>
> Is the issue related to the given below snapshot of log:
>
> org.apache.hadoop.mapred.TaskTracker: Task
> attempt_201305211246_0001_r_000001_0 is in commit-pending, task
> state:COMMIT_PENDING 2013-05-21 12:48:51,058 INFO
> org.apache.hadoop.mapred.TaskTracker:
> attempt_201305211246_0001_r_000001_0 0.0% reduce > copy > 2013-05-21
> 12:48:51,769 DEBUG org.apache.hadoop.mapred.TaskTracker:
> Got heartbeatResponse from JobTracker with responseId: 27 and 2
> actions 2013-05-21 12:48:51,769 INFO org.apache.hadoop.mapred.TaskTracker:
> Received commit task action for
> attempt_201305211246_0001_r_000000_0 2013-05-21 12:48:51,769
> INFO org.apache.hadoop.mapred.TaskTracker:
> Received commit task action for
> attempt_201305211246_0001_r_000001_0 2013-05-21 12:48:53,695
> INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201305211246_0001_r_000001_0 1.0% reduce >
> reduce 2013-05-21 12:48:53,777 INFO
>
> The issue says 'TaskTracker#reportProgress logging of 0.0-1.0 progress
> is followed by percent sign'
>
>
> 1.attempt_201305211246_0001_r_000001_0 0.0% reduce
> 2.attempt_201305211246_0001_r_000001_0 1.0% reduce
>
> If my understanding is correct the percent sign at the end of 0.0 and 1.0
>
> should be removed.
>
> Regards
>
> Niranjan Singh
>