You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Steven Willis <sw...@compete.com> on 2012/11/09 18:51:16 UTC
mapreduce.job.end-notification settings
I was looking at:
http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
And I noticed that there are some duplicate properties with different values and different descriptions:
mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
...
mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
Any idea which values and descriptions are correct?
-Steve
Re: mapreduce.job.end-notification settings
Posted by Ravi Prakash <ra...@ymail.com>.
D-oh!
Thanks for discovering this. Sorry for my silly mistake. Filed and patched https://issues.apache.org/jira/browse/MAPREDUCE-4786 .
Thanks
Ravi
________________________________
From: Harsh J <ha...@cloudera.com>
To: user@hadoop.apache.org
Sent: Friday, November 9, 2012 2:24 PM
Subject: Re: mapreduce.job.end-notification settings
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Ravi Prakash <ra...@ymail.com>.
D-oh!
Thanks for discovering this. Sorry for my silly mistake. Filed and patched https://issues.apache.org/jira/browse/MAPREDUCE-4786 .
Thanks
Ravi
________________________________
From: Harsh J <ha...@cloudera.com>
To: user@hadoop.apache.org
Sent: Friday, November 9, 2012 2:24 PM
Subject: Re: mapreduce.job.end-notification settings
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Ravi Prakash <ra...@ymail.com>.
D-oh!
Thanks for discovering this. Sorry for my silly mistake. Filed and patched https://issues.apache.org/jira/browse/MAPREDUCE-4786 .
Thanks
Ravi
________________________________
From: Harsh J <ha...@cloudera.com>
To: user@hadoop.apache.org
Sent: Friday, November 9, 2012 2:24 PM
Subject: Re: mapreduce.job.end-notification settings
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Ravi Prakash <ra...@ymail.com>.
D-oh!
Thanks for discovering this. Sorry for my silly mistake. Filed and patched https://issues.apache.org/jira/browse/MAPREDUCE-4786 .
Thanks
Ravi
________________________________
From: Harsh J <ha...@cloudera.com>
To: user@hadoop.apache.org
Sent: Friday, November 9, 2012 2:24 PM
Subject: Re: mapreduce.job.end-notification settings
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Harsh J <ha...@cloudera.com>.
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Harsh J <ha...@cloudera.com>.
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Harsh J <ha...@cloudera.com>.
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J
Re: mapreduce.job.end-notification settings
Posted by Harsh J <ha...@cloudera.com>.
Hey Steven,
Nice catch - this may be a bug we should fix.
There are 4 relevant properties in MR2:
Attempts: "mapreduce.job.end-notification.retry.attempts" and
"mapreduce.job.end-notification.max.attempts".
Retries: "mapreduce.job.end-notification.retry.interval" and
"mapreduce.job.end-notification.max.retry.interval".
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
Could you file a JIRA for this as well as there being an issue with
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis <sw...@compete.com> wrote:
> I was looking at:
>
> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>
> And I noticed that there are some duplicate properties with different values and different descriptions:
>
> mapreduce.job.end-notification.retry.attempts 0 Indicates how many times hadoop should attempt to contact the notification URL
> mapreduce.job.end-notification.retry.interval 30000 Indicates time in milliseconds between notification URL retry calls
> ...
> mapreduce.job.end-notification.retry.attempts 5 The number of times the submitter of the job wants to retry job end notification if it fails. This is capped by mapreduce.job.end-notification.max.attempts
> mapreduce.job.end-notification.retry.interval 1 The number of seconds the submitter of the job wants to wait before job end notification is retried if it fails. This is capped by mapreduce.job.end-notification.max.retry.interval
>
> Any idea which values and descriptions are correct?
>
> -Steve
--
Harsh J