You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Aaron T. Myers (Created) (JIRA)" <ji...@apache.org> on 2012/02/27 23:19:50 UTC
[jira] [Created] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
-----------------------------------------------------------------------
Key: HADOOP-8116
URL: https://issues.apache.org/jira/browse/HADOOP-8116
Project: Hadoop Common
Issue Type: Sub-task
Components: util
Affects Versions: HA Branch (HDFS-1623)
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8116:
-----------------------------------
Attachment: HADOOP-8116-HDFS-1623.patch
Thanks a lot for the review, Eli. Here's an updated patch which adds the comment you asked for. I'm going to commit this in a moment, and create the JIRA you asked for.
> HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
> -----------------------------------------------------------------------
>
> Key: HADOOP-8116
> URL: https://issues.apache.org/jira/browse/HADOOP-8116
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: HA Branch (HDFS-1623)
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HADOOP-8116-HDFS-1623.patch, HADOOP-8116-HDFS-1623.patch
>
>
> HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
> It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8116:
-----------------------------------
Attachment: HADOOP-8116-HDFS-1623.patch
Here's a patch which addresses the issue. I also noticed while fixing this issue that TestCopyMapper wouldn't have worked because HADOOP-8068 results in an additional exception wrapping, which the test wasn't taking into account.
> HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
> -----------------------------------------------------------------------
>
> Key: HADOOP-8116
> URL: https://issues.apache.org/jira/browse/HADOOP-8116
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: HA Branch (HDFS-1623)
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HADOOP-8116-HDFS-1623.patch
>
>
> HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
> It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217761#comment-13217761 ]
Aaron T. Myers commented on HADOOP-8116:
----------------------------------------
Oh, I should've mentioned - the two test cases which will fail without this patch are TestCopyMapper and TestRetriableCommand.
> HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
> -----------------------------------------------------------------------
>
> Key: HADOOP-8116
> URL: https://issues.apache.org/jira/browse/HADOOP-8116
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: HA Branch (HDFS-1623)
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HADOOP-8116-HDFS-1623.patch
>
>
> HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
> It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
Posted by "Aaron T. Myers (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers resolved HADOOP-8116.
------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
I've committed this to the HA branch and filed the following JIRA: MAPREDUCE-3934
Thanks a lot for the review, Eli.
> HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
> -----------------------------------------------------------------------
>
> Key: HADOOP-8116
> URL: https://issues.apache.org/jira/browse/HADOOP-8116
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: HA Branch (HDFS-1623)
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HADOOP-8116-HDFS-1623.patch, HADOOP-8116-HDFS-1623.patch
>
>
> HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
> It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219171#comment-13219171 ]
Hudson commented on HADOOP-8116:
--------------------------------
Integrated in Hadoop-Hdfs-HAbranch-build #92 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/92/])
HADOOP-8116. RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896. Contributed by Aaron T. Myers. (Revision 1294729)
Result = UNSTABLE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1294729
Files :
* /hadoop/common/branches/HDFS-1623/hadoop-common-project/hadoop-common/CHANGES.HDFS-1623.txt
* /hadoop/common/branches/HDFS-1623/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
* /hadoop/common/branches/HDFS-1623/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
> HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
> -----------------------------------------------------------------------
>
> Key: HADOOP-8116
> URL: https://issues.apache.org/jira/browse/HADOOP-8116
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: HA Branch (HDFS-1623)
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HADOOP-8116-HDFS-1623.patch, HADOOP-8116-HDFS-1623.patch
>
>
> HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
> It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8116) HA: RetriableCommand is using
RetryPolicy incorrectly after HADOOP-7896
Posted by "Eli Collins (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218342#comment-13218342 ]
Eli Collins commented on HADOOP-8116:
-------------------------------------
+1 looks great
Nit, the line where you pull out the nested exception in TestCopyMapper could use a comment indicating why this is necessary
Also wrt your comment that it's not kosher for RetriableCommand to be using the RetryPolicy classes, file a MR jira for that?
> HA: RetriableCommand is using RetryPolicy incorrectly after HADOOP-7896
> -----------------------------------------------------------------------
>
> Key: HADOOP-8116
> URL: https://issues.apache.org/jira/browse/HADOOP-8116
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: HA Branch (HDFS-1623)
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HADOOP-8116-HDFS-1623.patch
>
>
> HADOOP-7896 (on the HA branch) refactored RetryAction from an enum to a class, and also moved the act of sleeping to delay retries from the RetryPolicy implementations into RetryInvocationHandler. RetriableCommand, in the rewritten distcp tool, uses RetryPolicy and associated classes from o.a.h.io.retry. When MAPREDUCE-2765 was merged into the HA branch, RetriableCommand wasn't adjusted accordingly to make use of the new structure of the o.a.h.io.retry classes.
> It's probably generally not kosher for RetriableCommand to be using the RetryPolicy classes at all, since they're not really intended to be used except by RetryInvocationHandler. But, regardless, this JIRA aims to make distcp's use of the o.a.h.io.retry classes functional again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira