You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Mithun Radhakrishnan (JIRA)" <ji...@apache.org> on 2015/12/22 00:00:48 UTC
[jira] [Commented] (HADOOP-12473) distcp's ignoring failures should
be mutually exclusive with the atomic option
[ https://issues.apache.org/jira/browse/HADOOP-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067214#comment-15067214 ]
Mithun Radhakrishnan commented on HADOOP-12473:
-----------------------------------------------
[~jira.shegalov], [~liuml07], could you please explain the reasoning behind this fix?
If I understand correctly, Gera's orthogonal suggestion in HADOOP-12469 was to make {{-atomic}} and {{-i}} mutually exclusive. The latest patch in this JIRA doesn't seem to address this concern, AFAICT. It makes {{ignoreFailures}} an {{AtomicBoolean}}, which is not what Gera was getting at, I believe.
Also, [~jira.shegalov], why do you recommend that {{-atomic}} and {{-i}} be mutually exclusive? Aren't they orthogonal concerns? Why consider {{-atomic}} as incapable of ignoring copy-errors?
> distcp's ignoring failures should be mutually exclusive with the atomic option
> ------------------------------------------------------------------------------
>
> Key: HADOOP-12473
> URL: https://issues.apache.org/jira/browse/HADOOP-12473
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 2.7.1
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HADOOP-12473.000.patch, HADOOP-12473.001.patch, HADOOP-12473.002.patch
>
>
> In {{CopyMapper::handleFailure}}, the mapper handles failure and will ignore it if no it's config key is on. Ignoring failures should be mutually exclusive with the atomic option otherwise an incomplete dir is eligible for commit defeating the purpose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)