You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2017/12/05 01:52:00 UTC
[jira] [Commented] (HADOOP-3) Output directories are not cleaned up
before the reduces run
[ https://issues.apache.org/jira/browse/HADOOP-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16277883#comment-16277883 ]
Hudson commented on HADOOP-3:
-----------------------------
FAILURE: Integrated in Jenkins build HBase-1.4 #1049 (See [https://builds.apache.org/job/HBase-1.4/1049/])
HBASE-18942 hbase-hadoop2-compat module ignores hadoop-3 profile (apurtell: rev 9c8c0e08a66ced7dffb232facc51079e1de8a722)
* (edit) hbase-hadoop2-compat/pom.xml
> Output directories are not cleaned up before the reduces run
> ------------------------------------------------------------
>
> Key: HADOOP-3
> URL: https://issues.apache.org/jira/browse/HADOOP-3
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Priority: Minor
> Fix For: 0.1.0
>
> Attachments: clean-out-dir.patch, noclobber.patch
>
>
> The output directory for the reduces is not cleaned up and therefore if you can see left overs from previous runs, if they had more reduces. For example, if you run the application once with reduces=10 and then rerun with reduces=8, your output directory will have frag00000 to frag00009 with the first 8 fragments from the second run and the last 2 fragments from the first run.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org