You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Dennis Kubes (JIRA)" <ji...@apache.org> on 2007/10/12 17:57:51 UTC
[jira] Commented: (HADOOP-1795) Task.moveTaskOutputs is escaping
special characters in output filenames
[ https://issues.apache.org/jira/browse/HADOOP-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534319 ]
Dennis Kubes commented on HADOOP-1795:
--------------------------------------
This patch breaks the Injector job within Nutch.
java.io.IOException: Target file:/c:/nutch/hadoop/mapred/temp/inject-temp-479521103/_reduce_xtsclf/part-00000 already exists
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:125)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:116)
at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:180)
at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:380)
at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:452)
at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:469)
at org.apache.hadoop.mapred.Task.saveTaskOutput(Task.java:426)
> Task.moveTaskOutputs is escaping special characters in output filenames
> -----------------------------------------------------------------------
>
> Key: HADOOP-1795
> URL: https://issues.apache.org/jira/browse/HADOOP-1795
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Frédéric Bertin
> Priority: Critical
> Fix For: 0.15.0
>
> Attachments: HADOOP-1795.patch
>
>
> after a migration from 0.10.1 to 0.14.0, jobs can't generate output files with special characters in their name, just like '[' or ']' for example, because they are escaped during the {{Task.moveTaskOutputs}} process.
> For example, if you try to generate an output file named {{/foo/bar[0]}}, it ends up being named {{/foo/bar%5B0%5B}}.
> The culprit is {{Task.getFinalPath()}}, when it does {{relativePath.toString()}}, where I think it should do {{relativePath.getPath()}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.