You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Dennis Kubes (JIRA)" <ji...@apache.org> on 2007/10/12 17:57:51 UTC

[jira] Commented: (HADOOP-1795) Task.moveTaskOutputs is escaping special characters in output filenames

    [ https://issues.apache.org/jira/browse/HADOOP-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534319 ] 

Dennis Kubes commented on HADOOP-1795:
--------------------------------------

This patch breaks the Injector job within Nutch.

java.io.IOException: Target file:/c:/nutch/hadoop/mapred/temp/inject-temp-479521103/_reduce_xtsclf/part-00000 already exists
        at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:125)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:116)
        at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:180)
        at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:380)
        at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:452)
        at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:469)
        at org.apache.hadoop.mapred.Task.saveTaskOutput(Task.java:426)

> Task.moveTaskOutputs is escaping special characters in output filenames
> -----------------------------------------------------------------------
>
>                 Key: HADOOP-1795
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1795
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.14.0
>            Reporter: Frédéric Bertin
>            Priority: Critical
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1795.patch
>
>
> after a migration from 0.10.1 to 0.14.0, jobs can't generate output files with special characters in their name, just like '[' or ']' for example, because they are escaped during the {{Task.moveTaskOutputs}} process.
> For example, if you try to generate an output file named {{/foo/bar[0]}}, it ends up being named {{/foo/bar%5B0%5B}}.
> The culprit is {{Task.getFinalPath()}}, when it does {{relativePath.toString()}}, where I think it should do {{relativePath.getPath()}}.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.