You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Peter Romianowski (JIRA)" <ji...@apache.org> on 2009/12/09 19:52:18 UTC
[jira] Created: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
DistCp cannot handle -delete if destination is local filesystem
---------------------------------------------------------------
Key: MAPREDUCE-1285
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: distcp
Affects Versions: 0.20.1
Reporter: Peter Romianowski
The following exception is thrown:
{code}
Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated MAPREDUCE-1285:
----------------------------------------------
Resolution: Fixed
Fix Version/s: 0.22.0
Status: Resolved (was: Patch Available)
I have committed this. Thanks, Peter!
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Assignee: Peter Romianowski
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Peter Romianowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Romianowski updated MAPREDUCE-1285:
-----------------------------------------
Attachment: MAPREDUCE-1285.patch
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Peter Romianowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788254#action_12788254 ]
Peter Romianowski commented on MAPREDUCE-1285:
----------------------------------------------
I don't think that the patch breaks anything at this place. I only changed a single line in deleteNonexisting() where FilePair is not even used. The problem has been that the SequenceFile.Writer was tied to FileStatus.class but it didn't have to since we know the concrete implementation.
So the patch should be save even if someone introduces new fields in RawLocalFileStatus.
Did I miss something?
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788400#action_12788400 ]
Hadoop QA commented on MAPREDUCE-1285:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12427517/MAPREDUCE-1285-trunk.patch
against trunk revision 888761.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/178/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/178/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/178/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/178/console
This message is automatically generated.
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Assignee: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788263#action_12788263 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-1285:
---------------------------------------------------
Hi Peter, I am not saying that your patch is breaking anything. I mean that when RawLocalFileStatus is created, it is known that RawLocalFileStatus would break DistCp.FilePair.
It seems that your patch is not generated from trunk since the line numbers do not match. Could you generate a new patch from trunk?
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789536#action_12789536 ]
Hudson commented on MAPREDUCE-1285:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #171 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/171/])
. In DistCp.deleteNonexisting(..), get class from the parameter instead of using FileStatus.class. Contributed by Peter Romianowski
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Assignee: Peter Romianowski
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Peter Romianowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788290#action_12788290 ]
Peter Romianowski commented on MAPREDUCE-1285:
----------------------------------------------
Added patch against trunk.
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1285:
-------------------------------------
Status: Open (was: Patch Available)
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Peter Romianowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Romianowski updated MAPREDUCE-1285:
-----------------------------------------
Attachment: MAPREDUCE-1285-trunk.patch
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788245#action_12788245 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-1285:
---------------------------------------------------
Interestingly, there is a comment saying that RawLocalFileStatus would break DistCp.FilePair. (Note that CopyFiles is a former name of DistCp.)
{code}
static class RawLocalFileStatus extends FileStatus {
/* We can add extra fields here. It breaks at least CopyFiles.FilePair().
* We recognize if the information is already loaded by check if
* onwer.equals("").
*/
{code}
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Peter Romianowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Romianowski updated MAPREDUCE-1285:
-----------------------------------------
Status: Patch Available (was: Open)
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788486#action_12788486 ]
Hadoop QA commented on MAPREDUCE-1285:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12427517/MAPREDUCE-1285-trunk.patch
against trunk revision 888761.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/179/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/179/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/179/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/179/console
This message is automatically generated.
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Assignee: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated MAPREDUCE-1285:
----------------------------------------------
Hadoop Flags: [Reviewed]
+1 patch looks good.
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated MAPREDUCE-1285:
-------------------------------------
Assignee: Peter Romianowski
Status: Patch Available (was: Open)
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Assignee: Peter Romianowski
> Attachments: MAPREDUCE-1285-trunk.patch, MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if
destination is local filesystem
Posted by "Peter Romianowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788277#action_12788277 ]
Peter Romianowski commented on MAPREDUCE-1285:
----------------------------------------------
Patch is against 0.20.1 (git://github.com/optivo/hadoop-0.20.1.git). I'll add one for trunk.
> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-1285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Peter Romianowski
> Attachments: MAPREDUCE-1285.patch
>
>
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus is not class org.apache.hadoop.fs.FileStatus
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.