You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Chris Nauroth (JIRA)" <ji...@apache.org> on 2010/10/25 08:41:21 UTC
[jira] Created: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
hadoop fs -getmerge does not work using codebase from trunk.
------------------------------------------------------------
Key: HADOOP-7006
URL: https://issues.apache.org/jira/browse/HADOOP-7006
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.22.0
Reporter: Chris Nauroth
Fix For: 0.22.0
Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Chris Nauroth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Nauroth updated HADOOP-7006:
----------------------------------
Attachment: HADOOP-7006.patch
I've attached the patch. Could I please have a code review? Thank you.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Chris Nauroth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925045#action_12925045 ]
Chris Nauroth commented on HADOOP-7006:
---------------------------------------
Thanks, Aaron and Doug.
I'm following the directions from http://wiki.apache.org/hadoop/HowToContribute . I don't see the "Submit Patch" link to trigger Hudson though. Is there something else that I need to do?
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-7006:
---------------------------------
Attachment: HADOOP-7006.patch
Tests failed for me until I also changed the file merge to sort the files. This is probably a good idea anyway. Here's a new version of the patch with that change.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924625#action_12924625 ]
Aaron T. Myers commented on HADOOP-7006:
----------------------------------------
I've reviewed the patch, and it looks good to me.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925053#action_12925053 ]
Aaron T. Myers commented on HADOOP-7006:
----------------------------------------
Doug already marked the ticket patch available.
Not quite sure why Hudson hasn't picked it up yet - might just be taking some time to get around to it (the HDFS tests take quite a while to run.)
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-7006:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Chris!
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925330#action_12925330 ]
Hudson commented on HADOOP-7006:
--------------------------------
Integrated in Hadoop-Common-trunk #494 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/494/])
HADOOP-7006. Fix 'fs -getmerge' command to not be a no-op. Contributed by Chris Nauroth.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925099#action_12925099 ]
Doug Cutting commented on HADOOP-7006:
--------------------------------------
Ran test-patch by hand.
{code}
-1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 system tests framework. The patch passed system tests framework compile.
{code}
The javadoc test seems broken, as it fails even when I run test-patch with an empty patch file.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925159#action_12925159 ]
Hudson commented on HADOOP-7006:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #402 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/402/])
HADOOP-7006. Fix 'fs -getmerge' command to not be a no-op. Contributed by Chris Nauroth.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Chris Nauroth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924465#action_12924465 ]
Chris Nauroth commented on HADOOP-7006:
---------------------------------------
It appears that this was introduced with changeset 949658. Here is a partial diff of org.apache.hadoop.fs.FileUtil:
{noformat}
@@ -258,7 +258,7 @@
Configuration conf, String addString) throws IOException {
dstFile = checkDest(srcDir.getName(), dstFS, dstFile, false);
- if (!srcFS.getFileStatus(srcDir).isDir())
+ if (srcFS.getFileStatus(srcDir).isDirectory())
return false;
{noformat}
Notice that in addition to switching from isDir() to isDirectory(), this change also dropped the negation on the front of the condition. I'll attach a simple one-line patch to restore functionality. I've also added a unit test to cover the FileUtil.copyMerge API. I'd like to volunteer for HADOOP-6387 next, and I want this unit test in place as a regression test while I work on that.
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Fix For: 0.22.0
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7006) hadoop fs -getmerge does not work
using codebase from trunk.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-7006:
---------------------------------
Assignee: Chris Nauroth
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> hadoop fs -getmerge does not work using codebase from trunk.
> ------------------------------------------------------------
>
> Key: HADOOP-7006
> URL: https://issues.apache.org/jira/browse/HADOOP-7006
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 0.22.0
>
> Attachments: HADOOP-7006.patch, HADOOP-7006.patch
>
>
> Running the codebase from trunk, the hadoop fs -getmerge command does not work. As implemented in prior versions (i.e. 0.20.2), I could run hadoop fs -getmerge pointed at a directory containing multiple files. It would merge all files into a single file on the local file system. Running the same command using the codebase from trunk, it looks like nothing happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.