You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2013/02/26 11:32:12 UTC
[jira] [Commented] (MAPREDUCE-5029) Recursively take all files in
the directories of a root directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586994#comment-13586994 ]
Steve Loughran commented on MAPREDUCE-5029:
-------------------------------------------
Please can you see if this problem still exists on Hadoop trunk. There have been changes to do iterative enumeration of directory contents rather than the ls * -see {{FileSystem.listFiles()}}. If this method isn't being used in MR jobs, maybe it could be.
> Recursively take all files in the directories of a root directory
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-5029
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5029
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.20.2
> Reporter: Abhilash S R
>
> Suppose we have a root directories with 1000's of sub directories and in each directory there can be 100's of files.So while specifying the root directory in the input path in map-reduce the program crashes due to sub directories in the root directory.So if this feature is includes in latest version it will be great helpful for programers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira