You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Chris Nauroth (JIRA)" <ji...@apache.org> on 2014/11/17 19:41:35 UTC
[jira] [Commented] (HADOOP-11201) Hadoop Archives should support
globs resolving to files
[ https://issues.apache.org/jira/browse/HADOOP-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214968#comment-14214968 ]
Chris Nauroth commented on HADOOP-11201:
----------------------------------------
Hi, [~jira.shegalov]. This is a good find. Thank you for posting a patch.
>From reading this code, it's unclear to me why it ever bothered splitting directory paths into the separate {{justDirs}} list. The subsequent logic operates on the parents of the paths. A parent is by definition always going to be a directory. I think we can remove {{justDirs}} entirely and change the inner loop to iterate directly over {{paths}} instead. Do you agree?
If any of the other watchers can think of a reason for the {{justDirs}} logic, please comment. I couldn't find anything relevant in revision history.
> Hadoop Archives should support globs resolving to files
> -------------------------------------------------------
>
> Key: HADOOP-11201
> URL: https://issues.apache.org/jira/browse/HADOOP-11201
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools
> Affects Versions: 2.0.5-alpha
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Priority: Blocker
> Attachments: HADOOP-11201.v01.patch
>
>
> Consider the following scenario:
> {code}
> $ hadoop fs -ls /tmp/harsrc/dir2/dir3
> Found 5 items
> -rw-r--r-- 1 blah blah 0 2014-10-13 20:59 /tmp/harsrc/dir2/dir3/file31
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file32
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file33
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file34
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file35
> {code}
> Archive 'dir3/file3*':
> {code}
> $ hadoop archive -Dmapreduce.framework.name=local -archiveName fileStar.har -p /tmp/harsrc 'dir2/dir3/file*' /tmp/hardst_local
> $ hadoop fs -ls -R har:/tmp/hardst_local/fileStar.har
> drwxr-xr-x - blah blah 0 2014-10-13 22:32 har:///tmp/hardst_local/fileStar.har/dir2
> {code}
> Archiving dir3 (directory) which is equivalent to the above works.
> {code}
> $ hadoop archive -Dmapreduce.framework.name=local -archiveName dir3.har -p /tmp/harsrc 'dir2/dir3' /tmp/hardst_local
> $ hadoop fs -ls -R har:/tmp/hardst_local/dir3.har
> 14/10/14 02:06:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> drwxr-xr-x - blah blah 0 2014-10-13 22:32 har:///tmp/hardst_local/dir3.har/dir2
> drwxr-xr-x - blah blah 0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3
> -rw-r--r-- 1 blah blah 0 2014-10-13 20:59 har:///tmp/hardst_local/dir3.har/dir2/dir3/file31
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file32
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file33
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file34
> -rw-r--r-- 1 blah blah 0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file35
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)