You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jay Hacker (JIRA)" <ji...@apache.org> on 2011/02/09 15:21:04 UTC
[jira] Created: (HDFS-1616) hadoop fs -put and -copyFromLocal do
not support globs in the source path
hadoop fs -put and -copyFromLocal do not support globs in the source path
-------------------------------------------------------------------------
Key: HDFS-1616
URL: https://issues.apache.org/jira/browse/HDFS-1616
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.2
Environment: Cloudera CDH3b3
Reporter: Jay Hacker
Priority: Minor
I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't work:
{noformat}
$ ls
file1 file2
$ hadoop fs -put '*' .
put: File * does not exist.
{noformat}
This has probably gone unnoticed because your shell usually handles it, but a) I'd like to be able to call 'hadoop fs' programatically without a shell, and b) Hadoop globbing differs from shell globbing and it would be nice to be able to use it consistently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HDFS-1616) hadoop fs -put and -copyFromLocal do
not support globs in the source path
Posted by "Jay Hacker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jay Hacker resolved HDFS-1616.
------------------------------
Resolution: Duplicate
"Moved" to Hadoop Common as HADOOP-7141.
> hadoop fs -put and -copyFromLocal do not support globs in the source path
> -------------------------------------------------------------------------
>
> Key: HDFS-1616
> URL: https://issues.apache.org/jira/browse/HDFS-1616
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.2
> Environment: Cloudera CDH3b3
> Reporter: Jay Hacker
> Priority: Minor
>
> I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't work:
> {noformat}
> $ ls
> file1 file2
> $ hadoop fs -put '*' .
> put: File * does not exist.
> {noformat}
> This has probably gone unnoticed because your shell usually handles it, but a) I'd like to be able to call 'hadoop fs' programatically without a shell, b) it doesn't work in Pig or Grunt, where there is no shell helping you, and c) Hadoop globbing differs from shell globbing and it would be nice to be able to use it consistently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1616) hadoop fs -put and -copyFromLocal do
not support globs in the source path
Posted by "Jay Hacker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jay Hacker updated HDFS-1616:
-----------------------------
Description:
I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't work:
{noformat}
$ ls
file1 file2
$ hadoop fs -put '*' .
put: File * does not exist.
{noformat}
This has probably gone unnoticed because your shell usually handles it, but a) I'd like to be able to call 'hadoop fs' programatically without a shell, b) it doesn't work in Pig or Grunt, where there is no shell helping you, and c) Hadoop globbing differs from shell globbing and it would be nice to be able to use it consistently.
was:
I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't work:
{noformat}
$ ls
file1 file2
$ hadoop fs -put '*' .
put: File * does not exist.
{noformat}
This has probably gone unnoticed because your shell usually handles it, but a) I'd like to be able to call 'hadoop fs' programatically without a shell, and b) Hadoop globbing differs from shell globbing and it would be nice to be able to use it consistently.
> hadoop fs -put and -copyFromLocal do not support globs in the source path
> -------------------------------------------------------------------------
>
> Key: HDFS-1616
> URL: https://issues.apache.org/jira/browse/HDFS-1616
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.2
> Environment: Cloudera CDH3b3
> Reporter: Jay Hacker
> Priority: Minor
>
> I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't work:
> {noformat}
> $ ls
> file1 file2
> $ hadoop fs -put '*' .
> put: File * does not exist.
> {noformat}
> This has probably gone unnoticed because your shell usually handles it, but a) I'd like to be able to call 'hadoop fs' programatically without a shell, b) it doesn't work in Pig or Grunt, where there is no shell helping you, and c) Hadoop globbing differs from shell globbing and it would be nice to be able to use it consistently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1616) hadoop fs -put and -copyFromLocal do
not support globs in the source path
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992549#comment-12992549 ]
Harsh J Chouraria commented on HDFS-1616:
-----------------------------------------
This issue is a HADOOP (Common) one, not HDFS.
You can achieve a solution to globbing by Hadoop DFS by using {{FsShell.copy}} via {{hdfs -cp}} with a command like:
{{hdfs -cp 'file:/home/test/*.jar' /destination}}
This command does a globbing on its own (not shell-driven). Should also work inside Grunt/etc.{{.}}
To programmatically call {{fs}} sub-functions, it would be wise use the Hadoop's {{FileSystem}} API directly in your language instead. There are Python bindings for HDFS available, to start with, apart from the Java one provided.
Regarding glob differences in Shell/Hadoop, there isn't really a standard available to conform to (please correct me if I'm wrong). A good collection of cases is covered in {{test/o.a.h.fs.TestGlobPaths}}, which IMHO caters to most of globbing requirements (there's also a subset of regular-expression support available).
> hadoop fs -put and -copyFromLocal do not support globs in the source path
> -------------------------------------------------------------------------
>
> Key: HDFS-1616
> URL: https://issues.apache.org/jira/browse/HDFS-1616
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.2
> Environment: Cloudera CDH3b3
> Reporter: Jay Hacker
> Priority: Minor
>
> I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't work:
> {noformat}
> $ ls
> file1 file2
> $ hadoop fs -put '*' .
> put: File * does not exist.
> {noformat}
> This has probably gone unnoticed because your shell usually handles it, but a) I'd like to be able to call 'hadoop fs' programatically without a shell, b) it doesn't work in Pig or Grunt, where there is no shell helping you, and c) Hadoop globbing differs from shell globbing and it would be nice to be able to use it consistently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira