You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by John Meza <j_...@hotmail.com> on 2013/01/16 17:54:09 UTC
does "fs -put " create subdirectories?
Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the necassary subdirectories in hdfs?thanksJohn
RE: does "fs -put " create subdirectories?
Posted by John Meza <j_...@hotmail.com>.
Modifying the script to precede the put with a mkdir (which creates the missing subdirectories) fixes the issue.thanksJohn
Date: Wed, 16 Jan 2013 12:36:20 -0500
Subject: Re: does "fs -put " create subdirectories?
From: cembree@gmail.com
To: user@hadoop.apache.org
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It allows me to see if I've made a mistake in my planned placement of files instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
On 1.x, -put does mkdir the parent directories if they are non existent (akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.tput: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
--
Harsh J
RE: does "fs -put " create subdirectories?
Posted by John Meza <j_...@hotmail.com>.
Modifying the script to precede the put with a mkdir (which creates the missing subdirectories) fixes the issue.thanksJohn
Date: Wed, 16 Jan 2013 12:36:20 -0500
Subject: Re: does "fs -put " create subdirectories?
From: cembree@gmail.com
To: user@hadoop.apache.org
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It allows me to see if I've made a mistake in my planned placement of files instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
On 1.x, -put does mkdir the parent directories if they are non existent (akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.tput: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
--
Harsh J
RE: does "fs -put " create subdirectories?
Posted by John Meza <j_...@hotmail.com>.
Modifying the script to precede the put with a mkdir (which creates the missing subdirectories) fixes the issue.thanksJohn
Date: Wed, 16 Jan 2013 12:36:20 -0500
Subject: Re: does "fs -put " create subdirectories?
From: cembree@gmail.com
To: user@hadoop.apache.org
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It allows me to see if I've made a mistake in my planned placement of files instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
On 1.x, -put does mkdir the parent directories if they are non existent (akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.tput: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
--
Harsh J
RE: does "fs -put " create subdirectories?
Posted by John Meza <j_...@hotmail.com>.
Modifying the script to precede the put with a mkdir (which creates the missing subdirectories) fixes the issue.thanksJohn
Date: Wed, 16 Jan 2013 12:36:20 -0500
Subject: Re: does "fs -put " create subdirectories?
From: cembree@gmail.com
To: user@hadoop.apache.org
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It allows me to see if I've made a mistake in my planned placement of files instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
On 1.x, -put does mkdir the parent directories if they are non existent (akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.tput: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
--
Harsh J
Re: does "fs -put " create subdirectories?
Posted by Chris Embree <ce...@gmail.com>.
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It
allows me to see if I've made a mistake in my planned placement of files
instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
> On 1.x, -put does mkdir the parent directories if they are non existent
> (akin to mkdir -p).
>
> On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
> before -put.
>
> ➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
> put: `a/bc/dfe/fsdf.t': No such file or directory
>
>
> On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> Yes it does, you can just try ;)
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 1 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 2 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
>> /user/test2/subdir/anotherone/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>>
>> 2013/1/16, John Meza <j_...@hotmail.com>:
>> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
>> > necassary subdirectories in hdfs?thanksJohn
>>
>
>
>
> --
> Harsh J
>
Re: does "fs -put " create subdirectories?
Posted by Chris Embree <ce...@gmail.com>.
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It
allows me to see if I've made a mistake in my planned placement of files
instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
> On 1.x, -put does mkdir the parent directories if they are non existent
> (akin to mkdir -p).
>
> On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
> before -put.
>
> ➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
> put: `a/bc/dfe/fsdf.t': No such file or directory
>
>
> On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> Yes it does, you can just try ;)
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 1 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 2 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
>> /user/test2/subdir/anotherone/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>>
>> 2013/1/16, John Meza <j_...@hotmail.com>:
>> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
>> > necassary subdirectories in hdfs?thanksJohn
>>
>
>
>
> --
> Harsh J
>
Re: does "fs -put " create subdirectories?
Posted by Chris Embree <ce...@gmail.com>.
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It
allows me to see if I've made a mistake in my planned placement of files
instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
> On 1.x, -put does mkdir the parent directories if they are non existent
> (akin to mkdir -p).
>
> On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
> before -put.
>
> ➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
> put: `a/bc/dfe/fsdf.t': No such file or directory
>
>
> On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> Yes it does, you can just try ;)
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 1 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 2 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
>> /user/test2/subdir/anotherone/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>>
>> 2013/1/16, John Meza <j_...@hotmail.com>:
>> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
>> > necassary subdirectories in hdfs?thanksJohn
>>
>
>
>
> --
> Harsh J
>
Re: does "fs -put " create subdirectories?
Posted by Chris Embree <ce...@gmail.com>.
Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It
allows me to see if I've made a mistake in my planned placement of files
instead of blindly writing.
On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <ha...@cloudera.com> wrote:
> On 1.x, -put does mkdir the parent directories if they are non existent
> (akin to mkdir -p).
>
> On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
> before -put.
>
> ➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
> put: `a/bc/dfe/fsdf.t': No such file or directory
>
>
> On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> Yes it does, you can just try ;)
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 1 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
>> Found 2 items
>> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
>> /user/test/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
>> /user/test2/subdir/anotherone/
>> Found 1 items
>> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
>> /user/test2/subdir/anotherone/CHANGES.txt
>>
>>
>> 2013/1/16, John Meza <j_...@hotmail.com>:
>> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
>> > necassary subdirectories in hdfs?thanksJohn
>>
>
>
>
> --
> Harsh J
>
Re: does "fs -put " create subdirectories?
Posted by Harsh J <ha...@cloudera.com>.
On 1.x, -put does mkdir the parent directories if they are non existent
(akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
put: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Yes it does, you can just try ;)
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 1 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 2 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test2/subdir/anotherone/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
> /user/test2/subdir/anotherone/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
> /user/test2/subdir/anotherone/CHANGES.txt
>
>
> 2013/1/16, John Meza <j_...@hotmail.com>:
> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> > necassary subdirectories in hdfs?thanksJohn
>
--
Harsh J
Re: does "fs -put " create subdirectories?
Posted by Harsh J <ha...@cloudera.com>.
On 1.x, -put does mkdir the parent directories if they are non existent
(akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
put: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Yes it does, you can just try ;)
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 1 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 2 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test2/subdir/anotherone/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
> /user/test2/subdir/anotherone/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
> /user/test2/subdir/anotherone/CHANGES.txt
>
>
> 2013/1/16, John Meza <j_...@hotmail.com>:
> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> > necassary subdirectories in hdfs?thanksJohn
>
--
Harsh J
Re: does "fs -put " create subdirectories?
Posted by Harsh J <ha...@cloudera.com>.
On 1.x, -put does mkdir the parent directories if they are non existent
(akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
put: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Yes it does, you can just try ;)
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 1 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 2 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test2/subdir/anotherone/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
> /user/test2/subdir/anotherone/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
> /user/test2/subdir/anotherone/CHANGES.txt
>
>
> 2013/1/16, John Meza <j_...@hotmail.com>:
> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> > necassary subdirectories in hdfs?thanksJohn
>
--
Harsh J
Re: does "fs -put " create subdirectories?
Posted by Harsh J <ha...@cloudera.com>.
On 1.x, -put does mkdir the parent directories if they are non existent
(akin to mkdir -p).
On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd
before -put.
➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.t
put: `a/bc/dfe/fsdf.t': No such file or directory
On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Yes it does, you can just try ;)
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 1 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
> Found 2 items
> drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
> drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
> /user/test/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
> /user/test2/subdir/anotherone/CHANGES.txt
>
> hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls
> /user/test2/subdir/anotherone/
> Found 1 items
> -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
> /user/test2/subdir/anotherone/CHANGES.txt
>
>
> 2013/1/16, John Meza <j_...@hotmail.com>:
> > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> > necassary subdirectories in hdfs?thanksJohn
>
--
Harsh J
Re: does "fs -put " create subdirectories?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
Re: does "fs -put " create subdirectories?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
Re: does "fs -put " create subdirectories?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn
Re: does "fs -put " create subdirectories?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Yes it does, you can just try ;)
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/
Found 2 items
drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase
drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56
/user/test/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt
/user/test2/subdir/anotherone/CHANGES.txt
hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/
Found 1 items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza <j_...@hotmail.com>:
> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the
> necassary subdirectories in hdfs?thanksJohn