You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Christopher Maier <ch...@gm.com> on 2016/02/17 19:20:51 UTC
Hadoop fs mv pushes folder under subfolder of same name
Hi,
I've run into an odd condition in the Hadoop move command. When moving a directory 'a' under a directory 'b' which already has a directory 'a' under it, it moves under that subdirectory rather than failing.
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
Why is that happening? I've tried looking through the classes on grepcode, but I can't find a condition where this behavior will occur.
I've also tried directly running the 'rename' command for org.apache.hadoop.hdfs.DistributedFileSystem from Java, but it behaves as expected.
Every other condition I've been able to come up with (eg, 'b' contains directory 'c' which contains 'a') behaves as expected.
> hadoop version
Hadoop 2.2.0
Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
Compiled by jenkins on 2015-01-15T00:39Z
Compiled with protoc 2.5.0
>From source with checksum ad91691b86dceaaa5875773852f328d
This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> hdfs -mkdir abc
> hdfs -ls -R abc
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> hdfs -mv abc/a abc/b
mv: `abc/a': Input/output error
> hdfs -mv abc/b/a abc/b/c
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/b/a
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
Thanks,
Kit
Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.
Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
Re: Hadoop fs mv pushes folder under subfolder of same name
Posted by Chandeep Singh <cs...@chandeep.com>.
This is what I get on CDH 5.4.8:
[csingh]$ hadoop fs -ls -R abc
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:25 abc/a
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b/a
[csingh]$ hadoop fs -mv abc/a abc/b
mv: `abc/b/a': File exists
> On Feb 17, 2016, at 6:20 PM, Christopher Maier <ch...@gm.com> wrote:
>
> Hi,
>
> I’ve run into an odd condition in the Hadoop move command. When moving a directory ‘a’ under a directory ‘b’ which already has a directory ‘a’ under it, it moves under that subdirectory rather than failing.
>
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
>
> Why is that happening? I’ve tried looking through the classes on grepcode, but I can’t find a condition where this behavior will occur.
>
> I’ve also tried directly running the ‘rename’ command for org.apache.hadoop.hdfs.DistributedFileSystem from Java, but it behaves as expected.
>
> Every other condition I’ve been able to come up with (eg, ‘b’ contains directory ‘c’ which contains ‘a’) behaves as expected.
>
> > hadoop version
> Hadoop 2.2.0
> Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
> Compiled by jenkins on 2015-01-15T00:39Z
> Compiled with protoc 2.5.0
> From source with checksum ad91691b86dceaaa5875773852f328d
> This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> > hdfs -mkdir abc
> > hdfs -ls -R abc
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mv abc/a abc/b
> mv: `abc/a': Input/output error
> > hdfs -mv abc/b/a abc/b/c
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
>
> Thanks,
> Kit
>
>
>
> Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
Re: Hadoop fs mv pushes folder under subfolder of same name
Posted by Chandeep Singh <cs...@chandeep.com>.
This is what I get on CDH 5.4.8:
[csingh]$ hadoop fs -ls -R abc
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:25 abc/a
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b/a
[csingh]$ hadoop fs -mv abc/a abc/b
mv: `abc/b/a': File exists
> On Feb 17, 2016, at 6:20 PM, Christopher Maier <ch...@gm.com> wrote:
>
> Hi,
>
> I’ve run into an odd condition in the Hadoop move command. When moving a directory ‘a’ under a directory ‘b’ which already has a directory ‘a’ under it, it moves under that subdirectory rather than failing.
>
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
>
> Why is that happening? I’ve tried looking through the classes on grepcode, but I can’t find a condition where this behavior will occur.
>
> I’ve also tried directly running the ‘rename’ command for org.apache.hadoop.hdfs.DistributedFileSystem from Java, but it behaves as expected.
>
> Every other condition I’ve been able to come up with (eg, ‘b’ contains directory ‘c’ which contains ‘a’) behaves as expected.
>
> > hadoop version
> Hadoop 2.2.0
> Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
> Compiled by jenkins on 2015-01-15T00:39Z
> Compiled with protoc 2.5.0
> From source with checksum ad91691b86dceaaa5875773852f328d
> This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> > hdfs -mkdir abc
> > hdfs -ls -R abc
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mv abc/a abc/b
> mv: `abc/a': Input/output error
> > hdfs -mv abc/b/a abc/b/c
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
>
> Thanks,
> Kit
>
>
>
> Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
Re: Hadoop fs mv pushes folder under subfolder of same name
Posted by Chandeep Singh <cs...@chandeep.com>.
This is what I get on CDH 5.4.8:
[csingh]$ hadoop fs -ls -R abc
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:25 abc/a
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b/a
[csingh]$ hadoop fs -mv abc/a abc/b
mv: `abc/b/a': File exists
> On Feb 17, 2016, at 6:20 PM, Christopher Maier <ch...@gm.com> wrote:
>
> Hi,
>
> I’ve run into an odd condition in the Hadoop move command. When moving a directory ‘a’ under a directory ‘b’ which already has a directory ‘a’ under it, it moves under that subdirectory rather than failing.
>
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
>
> Why is that happening? I’ve tried looking through the classes on grepcode, but I can’t find a condition where this behavior will occur.
>
> I’ve also tried directly running the ‘rename’ command for org.apache.hadoop.hdfs.DistributedFileSystem from Java, but it behaves as expected.
>
> Every other condition I’ve been able to come up with (eg, ‘b’ contains directory ‘c’ which contains ‘a’) behaves as expected.
>
> > hadoop version
> Hadoop 2.2.0
> Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
> Compiled by jenkins on 2015-01-15T00:39Z
> Compiled with protoc 2.5.0
> From source with checksum ad91691b86dceaaa5875773852f328d
> This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> > hdfs -mkdir abc
> > hdfs -ls -R abc
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mv abc/a abc/b
> mv: `abc/a': Input/output error
> > hdfs -mv abc/b/a abc/b/c
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
>
> Thanks,
> Kit
>
>
>
> Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
Re: Hadoop fs mv pushes folder under subfolder of same name
Posted by Chandeep Singh <cs...@chandeep.com>.
This is what I get on CDH 5.4.8:
[csingh]$ hadoop fs -ls -R abc
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:25 abc/a
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b
drwxr-xr-x - hdfs supergroup 0 2016-02-17 18:26 abc/b/a
[csingh]$ hadoop fs -mv abc/a abc/b
mv: `abc/b/a': File exists
> On Feb 17, 2016, at 6:20 PM, Christopher Maier <ch...@gm.com> wrote:
>
> Hi,
>
> I’ve run into an odd condition in the Hadoop move command. When moving a directory ‘a’ under a directory ‘b’ which already has a directory ‘a’ under it, it moves under that subdirectory rather than failing.
>
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
>
> Why is that happening? I’ve tried looking through the classes on grepcode, but I can’t find a condition where this behavior will occur.
>
> I’ve also tried directly running the ‘rename’ command for org.apache.hadoop.hdfs.DistributedFileSystem from Java, but it behaves as expected.
>
> Every other condition I’ve been able to come up with (eg, ‘b’ contains directory ‘c’ which contains ‘a’) behaves as expected.
>
> > hadoop version
> Hadoop 2.2.0
> Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
> Compiled by jenkins on 2015-01-15T00:39Z
> Compiled with protoc 2.5.0
> From source with checksum ad91691b86dceaaa5875773852f328d
> This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> > hdfs -mkdir abc
> > hdfs -ls -R abc
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:28 abc/b
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mv abc/a abc/b
> mv: `abc/a': Input/output error
> > hdfs -mv abc/b/a abc/b/c
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:44 abc/b
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:30 abc/b/a
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c
> drwxr-x--x - dedwmktmedia supergroup 0 2016-02-17 12:29 abc/b/c/a
>
> Thanks,
> Kit
>
>
>
> Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.