copyToLocal

[Joseph Stein <cr...@gmail.com>]

Hi, so ok  am using copyToLocal through an automation script we have and
seeing odd results.

I am not sure if this is something I am doing wrong, defect, or known good
reason for it.  Let me know I would like to correct this in either my own
script, happy to give a try in the fs code fixing a bug or my own brain in
understanding because there is some good reason for this.

scenario

hadoop fs -copyToLocal event/2010_06_10/81ae7c24745211df9f6d002590008422
/data/2010_06_10/81ae7c24745211df9f6d002590008422

is resulting in my part files showing up in *
/data/2010_06_10/81ae7c24745211df9f6d002590008422/81ae7c24745211df9f6d002590008422
*
if i try [Hadoop 0.20.1]

hadoop fs -copyToLocal event/2010_06_10/81ae7c24745211df9f6d002590008422
/data/2010_06_10

is resulting in my part files showing up in just */data/2010_06_10*
(with no creation of the UUID directory like it did before)

my desired result is to have the files

from *event/2010_06_10/81ae7c24745211df9f6d002590008422

*and end up in */data/2010_06_10/81ae7c24745211df9f6d002590008422*

with the trailing directory creating itself like it does in the first
scenario but not duplicating it as it does....weirdly.

/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop
*/

Re: copyToLocal

[Joseph Stein <cr...@gmail.com>]

I think 3 weeks of no sleep caused this.  My automation script failed
leaving the directory there so when it re-ran THEN it caused this weirdness.

I guess copyToLocal if it sees a directory already existing it then appends
the directory as a child to the local (so in my first scenario
/data/2010_06_10/81ae7c24745211df9f6d002590008422 already was existing
because my script created it the first time).

Still odd behavior, whatever it is fine sorry to bother... i just added to
my automation script to remove the directory before i do a copyToLocal.


On Thu, Jun 10, 2010 at 7:12 PM, Joseph Stein <cr...@gmail.com> wrote:

> Hi, so ok  am using copyToLocal through an automation script we have and
> seeing odd results.
>
> I am not sure if this is something I am doing wrong, defect, or known good
> reason for it.  Let me know I would like to correct this in either my own
> script, happy to give a try in the fs code fixing a bug or my own brain in
> understanding because there is some good reason for this.
>
> scenario
>
> hadoop fs -copyToLocal event/2010_06_10/81ae7c24745211df9f6d002590008422
> /data/2010_06_10/81ae7c24745211df9f6d002590008422
>
> is resulting in my part files showing up in *
> /data/2010_06_10/81ae7c24745211df9f6d002590008422/81ae7c24745211df9f6d002590008422
> *
> if i try [Hadoop 0.20.1]
>
> hadoop fs -copyToLocal event/2010_06_10/81ae7c24745211df9f6d002590008422
> /data/2010_06_10
>
> is resulting in my part files showing up in just */data/2010_06_10*
> (with no creation of the UUID directory like it did before)
>
> my desired result is to have the files
>
> from *event/2010_06_10/81ae7c24745211df9f6d002590008422
>
> *and end up in */data/2010_06_10/81ae7c24745211df9f6d002590008422*
>
> with the trailing directory creating itself like it does in the first
> scenario but not duplicating it as it does....weirdly.
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop
> */
>
>


-- 
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
*/