Problem copying to local from DFS in Hadoop 0.12.1

[Rohan Mehta <ro...@visvo.com>]

All,

I am having problems with -copyToLocal in Hadoop version 0.12.1.
I have injected some data into the Distributed file system of Hadoop 0.12.1
When I try to do -copyToLocal on a file it works fine but when I try to 
do -copyToLocal on a directory I get the following error:

*[user@bob01 nutch]$ bin/hadoop fs -copyToLocal crawldb rohan/
copyToLocal: Target rohan/data already exists*

Same thing happens when i use -get or -cp option

System config:
- using cluster of 5 computers
- Nutch trunk version 0.9
- Hadoop truck version 0.12.1

how can I make this work ? any pointers are appreciated.

Thanks,
Rohan

-- 
This message has been scanned for viruses and
dangerous content and is believed to be clean.

Re: Problem copying to local from DFS in Hadoop 0.12.1

[Espen Amble Kolstad <es...@trank.no>]

Hi,

Here's a patch that I use to fix this:
Index: src/java/org/apache/hadoop/fs/ChecksumFileSystem.java
===================================================================
--- src/java/org/apache/hadoop/fs/ChecksumFileSystem.java
(revision 517190)
+++ src/java/org/apache/hadoop/fs/ChecksumFileSystem.java       (working
copy)
@@ -599,7 +599,7 @@
     } else {
       Path[] srcs = listPaths(src);
       for (Path srcFile : srcs) {
-        copyToLocalFile(srcFile, dst, copyCrc);
+        copyToLocalFile(srcFile, new Path(dst, srcFile.getName()),
copyCrc);
       }
     }
   }

- Espen

Rohan Mehta wrote:
> All,
> 
> I am having problems with -copyToLocal in Hadoop version 0.12.1.
> I have injected some data into the Distributed file system of Hadoop 0.12.1
> When I try to do -copyToLocal on a file it works fine but when I try to
> do -copyToLocal on a directory I get the following error:
> 
> *[user@bob01 nutch]$ bin/hadoop fs -copyToLocal crawldb rohan/
> copyToLocal: Target rohan/data already exists*
> 
> Same thing happens when i use -get or -cp option
> 
> System config:
> - using cluster of 5 computers
> - Nutch trunk version 0.9
> - Hadoop truck version 0.12.1
> 
> how can I make this work ? any pointers are appreciated.
> 
> Thanks,
> Rohan
> 

Re: Problem copying to local from DFS in Hadoop 0.12.1

[Nigel Daley <nd...@yahoo-inc.com>]

I filed https://issues.apache.org/jira/browse/HADOOP-1115

Thanks Rohan!

Nige

On Mar 13, 2007, at 8:56 AM, Rohan Mehta wrote:

> All,
>
> I am having problems with -copyToLocal in Hadoop version 0.12.1.
> I have injected some data into the Distributed file system of  
> Hadoop 0.12.1
> When I try to do -copyToLocal on a file it works fine but when I  
> try to do -copyToLocal on a directory I get the following error:
>
> *[user@bob01 nutch]$ bin/hadoop fs -copyToLocal crawldb rohan/
> copyToLocal: Target rohan/data already exists*
>
> Same thing happens when i use -get or -cp option
>
> System config:
> - using cluster of 5 computers
> - Nutch trunk version 0.9
> - Hadoop truck version 0.12.1
>
> how can I make this work ? any pointers are appreciated.
>
> Thanks,
> Rohan
>
> -- 
> This message has been scanned for viruses and
> dangerous content and is believed to be clean.
>