You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Michael Di Domenico <md...@gmail.com> on 2008/09/08 05:58:14 UTC
distcp failing
I'm attempting to load data into hadoop (version 0.17.1), from a
non-datanode machine in the cluster. I can run jobs and copyFromLocal works
fine, but when i try to use distcp i get the below. I'm don't understand
what the error, can anyone help?
Thanks
blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
file:///Users/mdidomenico/hadoop/1gTestfile /user/mdidomenico/1gTestfile
08/09/07 23:56:06 INFO util.CopyFiles:
srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
08/09/07 23:56:06 INFO util.CopyFiles:
destPath=/user/mdidomenico/1gTestfile1
08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
With failures, global counters are inaccurate; consider running with -i
Copy failed: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
/tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No such file
or directory
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
at
org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
at
org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
at
org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
at
org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
at org.apache.hadoop.ipc.Client.call(Client.java:557)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
at $Proxy1.submitJob(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy1.submitJob(Unknown Source)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
Re: distcp failing
Posted by Michael Di Domenico <md...@gmail.com>.
Looking in the task tracker log, i see this
This file does exist on my local workstation, but it does not exist on the
namenode/datanodes in my cluster. So it begs the question of if i
misunderstood the use of distcp or is there still something wrong?
I'm looking for something that will read a file from my workstation and load
it into the dfs, but instead of going through the namenode like
copyFromLocal seems to do, i'd like it to load the data via the datanodes
directly, if distcp doesn't do it this way, is there anything that will?
2008-09-09 14:00:54,418 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2008-09-09 14:00:54,662 INFO org.apache.hadoop.mapred.MapTask:
numReduceTasks: 0
2008-09-09 14:00:54,894 INFO org.apache.hadoop.util.CopyFiles: FAIL
1gTestfile : java.io.FileNotFoundException: File
file:/Users/mdidomenico/hadoop/1gTestfile does not exist.
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:242)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:116)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:274)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:380)
at org.apache.hadoop.util.CopyFiles$CopyFilesMapper.copy(CopyFiles.java:366)
at org.apache.hadoop.util.CopyFiles$CopyFilesMapper.map(CopyFiles.java:493)
at org.apache.hadoop.util.CopyFiles$CopyFilesMapper.map(CopyFiles.java:268)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
2008-09-09 14:01:03,950 WARN org.apache.hadoop.mapred.TaskTracker: Error
running child
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at
org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
On Tue, Sep 9, 2008 at 1:47 PM, Michael Di Domenico
<md...@gmail.com>wrote:
> Apparently, the fix to my original error is because hadoop is setup for a
> single local machine out of the box and i had to change these directories
> <property>
> <name>mapred.local.dir</name>
> <value>/hadoop/mapred/local</value>
> </property>
> <property>
> <name>mapred.system.dir</name>
> <value>/hadoop/mapred/system</value>
> </property>
> <property>
> <name>mapred.temp.dir</name>
> <value>/hadoop/mapred/temp</value>
> </property>
>
> to be hdfs instead of "hadoop.tmp.dir"
>
> So now distcp works as a non-hadoop user and mapred works as a non-hadoop
> user from the name node, however, from a workstation i get this now
>
> blue:hadoop-0.17.1 mdidomenico$ bin/hadoop distcp
> "file:///Users/mdidomenico/hadoop/1gTestfile" "1gTestfile-1"
> 08/09/09 13:44:19 INFO util.CopyFiles:
> srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
> 08/09/09 13:44:19 INFO util.CopyFiles: destPath=1gTestfile-1
> 08/09/09 13:44:20 INFO util.CopyFiles: srcCount=1
> 08/09/09 13:44:22 INFO mapred.JobClient: Running job: job_200809091332_0004
> 08/09/09 13:44:23 INFO mapred.JobClient: map 0% reduce 0%
> 08/09/09 13:44:31 INFO mapred.JobClient: Task Id :
> task_200809091332_0004_m_000000_0, Status : FAILED
> java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
> at
> org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>
> 08/09/09 13:44:50 INFO mapred.JobClient: Task Id :
> task_200809091332_0004_m_000000_1, Status : FAILED
> java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
> at
> org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>
> 08/09/09 13:45:07 INFO mapred.JobClient: Task Id :
> task_200809091332_0004_m_000000_2, Status : FAILED
> java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
> at
> org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>
> 08/09/09 13:45:26 INFO mapred.JobClient: map 100% reduce 100%
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1062)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
>
>
> On Tue, Sep 9, 2008 at 1:14 PM, Michael Di Domenico <
> mdidomenico4@gmail.com> wrote:
>
>> manually creating the "system" directory gets me past the first error, but
>> now i get this. i'm not necessarily sure its a step forward though, because
>> the map task never shows up in the jobtracker
>> [mdidomenico@orange hadoop-0.17.1]$ bin/hadoop distcp
>> "file:///home/mdidomenico/1gTestfile" "1gTestfile"
>> 08/09/09 13:12:06 INFO util.CopyFiles:
>> srcPaths=[file:/home/mdidomenico/1gTestfile]
>> 08/09/09 13:12:06 INFO util.CopyFiles: destPath=1gTestfile
>> 08/09/09 13:12:07 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:07 INFO dfs.DFSClient: Abandoning block
>> blk_5758513071638050362
>> 08/09/09 13:12:13 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:13 INFO dfs.DFSClient: Abandoning block
>> blk_1691495306775808049
>> 08/09/09 13:12:17 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:17 INFO dfs.DFSClient: Abandoning block
>> blk_1027634596973755899
>> 08/09/09 13:12:19 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:19 INFO dfs.DFSClient: Abandoning block
>> blk_4535302510016050282
>> 08/09/09 13:12:23 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:23 INFO dfs.DFSClient: Abandoning block
>> blk_7022658012001626339
>> 08/09/09 13:12:25 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:25 INFO dfs.DFSClient: Abandoning block
>> blk_-4509681241839967328
>> 08/09/09 13:12:29 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:29 INFO dfs.DFSClient: Abandoning block
>> blk_8318033979013580420
>> 08/09/09 13:12:31 WARN dfs.DFSClient: DataStreamer Exception:
>> java.io.IOException: Unable to create new block.
>> 08/09/09 13:12:31 WARN dfs.DFSClient: Error Recovery for block
>> blk_-4509681241839967328 bad datanode[0]
>> 08/09/09 13:12:35 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:35 INFO dfs.DFSClient: Abandoning block
>> blk_2848354798649979411
>> 08/09/09 13:12:41 WARN dfs.DFSClient: DataStreamer Exception:
>> java.io.IOException: Unable to create new block.
>> 08/09/09 13:12:41 WARN dfs.DFSClient: Error Recovery for block
>> blk_2848354798649979411 bad datanode[0]
>> Exception in thread "Thread-0" java.util.ConcurrentModificationException
>> at java.util.TreeMap$PrivateEntryIterator.nextEntry(Unknown
>> Source)
>> at java.util.TreeMap$KeyIterator.next(Unknown Source)
>> at org.apache.hadoop.dfs.DFSClient.close(DFSClient.java:217)
>> at
>> org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:214)
>> at
>> org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1324)
>> at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:224)
>> at
>> org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:209)
>> 08/09/09 13:12:41 INFO dfs.DFSClient: Exception in createBlockOutputStream
>> java.io.IOException: Could not read from stream
>> 08/09/09 13:12:41 INFO dfs.DFSClient: Abandoning block
>> blk_9189111926428577428
>>
>> On Tue, Sep 9, 2008 at 1:03 PM, Michael Di Domenico <
>> mdidomenico4@gmail.com> wrote:
>>
>>> a little more digging and it appears i cannot run distcp as someone other
>>> then hadoop on the namenode
>>> /tmp/hadoop-hadoop/mapred/system/job_200809091231_0005/job.xml
>>>
>>> looking at this directory from the error file the "system" directory does
>>> not exist on the namenode, i only have a "local" directory
>>>
>>>
>>> On Tue, Sep 9, 2008 at 12:41 PM, Michael Di Domenico <
>>> mdidomenico4@gmail.com> wrote:
>>>
>>>> i'm not sure that's the issue, i basically tarred up the hadoop
>>>> directory from the cluster and copied it over to the non-data node
>>>> but i do agree i've likely got a setting wrong, since i can run distcp
>>>> from the namenode and it works fine. the question is which one
>>>>
>>>> On Mon, Sep 8, 2008 at 7:04 PM, Aaron Kimball <aa...@spinnakerlabs.com>wrote:
>>>>
>>>>> It is likely that you mapred.system.dir and/or fs.default.namesettings are
>>>>> incorrect on the non-datanode machine that you are launching the task
>>>>> from.
>>>>> These two settings (in your conf/hadoop-site.xml file) must match the
>>>>> settings on the cluster itself.
>>>>>
>>>>> - Aaron
>>>>>
>>>>> On Sun, Sep 7, 2008 at 8:58 PM, Michael Di Domenico
>>>>> <md...@gmail.com>wrote:
>>>>>
>>>>> > I'm attempting to load data into hadoop (version 0.17.1), from a
>>>>> > non-datanode machine in the cluster. I can run jobs and
>>>>> copyFromLocal
>>>>> > works
>>>>> > fine, but when i try to use distcp i get the below. I'm don't
>>>>> understand
>>>>> > what the error, can anyone help?
>>>>> > Thanks
>>>>> >
>>>>> > blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
>>>>> > file:///Users/mdidomenico/hadoop/1gTestfile
>>>>> /user/mdidomenico/1gTestfile
>>>>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>>>>> > srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
>>>>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>>>>> > destPath=/user/mdidomenico/1gTestfile1
>>>>> > 08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
>>>>> > With failures, global counters are inaccurate; consider running with
>>>>> -i
>>>>> > Copy failed: org.apache.hadoop.ipc.RemoteException:
>>>>> java.io.IOException:
>>>>> > /tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No
>>>>> such
>>>>> > file
>>>>> > or directory
>>>>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
>>>>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
>>>>> > at
>>>>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
>>>>> > at
>>>>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
>>>>> > at
>>>>> > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
>>>>> > at
>>>>> > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>>>>> Source)
>>>>> > at java.lang.reflect.Method.invoke(Unknown Source)
>>>>> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>>> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> >
>>>>> > at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>>> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>> > at $Proxy1.submitJob(Unknown Source)
>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> > at
>>>>> >
>>>>> >
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> > at
>>>>> >
>>>>> >
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> > at java.lang.reflect.Method.invoke(Method.java:585)
>>>>> > at
>>>>> >
>>>>> >
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>> > at
>>>>> >
>>>>> >
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>> > at $Proxy1.submitJob(Unknown Source)
>>>>> > at
>>>>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
>>>>> > at
>>>>> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
>>>>> > at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
>>>>> > at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>> > at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>
Re: distcp failing
Posted by Michael Di Domenico <md...@gmail.com>.
Apparently, the fix to my original error is because hadoop is setup for a
single local machine out of the box and i had to change these directories
<property>
<name>mapred.local.dir</name>
<value>/hadoop/mapred/local</value>
</property>
<property>
<name>mapred.system.dir</name>
<value>/hadoop/mapred/system</value>
</property>
<property>
<name>mapred.temp.dir</name>
<value>/hadoop/mapred/temp</value>
</property>
to be hdfs instead of "hadoop.tmp.dir"
So now distcp works as a non-hadoop user and mapred works as a non-hadoop
user from the name node, however, from a workstation i get this now
blue:hadoop-0.17.1 mdidomenico$ bin/hadoop distcp
"file:///Users/mdidomenico/hadoop/1gTestfile" "1gTestfile-1"
08/09/09 13:44:19 INFO util.CopyFiles:
srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
08/09/09 13:44:19 INFO util.CopyFiles: destPath=1gTestfile-1
08/09/09 13:44:20 INFO util.CopyFiles: srcCount=1
08/09/09 13:44:22 INFO mapred.JobClient: Running job: job_200809091332_0004
08/09/09 13:44:23 INFO mapred.JobClient: map 0% reduce 0%
08/09/09 13:44:31 INFO mapred.JobClient: Task Id :
task_200809091332_0004_m_000000_0, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at
org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
08/09/09 13:44:50 INFO mapred.JobClient: Task Id :
task_200809091332_0004_m_000000_1, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at
org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
08/09/09 13:45:07 INFO mapred.JobClient: Task Id :
task_200809091332_0004_m_000000_2, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at
org.apache.hadoop.util.CopyFiles$CopyFilesMapper.close(CopyFiles.java:527)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
08/09/09 13:45:26 INFO mapred.JobClient: map 100% reduce 100%
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1062)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
On Tue, Sep 9, 2008 at 1:14 PM, Michael Di Domenico
<md...@gmail.com>wrote:
> manually creating the "system" directory gets me past the first error, but
> now i get this. i'm not necessarily sure its a step forward though, because
> the map task never shows up in the jobtracker
> [mdidomenico@orange hadoop-0.17.1]$ bin/hadoop distcp
> "file:///home/mdidomenico/1gTestfile" "1gTestfile"
> 08/09/09 13:12:06 INFO util.CopyFiles:
> srcPaths=[file:/home/mdidomenico/1gTestfile]
> 08/09/09 13:12:06 INFO util.CopyFiles: destPath=1gTestfile
> 08/09/09 13:12:07 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:07 INFO dfs.DFSClient: Abandoning block
> blk_5758513071638050362
> 08/09/09 13:12:13 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:13 INFO dfs.DFSClient: Abandoning block
> blk_1691495306775808049
> 08/09/09 13:12:17 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:17 INFO dfs.DFSClient: Abandoning block
> blk_1027634596973755899
> 08/09/09 13:12:19 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:19 INFO dfs.DFSClient: Abandoning block
> blk_4535302510016050282
> 08/09/09 13:12:23 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:23 INFO dfs.DFSClient: Abandoning block
> blk_7022658012001626339
> 08/09/09 13:12:25 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:25 INFO dfs.DFSClient: Abandoning block
> blk_-4509681241839967328
> 08/09/09 13:12:29 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:29 INFO dfs.DFSClient: Abandoning block
> blk_8318033979013580420
> 08/09/09 13:12:31 WARN dfs.DFSClient: DataStreamer Exception:
> java.io.IOException: Unable to create new block.
> 08/09/09 13:12:31 WARN dfs.DFSClient: Error Recovery for block
> blk_-4509681241839967328 bad datanode[0]
> 08/09/09 13:12:35 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:35 INFO dfs.DFSClient: Abandoning block
> blk_2848354798649979411
> 08/09/09 13:12:41 WARN dfs.DFSClient: DataStreamer Exception:
> java.io.IOException: Unable to create new block.
> 08/09/09 13:12:41 WARN dfs.DFSClient: Error Recovery for block
> blk_2848354798649979411 bad datanode[0]
> Exception in thread "Thread-0" java.util.ConcurrentModificationException
> at java.util.TreeMap$PrivateEntryIterator.nextEntry(Unknown Source)
> at java.util.TreeMap$KeyIterator.next(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.close(DFSClient.java:217)
> at
> org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:214)
> at
> org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1324)
> at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:224)
> at
> org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:209)
> 08/09/09 13:12:41 INFO dfs.DFSClient: Exception in createBlockOutputStream
> java.io.IOException: Could not read from stream
> 08/09/09 13:12:41 INFO dfs.DFSClient: Abandoning block
> blk_9189111926428577428
>
> On Tue, Sep 9, 2008 at 1:03 PM, Michael Di Domenico <
> mdidomenico4@gmail.com> wrote:
>
>> a little more digging and it appears i cannot run distcp as someone other
>> then hadoop on the namenode
>> /tmp/hadoop-hadoop/mapred/system/job_200809091231_0005/job.xml
>>
>> looking at this directory from the error file the "system" directory does
>> not exist on the namenode, i only have a "local" directory
>>
>>
>> On Tue, Sep 9, 2008 at 12:41 PM, Michael Di Domenico <
>> mdidomenico4@gmail.com> wrote:
>>
>>> i'm not sure that's the issue, i basically tarred up the hadoop directory
>>> from the cluster and copied it over to the non-data node
>>> but i do agree i've likely got a setting wrong, since i can run distcp
>>> from the namenode and it works fine. the question is which one
>>>
>>> On Mon, Sep 8, 2008 at 7:04 PM, Aaron Kimball <aa...@spinnakerlabs.com>wrote:
>>>
>>>> It is likely that you mapred.system.dir and/or fs.default.name settings
>>>> are
>>>> incorrect on the non-datanode machine that you are launching the task
>>>> from.
>>>> These two settings (in your conf/hadoop-site.xml file) must match the
>>>> settings on the cluster itself.
>>>>
>>>> - Aaron
>>>>
>>>> On Sun, Sep 7, 2008 at 8:58 PM, Michael Di Domenico
>>>> <md...@gmail.com>wrote:
>>>>
>>>> > I'm attempting to load data into hadoop (version 0.17.1), from a
>>>> > non-datanode machine in the cluster. I can run jobs and copyFromLocal
>>>> > works
>>>> > fine, but when i try to use distcp i get the below. I'm don't
>>>> understand
>>>> > what the error, can anyone help?
>>>> > Thanks
>>>> >
>>>> > blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
>>>> > file:///Users/mdidomenico/hadoop/1gTestfile
>>>> /user/mdidomenico/1gTestfile
>>>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>>>> > srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
>>>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>>>> > destPath=/user/mdidomenico/1gTestfile1
>>>> > 08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
>>>> > With failures, global counters are inaccurate; consider running with
>>>> -i
>>>> > Copy failed: org.apache.hadoop.ipc.RemoteException:
>>>> java.io.IOException:
>>>> > /tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No
>>>> such
>>>> > file
>>>> > or directory
>>>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
>>>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
>>>> > at
>>>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
>>>> > at
>>>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
>>>> > at
>>>> > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
>>>> > at
>>>> > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>>>> Source)
>>>> > at java.lang.reflect.Method.invoke(Unknown Source)
>>>> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> > at $Proxy1.submitJob(Unknown Source)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at
>>>> >
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> > at
>>>> >
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> > at java.lang.reflect.Method.invoke(Method.java:585)
>>>> > at
>>>> >
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> > at
>>>> >
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> > at $Proxy1.submitJob(Unknown Source)
>>>> > at
>>>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
>>>> > at
>>>> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
>>>> > at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
>>>> > at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>> > at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
>>>> >
>>>>
>>>
>>>
>>
>
Re: distcp failing
Posted by Michael Di Domenico <md...@gmail.com>.
manually creating the "system" directory gets me past the first error, but
now i get this. i'm not necessarily sure its a step forward though, because
the map task never shows up in the jobtracker
[mdidomenico@orange hadoop-0.17.1]$ bin/hadoop distcp
"file:///home/mdidomenico/1gTestfile" "1gTestfile"
08/09/09 13:12:06 INFO util.CopyFiles:
srcPaths=[file:/home/mdidomenico/1gTestfile]
08/09/09 13:12:06 INFO util.CopyFiles: destPath=1gTestfile
08/09/09 13:12:07 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:07 INFO dfs.DFSClient: Abandoning block
blk_5758513071638050362
08/09/09 13:12:13 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:13 INFO dfs.DFSClient: Abandoning block
blk_1691495306775808049
08/09/09 13:12:17 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:17 INFO dfs.DFSClient: Abandoning block
blk_1027634596973755899
08/09/09 13:12:19 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:19 INFO dfs.DFSClient: Abandoning block
blk_4535302510016050282
08/09/09 13:12:23 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:23 INFO dfs.DFSClient: Abandoning block
blk_7022658012001626339
08/09/09 13:12:25 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:25 INFO dfs.DFSClient: Abandoning block
blk_-4509681241839967328
08/09/09 13:12:29 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:29 INFO dfs.DFSClient: Abandoning block
blk_8318033979013580420
08/09/09 13:12:31 WARN dfs.DFSClient: DataStreamer Exception:
java.io.IOException: Unable to create new block.
08/09/09 13:12:31 WARN dfs.DFSClient: Error Recovery for block
blk_-4509681241839967328 bad datanode[0]
08/09/09 13:12:35 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:35 INFO dfs.DFSClient: Abandoning block
blk_2848354798649979411
08/09/09 13:12:41 WARN dfs.DFSClient: DataStreamer Exception:
java.io.IOException: Unable to create new block.
08/09/09 13:12:41 WARN dfs.DFSClient: Error Recovery for block
blk_2848354798649979411 bad datanode[0]
Exception in thread "Thread-0" java.util.ConcurrentModificationException
at java.util.TreeMap$PrivateEntryIterator.nextEntry(Unknown Source)
at java.util.TreeMap$KeyIterator.next(Unknown Source)
at org.apache.hadoop.dfs.DFSClient.close(DFSClient.java:217)
at
org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:214)
at
org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1324)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:224)
at
org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:209)
08/09/09 13:12:41 INFO dfs.DFSClient: Exception in createBlockOutputStream
java.io.IOException: Could not read from stream
08/09/09 13:12:41 INFO dfs.DFSClient: Abandoning block
blk_9189111926428577428
On Tue, Sep 9, 2008 at 1:03 PM, Michael Di Domenico
<md...@gmail.com>wrote:
> a little more digging and it appears i cannot run distcp as someone other
> then hadoop on the namenode
> /tmp/hadoop-hadoop/mapred/system/job_200809091231_0005/job.xml
>
> looking at this directory from the error file the "system" directory does
> not exist on the namenode, i only have a "local" directory
>
>
> On Tue, Sep 9, 2008 at 12:41 PM, Michael Di Domenico <
> mdidomenico4@gmail.com> wrote:
>
>> i'm not sure that's the issue, i basically tarred up the hadoop directory
>> from the cluster and copied it over to the non-data node
>> but i do agree i've likely got a setting wrong, since i can run distcp
>> from the namenode and it works fine. the question is which one
>>
>> On Mon, Sep 8, 2008 at 7:04 PM, Aaron Kimball <aa...@spinnakerlabs.com>wrote:
>>
>>> It is likely that you mapred.system.dir and/or fs.default.name settings
>>> are
>>> incorrect on the non-datanode machine that you are launching the task
>>> from.
>>> These two settings (in your conf/hadoop-site.xml file) must match the
>>> settings on the cluster itself.
>>>
>>> - Aaron
>>>
>>> On Sun, Sep 7, 2008 at 8:58 PM, Michael Di Domenico
>>> <md...@gmail.com>wrote:
>>>
>>> > I'm attempting to load data into hadoop (version 0.17.1), from a
>>> > non-datanode machine in the cluster. I can run jobs and copyFromLocal
>>> > works
>>> > fine, but when i try to use distcp i get the below. I'm don't
>>> understand
>>> > what the error, can anyone help?
>>> > Thanks
>>> >
>>> > blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
>>> > file:///Users/mdidomenico/hadoop/1gTestfile
>>> /user/mdidomenico/1gTestfile
>>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>>> > srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
>>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>>> > destPath=/user/mdidomenico/1gTestfile1
>>> > 08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
>>> > With failures, global counters are inaccurate; consider running with -i
>>> > Copy failed: org.apache.hadoop.ipc.RemoteException:
>>> java.io.IOException:
>>> > /tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No such
>>> > file
>>> > or directory
>>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
>>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
>>> > at
>>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
>>> > at
>>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
>>> > at
>>> > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
>>> > at
>>> > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>>> Source)
>>> > at java.lang.reflect.Method.invoke(Unknown Source)
>>> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> >
>>> > at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>> > at $Proxy1.submitJob(Unknown Source)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > at
>>> >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:585)
>>> > at
>>> >
>>> >
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>> > at
>>> >
>>> >
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>> > at $Proxy1.submitJob(Unknown Source)
>>> > at
>>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
>>> > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
>>> > at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
>>> > at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> > at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
>>> >
>>>
>>
>>
>
Re: distcp failing
Posted by Michael Di Domenico <md...@gmail.com>.
a little more digging and it appears i cannot run distcp as someone other
then hadoop on the namenode
/tmp/hadoop-hadoop/mapred/system/job_200809091231_0005/job.xml
looking at this directory from the error file the "system" directory does
not exist on the namenode, i only have a "local" directory
On Tue, Sep 9, 2008 at 12:41 PM, Michael Di Domenico <mdidomenico4@gmail.com
> wrote:
> i'm not sure that's the issue, i basically tarred up the hadoop directory
> from the cluster and copied it over to the non-data node
> but i do agree i've likely got a setting wrong, since i can run distcp from
> the namenode and it works fine. the question is which one
>
> On Mon, Sep 8, 2008 at 7:04 PM, Aaron Kimball <aa...@spinnakerlabs.com>wrote:
>
>> It is likely that you mapred.system.dir and/or fs.default.name settings
>> are
>> incorrect on the non-datanode machine that you are launching the task
>> from.
>> These two settings (in your conf/hadoop-site.xml file) must match the
>> settings on the cluster itself.
>>
>> - Aaron
>>
>> On Sun, Sep 7, 2008 at 8:58 PM, Michael Di Domenico
>> <md...@gmail.com>wrote:
>>
>> > I'm attempting to load data into hadoop (version 0.17.1), from a
>> > non-datanode machine in the cluster. I can run jobs and copyFromLocal
>> > works
>> > fine, but when i try to use distcp i get the below. I'm don't
>> understand
>> > what the error, can anyone help?
>> > Thanks
>> >
>> > blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
>> > file:///Users/mdidomenico/hadoop/1gTestfile /user/mdidomenico/1gTestfile
>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>> > srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
>> > 08/09/07 23:56:06 INFO util.CopyFiles:
>> > destPath=/user/mdidomenico/1gTestfile1
>> > 08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
>> > With failures, global counters are inaccurate; consider running with -i
>> > Copy failed: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
>> > /tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No such
>> > file
>> > or directory
>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
>> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
>> > at
>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
>> > at
>> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
>> > at
>> > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
>> > at
>> > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>> Source)
>> > at java.lang.reflect.Method.invoke(Unknown Source)
>> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> >
>> > at org.apache.hadoop.ipc.Client.call(Client.java:557)
>> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>> > at $Proxy1.submitJob(Unknown Source)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:585)
>> > at
>> >
>> >
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>> > at
>> >
>> >
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>> > at $Proxy1.submitJob(Unknown Source)
>> > at
>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
>> > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
>> > at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
>> > at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> > at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
>> >
>>
>
>
Re: distcp failing
Posted by Michael Di Domenico <md...@gmail.com>.
i'm not sure that's the issue, i basically tarred up the hadoop directory
from the cluster and copied it over to the non-data node
but i do agree i've likely got a setting wrong, since i can run distcp from
the namenode and it works fine. the question is which one
On Mon, Sep 8, 2008 at 7:04 PM, Aaron Kimball <aa...@spinnakerlabs.com>wrote:
> It is likely that you mapred.system.dir and/or fs.default.name settings
> are
> incorrect on the non-datanode machine that you are launching the task from.
> These two settings (in your conf/hadoop-site.xml file) must match the
> settings on the cluster itself.
>
> - Aaron
>
> On Sun, Sep 7, 2008 at 8:58 PM, Michael Di Domenico
> <md...@gmail.com>wrote:
>
> > I'm attempting to load data into hadoop (version 0.17.1), from a
> > non-datanode machine in the cluster. I can run jobs and copyFromLocal
> > works
> > fine, but when i try to use distcp i get the below. I'm don't understand
> > what the error, can anyone help?
> > Thanks
> >
> > blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
> > file:///Users/mdidomenico/hadoop/1gTestfile /user/mdidomenico/1gTestfile
> > 08/09/07 23:56:06 INFO util.CopyFiles:
> > srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
> > 08/09/07 23:56:06 INFO util.CopyFiles:
> > destPath=/user/mdidomenico/1gTestfile1
> > 08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
> > With failures, global counters are inaccurate; consider running with -i
> > Copy failed: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> > /tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No such
> > file
> > or directory
> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
> > at
> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
> > at
> > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
> > at
> > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
> > at
> > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >
> > at org.apache.hadoop.ipc.Client.call(Client.java:557)
> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> > at $Proxy1.submitJob(Unknown Source)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:585)
> > at
> >
> >
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> > at
> >
> >
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> > at $Proxy1.submitJob(Unknown Source)
> > at
> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
> > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
> > at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
> > at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> > at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
> >
>
Re: distcp failing
Posted by Aaron Kimball <aa...@spinnakerlabs.com>.
It is likely that you mapred.system.dir and/or fs.default.name settings are
incorrect on the non-datanode machine that you are launching the task from.
These two settings (in your conf/hadoop-site.xml file) must match the
settings on the cluster itself.
- Aaron
On Sun, Sep 7, 2008 at 8:58 PM, Michael Di Domenico
<md...@gmail.com>wrote:
> I'm attempting to load data into hadoop (version 0.17.1), from a
> non-datanode machine in the cluster. I can run jobs and copyFromLocal
> works
> fine, but when i try to use distcp i get the below. I'm don't understand
> what the error, can anyone help?
> Thanks
>
> blue:hadoop-0.17.1 mdidomenico$ time bin/hadoop distcp -overwrite
> file:///Users/mdidomenico/hadoop/1gTestfile /user/mdidomenico/1gTestfile
> 08/09/07 23:56:06 INFO util.CopyFiles:
> srcPaths=[file:/Users/mdidomenico/hadoop/1gTestfile]
> 08/09/07 23:56:06 INFO util.CopyFiles:
> destPath=/user/mdidomenico/1gTestfile1
> 08/09/07 23:56:07 INFO util.CopyFiles: srcCount=1
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> /tmp/hadoop-hadoop/mapred/system/job_200809072254_0005/job.xml: No such
> file
> or directory
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149)
> at
> org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155)
> at
> org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136)
> at
> org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175)
> at
> org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:557)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> at $Proxy1.submitJob(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at
>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at
>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at $Proxy1.submitJob(Unknown Source)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
> at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:604)
> at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
>