You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Matt Zytaruk <ma...@wavefire.com> on 2006/01/06 01:59:58 UTC
mapred system dir
Hello all, I seem to be having a problem with the latest version (as of
2006/01/04 anyways) of mapred. Everytime i try and do something, after
it's first set of tasks finishes, and it tries to start more, I get
this exception. The mapred system dir never seems to get created and
used, so it can't open the file because it doesnt exist. What's goin on?
java.io.IOException: Cannot open filename
/nutch-data/nutch/tmp/nutch/mapred/system/submit_euiwjv/job.xml
at org.apache.nutch.ipc.Client.call(Client.java:294)
at org.apache.nutch.ipc.RPC$Invoker.invoke(RPC.java:127)
at $Proxy1.open(Unknown Source)
at
org.apache.nutch.ndfs.NDFSClient$NDFSInputStream.openInfo(NDFSClient.java:256)
at
org.apache.nutch.ndfs.NDFSClient$NDFSInputStream.<init>(NDFSClient.java:242)
at org.apache.nutch.ndfs.NDFSClient.open(NDFSClient.java:79)
at
org.apache.nutch.fs.NDFSFileSystem.openRaw(NDFSFileSystem.java:66)
at
org.apache.nutch.fs.NFSDataInputStream$Checker.<init>(NFSDataInputStream.java:45)
at
org.apache.nutch.fs.NFSDataInputStream.<init>(NFSDataInputStream.java:221)
at
org.apache.nutch.fs.NutchFileSystem.open(NutchFileSystem.java:160)
at
org.apache.nutch.fs.NutchFileSystem.open(NutchFileSystem.java:149)
at
org.apache.nutch.fs.NDFSFileSystem.copyToLocalFile(NDFSFileSystem.java:221)
at
org.apache.nutch.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:346)
at
org.apache.nutch.mapred.TaskTracker$TaskInProgress.<init>(TaskTracker.java:332)
at
org.apache.nutch.mapred.TaskTracker.offerService(TaskTracker.java:232)
at org.apache.nutch.mapred.TaskTracker.run(TaskTracker.java:286)
at org.apache.nutch.mapred.TaskTracker.main(TaskTracker.java:651)
Thanks in advance
-Matt Zytaruk
Re: mapred system dir
Posted by Matt Zytaruk <ma...@wavefire.com>.
This is on Suse linux i believe. The user hasn't had any trouble
creating the folder in the past, and it creates the other temporary
folders fine. Just not the system one.
Stefan Groschupf wrote:
> What operation system do you use?
> Are you sure the user that runs the nutch process can create the folder?
>
> Am 06.01.2006 um 01:59 schrieb Matt Zytaruk:
>
>> Hello all, I seem to be having a problem with the latest version (as
>> of 2006/01/04 anyways) of mapred. Everytime i try and do
>> something, after it's first set of tasks finishes, and it tries to
>> start more, I get this exception. The mapred system dir never seems
>> to get created and used, so it can't open the file because it doesnt
>> exist. What's goin on?
>>
>> java.io.IOException: Cannot open filename /nutch-data/nutch/tmp/
>> nutch/mapred/system/submit_euiwjv/job.xml
>> at org.apache.nutch.ipc.Client.call(Client.java:294)
>> at org.apache.nutch.ipc.RPC$Invoker.invoke(RPC.java:127)
>> at $Proxy1.open(Unknown Source)
>> at org.apache.nutch.ndfs.NDFSClient$NDFSInputStream.openInfo
>> (NDFSClient.java:256)
>> at org.apache.nutch.ndfs.NDFSClient$NDFSInputStream.<init>
>> (NDFSClient.java:242)
>> at org.apache.nutch.ndfs.NDFSClient.open(NDFSClient.java:79)
>> at org.apache.nutch.fs.NDFSFileSystem.openRaw
>> (NDFSFileSystem.java:66)
>> at org.apache.nutch.fs.NFSDataInputStream$Checker.<init>
>> (NFSDataInputStream.java:45)
>> at org.apache.nutch.fs.NFSDataInputStream.<init>
>> (NFSDataInputStream.java:221)
>> at org.apache.nutch.fs.NutchFileSystem.open
>> (NutchFileSystem.java:160)
>> at org.apache.nutch.fs.NutchFileSystem.open
>> (NutchFileSystem.java:149)
>> at org.apache.nutch.fs.NDFSFileSystem.copyToLocalFile
>> (NDFSFileSystem.java:221)
>> at org.apache.nutch.mapred.TaskTracker
>> $TaskInProgress.localizeTask(TaskTracker.java:346)
>> at org.apache.nutch.mapred.TaskTracker$TaskInProgress.<init>
>> (TaskTracker.java:332)
>> at org.apache.nutch.mapred.TaskTracker.offerService
>> (TaskTracker.java:232)
>> at org.apache.nutch.mapred.TaskTracker.run(TaskTracker.java: 286)
>> at org.apache.nutch.mapred.TaskTracker.main(TaskTracker.java:
>> 651)
>>
>> Thanks in advance
>> -Matt Zytaruk
>>
>
>
>
Re: mapred system dir
Posted by Stefan Groschupf <sg...@media-style.com>.
What operation system do you use?
Are you sure the user that runs the nutch process can create the folder?
Am 06.01.2006 um 01:59 schrieb Matt Zytaruk:
> Hello all, I seem to be having a problem with the latest version
> (as of 2006/01/04 anyways) of mapred. Everytime i try and do
> something, after it's first set of tasks finishes, and it tries to
> start more, I get this exception. The mapred system dir never seems
> to get created and used, so it can't open the file because it
> doesnt exist. What's goin on?
>
> java.io.IOException: Cannot open filename /nutch-data/nutch/tmp/
> nutch/mapred/system/submit_euiwjv/job.xml
> at org.apache.nutch.ipc.Client.call(Client.java:294)
> at org.apache.nutch.ipc.RPC$Invoker.invoke(RPC.java:127)
> at $Proxy1.open(Unknown Source)
> at org.apache.nutch.ndfs.NDFSClient$NDFSInputStream.openInfo
> (NDFSClient.java:256)
> at org.apache.nutch.ndfs.NDFSClient$NDFSInputStream.<init>
> (NDFSClient.java:242)
> at org.apache.nutch.ndfs.NDFSClient.open(NDFSClient.java:79)
> at org.apache.nutch.fs.NDFSFileSystem.openRaw
> (NDFSFileSystem.java:66)
> at org.apache.nutch.fs.NFSDataInputStream$Checker.<init>
> (NFSDataInputStream.java:45)
> at org.apache.nutch.fs.NFSDataInputStream.<init>
> (NFSDataInputStream.java:221)
> at org.apache.nutch.fs.NutchFileSystem.open
> (NutchFileSystem.java:160)
> at org.apache.nutch.fs.NutchFileSystem.open
> (NutchFileSystem.java:149)
> at org.apache.nutch.fs.NDFSFileSystem.copyToLocalFile
> (NDFSFileSystem.java:221)
> at org.apache.nutch.mapred.TaskTracker
> $TaskInProgress.localizeTask(TaskTracker.java:346)
> at org.apache.nutch.mapred.TaskTracker$TaskInProgress.<init>
> (TaskTracker.java:332)
> at org.apache.nutch.mapred.TaskTracker.offerService
> (TaskTracker.java:232)
> at org.apache.nutch.mapred.TaskTracker.run(TaskTracker.java:
> 286)
> at org.apache.nutch.mapred.TaskTracker.main(TaskTracker.java:
> 651)
>
> Thanks in advance
> -Matt Zytaruk
>