You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by Lin Chia-Hung <cl...@googlemail.com> on 2010/10/30 10:24:49 UTC
hama system dir
I follow the getting started page at
http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
the hama system, but I keep getting the following error (saying the
system dir is null.)
After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
system dir as BSPMaster contains code that would throws ioexception if
no system dir; but even adding system folder (e.g. touch
/tmp/hadoop-${user}/bsp/system) it still throws this error.
How can I solve this problem?
2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
cleaning system directory: null
java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
at org.apache.hadoop.ipc.Client.call(Client.java:742)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy4.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
Re: hama system dir
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hello,
> Is there any restriction for hama to use hadoop version?
Yes, the current trunk version only works on the hadoop-0.20.x.
Please try again with
http://apache.mirror.cdnetworks.com//hadoop/core/hadoop-0.20.2/
Thanks.
On Mon, Nov 1, 2010 at 1:33 PM, Lin Chia-Hung <cl...@googlemail.com> wrote:
> I add the bsp.system.dir property to hama-site.xml, but the error
> message remains there; directly mkdir /tmp/hadoop-<user>/bsp/system in
> file system doesn't help either.
>
> The version of hama and hadoop (hdfs and common) I am using is trunk;
> zookeeper 3.3.1.
>
> Previously I tried using hadoop 0.20.x (as instruction in getting
> started - http://wiki.apache.org/hama/GettingStarted), but I notice in
> that version I can not execute fs command e.g. `hadoop fs -ls /' There
> is a hdfs property change which requires a patch; therefore I switch
> to use trunk (patch has been added and I can add my changes as well).
>
> Is there any restriction for hama to use hadoop version?
>
> Or any other places that I can check regarding to this problem?
>
> Thanks for help.
>
> 2010/10/30 Filipe David Manana <fd...@apache.org>:
>> Which version of Hama are you using?
>>
>> For the latest, make sure your XML config has the following:
>>
>> <property>
>> <name>bsp.system.dir</name>
>> <value>${hadoop.tmp.dir}/bsp/system</value>
>> <description>The shared directory where BSP stores control files.
>> </description>
>> </property>
>>
>> Let us know if it fixes the issue for you.
>>
>> regards
>>
>> On Sat, Oct 30, 2010 at 9:24 AM, Lin Chia-Hung <cl...@googlemail.com> wrote:
>>> I follow the getting started page at
>>> http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
>>> the hama system, but I keep getting the following error (saying the
>>> system dir is null.)
>>>
>>> After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
>>> system dir as BSPMaster contains code that would throws ioexception if
>>> no system dir; but even adding system folder (e.g. touch
>>> /tmp/hadoop-${user}/bsp/system) it still throws this error.
>>>
>>> How can I solve this problem?
>>>
>>> 2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
>>> cleaning system directory: null
>>> java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
>>> exception: java.io.EOFException
>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:742)
>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>>> at $Proxy4.getProtocolVersion(Unknown Source)
>>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>>> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
>>> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>>> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
>>> at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
>>> at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
>>> at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
>>> Caused by: java.io.EOFException
>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
>>>
>>
>>
>>
>> --
>> Filipe David Manana,
>> fdmanana@gmail.com, fdmanana@apache.org
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
>>
>
--
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org
Re: hama system dir
Posted by Lin Chia-Hung <cl...@googlemail.com>.
Thanks for clarifying. I think my problem is solved now.
Thanks again for help.
2010/11/1 Edward J. Yoon <ed...@apache.org>:
> http://markmail.org/search/?q=hama-user%20Hangs%20on%20Pi%20example#query:hama-user%20Hangs%20on%20Pi%20example+page:1+mid:fcxt4a6uylia7lp2+state:results
>
> Sorry, it's a bug. I'll fix it soon.
>
> Thanks. :)
>
> On Mon, Nov 1, 2010 at 2:19 PM, Lin Chia-Hung <cl...@googlemail.com> wrote:
>> After rolling back to use hadoop 0.20.2, I think this problem is solved.
>>
>> But I notice that when executing the pi example `hama jar
>> build/hama-0.2.0-dev-examples.jar pi`, seems like it will stays
>> infinite at the state as below (it won't finish like hello example):
>>
>> 10/11/01 13:01:01 INFO bsp.BSPJobClient: Running job: job_201011011257_0002
>>
>> Groom server processes are running on all machines and I can see the
>> estimated pi value printed in one of node's log. e.g.
>>
>> 2010-11-01 13:01:02,089 INFO
>> org.apache.hama.examples.PiEstimator$MyEstimator: Receives
>> messages:3.1372
>> 2010-11-01 13:01:02,090 INFO org.apache.hama.examples.PiEstimator$MyEstimator:
>> Estimated value of PI is 3.1372
>>
>> Is there any reason why it takes so a long time to accomplish a task?
>>
>>
>> 2010/11/1 Lin Chia-Hung <cl...@googlemail.com>:
>>> I add the bsp.system.dir property to hama-site.xml, but the error
>>> message remains there; directly mkdir /tmp/hadoop-<user>/bsp/system in
>>> file system doesn't help either.
>>>
>>> The version of hama and hadoop (hdfs and common) I am using is trunk;
>>> zookeeper 3.3.1.
>>>
>>> Previously I tried using hadoop 0.20.x (as instruction in getting
>>> started - http://wiki.apache.org/hama/GettingStarted), but I notice in
>>> that version I can not execute fs command e.g. `hadoop fs -ls /' There
>>> is a hdfs property change which requires a patch; therefore I switch
>>> to use trunk (patch has been added and I can add my changes as well).
>>>
>>> Is there any restriction for hama to use hadoop version?
>>>
>>> Or any other places that I can check regarding to this problem?
>>>
>>> Thanks for help.
>>>
>>> 2010/10/30 Filipe David Manana <fd...@apache.org>:
>>>> Which version of Hama are you using?
>>>>
>>>> For the latest, make sure your XML config has the following:
>>>>
>>>> <property>
>>>> <name>bsp.system.dir</name>
>>>> <value>${hadoop.tmp.dir}/bsp/system</value>
>>>> <description>The shared directory where BSP stores control files.
>>>> </description>
>>>> </property>
>>>>
>>>> Let us know if it fixes the issue for you.
>>>>
>>>> regards
>>>>
>>>> On Sat, Oct 30, 2010 at 9:24 AM, Lin Chia-Hung <cl...@googlemail.com> wrote:
>>>>> I follow the getting started page at
>>>>> http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
>>>>> the hama system, but I keep getting the following error (saying the
>>>>> system dir is null.)
>>>>>
>>>>> After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
>>>>> system dir as BSPMaster contains code that would throws ioexception if
>>>>> no system dir; but even adding system folder (e.g. touch
>>>>> /tmp/hadoop-${user}/bsp/system) it still throws this error.
>>>>>
>>>>> How can I solve this problem?
>>>>>
>>>>> 2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
>>>>> cleaning system directory: null
>>>>> java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
>>>>> exception: java.io.EOFException
>>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:742)
>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>>>>> at $Proxy4.getProtocolVersion(Unknown Source)
>>>>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>>>>> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
>>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
>>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
>>>>> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>>>>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
>>>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>>>>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
>>>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
>>>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>>>>> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
>>>>> at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
>>>>> at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
>>>>> at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
>>>>> Caused by: java.io.EOFException
>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Filipe David Manana,
>>>> fdmanana@gmail.com, fdmanana@apache.org
>>>>
>>>> "Reasonable men adapt themselves to the world.
>>>> Unreasonable men adapt the world to themselves.
>>>> That's why all progress depends on unreasonable men."
>>>>
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> edwardyoon@apache.org
> http://blog.udanax.org
>
Re: hama system dir
Posted by "Edward J. Yoon" <ed...@apache.org>.
http://markmail.org/search/?q=hama-user%20Hangs%20on%20Pi%20example#query:hama-user%20Hangs%20on%20Pi%20example+page:1+mid:fcxt4a6uylia7lp2+state:results
Sorry, it's a bug. I'll fix it soon.
Thanks. :)
On Mon, Nov 1, 2010 at 2:19 PM, Lin Chia-Hung <cl...@googlemail.com> wrote:
> After rolling back to use hadoop 0.20.2, I think this problem is solved.
>
> But I notice that when executing the pi example `hama jar
> build/hama-0.2.0-dev-examples.jar pi`, seems like it will stays
> infinite at the state as below (it won't finish like hello example):
>
> 10/11/01 13:01:01 INFO bsp.BSPJobClient: Running job: job_201011011257_0002
>
> Groom server processes are running on all machines and I can see the
> estimated pi value printed in one of node's log. e.g.
>
> 2010-11-01 13:01:02,089 INFO
> org.apache.hama.examples.PiEstimator$MyEstimator: Receives
> messages:3.1372
> 2010-11-01 13:01:02,090 INFO org.apache.hama.examples.PiEstimator$MyEstimator:
> Estimated value of PI is 3.1372
>
> Is there any reason why it takes so a long time to accomplish a task?
>
>
> 2010/11/1 Lin Chia-Hung <cl...@googlemail.com>:
>> I add the bsp.system.dir property to hama-site.xml, but the error
>> message remains there; directly mkdir /tmp/hadoop-<user>/bsp/system in
>> file system doesn't help either.
>>
>> The version of hama and hadoop (hdfs and common) I am using is trunk;
>> zookeeper 3.3.1.
>>
>> Previously I tried using hadoop 0.20.x (as instruction in getting
>> started - http://wiki.apache.org/hama/GettingStarted), but I notice in
>> that version I can not execute fs command e.g. `hadoop fs -ls /' There
>> is a hdfs property change which requires a patch; therefore I switch
>> to use trunk (patch has been added and I can add my changes as well).
>>
>> Is there any restriction for hama to use hadoop version?
>>
>> Or any other places that I can check regarding to this problem?
>>
>> Thanks for help.
>>
>> 2010/10/30 Filipe David Manana <fd...@apache.org>:
>>> Which version of Hama are you using?
>>>
>>> For the latest, make sure your XML config has the following:
>>>
>>> <property>
>>> <name>bsp.system.dir</name>
>>> <value>${hadoop.tmp.dir}/bsp/system</value>
>>> <description>The shared directory where BSP stores control files.
>>> </description>
>>> </property>
>>>
>>> Let us know if it fixes the issue for you.
>>>
>>> regards
>>>
>>> On Sat, Oct 30, 2010 at 9:24 AM, Lin Chia-Hung <cl...@googlemail.com> wrote:
>>>> I follow the getting started page at
>>>> http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
>>>> the hama system, but I keep getting the following error (saying the
>>>> system dir is null.)
>>>>
>>>> After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
>>>> system dir as BSPMaster contains code that would throws ioexception if
>>>> no system dir; but even adding system folder (e.g. touch
>>>> /tmp/hadoop-${user}/bsp/system) it still throws this error.
>>>>
>>>> How can I solve this problem?
>>>>
>>>> 2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
>>>> cleaning system directory: null
>>>> java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
>>>> exception: java.io.EOFException
>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:742)
>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>>>> at $Proxy4.getProtocolVersion(Unknown Source)
>>>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>>>> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
>>>> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>>>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
>>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>>>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
>>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
>>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>>>> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
>>>> at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
>>>> at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
>>>> at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
>>>> Caused by: java.io.EOFException
>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
>>>>
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>> fdmanana@gmail.com, fdmanana@apache.org
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>>
>>
>
--
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org
Re: hama system dir
Posted by Lin Chia-Hung <cl...@googlemail.com>.
After rolling back to use hadoop 0.20.2, I think this problem is solved.
But I notice that when executing the pi example `hama jar
build/hama-0.2.0-dev-examples.jar pi`, seems like it will stays
infinite at the state as below (it won't finish like hello example):
10/11/01 13:01:01 INFO bsp.BSPJobClient: Running job: job_201011011257_0002
Groom server processes are running on all machines and I can see the
estimated pi value printed in one of node's log. e.g.
2010-11-01 13:01:02,089 INFO
org.apache.hama.examples.PiEstimator$MyEstimator: Receives
messages:3.1372
2010-11-01 13:01:02,090 INFO org.apache.hama.examples.PiEstimator$MyEstimator:
Estimated value of PI is 3.1372
Is there any reason why it takes so a long time to accomplish a task?
2010/11/1 Lin Chia-Hung <cl...@googlemail.com>:
> I add the bsp.system.dir property to hama-site.xml, but the error
> message remains there; directly mkdir /tmp/hadoop-<user>/bsp/system in
> file system doesn't help either.
>
> The version of hama and hadoop (hdfs and common) I am using is trunk;
> zookeeper 3.3.1.
>
> Previously I tried using hadoop 0.20.x (as instruction in getting
> started - http://wiki.apache.org/hama/GettingStarted), but I notice in
> that version I can not execute fs command e.g. `hadoop fs -ls /' There
> is a hdfs property change which requires a patch; therefore I switch
> to use trunk (patch has been added and I can add my changes as well).
>
> Is there any restriction for hama to use hadoop version?
>
> Or any other places that I can check regarding to this problem?
>
> Thanks for help.
>
> 2010/10/30 Filipe David Manana <fd...@apache.org>:
>> Which version of Hama are you using?
>>
>> For the latest, make sure your XML config has the following:
>>
>> <property>
>> <name>bsp.system.dir</name>
>> <value>${hadoop.tmp.dir}/bsp/system</value>
>> <description>The shared directory where BSP stores control files.
>> </description>
>> </property>
>>
>> Let us know if it fixes the issue for you.
>>
>> regards
>>
>> On Sat, Oct 30, 2010 at 9:24 AM, Lin Chia-Hung <cl...@googlemail.com> wrote:
>>> I follow the getting started page at
>>> http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
>>> the hama system, but I keep getting the following error (saying the
>>> system dir is null.)
>>>
>>> After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
>>> system dir as BSPMaster contains code that would throws ioexception if
>>> no system dir; but even adding system folder (e.g. touch
>>> /tmp/hadoop-${user}/bsp/system) it still throws this error.
>>>
>>> How can I solve this problem?
>>>
>>> 2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
>>> cleaning system directory: null
>>> java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
>>> exception: java.io.EOFException
>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:742)
>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>>> at $Proxy4.getProtocolVersion(Unknown Source)
>>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>>> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
>>> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>>> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
>>> at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
>>> at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
>>> at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
>>> Caused by: java.io.EOFException
>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
>>>
>>
>>
>>
>> --
>> Filipe David Manana,
>> fdmanana@gmail.com, fdmanana@apache.org
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
>>
>
Re: hama system dir
Posted by Lin Chia-Hung <cl...@googlemail.com>.
I add the bsp.system.dir property to hama-site.xml, but the error
message remains there; directly mkdir /tmp/hadoop-<user>/bsp/system in
file system doesn't help either.
The version of hama and hadoop (hdfs and common) I am using is trunk;
zookeeper 3.3.1.
Previously I tried using hadoop 0.20.x (as instruction in getting
started - http://wiki.apache.org/hama/GettingStarted), but I notice in
that version I can not execute fs command e.g. `hadoop fs -ls /' There
is a hdfs property change which requires a patch; therefore I switch
to use trunk (patch has been added and I can add my changes as well).
Is there any restriction for hama to use hadoop version?
Or any other places that I can check regarding to this problem?
Thanks for help.
2010/10/30 Filipe David Manana <fd...@apache.org>:
> Which version of Hama are you using?
>
> For the latest, make sure your XML config has the following:
>
> <property>
> <name>bsp.system.dir</name>
> <value>${hadoop.tmp.dir}/bsp/system</value>
> <description>The shared directory where BSP stores control files.
> </description>
> </property>
>
> Let us know if it fixes the issue for you.
>
> regards
>
> On Sat, Oct 30, 2010 at 9:24 AM, Lin Chia-Hung <cl...@googlemail.com> wrote:
>> I follow the getting started page at
>> http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
>> the hama system, but I keep getting the following error (saying the
>> system dir is null.)
>>
>> After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
>> system dir as BSPMaster contains code that would throws ioexception if
>> no system dir; but even adding system folder (e.g. touch
>> /tmp/hadoop-${user}/bsp/system) it still throws this error.
>>
>> How can I solve this problem?
>>
>> 2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
>> cleaning system directory: null
>> java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
>> exception: java.io.EOFException
>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
>> at org.apache.hadoop.ipc.Client.call(Client.java:742)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>> at $Proxy4.getProtocolVersion(Unknown Source)
>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
>> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
>> at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
>> at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
>> at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
>> Caused by: java.io.EOFException
>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
>>
>
>
>
> --
> Filipe David Manana,
> fdmanana@gmail.com, fdmanana@apache.org
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
>
Re: hama system dir
Posted by Filipe David Manana <fd...@apache.org>.
Which version of Hama are you using?
For the latest, make sure your XML config has the following:
<property>
<name>bsp.system.dir</name>
<value>${hadoop.tmp.dir}/bsp/system</value>
<description>The shared directory where BSP stores control files.
</description>
</property>
Let us know if it fixes the issue for you.
regards
On Sat, Oct 30, 2010 at 9:24 AM, Lin Chia-Hung <cl...@googlemail.com> wrote:
> I follow the getting started page at
> http://wiki.apache.org/hama/GettingStarted#Setting_up_Hama setting up
> the hama system, but I keep getting the following error (saying the
> system dir is null.)
>
> After checking the /tmp/hadoop-${user}/bsp/, seems like there is no
> system dir as BSPMaster contains code that would throws ioexception if
> no system dir; but even adding system folder (e.g. touch
> /tmp/hadoop-${user}/bsp/system) it still throws this error.
>
> How can I solve this problem?
>
> 2010-10-30 16:16:59,585 INFO org.apache.hama.bsp.BSPMaster: problem
> cleaning system directory: null
> java.io.IOException: Call to lab01/140.127.220.29:9000 failed on local
> exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
> at org.apache.hadoop.ipc.Client.call(Client.java:742)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
> at $Proxy4.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:169)
> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:131)
> at org.apache.hama.bsp.BSPMaster.startTracker(BSPMaster.java:237)
> at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:230)
> at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:43)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:53)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
>
--
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."