You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Lyu Xuedong <lx...@gmail.com> on 2013/04/16 15:55:21 UTC
Why my distributed mode does not work?
My project can work well in eclipse, but when I export it as a jar file
and submit it to a two-node hama cluster, some errors and fatal occur if
the input file is over 64M.
I suspect that my configure files of hadoop or hama is somewhere not
right, but a pi estimator can run normally on my cluster and
GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole
day, nothing improved.
What in my input file are a large number of RDF triples: "<subject>
<predicate> <object> ." My task is to create vertices for subjects and
objects. Predicates are subjects' edges.
What should I do ?
Terminal output:
13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2
13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015
13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
sync.ZKSyncClient: Initializing ZK Sync Client
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
hadoop1/1.2.3.4:61002
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
Starting SocketReader
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
IPC Server Responder: starting
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
IPC Server handler 0 on 61002: starting
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
IPC Server listener on 61002: starting
attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask:
Error running bsp setup and bsp function.
attempt_201304161357_0015_000000_0: java.lang.NullPointerException
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
Stopping server on 61002
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
IPC Server handler 0 on 61002: exiting
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
Stopping IPC Server listener on 61002
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
Stopping IPC Server Responder
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
metrics.RpcInstrumentation: shut down
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask:
Shutting down ping service.
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL
bsp.GroomServer: Error running child
attempt_201304161357_0015_000000_0: java.lang.NullPointerException
attempt_201304161357_0015_000000_0: java.lang.NullPointerException
13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
tasklogs:
attempt_201304161357_0015_000000_0.log
13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
Zookeeper! At hadoop1/1.2.3.4:61002
13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting
13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
address:hadoop1 port:61002
13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting
13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
function.
java.lang.NullPointerException
13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting
13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002
13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
java.lang.NullPointerException
java.lang.NullPointerException
attempt_201304161357_0015_000001_0.log
13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
Zookeeper! At hadoop1/1.2.3.4:61001
13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
NoNode for /bsp/job_201304161357_0015/peers
13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
address:hadoop1 port:61001
13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting
13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting
13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
at org.apache.hadoop.ipc.Client.call(Client.java:1071)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy3.put(Unknown Source)
at
org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
Re: Why my distributed mode does not work?
Posted by "Edward J. Yoon" <ed...@apache.org>.
P.S., please see our official wiki.
On Sat, Apr 20, 2013 at 9:34 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Yes, the repair function has deleted.
>
>
> On Sat, Apr 20, 2013 at 7:26 PM, Lyu Xuedong <lx...@gmail.com> wrote:
>> Hi, Edward,
>>
>> Thank you.
>> I updated the version. There comes a new problem.
>>
>> In the /Apache Hama BSP Programming Model
>> (http://people.apache.org/~tjungblut/downloads/hamadocs/ApacheHamaBSPProgrammingmodel_06.pdf)/
>> there is a paragraph describes 'Graph repair' : "Hama requires a graph to be
>> completed before feeding it to an algorithm. By complete we mean that every
>> vertex that is referenced by an edge must somewhere be a vertex in the
>> graph. In many cases of leafs this is not always the case, therefore we have
>> added a repair functionality which is traversing the whole graph for leafs
>> and adding them to the vertex structure to prevent algorithms from breaking
>> with NullPointerExceptions when it does not find a referenced vertex. You
>> can turn this feature on by setting it in your configuration like this:
>> conf.setBoolean(GraphJobRunner.GRAPH_REPAIR, true);"
>>
>> I followed the guidance but got a hint 'GRAPH_REPAIR cannot be resolved or
>> is not a field' programming in Eclipse. I read the source code later and
>> there is really no variable named GRAPH_REPAIR in GraphJobRunner.java which
>> exists in the version 0.5.0. Is this function not supported any longer ? How
>> can I make a 'repaired graph'?
>> Thank you.
>>
>>
>> On 04/17/2013 12:51 PM, Edward J. Yoon wrote:
>>>
>>> Please use 0.6.1 and try your application with small data again.
>>>
>>> See also http://hama.apache.org/run_examples.html
>>>
>>> On Wed, Apr 17, 2013 at 10:27 AM, Lyu Xuedong <lx...@gmail.com> wrote:
>>>>
>>>> hama: 0.6.0
>>>> hadoop : 1.0.4
>>>> JDK : 1.6
>>>> OS : ubuntu 12.04
>>>>
>>>>
>>>> On 04/17/2013 05:47 AM, Edward J. Yoon wrote:
>>>>>
>>>>> Your version?
>>>>>
>>>>> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <lx...@gmail.com> wrote:
>>>>>>
>>>>>> Hi, Edward, thank you. But your suggestion seems do not work, do you
>>>>>> have
>>>>>> some other advice ?
>>>>>>
>>>>>> Is java.lang.NullPointerException related with heap size ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> 在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
>>>>>>
>>>>>>> I guess you need to increase the child processor JVM heap size.
>>>>>>>
>>>>>>> - conf/hama-site.xml:
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>bsp.child.java.opts</name>
>>>>>>> <value>-Xmx2048m</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> My project can work well in eclipse, but when I export it as a jar
>>>>>>>> file
>>>>>>>> and
>>>>>>>> submit it to a two-node hama cluster, some errors and fatal occur if
>>>>>>>> the
>>>>>>>> input file is over 64M.
>>>>>>>> I suspect that my configure files of hadoop or hama is somewhere not
>>>>>>>> right,
>>>>>>>> but a pi estimator can run normally on my cluster and
>>>>>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a
>>>>>>>> whole
>>>>>>>> day,
>>>>>>>> nothing improved.
>>>>>>>> What in my input file are a large number of RDF triples: "<subject>
>>>>>>>> <predicate> <object> ." My task is to create vertices for subjects
>>>>>>>> and
>>>>>>>> objects. Predicates are subjects' edges.
>>>>>>>> What should I do ?
>>>>>>>>
>>>>>>>> Terminal output:
>>>>>>>>
>>>>>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>>>>>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to
>>>>>>>> process : 2
>>>>>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job:
>>>>>>>> job_201304161357_0015
>>>>>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>>>>>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>>>>>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>>>>>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>>>>>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>>>>>>>> hadoop1/1.2.3.4:61002
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> ipc.Server:
>>>>>>>> Starting SocketReader
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> ipc.Server:
>>>>>>>> IPC
>>>>>>>> Server Responder: starting
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> ipc.Server:
>>>>>>>> IPC
>>>>>>>> Server handler 0 on 61002: starting
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>>> ipc.Server:
>>>>>>>> IPC
>>>>>>>> Server listener on 61002: starting
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR
>>>>>>>> bsp.BSPTask:
>>>>>>>> Error running bsp setup and bsp function.
>>>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>>> ipc.Server:
>>>>>>>> Stopping server on 61002
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>>> ipc.Server:
>>>>>>>> IPC
>>>>>>>> Server handler 0 on 61002: exiting
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>>> ipc.Server:
>>>>>>>> Stopping IPC Server listener on 61002
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>>> ipc.Server:
>>>>>>>> Stopping IPC Server Responder
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>>> metrics.RpcInstrumentation: shut down
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR
>>>>>>>> bsp.BSPTask:
>>>>>>>> Shutting down ping service.
>>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL
>>>>>>>> bsp.GroomServer:
>>>>>>>> Error running child
>>>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>>>>>>>
>>>>>>>> tasklogs:
>>>>>>>> attempt_201304161357_0015_000000_0.log
>>>>>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>>>>> to
>>>>>>>> Zookeeper! At hadoop1/1.2.3.4:61002
>>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>>>>> starting
>>>>>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>>>>> address:hadoop1 port:61002
>>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002:
>>>>>>>> starting
>>>>>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>>>>>>> function.
>>>>>>>> java.lang.NullPointerException
>>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>>>>> exiting
>>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on
>>>>>>>> 61002
>>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>>>>>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>>>>>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>>>>>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>>>>>>>> java.lang.NullPointerException
>>>>>>>> java.lang.NullPointerException
>>>>>>>>
>>>>>>>> attempt_201304161357_0015_000001_0.log
>>>>>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>>>>> to
>>>>>>>> Zookeeper! At hadoop1/1.2.3.4:61001
>>>>>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>>>>>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
>>>>>>>> =
>>>>>>>> NoNode for /bsp/job_201304161357_0015/peers
>>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>>>>>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>>>>> address:hadoop1 port:61001
>>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001:
>>>>>>>> starting
>>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001:
>>>>>>>> starting
>>>>>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>>>>>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>>>>>>>> exception: java.io.EOFException
>>>>>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>>>>>>> at $Proxy3.put(Unknown Source)
>>>>>>>> at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>>>>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>>>>>>>> at
>>>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>>>>>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>>>>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>>>>>>> at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>>>>>>>> Caused by: java.io.EOFException
>>>>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>>>>> at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>>>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards, Edward J. Yoon
>>>>>>> @eddieyoon
>>>>>
>>>>>
>>>>>
>>>
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Why my distributed mode does not work?
Posted by "Edward J. Yoon" <ed...@apache.org>.
Yes, the repair function has deleted.
On Sat, Apr 20, 2013 at 7:26 PM, Lyu Xuedong <lx...@gmail.com> wrote:
> Hi, Edward,
>
> Thank you.
> I updated the version. There comes a new problem.
>
> In the /Apache Hama BSP Programming Model
> (http://people.apache.org/~tjungblut/downloads/hamadocs/ApacheHamaBSPProgrammingmodel_06.pdf)/
> there is a paragraph describes 'Graph repair' : "Hama requires a graph to be
> completed before feeding it to an algorithm. By complete we mean that every
> vertex that is referenced by an edge must somewhere be a vertex in the
> graph. In many cases of leafs this is not always the case, therefore we have
> added a repair functionality which is traversing the whole graph for leafs
> and adding them to the vertex structure to prevent algorithms from breaking
> with NullPointerExceptions when it does not find a referenced vertex. You
> can turn this feature on by setting it in your configuration like this:
> conf.setBoolean(GraphJobRunner.GRAPH_REPAIR, true);"
>
> I followed the guidance but got a hint 'GRAPH_REPAIR cannot be resolved or
> is not a field' programming in Eclipse. I read the source code later and
> there is really no variable named GRAPH_REPAIR in GraphJobRunner.java which
> exists in the version 0.5.0. Is this function not supported any longer ? How
> can I make a 'repaired graph'?
> Thank you.
>
>
> On 04/17/2013 12:51 PM, Edward J. Yoon wrote:
>>
>> Please use 0.6.1 and try your application with small data again.
>>
>> See also http://hama.apache.org/run_examples.html
>>
>> On Wed, Apr 17, 2013 at 10:27 AM, Lyu Xuedong <lx...@gmail.com> wrote:
>>>
>>> hama: 0.6.0
>>> hadoop : 1.0.4
>>> JDK : 1.6
>>> OS : ubuntu 12.04
>>>
>>>
>>> On 04/17/2013 05:47 AM, Edward J. Yoon wrote:
>>>>
>>>> Your version?
>>>>
>>>> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <lx...@gmail.com> wrote:
>>>>>
>>>>> Hi, Edward, thank you. But your suggestion seems do not work, do you
>>>>> have
>>>>> some other advice ?
>>>>>
>>>>> Is java.lang.NullPointerException related with heap size ?
>>>>>
>>>>>
>>>>>
>>>>> 在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
>>>>>
>>>>>> I guess you need to increase the child processor JVM heap size.
>>>>>>
>>>>>> - conf/hama-site.xml:
>>>>>>
>>>>>> <property>
>>>>>> <name>bsp.child.java.opts</name>
>>>>>> <value>-Xmx2048m</value>
>>>>>> </property>
>>>>>>
>>>>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> My project can work well in eclipse, but when I export it as a jar
>>>>>>> file
>>>>>>> and
>>>>>>> submit it to a two-node hama cluster, some errors and fatal occur if
>>>>>>> the
>>>>>>> input file is over 64M.
>>>>>>> I suspect that my configure files of hadoop or hama is somewhere not
>>>>>>> right,
>>>>>>> but a pi estimator can run normally on my cluster and
>>>>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a
>>>>>>> whole
>>>>>>> day,
>>>>>>> nothing improved.
>>>>>>> What in my input file are a large number of RDF triples: "<subject>
>>>>>>> <predicate> <object> ." My task is to create vertices for subjects
>>>>>>> and
>>>>>>> objects. Predicates are subjects' edges.
>>>>>>> What should I do ?
>>>>>>>
>>>>>>> Terminal output:
>>>>>>>
>>>>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>>>>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to
>>>>>>> process : 2
>>>>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job:
>>>>>>> job_201304161357_0015
>>>>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>>>>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>>>>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>>>>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>>>>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>>>>>>> hadoop1/1.2.3.4:61002
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> ipc.Server:
>>>>>>> Starting SocketReader
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> ipc.Server:
>>>>>>> IPC
>>>>>>> Server Responder: starting
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> ipc.Server:
>>>>>>> IPC
>>>>>>> Server handler 0 on 61002: starting
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>>> ipc.Server:
>>>>>>> IPC
>>>>>>> Server listener on 61002: starting
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR
>>>>>>> bsp.BSPTask:
>>>>>>> Error running bsp setup and bsp function.
>>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>> ipc.Server:
>>>>>>> Stopping server on 61002
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>> ipc.Server:
>>>>>>> IPC
>>>>>>> Server handler 0 on 61002: exiting
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>> ipc.Server:
>>>>>>> Stopping IPC Server listener on 61002
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>> ipc.Server:
>>>>>>> Stopping IPC Server Responder
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>>> metrics.RpcInstrumentation: shut down
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR
>>>>>>> bsp.BSPTask:
>>>>>>> Shutting down ping service.
>>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL
>>>>>>> bsp.GroomServer:
>>>>>>> Error running child
>>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>>>>>>
>>>>>>> tasklogs:
>>>>>>> attempt_201304161357_0015_000000_0.log
>>>>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>>>> to
>>>>>>> Zookeeper! At hadoop1/1.2.3.4:61002
>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>>>> starting
>>>>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>>>> address:hadoop1 port:61002
>>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002:
>>>>>>> starting
>>>>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>>>>>> function.
>>>>>>> java.lang.NullPointerException
>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>>>> exiting
>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on
>>>>>>> 61002
>>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>>>>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>>>>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>>>>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>>>>>>> java.lang.NullPointerException
>>>>>>> java.lang.NullPointerException
>>>>>>>
>>>>>>> attempt_201304161357_0015_000001_0.log
>>>>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>>>> to
>>>>>>> Zookeeper! At hadoop1/1.2.3.4:61001
>>>>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>>>>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
>>>>>>> =
>>>>>>> NoNode for /bsp/job_201304161357_0015/peers
>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>>>>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>>>> address:hadoop1 port:61001
>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001:
>>>>>>> starting
>>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001:
>>>>>>> starting
>>>>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>>>>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>>>>>>> exception: java.io.EOFException
>>>>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>>>>>> at $Proxy3.put(Unknown Source)
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>>>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>>>>>>> at
>>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>>>>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>>>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>>>>>>> Caused by: java.io.EOFException
>>>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards, Edward J. Yoon
>>>>>> @eddieyoon
>>>>
>>>>
>>>>
>>
>>
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Why my distributed mode does not work?
Posted by Lyu Xuedong <lx...@gmail.com>.
Hi, Edward,
Thank you.
I updated the version. There comes a new problem.
In the /Apache Hama BSP Programming Model
(http://people.apache.org/~tjungblut/downloads/hamadocs/ApacheHamaBSPProgrammingmodel_06.pdf)/
there is a paragraph describes 'Graph repair' : "Hama requires a graph
to be completed before feeding it to an algorithm. By complete we mean
that every vertex that is referenced by an edge must somewhere be a
vertex in the graph. In many cases of leafs this is not always the case,
therefore we have added a repair functionality which is traversing the
whole graph for leafs and adding them to the vertex structure to prevent
algorithms from breaking with NullPointerExceptions when it does not
find a referenced vertex. You can turn this feature on by setting it in
your configuration like this:
conf.setBoolean(GraphJobRunner.GRAPH_REPAIR, true);"
I followed the guidance but got a hint 'GRAPH_REPAIR cannot be resolved
or is not a field' programming in Eclipse. I read the source code later
and there is really no variable named GRAPH_REPAIR in
GraphJobRunner.java which exists in the version 0.5.0. Is this function
not supported any longer ? How can I make a 'repaired graph'?
Thank you.
On 04/17/2013 12:51 PM, Edward J. Yoon wrote:
> Please use 0.6.1 and try your application with small data again.
>
> See also http://hama.apache.org/run_examples.html
>
> On Wed, Apr 17, 2013 at 10:27 AM, Lyu Xuedong <lx...@gmail.com> wrote:
>> hama: 0.6.0
>> hadoop : 1.0.4
>> JDK : 1.6
>> OS : ubuntu 12.04
>>
>>
>> On 04/17/2013 05:47 AM, Edward J. Yoon wrote:
>>> Your version?
>>>
>>> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <lx...@gmail.com> wrote:
>>>> Hi, Edward, thank you. But your suggestion seems do not work, do you have
>>>> some other advice ?
>>>>
>>>> Is java.lang.NullPointerException related with heap size ?
>>>>
>>>>
>>>>
>>>> 在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
>>>>
>>>>> I guess you need to increase the child processor JVM heap size.
>>>>>
>>>>> - conf/hama-site.xml:
>>>>>
>>>>> <property>
>>>>> <name>bsp.child.java.opts</name>
>>>>> <value>-Xmx2048m</value>
>>>>> </property>
>>>>>
>>>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com>
>>>>> wrote:
>>>>>> My project can work well in eclipse, but when I export it as a jar file
>>>>>> and
>>>>>> submit it to a two-node hama cluster, some errors and fatal occur if
>>>>>> the
>>>>>> input file is over 64M.
>>>>>> I suspect that my configure files of hadoop or hama is somewhere not
>>>>>> right,
>>>>>> but a pi estimator can run normally on my cluster and
>>>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole
>>>>>> day,
>>>>>> nothing improved.
>>>>>> What in my input file are a large number of RDF triples: "<subject>
>>>>>> <predicate> <object> ." My task is to create vertices for subjects and
>>>>>> objects. Predicates are subjects' edges.
>>>>>> What should I do ?
>>>>>>
>>>>>> Terminal output:
>>>>>>
>>>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>>>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to
>>>>>> process : 2
>>>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job:
>>>>>> job_201304161357_0015
>>>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>>>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>>>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>>>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>>>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>>>>>> hadoop1/1.2.3.4:61002
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>>> Starting SocketReader
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>>> IPC
>>>>>> Server Responder: starting
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>>> IPC
>>>>>> Server handler 0 on 61002: starting
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>>> IPC
>>>>>> Server listener on 61002: starting
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR
>>>>>> bsp.BSPTask:
>>>>>> Error running bsp setup and bsp function.
>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>>> Stopping server on 61002
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>>> IPC
>>>>>> Server handler 0 on 61002: exiting
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>>> Stopping IPC Server listener on 61002
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>>> Stopping IPC Server Responder
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>>> metrics.RpcInstrumentation: shut down
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR
>>>>>> bsp.BSPTask:
>>>>>> Shutting down ping service.
>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL
>>>>>> bsp.GroomServer:
>>>>>> Error running child
>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>>>>>
>>>>>> tasklogs:
>>>>>> attempt_201304161357_0015_000000_0.log
>>>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>>> to
>>>>>> Zookeeper! At hadoop1/1.2.3.4:61002
>>>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>>> starting
>>>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>>> address:hadoop1 port:61002
>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002:
>>>>>> starting
>>>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>>>>> function.
>>>>>> java.lang.NullPointerException
>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>>>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>>> exiting
>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on
>>>>>> 61002
>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>>>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>>>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>>>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>>>>>> java.lang.NullPointerException
>>>>>> java.lang.NullPointerException
>>>>>>
>>>>>> attempt_201304161357_0015_000001_0.log
>>>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>>> to
>>>>>> Zookeeper! At hadoop1/1.2.3.4:61001
>>>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>>>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>>>>>> NoNode for /bsp/job_201304161357_0015/peers
>>>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>>>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>>> address:hadoop1 port:61001
>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001:
>>>>>> starting
>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001:
>>>>>> starting
>>>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>>>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>>>>>> exception: java.io.EOFException
>>>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>>>>> at $Proxy3.put(Unknown Source)
>>>>>> at
>>>>>>
>>>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>>>>>> at
>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>>>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>>>>> at
>>>>>>
>>>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>>>>>> Caused by: java.io.EOFException
>>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>>> @eddieyoon
>>>
>>>
>
>
Re: Why my distributed mode does not work?
Posted by "Edward J. Yoon" <ed...@apache.org>.
Please use 0.6.1 and try your application with small data again.
See also http://hama.apache.org/run_examples.html
On Wed, Apr 17, 2013 at 10:27 AM, Lyu Xuedong <lx...@gmail.com> wrote:
> hama: 0.6.0
> hadoop : 1.0.4
> JDK : 1.6
> OS : ubuntu 12.04
>
>
> On 04/17/2013 05:47 AM, Edward J. Yoon wrote:
>>
>> Your version?
>>
>> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <lx...@gmail.com> wrote:
>>>
>>> Hi, Edward, thank you. But your suggestion seems do not work, do you have
>>> some other advice ?
>>>
>>> Is java.lang.NullPointerException related with heap size ?
>>>
>>>
>>>
>>> 在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
>>>
>>>> I guess you need to increase the child processor JVM heap size.
>>>>
>>>> - conf/hama-site.xml:
>>>>
>>>> <property>
>>>> <name>bsp.child.java.opts</name>
>>>> <value>-Xmx2048m</value>
>>>> </property>
>>>>
>>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com>
>>>> wrote:
>>>>>
>>>>> My project can work well in eclipse, but when I export it as a jar file
>>>>> and
>>>>> submit it to a two-node hama cluster, some errors and fatal occur if
>>>>> the
>>>>> input file is over 64M.
>>>>> I suspect that my configure files of hadoop or hama is somewhere not
>>>>> right,
>>>>> but a pi estimator can run normally on my cluster and
>>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole
>>>>> day,
>>>>> nothing improved.
>>>>> What in my input file are a large number of RDF triples: "<subject>
>>>>> <predicate> <object> ." My task is to create vertices for subjects and
>>>>> objects. Predicates are subjects' edges.
>>>>> What should I do ?
>>>>>
>>>>> Terminal output:
>>>>>
>>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to
>>>>> process : 2
>>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job:
>>>>> job_201304161357_0015
>>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>>>>> hadoop1/1.2.3.4:61002
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>> Starting SocketReader
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>> IPC
>>>>> Server Responder: starting
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>> IPC
>>>>> Server handler 0 on 61002: starting
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>>> IPC
>>>>> Server listener on 61002: starting
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR
>>>>> bsp.BSPTask:
>>>>> Error running bsp setup and bsp function.
>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>> Stopping server on 61002
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>> IPC
>>>>> Server handler 0 on 61002: exiting
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>> Stopping IPC Server listener on 61002
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>>> Stopping IPC Server Responder
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>>> metrics.RpcInstrumentation: shut down
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR
>>>>> bsp.BSPTask:
>>>>> Shutting down ping service.
>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL
>>>>> bsp.GroomServer:
>>>>> Error running child
>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>>>>
>>>>> tasklogs:
>>>>> attempt_201304161357_0015_000000_0.log
>>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>> to
>>>>> Zookeeper! At hadoop1/1.2.3.4:61002
>>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>> starting
>>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>> address:hadoop1 port:61002
>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002:
>>>>> starting
>>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>>>> function.
>>>>> java.lang.NullPointerException
>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002:
>>>>> exiting
>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on
>>>>> 61002
>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>>>>> java.lang.NullPointerException
>>>>> java.lang.NullPointerException
>>>>>
>>>>> attempt_201304161357_0015_000001_0.log
>>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>>> to
>>>>> Zookeeper! At hadoop1/1.2.3.4:61001
>>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>>>>> NoNode for /bsp/job_201304161357_0015/peers
>>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>>> address:hadoop1 port:61001
>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001:
>>>>> starting
>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001:
>>>>> starting
>>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>>>>> exception: java.io.EOFException
>>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>>>> at $Proxy3.put(Unknown Source)
>>>>> at
>>>>>
>>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>>>>> at
>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>>>> at
>>>>>
>>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>>>>> Caused by: java.io.EOFException
>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon
>>>> @eddieyoon
>>
>>
>>
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Why my distributed mode does not work?
Posted by Lyu Xuedong <lx...@gmail.com>.
hama: 0.6.0
hadoop : 1.0.4
JDK : 1.6
OS : ubuntu 12.04
On 04/17/2013 05:47 AM, Edward J. Yoon wrote:
> Your version?
>
> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <lx...@gmail.com> wrote:
>> Hi, Edward, thank you. But your suggestion seems do not work, do you have some other advice ?
>>
>> Is java.lang.NullPointerException related with heap size ?
>>
>>
>>
>> 在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
>>
>>> I guess you need to increase the child processor JVM heap size.
>>>
>>> - conf/hama-site.xml:
>>>
>>> <property>
>>> <name>bsp.child.java.opts</name>
>>> <value>-Xmx2048m</value>
>>> </property>
>>>
>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com> wrote:
>>>> My project can work well in eclipse, but when I export it as a jar file and
>>>> submit it to a two-node hama cluster, some errors and fatal occur if the
>>>> input file is over 64M.
>>>> I suspect that my configure files of hadoop or hama is somewhere not right,
>>>> but a pi estimator can run normally on my cluster and
>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole day,
>>>> nothing improved.
>>>> What in my input file are a large number of RDF triples: "<subject>
>>>> <predicate> <object> ." My task is to create vertices for subjects and
>>>> objects. Predicates are subjects' edges.
>>>> What should I do ?
>>>>
>>>> Terminal output:
>>>>
>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2
>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015
>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>>>> hadoop1/1.2.3.4:61002
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>>> Starting SocketReader
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>>>> Server Responder: starting
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>>>> Server handler 0 on 61002: starting
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>>>> Server listener on 61002: starting
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask:
>>>> Error running bsp setup and bsp function.
>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>> Stopping server on 61002
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: IPC
>>>> Server handler 0 on 61002: exiting
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>> Stopping IPC Server listener on 61002
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>>> Stopping IPC Server Responder
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>>> metrics.RpcInstrumentation: shut down
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask:
>>>> Shutting down ping service.
>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL bsp.GroomServer:
>>>> Error running child
>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>>>
>>>> tasklogs:
>>>> attempt_201304161357_0015_000000_0.log
>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>>>> Zookeeper! At hadoop1/1.2.3.4:61002
>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting
>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>> address:hadoop1 port:61002
>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting
>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>>> function.
>>>> java.lang.NullPointerException
>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting
>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002
>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>>>> java.lang.NullPointerException
>>>> java.lang.NullPointerException
>>>>
>>>> attempt_201304161357_0015_000001_0.log
>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>>>> Zookeeper! At hadoop1/1.2.3.4:61001
>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>>>> NoNode for /bsp/job_201304161357_0015/peers
>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>>>> address:hadoop1 port:61001
>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting
>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting
>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>>>> exception: java.io.EOFException
>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>>> at $Proxy3.put(Unknown Source)
>>>> at
>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>>>> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>>> at
>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>>>> Caused by: java.io.EOFException
>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>> at
>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>
>
Re: Why my distributed mode does not work?
Posted by "Edward J. Yoon" <ed...@apache.org>.
Your version?
On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <lx...@gmail.com> wrote:
> Hi, Edward, thank you. But your suggestion seems do not work, do you have some other advice ?
>
> Is java.lang.NullPointerException related with heap size ?
>
>
>
> 在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
>
>> I guess you need to increase the child processor JVM heap size.
>>
>> - conf/hama-site.xml:
>>
>> <property>
>> <name>bsp.child.java.opts</name>
>> <value>-Xmx2048m</value>
>> </property>
>>
>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com> wrote:
>>> My project can work well in eclipse, but when I export it as a jar file and
>>> submit it to a two-node hama cluster, some errors and fatal occur if the
>>> input file is over 64M.
>>> I suspect that my configure files of hadoop or hama is somewhere not right,
>>> but a pi estimator can run normally on my cluster and
>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole day,
>>> nothing improved.
>>> What in my input file are a large number of RDF triples: "<subject>
>>> <predicate> <object> ." My task is to create vertices for subjects and
>>> objects. Predicates are subjects' edges.
>>> What should I do ?
>>>
>>> Terminal output:
>>>
>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2
>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015
>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>>> hadoop1/1.2.3.4:61002
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>>> Starting SocketReader
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>>> Server Responder: starting
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>>> Server handler 0 on 61002: starting
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>>> Server listener on 61002: starting
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask:
>>> Error running bsp setup and bsp function.
>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>> Stopping server on 61002
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: IPC
>>> Server handler 0 on 61002: exiting
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>> Stopping IPC Server listener on 61002
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>>> Stopping IPC Server Responder
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>>> metrics.RpcInstrumentation: shut down
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask:
>>> Shutting down ping service.
>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL bsp.GroomServer:
>>> Error running child
>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>>
>>> tasklogs:
>>> attempt_201304161357_0015_000000_0.log
>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>>> Zookeeper! At hadoop1/1.2.3.4:61002
>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting
>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>>> address:hadoop1 port:61002
>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting
>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>> function.
>>> java.lang.NullPointerException
>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting
>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002
>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>>> java.lang.NullPointerException
>>> java.lang.NullPointerException
>>>
>>> attempt_201304161357_0015_000001_0.log
>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>>> Zookeeper! At hadoop1/1.2.3.4:61001
>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>>> NoNode for /bsp/job_201304161357_0015/peers
>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>>> address:hadoop1 port:61001
>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting
>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting
>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>>> exception: java.io.EOFException
>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>> at $Proxy3.put(Unknown Source)
>>> at
>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>>> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>> at
>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>>> Caused by: java.io.EOFException
>>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>> at
>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Why my distributed mode does not work?
Posted by Lvxuedong <lx...@gmail.com>.
Hi, Edward, thank you. But your suggestion seems do not work, do you have some other advice ?
Is java.lang.NullPointerException related with heap size ?
在 2013-4-16，22:13，"Edward J. Yoon" <ed...@apache.org> 写道：
> I guess you need to increase the child processor JVM heap size.
>
> - conf/hama-site.xml:
>
> <property>
> <name>bsp.child.java.opts</name>
> <value>-Xmx2048m</value>
> </property>
>
> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com> wrote:
>> My project can work well in eclipse, but when I export it as a jar file and
>> submit it to a two-node hama cluster, some errors and fatal occur if the
>> input file is over 64M.
>> I suspect that my configure files of hadoop or hama is somewhere not right,
>> but a pi estimator can run normally on my cluster and
>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole day,
>> nothing improved.
>> What in my input file are a large number of RDF triples: "<subject>
>> <predicate> <object> ." My task is to create vertices for subjects and
>> objects. Predicates are subjects' edges.
>> What should I do ?
>>
>> Terminal output:
>>
>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2
>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015
>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>> sync.ZKSyncClient: Initializing ZK Sync Client
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
>> hadoop1/1.2.3.4:61002
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
>> Starting SocketReader
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>> Server Responder: starting
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>> Server handler 0 on 61002: starting
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
>> Server listener on 61002: starting
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask:
>> Error running bsp setup and bsp function.
>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>> Stopping server on 61002
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: IPC
>> Server handler 0 on 61002: exiting
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>> Stopping IPC Server listener on 61002
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
>> Stopping IPC Server Responder
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
>> metrics.RpcInstrumentation: shut down
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask:
>> Shutting down ping service.
>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL bsp.GroomServer:
>> Error running child
>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>>
>> tasklogs:
>> attempt_201304161357_0015_000000_0.log
>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>> Zookeeper! At hadoop1/1.2.3.4:61002
>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting
>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
>> address:hadoop1 port:61002
>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting
>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
>> function.
>> java.lang.NullPointerException
>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting
>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002
>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
>> java.lang.NullPointerException
>> java.lang.NullPointerException
>>
>> attempt_201304161357_0015_000001_0.log
>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>> Zookeeper! At hadoop1/1.2.3.4:61001
>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for /bsp/job_201304161357_0015/peers
>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
>> address:hadoop1 port:61001
>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting
>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting
>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
>> exception: java.io.EOFException
>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>> at $Proxy3.put(Unknown Source)
>> at
>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
>> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
>> Caused by: java.io.EOFException
>> at java.io.DataInputStream.readInt(DataInputStream.java:375)
>> at
>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
Re: Why my distributed mode does not work?
Posted by "Edward J. Yoon" <ed...@apache.org>.
I guess you need to increase the child processor JVM heap size.
- conf/hama-site.xml:
<property>
<name>bsp.child.java.opts</name>
<value>-Xmx2048m</value>
</property>
On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <lx...@gmail.com> wrote:
> My project can work well in eclipse, but when I export it as a jar file and
> submit it to a two-node hama cluster, some errors and fatal occur if the
> input file is over 64M.
> I suspect that my configure files of hadoop or hama is somewhere not right,
> but a pi estimator can run normally on my cluster and
> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole day,
> nothing improved.
> What in my input file are a large number of RDF triples: "<subject>
> <predicate> <object> ." My task is to create vertices for subjects and
> objects. Predicates are subjects' edges.
> What should I do ?
>
> Terminal output:
>
> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2
> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015
> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
> sync.ZKSyncClient: Initializing ZK Sync Client
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
> hadoop1/1.2.3.4:61002
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server:
> Starting SocketReader
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
> Server Responder: starting
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
> Server handler 0 on 61002: starting
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO
> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002
> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC
> Server listener on 61002: starting
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask:
> Error running bsp setup and bsp function.
> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
> Stopping server on 61002
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: IPC
> Server handler 0 on 61002: exiting
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
> Stopping IPC Server listener on 61002
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server:
> Stopping IPC Server Responder
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO
> metrics.RpcInstrumentation: shut down
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask:
> Shutting down ping service.
> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL bsp.GroomServer:
> Error running child
> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
> attempt_201304161357_0015_000000_0: java.lang.NullPointerException
> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.
>
> tasklogs:
> attempt_201304161357_0015_000000_0.log
> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
> Zookeeper! At hadoop1/1.2.3.4:61002
> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting
> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer
> address:hadoop1 port:61002
> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting
> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp
> function.
> java.lang.NullPointerException
> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting
> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002
> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
> java.lang.NullPointerException
> java.lang.NullPointerException
>
> attempt_201304161357_0015_000001_0.log
> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
> Zookeeper! At hadoop1/1.2.3.4:61001
> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /bsp/job_201304161357_0015/peers
> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer
> address:hadoop1 port:61001
> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting
> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting
> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local
> exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
> at org.apache.hadoop.ipc.Client.call(Client.java:1071)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
> at $Proxy3.put(Unknown Source)
> at
> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> at
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>
>
>
>
>
>
>
>
>
--
Best Regards, Edward J. Yoon
@eddieyoon