You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Artie Pesh-Imam <ar...@tapad.com> on 2013/11/07 22:25:27 UTC
Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
Re: Null Pointer Exception when running a job
Posted by Sertuğ Kaya <se...@agmlab.com>.
Hello Artie,
Sorry for late response, can you give more details about your hadoop
cluster, how many datanodes, memory of nodes, map task capacity etc?
Since you can make this work with a small dataset, it should be a
problem about scaling up.
On 19-11-2013 19:51, Artie Pesh-Imam wrote:
> Is there no other suggestions on what to do here? I really don’t have
> any idea about what’s going on? I actually ran this against a single
> extract from our graph.
>
> Here are some more logs.
>
> Again this is running giraph 1.0 against hadoop 2.
> Thanks,
> 2013-11-19 17:06:04,097 WARN org.apache.hadoop.mapred.Child: Error running child
> java.lang.IllegalStateException: run: Caught an unrecoverable exception null
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262)
> Caused by: java.lang.NullPointerException
> at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
> at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
> at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
> at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
> at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
> at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
> at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
> --
> Artie Pesh-Imam
> Software Engineer @ Tapad
> 201-253-9647
> ------------------------------------------------------------------------
> From: Artie Pesh-Imam Artie Pesh-Imam <ma...@tapad.com>
> Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
> <ma...@tapad.com>
> Date: November 11, 2013 at 10:55:54 AM
> To: Sertuğ Kaya sertug.kaya@agmlab.com
> <ma...@agmlab.com>, Mohammad Islam mislam77@yahoo.com
> <ma...@yahoo.com>, user@giraph.apache.org
> user@giraph.apache.org <ma...@giraph.apache.org>
> Subject: Re: Null Pointer Exception when running a job
>> I can certainly try this. Are there any other configurations which
>> are worth exploring?
>>
>>
>> --
>> Artie Pesh-Imam
>> Software Engineer @ Tapad
>> 201-253-9647
>> ------------------------------------------------------------------------
>> From: Sertuğ Kaya Sertuğ Kaya <ma...@agmlab.com>
>> Reply: user@giraph.apache.org user@giraph.apache.org
>> <ma...@giraph.apache.org>
>> Date: November 9, 2013 at 3:58:39 PM
>> To: user@giraph.apache.org user@giraph.apache.org
>> <ma...@giraph.apache.org>, Mohammad Islam mislam77@yahoo.com
>> <ma...@yahoo.com>
>> Subject: Re: Null Pointer Exception when running a job
>>> Hello Artie;
>>> I also had such problems when working on large datasets. It probably
>>> needs a better hadoop/mapreduce configuration. For example did you
>>> try increasing the memory of task tracker child processes in
>>> mapred-site.xml such as:
>>>> <property>
>>>> <name>mapred.child.java.opts</name>
>>>> <value>-Xmx3000m</value>
>>>> </property>
>>> Regards;
>>> Sertug
>>>
>>> On 08-11-2013 01:26, Artie Pesh-Imam wrote:
>>>> Hey
>>>>
>>>> It does work against a smaller data set and Im running 1.0.0 for
>>>> hadoop 2.
>>>>
>>>>
>>>> --
>>>> Artie Pesh-Imam
>>>> Software Engineer @ Tapad
>>>> 201-253-9647
>>>> ------------------------------------------------------------------------
>>>> From: Mohammad Islam Mohammad Islam <ma...@yahoo.com>
>>>> Reply: user@giraph.apache.org user@giraph.apache.org
>>>> <ma...@giraph.apache.org>, Mohammad Islam mislam77@yahoo.com
>>>> <ma...@yahoo.com>
>>>> Date: November 7, 2013 at 6:20:53 PM
>>>> To: user@giraph.apache.org user@giraph.apache.org
>>>> <ma...@giraph.apache.org>
>>>> Subject: Re: Null Pointer Exception when running a job
>>>>> Hi Artie,
>>>>> Hopefully someone would give you a concrete answer.
>>>>>
>>>>> My rambling comments:
>>>>> Did you try with a smaller number of vertices? I want to know if
>>>>> the problem is due to increased load.
>>>>> What version of Giraph are you using?
>>>>>
>>>>> Regards,
>>>>> Mohammad
>>>>>
>>>>>
>>>>> On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam
>>>>> <ar...@tapad.com> wrote:
>>>>> Please let me know if any additional context would be useful. With
>>>>> no detail in the error message, I have no idea what the root issue
>>>>> could be.
>>>>>
>>>>> --
>>>>> Artie Pesh-Imam
>>>>> Software Engineer @ Tapad
>>>>> 201-253-9647
>>>>> ------------------------------------------------------------------------
>>>>> From: Artie Pesh-Imam Artie Pesh-Imam
>>>>> <ma...@tapad.com>
>>>>> Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
>>>>> <ma...@tapad.com>
>>>>> Date: November 7, 2013 at 4:25:30 PM
>>>>> To: user@giraph.apache.org user@giraph.apache.org
>>>>> <ma...@giraph.apache.org>
>>>>> Subject: Null Pointer Exception when running a job
>>>>> Hi all,
>>>>>
>>>>> Im running a simple connected components algorithm against a graph
>>>>> of almost 200 million vertices.
>>>>>
>>>>> I have pretty much no guess on what this issue could be:
>>>>> java.lang.IllegalStateException: run: Caught an unrecoverable exception null
>>>>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>>>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>>>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>>>>> at org.apache.hadoop.mapred.Child.main(Child.java:262)
>>>>> Caused by: java.lang.NullPointerException
>>>>> at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
>>>>> at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
>>>>> at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
>>>>> at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
>>>>> at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
>>>>> at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
>>>>> at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
>>>>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
>>>>>
>>>>> --
>>>>> Artie Pesh-Imam
>>>>> Software Engineer @ Tapad
>>>>> 201-253-9647
>>>>>
>>>>>
>>>
Re: Null Pointer Exception when running a job
Posted by Artie Pesh-Imam <ar...@tapad.com>.
Is there no other suggestions on what to do here? I really don’t have any idea about what’s going on? I actually ran this against a single extract from our graph.
Here are some more logs.
Again this is running giraph 1.0 against hadoop 2.
Thanks,
2013-11-19 17:06:04,097 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 11, 2013 at 10:55:54 AM
To: Sertuğ Kaya sertug.kaya@agmlab.com, Mohammad Islam mislam77@yahoo.com, user@giraph.apache.org user@giraph.apache.org
Subject: Re: Null Pointer Exception when running a job
I can certainly try this. Are there any other configurations which are worth exploring?
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Sertuğ Kaya Sertuğ Kaya
Reply: user@giraph.apache.org user@giraph.apache.org
Date: November 9, 2013 at 3:58:39 PM
To: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Subject: Re: Null Pointer Exception when running a job
Hello Artie;
I also had such problems when working on large datasets. It probably needs a better hadoop/mapreduce configuration. For example did you try increasing the memory of task tracker child processes in mapred-site.xml such as:
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx3000m</value>
</property>
Regards;
Sertug
On 08-11-2013 01:26, Artie Pesh-Imam wrote:
Hey
It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Mohammad Islam Mohammad Islam
Reply: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Date: November 7, 2013 at 6:20:53 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Re: Null Pointer Exception when running a job
Hi Artie,
Hopefully someone would give you a concrete answer.
My rambling comments:
Did you try with a smaller number of vertices? I want to know if the problem is due to increased load.
What version of Giraph are you using?
Regards,
Mohammad
On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam <ar...@tapad.com> wrote:
Please let me know if any additional context would be useful. With no detail in the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
Re: Null Pointer Exception when running a job
Posted by Artie Pesh-Imam <ar...@tapad.com>.
I can certainly try this. Are there any other configurations which are worth exploring?
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Sertuğ Kaya Sertuğ Kaya
Reply: user@giraph.apache.org user@giraph.apache.org
Date: November 9, 2013 at 3:58:39 PM
To: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Subject: Re: Null Pointer Exception when running a job
Hello Artie;
I also had such problems when working on large datasets. It probably needs a better hadoop/mapreduce configuration. For example did you try increasing the memory of task tracker child processes in mapred-site.xml such as:
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx3000m</value>
</property>
Regards;
Sertug
On 08-11-2013 01:26, Artie Pesh-Imam wrote:
Hey
It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Mohammad Islam Mohammad Islam
Reply: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Date: November 7, 2013 at 6:20:53 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Re: Null Pointer Exception when running a job
Hi Artie,
Hopefully someone would give you a concrete answer.
My rambling comments:
Did you try with a smaller number of vertices? I want to know if the problem is due to increased load.
What version of Giraph are you using?
Regards,
Mohammad
On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam <ar...@tapad.com> wrote:
Please let me know if any additional context would be useful. With no detail in the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
Re: Null Pointer Exception when running a job
Posted by Sertuğ Kaya <se...@agmlab.com>.
Hello Artie;
I also had such problems when working on large datasets. It probably
needs a better hadoop/mapreduce configuration. For example did you try
increasing the memory of task tracker child processes in mapred-site.xml
such as:
> <property>
> <name>mapred.child.java.opts</name>
> <value>-Xmx3000m</value>
> </property>
Regards;
Sertug
On 08-11-2013 01:26, Artie Pesh-Imam wrote:
> Hey
>
> It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
>
>
> --
> Artie Pesh-Imam
> Software Engineer @ Tapad
> 201-253-9647
> ------------------------------------------------------------------------
> From: Mohammad Islam Mohammad Islam <ma...@yahoo.com>
> Reply: user@giraph.apache.org user@giraph.apache.org
> <ma...@giraph.apache.org>, Mohammad Islam mislam77@yahoo.com
> <ma...@yahoo.com>
> Date: November 7, 2013 at 6:20:53 PM
> To: user@giraph.apache.org user@giraph.apache.org
> <ma...@giraph.apache.org>
> Subject: Re: Null Pointer Exception when running a job
>> Hi Artie,
>> Hopefully someone would give you a concrete answer.
>>
>> My rambling comments:
>> Did you try with a smaller number of vertices? I want to know if the
>> problem is due to increased load.
>> What version of Giraph are you using?
>>
>> Regards,
>> Mohammad
>>
>>
>> On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam
>> <ar...@tapad.com> wrote:
>> Please let me know if any additional context would be useful. With no
>> detail in the error message, I have no idea what the root issue could be.
>>
>> --
>> Artie Pesh-Imam
>> Software Engineer @ Tapad
>> 201-253-9647
>> ------------------------------------------------------------------------
>> From: Artie Pesh-Imam Artie Pesh-Imam <ma...@tapad.com>
>> Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
>> <ma...@tapad.com>
>> Date: November 7, 2013 at 4:25:30 PM
>> To: user@giraph.apache.org user@giraph.apache.org
>> <ma...@giraph.apache.org>
>> Subject: Null Pointer Exception when running a job
>> Hi all,
>>
>> Im running a simple connected components algorithm against a graph of
>> almost 200 million vertices.
>>
>> I have pretty much no guess on what this issue could be:
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception null
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>> at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> Caused by: java.lang.NullPointerException
>> at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
>> at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
>> at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
>> at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
>> at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
>> at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
>> at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
>>
>>
>> --
>> Artie Pesh-Imam
>> Software Engineer @ Tapad
>> 201-253-9647
>>
>>
Re: Null Pointer Exception when running a job
Posted by Artie Pesh-Imam <ar...@tapad.com>.
Hey
It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Mohammad Islam Mohammad Islam
Reply: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Date: November 7, 2013 at 6:20:53 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Re: Null Pointer Exception when running a job
Hi Artie,
Hopefully someone would give you a concrete answer.
My rambling comments:
Did you try with a smaller number of vertices? I want to know if the problem is due to increased load.
What version of Giraph are you using?
Regards,
Mohammad
On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam <ar...@tapad.com> wrote:
Please let me know if any additional context would be useful. With no detail in the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
Re: Null Pointer Exception when running a job
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Artie,
Hopefully someone would give you a concrete answer.
My rambling comments:
Did you try with a smaller number of vertices? I want to know if the problem is due to increased load.
What version of Giraph are you using?
Regards,
Mohammad
On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam <ar...@tapad.com> wrote:
Please let me know if any additional context would be useful. With no detail in the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
________________________________
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph
of almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157) at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171) at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410) at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316) at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
Re: Null Pointer Exception when running a job
Posted by Artie Pesh-Imam <ar...@tapad.com>.
Please let me know if any additional context would be useful. With no detail in the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647