You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Jyoti Yadav <ra...@gmail.com> on 2013/11/20 08:42:25 UTC
Need your help to serialize an boolean array....
Hi folks..
I am implementing one program where I need to pass message as boolean
array..
While implementing my MyMessageWritable.java class,I need to define
readfields() and write() functions...
I tried my luck but failed...Program is compiling fine but not running...
I am taking boolean array as
boolean[] bitstring=new boolean[128];
pls help...I am badly tangled...
Thanks
Jyoti
Re: Need your help to serialize an boolean array....
Posted by Jyoti Yadav <ra...@gmail.com>.
Thanks a lot for fixing the problem...Now its working fine..my silly
mistake troubled me a lot..
Thanks once again for your ideas..:)
On Thu, Nov 21, 2013 at 2:46 AM, Claudio Martella <
claudio.martella@gmail.com> wrote:
> I think your problem is the fact that you instance the boolean[] only in
> the parametrized constructor. In the readFields method you should make sure
> that the array has been allocated. if not you should instance an array of
> length "size", as you just read from the byte stream.
>
>
> On Wed, Nov 20, 2013 at 12:25 PM, Jyoti Yadav <ra...@gmail.com>wrote:
>
>> 2013-11-20 15:46:09,202 ERROR
>> org.apache.giraph.utils.LogStacktraceCallable: Execution of callable failed
>>
>> *java.lang.NullPointerException at
>> org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)*
>> at
>> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
>> at
>> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
>> at
>> org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
>> at
>> org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
>> at
>> org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
>> at
>> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
>> at
>> org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
>> at
>> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
>> at
>> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
>> at
>> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
>> at
>> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:724)
>> 2013-11-20 15:46:09,204 ERROR org.apache.giraph.worker.BspServiceWorker:
>> unregisterHealth: Got failure, unregistering health on
>> /_hadoopBsp/job_201311200901_0018/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/kanha-Vostro-1014_1
>> on superstep 0
>> 2013-11-20 15:46:09,261 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
>> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
>> 2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO:
>> Initialized cache for UID to User mapping with a cache timeout of 14400
>> seconds.
>> 2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO: Got
>> UserName hduser for UID 1001 from the native implementation
>> 2013-11-20 15:46:09,351 WARN org.apache.hadoop.mapred.Child: Error
>> running child
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception
>> waitFor: ExecutionException occurred while waiting for
>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>> at org.apache.hadoop.mapred.Child.main(Child.java:253)
>> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
>> occurred while waiting for
>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
>> at
>> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181)
>> at
>> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:139)
>> at
>> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:124)
>> at
>> org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:87)
>> at
>> org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:221)
>> at
>> org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:741)
>> at
>> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:286)
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
>> ... 7 more
>> Caused by: java.util.concurrent.ExecutionException:
>> java.lang.NullPointerException
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
>> at
>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:300)
>> at
>> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173)
>> ... 14 more
>> Caused by: java.lang.NullPointerException
>> at
>> org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)
>> at
>> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
>> at
>> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
>> at
>> org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
>> at
>> org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
>> at
>> org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
>> at
>> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
>> at
>> org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
>> at
>> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
>> at
>> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
>> at
>> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
>> at
>> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:724)
>> 2013-11-20 15:46:09,356 INFO org.apache.hadoop.mapred.Task: Runnning
>> cleanup for the task
>> 2013-11-20 15:46:09,408 WARN org.apache.hadoop.conf.Configuration:
>> /app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
>> attempt to override final parameter: dfs.data.dir; Ignoring.
>> 2013-11-20 15:46:09,412 WARN org.apache.hadoop.conf.Configuration:
>> /app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
>> attempt to override final parameter: dfs.name.dir; Ignoring.
>>
>>
>> On Wed, Nov 20, 2013 at 4:52 PM, Sebastian Schelter <
>> ssc.open@googlemail.com> wrote:
>>
>>> What is the exact error message?
>>>
>>> On 20.11.2013 12:11, Jyoti Yadav wrote:
>>> > Thanks Sir for your suggestions...
>>> > I looked into the log files...
>>> > It is showing some error at readFields() method in
>>> > ClosenessMessageWritable.java file..You please have a look again
>>> whether
>>> > boolean array is serialized properly or not...
>>> >
>>> > I am doubtful about these serialization functions..
>>> >
>>> > Thanks..
>>> >
>>> >
>>> > On Wed, Nov 20, 2013 at 4:16 PM, Sebastian Schelter <
>>> ssc.open@googlemail.com
>>> >> wrote:
>>> >
>>> >> It says "Failed map tasks=1", you should have a deeper look into the
>>> >> logfiles or into the web console of Hadoop to find out why the map
>>> task
>>> >> fails.
>>> >>
>>> >> On 20.11.2013 11:44, Jyoti Yadav wrote:
>>> >>> 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
>>> >>
>>> >>
>>> >
>>>
>>>
>>
>
>
> --
> Claudio Martella
> claudio.martella@gmail.com
>
Re: Need your help to serialize an boolean array....
Posted by Claudio Martella <cl...@gmail.com>.
I think your problem is the fact that you instance the boolean[] only in
the parametrized constructor. In the readFields method you should make sure
that the array has been allocated. if not you should instance an array of
length "size", as you just read from the byte stream.
On Wed, Nov 20, 2013 at 12:25 PM, Jyoti Yadav <ra...@gmail.com>wrote:
> 2013-11-20 15:46:09,202 ERROR
> org.apache.giraph.utils.LogStacktraceCallable: Execution of callable failed
>
> *java.lang.NullPointerException at
> org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)*
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
> at
> org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
> at
> org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
> at
> org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
> at
> org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
> at
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> 2013-11-20 15:46:09,204 ERROR org.apache.giraph.worker.BspServiceWorker:
> unregisterHealth: Got failure, unregistering health on
> /_hadoopBsp/job_201311200901_0018/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/kanha-Vostro-1014_1
> on superstep 0
> 2013-11-20 15:46:09,261 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO:
> Initialized cache for UID to User mapping with a cache timeout of 14400
> seconds.
> 2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO: Got
> UserName hduser for UID 1001 from the native implementation
> 2013-11-20 15:46:09,351 WARN org.apache.hadoop.mapred.Child: Error running
> child
> java.lang.IllegalStateException: run: Caught an unrecoverable exception
> waitFor: ExecutionException occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
> occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
> at
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181)
> at
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:139)
> at
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:124)
> at
> org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:87)
> at
> org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:221)
> at
> org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:741)
> at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:286)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
> ... 7 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
> at
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:300)
> at
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173)
> ... 14 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
> at
> org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
> at
> org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
> at
> org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
> at
> org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
> at
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> 2013-11-20 15:46:09,356 INFO org.apache.hadoop.mapred.Task: Runnning
> cleanup for the task
> 2013-11-20 15:46:09,408 WARN org.apache.hadoop.conf.Configuration:
> /app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
> attempt to override final parameter: dfs.data.dir; Ignoring.
> 2013-11-20 15:46:09,412 WARN org.apache.hadoop.conf.Configuration:
> /app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
> attempt to override final parameter: dfs.name.dir; Ignoring.
>
>
> On Wed, Nov 20, 2013 at 4:52 PM, Sebastian Schelter <
> ssc.open@googlemail.com> wrote:
>
>> What is the exact error message?
>>
>> On 20.11.2013 12:11, Jyoti Yadav wrote:
>> > Thanks Sir for your suggestions...
>> > I looked into the log files...
>> > It is showing some error at readFields() method in
>> > ClosenessMessageWritable.java file..You please have a look again whether
>> > boolean array is serialized properly or not...
>> >
>> > I am doubtful about these serialization functions..
>> >
>> > Thanks..
>> >
>> >
>> > On Wed, Nov 20, 2013 at 4:16 PM, Sebastian Schelter <
>> ssc.open@googlemail.com
>> >> wrote:
>> >
>> >> It says "Failed map tasks=1", you should have a deeper look into the
>> >> logfiles or into the web console of Hadoop to find out why the map task
>> >> fails.
>> >>
>> >> On 20.11.2013 11:44, Jyoti Yadav wrote:
>> >>> 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
>> >>
>> >>
>> >
>>
>>
>
--
Claudio Martella
claudio.martella@gmail.com
Re: Need your help to serialize an boolean array....
Posted by Ameya Vilankar <am...@gmail.com>.
Hi,
Can you please confirm that the line number 45 in ClosenessMessageWritable
is as follows:
this.previous_bitstring[i] = in.readBoolean();
If its then try the following. Hopefully it will fix your problem
@Override
public void readFields(DataInput in) throws IOException {
size=in.readInt();
previous_bitstring = new boolean[size];
for(int i=0;i<size;i++)
{
previous_bitstring[i] = DataInput.readBoolean();
}
}
On Nov 20, 2013 3:26 AM, "Jyoti Yadav" <ra...@gmail.com> wrote:
> 2013-11-20 15:46:09,202 ERROR
> org.apache.giraph.utils.LogStacktraceCallable: Execution of callable failed
>
> *java.lang.NullPointerException at
> org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)*
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
> at
> org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
> at
> org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
> at
> org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
> at
> org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
> at
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> 2013-11-20 15:46:09,204 ERROR org.apache.giraph.worker.BspServiceWorker:
> unregisterHealth: Got failure, unregistering health on
> /_hadoopBsp/job_201311200901_0018/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/kanha-Vostro-1014_1
> on superstep 0
> 2013-11-20 15:46:09,261 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO:
> Initialized cache for UID to User mapping with a cache timeout of 14400
> seconds.
> 2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO: Got
> UserName hduser for UID 1001 from the native implementation
> 2013-11-20 15:46:09,351 WARN org.apache.hadoop.mapred.Child: Error running
> child
> java.lang.IllegalStateException: run: Caught an unrecoverable exception
> waitFor: ExecutionException occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
> occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
> at
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181)
> at
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:139)
> at
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:124)
> at
> org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:87)
> at
> org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:221)
> at
> org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:741)
> at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:286)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
> ... 7 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
> at
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:300)
> at
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173)
> ... 14 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
> at
> org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
> at
> org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
> at
> org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
> at
> org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
> at
> org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
> at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
> at
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> 2013-11-20 15:46:09,356 INFO org.apache.hadoop.mapred.Task: Runnning
> cleanup for the task
> 2013-11-20 15:46:09,408 WARN org.apache.hadoop.conf.Configuration:
> /app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
> attempt to override final parameter: dfs.data.dir; Ignoring.
> 2013-11-20 15:46:09,412 WARN org.apache.hadoop.conf.Configuration:
> /app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
> attempt to override final parameter: dfs.name.dir; Ignoring.
>
>
> On Wed, Nov 20, 2013 at 4:52 PM, Sebastian Schelter <
> ssc.open@googlemail.com> wrote:
>
>> What is the exact error message?
>>
>> On 20.11.2013 12:11, Jyoti Yadav wrote:
>> > Thanks Sir for your suggestions...
>> > I looked into the log files...
>> > It is showing some error at readFields() method in
>> > ClosenessMessageWritable.java file..You please have a look again whether
>> > boolean array is serialized properly or not...
>> >
>> > I am doubtful about these serialization functions..
>> >
>> > Thanks..
>> >
>> >
>> > On Wed, Nov 20, 2013 at 4:16 PM, Sebastian Schelter <
>> ssc.open@googlemail.com
>> >> wrote:
>> >
>> >> It says "Failed map tasks=1", you should have a deeper look into the
>> >> logfiles or into the web console of Hadoop to find out why the map task
>> >> fails.
>> >>
>> >> On 20.11.2013 11:44, Jyoti Yadav wrote:
>> >>> 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
>> >>
>> >>
>> >
>>
>>
>
Re: Need your help to serialize an boolean array....
Posted by Jyoti Yadav <ra...@gmail.com>.
2013-11-20 15:46:09,202 ERROR
org.apache.giraph.utils.LogStacktraceCallable: Execution of callable failed
*java.lang.NullPointerException at
org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)*
at
org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
at
org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
at
org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
at
org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
at
org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
at
org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
2013-11-20 15:46:09,204 ERROR org.apache.giraph.worker.BspServiceWorker:
unregisterHealth: Got failure, unregistering health on
/_hadoopBsp/job_201311200901_0018/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/kanha-Vostro-1014_1
on superstep 0
2013-11-20 15:46:09,261 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO:
Initialized cache for UID to User mapping with a cache timeout of 14400
seconds.
2013-11-20 15:46:09,348 INFO org.apache.hadoop.io.nativeio.NativeIO: Got
UserName hduser for UID 1001 from the native implementation
2013-11-20 15:46:09,351 WARN org.apache.hadoop.mapred.Child: Error running
child
java.lang.IllegalStateException: run: Caught an unrecoverable exception
waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@16ea72c
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:139)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:124)
at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:87)
at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:221)
at
org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:741)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:286)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
... 7 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:300)
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.giraph.examples.ClosenessMessageWritable.readFields(ClosenessMessageWritable.java:45)
at
org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:77)
at
org.apache.giraph.utils.ByteArrayVertexIdMessages.readData(ByteArrayVertexIdMessages.java:34)
at
org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:221)
at
org.apache.giraph.comm.messages.primitives.LongByteArrayMessageStore.addPartitionMessages(LongByteArrayMessageStore.java:149)
at
org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:73)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
at
org.apache.giraph.comm.SendMessageCache.flush(SendMessageCache.java:247)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:415)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:203)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
2013-11-20 15:46:09,356 INFO org.apache.hadoop.mapred.Task: Runnning
cleanup for the task
2013-11-20 15:46:09,408 WARN org.apache.hadoop.conf.Configuration:
/app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
attempt to override final parameter: dfs.data.dir; Ignoring.
2013-11-20 15:46:09,412 WARN org.apache.hadoop.conf.Configuration:
/app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201311200901_0018/job.xml:a
attempt to override final parameter: dfs.name.dir; Ignoring.
On Wed, Nov 20, 2013 at 4:52 PM, Sebastian Schelter <ssc.open@googlemail.com
> wrote:
> What is the exact error message?
>
> On 20.11.2013 12:11, Jyoti Yadav wrote:
> > Thanks Sir for your suggestions...
> > I looked into the log files...
> > It is showing some error at readFields() method in
> > ClosenessMessageWritable.java file..You please have a look again whether
> > boolean array is serialized properly or not...
> >
> > I am doubtful about these serialization functions..
> >
> > Thanks..
> >
> >
> > On Wed, Nov 20, 2013 at 4:16 PM, Sebastian Schelter <
> ssc.open@googlemail.com
> >> wrote:
> >
> >> It says "Failed map tasks=1", you should have a deeper look into the
> >> logfiles or into the web console of Hadoop to find out why the map task
> >> fails.
> >>
> >> On 20.11.2013 11:44, Jyoti Yadav wrote:
> >>> 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
> >>
> >>
> >
>
>
Re: Need your help to serialize an boolean array....
Posted by Sebastian Schelter <ss...@googlemail.com>.
What is the exact error message?
On 20.11.2013 12:11, Jyoti Yadav wrote:
> Thanks Sir for your suggestions...
> I looked into the log files...
> It is showing some error at readFields() method in
> ClosenessMessageWritable.java file..You please have a look again whether
> boolean array is serialized properly or not...
>
> I am doubtful about these serialization functions..
>
> Thanks..
>
>
> On Wed, Nov 20, 2013 at 4:16 PM, Sebastian Schelter <ssc.open@googlemail.com
>> wrote:
>
>> It says "Failed map tasks=1", you should have a deeper look into the
>> logfiles or into the web console of Hadoop to find out why the map task
>> fails.
>>
>> On 20.11.2013 11:44, Jyoti Yadav wrote:
>>> 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
>>
>>
>
Re: Need your help to serialize an boolean array....
Posted by Jyoti Yadav <ra...@gmail.com>.
Thanks Sir for your suggestions...
I looked into the log files...
It is showing some error at readFields() method in
ClosenessMessageWritable.java file..You please have a look again whether
boolean array is serialized properly or not...
I am doubtful about these serialization functions..
Thanks..
On Wed, Nov 20, 2013 at 4:16 PM, Sebastian Schelter <ssc.open@googlemail.com
> wrote:
> It says "Failed map tasks=1", you should have a deeper look into the
> logfiles or into the web console of Hadoop to find out why the map task
> fails.
>
> On 20.11.2013 11:44, Jyoti Yadav wrote:
> > 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
>
>
Re: Need your help to serialize an boolean array....
Posted by Sebastian Schelter <ss...@googlemail.com>.
It says "Failed map tasks=1", you should have a deeper look into the
logfiles or into the web console of Hadoop to find out why the map task
fails.
On 20.11.2013 11:44, Jyoti Yadav wrote:
> 13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
Re: Need your help to serialize an boolean array....
Posted by Jyoti Yadav <ra...@gmail.com>.
Hi Sebastian...
problem is that while sending message in superstep 0 ,i am not able to
retrieve in superstep 1..
while running the program,program ends with the following details...
13/11/20 15:45:23 INFO job.GiraphJob: run: Tracking URL:
http://localhost:50030/jobdetails.jsp?jobid=job_201311200901_0018
13/11/20 15:46:33 INFO
job.HaltApplicationUtils$DefaultHaltInstructionsWriter:
writeHaltInstructions: To halt after next superstep execute:
'bin/halt-application --zkServer kanha-Vostro-1014:22181 --zkNode
/_hadoopBsp/job_201311200901_0018/_haltComputation'
13/11/20 15:46:33 INFO mapred.JobClient: Running job: job_201311200901_0018
13/11/20 15:46:34 INFO mapred.JobClient: map 100% reduce 0%
13/11/20 15:56:20 INFO mapred.JobClient: map 50% reduce 0%
13/11/20 15:56:28 INFO mapred.JobClient: Job complete: job_201311200901_0018
13/11/20 15:56:28 INFO mapred.JobClient: Counters: 6
13/11/20 15:56:28 INFO mapred.JobClient: Job Counters
13/11/20 15:56:28 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=1282572
13/11/20 15:56:28 INFO mapred.JobClient: Total time spent by all
reduces waiting after reserving slots (ms)=0
13/11/20 15:56:28 INFO mapred.JobClient: Total time spent by all maps
waiting after reserving slots (ms)=0
13/11/20 15:56:28 INFO mapred.JobClient: Launched map tasks=2
13/11/20 15:56:28 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/11/20 15:56:28 INFO mapred.JobClient: Failed map tasks=1
For debugging purpose while i am printing System.out.println() statement,
in superstep 1 ,it is not printing...It means that control is not entering
into Superstep 2...
Thanks....
On Wed, Nov 20, 2013 at 3:43 PM, Sebastian Schelter <ssc.open@googlemail.com
> wrote:
> Hi Jyoti,
>
> the code looks good in general, what error do you get?
>
> Btw, I have two students of mine implementing closeness centrality on
> Giraph, if you want, I can hook you up with them, they are pretty far
> with their implementation.
>
> Best,
> Sebastian
>
>
>
> On 20.11.2013 10:13, Jyoti Yadav wrote:
> > Thanks Sebastian for your reply...
> >
> > All three files are shown below..
> >
> > 1.Main program file..
> > 2.VertexValueWritable file.
> > 3.MessageWritable file..
> >
> >
> >
> >
> ..................................************************.........................................
> >
> >
> > package org.apache.giraph.examples;
> > import org.apache.giraph.examples.utils.ClosenessVertexValueWritable;
> > import org.apache.giraph.graph.BasicComputation;
> > import org.apache.giraph.conf.LongConfOption;
> > import org.apache.giraph.edge.Edge;
> > import org.apache.giraph.graph.Vertex;
> > import org.apache.hadoop.io.DoubleWritable;
> > import org.apache.hadoop.io.FloatWritable;
> > import org.apache.hadoop.io.LongWritable;
> > import org.apache.log4j.Logger;
> >
> > import java.io.IOException;
> >
> >
> > public class Closeness extends BasicComputation<LongWritable,
> > ClosenessVertexValueWritable, FloatWritable, ClosenessMessageWritable>
> > {
> > private static final Logger LOG =Logger.getLogger(Closeness.class);
> > @Override
> > public void compute(Vertex<LongWritable,
> > ClosenessVertexValueWritable, FloatWritable>
> > vertex,Iterable<ClosenessMessageWritable> messages) throws IOException
> > {
> > if(getSuperstep()==0)
> > {
> > boolean[] bitstring=new boolean[128];
> > for(int i=0;i<128;i++)
> > {
> > bitstring[i]=false;
> > }
> > vertex.setValue(new
> > ClosenessVertexValueWritable(0.0,0.0,bitstring));
> > boolean[] bitstring1=new boolean[128];
> > bitstring1=vertex.getValue().get_previous_bitstring();
> >
> >
> > for (Edge<LongWritable, FloatWritable> edge :
> > vertex.getEdges())
> > {
> >
> >
> > sendMessage(edge.getTargetVertexId(), new
> > ClosenessMessageWritable(bitstring));
> >
> > System.out.println("superstep is
> > ="+getSuperstep()+"vertex is ="+vertex.getId().get());
> >
> >
> >
> > }
> >
> > }
> >
> > if(getSuperstep()==1)
> > {
> > boolean[] bitstring2=new boolean[128];
> >
> > for (ClosenessMessageWritable message : messages)
> > {
> > System.out.println("superstep is
> ="+getSuperstep()+"vertex
> > is ="+vertex.getId().get());
> >
> >
> > bitstring2=message.get_previous_bitstring();
> >
> > }
> >
> >
> > vertex.voteToHalt();
> > }
> >
> >
> >
> >
> >
> > }
> > }
> >
> >
> >
> >
> .............................................*************************************.....................................................
> >
> >
> >
> > package org.apache.giraph.examples.utils;
> >
> > import java.io.*;
> > import org.apache.hadoop.io.Writable;
> > import org.apache.hadoop.io.LongWritable;
> > import org.apache.hadoop.io.WritableComparator;
> > import java.util.Arrays;
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > public class ClosenessVertexValueWritable implements Writable {
> >
> > private double vertex_value;
> > private double previous_no_of_neighbors;
> > private boolean[] previous_bitstring;
> >
> > public ClosenessVertexValueWritable()
> > {
> > vertex_value=0.0;
> > previous_no_of_neighbors=0.0;
> > previous_bitstring=new boolean[128];
> >
> > }
> >
> > public ClosenessVertexValueWritable(double vertex_value1,double
> > previous_no_of_neighbors1,boolean[] previous_bitstring1)
> > {
> >
> > vertex_value=vertex_value1;
> > previous_no_of_neighbors=previous_no_of_neighbors1;
> > previous_bitstring=new boolean[128];
> > previous_bitstring=previous_bitstring1;
> >
> >
> >
> >
> > public double get_vertex_value() { return vertex_value; }
> > public double get_previous_no_of_neighbors() { return
> > previous_no_of_neighbors; }
> > public boolean[] get_previous_bitstring() { return previous_bitstring;
> }
> >
> >
> > @Override
> > public void readFields(DataInput in) throws IOException {
> > vertex_value=in.readDouble();
> > previous_no_of_neighbors=in.readDouble();
> >
> > for(int i=0;i<128;i++)
> > {
> > this.previous_bitstring[i] = in.readBoolean();
> > }
> > }
> > @Override
> > public void write(DataOutput out) throws IOException {
> > out.writeDouble(vertex_value);
> >
> > out.writeDouble(previous_no_of_neighbors);
> >
> > for(int i=0;i<128;i++)
> > {
> > out.writeBoolean(previous_bitstring[i]);
> > }
> >
> >
> > }
> >
> > @Override
> > public String toString()
> > {
> > String str="";
> >
> > //str=Arrays.toString(previous_bitstring);
> >
> > return Double.toString(vertex_value)+
> >
> Double.toString(previous_no_of_neighbors)+Arrays.toString(previous_bitstring);
> > }
> >
> >
> >
> > }
> >
> >
> >
> ..................................*****************************************...................................................
> >
> >
> >
> > package org.apache.giraph.examples;
> >
> > import java.io.*;
> > import org.apache.hadoop.io.Writable;
> > import org.apache.hadoop.io.LongWritable;
> > import org.apache.hadoop.io.WritableComparator;
> > import java.util.Arrays;
> >
> >
> > public class ClosenessMessageWritable implements Writable {
> >
> >
> >
> > private int size;
> >
> > private boolean[] previous_bitstring;
> >
> > public ClosenessMessageWritable() {}
> >
> > public ClosenessMessageWritable(boolean[] previous_bitstring1)
> > {
> >
> > size=128;
> > previous_bitstring=new boolean[128];
> > previous_bitstring=previous_bitstring1;
> >
> >
> >
> >
> > }
> >
> >
> > public boolean[] get_previous_bitstring() { return previous_bitstring;
> }
> >
> > @Override
> > public void readFields(DataInput in) throws IOException {
> >
> >
> > size=in.readInt();
> >
> > for(int i=0;i<size;i++)
> > {
> > boolean b=in.readBoolean();
> > previous_bitstring[i] = b;
> > }
> >
> >
> > }
> > @Override
> > public void write(DataOutput out) throws IOException {
> >
> > out.writeInt(size);
> >
> > for(int i=0;i<size;i++)
> > {
> > boolean b=previous_bitstring[i];
> > out.writeBoolean(b);
> > }
> >
> >
> >
> > }
> >
> >
> > @Override
> > public String toString()
> > {
> > String str="";
> >
> > str=Arrays.toString(previous_bitstring);
> >
> > return str;
> > }
> >
> >
> >
> >
> > }
> >
> >
> > Please check where i am going wrong...
> > Thanks...
> >
> >
> >
> > On Wed, Nov 20, 2013 at 1:31 PM, Sebastian Schelter <
> ssc.open@googlemail.com
> >> wrote:
> >
> >> What errors do you exactly get? Can you show the whole implementation of
> >> your vertex?
> >>
> >> On 20.11.2013 08:42, Jyoti Yadav wrote:
> >>> Hi folks..
> >>>
> >>> I am implementing one program where I need to pass message as boolean
> >>> array..
> >>> While implementing my MyMessageWritable.java class,I need to define
> >>> readfields() and write() functions...
> >>>
> >>> I tried my luck but failed...Program is compiling fine but not
> running...
> >>>
> >>>
> >>> I am taking boolean array as
> >>>
> >>> boolean[] bitstring=new boolean[128];
> >>>
> >>>
> >>> pls help...I am badly tangled...
> >>>
> >>> Thanks
> >>>
> >>> Jyoti
> >>>
> >>
> >>
> >
>
>
Re: Need your help to serialize an boolean array....
Posted by Sebastian Schelter <ss...@googlemail.com>.
Hi Jyoti,
the code looks good in general, what error do you get?
Btw, I have two students of mine implementing closeness centrality on
Giraph, if you want, I can hook you up with them, they are pretty far
with their implementation.
Best,
Sebastian
On 20.11.2013 10:13, Jyoti Yadav wrote:
> Thanks Sebastian for your reply...
>
> All three files are shown below..
>
> 1.Main program file..
> 2.VertexValueWritable file.
> 3.MessageWritable file..
>
>
>
> ..................................************************.........................................
>
>
> package org.apache.giraph.examples;
> import org.apache.giraph.examples.utils.ClosenessVertexValueWritable;
> import org.apache.giraph.graph.BasicComputation;
> import org.apache.giraph.conf.LongConfOption;
> import org.apache.giraph.edge.Edge;
> import org.apache.giraph.graph.Vertex;
> import org.apache.hadoop.io.DoubleWritable;
> import org.apache.hadoop.io.FloatWritable;
> import org.apache.hadoop.io.LongWritable;
> import org.apache.log4j.Logger;
>
> import java.io.IOException;
>
>
> public class Closeness extends BasicComputation<LongWritable,
> ClosenessVertexValueWritable, FloatWritable, ClosenessMessageWritable>
> {
> private static final Logger LOG =Logger.getLogger(Closeness.class);
> @Override
> public void compute(Vertex<LongWritable,
> ClosenessVertexValueWritable, FloatWritable>
> vertex,Iterable<ClosenessMessageWritable> messages) throws IOException
> {
> if(getSuperstep()==0)
> {
> boolean[] bitstring=new boolean[128];
> for(int i=0;i<128;i++)
> {
> bitstring[i]=false;
> }
> vertex.setValue(new
> ClosenessVertexValueWritable(0.0,0.0,bitstring));
> boolean[] bitstring1=new boolean[128];
> bitstring1=vertex.getValue().get_previous_bitstring();
>
>
> for (Edge<LongWritable, FloatWritable> edge :
> vertex.getEdges())
> {
>
>
> sendMessage(edge.getTargetVertexId(), new
> ClosenessMessageWritable(bitstring));
>
> System.out.println("superstep is
> ="+getSuperstep()+"vertex is ="+vertex.getId().get());
>
>
>
> }
>
> }
>
> if(getSuperstep()==1)
> {
> boolean[] bitstring2=new boolean[128];
>
> for (ClosenessMessageWritable message : messages)
> {
> System.out.println("superstep is ="+getSuperstep()+"vertex
> is ="+vertex.getId().get());
>
>
> bitstring2=message.get_previous_bitstring();
>
> }
>
>
> vertex.voteToHalt();
> }
>
>
>
>
>
> }
> }
>
>
>
> .............................................*************************************.....................................................
>
>
>
> package org.apache.giraph.examples.utils;
>
> import java.io.*;
> import org.apache.hadoop.io.Writable;
> import org.apache.hadoop.io.LongWritable;
> import org.apache.hadoop.io.WritableComparator;
> import java.util.Arrays;
>
>
>
>
>
>
>
>
>
>
> public class ClosenessVertexValueWritable implements Writable {
>
> private double vertex_value;
> private double previous_no_of_neighbors;
> private boolean[] previous_bitstring;
>
> public ClosenessVertexValueWritable()
> {
> vertex_value=0.0;
> previous_no_of_neighbors=0.0;
> previous_bitstring=new boolean[128];
>
> }
>
> public ClosenessVertexValueWritable(double vertex_value1,double
> previous_no_of_neighbors1,boolean[] previous_bitstring1)
> {
>
> vertex_value=vertex_value1;
> previous_no_of_neighbors=previous_no_of_neighbors1;
> previous_bitstring=new boolean[128];
> previous_bitstring=previous_bitstring1;
>
>
>
>
> public double get_vertex_value() { return vertex_value; }
> public double get_previous_no_of_neighbors() { return
> previous_no_of_neighbors; }
> public boolean[] get_previous_bitstring() { return previous_bitstring; }
>
>
> @Override
> public void readFields(DataInput in) throws IOException {
> vertex_value=in.readDouble();
> previous_no_of_neighbors=in.readDouble();
>
> for(int i=0;i<128;i++)
> {
> this.previous_bitstring[i] = in.readBoolean();
> }
> }
> @Override
> public void write(DataOutput out) throws IOException {
> out.writeDouble(vertex_value);
>
> out.writeDouble(previous_no_of_neighbors);
>
> for(int i=0;i<128;i++)
> {
> out.writeBoolean(previous_bitstring[i]);
> }
>
>
> }
>
> @Override
> public String toString()
> {
> String str="";
>
> //str=Arrays.toString(previous_bitstring);
>
> return Double.toString(vertex_value)+
> Double.toString(previous_no_of_neighbors)+Arrays.toString(previous_bitstring);
> }
>
>
>
> }
>
>
> ..................................*****************************************...................................................
>
>
>
> package org.apache.giraph.examples;
>
> import java.io.*;
> import org.apache.hadoop.io.Writable;
> import org.apache.hadoop.io.LongWritable;
> import org.apache.hadoop.io.WritableComparator;
> import java.util.Arrays;
>
>
> public class ClosenessMessageWritable implements Writable {
>
>
>
> private int size;
>
> private boolean[] previous_bitstring;
>
> public ClosenessMessageWritable() {}
>
> public ClosenessMessageWritable(boolean[] previous_bitstring1)
> {
>
> size=128;
> previous_bitstring=new boolean[128];
> previous_bitstring=previous_bitstring1;
>
>
>
>
> }
>
>
> public boolean[] get_previous_bitstring() { return previous_bitstring; }
>
> @Override
> public void readFields(DataInput in) throws IOException {
>
>
> size=in.readInt();
>
> for(int i=0;i<size;i++)
> {
> boolean b=in.readBoolean();
> previous_bitstring[i] = b;
> }
>
>
> }
> @Override
> public void write(DataOutput out) throws IOException {
>
> out.writeInt(size);
>
> for(int i=0;i<size;i++)
> {
> boolean b=previous_bitstring[i];
> out.writeBoolean(b);
> }
>
>
>
> }
>
>
> @Override
> public String toString()
> {
> String str="";
>
> str=Arrays.toString(previous_bitstring);
>
> return str;
> }
>
>
>
>
> }
>
>
> Please check where i am going wrong...
> Thanks...
>
>
>
> On Wed, Nov 20, 2013 at 1:31 PM, Sebastian Schelter <ssc.open@googlemail.com
>> wrote:
>
>> What errors do you exactly get? Can you show the whole implementation of
>> your vertex?
>>
>> On 20.11.2013 08:42, Jyoti Yadav wrote:
>>> Hi folks..
>>>
>>> I am implementing one program where I need to pass message as boolean
>>> array..
>>> While implementing my MyMessageWritable.java class,I need to define
>>> readfields() and write() functions...
>>>
>>> I tried my luck but failed...Program is compiling fine but not running...
>>>
>>>
>>> I am taking boolean array as
>>>
>>> boolean[] bitstring=new boolean[128];
>>>
>>>
>>> pls help...I am badly tangled...
>>>
>>> Thanks
>>>
>>> Jyoti
>>>
>>
>>
>
Re: Need your help to serialize an boolean array....
Posted by Jyoti Yadav <ra...@gmail.com>.
Thanks Sebastian for your reply...
All three files are shown below..
1.Main program file..
2.VertexValueWritable file.
3.MessageWritable file..
..................................************************.........................................
package org.apache.giraph.examples;
import org.apache.giraph.examples.utils.ClosenessVertexValueWritable;
import org.apache.giraph.graph.BasicComputation;
import org.apache.giraph.conf.LongConfOption;
import org.apache.giraph.edge.Edge;
import org.apache.giraph.graph.Vertex;
import org.apache.hadoop.io.DoubleWritable;
import org.apache.hadoop.io.FloatWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.log4j.Logger;
import java.io.IOException;
public class Closeness extends BasicComputation<LongWritable,
ClosenessVertexValueWritable, FloatWritable, ClosenessMessageWritable>
{
private static final Logger LOG =Logger.getLogger(Closeness.class);
@Override
public void compute(Vertex<LongWritable,
ClosenessVertexValueWritable, FloatWritable>
vertex,Iterable<ClosenessMessageWritable> messages) throws IOException
{
if(getSuperstep()==0)
{
boolean[] bitstring=new boolean[128];
for(int i=0;i<128;i++)
{
bitstring[i]=false;
}
vertex.setValue(new
ClosenessVertexValueWritable(0.0,0.0,bitstring));
boolean[] bitstring1=new boolean[128];
bitstring1=vertex.getValue().get_previous_bitstring();
for (Edge<LongWritable, FloatWritable> edge :
vertex.getEdges())
{
sendMessage(edge.getTargetVertexId(), new
ClosenessMessageWritable(bitstring));
System.out.println("superstep is
="+getSuperstep()+"vertex is ="+vertex.getId().get());
}
}
if(getSuperstep()==1)
{
boolean[] bitstring2=new boolean[128];
for (ClosenessMessageWritable message : messages)
{
System.out.println("superstep is ="+getSuperstep()+"vertex
is ="+vertex.getId().get());
bitstring2=message.get_previous_bitstring();
}
vertex.voteToHalt();
}
}
}
.............................................*************************************.....................................................
package org.apache.giraph.examples.utils;
import java.io.*;
import org.apache.hadoop.io.Writable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.WritableComparator;
import java.util.Arrays;
public class ClosenessVertexValueWritable implements Writable {
private double vertex_value;
private double previous_no_of_neighbors;
private boolean[] previous_bitstring;
public ClosenessVertexValueWritable()
{
vertex_value=0.0;
previous_no_of_neighbors=0.0;
previous_bitstring=new boolean[128];
}
public ClosenessVertexValueWritable(double vertex_value1,double
previous_no_of_neighbors1,boolean[] previous_bitstring1)
{
vertex_value=vertex_value1;
previous_no_of_neighbors=previous_no_of_neighbors1;
previous_bitstring=new boolean[128];
previous_bitstring=previous_bitstring1;
public double get_vertex_value() { return vertex_value; }
public double get_previous_no_of_neighbors() { return
previous_no_of_neighbors; }
public boolean[] get_previous_bitstring() { return previous_bitstring; }
@Override
public void readFields(DataInput in) throws IOException {
vertex_value=in.readDouble();
previous_no_of_neighbors=in.readDouble();
for(int i=0;i<128;i++)
{
this.previous_bitstring[i] = in.readBoolean();
}
}
@Override
public void write(DataOutput out) throws IOException {
out.writeDouble(vertex_value);
out.writeDouble(previous_no_of_neighbors);
for(int i=0;i<128;i++)
{
out.writeBoolean(previous_bitstring[i]);
}
}
@Override
public String toString()
{
String str="";
//str=Arrays.toString(previous_bitstring);
return Double.toString(vertex_value)+
Double.toString(previous_no_of_neighbors)+Arrays.toString(previous_bitstring);
}
}
..................................*****************************************...................................................
package org.apache.giraph.examples;
import java.io.*;
import org.apache.hadoop.io.Writable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.WritableComparator;
import java.util.Arrays;
public class ClosenessMessageWritable implements Writable {
private int size;
private boolean[] previous_bitstring;
public ClosenessMessageWritable() {}
public ClosenessMessageWritable(boolean[] previous_bitstring1)
{
size=128;
previous_bitstring=new boolean[128];
previous_bitstring=previous_bitstring1;
}
public boolean[] get_previous_bitstring() { return previous_bitstring; }
@Override
public void readFields(DataInput in) throws IOException {
size=in.readInt();
for(int i=0;i<size;i++)
{
boolean b=in.readBoolean();
previous_bitstring[i] = b;
}
}
@Override
public void write(DataOutput out) throws IOException {
out.writeInt(size);
for(int i=0;i<size;i++)
{
boolean b=previous_bitstring[i];
out.writeBoolean(b);
}
}
@Override
public String toString()
{
String str="";
str=Arrays.toString(previous_bitstring);
return str;
}
}
Please check where i am going wrong...
Thanks...
On Wed, Nov 20, 2013 at 1:31 PM, Sebastian Schelter <ssc.open@googlemail.com
> wrote:
> What errors do you exactly get? Can you show the whole implementation of
> your vertex?
>
> On 20.11.2013 08:42, Jyoti Yadav wrote:
> > Hi folks..
> >
> > I am implementing one program where I need to pass message as boolean
> > array..
> > While implementing my MyMessageWritable.java class,I need to define
> > readfields() and write() functions...
> >
> > I tried my luck but failed...Program is compiling fine but not running...
> >
> >
> > I am taking boolean array as
> >
> > boolean[] bitstring=new boolean[128];
> >
> >
> > pls help...I am badly tangled...
> >
> > Thanks
> >
> > Jyoti
> >
>
>
Re: Need your help to serialize an boolean array....
Posted by Sebastian Schelter <ss...@googlemail.com>.
What errors do you exactly get? Can you show the whole implementation of
your vertex?
On 20.11.2013 08:42, Jyoti Yadav wrote:
> Hi folks..
>
> I am implementing one program where I need to pass message as boolean
> array..
> While implementing my MyMessageWritable.java class,I need to define
> readfields() and write() functions...
>
> I tried my luck but failed...Program is compiling fine but not running...
>
>
> I am taking boolean array as
>
> boolean[] bitstring=new boolean[128];
>
>
> pls help...I am badly tangled...
>
> Thanks
>
> Jyoti
>