Problem: generating data to be sorted

[Richard Yang <ri...@richardyang.net>]

Hello everyone,

 

I have encountered the following error when I try to use randomwriter
(http://wiki.apache.org/lucene-hadoop/RandomWriter) 

I also changed the configuration to 

Test.randomwriter.mpas_per_host=5

Test.randomwriter.bytes_per_map=1024*1024

In short, I am trying to generate only one megabyte of data

After running

bin/hadoop jar hadoop-0.11.2-examples.jar randomwriter 030307kuku RW.conf,
here is the error message:



It seems that the namenode has problem creating new files to store results.
Yet, what troubles (maybe confuses) me is the fact that I just finished
running sample grep programs without any error.

 

 

Currently, there are 3 datanodes and one of them as namenode. I have run 5
sample grep programs concurrently and they all finished fine most of time.
Sometimes the progresses would halt for 5 grep programs and I will need to
reboot every node and start it over again.  Does anybody know how to debug
or fix this kind of problem?? Thank you.

 

Best Regards

 

Richard Yang

 <ma...@richardyang.net> richardyang@richardyang.net

 <ma...@gmail.com> kusanagiyang@gmail.com